examples/2. Predict States Energy Data.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# FOST--Predict States Energy Data\n",
    "\n",
    "> State Energy Data Systems (SEDS) data for all US states, including DC, from 1960 to 2014F.</br> \n",
    "The data is a composition of the State Energy Data Systems (SEDS) data for all US states, including DC, from 2016 to 2014F, for data released June 29, 2016. It has been tidied from a wide format to a long format, and includes unit codes for the values associated with the observations for each MSN code for each state for each year.</br> \n",
    "Link: https://www.kaggle.com/nathanto/seds-1960-2014F\n",
    "\n",
    "We are going to use FOST and states energy temporal and spatial data to predict next several years."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import fostool\n",
    "from fostool.pipeline import Pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Download Data\n",
    "\n",
    "You should have `FOST_example_data/Energy/graph.csv`,`FOST_example_data/Energy/train.csv` and `FOST_example_data/Turbine/train.csv` after running this block."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2021-11-11 14:01:25--  https://onedrive.live.com/download?cid=4E3B5BF06F46DFD5&resid=4E3B5BF06F46DFD5%21461&authkey=ACK4NyYumsXm8IY\n",
      "Resolving onedrive.live.com (onedrive.live.com)... 13.107.42.13\n",
      "Connecting to onedrive.live.com (onedrive.live.com)|13.107.42.13|:443... connected.\n",
      "HTTP request sent, awaiting response... 302 Found\n",
      "Location: https://ife64g.dm.files.1drv.com/y4m41J1TZTZKXwLezz85NBSVDMW2c7UChD9doqRFaLZzQvc2o9-Ahq_mNTqFhZIUDuB3k7WCZ_k92xLita2m83sSkjVpo8CgtztWt1j04hei-YfmDxlHTrPCw1yPBlMeEehuCqOlQTP93D50XpTh-csf3oJ-nHJY7Wb72cXAfouoq6tsliDeVPlNH7MIcSYh2adMmf1zJN5myeWacpPYdxk6Q/FOST_example_data.zip?download&psid=1 [following]\n",
      "--2021-11-11 14:01:26--  https://ife64g.dm.files.1drv.com/y4m41J1TZTZKXwLezz85NBSVDMW2c7UChD9doqRFaLZzQvc2o9-Ahq_mNTqFhZIUDuB3k7WCZ_k92xLita2m83sSkjVpo8CgtztWt1j04hei-YfmDxlHTrPCw1yPBlMeEehuCqOlQTP93D50XpTh-csf3oJ-nHJY7Wb72cXAfouoq6tsliDeVPlNH7MIcSYh2adMmf1zJN5myeWacpPYdxk6Q/FOST_example_data.zip?download&psid=1\n",
      "Resolving ife64g.dm.files.1drv.com (ife64g.dm.files.1drv.com)... 13.107.42.12\n",
      "Connecting to ife64g.dm.files.1drv.com (ife64g.dm.files.1drv.com)|13.107.42.12|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 7891061 (7.5M) [application/zip]\n",
      "Saving to: ‘Dataset.zip’\n",
      "\n",
      "Dataset.zip         100%[===================>]   7.53M  11.3MB/s    in 0.7s    \n",
      "\n",
      "2021-11-11 14:01:28 (11.3 MB/s) - ‘Dataset.zip’ saved [7891061/7891061]\n",
      "\n",
      "Archive:  Dataset.zip\n",
      "   creating: FOST_example_data/Energy/\n",
      "  inflating: FOST_example_data/Energy/graph.csv  \n",
      "  inflating: FOST_example_data/Energy/train.csv  \n",
      "   creating: FOST_example_data/Turbine/\n",
      "  inflating: FOST_example_data/Turbine/train.csv  \n"
     ]
    }
   ],
   "source": [
    "!wget -O Dataset.zip \"https://onedrive.live.com/download?cid=4E3B5BF06F46DFD5&resid=4E3B5BF06F46DFD5%21461&authkey=ACK4NyYumsXm8IY\"\n",
    "\n",
    "!unzip Dataset.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparing Data\n",
    "\n",
    "Load the training data and graph data from the .csv file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_path = 'FOST_example_data/Energy/train.csv'\n",
    "graph_path = 'FOST_example_data/Energy/graph.csv'\n",
    "#predict for next 5 years\n",
    "lookahead = 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Node</th>\n",
       "      <th>Date</th>\n",
       "      <th>TARGET</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Alaska</td>\n",
       "      <td>1960-01-01</td>\n",
       "      <td>800592.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Alaska</td>\n",
       "      <td>1961-01-01</td>\n",
       "      <td>933600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Alaska</td>\n",
       "      <td>1962-01-01</td>\n",
       "      <td>1013979.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Alaska</td>\n",
       "      <td>1963-01-01</td>\n",
       "      <td>1046760.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Alaska</td>\n",
       "      <td>1964-01-01</td>\n",
       "      <td>1100728.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0    Node        Date     TARGET\n",
       "0           0  Alaska  1960-01-01   800592.0\n",
       "1           1  Alaska  1961-01-01   933600.0\n",
       "2           2  Alaska  1962-01-01  1013979.0\n",
       "3           3  Alaska  1963-01-01  1046760.0\n",
       "4           4  Alaska  1964-01-01  1100728.0"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "pd.read_csv(train_path).head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>node_0</th>\n",
       "      <th>node_1</th>\n",
       "      <th>weight</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>Alabama</td>\n",
       "      <td>Arkansas</td>\n",
       "      <td>0.074517</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>Alabama</td>\n",
       "      <td>Florida</td>\n",
       "      <td>0.075873</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>Alabama</td>\n",
       "      <td>Georgia</td>\n",
       "      <td>0.155103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>Alabama</td>\n",
       "      <td>Illinois</td>\n",
       "      <td>0.054406</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>Alabama</td>\n",
       "      <td>Indiana</td>\n",
       "      <td>0.056332</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0   node_0    node_1    weight\n",
       "0           0  Alabama  Arkansas  0.074517\n",
       "1           1  Alabama   Florida  0.075873\n",
       "2           2  Alabama   Georgia  0.155103\n",
       "3           3  Alabama  Illinois  0.054406\n",
       "4           4  Alabama   Indiana  0.056332"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.read_csv(graph_path).head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create Pipeline\n",
    "\n",
    "Create Pipeline and preprocess training data, you are encouraged to change params like `lookahead`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:14 fostool/task/config_handler.py 26 \\ - INFO - yaml handler load path: /home/xiaofan/anaconda3/envs/test/lib/python3.8/site-packages/fostool/config/default.yaml\n",
      "2021-11-11 14:15:15 fostool/dataset/data_utils.py 402 \\ - INFO - Detected Sample Frequency: <DateOffset: years=1>.\n",
      "2021-11-11 14:15:15 fostool/dataset/data_utils.py 426 \\ - INFO - 2805 Rows Before Time Reindex.\n",
      "2021-11-11 14:15:15 fostool/dataset/data_utils.py 428 \\ - INFO - 2805 Rows After Time Reindex.\n",
      "2021-11-11 14:15:15 fostool/dataset/data_utils.py 429 \\ - INFO - --------------------\n",
      "2021-11-11 14:15:15 fostool/dataset/data_utils.py 457 \\ - INFO - 2805 Rows Before Fill Missing.\n",
      "2021-11-11 14:15:15 fostool/dataset/data_utils.py 461 \\ - INFO - 2805 Rows After Fill Missing.\n",
      "2021-11-11 14:15:15 fostool/dataset/data_utils.py 462 \\ - INFO - --------------------\n"
     ]
    }
   ],
   "source": [
    "fost = Pipeline(lookahead=lookahead, train_path=train_path, graph_path=graph_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training\n",
    "\n",
    "Train the models that fit your data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 0, train loss 0.5394601225852966, val loss 0.7979480028152466\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.7979480028152466\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 1, train loss 0.4595740536848704, val loss 0.663418710231781\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.663418710231781\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 2, train loss 0.41446834802627563, val loss 0.5555686354637146\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.5555686354637146\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 3, train loss 0.3648509482542674, val loss 0.45720306038856506\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.45720306038856506\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 4, train loss 0.33784282207489014, val loss 0.37851011753082275\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.37851011753082275\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 5, train loss 0.31920791665712994, val loss 0.3321484923362732\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.3321484923362732\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 6, train loss 0.3163359959920247, val loss 0.3119962215423584\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.3119962215423584\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 7, train loss 0.29853253563245136, val loss 0.31145018339157104\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.31145018339157104\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 8, train loss 0.2969876527786255, val loss 0.3181919753551483\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 128 \\ - INFO - On epoch 9, train loss 0.27941952149073285, val loss 0.3063599169254303\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:19 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.3063599169254303\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 10, train loss 0.2752908368905385, val loss 0.283165842294693\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.283165842294693\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 11, train loss 0.2565777003765106, val loss 0.26976269483566284\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.26976269483566284\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 12, train loss 0.2526879459619522, val loss 0.26443466544151306\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.26443466544151306\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 13, train loss 0.231986070672671, val loss 0.27486035227775574\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 14, train loss 0.23411838710308075, val loss 0.2611584961414337\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.2611584961414337\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 15, train loss 0.22496947149435678, val loss 0.2302684783935547\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.2302684783935547\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 16, train loss 0.2218333880106608, val loss 0.20190982520580292\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.20190982520580292\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 17, train loss 0.19751046101252237, val loss 0.20237614214420319\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 18, train loss 0.19805975258350372, val loss 0.21329186856746674\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 19, train loss 0.19002187252044678, val loss 0.22794929146766663\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 20, train loss 0.18479381004969278, val loss 0.224673792719841\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 21, train loss 0.1849053998788198, val loss 0.23536279797554016\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 22, train loss 0.18254772325356802, val loss 0.2184980809688568\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 23, train loss 0.16534507274627686, val loss 0.18280720710754395\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.18280720710754395\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 24, train loss 0.16531549394130707, val loss 0.1771249622106552\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_3164_7, current best val loss 0.1771249622106552\n",
      "2021-11-11 14:15:20 fostool/tools/trainer.py 128 \\ - INFO - On epoch 25, train loss 0.16057217121124268, val loss 0.19806890189647675\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 26, train loss 0.16522611180941263, val loss 0.21463696658611298\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 27, train loss 0.16231742004553476, val loss 0.23201657831668854\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 28, train loss 0.16854546467463175, val loss 0.2112753540277481\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 29, train loss 0.15290498485167822, val loss 0.19457682967185974\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 30, train loss 0.15604684750239053, val loss 0.18524982035160065\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 31, train loss 0.15209218859672546, val loss 0.18770956993103027\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 32, train loss 0.1536823312441508, val loss 0.20137137174606323\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 33, train loss 0.15817637741565704, val loss 0.20276322960853577\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 34, train loss 0.1504213660955429, val loss 0.19524480402469635\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 128 \\ - INFO - On epoch 0, train loss 0.5128050645192465, val loss 0.8280712962150574\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:21 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_a54c_7, current best val loss 0.8280712962150574\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 128 \\ - INFO - On epoch 1, train loss 0.4279733995596568, val loss 0.5913812518119812\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_a54c_7, current best val loss 0.5913812518119812\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 128 \\ - INFO - On epoch 2, train loss 0.36069775621096295, val loss 0.38592806458473206\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_a54c_7, current best val loss 0.38592806458473206\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 128 \\ - INFO - On epoch 3, train loss 0.31865084171295166, val loss 0.2874174416065216\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_a54c_7, current best val loss 0.2874174416065216\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 128 \\ - INFO - On epoch 4, train loss 0.30879228313763935, val loss 0.27310726046562195\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_a54c_7, current best val loss 0.27310726046562195\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 128 \\ - INFO - On epoch 5, train loss 0.28743701179822284, val loss 0.34100407361984253\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 128 \\ - INFO - On epoch 6, train loss 0.2697569827238719, val loss 0.3028898537158966\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 128 \\ - INFO - On epoch 7, train loss 0.24229651192824045, val loss 0.25022390484809875\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:22 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_a54c_7, current best val loss 0.25022390484809875\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 128 \\ - INFO - On epoch 8, train loss 0.2336580604314804, val loss 0.21356499195098877\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_a54c_7, current best val loss 0.21356499195098877\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 128 \\ - INFO - On epoch 9, train loss 0.221748153368632, val loss 0.21577654778957367\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 128 \\ - INFO - On epoch 10, train loss 0.2125046302874883, val loss 0.2698751986026764\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 128 \\ - INFO - On epoch 11, train loss 0.20003477732340494, val loss 0.21675655245780945\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 128 \\ - INFO - On epoch 12, train loss 0.18964122732480368, val loss 0.17032350599765778\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_a54c_7, current best val loss 0.17032350599765778\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 128 \\ - INFO - On epoch 13, train loss 0.18830854694048563, val loss 0.1962081640958786\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 128 \\ - INFO - On epoch 14, train loss 0.1811235100030899, val loss 0.3195156455039978\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 128 \\ - INFO - On epoch 15, train loss 0.16702266534169516, val loss 0.19359290599822998\n",
      "2021-11-11 14:15:23 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 128 \\ - INFO - On epoch 16, train loss 0.15275723735491434, val loss 0.14840395748615265\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_a54c_7, current best val loss 0.14840395748615265\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 128 \\ - INFO - On epoch 17, train loss 0.15524888535340628, val loss 0.3044474422931671\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 128 \\ - INFO - On epoch 18, train loss 0.155009796222051, val loss 0.24788565933704376\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 128 \\ - INFO - On epoch 19, train loss 0.14534358183542886, val loss 0.21676568686962128\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 128 \\ - INFO - On epoch 20, train loss 0.14414747059345245, val loss 0.22795738279819489\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 128 \\ - INFO - On epoch 21, train loss 0.14441105226675668, val loss 0.30761241912841797\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 128 \\ - INFO - On epoch 22, train loss 0.14711840947469076, val loss 0.15385785698890686\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 128 \\ - INFO - On epoch 23, train loss 0.1481392482916514, val loss 0.37413567304611206\n",
      "2021-11-11 14:15:24 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 24, train loss 0.15062629679838815, val loss 0.17676302790641785\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 25, train loss 0.13748787343502045, val loss 0.2705700099468231\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 26, train loss 0.13178216914335886, val loss 0.2439652532339096\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 0, train loss 0.7239524920781454, val loss 0.44722795486450195\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_b4d0_7, current best val loss 0.44722795486450195\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 1, train loss 0.3949432671070099, val loss 0.27110227942466736\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_b4d0_7, current best val loss 0.27110227942466736\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 2, train loss 0.33222198486328125, val loss 0.2399486005306244\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_b4d0_7, current best val loss 0.2399486005306244\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 3, train loss 0.31109825770060223, val loss 0.208534374833107\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_b4d0_7, current best val loss 0.208534374833107\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 4, train loss 0.25795629620552063, val loss 0.21225526928901672\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 5, train loss 0.22237334152062735, val loss 0.21629072725772858\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 6, train loss 0.20101024210453033, val loss 0.16576029360294342\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_b4d0_7, current best val loss 0.16576029360294342\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 7, train loss 0.19852790236473083, val loss 0.12593558430671692\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_b4d0_7, current best val loss 0.12593558430671692\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 8, train loss 0.19437583784262338, val loss 0.138005331158638\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 9, train loss 0.18794843554496765, val loss 0.18218281865119934\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 10, train loss 0.18902543683846793, val loss 0.1922350525856018\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 11, train loss 0.18849365909894308, val loss 0.13523384928703308\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:25 fostool/tools/trainer.py 128 \\ - INFO - On epoch 12, train loss 0.187355175614357, val loss 0.13289275765419006\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 13, train loss 0.1795682062705358, val loss 0.19869698584079742\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 14, train loss 0.17216946184635162, val loss 0.20334398746490479\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 15, train loss 0.17404891550540924, val loss 0.17774806916713715\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 16, train loss 0.1694688449303309, val loss 0.17364545166492462\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 17, train loss 0.17082436879475912, val loss 0.181337371468544\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 0, train loss 0.4989210516214371, val loss 0.9685775637626648\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.9685775637626648\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 1, train loss 0.4736110419034958, val loss 0.8770714402198792\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.8770714402198792\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 2, train loss 0.4557618349790573, val loss 0.7948082685470581\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.7948082685470581\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 3, train loss 0.4413255304098129, val loss 0.7176408767700195\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.7176408767700195\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 4, train loss 0.42836080491542816, val loss 0.6382026672363281\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.6382026672363281\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 5, train loss 0.41627390682697296, val loss 0.5655919909477234\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.5655919909477234\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 6, train loss 0.40416762232780457, val loss 0.5082083344459534\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.5082083344459534\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 7, train loss 0.39610573649406433, val loss 0.46088749170303345\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.46088749170303345\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 8, train loss 0.3844829946756363, val loss 0.43058425188064575\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.43058425188064575\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 9, train loss 0.3736089617013931, val loss 0.41216790676116943\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.41216790676116943\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 10, train loss 0.36150966584682465, val loss 0.39457136392593384\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.39457136392593384\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 128 \\ - INFO - On epoch 11, train loss 0.3486785441637039, val loss 0.3744224011898041\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:26 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.3744224011898041\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 12, train loss 0.3355981111526489, val loss 0.3543141186237335\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.3543141186237335\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 13, train loss 0.3223126232624054, val loss 0.33156681060791016\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.33156681060791016\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 14, train loss 0.3085220009088516, val loss 0.3110785484313965\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.3110785484313965\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 15, train loss 0.2941538691520691, val loss 0.2926771640777588\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.2926771640777588\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 16, train loss 0.2797641158103943, val loss 0.27119579911231995\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.27119579911231995\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 17, train loss 0.26719844341278076, val loss 0.2490113377571106\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.2490113377571106\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 18, train loss 0.25248514115810394, val loss 0.23279023170471191\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.23279023170471191\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 19, train loss 0.23975571244955063, val loss 0.21798774600028992\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.21798774600028992\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 20, train loss 0.22735927999019623, val loss 0.20740114152431488\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.20740114152431488\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 21, train loss 0.2168155387043953, val loss 0.2006959319114685\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.2006959319114685\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 22, train loss 0.20658039301633835, val loss 0.19927291572093964\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.19927291572093964\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 23, train loss 0.19848576188087463, val loss 0.1965406835079193\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.1965406835079193\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 24, train loss 0.1916584149003029, val loss 0.18720144033432007\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.18720144033432007\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 25, train loss 0.18705089390277863, val loss 0.17548242211341858\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.17548242211341858\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 26, train loss 0.18024513125419617, val loss 0.16546601057052612\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.16546601057052612\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 27, train loss 0.17624570429325104, val loss 0.16029928624629974\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.16029928624629974\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 28, train loss 0.1721898391842842, val loss 0.1510169953107834\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.1510169953107834\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 29, train loss 0.16910162568092346, val loss 0.14257390797138214\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.14257390797138214\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 128 \\ - INFO - On epoch 30, train loss 0.1657581850886345, val loss 0.13868634402751923\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:27 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.13868634402751923\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 31, train loss 0.1644994467496872, val loss 0.13286159932613373\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.13286159932613373\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 32, train loss 0.1604553982615471, val loss 0.12635447084903717\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.12635447084903717\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 33, train loss 0.15988358110189438, val loss 0.12299881130456924\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.12299881130456924\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 34, train loss 0.15908877551555634, val loss 0.12677741050720215\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 35, train loss 0.15692659467458725, val loss 0.1255977600812912\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 36, train loss 0.1548689901828766, val loss 0.127029150724411\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 37, train loss 0.1511954367160797, val loss 0.12296372652053833\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.12296372652053833\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 38, train loss 0.14986146986484528, val loss 0.12019504606723785\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.12019504606723785\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 39, train loss 0.1490846760571003, val loss 0.11929481476545334\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_c234_14, current best val loss 0.11929481476545334\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 40, train loss 0.14672011137008667, val loss 0.12148523330688477\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 41, train loss 0.14542733132839203, val loss 0.12417815625667572\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 42, train loss 0.14416511356830597, val loss 0.13056731224060059\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 43, train loss 0.14390427619218826, val loss 0.131645068526268\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 44, train loss 0.14166425168514252, val loss 0.13266411423683167\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 45, train loss 0.14015454053878784, val loss 0.12816159427165985\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 46, train loss 0.1422819271683693, val loss 0.12521307170391083\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 47, train loss 0.13755029439926147, val loss 0.132210835814476\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 48, train loss 0.13785485923290253, val loss 0.13624869287014008\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 128 \\ - INFO - On epoch 49, train loss 0.13489189743995667, val loss 0.1359129697084427\n",
      "2021-11-11 14:15:28 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 128 \\ - INFO - On epoch 0, train loss 0.5064907670021057, val loss 0.9206510782241821\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.9206510782241821\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 128 \\ - INFO - On epoch 1, train loss 0.4697325527667999, val loss 0.7551189064979553\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.7551189064979553\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 128 \\ - INFO - On epoch 2, train loss 0.4372527450323105, val loss 0.6295667886734009\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.6295667886734009\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 128 \\ - INFO - On epoch 3, train loss 0.4148876368999481, val loss 0.513950526714325\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.513950526714325\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 128 \\ - INFO - On epoch 4, train loss 0.38407059013843536, val loss 0.45779046416282654\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.45779046416282654\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 128 \\ - INFO - On epoch 5, train loss 0.3687244653701782, val loss 0.3886502981185913\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.3886502981185913\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 128 \\ - INFO - On epoch 6, train loss 0.3396373987197876, val loss 0.33999690413475037\n",
      "2021-11-11 14:15:29 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.33999690413475037\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 128 \\ - INFO - On epoch 7, train loss 0.31710055470466614, val loss 0.3236995339393616\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.3236995339393616\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 128 \\ - INFO - On epoch 8, train loss 0.2977634221315384, val loss 0.32261067628860474\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.32261067628860474\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 128 \\ - INFO - On epoch 9, train loss 0.27416181564331055, val loss 0.2702319324016571\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.2702319324016571\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 128 \\ - INFO - On epoch 10, train loss 0.25087909400463104, val loss 0.2414209395647049\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.2414209395647049\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 128 \\ - INFO - On epoch 11, train loss 0.2286624312400818, val loss 0.21285700798034668\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.21285700798034668\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 128 \\ - INFO - On epoch 12, train loss 0.21459725499153137, val loss 0.20052719116210938\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.20052719116210938\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 128 \\ - INFO - On epoch 13, train loss 0.20111408084630966, val loss 0.2029796540737152\n",
      "2021-11-11 14:15:30 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 128 \\ - INFO - On epoch 14, train loss 0.18258561193943024, val loss 0.17640642821788788\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.17640642821788788\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 128 \\ - INFO - On epoch 15, train loss 0.17769883573055267, val loss 0.18406274914741516\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 128 \\ - INFO - On epoch 16, train loss 0.1675204187631607, val loss 0.15178079903125763\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.15178079903125763\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:31 fostool/tools/trainer.py 128 \\ - INFO - On epoch 17, train loss 0.16792144626379013, val loss 0.14602099359035492\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.14602099359035492\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 128 \\ - INFO - On epoch 18, train loss 0.17501761764287949, val loss 0.14043864607810974\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.14043864607810974\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 128 \\ - INFO - On epoch 19, train loss 0.15896540880203247, val loss 0.13558267056941986\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.13558267056941986\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 128 \\ - INFO - On epoch 20, train loss 0.1536787822842598, val loss 0.1836663782596588\n",
      "2021-11-11 14:15:31 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 128 \\ - INFO - On epoch 21, train loss 0.15330833196640015, val loss 0.127339169383049\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.127339169383049\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 128 \\ - INFO - On epoch 22, train loss 0.1464262679219246, val loss 0.12553469836711884\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.12553469836711884\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 128 \\ - INFO - On epoch 23, train loss 0.13734953850507736, val loss 0.1364860236644745\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 128 \\ - INFO - On epoch 24, train loss 0.14162034541368484, val loss 0.12794920802116394\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 128 \\ - INFO - On epoch 25, train loss 0.1439596638083458, val loss 0.12167468667030334\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.12167468667030334\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 128 \\ - INFO - On epoch 26, train loss 0.131245918571949, val loss 0.1385733187198639\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 128 \\ - INFO - On epoch 27, train loss 0.13697096705436707, val loss 0.11020970344543457\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_b2aa_14, current best val loss 0.11020970344543457\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 128 \\ - INFO - On epoch 28, train loss 0.125806525349617, val loss 0.11163068562746048\n",
      "2021-11-11 14:15:32 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 128 \\ - INFO - On epoch 29, train loss 0.1286429427564144, val loss 0.14303375780582428\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 128 \\ - INFO - On epoch 30, train loss 0.12602576985955238, val loss 0.11722218990325928\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 128 \\ - INFO - On epoch 31, train loss 0.12244827672839165, val loss 0.11089543253183365\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 128 \\ - INFO - On epoch 32, train loss 0.11862198263406754, val loss 0.1369405835866928\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 128 \\ - INFO - On epoch 33, train loss 0.12211660668253899, val loss 0.11146719753742218\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 128 \\ - INFO - On epoch 34, train loss 0.11321401968598366, val loss 0.11618875712156296\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 128 \\ - INFO - On epoch 35, train loss 0.11360686272382736, val loss 0.15730153024196625\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 128 \\ - INFO - On epoch 36, train loss 0.11201131716370583, val loss 0.12389474362134933\n",
      "2021-11-11 14:15:33 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 37, train loss 0.11164798587560654, val loss 0.11376441270112991\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 0, train loss 0.6218684613704681, val loss 0.9174932241439819\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.9174932241439819\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 1, train loss 0.48356637358665466, val loss 0.7049474120140076\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.7049474120140076\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 2, train loss 0.4019864797592163, val loss 0.5375086069107056\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.5375086069107056\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 3, train loss 0.343786284327507, val loss 0.40681466460227966\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.40681466460227966\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 4, train loss 0.3061283975839615, val loss 0.29898977279663086\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.29898977279663086\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 5, train loss 0.2710004299879074, val loss 0.2223901003599167\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.2223901003599167\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 6, train loss 0.25259408354759216, val loss 0.179073303937912\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.179073303937912\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 7, train loss 0.23375393450260162, val loss 0.16690275073051453\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.16690275073051453\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 8, train loss 0.22119949012994766, val loss 0.16712988913059235\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 9, train loss 0.21028511971235275, val loss 0.16471993923187256\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.16471993923187256\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 10, train loss 0.20276767015457153, val loss 0.16127751767635345\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.16127751767635345\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 11, train loss 0.1937788724899292, val loss 0.15597397089004517\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.15597397089004517\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 12, train loss 0.1868366375565529, val loss 0.14444275200366974\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.14444275200366974\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 128 \\ - INFO - On epoch 13, train loss 0.18406909704208374, val loss 0.14623060822486877\n",
      "2021-11-11 14:15:34 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 14, train loss 0.17783478647470474, val loss 0.13780182600021362\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.13780182600021362\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 15, train loss 0.17257457971572876, val loss 0.12753598392009735\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.12753598392009735\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 16, train loss 0.16954579204320908, val loss 0.11959992349147797\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.11959992349147797\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 17, train loss 0.16658756881952286, val loss 0.11025789380073547\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_5eb1_14, current best val loss 0.11025789380073547\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 18, train loss 0.16397182643413544, val loss 0.11134684830904007\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 19, train loss 0.16055293381214142, val loss 0.12205851078033447\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 20, train loss 0.15815556049346924, val loss 0.1362987756729126\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 21, train loss 0.15533671528100967, val loss 0.1398431360721588\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 22, train loss 0.1538809984922409, val loss 0.1265815943479538\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 23, train loss 0.1510363221168518, val loss 0.1312919706106186\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 24, train loss 0.14862485229969025, val loss 0.13382823765277863\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 25, train loss 0.14694447070360184, val loss 0.13463786244392395\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 26, train loss 0.14520879089832306, val loss 0.13067297637462616\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 27, train loss 0.1443067491054535, val loss 0.12293653935194016\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 0, train loss 0.577772706747055, val loss 1.0590466260910034\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 1.0590466260910034\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 1, train loss 0.5452745854854584, val loss 0.9661420583724976\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.9661420583724976\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 2, train loss 0.5499700009822845, val loss 0.9161409735679626\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.9161409735679626\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 128 \\ - INFO - On epoch 3, train loss 0.5405254065990448, val loss 0.8712778091430664\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:35 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.8712778091430664\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 4, train loss 0.5215187966823578, val loss 0.822579026222229\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.822579026222229\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 5, train loss 0.504781499505043, val loss 0.7849721312522888\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.7849721312522888\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 6, train loss 0.48410122096538544, val loss 0.7421674132347107\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.7421674132347107\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 7, train loss 0.4950513541698456, val loss 0.7017943859100342\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.7017943859100342\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 8, train loss 0.5054904818534851, val loss 0.6729130148887634\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.6729130148887634\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 9, train loss 0.4880274534225464, val loss 0.6405434012413025\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.6405434012413025\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 10, train loss 0.4538150280714035, val loss 0.6172938346862793\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.6172938346862793\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 11, train loss 0.4704024940729141, val loss 0.6018126010894775\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.6018126010894775\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 12, train loss 0.4755554646253586, val loss 0.5920253992080688\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.5920253992080688\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 13, train loss 0.4673824906349182, val loss 0.5800316333770752\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.5800316333770752\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 14, train loss 0.4204444885253906, val loss 0.5597347617149353\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.5597347617149353\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 15, train loss 0.4113401174545288, val loss 0.5470089316368103\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.5470089316368103\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 16, train loss 0.4231562465429306, val loss 0.5253322720527649\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.5253322720527649\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 17, train loss 0.37680698931217194, val loss 0.4930296838283539\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.4930296838283539\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 18, train loss 0.37924791872501373, val loss 0.46332287788391113\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.46332287788391113\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 19, train loss 0.3513939678668976, val loss 0.4349636733531952\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.4349636733531952\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 20, train loss 0.32729001343250275, val loss 0.41378113627433777\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.41378113627433777\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 128 \\ - INFO - On epoch 21, train loss 0.30249299108982086, val loss 0.3886920213699341\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:36 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.3886920213699341\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 22, train loss 0.2902710288763046, val loss 0.37024086713790894\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.37024086713790894\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 23, train loss 0.2655092030763626, val loss 0.343049556016922\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.343049556016922\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 24, train loss 0.2429555281996727, val loss 0.3149363696575165\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.3149363696575165\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 25, train loss 0.21670860797166824, val loss 0.29455119371414185\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.29455119371414185\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 26, train loss 0.19650879502296448, val loss 0.27794671058654785\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.27794671058654785\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 27, train loss 0.17310338467359543, val loss 0.26174530386924744\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.26174530386924744\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 28, train loss 0.16181616485118866, val loss 0.24940170347690582\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.24940170347690582\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 29, train loss 0.14546076953411102, val loss 0.22581085562705994\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.22581085562705994\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 30, train loss 0.13840437680482864, val loss 0.22890672087669373\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 31, train loss 0.13217563182115555, val loss 0.21116232872009277\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.21116232872009277\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 32, train loss 0.14049670845270157, val loss 0.21899639070034027\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 33, train loss 0.13120657205581665, val loss 0.2256985753774643\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 34, train loss 0.1237565279006958, val loss 0.1849677711725235\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 136 \\ - INFO - For model KRNNModel_5c90_21, current best val loss 0.1849677711725235\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 35, train loss 0.12749719992280006, val loss 0.18587499856948853\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 36, train loss 0.1231389008462429, val loss 0.20072869956493378\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 37, train loss 0.12095645442605019, val loss 0.21771851181983948\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 38, train loss 0.11944562941789627, val loss 0.2139943242073059\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 128 \\ - INFO - On epoch 39, train loss 0.1132764033973217, val loss 0.2060302346944809\n",
      "2021-11-11 14:15:37 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 128 \\ - INFO - On epoch 40, train loss 0.11504939571022987, val loss 0.21019762754440308\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 128 \\ - INFO - On epoch 41, train loss 0.11305584013462067, val loss 0.2370913326740265\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 128 \\ - INFO - On epoch 42, train loss 0.11171175166964531, val loss 0.25909268856048584\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 128 \\ - INFO - On epoch 43, train loss 0.10788455232977867, val loss 0.23453669250011444\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 128 \\ - INFO - On epoch 44, train loss 0.10709705948829651, val loss 0.21734027564525604\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 128 \\ - INFO - On epoch 0, train loss 0.5980601012706757, val loss 0.9694886207580566\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.9694886207580566\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 128 \\ - INFO - On epoch 1, train loss 0.5306427776813507, val loss 0.8687199354171753\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.8687199354171753\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 128 \\ - INFO - On epoch 2, train loss 0.5313378274440765, val loss 0.8179412484169006\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.8179412484169006\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 128 \\ - INFO - On epoch 3, train loss 0.508684515953064, val loss 0.746424674987793\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:38 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.746424674987793\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 128 \\ - INFO - On epoch 4, train loss 0.48511914908885956, val loss 0.6498120427131653\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.6498120427131653\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 128 \\ - INFO - On epoch 5, train loss 0.4629763215780258, val loss 0.5489436388015747\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.5489436388015747\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 128 \\ - INFO - On epoch 6, train loss 0.4254409074783325, val loss 0.49133387207984924\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.49133387207984924\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 128 \\ - INFO - On epoch 7, train loss 0.3875773400068283, val loss 0.46054187417030334\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.46054187417030334\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 128 \\ - INFO - On epoch 8, train loss 0.3588222861289978, val loss 0.42895761132240295\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.42895761132240295\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 128 \\ - INFO - On epoch 9, train loss 0.31732015311717987, val loss 0.34649959206581116\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:39 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.34649959206581116\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 128 \\ - INFO - On epoch 10, train loss 0.2622882202267647, val loss 0.28953641653060913\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.28953641653060913\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 128 \\ - INFO - On epoch 11, train loss 0.2287004590034485, val loss 0.25129836797714233\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.25129836797714233\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 128 \\ - INFO - On epoch 12, train loss 0.1936206892132759, val loss 0.23046232759952545\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.23046232759952545\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 128 \\ - INFO - On epoch 13, train loss 0.16879475116729736, val loss 0.2658417522907257\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 128 \\ - INFO - On epoch 14, train loss 0.1571466624736786, val loss 0.20644652843475342\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.20644652843475342\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 128 \\ - INFO - On epoch 15, train loss 0.13896790146827698, val loss 0.1710038036108017\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.1710038036108017\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 128 \\ - INFO - On epoch 16, train loss 0.12600290775299072, val loss 0.20549549162387848\n",
      "2021-11-11 14:15:40 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 128 \\ - INFO - On epoch 17, train loss 0.12642190977931023, val loss 0.1570553332567215\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.1570553332567215\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 128 \\ - INFO - On epoch 18, train loss 0.12549729645252228, val loss 0.1454012095928192\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.1454012095928192\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 128 \\ - INFO - On epoch 19, train loss 0.10693679004907608, val loss 0.17400552332401276\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 128 \\ - INFO - On epoch 20, train loss 0.11154115572571754, val loss 0.12965363264083862\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:41 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.12965363264083862\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 128 \\ - INFO - On epoch 21, train loss 0.09823064506053925, val loss 0.1291753202676773\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.1291753202676773\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 128 \\ - INFO - On epoch 22, train loss 0.10100039467215538, val loss 0.13323839008808136\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:41 fostool/tools/trainer.py 128 \\ - INFO - On epoch 23, train loss 0.09880898147821426, val loss 0.12909503281116486\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.12909503281116486\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 128 \\ - INFO - On epoch 24, train loss 0.09568032249808311, val loss 0.12953659892082214\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 128 \\ - INFO - On epoch 25, train loss 0.08963852003216743, val loss 0.12609662115573883\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.12609662115573883\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 128 \\ - INFO - On epoch 26, train loss 0.08754115924239159, val loss 0.12146542966365814\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.12146542966365814\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 128 \\ - INFO - On epoch 27, train loss 0.08422372862696648, val loss 0.12412112951278687\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 128 \\ - INFO - On epoch 28, train loss 0.0833200216293335, val loss 0.12108147889375687\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.12108147889375687\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 128 \\ - INFO - On epoch 29, train loss 0.08637012913823128, val loss 0.12178431451320648\n",
      "2021-11-11 14:15:42 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 128 \\ - INFO - On epoch 30, train loss 0.08180632442235947, val loss 0.12592555582523346\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 128 \\ - INFO - On epoch 31, train loss 0.08036371693015099, val loss 0.12158860266208649\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 128 \\ - INFO - On epoch 32, train loss 0.08171738684177399, val loss 0.11939498782157898\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 136 \\ - INFO - For model SandwichModel_ff1a_21, current best val loss 0.11939498782157898\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 128 \\ - INFO - On epoch 33, train loss 0.07620701566338539, val loss 0.12461023777723312\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 128 \\ - INFO - On epoch 34, train loss 0.07578901574015617, val loss 0.1238127201795578\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 128 \\ - INFO - On epoch 35, train loss 0.07644329220056534, val loss 0.12392397969961166\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 128 \\ - INFO - On epoch 36, train loss 0.07271324098110199, val loss 0.13151752948760986\n",
      "2021-11-11 14:15:43 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 128 \\ - INFO - On epoch 37, train loss 0.07496359571814537, val loss 0.12263432890176773\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 128 \\ - INFO - On epoch 38, train loss 0.0749073475599289, val loss 0.12353097647428513\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 128 \\ - INFO - On epoch 39, train loss 0.0712229385972023, val loss 0.12882301211357117\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 128 \\ - INFO - On epoch 40, train loss 0.0721512921154499, val loss 0.12447108328342438\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 128 \\ - INFO - On epoch 41, train loss 0.06995263695716858, val loss 0.12400712072849274\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 128 \\ - INFO - On epoch 42, train loss 0.06694117933511734, val loss 0.12539255619049072\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 128 \\ - INFO - On epoch 0, train loss 0.8915237188339233, val loss 2.0648345947265625\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:44 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 2.0648345947265625\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 1, train loss 0.6819222569465637, val loss 1.5201629400253296\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 1.5201629400253296\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 2, train loss 0.5558416843414307, val loss 1.0965243577957153\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 1.0965243577957153\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 3, train loss 0.4752289056777954, val loss 0.7989023923873901\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.7989023923873901\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 4, train loss 0.41008418798446655, val loss 0.581897497177124\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.581897497177124\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 5, train loss 0.3524443954229355, val loss 0.42024195194244385\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.42024195194244385\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 6, train loss 0.30850282311439514, val loss 0.31394436955451965\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.31394436955451965\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 7, train loss 0.26325053721666336, val loss 0.24890796840190887\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.24890796840190887\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 8, train loss 0.23259367793798447, val loss 0.20851270854473114\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.20851270854473114\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 9, train loss 0.21439716219902039, val loss 0.18368573486804962\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.18368573486804962\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 10, train loss 0.19818440824747086, val loss 0.166077122092247\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.166077122092247\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 11, train loss 0.18535909801721573, val loss 0.1528259962797165\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.1528259962797165\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 12, train loss 0.18406111001968384, val loss 0.14431732892990112\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.14431732892990112\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 13, train loss 0.18111687898635864, val loss 0.14123320579528809\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.14123320579528809\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 14, train loss 0.16339551657438278, val loss 0.13889794051647186\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.13889794051647186\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 15, train loss 0.15396948158740997, val loss 0.1345948427915573\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.1345948427915573\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 16, train loss 0.14292233437299728, val loss 0.12846024334430695\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.12846024334430695\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 17, train loss 0.1462065875530243, val loss 0.12104033678770065\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.12104033678770065\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 18, train loss 0.13624054193496704, val loss 0.11333134770393372\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.11333134770393372\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 19, train loss 0.1353704109787941, val loss 0.11041393876075745\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.11041393876075745\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 20, train loss 0.1322757937014103, val loss 0.10943283885717392\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.10943283885717392\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 21, train loss 0.12949059903621674, val loss 0.10735538601875305\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.10735538601875305\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 22, train loss 0.12988434731960297, val loss 0.10683534294366837\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.10683534294366837\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 23, train loss 0.1199214830994606, val loss 0.10927733033895493\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 24, train loss 0.11524248868227005, val loss 0.11178633570671082\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 128 \\ - INFO - On epoch 25, train loss 0.11462756246328354, val loss 0.10868146270513535\n",
      "2021-11-11 14:15:45 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 26, train loss 0.11144504323601723, val loss 0.10283633321523666\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.10283633321523666\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 27, train loss 0.11179768666625023, val loss 0.09745516628026962\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.09745516628026962\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 28, train loss 0.10479810833930969, val loss 0.0964360311627388\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.0964360311627388\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 29, train loss 0.10669497027993202, val loss 0.09576395899057388\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.09576395899057388\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 30, train loss 0.10532303899526596, val loss 0.09552318602800369\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.09552318602800369\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 31, train loss 0.10176360607147217, val loss 0.0952601283788681\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.0952601283788681\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 32, train loss 0.09915884584188461, val loss 0.09528665989637375\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 33, train loss 0.09998460859060287, val loss 0.09668238461017609\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 34, train loss 0.09762973710894585, val loss 0.09848382323980331\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 35, train loss 0.09749258682131767, val loss 0.09904992580413818\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 36, train loss 0.09478767961263657, val loss 0.09650077670812607\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 37, train loss 0.09311595559120178, val loss 0.09450780600309372\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 136 \\ - INFO - For model MLP_Res_c5cf_21, current best val loss 0.09450780600309372\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 38, train loss 0.09231722727417946, val loss 0.09557486325502396\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 39, train loss 0.09181949868798256, val loss 0.09991221129894257\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 40, train loss 0.09182250872254372, val loss 0.10363520681858063\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 41, train loss 0.0921165719628334, val loss 0.10350822657346725\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 42, train loss 0.08867720514535904, val loss 0.10065094381570816\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 43, train loss 0.08876508474349976, val loss 0.09790736436843872\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 44, train loss 0.08764931932091713, val loss 0.09880299866199493\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 45, train loss 0.08975846320390701, val loss 0.1016286239027977\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 46, train loss 0.08765417337417603, val loss 0.10523726791143417\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 128 \\ - INFO - On epoch 47, train loss 0.0855916403234005, val loss 0.10733595490455627\n",
      "2021-11-11 14:15:46 fostool/tools/trainer.py 130 \\ - INFO - ------------\n"
     ]
    }
   ],
   "source": [
    "fost.fit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predict\n",
    "\n",
    "Predict results using the models trained in the previous step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-10 16:16:06 fostool/task/fusion.py 67 \\ - INFO -    val_loss            model_name\n",
      "5  0.099834       MLP_Res_c5cf_21\n",
      "6  0.125119  SandwichModel_a54c_7\n"
     ]
    }
   ],
   "source": [
    "result = fost.predict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Plot\n",
    "\n",
    "Display of predicted data.\n",
    "\n",
    "You are encouraged to change params like `node_name`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2021-11-10 16:16:06 fostool/visualizer/plot.py 66 \\ - INFO - Unspecified lookback_size, use default lookback_size: 25.\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAscAAAGtCAYAAAD6cdmPAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAB8xUlEQVR4nO3dd3hUVfoH8O+UJJPey6SQkJCQ0BJIKIl0CxZEV0FREVER67pYVt21rLqu7M91V7GLooCo2BXBgggBhEjvJQWSQHpPZlKnnN8fKVJCSLkzd2by/TyPzyNz79z7zkkyeXPmPe9RCCEEiIiIiIgISrkDICIiIiKyFUyOiYiIiIjaMDkmIiIiImrD5JiIiIiIqA2TYyIiIiKiNkyOiYiIiIjaMDkmIuojhUKBnJwci97j448/xmWXXSbJtZYtW4bx48dLci25nf1aPDw8cOLECRkjIiJ7x+SYiGxSVFQUXF1d4eHhgeDgYMybNw96vV7usGRzyy23YN26dVa/77PPPos5c+ZY/b69pdfrER0d3eU5eXl5UCgUMBqNVoqKiOwJk2Mislnff/899Ho99uzZg127duGFF16QOySyICEEzGaz3GEQUT/H5JiIbF5YWBiuuOIKHDp0CADw+++/Iy0tDT4+PkhMTER6enrHuZMnT8bTTz+Niy66CJ6enrjssstQUVEB4I8Zw+XLl2PAgAEICAjAv/71r47n7tixAykpKfDy8kJwcDAefvhhAMBVV12F119//YyYRowYgW+++eaMx7Zv346QkBCYTKaOx7755huMGDGi4/qpqanw8fGBVqvFAw88gJaWlo5zFQoF3nnnHcTGxsLHxwf3338/2jcxPbt84C9/+QsiIiLg5eWF5ORkbNmy5bzjV1lZiRkzZsDLywtjxozB8ePHzzh+vmv99NNPePHFF/HZZ5/Bw8MDiYmJAIAPP/wQCQkJ8PT0RHR0NN59993z3nvZsmW46KKL8MADD8Db2xvx8fH49ddfO45PnjwZTz75JC666CK4ubnhxIkTOHbsGC699FL4+flh8ODB+Pzzz7v9Wk4vcWlsbMQjjzyCyMhIeHt7Y/z48WhsbMTEiRMBAD4+PvDw8EBGRsZ54yeifkgQEdmgyMhI8csvvwghhDh58qQYMmSIeOqpp0RBQYHw8/MTa9euFSaTSaxbt074+fmJsrIyIYQQkyZNEtHR0SIzM1M0NDSISZMmiccff1wIIURubq4AIObPny8aGhrEvn37hLOzszhy5IgQQohx48aJFStWCCGE0Ol0IiMjQwghxGeffSbGjBnTEdu+ffuEn5+faG5uFkIIAUBkZ2cLIYSIjo4W69at6zh35syZYtGiRUIIIXbt2iUyMjKEwWAQubm5Ij4+Xrzyyisd5wIQV111laiurhb5+fkiICBA/Pjjj0IIIT788ENx0UUXdZz70UcfiYqKCmEwGMTLL78sgoODRWNjY6djeeONN4pZs2YJvV4vDh48KEJDQ7t9rX/84x/illtuOeN6a9asETk5OcJsNov09HTh6uoqdu/e3em9P/zwQ6FSqcT//vc/0dLSIlatWiW8vLxEZWVlx9crIiJCHDp0SBgMBlFTUyPCw8PFBx98IAwGg9izZ4/w9/cXhw8f7tZrOf1rcd9994lJkyaJgoICYTQaxdatW0VTU1PH94HBYOg0ZiLq3+wyOb799ttFYGCgGDp06AXPXbhwoUhMTBSJiYkiNjZWeHt7Wz5AIuqzyMhI4e7uLry9vcWAAQPEvffeKxoaGsS///1vMWfOnDPOveyyy8SyZcuEEK3J1j//+c+OY2+++aaYNm2aEOKP5PjUqVMdx0ePHi0+/fRTIYQQEyZMEM8884woLy8/4/qNjY3Cx8dHZGVlCSGEeOSRR8S9997bcfz0hOzJJ58Ut99+uxBCiLq6OuHm5iby8vI6fY2vvPKKuPbaa8+4zpYtWzr+PWvWrI7E+uzk+Gw+Pj5i37595zxuNBqFWq0WR48e7Xjsb3/7W7ev1VlyfLZrrrlGvPrqq50e+/DDD4VWqxVms7njsdGjR3f8ETJp0iTx9NNPdxxbtWqVGD9+/BnXWLBggXj22We79VravxYmk0loNJpOx4TJMRF1xS7LKubNm4effvqpW+e+8sor2LdvH/bt24c///nPuO666ywcHRFJ5dtvv0VNTQ3y8/Px1ltvwdXVFfn5+fjiiy/g4+PT8d9vv/2G4uLijueFhIR0/L+bm9s5C/nOd3zp0qXIyspCfHw8Ro8ejTVr1gAANBoNbrzxRqxcuRJmsxmffvopbr311k5jvvnmm/H111+jubkZX3/9NUaNGoXIyEgAQFZWFqZPn46QkBB4eXnh73//e0fJR3djb/fyyy8jISEB3t7e8PHxQW1t7TnXAoDy8nIYjUZERER0PNYeT0+v1e7HH3/EuHHj4OfnBx8fH/zwww9dnh8WFgaFQnHG/YuKijr+fXps+fn52L59+xlf348//hglJSXdei3tKioq0NTUhJiYmPPGRUTUGbtMjidOnAg/P78zHjt+/Dguv/xyJCcnY8KECTh27Ng5z/v0009x0003WStMIrKAiIgI3Hrrraipqen4r76+Hk888USfrx0bG4tPP/0UZWVlePzxxzFz5kzU19cDAG677TZ8/PHH+PXXX+Hm5obU1NROrzFkyBBERkbixx9/xCeffIKbb76549i9996L+Ph4ZGdno66uDi+++GJHTXFPbNmyBS+99BI+//xzVFdXo6amBt7e3p1eKzAwEGq1GqdOnep47OTJk92+1ulJLQA0Nzfj+uuvx6OPPorS0lLU1NTgyiuv7PJ1FBYWnnH85MmTCA0N7fj36feIiIjApEmTzvj66vV6vP322xd8LacLCAiARqM5pya5s9dERHQ6u0yOO7NgwQK8/vrr2L17N15++WXcd999ZxzPz89Hbm4upk6dKlOERCSFOXPm4Pvvv8fPP/8Mk8mEpqYmpKeno6CgoM/XXrlyJcrLy6FUKuHj4wMAUCpb3yZTU1OhVCrxyCOPnHfWuN3NN9+MxYsXY/PmzZg1a1bH4zqdDl5eXvDw8MCxY8fw9ttv9ypOnU4HtVqNwMBAGI1GPP/886irq+v0XJVKheuuuw7PPvssGhoacOTIESxfvrzb1woODkZeXl5HF4mWlhY0Nzd3JKo//vjjBVvMlZWV4bXXXoPBYMAXX3yBo0eP4sorr+z03OnTpyMrKwsfffQRDAYDDAYDdu7ciaNHj17wtZxOqVTijjvuwMMPP4yioiKYTCZkZGR0xK5UKtkPmYg65RDJsV6vx7Zt2zBr1iwkJSXh7rvvPuMjVgBYtWoVZs6cCZVKJVOURCSFiIgIfPfdd3jxxRcRGBiIiIgI/Oc//5GkBdhPP/2EoUOHwsPDA3/5y1+watUquLq6dhyfO3cuDh48eMG+vzfddBM2bdqEqVOnIiAgoOPxl19+GZ988gk8PT1x11134cYbb+xVnNOmTcPll1+OuLg4REZGQqPRnFFqcLY33ngDer0eISEhmDdvHm6//fZuX6s9uff398eoUaPg6emJ1157DTfccAN8fX3xySefYMaMGV3GO3bsWGRnZyMgIABPPvkkvvzyS/j7+3d6rqenJ9atW4dVq1YhNDQUISEhePzxx9Hc3HzB13K2l19+GcOHD8fo0aPh5+eHxx9/HGazGW5ubh0dMnx8fPD77793GT8R9S8K0ZvP9GxAXl4epk+fjkOHDqGurg6DBw8+JyE+3ciRI/Hmm28iLS3NilESkSNZsWIFlixZgt9++03uUOzGsmXL8P7773PMiMhuOMTMsZeXFwYOHIgvvvgCQGsj+f3793ccP3bsGKqrq89bI0hEdCENDQ146623sGDBArlDISIiC7LL5Pimm25CamoqMjMzER4ejqVLl+Ljjz/G0qVLkZiYiKFDh+K7777rOH/VqlWYPXs2F2EQUa/8/PPPCAwMRHBw8BkL7IiIyPHYbVkFEREREZHU7HLmmIiIiIjIEtRyB9BTAQEBiIqKkjsMm2QwGODk5CR3GHaP4ygdjqU0OI7S4VhKg+MoDY6jdLozlnl5eV1uWNTO7pLjqKgo7Nq1S+4wbFJRUdEZjfWpdziO0uFYSoPjKB2OpTQ4jtLgOEqnO2OZkpLSrWuxrIKIiIiIqA2TYyIiIiKiNkyOiYiIiIja2F3NMRERERG1LkIrKChAU1OT3KHIzmQyoba2FgCg0WgQHh7e68WOTI6JiIiI7FBBQQE8PT0RFRXV7zc6a2lpgbOzM4QQqKysREFBAQYOHNira7GsgoiIiMgONTU1wd/fv98nxqdTKBTw9/fv02w6k2MiIiIiO8XE+Fx9HRMmx0REREREbZgcExEREVGPVVZWIikpCUlJSQgJCUFYWFjHv1taWrp87q5du/Dggw9aKdKe4YI8IiIiIuoxf39/7Nu3DwDw7LPPwsPDA48++mjHcaPRCLW681QzJSWl2zvWWRtnjomIiIhIEvPmzcM999yDsWPH4rHHHsOOHTuQmpqKkSNHIi0tDZmZmQCA9PR0TJ8+HUBrYn3HHXdg8uTJiI6OxmuvvSbnS+DMMREREZG9e+77wzhSVCfpNYeEeuEfVw/t8fMKCgqwbds2qFQq1NXVYcuWLVCr1Vi/fj3+/ve/46uvvjrnOceOHcPGjRuh0+kwePBg3Hvvvb3uU9xXTI6JiIiISDKzZs2CSqUCANTW1uK2225DdnY2FAoFDAZDp8+56qqr4OLiAhcXFwQFBaG0tBTh4eHWDLsDk2MiIiIiO9ebGV5LcXd37/j/p59+GlOmTME333yDvLw8TJ48udPnuLi4dPy/SqWC0Wi0dJjnxZpjIqJ+xGgy44PfcpFTppc7FCLqB2praxEWFgYAWLZsmbzBdBOTYyKifqLJYMJ9H+/B82uO4PUN2XKHQ0T9wGOPPYa//e1vGDlypKyzwT3Bsgoion5A12TAXSt24fcTVQj3dcW245UQQnB3LSKSxLPPPtvp46mpqcjKyur49wsvvAAAmDx5ckeJxdnPPXTokCVC7DbOHBMRObgKfTNueu937MqrxuLZSXhgyiCU65pxvJylFUREZ+PMMRGRAyuobsCtS3eguLYR792WgimDg3CysgEAsO14JQYFecocIRGRbeHMMRGRg8oq1WHm2xmo1Dfj4/ljMWVwEAAgws8VYT6u2JZTKXOERES2h8kxEZED2nOyGrPeyYBZCHx+TyqSI/06jikUCqTF+OP33EqYzULGKImIbA+TYyIiB7Mpqxy3vLcdPm5O+OreNMSHeJ1zTtogf9Q0GHC0RNodtYiI7B2TYyIiB/L9/iLMX74TUQHu+PKeNET4uXV6Xmp0AAAg4zhLK4iITmfx5NhkMmHkyJGYPn36Ocfy8/Nx8cUXY8SIEZg8eTIKCgosHQ4RkcP66Pd8PLhqL0YO8MVnd49DoKfLec8N8dYgOsAd25gcE1EfqFQqJCUlYdiwYZg1axYaGhp6fa158+bhyy+/BADMnz8fR44cOe+56enp2LZtW6/v1RWLJ8eLFy9GQkJCp8ceffRRzJ07FwcOHMAzzzyDv/3tb5YOh4jI4Qgh8Nqv2Xj620O4OD4IK+4YAy+N0wWflxrjj+0nKmEwma0QJRE5IldXV+zbtw+HDh2Cs7Mz3nnnnTOO93bjj/fffx9Dhgw573G7TY4LCgqwdu1azJ8/v9PjR44cwdSpUwEAU6ZMwXfffWfJcIiIHI7ZLPDc90fwv1+ycP2ocLwzJxkaJ1W3npsWE4D6FhMOFtZaOEoi6g8mTJiAnJwcpKenY8KECZgxYwaGDBkCk8mEv/71rxg9ejRGjBiBd999F0DrH/YPPPAABg8ejEsuuQRlZWUd15o8eTJ27doFAPjpp58watQoJCYm4uKLL0ZeXh7eeecdvPLKK0hKSsKWLVskfR0W7XO8cOFCvPTSS9DpdJ0eT0xMxNdff42//OUv+Oabb6DT6VBZWQl/f/8zzluyZAmWLFkCACgpKUFRUZElw7Zb5eXlcofgEDiO0uFYSuN842g0CbywPh/rMqsxe2QgHrgoAGWlJd2+bpSbAQDw895chKgbJYnV1vF7UhocR2n0dRxNJhNaWloAAKpfnoSiVNqd5UTwMJgu/dcFz2tpaYHRaMTatWtx2WWXwWAwYM+ePdizZw8GDhyId999Fx4eHti6dSuam5s7dsfbv38/jh49in379qG0tBRJSUmYO3cuWlpaIISAwWBAYWEh7rrrLqxfvx4DBw5EVVUV/Pz8cNddd8Hd3R0PP/wwgHNnqE0mU6/zRYslx2vWrEFQUBCSk5ORnp7e6Tkvv/wyHnjgASxbtgwTJ05EWFgYVKpzZzwWLFiABQsWAABSUlIQGhpqqbDtHsdGGhxH6XAspXH2ODa2mHDfx7uxMbMaj10+GPdOiunxVtChAOJD8nG43NCvvk796bVaEsdRGn0Zx9raWjg7O7f+Q6kCFBIXBChVULVf/zwaGxsxZswYAK0zx3fffTe2bduGMWPGYPDgwQCADRs24MCBA/jmm2864s7Pz8e2bdtwyy23wNXVFVFRUZg6dSrUajWcnZ2hUCjg5OSEPXv2YOLEiR3XCgkJAdBa69x+brvT/1+lUvV6bC2WHG/duhWrV6/GDz/8gKamJtTV1WHOnDlYuXJlxzmhoaH4+uuvAQB6vR5fffUVfHx8LBUSEZFDqG0w4I7lO7H3ZDUWXTccN40Z0OtrpcUE4OPt+Wg2muCi7l45BhHZoCv+Lctt22uOz+bu7t7x/0IIvP7665g2bdoZ5/zwww+WDq9XLFZzvGjRIhQUFCAvLw+rVq3C1KlTz0iMAaCiogJms7nj/DvuuMNS4RAROYTSuibc8G4GDhbU4s2bR/UpMQZaF+U1G83Ye7JGmgCJiM4ybdo0vP322zAYWku5srKyUF9fj4kTJ+Kzzz6DyWRCcXExNm7ceM5zx40bh82bNyM3NxcAUFVVBQDw9PQ8b9luX1m9z/EzzzyD1atXA2hdaTh48GDExcWhtLQUTz75pLXDISKyG3kV9Zj5zjYUVDfgw9tH44rh2j5fc8xAPygVYEs3IrKY+fPnY8iQIRg1ahSGDRuGu+++G0ajEX/6058QGxuLIUOGYO7cuUhNTT3nuYGBgViyZAmuu+46JCYm4sYbbwQAXH311fjmm28ssiBPIYSwq71DU1JSOlYv0pmKiopYAyYBjqN0OJbSKCoqQjXccdsHO2Eym7Hs9jFIjPCR7PrXvPEbnNVKfHFPmmTXtFX8npQGx1EafR3Ho0ePnrddbn/T0tJyRs1xZ2PT3RySO+QREdm4vYV6zH73dzipFPjinjRJE2MASI0JwN6TNWho6V0/UiIiR8LkmIjIhv2WXYGHvs1BkJcLvro3DYOCPCS/R1qMP4xmgZ151ZJfm4jI3jA5JiKyUUaTGU99exBaL2d8cU8aQn1cLXKflChfOKkUyGDdMZHdsbPqWKvo65gwOSYislFrDhQjr7IB96SFws+9616jfeHmrMbICF9kHK+w2D2ISHoajQaVlZVMkE8jhEBlZSU0Gk2vr2HRHfKIiKh3TGaB1zdkIz7EExOivS1+v3Ex/nhjQzZqGw3wdnWy+P2IqO/Cw8NRUFDAHQvRuiNe+0ZyGo0G4eHhvb4Wk2MiIhv0w8FiHC+vxxs3j4SyZxvf9UpajD9e+zUbO3KrcOmQYMvfkIj6zMnJCQMHDpQ7DJsgZQcVllUQEdkYc9us8aAgD1wxrO+9jLtj5AAfuKiV2MbSCiLq55gcExHZmHVHSpBVqsefpw6CyhrTxgBc1CqMjvLjojwi6veYHBMR2RAhBBb/moPoAHdMH2HdTRZSY/xxrESHSn2zVe9LRGRLmBwTEdmQ9UfLcLS4DvdPsd6scbu0GH8AwO8nqqx6XyIiW8LkmIjIRggh8Nqv2Rjg54Zrkqy/Ne/wMG94uKhZd0xE/RqTYyIiG5GeWY6DhbW4f0oM1Crrvz2rVUqMGci6YyLq35gcExHZgNZa42yE+bjiTyN735+zr9Ji/HGioh7FtY2yxUBEJCcmx0RENuC3nArsO1WD+6bEwFkt31tzalvdsdyzx1mlOpyqapA1BiLqn5gcExHJTAiBxeuzofXWYGayfLPGAJAQ4gUfNydskzE5bjKYMHvJ75j++m/ILNHJFgcR9U9MjomIZJZxohK78qtxz6QYuKhVssaiVCqQGu2PjOOVEELIEsPaA8Woqm+BySwwZ+l25FfWyxIHEfVPTI6JiGT2+q85CPJ0wY2jI+QOBUBr3XFhTSNOVclTd7wiIw8xge74+r40GE1m3PL+dpTUNskSCxH1P0yOiYhktCO3ChknKnH3pBhonOSdNW7XXncsR0u3fadqsL+gFrelRSEu2BPL7xiDmgYD5izdjqr6FqvHQ0T9D5NjIiIZvb4hGwEezrh5zAC5Q+kQE+iBQE8XWeqOV2zLg4eLGteNaq29HhHug/dvS8Gpqgbc9sEO6JoMVo+JiPoXJsdERDLZnV+NLdkVuGtCNFydbWPWGAAUCgXSYvyxzcp1xxX6Zqw5UIzrR4XBw0Xd8fi4aH+8PWcUjhbX4c7lu9BkMFktJiLqf5gcExHJ5PUN2fB1c8KccZFyh3KOtBh/VOibkVOmt9o9P9t5Ci0mM25NjTrn2NT4YPzvxiTszKvCvSt3o8VotlpcRNS/MDkmIpLBgYIapGeWY/6EaLifNktqK9JiAgC0dtKwBqPJjJW/52P8oAAMCvLo9JwZiaH417XDsTGzHA9/vg8mszzdNIjIsTE5JiKSwWu/5sDb1QlzU21v1hgAIvzcEO7rim051kmO1x8tRXFt0wXH4+axA/C3K+Kx5kAxnvr2kGzt5ojIcdnedAURkYM7VFiL9UdL8fClcfDUOMkdznmlxfjj58OlMJsFlEqFRe+1fFs+wnxccXFC8AXPvXtSDOqaDHhz43F4adR44op4KBSWjY+I+g/OHBMRWdkbG3Lg6aLGbWlRcofSpdQYf9Q2GnCkuM6i98kq1SHjRCXmjIuEqptJ+KOXDcat4yLx7uYTeCv9uEXjI6L+hckxEZEVZZbo8NPhEtx+URS8XW131hgAUqPb6o4t3NJtRUYenNXKHm2ColAo8NyMofjTyDD85+dMfJSRZ7kAiahfYXJMRGRFr2/IhruzCneMHyh3KBcU4q1BdKC7RTcDqWsy4Os9hZiRGAo/d+cePVepVOClmSNwSUIwnv7uML7ZW2ChKImoP2FyTERkJTllOqw9WIzb0qLg49azRFAuaTH+2JFbBYPJMq3Tvt5dgIYWE27rpH1bdziplHjj5pFIjfbHo18cwC9HSqUNkIj6HSbHRERW8saGHLg6qTB/QrTcoXRbWkwA6ltMOFhYK/m1zWaBFRn5GDnAB8PDvXt9HY2TCu/dloJhYd64/5M92JZj/W2vichxMDkmIrKC3Ip6rN5fhDnjIntcPiCncdH+ACxTd7z1eAVOVNT3etb4dB4uaiy/fTQG+rtj/opd2Huyuu8BElG/xOSYiMgK3tyYAyeVEnfZ0awxAPi5OyNB62WRuuPl2/IR4OGMK4aHSHI9HzdnfHTnGAR4uGDehzuRWaKT5LpE1L8wOSYisrCTlQ34Zm8hbhkbiUBPF7nD6bHUaH/syqtGk8Ek2TVPVTXg12OluGnMALioVZJdN8hLg4/nj4XGSYk5S7cjv7JesmsTUf/A5JiIyMLeSs+BSqnA3ZPsa9a4XVqMP5qNZuw9WSPZNVduz4dSocDNYwdIds12EX5uWHnnWBhNZtzy/naU1DZJfg8iclxMjomILKigugFf7i7A7NERCPbSyB1Or4yJ9oNSAWRIVFrRZDDhs52nMG1oMLTerpJc82yxwZ5YfscY1DQYMGfpdlTVt1jkPkTkeJgcExFZ0DubjkOhAO6ZFCN3KL3mpXHC8HAfbJNoUd7q/UWoaTBgrgQL8boyItwH79+WglNVDbjtgx3QNRksej8icgxMjomILKS4thGf7yzArJQIhPpYZobUWtJi/LHvVA0aWox9uo4QAsu35WFwsCfGDvSTKLrzGxftj7fnjMLR4jrcuXwXWoyW6ddMRI6DyTERkYW8u+kEzELgXjueNW6XFuMPo1lgZ17fWqTtOVmDw0V1uDU1EgqFQqLoujY1PhgvXDsMO3KrsKdQb5V7EpH9YnJMRGQBZXVN+GTHSVw/KhwRfm5yh9NnKZF+cFIp+tzSbUVGHjxd1PjTyDCJIuueCXGBAIASHWuPiahrTI6JiCzg3c0nYDIL3DfF/meNAcDVWYWREb592gykTNeEHw4WY2ZKONxd1BJGd2HBni5QKoByPeuOiahrTI6JiCRWoW/Gx9vzcU1SKCL93eUORzKpMf44VFiL2obeJZirdpyCwSRw67hIiSO7MLVKiSBPDUo5c0xEF8DkmIhIYu9tOYEWoxn3TxkkdyiSSovxh1kA23N7PntsMJnx8fZ8TIwLRHSghwWiu7AQbw3K9EyOiahrTI6JiCRUVd+CjzLycXViKGJkSgItJWmADzROSmSc6HlyvO5wKUrrmnFbqvVnjduF+mhQxrIKIroAJsdERBJa+tsJNBpMeMDBZo0BwEWtwugov17VHS/PyEOEnysmDw6yQGTdo/V2RZnOACGEbDEQke1jckxEJJGahhYs35aPK4dpERvsKXc4FpEa449jJTpU6Ju7/ZyjxXXYkVuFW8dFQqW0Tvu2zmi9NWgymlHbyNljIjo/JsdERBJ5b8sJ1LcY8eeLHW/WuF1aTAAA4PcelFasyMiHi1qJG1IiLBVWt7RvVV1U0yRrHERk25gcExFJoFLfjA+35uGq4VrEh3jJHY7FDAv1goeLuttbSdc2GvDt3kJcmxQGHzdnC0fXNa2PBgBQUtcoaxxEZNuYHBMRSeCdTcfRZDBh4SVxcodiUWqVEmMHdr/u+MvdBWg0mHCrjAvx2mm9W5NjzhwTUVcsnhybTCaMHDkS06dPP+fYyZMnMWXKFIwcORIjRozADz/8YOlwiIgkV1bXhBUZ+bh2ZBgGBTlWh4rOpMb4I7eiHsW1Xc/Ams0CH2XkISXSF8PCvK0U3fkFeWqgUuCCcRNR/2bx5Hjx4sVISEjo9NgLL7yAG264AXv37sWqVatw3333WTocIiLJvbkxByazwF8ujpU7FKtorzu+0Ozx5uxy5FU2YG5alBWiujCVUoEAdycU13LmmIjOz6LJcUFBAdauXYv58+d3elyhUKCurg4AUFtbi9DQUEuGQ0QkucKaRny64xRmpYQ71G54XYkP8YSvm9MF645XZOQj0NMFlw8NsVJkFxbo4YxillUQURcsurn9woUL8dJLL0Gn03V6/Nlnn8Vll12G119/HfX19Vi/fn2n5y1ZsgRLliwBAJSUlKCoqMhiMduz8vJyuUNwCBxH6fSHsfy/X09CCIFZQ70s9t5ki+OYFOqOLVmlKCwshEJxbnu2gppmbDxWhtvHhKCirESGCDvn7WxGfqWev0f6yBa/J+0Rx1E6Uo6lxZLjNWvWICgoCMnJyUhPT+/0nE8//RTz5s3DI488goyMDNx66604dOgQlMozJ7QXLFiABQsWAABSUlI4w9wFjo00OI7SceSxzK+sxw9Hq3DL2AEYNTjKoveytXGcOsyAjTmHYNT4dDpj/uHeI1ApFbj7kqEI9tLIEGHnBvgXYldRBbRabadJPXWfrX1P2iuOo3SkGkuLlVVs3boVq1evRlRUFGbPno0NGzZgzpw5Z5yzdOlS3HDDDQCA1NRUNDU1oaKiwlIhERFJavGv2VApFbjfAXfDu5DUaH8A6LS0orHFhM92nsLlw0JsKjEGgCAPJzQbzahu4EYgjsxsFoDZLHcYZKcslhwvWrQIBQUFyMvLw6pVqzB16lSsXLnyjHMGDBiAX3/9FQBw9OhRNDU1ITAw0FIhERFJJqdMj2/3FmJuaiSCbCwBtIaYQHcEebp0mhx/t68QdU1G3GYjC/FOF+TZ2mu5qIYdKxzZznfuxt7/Xt2aJBP1kNX7HD/zzDNYvXo1AOC///0v3nvvPSQmJuKmm27CsmXL+DEXEdmFV9dnQeOkwj2TYuQORRYKhQJpMf7IOF4JIf5IQIQQWJ6RjwStF1IifWWMsHPBHk4AgBJ2rHBYZbomZBQLjKz/DcriPXKHQ3bIogvy2k2ePBmTJ08GADz//PMdjw8ZMgRbt261RghERJI5WlyHNQeKcf+UGPh7uMgdjmzSYgLw7b4i5JTpERvsCQDYlV+No8V1+Pd1w21ysqN95pi9jh3XO+kn8KXpcvzZ/ReoNr4IzPlK7pDIznCHPCKiHnrllyx4uqhx14RouUORVWrMuXXHy7flwUujxjVJYXKF1SVfVzXUSgWKOHPskMrqmvDx9nxMGzkIqvELgZz1wMntcodFdobJMRFRDxwsqMW6I6WYPyEaPm7Ococjqwg/N0T4uWLb8daF1KV1TfjpUAluHB0BV2eVzNF1TqVUINhLw7IKB/X2puMwmgUemDoIGHMX4B4IbHxB7rDIzjA5JiKLOFhQi7omo9xhSO6/v2TCx80Jd4yPkjsUm5Aa7Y/fT1TBZBb4ZPtJmITAnHGRcofVpVAfDRfkOaDSuiZ8vP0krh8V1tpe0NkdGP8QkLsZyN0id3hkR5gcE5HkDCYzZr27Da9tKZQ7FEntzq9GemY57p4YA0+Nk9zh2IS0mADUNhqwv6AGn+w4iSmDg2x+p8AQb1duIe2A3k4/DrNZ4IEpp23jnnIH4KkFNr4ICHauoO5hckxEkjtRXo8mgxnpOTVoMpjkDkcy//slEwEezrgtzbZnRq2pve74ue+PoFzXjLmptj82od6tZRWCyZLDKKltwic7TuL6UeEY4O/2xwEnV2DCI8DJbcCJjfIFSHaFyTERSe5YSR0AoMFgxoZjZTJHI42M45XYmlOJeycPgpuzVRr92IVgLw1iAt2x/1QNovzdMDHW9nvVa701aDGZUVnfIncoJJG303NaZ42ndrIhz6i5gFc4sOFfnD2mbmFyTESSyyzRQa1UwM9NjdX7iuQOp8+EEPjfL5kI9nLBLWMHyB2OzUmLCQAA3JoaBaXS9tq3nS3E2xUAUFzD0gpHUFLbhE93nMLM5HBE+Lmde4LaBZj4KFC4C8j+xfoBkt1hckxEkjtWokNMoAcuifPFhswy1DXZ91a9m7MrsDOvGg9MjYXGyTa7MMjp2pFhSIn0xczkcLlD6ZZQn9YdDYvY69ghvJWeA7MQXW/jPnIO4BMJbOTsMV0Yk2MiklxmiQ6DQzxxaZwvWoxm/HyoRO6Qek0Igf+ty0SYjytuTImQOxyblBzpiy/vTYO3q30sUtS2zRyznZv9K6ppxKodpzArJaLzWeN2Kidg0uNA8T7g2FqrxUf2ickxEUmqrsmAwppGxGs9MSTYDQP83LB6v/2WVqw/Wob9BbX4y8WxcFbzLdMR+Ls7w0ml4MyxA3g7/TgEBO6f0o1t3EfcCPgPau1cYTZbPjiyW3ynJyJJZZboAADxIZ5QKBSYkRiKrTkVKNc1yxxZz5nNAv/7JQtR/m64bpRt7vhGPadUKhDirWHNsZ0rqmnEZztbZ43DfbuYNW6nUgOTngDKDgNHv7N8gGS3mBwTkaSOdSTHXgCAGUmhMAvgh4PFcobVKz8eKsHR4jr85ZJYqFV8u3QkWm9XllXYuTc35rTNGndRa3y2YdcBgfHAxkWA2XHaTJK0+G5PRJI6VlwHT40aWu/WRU9xwZ6ID/HEd/vsa0MQk1nglfVZGBTkgRmJnDV2NFpvDcsq7FhhTSM+33UKN6REIMzHtftPVKqAyU8AFZnAoa8sFyDZNSbHRCSpzBJdR0lFuxlJodhzsganqhpkjKxnVu8vRE6ZHg9dEgeVHbQno57ReruitK4JZjM7F9ijNzfmAEDPZo3bJVwDBA8D0v8NmBxvi3vqOybHRCQZIURbcux1xuNXjwgFAHx/wD4W5hlMZixen40ErReuGBYidzhkAaE+GhhMAhX19lcL398VVDfgi12ncOPoCIT2ZNa4nVIJTPk7UHUcOLBK+gDJ7jE5JiLJFNY0QtdsxOAQzzMej/BzQ3Kkr91sCPL1ngLkVTbg4Uvj7GJTC+o5LTcCsVtvbjwOBRS9mzVuN/hKQJsEbPo/wGTffdhJekyOiUgyp3eqONuMxFAcK9F1nGOrWoxmvPZrDhLDvXFJQpDc4ZCFtNfEF7Pu2K6cqmqdNZ49JqLjD5xeUSiAKU8CNSeBvSulC5AcApNjIpJMe6eKuE6S4yuHa6FSKrB6v20vzPts1ykU1jTi4csGn1E3TY7lj+SYM8f25K30HCgVCtw7uRt9jS8k9lIgfDSw+WXAyPIa+gOTYyKSzLESHcJ8XOGlOXentEBPF6TF+OP7/cUQNrp9a5PBhDc2ZGN0lC8mxgbIHQ5ZkJ+7M5zVSibHdqR11rgAN/V11rhd++xxXQGwe3nfr0cOg8kxEUkms6Su05KKdjMSQ3GyqgH7TtVYL6ge+Hj7SZTWNePhSzlr7OgUCkVrO7callXYizc25ECpVODeyX2oNT5b9GQg8iJgy38BA78XqBWTYyKSRLPRhBPl9YjXnj85njYsBM5qJb6zwYV5DS1GvJ2eg4sG+SM1xl/ucMgKtN4abgRiJ05WNuCrPQW4ecwAhLSVxEhCoWjtXKEvAXZ9IN11ya4xOSYiSRwvq4fRLDD4rDZup/PSOGHq4CCsOVAMk431l12+LR8V+hY8fOlguUMhKwn1dmVZhZ14Y2M2VEqJao3PFjUeGDgJ+O0VoKVe+uuT3WFyTESSyCytA9B5p4rTXZMUigp9MzKOV1ojrG7RNRnw7ubjmDI4EMmRvnKHQ1YS4q1BSV2Tzf2hRmfKr6zHV3sKcfPYAQj2knDW+HRTnwLqy4EdSyxzfbIrTI6JSBLHinVwVikxMMC9y/OmxAfBw0VtU10rPvgtDzUNBs4a9zNaH1eYzAIVenYqsGVvbMiBWqnAvZMsMGvcLmIMMOhSYOtioKnOcvchu8DkmIgkcaxEh5ggDzipun5b0TipcNnQYPx4qATNRpOVoju/moYWvL/lBKYNDcbwcG+5wyErCm2rXeWiPNuVV1GPr/cW4paxkQiy1Kxxuyl/Axqrge3vWvY+ZPOYHBORJFq3je66pKLdNUlh0DUZkZ5ZbuGoLuy9LSegbzHioUvj5A6FrCyEvY5t3utts8b3TIq2/M3Cklt3zst4HWissfz9yGYxOSaiPqtpaEFJXVO3k+OLYvzh7+6M1fvl7VpRqW/Gh1vzcNVwLeK7WEhIjim0rVcuZ45tU15FPb7dV4g546wwa9xuyt+Bplog403r3I9sEpNjIuqz9p3xBnczOVarlLhqhBbrj5RC32y0ZGhdejv9OJoMJiy8hLPG/ZGPmxM0Tkq2c7NRr23IhpNKgbutMWvcLmQ4MOQa4Pe3gYYq692XbAqTYyLqs8y25DhB2/3Z1xmJoWg2mvHLkRJLhdWl4+V6LM/Iw/WjwjEoyEOWGEherRuBsJ2bLcqtqMe3ewsxZ2wkgjytNGvcbvLfgBY9sO01696XbAaTYyLqs2MldfBxc0KQp0u3nzNqgC/CfFyxWoYNQYQQeHb1YWicVHjs8nir359sh9Zbg6JallXYmtd/zYazWom7Ldmh4nyCEoBh17cuzNPLvy6CrI/JMRH12bESHQYHe/Zoy2WlUoGrE0OxJbsCVfUtFozuXD8dKsGW7Ao8cmkcAnuQ0JPj0Xq7sqzCxhwv1+PbfYWYmxol38/n5CcAYxOw9VV57k+yYnJMRH1iNgtkleh6VFLRbkZiKIxmgR8OFlsgss41tBjxzzVHEB/iiTnjIq12X7JNoT4alNY1wWgyyx0KtXljQw5c1CosmGjFWuOzBcQCI2YDO98H6qz3/kS2gckxEfVJQXUj6ltM3V6Md7oErScGBXlYtbTizY05KKptwj+vHQb1BXoyk+ML8dbALIAyHTcCsQXHy/X4bl8h5qZGIsBD5k91Jj0GmI2t20pTv8LfDETUJ8dKWneT6k1yrFAocE1iKHbkVVmlndaJcj2WbD6B60aGYXSUn8XvR7avvZ0bF+XZhtd+zZZ/1rid30Ag6RZg94dAbYHc0ZAVMTkmoj5p71QxOLjnyTEAXJ0YCgBYc8Cys8dCCDz7/RFo1Co8cSUX4VErrU/7RiBclCe3nDI9Vu8vwty0SPjLPWvcbuKjgBDA5pfljoSsiMkxEfXJsRIdBvi5wd1F3avnRwW4IzHCB99ZuLTi58Ol2JxVjocujbN+ayiyWVqvtpnjGs4cy+21X7Ph6qTC3RNl6FBxPj4DgOTbgL0fAdV5ckdDVsLkmIj65FhJXa9KKk43IzEUh4vqcLxcL1FUZ2psMXUswpubykV49AcvVzXcnFUsq5BZdqkO3x8owm1pUfBzd5Y7nDNNeARQqIDN/5E7ErISJsdE1GtNBhNyK+qR0MfkePoILRQKWGxh3lvpOSisacRzM4ZyER6doXUjEA3LKmT22oYcuDmpcNcEG6g1PptXKDD6TmDfp0DZMbmjISvgbwki6rWcMj3MAhgc0vM2bqcL9tIgNdofq/cXQQghUXStcivq8e6mE7g2KRRjo/0lvTY5Bq23K4o4cyybvIp6rLHVWeN24x8CNN7A8ulA4W65oyELY3JMRL12rH0xXh9njoHW0orcinocKqzr87XaCSHw3PeH4axW4u9XJkh2XXIsWm8NSjhzLJvd+dUQArhuVLjcoZyfRxBw5zrAyQ348Crg2A9yR0QWxOSYiHrtWHEdXNRKRPm79flaVwzTwkmlwHf7CiWIrNUvR0qRnlmOhZfEIsiLi/Coc1ofV5TpmmHgRiCyyC7Tw0mlkOR9xKICYoH561u3l/7sFmDHe3JHRBbC5JjoNBX6ZqzYWYLSOn7E2h2ZpTrEBntIUsfr7eaESXFBWHOgGGZz30srmgwmPL/mCOKCPXBbWlSfr0eOS+utgRDgz71Mcsp0iA6Q5n3E4jyCgHlrgLjLgR8eBdY9BZj5R5WjsYPvRCLr0DcbcdsHO/BORjGmvpyOJZuPo8XIN72uHCvRYXBw3+qNTzcjKRQldU3YkVfV52u9lX4cBdWNeP6aYXCyh1+6JButd3uvYybHcsgu02NQsIfcYXSfsztw40pg9F3AtteBL28HDPzecST8jUEEoMVoxj0f7caxEh2emBqBcdH+ePGHY7hi8Wb8ll0hd3g2qVLfjHJdMxK0fa83bndJQhDcnFV97nmcX1mPdzYdx4zEUIzjIjy6gFAf7pInl8YWE05WNSA2yI6SYwBQqoAr/wNc9gJw5FtgxTVAQ9//qCfbwOSY+j2zWeDRL/bjt5wK/N/1IzBjWACWzhuNpbelwGASmLN0O+77eDcKrbC9sT3JlHAxXjs3ZzUuHRKMHw8V92nW/rnvj8BJqcCTV3ERHl1YSPvMMX/Gre54uR5CAHG93GFTVgoFkPZnYNYyoGgvsPRSoOqE3FGRBJgcU78mhMC/fjiK1fuL8NjlgzEz+Y/V0hcnBGPdQxPxyKVx2HCsDJf8dxPe3JiDZqNJxohtR3univg+tnE72zVJoahpMGBLdnmvnr/+SCk2HCvDXy6JRTAX4VE3eGmc4OGi5syxDHLKWjf+sbuZ49MN/RNw22qgoRJ4/1KgYJfcEVEfMTmmfu29LSew9LdczEuLwr2Tzt2yVOOkwp8vjsX6hydhUlwg/vNzJqa9shkbM8tkiNa2HCupg7+7MwI9XSS97vhBgfBxc8Lq/T0vrWgymPDcmsOIDfLA7RcNlDQucmzcCEQe2WU6qJUKRPq7yx1K3wwYB9y5HnDxAJZNB46tlTsi6gOLJ8cmkwkjR47E9OnTzzn20EMPISkpCUlJSYiLi4OPj4+lwyHq8PWeArz4wzFcNUKLZ6YPgUKhOO+54b5ueOfWZKy4YwyUSgVu/3An5i/fhZOVDVaM2LZklugkLalo56xW4ophWvxypBQNLcYePfedTcdxqqoRz10zlIvwqEe0Pq6cOZZBdqkeUQHucFY7wM9rwKDWBDl4KLDqFmD7Erkjol6y+Hfj4sWLkZDQed3fK6+8gn379mHfvn3485//jOuuu87S4RABADZlleOxLw8gNdof/7shEUrl+RPj002MC8RPf5mIJ66Ix7bjFbjklU145ZcsNBn6V6mFySyQVaqXvKSi3TVJoWhoMWH90e7P0J+sbMDb6ccxfYQWaTEBFomLHJfWS4OiGibH1pZTprfvkoqzeQQCt30PDL4S+PGvwM9PstWbHbJoclxQUIC1a9di/vz5Fzz3008/xU033WTJcIgAAPtP1eDelbsRG+yJd+cmw0Wt6tHzndVK3DMpBr8+MgnThoZg8a/ZuOR/m7DucInkWx/bqpNVDWg0mBBvgZljABgT5YcQLw1W96BrxfNrDkPFRXjUS1ofDSr0zWzfaEXNRhPyKusdKzkGAGc34MaPgDF3AxlvAF/OAwws2bEnaktefOHChXjppZeg0+m6PC8/Px+5ubmYOnVqp8eXLFmCJUtaP54oKSlBUVHf2jw5qvLy3i1g6k9O1TTh7i+y4a1R4f+uHAB9VTn0Z53Tk3H826RgTItxw3/TC7Dgo90YF+mJhyaFI8LHsReCZeTUAAD81E1d/jz25XtySowXvthfhmMnTsJL0/Vb1dbcWqw/Wob7LgqFqK9GUX11r+9ri/izLZ3zjaWbaAYAHMjOR6i3tHX0jkiK78njFY0wC8Df2eiYv9cT/wJ3pQ+8fn8JhsqTqJr2Jswa3zNO4c+2dKQcS4slx2vWrEFQUBCSk5ORnp7e5bmrVq3CzJkzoVJ1PoO3YMECLFiwAACQkpKC0NBQqcN1GByb8yvTNeHRldugVCrx8V2piA48/2xFT8YxNBSYlhyLFRn5ePWXLNz6cSbmTxiIB6YOgpuzRf/+lE3pYT0UCuCiodFwde565r2335M3X+SOT/eWYV8FMHvM+a/RZDDh9ZWZiAl0x8IrEh2jdrET/NmWTmdjGa93AnASJhcvhIayN3Z39PV7cld5a0I8ZnAEQrWWKdGSXdjfgYghcP56AULW3ArM+RLwiz7jFP5sS0eqsbTYb5GtW7di9erViIqKwuzZs7FhwwbMmTOn03NXrVrFkgqyKF2TAbd/uBMVuhZ8MG90l4lxbziplLhz/ED8+ugkTE/U4q3047j4v5uw9kCxQ5ZaZJboEOXvfsHEuC+GhXlhYID7BbtWLNl8AierGvDcjGEOmxiT5YW2fdpTwi2krSanVAelAogOtPNOFRcy9NrWOuTGarZ6sxMW+02yaNEiFBQUIC8vD6tWrcLUqVOxcuXKc847duwYqqurkZqaaqlQqJ9rMZpxz8rdyCzR4a05o5AU4WOxewV5avC/G5Lw5T2p8HVzxv2f7MEt72/HifKzizfsW2apDoMt3LRfoVBgRmIoMk5UovQ8Ccupqga8uTEHVw3XYnwsF+FR74V4t+6Sx0V51pNdpkeUv3uP133YpQFjgfnrARfP1lZvR9fIHRF1werTLM888wxWr17d8e9Vq1Zh9uzZXbbRIuqt9t3vtuZU4v+uH4Epg4Osct+UKD98/+fx+Oc1Q3GosBb3fbzHKve1hoYWI/Iq6xEv4bbR5zMjKRRCAGsOFHd6/Pk1R6BUcBEe9Z2HixqeGjV7HVtRdpkegxxtMV5X/GNaE+SQYcBnc4Df35E7IjoPqxRETp48GZMnTwYAPP/882cce/bZZ60RAvVDQgi8sLZ197vHL4/H9aftfmcNKqUCt6ZGQd9swv/9dAyV+mb4e9j/Qp/s0tbtXi3VqeJ0MYEeGBbmhdX7CnHn+DM39diYWYZfjpTiscsHI9TH1eKxkOML9WavY2tpMZqRV1GPaUOD5Q7FutwDgLmrga/vAn56HN7xu4BJDwIhw1u3oyabwAI9clhLNp/AB1tzcftFUbhnUvSFn2Aho6NaVyfvzneMDgqZbdtGD7ZQj+OzzUgMxf6CWuRV1Hc81mw04bnVhxEd4I754+X72pJj0fpwlzxrya+sh9EsEBtk+T+ybY6zG3DDCiD1AbhlfgW8OwF4dQTw4xNA7hbA1LPNj0h6TI7JIX29pwCLfjyG6SO0ePqqrne/s7Th4d5wViuxy0GS46MldXB1UmGAn5tV7jd9ROvq4+9PW5j33uYTyKtswLMzhnIRHklG661BMWuOrSK7rHUdRr8qqzidUgVM+xdK52wGZrzRuqverg+A5dOBlwcB39zbugV1S//dhVVOjtlnivq19MwyPPblAVw0yB//7cHud5biolYhMdwbO/OqZI1DKpklOsQFe0BlpXEN9XHFmIF++G5/ER6YOgiFNY14Y2MOrhgWgolxgVaJgfoHrbcrKutb0GQwQePUDxaJySi7tLUdZIzEnYPsjdnVH4i5FRh1K9CsB45vAI6tATLXAvs/AdSuwKCLgfirgLjLATc/uUPuF5gck0PZf6oG9328B3HBnnhnTs93v7OUlCg/vL/lBBpbTBZtf2ZpQggcK9HhkgTrLGxsNyMxFE99ewhHi3VY/GsWFFDgqelDrBoDOT6td2s7t9K6JkT6O3h7MZlllekwwM/Nrt8PJefiAQyZ0fqfyQDkb22dPT62tjVhVqiAyLTWRHnwlYBvpNwROyx+HkkOI7eiHrcv2wl/D2csu2M0PDVOcofUYXSULwwmgf0FNXKH0ifl+mZU1bcg3kr1xu2uHK6FWqnA098dws+HS/HA1EEI4yI8kpiW7dysJqdU73jbRktJ5QRETwau/A/w0GFgQTow/iGgvgL46Qlg8QjgnQlA+v8BJYcAB+ynLyfOHJNDKNM1Ye4H2wEAK+4YiyBP29q+OXlA60dhu/KqMC7afnffal+MZ41OFafzc3fGhNgAbMwsx8AAd8yfMPDCTyLqIW3bRiBclGdZRpMZJyr0mBJv3U+g7JZCAYSObP3v4qeByuN/zCinLwLSXwR8IoH46a2zygPGtdY0U69x5pjsnq7JgHkf7ESlvgUfzhuNgQG293Got5sTBgd7YmeefS/K+6NThfVXmLe34vvH1UNsplyGHEto28wx27lZVn5VAwwmwZnj3vKPAS56ELjzZ+DRLODq14DAeGDne8CyK4HXRwHb322tYaZe4cwx2bVmown3rNyNrFId3r8tBYkW3P2ur1KifLF6XxFMZmG1xWxSO1qsQ6Cniyz9mq8arkXS4z4I97VOlwzqf1ydVfBxc+LMsYVll7YmbbHBTI77zCMISL6t9b9mHZD1M7BjCfDjY8DGF4GU24ExdwNeWrkjtSucOSa7JYTA418ewNacSrw0cwQmW2n3u94aHeUHXbMRx0rq5A6l1zJL66xeUtFOoVAwMSaLC/FiOzdLyylr/QSqv3eqkJyLJzB8JnDnOuDOX4CBE4Gti4FXh7e2his9LHeEdoPJMdmtr/YU4tt9RXjk0jhcN8q6u9/1RkrbZiC77LS0wmgyI6tUL1tyTGQNoT7cJc/Sssv0CPNxhbsLP7y2mIgxwI0fAX/e3Tp7fORb4O004KM/ATm/cgHfBTA5Jrt0qqoBz64+jLED/XDflEFyh9MtYT6u0Hpr7LbfcV5lA1qMZqvtjEckB603d8mztKxSPUsqrMUv+o+OF1Ofbp09Xnkd8PZFwL5PAGOL3BHaJCbHZHdMZoFHPt8PAPjvDYl2U7+rUCiQEuWHnXlVEHb4V7tcnSqIrEnrrUF1gwGNLSa5Q3FIJrPA8XI94oL5PmJVbn7AxEeBhQeBa94CIIBv720tudjyX6DRPj/RtBQmx2R33ttyAjvyqvDcjKF2V4M6OsoXpXXNKKi2v5mpYyV1UCr68Xav1C+09zouqWNphSWcqmr9BIrvIzJRuwAjbwHu3QbM+QoISgB+fR7431Dgh8eAqly5I7QJTI7JrhwpqsN/12XiimEhuG5UmNzh9NjoqLZ+x/n2V1pxrESHgQHu3FaXHFpHr+Ma+/sD1h5kl7V1qmByLC+FAhh0CTD3W+Ce31p35dv1QWsbuM/nAqd2yh2hrJgck91oMpjw0Gf74OvmjBf/NBwKhX2UU5wuLtgTnhq1XfY7zizRWX1nPCJr69glj4vyLCK7rVMFZ45tSMhw4E/vAAsPABf9BTiRDiy9BFh6GXD0e8Dc/0qMmByT3Xj550xklurw0swR8HV3ljucXlEpFUiO9MUuO1uUp2824mRVA+uNyeFpvVtnjku4KM8ickr10Hpr4KlxkjsUOptXKHDJs8BDR4DL/w/QlQCfzQHeHNO6K18/wuSY7MK2nAq8/1subh0XafP9jC9kdJQfskr1qGmwn1XCWaXy7YxHZE0aJxX83J05c2wh2WV6zhrbOhcPYNw9wJ/3ALOWAQ1VwMrrgfoKuSOzGibHZPNqGw149Iv9iA5wx9+vTJA7nD5LiWztd7w7335KK/7oVMGyCnJ8rRuBcOZYamazQE6ZHrFB/CPbLqjUwNA/ATd/BuiKgU9uAFrq5Y7KKpgck817dvVhlOqa8cqNSXB1tv/FYIkRPnBSKeyq7vhYcR3cnVUI93WVOxQiiwv10XAjEAsorGlEo8HEHsf2JmIMMPMDoGgv8OUdgMkod0QWx+SYbNqaA0X4Zm8hHpwai8QIH7nDkYTGSYXhYd52VXd8rESHuBBPKO2kpzRRX2i9uUueJbQvxotjcmx/4q8CrngJyPoJ+OERh99hj8kx2ayS2iY8+c0hJEX44P4pMXKHI6nRUX44UFCLJoPtrwIWQiCzVMfFeNRvhHhrUNtoQEOL48+QWVN2aWsbt0GBfC+xS2PuAsY/BOxeBmx5We5oLIrJMdkks1ngr1/uR4vRjFduTIJa5VjfqilRfmgxmXGwsFbuUC6otK4ZNQ0G1htTvxHa3uuYs8eSyi7TI8jTBd5u7FRhty7+BzDiRmDDC63bTzsox8o4yGF89Hs+tmRX4KnpCRgY4C53OJJLbluUt9MOSiuOldQBYKcK6j/aex0X1zA5llJ2mZ71xvZOoQBmvAEMnASs/jOQ86vcEVkEk2OyOTllOrz4w1FMjQ/CzWMGyB2ORfi5O2NQkAd22cGivGMdnSqYHFP/ENqxEYh1O1YYTWbMX74Law8UW/W+1iCEQE6pjp0qHIHaGbjxIyAwvnU3veL9ckckOSbHZFNajGYs/Gwf3F3U+Pf19rkLXneNjmrdDMRstu2FDZklOoR4aeDjZp8brxD1VLC3CwDrzxwfKa7D+qOleOizffj9RKVV721pxbVNqG8xscexo9B4A7d8CWh8gI9nAdX5ckckKSbHZFNe35CNQ4V1ePFPwxHkqZE7HItKifRDXZMR2WV6uUPp0rESHUsqqF9xUasQ4OGMkjrrzhzvyG0tswrx1mDBil3Iaevu4Aja3+dimRw7Di8tMOdLwNgEfDyzdbMQB8HkmGzG7vxqvLkxB7OSw3H5sBC5w7G40VF+AGy77thgMiOnTId4LZNj6l+03q4osvLM8e8nqhDl74aP54+Fs1qFeR/uRLmu2aoxWEp2aXsbN76XOJSgBGD2J0B1HrDqZsDgGHX6TI7JJtQ3G/Hw5/sQ6uOKZ64eInc4VhHh54ogTxeb7necW1EPg0mw3pj6nRBvDYqtWHNsNgvszKvCmIF+iPBzwwfzUlCpb8Gdy3c6REu57FI9Ajyc4evO8iyHEzUe+NO7wMkM4Ou7ALPttyi9ECbHZBNeWHsEJ6sa8MqNSfDU9I82PwqFAqOj/Gx6p7z2xXiDg9nGjfqXUG/r7pKXVaZDbaMBYwb6AwBGhPvg9ZtG4lBhLR78dC9MNr424UKyy3SsN3Zkw64Dpr0IHF0N/Px3u98khMkxyW79kVJ8uuMU7pkU01Fq0F+kRPmisKYRRTXWrW3srmPFdVArFYgJcrx2ekRd0fq4QtdkhL7ZOrO220+0foI0duAf74GXDAnGszOGYv3RMjz//WEIO004hBCtbdzYqcKxpd4PjLsP2P4OkPGG3NH0CZNjklWFvhlPfH0AQ7ReeOiSOLnDsbr2PwZ25dvm7HFmiQ7Rge5wUavkDoXIqrTebRuBWOkP1x25VQj11iDc1/WMx+emRuGuCQOxPCMfS3/LtUosUivTNUPXZGSP4/7gsn8BQ64F1j0FHPxS7mh6jckxyUYIgSe+Ooi6JiNenZ0EZ3X/+3aMD/GEu7PKZuuOWztVsKSC+p+OjUCsUFohhMD23NZ6487aV/7tigRcOTwE//rhKH48aH89kDu2jWZZheNTKlvrjwekAd/eC+RukTuiXul/2QjZjM93ncL6o6V4/PL4fruCWa1SYlSkr03WHdc1GVBY08jFeNQvdcwcW2FR3omKelTomzE22r/T40qlAv+7IQkjI3yw8LN92G2jnzSdT3ZbSzqWVfQTThpg9seA70Bg1S1A6RG5I+oxJscki/zKejz3/RGkxfjj9rQoucORVUqkH46V1KGuySB3KGfI4s541I8Fe2mgUMAq7dza+xuPGXj+NRcaJxXev200tN4a3LViF/Iq6i0el1SySvXwcXNCgAc7VfQbbn7AnK8AJ9fWHsi1hXJH1CNMjsnqTGaBhz/fD7VSgZdnJUKpdNxd8LpjdJQvhAD22NhsUEenCibH1A85q5UI8HBBiRXKKnbkViHAwwXRAV0vfPVzd8ay28dACIF5H+5AVX2LxWOTQk6ZDnFBng694yl1wieidZOQprrWXfSaauWOqNuYHJPVvbPpOHbnV+Of1w5DqI/rhZ/g4JIG+EClVGCXjZVWHCupg6eLGmH8GlE/FeqtQZGFyyqEENh+ohJjz1NvfLaoAHe8f1sKimqbcNeKXWgy2HZPWSEEskr1GMTFeP1TyHDgxo+AikzgszmA0T7+oDtvcnzllVciLy/PiqFQf3CosBav/JKFqxNDcU1SmNzh2AQ3ZzWGhXrZ3E55mW3bRnO2h/qrECv0Oi6obkRRbVOXJRVnS470w6s3JmHPyWo88vl+mG24B3KFvgW1jQZuG92fxUwBrnkTyN0MfHcfYDbLHdEFnTc5vv3223HZZZfhX//6FwwG26qFJPvUZDBh4Wf7EODhgheuGSZ3ODYlJcoP+07VoMVoG28aQoi2ThUsqaD+S+vtavGyiu7UG3fmyuFa/P2KBKw9WIx//3TMEqFJgovxCACQOBu4+Bng4BfAr8/JHc0Fqc93YNasWbjiiivwz3/+EykpKbj11luhVP6RSz/88MNWCZAcx3ubTyCnTI+Vd46Ft1v/2AWvu0ZH+WLpb7k4VFSLUQN85Q4HRbVN0DUZEa9lGzfqv0J9NNA3G1HXZICXhXbu3J5bCW9XJwzuRcee+RMG4lR1A5ZsPoEIX1fcmholfYB9lFPW2saNPY4J4x9uXZi39VXAOxwYc5fcEZ1XlzXHzs7OcHd3R3NzM3Q63Rn/EfWEwWTGR7/nY1JcIMbHBsgdjs1JjmzbDMRGSisyS+oAsFMF9W8dvY4t2LFiR24VRkf59WphskKhwD+uHopLEoLwj9WH8evRUgtE2DfZpXp4atQI8nSROxSSm0IBXPkfYPCVQM56m95i+rwzxz/99BMefvhhzJgxA3v27IGbm5s14yIH8/PhEpTpmvHv6yPlDsUmBXq6YGCAO3bmVWPBRLmjAY4Ws1MFUXuv46LaRov8LJTWNSGvsgG3jO39+6JKqcBrN43E7CW/44FP9uLzu1MxPNxbwij7JrtMh9ggD65doFZKFTDzA0Cpbk2WbdR5Z47/9a9/4YsvvsC///1vJsbUZysy8jHAzw2T4oLkDsVmpUT6YldeFYQN/DWdWaJDmI+rxT5KJrIH2rZOLZaqO97eVm88Nrpn9cZnc3NW4/3bUuDn7ow7lu9EQXWDFOFJIrtUz3pjOpOTK6Cy7d8t502Ot2zZgqFDh1ozFnJQR4vrsCO3CreOi4Sqn/c07sroKD9UNxhwvFz+5v6ZXIxHhCBPFygVQHGNZdq57cithLuzCkMkqO0P8tRg+R2j0Www4fYPd6K2Uf6F9JX6ZlTWt7DemOwO+xyTxa3IyIeLWolZKeFyh2LTUqJaF+LJ3dKtxWjG8XI9642p33NSKRHo6YIiC80c78itQnKUH9QqaX4VDwryxJK5KcirrMc9H+2WvfvNH4vx+F5C9oXJMVlUbYMB3+4txLVJYfBx49ahXRkY4I4AD2fZk+Pj5XoYzYIzx0SwXDu3qvoWZJXqMbaHLdwuZFy0P/4zMxEZJyrxxFcHZC3Tym5PjtnjmOzMeRfkEUnhi92n0Ggw4dZULsS7EIVCgZRIP9l3ysts2zY6PoRt3IhCfTQdW6lLqb2/sdTJMQBcOzIMBdUNeHldFsJ9XfHwZYMlv0d35JTp4e6s6ljYSGQvOHNMFmM2C6z8PR8pkb4YFmY7q6dtWUqUL05WNaC0zrIbD3TlaEkdnFQKRAe6yxYDka0I8XJFcU2T5DOwO3Kr4KJWWqyzxP1TBmH26Ai8tiEHn+88ZZF7XEh2mQ6DgrnLJtkfiyfHJpMJI0eOxPTp0zs9/vnnn2PIkCEYOnQobr75ZkuHQ1a0ObsceZUNmJsWJXcodmN0VHu/Y/lmjzNLdIgJ9ICTRHWQRPYs1EeDRoMJdY1GSa+7I68Sowb4wkWtkvS67RQKBf557TBMiA3Ak98eRIW+2SL36UprpwqWVJD9sfhvv8WLFyMhIaHTY9nZ2Vi0aBG2bt2Kw4cP49VXX7V0OGRFKzLyEeDhgsuHhsgdit0YEuoFVyeVrHXHx4p1XIxH1KZ9I5CiWuk6VtQ1GXCkqK7HW0b3lJNKiSevSoDBJPDToRKL3utstQ0GlOmamRyTXbJoclxQUIC1a9di/vz5nR5/7733cP/998PXt3WVflAQe+A6ivzKemzMLMPNYwfAWc0ZyO5yUikxcoAPduXLkxzXNhhQUtfEbaOJ2oS01csWS5gc786rhllYpt74bIODPRET6I41B4osfq/TZZe11mmzjRvZI4suyFu4cCFeeuml8243nZWVBQC46KKLYDKZ8Oyzz+Lyyy8/57wlS5ZgyZIlAICSkhIUFVn3h9xelJeXyx1Ch3e2FEKpAC6OdLG7r5fc4xjv74RlOyuRnXcK7s6W+cj1fPYWtq4uD3RqkeTrJvdYOgqOo3R6OpaqphYAwLH8UsR7mSSJ4deDhVArFQhxarLK++OkgZ74cEcJDmTnI8Bdms0XLjSOO7MqAADeaLC73wHWxJ9t6Ug5lhZLjtesWYOgoCAkJycjPT2903OMRiOys7ORnp6OgoICTJw4EQcPHoSPj88Z5y1YsAALFiwAAKSkpCA0NNRSYds9WxibxhYTfjh2CJcP0yIxzj67VMg5jlOGO+GDHSUoMWgwISrQqvf+JS8PAJA2NKrj4+S+soXvSUfAcZROT8Yy2CygUh5BA1wk+xocLs9FYoQPoiOt0/v95vGe+GBHCfaUmTHvIum+j7oaj/LdNXB1UmFkXBSU3PypS/zZlo5UY2mxz7u3bt2K1atXIyoqCrNnz8aGDRswZ86cM84JDw/HjBkz4OTkhIEDByIuLg7Z2dmWComsZPX+QtQ2GjB3nH0mxnIbOcAXSgWwU4ZFecdKdPB2dUKIF1svEQGASqlAkKeLZDXHDS1GHCyotXi98ekGBXkiPsQTaw8WW+2e2WU6DAryYGJMdsliyfGiRYtQUFCAvLw8rFq1ClOnTsXKlSvPOOfaa6/tmFWuqKhAVlYWoqOjLRUSWYEQAsu35SM+xNOqb/6OxMNFjSGhXtglw6K8zJI6DA5h6yWi02m9NZJtBLL3ZA2MZmGVeuPTXTVci5151ZLWTnclp4ydKsh+WX2l1DPPPIPVq1cDAKZNmwZ/f38MGTIEU6ZMwX/+8x/4+/tbOySS0O78ahwprsPc1CgmWH2QEumHvSdrYDBZb/tXs1kgs4SdKojOpvVxRbFEyfH2E5VQKoDkSF9Jrtdd0xNbP25ee8Dys8e6JgOKa5swiIvxyE5ZJTmePHky1qxZAwB4/vnnMWPGDACtfRj/97//4ciRIzh48CBmz55tjXDIglZk5MNTo8a1I1lD1Rejo/zQaDDhSFGd1e5ZWNOI+hYTd8YjOkuotwZFNY2SbASyPbcKQ0O94amRZmFcdw0McMfQUC+ssUJynNOxbTT/0Cb7xB5bJJmyuib8cLAYN6REwM2ZO5P3RUpU66ySNfsdt2+RO5gzx0RnCPF2RbPRjJoGQ5+u02w0Ye+pGquXVLSbPiIU+07V4FRVg0Xvk13anhxz5pjsE5NjksynO07BaBaYw4V4fRbspcEAPzer7pR3rLh1lprJMdGZQtt6Hfd1Ud7+U7VoMZplW48xfYQWAPCDhRfmZZfp4KJWIsLPzaL3IbIUJsckCYPJjI+352NSXCAGBrjLHY5DSInyxa78Kkk+yu2OY6U6RPi5wsOFs/5Ep9P6tLY1LK7pW93xjtxKAH9sE29tEX5uSAz3tnhpRXaZHjGBHlCxUwXZKSbHJImfD5egTNeM29I4ayyV0VF+qNC3IK/Ssh+Btsss0WFwMOuNic6mlWiXvO25VYgP8YSvu7MUYfXK9BGhOFhYi7yKeovdI7tUz53xyK4xOSZJrMjIxwA/N0yK4xbgUhltxbrjJoMJuRX1SNCypILobAEeLlArFX3qWGEwmbE7v1r2FpdXtZVWWKrncX2zEYU1jaw3JrvG5Jj67GhxHXbkVmHOuAH8GE1CMYEe8HVzskq/45wyPUxmwXpjok6olAoEe2n6lBwfLqpDQ4tJ9uQ41McVyZG+FiutOF7euhhvEDtVkB1jckx9tiIjHy5qJW5IiZA7FIeiUCiQHOlnlUV57Z0q2OOYqHPatnZuvdVebyx3cgy0bghytLiuI5GVUkenCpZVkB1jckx9UttgwLd7C3FtUhh83OSro3NUo6N8caKiHhX6Zovdo8lgwqodJ+HurEKUPxdTEnVG6+OKkrrezxxvP1GF6AB3BHnKvzX7VSO0UCiANfulnz3OLtPDSaVAJDtVkB1jckx98sXuU2g0mHBrKhfiWUJK26p2S80em8wCD366F7tPVmPR9SOgVvEtgagzod6tZRW96R5jMgvsyKuyiVljoLVV5OgoP6w5UCT5tbNLdYgO8OB7Cdk1fvdSr5nNAit/z0dKpC+GhXnLHY5DGhbmBRe10iJ1x0IIPPXtIaw7Uop/TB+CGYnc1ZDofEK8NWgxmlFZ39Lj52aW6KBrMmJstG0kxwBw9Qgtssv0yCrVSXrd7DJ2qiD7x+SYem1zdjnyKhs4a2xBLmoVEiN8sDNf+pnjV9Zn49MdJ3H/lBjMu2ig5NcnciRa79ZexyW9WJT3R72xv6Qx9cXlw7RQKoA1+6WbPW5sMeFUdQO3jSa7x+SYem1FRj4CPFxwxTCt3KE4tNFRvjhcWIuGFqNk1/woIw+v/ZqNG1LC8ehlgyW7LpGjCvVp2yWvF4vytudWIczHFWFtm4nYgkBPF4yL9seaA8WSbTR0vFwPIbgYj+wfk2PqlZOVDdiYWYabxw6As5rfRpaUEuUHo1lg36kaSa73w8FiPLP6MC6OD8KLfxoOhYLt94guJKRjI5CezRwLIbAjt8qmSiraTR8RihMV9TjStnV8X+WUtXWqYI9jsnPMaqhXVm7Ph0qhwC1jB8gdisMbNcAXCoU0i/K2Ha/AwlX7MGqAL964eRQXzRB1U4C7C5xUPd8I5Hh5PSrrWzDWRhbjne7yYSFQKRWS9TzOLtNBrVQgkl1vyM7xNyP1WGOLCZ/tPIVpQ0MQ7CV/WyJH5+3qhMHBnn3eKe9wUS0WrNiNSH83LL0tBa7OKokiJHJ8SqUCId6aHm8hvd0G643b+bk746JBAVgrUWlFdqkeUQHu/DSR7B6/g6nHVu8vRG2jAXO5EM9qRkf5YU9+NYwmc6+ef7KyAfM+3AkvjRor7hzDntREvaD1ckVxTc9mjnfkViHI0wVR/rbZ93f6cC1OVjXgYGFtn6+VU6ZnSQU5BCbH1CNCCCzflo/4EE+b6dnZH6RE+aK+xdSxk11PVOibMfeD7TCYzFhx55iOVfdE1DNaHw2K67o/cyyEwPYTrf2NbbW2f9rQEDip+l5a0WQwIa+ynskxOQQmx9Qju/OrcaS4DnNTo2z2zd4Rje7YDKRnpRX6ZiNu/3AnSuqasPS20RjEFktEvab1dkVJbRPM5u6VIJyqakRJXZNN1hu383ZzwoTYwD6XVuRW1MMsgNhgvseQ/WNyTD2yIiMfnho1rh3JDSOsKbStDVRP+h23GM2456PdOFJch7duGYXkSF8LRkjk+EJ9NDCYBCrqu7ede3u98dho26s3Pt30EVoU1jRibx864mS3d6pgGzdyAEyOqdvK6prww8FizEqOgJuzWu5w+p2UKF/syqvq1uyO2SzwyBf78VtOBf593XBMjQ+2QoREji2kbQFyd+uOd+RWwdfNCYMCbTthvGRIMJxVSqzZ3/vSipxSHZQKYGAAO1WQ/WNyTN326Y5TMJoFd8STyegoP5TWNaOguuuaRyEEnl9zBN/vL8Ljl8djVkqElSIkcmyhbZt4dLed2/bcKoyO8oNSadslaF4aJ0waHIgfDhZ3u2TkbNllekT5u8NFzS44ZP+YHFO3GExmfLw9H5PiAjkzIJP2uuMLtXR7e9NxLNuWhzsuGoh7JkVbIzSifkHbsRHIhRflFdc24mRVg82XVLSbPkKLkrom7OrlVvXZZXoM4mI8chBMjqlbfj5cgjJdM25L46yxXGKDPOClUWNnF5uBfL7rFF76KRMzEkPx1FUJXDRJJCE/d2c4q5Xdmjnekdv6R6wtL8Y73SUJwXBRK7H2QFGPn9tiNCOvop71xuQwmBxTt6zIyEeEnysmxQXJHUq/pVQqkBLld96Z41+PluJvXx/EhNgAvDwr0eY/yiWyNwqFAlpvTbeS4+25VfB0USNB62WFyPrO3UWNqfFB+OFQCUw9LK3Ir6yH0SwQy2445CCYHNMFHS2uw47cKtw6LhIqJlyySonyRU6ZHlX1LWc8vju/Cvd/sgdDQ73w9pxk7lBFZCFabw2Kay5cVrEjtwopUb529Z45fUQoynXNHV02uiurlJ0qyLHwNyhd0IqMfLiolbiBC7tk1153vPu0usDsUh3uWLYLWm9XfDBvNDxc2EmEyFK03q4XnDmu0Dcjp0xvk1tGd2VqfBDcnFU93hAku0wHhQKIsfGuHETdxeSYulTbYMC3ewtxbVIYtxy2AcPDvOGsUnZsBlJU04i5H+yAs1qJFXeMQYCHi8wREjk2rbcGpXVNXZYe7GyrN7a3XURdnVW4OCEYPx0q6dFW9dllegzwc4PGiZ0qyDEwOaYufbH7FBoNJrZvsxEaJxVGhHtjZ14VahpaMPeDHdA3GbHs9tGI8HOTOzwih6f1cYXRLFChP/9GINtzq+DqpMLwMG8rRiaN6SO0qKpvQcaJ7pdW5JTquW00ORQmx3ReZrPAyt/zkRzpi2F2+CbvqFKi/HCwsBa3L9uJk5UNWDI3BUND+fUhsgZt20YgRV3UHe/IrcKoSB+7rP2fFBcIDxd1tzcEMZrMOFGh59b05FDs7yeXrGZzdjnyKhswl7PGNmV0lC8MJoF9p2qweHYSUmPsq66RyJ5pfVqT45Lz1B3XNhhwtKQOY6Ls8+dS46TCpUOC8dPhErQYL1xakV/VAINJcOaYHAqTYzqvFRn5CPBwwRXDtHKHQqcZPdAPUf5u+Ne1w3HFcH5tiKwp1Lt1l7yi8yTHu/KrIAQwNtq+6o1PN32EFrWNBmzNqbjgudnsVEEOiMvaqVMnyvXYmFmGP08ZZJcfDToyL40T0v86Re4wiPolHzcnuKiV523ntiO3Cs4qJZIifKwbmIQmxAbCU6PGmgPFmBLfdW/77FIdAHaqIMfCrIc69d6WXDiplLg1NUruUIiIbIZCoUCojyuK6zqfOd6eW4XECG+77tzgrFZi2tAQrDtSgmajqctzs8v0CPd1hTtbSJIDYXJM5yjXNeOrPQW4flQ4Aj3ZGoyI6HTn2wikvtmIg4W1GGtn/Y07M32EFromIzZndV1akV3GThXkeJgc0zlWZOTBYDLjrgkD5Q6FiMjmnG8jkD0nq2EyC7vrb9yZiwYFwNfNCWsOFJ33HJNZ4Hi5HrHB7FRBjoXJMZ2hvtmIFRn5uDQhGNGsISMiOofWW4MyXfM5G2XsyK2CSqnAqEhfmSKTjpNKicuHhWD9kVI0GTovrThV1YAWoxmDOHNMDobJMZ3h812nUNtowN2TYuQOhYjIJml9NDCZBcrP2ghk+4kqDAvzdpgt3K8aHor6FhPSM8s6PZ5d1tapgskxORgmx9TBaDLj/S25SIn0RbIDzHwQEVlCRzu3mj9KK5oMJuw7VYOxDlBS0W5ctB/83Z3x/YHONwTJLmvtVMGZY3I0TI6pw9qDxSisaeSsMRFRF0K8WzcCKa79Y1He/lM1aDGZMSbKcZJjtUqJK4aHYMPRMjS0GM85nlOqh9ZbA0+NkwzREVkOk2MCAAghsGTzCcQEuuPiC/S1JCLqz9pnjk/fJW97bhUUCmC0AyXHADB9RCgaDSb8evTc0oqsMh1njckhMTkmAMC245U4XFSHuyZEQ6lUyB0OEZHN8nJVw81ZdUZZxY7cKsSHeMHbzbFmUUdH+SHI0wVrzyqtMAuBnDI94tipghwQk2MCALyz6TgCPV1w7cgwuUMhIrJpCoUCId6ajrIKg8mM3fnVDlVv3E6lVODK4VpszCyDvvmP0oqSuhY0GcxcjEcOickx4UhRHbZkV2BeWpRd7+pERGQtoaf1Oj5YWItGg8khk2OgdUOQZqMZ64+UdjyWW9X62mODmRyT42FyTFiy+TjcnFWYMzZS7lCIiOyC9rSZ4x25VQCA0Q6aHI8a4Autt+aMDUHy2pLjQYEsqyDHw+S4nyusacT3B4px05gBDlcrR0RkKe0bgRhMZuzIrUJMoDsCPFzkDssilEoFrhquxaasctQ2GgC0zhwHebrw9wY5JCbH/dwHv+UCAO4Yz62iiYi6S+vjCiFaO1bszK3C2Gh/uUOyqKtGaGEwCfzSVlqRV9XEkgpyWEyO+7HaBgM+3XESV4/QIszHVe5wiIjshrat1/GGY2XQNRsdtt64XVKED8J9XbHmQBGEEK3JcRBLKsgxWTw5NplMGDlyJKZPn37OsWXLliEwMBBJSUlISkrC+++/b+lw6DQrt+ejocWEBRO56QcRUU9o23odf7uvEAAwxsGTY4VCgatGaPFbdgWOFNehwWDmzDE5LItvAL948WIkJCSgrq6u0+M33ngj3njjDUuHQWdpMpiwbFseJsQGYEiol9zhEBHZFa1P68zx3pM1GODn1pEsO7KrR4Ti3U0n8MaGHADgzDE5LIvOHBcUFGDt2rWYP3++JW9DvfDt3kKU65pxD7eKJiLqMS+NEzxcWueXHH3WuN3QUC9E+bvhx0MlAMAex+SwLDpzvHDhQrz00kvQ6XTnPeerr77C5s2bERcXh1deeQURERHnnLNkyRIsWbIEAFBSUoKioqJzziGgvLy8W+eZhcBbG7MQF+iKKNdmjudZujuOdGEcS2lwHKUj5VgGuquhbzYizlfZb95HJw70RF5lA7xdlGisrUBjrdwR2Tf+bEtHyrG0WHK8Zs0aBAUFITk5Genp6Z2ec/XVV+Omm26Ci4sL3n33Xdx2223YsGHDOectWLAACxYsAACkpKQgNDTUUmHbve6MzS9HSnGyuhmLZychLIw74nWG32PS4VhKg+MoHanGMtz/FHKrmnD5yBiE+rtJck1bd9NFHlixqxQxAW78npQIx1E6Uo2lxcoqtm7ditWrVyMqKgqzZ8/Ghg0bMGfOnDPO8ff3h4tLa1/I+fPnY/fu3ZYKh07z7qbjCPNxxVXDtXKHQkRktxK0XogOdEeEn+PXG7eLD/HEhNgApEZxrQo5Loslx4sWLUJBQQHy8vKwatUqTJ06FStXrjzjnOLi4o7/X716NRISEiwVDrXZnV+FXfnVmD9hINQqdvIjIuqtRy8bjNUPjIdCoZA7FKtRKBT46M6xuCU5WO5QiCzG4t0qzvbMM88gJSUFM2bMwGuvvYbVq1dDrVbDz88Py5Yts3Y4/c67m07A29UJN6ScW9tNRETd56xWwlnNSQYiR2OV5Hjy5MmYPHkyAOD555/veHzRokVYtGiRNUIgACfK9fjlaCkemDII7i5W/7uIiIiIyObxT95+5L0tuXBSKTE3NUruUIiIiIhsEpPjfqJc14yv9hRgZnI4Aj1d5A6HiIiIyCYxOe4nlm/Lg8Fkxl0TouUOhYiIiMhmMTnuB+qbjfjo93xcNiQYAwPc5Q6HiIiIyGYxOe4HPtt5CrWNBtzNraKJiIiIusTk2MEZTWYs/S0Xo6N8MWqAr9zhEBEREdk0JscObu3BYhTWNGLBRM4aExEREV0Ik2MHJoTAu5tOICbQHRfHB8kdDhEREZHNY3LswLbmVOJIcR0WTIyGUtl/tjclIiIi6i0mxw7s3c3HEejpgmtHhskdChEREZFdYHLsoA4X1WJLdgVuvygKLmqV3OEQERER2QUmxw7qvc0n4O6swi1jI+UOhYiIiMhuMDl2QAXVDfj+QDFuGjMA3q5OcodDREREZDeYHDugD37LgwLAHeMHyh0KERERkV1hcuxgahsMWLXzJK5ODEWoj6vc4RARERHZFSbHDmbl9nw0tJiwYGK03KEQERER2R0mxw6k2WjGh1vzMDEuEAlaL7nDISIiIrI7TI4dyM/HqlChb8bdnDUmIiIi6hUmxw7CbBb4ZG8ZhoV5IS3GX+5wiIiIiOwSk2MHsf5oKU5WN2PBxBgoFNwqmoiIiKg3mBw7iE92nESwpxOuHBYidyhEREREdovJsQNoMpiQcbwSE6N9oFbxS0pERETUW8ykHMCO3Co0G80YF+kpdyhEREREdo3JsQPYlFUOZ7USI8OYHBMRERH1BZNjB7A5qxxjB/pB48QvJxEREVFfMJuyc0U1jcgu02NibKDcoRARERHZPSbHdm5zVjkAYNJgJsdEREREfcXk2M5tyiqH1luD2CAPuUMhIiIisntMju2Y0WTGbzkVmBgbyI0/iIiIiCTA5NiO7TtVA12TkSUVRERERBJhcmzHNmWVQ6kALooJkDsUIiIiIofA5NiObc4qx8gBvvB2c5I7FCIiIiKHwOTYTlXVt+BAYS1buBERERFJiMmxndqSXQ4h2MKNiIiISEpMju3Upqxy+Lo5YXiYt9yhEBERETkMJsd2yGwW2JxVgfGxgVAp2cKNiIiISCpMju3Q0ZI6VOibMSmOJRVEREREUmJybIc2tW0ZPTGWLdyIiIiIpKSWOwBbJ4TAjtwqCADjov3lDgdAawu3BK0Xgrw0codCRERE5FA4c9wNf//mIF7+OVPuMAAA+mYjduVVY2IcZ42JiIiIpMbk+AIUCgVmJkdgV341TpTr5Q4HGccrYTQL1hsTERERWQCT4264blQYlArgqz0FcoeCTVllcHNWISXST+5QiIiIiBwOk+NuCPbSYGJcIL7aXQiTWcgWhxACm7LKkRbjD2c1v3REREREUmOG1U0zk8NRUteErTkVssWQV9mAU1WNLKkgIiIishAmx910SUIwvF2d8OVu+UorNmWWAQAmMjkmIiIisggmx92kcVJhRmIofj5cgtpGgywxbM6uQJS/GyL93WW5PxEREZGjY3LcA7NSwtFsNGPNgSKr37vZaELG8UrOGhMRERFZkMWTY5PJhJEjR2L69OnnPeerr76CQqHArl27LB1OnwwP80ZcsIcspRW78qrRaDCx3piIiIjIgiyeHC9evBgJCQnnPa7T6bB48WKMHTvW0qH0WWvP43DsPVmDnDKdVe+9KascziqlzezSR0REROSILJocFxQUYO3atZg/f/55z3n66afx+OOPQ6Oxj62Qrx0ZBpVSgS93F1r1vpsyy5ES5Qt3F+74TURERGQpFs20Fi5ciJdeegk6XeezrHv27MGpU6dw1VVX4T//+c95r7NkyRIsWbIEAFBSUoKiIuvX/J5u3ABPfLkrHzcP94RaqbD4/cr1Lcgs1eH+QaFdvvby8nKLx9IfcBylw7GUBsdROhxLaXAcpcFxlI6UY2mx5HjNmjUICgpCcnIy0tPTzzluNpvx8MMPY9myZRe81oIFC7BgwQIAQEpKCkJDQyWOtmfmXKTAvR/vwfF6J0wZHGTx+/228xQAYHpKDEK1Xl2eK/fYOAqOo3Q4ltLgOEqHYykNjqM0OI7SkWosLVZWsXXrVqxevRpRUVGYPXs2NmzYgDlz5nQc1+l0OHToECZPnoyoqCj8/vvvmDFjhs0vygOAixOC4etmvZ7Hm7LLEeTpgvgQT6vcj4iIiKi/slhyvGjRIhQUFCAvLw+rVq3C1KlTsXLlyo7j3t7eqKioQF5eHvLy8jBu3DisXr0aKSkplgpJMs5qJa5JCsMvh0tR09Bi0XuZzAK/ZVdgYlwgFArLl3AQERER9WdW73P8zDPPYPXq1da+reRmJoejxWTG9/stW/+8v6AGtY0GtnAjIiIisgKrtD6YPHkyJk+eDAB4/vnnOz2ns7pkWzY01AvxIZ74cncBbk2Nsth9NmWWQ6EAxg8KsNg9iIiIiKgVd8jrpfaex/sLapFVarmex5uzy5EY7gNfd2eL3YOIiIiIWjE57oNrR4ZBrVRYbGFedX0L9p+qYUkFERERkZUwOe6DAA8XTIkPwtd7CmEwmSW//m85FTALYCKTYyIiIiKrYHLcRzOTw1Ghb8bmLOkbeW/OKoe3qxMSw70lvzYRERERnYvJcR9NjQ+Cv7uz5KUVQghszi7H+EEBUKv4ZSIiIiKyBmZdfeSkau15vP5oKarqpet5nFmqQ2ldM+uNiYiIiKyIybEEZqWEw2ASWL2vULJrbspsLdOYEMcWbkRERETWwuRYAglaLwwN9cKXe6QrrdicXY7BwZ7QertKdk0iIiIi6hqTY4nMTA7HocI6HC2u6/O16puN2JlbjUmDWVJBREREZE1MjiVyTVIYnFTS9Dz+/UQlWkxmTIxlckxERERkTUyOJeLn7oyL44Px7d6+9zzenFUOVycVUqJ8JYqOiIiIiLqDybGEZiaHo7K+BRuPlfXpOpuyyjEu2g8aJ5VEkRERERFRdzA5ltCkwYEI8HDpU2lFfmU98iob2MKNiIiISAZMjiXkpFLiTyNDseFYGSr0zb26RvtOe9wymoiIiMj6mBxLbGZyBIxmge/2FfXq+ZuyKhDh54qBAe4SR0ZEREREF8LkWGKDQzwxIty7V6UVLUYzth2vwKS4QCgUCgtER0RERERdYXJsATOTw3G0uA6HCmt79Lxd+VVoaDGxhRsRERGRTJgcW8CMxFA4q5Q9nj3enFUBtVKBtEHcMpqIiIhIDkyOLcDHzRmXDgnGd/sK0WLsfs/jTVnlSI70hYeL2oLREREREdH5MDm2kJnJ4ahuMGDDsdJunV9W14SjxXXcMpqIiIhIRkyOLWRCbACCPLvf83hzdgUAsN6YiIiISEZMji1ErVLiT6PCsDGzHGW6pguevzmrHAEeLhii9bJCdERERETUGSbHFjQrORwms8B3e7vueWwyC2zJLsfEuAAolWzhRkRERCQXJscWNCjIE0kRPvhydwGEEOc972BhLaobDNwymoiIiEhmTI4tbGZyODJLdTjYRc/jzVnlUCiA8WzhRkRERCQrJscWdnViKJzVXfc83pRVjuFh3vD3cLFiZERERER0NibHFubt6oRpQ0Pw3b4iNBtN5xyvbTBg78lqllQQERER2QAmx1YwMzkctY0GrD9Sds6xrccrYBbARCbHRERERLJjcmwF4wcFIMRLgy93nzrn2Oascnhq1BgZ4WP9wIiIiIjoDEyOrUClVOC6UWHYlFWO0ro/eh4LIbApqxzjBwVAreKXgoiIiEhuzMisZGZyOMwC+GZvYcdj2WV6FNc2saSCiIiIyEYwObaS6EAPJEf6ntHzeHNWOQDWGxMRERHZCibHVjQzORw5ZXrsO1UDoLWF26AgD4T5uMobGBEREREBYHJsVVeN0ELj1NrzuLHFhO25VWzhRkRERGRD1HIH0J94aZxw+dAQrN5fhAmxgWgxmllSQURERGRDOHNsZTOTI6BrMuLFH47CRa3E2IF+codERERERG2YHFtZWow/Qr01OFnVgLHR/tA4qeQOiYiIiIjaMDm2MqVSgeuTwwGA9cZERERENoY1xzK4ZWwkjhTVYfoIrdyhEBEREdFpmBzLIMRbg6XzRssdBhERERGdhWUVRERERERtmBwTEREREbVhckxERERE1IbJMRERERFRGybHRERERERtmBwTEREREbVhckxERERE1MbiybHJZMLIkSMxffr0c4698847GD58OJKSkjB+/HgcOXLE0uEQEREREZ2XxZPjxYsXIyEhodNjN998Mw4ePIh9+/bhsccew8MPP2zpcIiIiIiIzsuiyXFBQQHWrl2L+fPnd3rcy8ur4//r6+uhUCgsGQ4RERERUZcsun30woUL8dJLL0Gn0533nDfffBP/+9//0NLSgg0bNnR6zpIlS7BkyRIAQElJCYqKiiwSr70rLy+XOwSHwHGUDsdSGhxH6XAspcFxlAbHUTpSjqXFkuM1a9YgKCgIycnJSE9PP+95999/P+6//3588skneOGFF7B8+fJzzlmwYAEWLFgAAEhJSUFoaKilwrZ7HBtpcBylw7GUBsdROhxLaXAcpcFxlI5UY2mxsoqtW7di9erViIqKwuzZs7FhwwbMmTPnvOfPnj0b3377raXCISIiIiK6IIslx4sWLUJBQQHy8vKwatUqTJ06FStXrjzjnOzs7I7/X7t2LWJjYy0VDhERERHRBVm05rgzzzzzDFJSUjBjxgy88cYbWL9+PZycnODr69tpSQURERERkbUohBBC7iB6IiAgAFFRUXKHYZPKy8sRGBgodxh2j+MoHY6lNDiO0uFYSoPjKA2Oo3S6M5Z5eXmoqKi44LXsLjmm80tJScGuXbvkDsPucRylw7GUBsdROhxLaXAcpcFxlI6UY8nto4mIiIiI2jA5JiIiIiJqw+TYgbT3gqa+4ThKh2MpDY6jdDiW0uA4SoPjKB0px5I1x0REREREbThzTERERETUhskxEREREVEbJsc27I477kBQUBCGDRvW8dj+/fuRmpqK4cOH4+qrr0ZdXR0AoKWlBbfffjuGDx+OxMREpKenAwAaGhpw1VVXIT4+HkOHDsUTTzwhx0uRnRRj2X5swYIFiIuLQ3x8PL766itrvxRZnTp1ClOmTMGQIUMwdOhQLF68GABQVVWFSy+9FLGxsbj00ktRXV0NABBC4MEHH8SgQYMwYsQI7Nmzp+Nay5cvR2xsLGJjY/vdBkBSjiMA1NXVITw8HA888IDVX4vcpBzLxx57DEOHDkVCQgIefPBB9Keqw56O47Fjx5CamgoXFxe8/PLLF7xOfyHVOAJATU0NZs6cifj4eCQkJCAjI8Pqr0dOPR3Ljz/+GCNGjMDw4cORlpaG/fv3d1zrp59+wuDBgzFo0CD8+9//vvDNBdmsTZs2id27d4uhQ4d2PJaSkiLS09OFEEIsXbpUPPXUU0IIId544w0xb948IYQQpaWlYtSoUcJkMon6+nqxYcMGIYQQzc3NYvz48eKHH36w8iuRnxRjKYQQzzzzjHjyySeFEEKYTCZRXl5uzZchu6KiIrF7924hhBB1dXUiNjZWHD58WPz1r38VixYtEkIIsWjRIvHYY48JIYRYu3atuPzyy4XZbBYZGRlizJgxQgghKisrxcCBA0VlZaWoqqoSAwcOFFVVVfK8KBlINY7tHnzwQXHTTTeJ+++/37ovxAZINZZbt24VaWlpwmg0CqPRKMaNGyc2btwoy2uSQ0/HsbS0VOzYsUP8/e9/F//5z38ueJ3+QqpxFEKIuXPnivfee08I0fr7u7q62novxAb0dCy3bt3a8Xvkhx9+6PjZNhqNIjo6Whw/flw0NzeLESNGXPB7ksmxjcvNzT0jofPy8hJms1kIIcTJkydFQkKCEEKI++67T6xYsaLjvKlTp4rt27efc70HH3xQLFmyxMJR2yYpxjI8PFzo9XorRm3bZsyYIdatWyfi4uJEUVGREKL1DS0uLk4IIcSCBQvEJ5980nF++3mffPKJWLBgQcfjZ5/X3/R2HIUQYteuXeLGG28UH374Yb9Mjs/W27Hctm2bGDVqlGhoaBD19fUiOTlZHDlyRJbXYAsuNI7t/vGPf5yT1HV2nf6qt+NYU1MjoqKiOn5HUffHUgghqqqqRGhoqBBCiG3btonLLrus49iLL74oXnzxxS7vxbIKOzN06FB89913AIAvvvgCp06dAgAkJiZi9erVMBqNyM3Nxe7duzuOtaupqcH333+Piy++2Opx26KejmVNTQ0A4Omnn8aoUaMwa9YslJaWyhW+7PLy8rB3716MHTsWpaWl0Gq1AICQkJCOcSksLERERETHc8LDw1FYWHjex/ujvoyj2WzGI488cs7Hsf1VX8YyNTUVU6ZMgVarhVarxbRp05CQkCDL65Bbd8axp9fpj/oyjrm5uQgMDMTtt9+OkSNHYv78+aivr7dG2Dapp2O5dOlSXHHFFQDO/zPfFSbHduaDDz7AW2+9heTkZOh0Ojg7OwNorakNDw9HSkoKFi5ciLS0NKhUqo7nGY1G3HTTTXjwwQcRHR0tV/g2padjaTQaUVBQgLS0NOzZswepqal49NFHZX4V8tDr9bj++uvx6quvwsvL64xjCoUCCoVCpsjsS1/H8a233sKVV16J8PBwS4ZpF/o6ljk5OTh69CgKCgpQWFiIDRs2YMuWLZYM2SZJ9bPd1XX6g76Oo9FoxJ49e3Dvvfdi7969cHd3716trAPq6Vhu3LgRS5cuxf/93//1+p7qXj+TZBEfH49169YBALKysrB27VoAgFqtxiuvvNJxXlpaGuLi4jr+vWDBAsTGxmLhwoVWjdeW9XQs/f394ebmhuuuuw4AMGvWLCxdutT6gcvMYDDg+uuvxy233NIxFsHBwSguLoZWq0VxcTGCgoIAAGFhYWd8glFQUICwsDCEhYWdsdCxoKAAkydPtubLkJ0U45iRkYEtW7bgrbfegl6vR0tLCzw8PPrdL1EpxnLlypUYN24cPDw8AABXXHEFMjIyMGHCBOu/IJn0ZBx7ep3+RIpxDA8PR3h4eMes+8yZM/vdzzXQ87E8cOAA5s+fjx9//BH+/v4Azv8z3xXOHNuZsrIyAIDZbMYLL7yAe+65B0BrV4r2j1x++eUXqNVqDBkyBADw1FNPoba2Fq+++qosMduqno6lQqHA1Vdf3ZHU/frrrx1j3F8IIXDnnXciISEBDz/8cMfjM2bM6Og4sXz5clxzzTUdj69YsQJCCPz+++/w9vbu+Mh63bp1qK6uRnV1NdatW4dp06bJ8prkINU4fvzxxzh58iTy8vLw8ssvY+7cuf3uF6hUYzlgwABs2rQJRqMRBoMBmzZt6ldlFT0dx55ep7+QahxDQkIQERGBzMxMAPx9052xPHnyJK677jp89NFHZ0wOjh49GtnZ2cjNzUVLSwtWrVqFGTNmXPDmZKNmz54tQkJChFqtFmFhYeL9998Xr776qoiNjRWxsbHi8ccf7yjWz83NFXFxcSI+Pl5cfPHFIi8vTwghxKlTpwQAER8fLxITE0ViYmLH6tf+RIqxFEKIvLw8MWHCBDF8+HAxdepUkZ+fL9dLksWWLVsEADF8+PCO76e1a9eKiooKMXXqVDFo0CBx8cUXi8rKSiGEEGazWdx3330iOjpaDBs2TOzcubPjWkuXLhUxMTEiJiZGfPDBB3K9JFlIOY7t+uuCPKnG0mg0igULFoj4+HiRkJAgHnroITlfltX1dByLi4tFWFiY8PT0FN7e3iIsLEzU1tae9zr9hVTjKIQQe/fuFcnJyWL48OHimmuu6VcdfYTo+VjeeeedwsfHp+Pc5OTkjmutXbtWxMbGiujoaPHCCy9c8N7cPpqIiIiIqA3LKoiIiIiI2jA5JiIiIiJqw+SYiIiIiKgNk2MiIiIiojZMjomIiIiI2jA5JiKycSqVCklJSRg6dCgSExPx3//+F2azucvn5OXl4ZNPPrFShEREjoPJMRGRjXN1dcW+fftw+PBh/PLLL/jxxx/x3HPPdfkcJsdERL3D5JiIyI4EBQVhyZIleOONNyCEQF5eHiZMmIBRo0Zh1KhR2LZtGwDgiSeewJYtW5CUlIRXXnkFJpMJf/3rXzF69GiMGDEC7777rsyvhIjINnETECIiG+fh4QG9Xn/GYz4+PsjMzISnpyeUSiU0Gg2ys7Nx0003YdeuXUhPT8fLL7+MNWvWAACWLFmCsrIyPPXUU2hubsZFF12EL774AgMHDpTjJRER2Sy13AEQEVHvGQwGPPDAA9i3bx9UKhWysrI6PW/dunU4cOAAvvzySwBAbW0tsrOzmRwTEZ2FyTERkZ05ceIEVCoVgoKC8NxzzyE4OBj79++H2WyGRqPp9DlCCLz++uuYNm2alaMlIrIvrDkmIrIj5eXluOeee/DAAw9AoVCgtrYWWq0WSqUSH330EUwmEwDA09MTOp2u43nTpk3D22+/DYPBAADIyspCfX29LK+BiMiWceaYiMjGNTY2IikpCQaDAWq1GrfeeisefvhhAMB9992H66+/HitWrMDll18Od3d3AMCIESOgUqmQmJiIefPm4S9/+Qvy8vIwatQoCCEQGBiIb7/9VsZXRURkm7ggj4iIiIioDcsqiIiIiIjaMDkmIiIiImrD5JiIiIiIqA2TYyIiIiKiNkyOiYiIiIjaMDkmIiIiImrD5JiIiIiIqM3/A6r+QIoMgdm9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fost.plot(result, node_name='Pennsylvania')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}