tutorials/data_driven/LSTM/train_rnn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c064b8dd-84ac-4da4-9701-9cc296414650",
   "metadata": {},
   "source": [
    "# Tutorial: How to train a rnn using grainlearning.rnn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0d3daa5-ced1-4a89-9a27-4efcaf88f886",
   "metadata": {},
   "source": [
    "## ⚙️ Install grainlearning package\n",
    "(Not necessary if you are running jupyter-lab on an environment where grainlearning and rnn dependencies are installed)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4e11d1c8-a9a2-47dd-81cd-59369c5b79c6",
   "metadata": {},
   "source": [
    "`pip install grainlearning --extras \"rnn\"`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "915d05b1-c05e-4a4b-811b-e946d2cf823f",
   "metadata": {},
   "source": [
    "## Option 1: Train using wandb"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "11e50e9d-c1c0-47ce-8d0a-017d94a1e56e",
   "metadata": {},
   "outputs": [],
   "source": [
    "import grainlearning.rnn.train as train_rnn\n",
    "import wandb"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "159e5115",
   "metadata": {},
   "source": [
    "We define a dictionary `my_config` containing the values of several parameters that are needed for preparing the data, creating the RNN model and training it. Extensive lists of the keywords and the meaning of the values for [preparing data]() and [preparing model and training](https://grainlearning.readthedocs.io/en/latest/api.html#grainlearning.rnn.train.get_default_config)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "04d4f2e5-8bd2-4102-9121-eefd4512dbb1",
   "metadata": {},
   "outputs": [],
   "source": [
    "my_config = {\n",
    "        'raw_data': '/Users/luisaorozco/Documents/Projects/GrainLearning/data/TriaxialCompression/triaxial_compression_variable_input.hdf5',\n",
    "        'pressure': 'All',\n",
    "        'experiment_type': 'drained',\n",
    "        'add_pressure': True,\n",
    "        'add_e0': True,\n",
    "        'train_frac': 0.7,\n",
    "        'val_frac': 0.15,\n",
    "        'window_size': 20,\n",
    "        'window_step': 1,\n",
    "        'patience': 25,\n",
    "        'epochs': 10,\n",
    "        'learning_rate': 1e-4,\n",
    "        'lstm_units': 250,\n",
    "        'dense_units': 250,\n",
    "        'batch_size': 256,\n",
    "        'standardize_outputs': True,\n",
    "        'save_weights_only': True\n",
    "    }"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b3eafc2e",
   "metadata": {},
   "source": [
    "Creating a preprocessor of the data. We choose the preprocessor that is tailored for triaxial compression experiments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "cd637abc",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No add_experiment_type specified in config, using default True.\n",
      "No pad_length specified in config, using default 0.\n"
     ]
    }
   ],
   "source": [
    "from grainlearning.rnn import preprocessor\n",
    "preprocessor_TC = preprocessor.PreprocessorTriaxialCompression(**my_config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "cf0caf89-0cd6-442f-8bda-974bf37e2116",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Failed to detect the name of this notebook, you can set it manually with the WANDB_NOTEBOOK_NAME environment variable to enable code saving.\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mluisaforozco\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9489a9da0b9c4fa3acf8894667e78ad2",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Label(value='Waiting for wandb.init()...\\r'), FloatProgress(value=0.0167534875, max=1.0)))"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "wandb version 0.15.0 is available!  To upgrade, please run:\n",
       " $ pip install wandb --upgrade"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Tracking run with wandb version 0.13.7"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Run data is saved locally in <code>/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel</code>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Syncing run <strong><a href=\"https://wandb.ai/luisaforozco/grainLearning-tutorials_rnn/runs/qq361mel\" target=\"_blank\">astral-snow-7</a></strong> to <a href=\"https://wandb.ai/luisaforozco/grainLearning-tutorials_rnn\" target=\"_blank\">Weights & Biases</a> (<a href=\"https://wandb.me/run\" target=\"_blank\">docs</a>)<br/>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No pad_length specified in config, using default 0.\n",
      "2023-04-25 14:45:23.639273: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:305] Could not identify NUMA node of platform GPU ID 0, defaulting to 0. Your kernel may not have been built with NUMA support.\n",
      "2023-04-25 14:45:23.640027: I tensorflow/core/common_runtime/pluggable_device/pluggable_device_factory.cc:271] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 0 MB memory) -> physical PluggableDevice (device: 0, name: METAL, pci bus id: <undefined>)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Metal device set to: Apple M1 Pro\n",
      "\n",
      "systemMemory: 32.00 GB\n",
      "maxCacheSize: 10.67 GB\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[34m\u001b[1mwandb\u001b[0m: \u001b[33mWARNING\u001b[0m The save_model argument by default saves the model in the HDF5 format that cannot save custom objects like subclassed models and custom layers. This behavior will be deprecated in a future release in favor of the SavedModel format. Meanwhile, the HDF5 model is saved as W&B files and the SavedModel as W&B Artifacts.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-25 14:45:25.060938: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz\n",
      "2023-04-25 14:45:25.070371: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n",
      "2023-04-25 14:45:25.336914: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   1/2214 [..............................] - ETA: 52:22 - loss: 1.1206 - mae: 0.7866"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-25 14:45:25.768454: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - ETA: 0s - loss: 0.2972 - mae: 0.3581"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-25 14:46:16.049515: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n",
      "2023-04-25 14:46:16.121745: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n",
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.1s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 58s 25ms/step - loss: 0.2972 - mae: 0.3581 - val_loss: 0.1388 - val_mae: 0.2406\n",
      "Epoch 2/10\n",
      "2212/2214 [============================>.] - ETA: 0s - loss: 0.0952 - mae: 0.1920"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 42s 19ms/step - loss: 0.0951 - mae: 0.1920 - val_loss: 0.0864 - val_mae: 0.1804\n",
      "Epoch 3/10\n",
      "2211/2214 [============================>.] - ETA: 0s - loss: 0.0745 - mae: 0.1650"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 41s 19ms/step - loss: 0.0745 - mae: 0.1649 - val_loss: 0.0763 - val_mae: 0.1642\n",
      "Epoch 4/10\n",
      "2214/2214 [==============================] - ETA: 0s - loss: 0.0650 - mae: 0.1536"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 41s 19ms/step - loss: 0.0650 - mae: 0.1536 - val_loss: 0.0592 - val_mae: 0.1445\n",
      "Epoch 5/10\n",
      "2214/2214 [==============================] - ETA: 0s - loss: 0.0503 - mae: 0.1337"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 41s 18ms/step - loss: 0.0503 - mae: 0.1337 - val_loss: 0.0481 - val_mae: 0.1195\n",
      "Epoch 6/10\n",
      "2213/2214 [============================>.] - ETA: 0s - loss: 0.0356 - mae: 0.1105"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 41s 19ms/step - loss: 0.0356 - mae: 0.1105 - val_loss: 0.0345 - val_mae: 0.0956\n",
      "Epoch 7/10\n",
      "2212/2214 [============================>.] - ETA: 0s - loss: 0.0258 - mae: 0.0927"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 41s 18ms/step - loss: 0.0258 - mae: 0.0927 - val_loss: 0.0287 - val_mae: 0.0880\n",
      "Epoch 8/10\n",
      "2213/2214 [============================>.] - ETA: 0s - loss: 0.0223 - mae: 0.0854"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 41s 19ms/step - loss: 0.0223 - mae: 0.0854 - val_loss: 0.0258 - val_mae: 0.0809\n",
      "Epoch 9/10\n",
      "2212/2214 [============================>.] - ETA: 0s - loss: 0.0197 - mae: 0.0795"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 41s 18ms/step - loss: 0.0196 - mae: 0.0795 - val_loss: 0.0220 - val_mae: 0.0757\n",
      "Epoch 10/10\n",
      "2214/2214 [==============================] - ETA: 0s - loss: 0.0161 - mae: 0.0720"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Found untraced functions such as lstm_cell_layer_call_fn, lstm_cell_layer_call_and_return_conditional_losses while saving (showing 2 of 2). These functions will not be directly callable after loading.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best/assets\n",
      "\u001b[34m\u001b[1mwandb\u001b[0m: Adding directory to artifact (/Users/luisaorozco/Documents/Projects/GrainLearning/grainLearning/tutorials/rnn/wandb/run-20230425_144520-qq361mel/files/model-best)... Done. 0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 41s 19ms/step - loss: 0.0161 - mae: 0.0720 - val_loss: 0.0176 - val_mae: 0.0689\n",
      "476/476 [==============================] - 4s 9ms/step - loss: 0.0173 - mae: 0.0696\n",
      "test loss = 0.01731450855731964, test mae = 0.06963476538658142\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "Waiting for W&B process to finish... <strong style=\"color:green\">(success).</strong>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<style>\n",
       "    table.wandb td:nth-child(1) { padding: 0 10px; text-align: left ; width: auto;} td:nth-child(2) {text-align: left ; width: 100%}\n",
       "    .wandb-row { display: flex; flex-direction: row; flex-wrap: wrap; justify-content: flex-start; width: 100% }\n",
       "    .wandb-col { display: flex; flex-direction: column; flex-basis: 100%; flex: 1; padding: 10px; }\n",
       "    </style>\n",
       "<div class=\"wandb-row\"><div class=\"wandb-col\"><h3>Run history:</h3><br/><table class=\"wandb\"><tr><td>epoch</td><td>▁▂▃▃▄▅▆▆▇█</td></tr><tr><td>loss</td><td>█▃▂▂▂▁▁▁▁▁</td></tr><tr><td>mae</td><td>█▄▃▃▃▂▂▁▁▁</td></tr><tr><td>test_loss</td><td>▁</td></tr><tr><td>test_mae</td><td>▁</td></tr><tr><td>val_loss</td><td>█▅▄▃▃▂▂▁▁▁</td></tr><tr><td>val_mae</td><td>█▆▅▄▃▂▂▁▁▁</td></tr></table><br/></div><div class=\"wandb-col\"><h3>Run summary:</h3><br/><table class=\"wandb\"><tr><td>best_epoch</td><td>9</td></tr><tr><td>best_val_loss</td><td>0.01758</td></tr><tr><td>epoch</td><td>9</td></tr><tr><td>loss</td><td>0.01614</td></tr><tr><td>mae</td><td>0.07196</td></tr><tr><td>test_loss</td><td>0.01731</td></tr><tr><td>test_mae</td><td>0.06963</td></tr><tr><td>val_loss</td><td>0.01758</td></tr><tr><td>val_mae</td><td>0.06888</td></tr></table><br/></div></div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Synced <strong style=\"color:#cdcd00\">astral-snow-7</strong>: <a href=\"https://wandb.ai/luisaforozco/grainLearning-tutorials_rnn/runs/qq361mel\" target=\"_blank\">https://wandb.ai/luisaforozco/grainLearning-tutorials_rnn/runs/qq361mel</a><br/>Synced 6 W&B file(s), 1 media file(s), 31 artifact file(s) and 2 other file(s)"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Find logs at: <code>./wandb/run-20230425_144520-qq361mel/logs</code>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "history_wandb = train_rnn.train(preprocessor_TC, config=my_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f2e5d15-7e95-4fc5-a05d-0dadbd52177d",
   "metadata": {},
   "source": [
    "You have trained your first model! 🥳️\n",
    "\n",
    "Note that we have only ran 10 epochs, you can change that in the config dictionary that we defined.\n",
    "\n",
    "Also, the results have probably been synced to the wandb project that you had by default, we explain how to set that up in the documentation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e314bee1-f7b5-47e8-9e77-311f104d9bb7",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Option 2: Train using plain tensorflow "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "590c4f1e-26a4-486a-a988-3ab299202185",
   "metadata": {},
   "outputs": [],
   "source": [
    "import grainlearning.rnn.train as train_rnn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fcf30f75-1e05-4d04-93be-016a5bba7c89",
   "metadata": {},
   "outputs": [],
   "source": [
    "# dictionary with the configuration of the training\n",
    "my_config = {\n",
    "        'raw_data': '/Users/luisaorozco/Documents/Projects/GrainLearning/data/TriaxialCompression/triaxial_compression_variable_input.hdf5',\n",
    "        'pressure': 'All',\n",
    "        'experiment_type': 'drained',\n",
    "        'add_pressure': True,\n",
    "        'add_e0': True,\n",
    "        'train_frac': 0.7,\n",
    "        'val_frac': 0.15,\n",
    "        'window_size': 20,\n",
    "        'window_step': 1,\n",
    "        'patience': 25,\n",
    "        'epochs': 10,\n",
    "        'learning_rate': 1e-4,\n",
    "        'lstm_units': 250,\n",
    "        'dense_units': 250,\n",
    "        'batch_size': 256,\n",
    "        'standardize_outputs': True,\n",
    "        'save_weights_only': True\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "47a246f4",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No add_experiment_type specified in config, using default True.\n",
      "No pad_length specified in config, using default 0.\n"
     ]
    }
   ],
   "source": [
    "# Creating the preprocessor of the data\n",
    "from grainlearning.rnn import preprocessor\n",
    "preprocessor_TC = preprocessor.PreprocessorTriaxialCompression(**my_config)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "468f8134-1145-4eab-8636-b83afc028e9f",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "No pad_length specified in config, using default 0.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-25 14:56:35.261345: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n",
      "2023-04-25 14:56:35.439760: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "   4/2214 [..............................] - ETA: 47s - loss: 1.0562 - mae: 0.7639  "
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-25 14:56:35.527022: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - ETA: 0s - loss: 0.2972 - mae: 0.3581"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-04-25 14:57:11.579682: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n",
      "2023-04-25 14:57:11.647852: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:113] Plugin optimizer for device_type GPU is enabled.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2214/2214 [==============================] - 41s 18ms/step - loss: 0.2972 - mae: 0.3581 - val_loss: 0.1388 - val_mae: 0.2406\n",
      "Epoch 2/10\n",
      "2214/2214 [==============================] - 40s 18ms/step - loss: 0.0951 - mae: 0.1920 - val_loss: 0.0864 - val_mae: 0.1804\n",
      "Epoch 3/10\n",
      "2214/2214 [==============================] - 39s 18ms/step - loss: 0.0745 - mae: 0.1649 - val_loss: 0.0763 - val_mae: 0.1642\n",
      "Epoch 4/10\n",
      "2214/2214 [==============================] - 40s 18ms/step - loss: 0.0650 - mae: 0.1536 - val_loss: 0.0592 - val_mae: 0.1445\n",
      "Epoch 5/10\n",
      "2214/2214 [==============================] - 39s 18ms/step - loss: 0.0503 - mae: 0.1337 - val_loss: 0.0481 - val_mae: 0.1195\n",
      "Epoch 6/10\n",
      "2214/2214 [==============================] - 40s 18ms/step - loss: 0.0356 - mae: 0.1105 - val_loss: 0.0345 - val_mae: 0.0956\n",
      "Epoch 7/10\n",
      "2214/2214 [==============================] - 39s 18ms/step - loss: 0.0258 - mae: 0.0927 - val_loss: 0.0287 - val_mae: 0.0880\n",
      "Epoch 8/10\n",
      "2214/2214 [==============================] - 39s 18ms/step - loss: 0.0223 - mae: 0.0854 - val_loss: 0.0258 - val_mae: 0.0809\n",
      "Epoch 9/10\n",
      "2214/2214 [==============================] - 39s 18ms/step - loss: 0.0196 - mae: 0.0795 - val_loss: 0.0220 - val_mae: 0.0757\n",
      "Epoch 10/10\n",
      "2214/2214 [==============================] - 39s 18ms/step - loss: 0.0161 - mae: 0.0720 - val_loss: 0.0176 - val_mae: 0.0689\n",
      "476/476 [==============================] - 4s 9ms/step - loss: 0.0173 - mae: 0.0696\n",
      "test loss = 0.01731450855731964, test mae = 0.06963476538658142\n"
     ]
    }
   ],
   "source": [
    "history_simple = train_rnn.train_without_wandb(preprocessor_TC, config=my_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51a4f455-cff8-4c21-a19c-7f233a36882c",
   "metadata": {},
   "source": [
    "### 📉 Plot the training history "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5db6741-0fd6-478b-9c1f-fa2b1b0c6e1e",
   "metadata": {},
   "source": [
    "⚙️ First, install dependencies for ploting on jupyter notebooks"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "4387d65b-2252-4288-b469-50461f715832",
   "metadata": {},
   "source": [
    "`pip install matplotlib`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "c05631e3-a9ec-44cb-9a52-abaa3405edf8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x313c53ca0>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAkAAAAGwCAYAAABB4NqyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABZhElEQVR4nO3dd3hUZd7/8ffMJJn0RkglkFBERJIAgQhY1yhYEHd1QWUfiq4+P+u6rAV2H0HFFUV0EVFRdhU76lqwIJYoqIiiYOi9JZBGSe+Zmd8fk0yItISUk2Q+r+uaK5MzZ875jonm432+575NDofDgYiIiIgbMRtdgIiIiEhbUwASERERt6MAJCIiIm5HAUhERETcjgKQiIiIuB0FIBEREXE7CkAiIiLidjyMLqA9stvtZGVlERAQgMlkMrocERERaQSHw0FxcTHR0dGYzScf41EAOo6srCxiY2ONLkNEREROQ2ZmJt26dTvpPgpAxxEQEAA4/wEGBgYaXI2IiIg0RlFREbGxsa6/4yejAHQcdZe9AgMDFYBEREQ6mMa0r6gJWkRERNyOApCIiIi4HQUgERERcTvqARIRkVZns9morq42ugzp4Dw9PbFYLC1yLAUgERFpNQ6Hg5ycHAoKCowuRTqJ4OBgIiMjmz1PnwKQiIi0mrrwEx4ejq+vryaXldPmcDgoKysjLy8PgKioqGYdTwFIRERahc1mc4WfLl26GF2OdAI+Pj4A5OXlER4e3qzLYe2iCfrZZ58lLi4Ob29vUlJSWL169Qn3ff/990lOTiY4OBg/Pz+SkpJ47bXXGuzjcDiYPn06UVFR+Pj4kJqayo4dO1r7Y4iIyFHqen58fX0NrkQ6k7rfp+b2lBkegN5++22mTJnCjBkzWLt2LYmJiYwcOdI1xPVboaGh/OMf/2DVqlWsX7+eyZMnM3nyZD7//HPXPrNnz2bevHksWLCAn376CT8/P0aOHElFRUVbfSwREamly17Sklrq98nkcDgcLXKk05SSksKQIUOYP38+4FyINDY2ljvvvJOpU6c26hiDBg3iiiuuYObMmTgcDqKjo/nb3/7GPffcA0BhYSEREREsWrSI66677pTHKyoqIigoiMLCQs0ELSJymioqKtizZw/x8fF4e3sbXY50Eif7vWrK329DR4CqqqpYs2YNqamprm1ms5nU1FRWrVp1yvc7HA7S0tLYtm0b559/PgB79uwhJyenwTGDgoJISUk54TErKyspKipq8BAREZHOy9AAdOjQIWw2GxEREQ22R0REkJOTc8L3FRYW4u/vj5eXF1dccQXPPPMMl1xyCYDrfU055qxZswgKCnI9tBK8iIi0pLi4OObOndvo/ZcvX47JZGr16QMWLVpEcHBwq56jveqQd4EFBASQnp5OSUkJaWlpTJkyhZ49e3LhhRee1vGmTZvGlClTXN/XrSbb0hwOB/vzy7GYTUQH+7T48UVEpGVceOGFJCUlNSm0nMzPP/+Mn59fo/cfPnw42dnZBAUFtcj55ViGBqCwsDAsFgu5ubkNtufm5hIZGXnC95nNZnr37g1AUlISW7ZsYdasWVx44YWu9+Xm5jaYIyA3N5ekpKTjHs9qtWK1Wpv5aU7tn59u4d/f7+HP58bzf1ee1ernExGR1uNwOLDZbHh4nPpPadeuXZt0bC8vr5P+HZTmM/QSmJeXF4MHDyYtLc21zW63k5aWxrBhwxp9HLvdTmVlJQDx8fFERkY2OGZRURE//fRTk47ZGvpGBgCwfn+hoXWIiBjF4XBQVlVjyKOx9/xMmjSJFStW8PTTT2MymTCZTOzdu9d1Weqzzz5j8ODBWK1Wvv/+e3bt2sWYMWOIiIjA39+fIUOG8NVXXzU45m8vgZlMJv7973/z+9//Hl9fX/r06cNHH33kev23l8DqLlV9/vnn9OvXD39/f0aNGkV2drbrPTU1Ndx1110EBwfTpUsX7r//fiZOnMjVV1/dpJ/R888/T69evfDy8qJv374NpppxOBw8+OCDdO/eHavVSnR0NHfddZfr9eeee44+ffrg7e1NREQE1157bZPO3ZYMvwQ2ZcoUJk6cSHJyMkOHDmXu3LmUlpYyefJkACZMmEBMTAyzZs0CnP06ycnJ9OrVi8rKSpYuXcprr73G888/Dzh/qe6++24eeeQR+vTpQ3x8PA888ADR0dFN/iVoaYmxwQBszCrEZndgMevWUBFxL+XVNs6a/vmpd2wFmx8eia/Xqf/sPf3002zfvp2zzz6bhx9+GHCO4OzduxeAqVOnMmfOHHr27ElISAiZmZlcfvnl/POf/8RqtfLqq68yevRotm3bRvfu3U94noceeojZs2fzxBNP8MwzzzB+/Hj27dtHaGjocfcvKytjzpw5vPbaa5jNZv70pz9xzz338MYbbwDw+OOP88Ybb/Dyyy/Tr18/nn76aT788EMuuuiiRv8z+uCDD/jLX/7C3LlzSU1N5ZNPPmHy5Ml069aNiy66iPfee49//etfLF68mP79+5OTk8O6desA+OWXX7jrrrt47bXXGD58OEeOHOG7775r9LnbmuEBaNy4cRw8eJDp06eTk5NDUlISy5YtczUxZ2RkYDbXD1SVlpZy2223sX//fnx8fDjzzDN5/fXXGTdunGuf++67j9LSUm655RYKCgo499xzWbZsmeG3Yfbq6o+vl4WyKhs780pcI0IiItJ+BAUF4eXlha+v73EvQz388MOuG2/AOT9dYmKi6/uZM2fywQcf8NFHH3HHHXec8DyTJk3i+uuvB+DRRx9l3rx5rF69mlGjRh13/+rqahYsWECvXr0AuOOOO1wBDeCZZ55h2rRp/P73vwdg/vz5LF26tAmfHObMmcOkSZO47bbbAOcgxY8//sicOXO46KKLyMjIIDIyktTUVDw9PenevTtDhw4FnH+v/fz8uPLKKwkICKBHjx4MHDiwSedvS4YHIHD+EE/0S7J8+fIG3z/yyCM88sgjJz2eyWTi4YcfbvCL0R5YzCbOjgli9Z4jrNtfoAAkIm7Hx9PC5odHGnbulpCcnNzg+5KSEh588EE+/fRTsrOzqampoby8nIyMjJMeJyEhwfXcz8+PwMDAE04CDM4ZkOvCDzjXwqrbv7CwkNzcXFcYAbBYLAwePBi73d7oz7ZlyxZuueWWBttGjBjB008/DcAf//hH5s6dS8+ePRk1ahSXX345o0ePxsPDg0suuYQePXq4Xhs1apTrEl97ZPhM0O4msZuzo3/9/gJjCxERMYDJZMLXy8OQR0vNIPzbu7nuuecePvjgAx599FG+++470tPTGTBgAFVVVSc9jqen5zH/bE4WVo63f1vPZRwbG8u2bdt47rnn8PHx4bbbbuP888+nurqagIAA1q5dy1tvvUVUVBTTp08nMTGx1W/lP10KQG0soVswoEZoEZH2zMvLC5vN1qh9V65cyaRJk/j973/PgAEDiIyMdPULtZWgoCAiIiL4+eefXdtsNhtr165t0nH69evHypUrG2xbuXIlZ51Vf+eyj48Po0ePZt68eSxfvpxVq1axYcMGADw8PEhNTWX27NmsX7+evXv38vXXXzfjk7WednEJzJ0k1gagLdlFVNbYsHq0zJCsiIi0nLi4OH766Sf27t2Lv7//CRuTAfr06cP777/P6NGjMZlMPPDAA0267NRS7rzzTmbNmkXv3r0588wzeeaZZ8jPz2/SyNe9997L2LFjGThwIKmpqXz88ce8//77rrvaFi1ahM1mIyUlBV9fX15//XV8fHzo0aMHn3zyCbt37+b8888nJCSEpUuXYrfb6du3b2t95GbRCFAbiw31IcTXk2qbgy3ZxUaXIyIix3HPPfdgsVg466yz6Nq160n7eZ566ilCQkIYPnw4o0ePZuTIkQwaNKgNq3W6//77uf7665kwYQLDhg3D39+fkSNHNukGoKuvvpqnn36aOXPm0L9/f1544QVefvll10TDwcHBLFy4kBEjRpCQkMBXX33Fxx9/TJcuXQgODub999/nd7/7Hf369WPBggW89dZb9O/fv5U+cfMYvhhqe9Tai6FOeGk1324/yMNj+jNhWFyLH19EpD3QYqjGstvt9OvXj7FjxzJz5kyjy2kxnWIxVHeVVNsIvS5TfUAiItIy9u3bx8KFC9m+fTsbNmzg1ltvZc+ePdxwww1Gl9YuKQAZoL4RusDQOkREpPMwm80sWrSIIUOGMGLECDZs2MBXX31Fv379jC6tXVITtAESYp0jQDsPllBSWYO/VT8GERFpntjY2GPu4JIT0wiQAcIDvIkK8sbhgI0HdBlMRESkrSkAGSRBEyKKiIgYRgHIIHV9QOs0IaKIiEibUwAySKIaoUVERAyjAGSQAbWXwDKPlHO4pNLgakRERNyLApBBgnw8iQ9zLqi3Xo3QIiKdTlxcHHPnznV9bzKZ+PDDD0+4/969ezGZTKSnpzfrvC11nFOZNGkSV199daueozXp/msDJXQLYs+hUtZnFnJR33CjyxERkVaUnZ1NSEhIix5z0qRJFBQUNAhWsbGxZGdnExYW1qLn6mw0AmQg9QGJiLiPyMhIrFZrq5/HYrEQGRmJh4fGOE5GAchAibUTIq7bX4iWZBMRaR9efPFFoqOjj1nRfcyYMdx4440A7Nq1izFjxhAREYG/vz9DhgxxrZh+Ir+9BLZ69WoGDhyIt7c3ycnJ/Prrrw32t9ls3HTTTcTHx+Pj40Pfvn15+umnXa8/+OCDvPLKKyxZsgSTyYTJZGL58uXHvQS2YsUKhg4ditVqJSoqiqlTp1JTU+N6/cILL+Suu+7ivvvuIzQ0lMjISB588MEm/XOrrKzkrrvuIjw8HG9vb84991x+/vln1+v5+fmMHz+erl274uPjQ58+fXj55ZcBqKqq4o477iAqKgpvb2969OjBrFmzmnT+plI8NNBZUUFYzCYOlVSSXVhBdLCP0SWJiLQuhwOqy4w5t6cvmEyn3O2Pf/wjd955J9988w0XX3wxAEeOHGHZsmUsXboUgJKSEi6//HL++c9/YrVaefXVVxk9ejTbtm2je/fupzxHSUkJV155JZdccgmvv/46e/bs4S9/+UuDfex2O926dePdd9+lS5cu/PDDD9xyyy1ERUUxduxY7rnnHrZs2UJRUZErSISGhpKVldXgOAcOHODyyy9n0qRJvPrqq2zdupWbb74Zb2/vBiHnlVdeYcqUKfz000+sWrWKSZMmMWLECC655JJTfh6A++67j/fee49XXnmFHj16MHv2bEaOHMnOnTsJDQ3lgQceYPPmzXz22WeEhYWxc+dOysvLAZg3bx4fffQR77zzDt27dyczM5PMzMxGnfd0KQAZyMfLwhkRAWzJLmL9/gIFIBHp/KrL4NFoY8799yzw8jvlbiEhIVx22WW8+eabrgD03//+l7CwMC666CIAEhMTSUxMdL1n5syZfPDBB3z00UfccccdpzzHm2++id1u5z//+Q/e3t7079+f/fv3c+utt7r28fT05KGHHnJ9Hx8fz6pVq3jnnXcYO3Ys/v7++Pj4UFlZSWRk5AnP9dxzzxEbG8v8+fMxmUyceeaZZGVlcf/99zN9+nTMZufFoISEBGbMmAFAnz59mD9/PmlpaY0KQKWlpTz//PMsWrSIyy67DICFCxfy5Zdf8p///Id7772XjIwMBg4cSHJyMuBsEq+TkZFBnz59OPfcczGZTPTo0eOU52wuXQIzWGK3+stgIiLSPowfP5733nuPykrnNCVvvPEG1113nSsslJSUcM8999CvXz+Cg4Px9/dny5YtZGRkNOr4W7ZsISEhAW9vb9e2YcOGHbPfs88+y+DBg+natSv+/v68+OKLjT7H0ecaNmwYpqNGv0aMGEFJSQn79+93bUtISGjwvqioKPLy8hp1jl27dlFdXc2IESNc2zw9PRk6dChbtmwB4NZbb2Xx4sUkJSVx33338cMPP7j2nTRpEunp6fTt25e77rqLL774okmf8XRoBMhgCd2CWfxzphqhRcQ9ePo6R2KMOncjjR49GofDwaeffsqQIUP47rvv+Ne//uV6/Z577uHLL79kzpw59O7dGx8fH6699lqqqqparNzFixdzzz338OSTTzJs2DACAgJ44okn+Omnn1rsHEfz9PRs8L3JZDqmD6o5LrvsMvbt28fSpUv58ssvufjii7n99tuZM2cOgwYNYs+ePXz22Wd89dVXjB07ltTUVP773/+22Pl/SwHIYPVrghVitzswm099fVpEpMMymRp1Gcpo3t7e/OEPf+CNN95g586d9O3bl0GDBrleX7lyJZMmTeL3v/894BwR2rt3b6OP369fP1577TUqKipco0A//vhjg31WrlzJ8OHDue2221zbdu3a1WAfLy8vbDbbKc/13nvv4XA4XKNAK1euJCAggG7dujW65pPp1asXXl5erFy50nX5qrq6mp9//pm7777btV/Xrl2ZOHEiEydO5LzzzuPee+9lzpw5AAQGBjJu3DjGjRvHtddey6hRozhy5AihoaEtUuNv6RKYwfpGBmD1MFNcUcPew6VGlyMiIrXGjx/Pp59+yksvvcT48eMbvNanTx/ef/990tPTWbduHTfccEOTRktuuOEGTCYTN998M5s3b2bp0qWuIHD0OX755Rc+//xztm/fzgMPPNDgripw9tGsX7+ebdu2cejQIaqrq48512233UZmZiZ33nknW7duZcmSJcyYMYMpU6a4Luk1l5+fH7feeiv33nsvy5YtY/Pmzdx8882UlZVx0003ATB9+nSWLFnCzp072bRpE5988gn9+vUD4KmnnuKtt95i69atbN++nXfffZfIyEiCg4NbpL7jUQAymKfFzFnRgQCs02UwEZF243e/+x2hoaFs27aNG264ocFrTz31FCEhIQwfPpzRo0czcuTIBiNEp+Lv78/HH3/Mhg0bGDhwIP/4xz94/PHHG+zzv//7v/zhD39g3LhxpKSkcPjw4QajQQA333wzffv2JTk5ma5du7Jy5cpjzhUTE8PSpUtZvXo1iYmJ/L//9/+46aab+L//+78m/NM4tccee4xrrrmG//mf/2HQoEHs3LmTzz//3DX5o5eXF9OmTSMhIYHzzz8fi8XC4sWLAQgICGD27NkkJyczZMgQ9u7dy9KlS1ssoB2PyaEJaI5RVFREUFAQhYWFBAYGtvr5HvxoE4t+2Muk4XE8eFX/Vj+fiEhbqKioYM+ePcTHxzdo9hVpjpP9XjXl77dGgNqBugkR1QgtIiLSNhSA2oGE2iUxNmUVUW1ruY57EREROT4FoHYgvosfAVYPKmvsbM8tNrocERGRTk8BqB0wm00MOOp2eBEREWldCkDtRIJWhheRTkr32khLaqnfJwWgdsK1JEamRoBEpHOom1m4rMygxU+lU6r7ffrtzNVNpZmg24mE2GAAtuUWU1Ftw9vTYmxBIiLNZLFYCA4Odq0n5evr22A9KpGmcDgclJWVkZeXR3BwMBZL8/5OKgC1E9FB3oT5e3GopIpNWUUM7hFidEkiIs1Wt0p5YxfVFDmV4OBg1+9VcygAtRMmk4mEbsF8vTWPdZkFCkAi0imYTCaioqIIDw8/7jINIk3h6enZ7JGfOgpA7UhCtyC+3pqnRmgR6XQsFkuL/eESaQlqgm5HEmv7gHQrvIiISOtSAGpHEmtvhd99qJTCcg0Vi4iItBYFoHYk1M+LbiE+AGw8oFEgERGR1qIA1M7UjQKtUx+QiIhIq1EAamcS6pbE0ISIIiIirUYBqJ3RkhgiIiKtTwGonRnQLQiTCbIKKzhYXGl0OSIiIp2SAlA742/1oFdXf0CjQCIiIq1FAagdSnAtjFpgbCEiIiKdlAJQO1R/J5gaoUVERFqDAlA7VD8jdAEOh8PYYkRERDohBaB2qF9UAJ4WE/ll1ezPLze6HBERkU5HAagdsnpYODMyENCEiCIiIq1BAaidck2IqD4gERGRFtcuAtCzzz5LXFwc3t7epKSksHr16hPuu3DhQs477zxCQkIICQkhNTX1mP0nTZqEyWRq8Bg1alRrf4wW5WqE1p1gIiIiLc7wAPT2228zZcoUZsyYwdq1a0lMTGTkyJHk5eUdd//ly5dz/fXX880337Bq1SpiY2O59NJLOXDgQIP9Ro0aRXZ2tuvx1ltvtcXHaTEJsc4RoI0HCrHZ1QgtIiLSkkwOg28zSklJYciQIcyfPx8Au91ObGwsd955J1OnTj3l+202GyEhIcyfP58JEyYAzhGggoICPvzww0bVUFlZSWVl/azLRUVFxMbGUlhYSGBgYNM/VAuosdkZ8OAXlFfb+PKv59MnIsCQOkRERDqKoqIigoKCGvX329ARoKqqKtasWUNqaqprm9lsJjU1lVWrVjXqGGVlZVRXVxMaGtpg+/LlywkPD6dv377ceuutHD58+ITHmDVrFkFBQa5HbGzs6X2gFuRhMXN2TF0jtPqAREREWpKhAejQoUPYbDYiIiIabI+IiCAnJ6dRx7j//vuJjo5uEKJGjRrFq6++SlpaGo8//jgrVqzgsssuw2azHfcY06ZNo7Cw0PXIzMw8/Q/VgrQwqoiISOvwMLqA5njsscdYvHgxy5cvx9vb27X9uuuucz0fMGAACQkJ9OrVi+XLl3PxxRcfcxyr1YrVam2TmptCS2KIiIi0DkNHgMLCwrBYLOTm5jbYnpubS2Rk5EnfO2fOHB577DG++OILEhISTrpvz549CQsLY+fOnc2uuS0l1c4IvSW7mKoau7HFiIiIdCKGBiAvLy8GDx5MWlqaa5vdbictLY1hw4ad8H2zZ89m5syZLFu2jOTk5FOeZ//+/Rw+fJioqKgWqbutdA/1JdjXkyqbna05RUaXIyIi0mkYfhv8lClTWLhwIa+88gpbtmzh1ltvpbS0lMmTJwMwYcIEpk2b5tr/8ccf54EHHuCll14iLi6OnJwccnJyKCkpAaCkpIR7772XH3/8kb1795KWlsaYMWPo3bs3I0eONOQzni6TycSAmNrLYGqEFhERaTGG9wCNGzeOgwcPMn36dHJyckhKSmLZsmWuxuiMjAzM5vqc9vzzz1NVVcW1117b4DgzZszgwQcfxGKxsH79el555RUKCgqIjo7m0ksvZebMme2yz+dUErsF892OQ6zPLIBzehhdjoiISKdg+DxA7VFT5hFobV9syuGW19bQNyKAz/96vqG1iIiItGcdZh4gObXE2kboHXnFlFXVGFuMiIhIJ6EA1M5FBHoTEWjF7oCNB9QILSIi0hIUgDoATYgoIiLSshSAOoDEbroTTEREpCUpAHUAdSNAmhFaRESkZSgAdQB1S2JkHCkjv7TK4GpEREQ6PgWgDiDY14u4Lr4ArD+gy2AiIiLNpQDUQbgaoXUZTEREpNkUgDqIBDVCi4iItBgFoA6ibkJE3QovIiLSfApAHUT/6EDMJsgrriSnsMLockRERDo0BaAOwtfLgzMiAgBYp1EgERGRZlEA6kDq+oB0GUxERKR5FIA6kPolMdQILSIi0hwKQB1I4lEzQjscDmOLERER6cAUgDqQvpEBeFnMFFXUsPdwmdHliIiIdFgKQB2Il4eZs6IDAfUBiYiINIcCUAfjWhk+U31AIiIip0sBqIOpb4QuMLQOERGRjkwBqINJjHWOAG3MKqTGZje4GhERkY5JAaiD6Rnmj7/Vg4pqOzvySowuR0REpENSAOpgzGYTZ8eoEVpERKQ5FIA6INd8QJoQUURE5LQoAHVAaoQWERFpHgWgDqhuTbCt2cVUVNsMrkZERKTjUQDqgLqF+BDq50WN3cHm7CKjyxEREelwFIA6IJPJ5JoQcX1mgbHFiIiIdEAKQB2UVoYXERE5fQpAHVTdhIjr1AgtIiLSZApAHVTdCNDuQ6UUV1QbW4yIiEgHowDUQYX5W4kJ9sHhgA0HdBlMRESkKRSAOrC62+HVByQiItI0CkAdmCZEFBEROT0KQB1Y3a3w6zI1AiQiItIUCkAd2Nm1AehAQTmHSyoNrkZERKTjUADqwAK9PenZ1Q9QH5CIiEhTKAB1cEm1fUDpmhFaRESk0RSAOrj6O8EKjC1ERESkA1EA6uASYoMB5yUwh8NhbDEiIiIdhAJQB3dWVCAeZhOHS6s4UFBudDkiIiIdggJQB+ftaaFvZACgRmgREZHGUgDqBOomRNTCqCIiIo2jANQJ1E2IuF4TIoqIiDSKAlAnUDcCtPFAIXa7GqFFRERORQGoEzgjwh9vTzPFlTXsPlRqdDkiIiLtngJQJ+BhMdM/WvMBiYiINJYCUCeRWNcIrRmhRURETkkBqJNIjK1dGV63wouIiJySAlAnUdcIvTm7iKoau7HFiIiItHPtIgA9++yzxMXF4e3tTUpKCqtXrz7hvgsXLuS8884jJCSEkJAQUlNTj9nf4XAwffp0oqKi8PHxITU1lR07drT2xzBUXBdfAr09qKqxsz232OhyRERE2jXDA9Dbb7/NlClTmDFjBmvXriUxMZGRI0eSl5d33P2XL1/O9ddfzzfffMOqVauIjY3l0ksv5cCBA659Zs+ezbx581iwYAE//fQTfn5+jBw5koqKirb6WG3OZDJpQkQREZFGMjkMXkEzJSWFIUOGMH/+fADsdjuxsbHceeedTJ069ZTvt9lshISEMH/+fCZMmIDD4SA6Opq//e1v3HPPPQAUFhYSERHBokWLuO666445RmVlJZWVla7vi4qKiI2NpbCwkMDAwBb6pK1v9rKtPLd8F+OSY3n82gSjyxEREWlTRUVFBAUFNervt6EjQFVVVaxZs4bU1FTXNrPZTGpqKqtWrWrUMcrKyqiuriY0NBSAPXv2kJOT0+CYQUFBpKSknPCYs2bNIigoyPWIjY1txqcyjkaAREREGsfQAHTo0CFsNhsRERENtkdERJCTk9OoY9x///1ER0e7Ak/d+5pyzGnTplFYWOh6ZGZmNvWjtAt1d4LtyCuhvMpmcDUiIiLtl4fRBTTHY489xuLFi1m+fDne3t6nfRyr1YrVam3ByowRGehN1wArB4sr2ZRVSHJcqNEliYiItEuGjgCFhYVhsVjIzc1tsD03N5fIyMiTvnfOnDk89thjfPHFFyQk1Pe71L3vdI7Z0ZlMJtfCqJoPSERE5MQMDUBeXl4MHjyYtLQ01za73U5aWhrDhg074ftmz57NzJkzWbZsGcnJyQ1ei4+PJzIyssExi4qK+Omnn056zM5CM0KLiIicmuGXwKZMmcLEiRNJTk5m6NChzJ07l9LSUiZPngzAhAkTiImJYdasWQA8/vjjTJ8+nTfffJO4uDhXX4+/vz/+/v6YTCbuvvtuHnnkEfr06UN8fDwPPPAA0dHRXH311UZ9zDaTEBsMaE0wERGRkzE8AI0bN46DBw8yffp0cnJySEpKYtmyZa4m5oyMDMzm+oGq559/nqqqKq699toGx5kxYwYPPvggAPfddx+lpaXccsstFBQUcO6557Js2bJm9Ql1FAkxzktgew+XUVhWTZCvp8EViYiItD+GzwPUHjVlHoH26PzZ35BxpIzXbhrKeX26Gl2OiIhIm+gw8wBJ60iobYRer0ZoERGR41IA6oTUCC0iInJyCkCdkEaARERETk4BqBM6OyYIswlyiirIK+q8C8CKiIicLgWgTsjP6kHvcH9AEyKKiIgcjwJQJ1W3MKrmAxIRETmWAlAnlVg7IaJGgERERI6lANRJJboaoQvQVE8iIiINKQB1UmdGBuJlMVNQVk3GkTKjyxEREWlXFIA6KS8PM/2iAgBdBhMREfktBaBOzNUIrQkRRUREGlAA6sQ0IaKIiMjxKQB1YnV3gm3MKsRmVyO0iIhIHQWgTqxXV398vSyUVdnYmVdidDkiIiLthgJQJ2Yxmzg7xnkZbJ0mRBQREXFRAOrkjp4PSERERJwUgDq5uj4gNUKLiIjUUwDq5BJrb4Xfkl1EZY3N2GJERETaCQWgTq5biA8hvp5U2xxsyS42uhwREZF2QQGokzOZTFoZXkRE5DcUgNxAXSP0ukz1AYmIiIACkFvQCJCIiEhDCkBuICHWOQK082AJJZU1BlcjIiJiPAUgNxAe4E1UkDcOB2w8oMtgIiIiCkBuIkETIoqIiLgoALmJuj6gdZoQUURERAHIXSS5ZoQuMLQOERGR9kAByE3ULYqaeaScI6VVBlcjIiJiLAUgNxHk40nPMD9AK8OLiIgoALkRVyO0JkQUERE3pwDkRjQhooiIiJMCkBtJrJ0Qcd3+QhwOh8HViIiIGEcByI2cFRWExWziUEkl2YUVRpcjIiJiGAUgN+LjZeGMiABAl8FERMS9NSkAzZ49m/Lyctf3K1eupLKy0vV9cXExt912W8tVJy3OtTK8JkQUERE31qQANG3aNIqLi13fX3bZZRw4cMD1fVlZGS+88ELLVSctTo3QIiIiTQxAv22cVSNtx1PXCL1+fyF2u35+IiLintQD5GbOiAjA6mGmuKKGvYdLjS5HRETEEApAbsbTYqZ/dCCgGaFFRMR9eTT1Df/+97/x9/cHoKamhkWLFhEWFgbQoD9I2q+EbsGszShgXWYhvx/YzehyRERE2lyTAlD37t1ZuHCh6/vIyEhee+21Y/aR9q2+D6jA2EJEREQM0qQAtHfv3lYqQ9pS3Z1gm7KKqLbZ8bToSqiIiLgX/eVzQ/Fd/AiwelBZY2d7ri5bioiI+2lSAFq1ahWffPJJg22vvvoq8fHxhIeHc8sttzSYGFHaJ7PZxIBu9bfDi4iIuJsmBaCHH36YTZs2ub7fsGEDN910E6mpqUydOpWPP/6YWbNmtXiR0vI0IaKIiLizJgWg9PR0Lr74Ytf3ixcvJiUlhYULFzJlyhTmzZvHO++80+JFSstzLYmRqREgERFxP00KQPn5+URERLi+X7FiBZdddpnr+yFDhpCZmdly1UmrSYgNBmBbbjEV1TZjixEREWljTQpAERER7NmzB4CqqirWrl3LOeec43q9uLgYT0/Plq1QWkV0kDdh/lZsdgebsoqMLkdERKRNNSkAXX755UydOpXvvvuOadOm4evry3nnned6ff369fTq1atJBTz77LPExcXh7e1NSkoKq1evPuG+mzZt4pprriEuLg6TycTcuXOP2efBBx/EZDI1eJx55plNqskdmEymoy6DFRhbjIiISBtrUgCaOXMmHh4eXHDBBSxcuJAXX3wRLy8v1+svvfQSl156aaOP9/bbbzNlyhRmzJjB2rVrSUxMZOTIkeTl5R13/7KyMnr27Mljjz1GZGTkCY/bv39/srOzXY/vv/++8R/SjagRWkRE3FWTJkIMCwvj22+/pbCwEH9/fywWS4PX3333XQICAhp9vKeeeoqbb76ZyZMnA7BgwQI+/fRTXnrpJaZOnXrM/kOGDGHIkCEAx329joeHx0kDkjglxOpWeBERcU9NCkA33nhjo/Z76aWXTrlPVVUVa9asYdq0aa5tZrOZ1NRUVq1a1ZSyjrFjxw6io6Px9vZm2LBhzJo166RLdFRWVjaYv6ioyD16YhJrR4B2HyqlsLyaIB/1b4mIiHto0iWwRYsW8c0331BQUEB+fv4JH41x6NAhbDZbg7vKwNlonZOT05SyGkhJSWHRokUsW7aM559/nj179nDeeeeddKHWWbNmERQU5HrExsae9vk7klA/L7qF+ACw8YBGgURExH00aQTo1ltv5a233mLPnj1MnjyZP/3pT4SGhrZWbafl6NvyExISSElJoUePHrzzzjvcdNNNx33PtGnTmDJliuv7oqIitwlBid2C2Z9fzrr9BYzoHWZ0OSIiIm2iSSNAzz77LNnZ2dx33318/PHHxMbGMnbsWD7//HMcDkeTThwWFobFYiE3N7fB9tzc3Bbt3wkODuaMM85g586dJ9zHarUSGBjY4OEuEuqWxNCEiCIi4kaavBiq1Wrl+uuv58svv2Tz5s3079+f2267jbi4OEpKShp9HC8vLwYPHkxaWpprm91uJy0tjWHDhjW1rBMqKSlh165dREVFtdgxOxPdCSYiIu6oSZfAfstsNmMymXA4HNhsTZ9NeMqUKUycOJHk5GSGDh3K3LlzKS0tdd0VNmHCBGJiYlzri1VVVbF582bX8wMHDpCeno6/vz+9e/cG4J577mH06NH06NGDrKwsZsyYgcVi4frrr2/OR+20BnQLwmSCrMIKDhZX0jXAanRJIiIira7JI0CVlZW89dZbXHLJJZxxxhls2LCB+fPnk5GRgb+/f5OONW7cOObMmcP06dNJSkoiPT2dZcuWuRqjMzIyyM7Odu2flZXFwIEDGThwINnZ2cyZM4eBAwfy5z//2bXP/v37uf766+nbty9jx46lS5cu/Pjjj3Tt2rWpH9Ut+Fs96N3V+XPTKJCIiLgLk6MJzTu33XYbixcvJjY2lhtvvJHx48cTFtb5GmeLiooICgqisLDQLfqB/vbOOt5bu5+7Lu7DlEvOMLocERGR09KUv99NugS2YMECunfvTs+ePVmxYgUrVqw47n7vv/9+Uw4rBkuMDeK9tfu1JIaIiLiNJgWgCRMmYDKZWqsWMcjRjdAOh0M/YxER6fSaFIAWLVrUSmWIkfpFBeBpMZFfVs3+/HJiQ32NLklERKRVNbkJWjofq4eFMyOd10rXqRFaRETcgAJQW6ts/FxJbck1IaIWRhURETegANSW9q6EeQNh66dGV3KMuoVR1QgtIiLuQAGoLW39BErz4P3/hYPbja6mgYRY5wjQxgOF2OxNW9ZERESko1EAakuXPAw9RkBVMSy+ASraz+Wm3l398fG0UFplY/fB9nmZTkREpKUoALUliyf88RUIjIHDO5wjQXa70VUB4GExMyDGOQq0Tn1AIiLSySkAtTX/rjDuNbBYYftn8O1soytyqW+ELjC2EBERkVamAGSEmMFw5b+cz5fPgq1Lja2nVkJsMKBGaBER6fwUgIwycDwMvcX5/P1b2kVTdGLtCNCW7GKqatrHpTkREZHWoABkpJGPQvfhzqbot8dDRZGh5XQP9SXY15Mqm52tOcbWIiIi0poUgIxk8YSxr0BANBzaDh8Y2xRtMpnUCC0iIm5BAcho/uEw7nVnU/S2pfDtE4aWUzch4nr1AYmISCemANQedBsMVz7lfL78Udj2mWGlaEkMERFxBwpA7cXAP8GQPzufv38LHNphSBmJtXeC7cgrpqyqxpAaREREWpsCUHsychZ0HwaVRbUzRbd9I3JEoDcRgVbsDth4QI3QIiLSOSkAtSceXs6Zouuaoj+81ZCmaFcfkCZEFBGRTkoBqL0JiKidKdrLuXjqd3PavIS6y2C6E0xERDorBaD2qFsyXPGk8/k3j8K2ZW16ei2JISIinZ0CUHs1aAIk3wQ44P2b4dDONjt1QkwwAPsOl5FfWtVm5xUREWkrCkDt2ajHIPac+qboyuI2OW2QrydxXXwBWH9Al8FERKTzUQBqzzy8YOyrEBAFh7bBB/+vzZqiEzQhooiIdGIKQO1dQASMPaop+vsn2+S0dX1AaoQWEZHOSAGoI4gdApfX3g329T9h++etfsq6O8HUCC0iIp2RAlBHMXgiJN8IOOC9m+HwrlY9Xf/oQMwmyCuuJKewolXPJSIi0tYUgDqSUY9DbApUFrZ6U7SvlwdnRAQAsE6jQCIi0skoAHUkdU3R/pFwcKtzpmiHo9VOpxmhRUSks1IA6mgCIp0zRZs9YcvH8F3rNUUnxGpleBER6ZwUgDqi2KFwRV1T9COw48tWOU3dCNC6zAIcrTjSJCIi0tYUgDqqwZOcDxzw3k2t0hTdNzIALw8zRRU17D1c1uLHFxERMYoCUEd22WzoNhQqCmHxeKgsadHDe1rMnBUVCKgPSEREOhcFoI7Mw1rbFB0BB7fAkttavCk6sW5CxEz1AYmISOehANTRBUY5Z4o2e8LmJfD9v1r08Am6E0xERDohBaDOoHsKXD7b+TztYdjxVYsdOrH2TrCNWYXU2NpmHTIREZHWpgDUWSTfCIMm4myKvhGO7G6Rw/YM88ff6kFFtZ0deS3bYyQiImIUBaDO5PInoNuQFm2KNptNnB2jRmgREelcFIA6Ew+rsx/IPwLyNsOS21ukKbpuYVStDC8iIp2FAlBnExjlvDPM7AmbP4SVc5t9SC2JISIinY0CUGfU/Ry47HHn868egp3Na4pOqL0Vfmt2MRXVtuZWJyIiYjgFoM4q+UYY+D+AA/7bvKbomGAfuvh5UWN3sDm7qOVqFBERMYgCUGdlMsEVT0JMcm1T9J+gqvQ0D2VyjQKtzyxowSJFRESMoQDUmXlYnSvH+4VD3qZmNUXXT4ioRmgREen4FIA6u8Do2qZoD9j0Aax8+rQOUzch4jo1QouISCegAOQOegyDUY85n6c9BDvTmnyIuhGg3YdKKa6obsHiRERE2p4CkLsY8mcY+Cdw2Gubovc06e1h/lZign1wOGDDAV0GExGRjk0ByF2YTHD5kxAzGCoK4O2mN0W7GqHVByQiIh2cApA78fR2zhTt1xVyN8KSO5rUFF03I7QmRBQRkY7O8AD07LPPEhcXh7e3NykpKaxevfqE+27atIlrrrmGuLg4TCYTc+fObfYx3U5QzFFN0e/DD880+q11I0DrMjUCJCIiHZuhAejtt99mypQpzJgxg7Vr15KYmMjIkSPJy8s77v5lZWX07NmTxx57jMjIyBY5plvqMby+KfqrGbDr60a9bUBMECYTHCgo53BJZSsWKCIi0roMDUBPPfUUN998M5MnT+ass85iwYIF+Pr68tJLLx13/yFDhvDEE09w3XXXYbVaW+SYbmvInyFpfH1TdP7eU74lwNuTnmF+gPqARESkYzMsAFVVVbFmzRpSU1PrizGbSU1NZdWqVW16zMrKSoqKiho8Oj2TCa54CqIHQnl+7UzRZad8W93CqOmaEVpERDowwwLQoUOHsNlsRERENNgeERFBTk5Omx5z1qxZBAUFuR6xsbGndf4Ox9Mbxr1e2xS9AT6685RN0fV3ghW0QYEiIiKtw/Am6PZg2rRpFBYWuh6ZmZlGl9R2grrBH19xNkVv/C+smn/S3RNcd4IV4jjNZTVERESMZlgACgsLw2KxkJub22B7bm7uCRucW+uYVquVwMDABg+3EjcCRj7qfP7ldNj1zQl3PSsqEA+zicOlVRwoKG+jAkVERFqWYQHIy8uLwYMHk5ZWvyyD3W4nLS2NYcOGtZtjuo2ht0DiDUc1Re877m7enhb6RgYAaoQWEZGOy9BLYFOmTGHhwoW88sorbNmyhVtvvZXS0lImT54MwIQJE5g2bZpr/6qqKtLT00lPT6eqqooDBw6Qnp7Ozp07G31MOQGTCa58CqKSoPwIvD3+hE3RdeuC/bDrkC6DiYhIh+Rh5MnHjRvHwYMHmT59Ojk5OSQlJbFs2TJXE3NGRgZmc31Gy8rKYuDAga7v58yZw5w5c7jgggtYvnx5o44pJ+HpA9e9AS9cADkb4OO74A8LneHoKEPjQ3hrdQav/5jBhv2F3HVxH353Zjim3+wnIiLSXpkc+l/4YxQVFREUFERhYaH79QMB7PkOXh0DDpuzN2jY7Q1ettsdzP58G6/8sJfyahsA/aMDueviPlzSLwKzWUFIRETaXlP+fisAHYfbByCAHxfAsvvBZIH/+QB6XnDMLodKKvn3d3t4ddVeyqqcQejMyADuurgPo/pHKgiJiEibUgBqJgUgnPMBfXgrrHsLfELhf1dAcPfj7nqktIqXvt/Doh/2UlJZA0CfcH/uvLgPVwyIwqIgJCIibUABqJkUgGpVl8NLIyF7HUQmwI2fg5fvCXcvLKvmpZV7eGnlHoornEGoZ1c/7vxdb0YnRONh0bRTIiLSehSAmkkB6CgFmfDiBVB2GAaMhT+8eExT9G8VVVTzysq9/Pv7PRSWVwMQ18WX2y/qzdUDY/BUEBIRkVagANRMCkC/sedbePXq2qboWTDstka9rbiimtd+3MfCb3eTX+YMQrGhPtx+YW/+MKgbXh4KQiIi0nIUgJpJAeg4Vj0Hn09zNkVP+BDiz2/0W0sra3jjp328+O1uDpVUARAT7MOtF/bij8ndsHpYWqloERFxJwpAzaQAdBwOB3zwv7D+bfDtArcsP2FT9ImUV9l4c3UGC1bs4mBxJQCRgd7cemEvxg2JxdtTQUhERE6fAlAzKQCdQFWZsyk6Zz1EJTqboj19mnyYimobb/+cyfPLd5FTVAFAeICV/72gFzcM7Y6Pl4KQiIg0nQJQMykAnURBhnOm6PIj4BcOPYZBjxHQfRhE9Adz48NLZY2Nd3/Zz/PLd7kWVg3z9+KW83vyp3N64Otl6ETlIiLSwSgANZMC0Cns+c65VljFbxZDtQZB9xToMdwZiqKSwMPrlIerqrHz/tr9zP9mJ/vznUEo1M+Lm8/ryf8M64G/VUFIREROTQGomRSAGqG6Ag6sgYwfYN8PkLkaqkoa7uPhA92SawPRcOg2BLz8TnxIm50Pfj3As9/sZN9h50Kswb6e/PnceCYMjyPQ27M1P5GIiHRwCkDNpAB0Gmw1zt6gjFXOQLTvB+dlsqOZPZyjQq7LZueAT8gxh6qx2floXRbzv97J7kOlAAR6e3DjufFMHh5PkK+CkIiIHEsBqJkUgFqA3Q6HtsO+lfWhqOjAb3YyQfhZtSNEw6D7cAiMcr1qszv4ZH0Wz3y9k515ztGlAKsHk0bEceOIeEL8Tn15TURE3IcCUDMpALUCh8PZQL3vh/rLZod3HrtfSLxzdKjHMGcwConH7oDPNuYwL20H23KLAfDzsjBheBx/PjeeLv7WNv4wIiLSHikANZMCUBspyasNRKucI0U5G4Hf/Dr6R7p6iOyxw/jiUCjzvt7F5uwiAHw8LfzPsB7cfF5PugYoCImIuDMFoGZSADJIeYGzmbpuhOjAWrBXN9zHOxhH93PY4Z3Ii/vC+TC3KzV44O1p5oahPfjfC3oSEehtSPkiImIsBaBmUgBqJ6rLYf8v9SNEmauhuqzBLjaLDxvNfUkr683Pjr5sMp/B74f05v9d2IuooKZP0igiIh2XAlAzKQC1U7ZqyF7fsLG6oqDBLlUOCxscPVnj6Id373O5eORVxERGHf94IiLSqSgANZMCUAdht8PBra5A5Nj3A6bi7Ia7OEzk+PQm4IzzCeh7vrOfyD/coIJFRKQ1KQA1kwJQB+VwQP5e2PcDeZu+wb53JZE1Wcfu16W3c+mOurvNgnuAydTm5YqISMtSAGomBaDOY93mrXyX9jEBuT8x1LyNvqZMzKbf/MpHnA3Jk2HAWPDWz1tEpKNSAGomBaDO59eMfJ75eie/bN1Nsnk7KeatXOq/m7iq7ZjsNc6dvPxhwB9hyE0QOcDYgkVEpMkUgJpJAajz2rC/kHlf7+DLzbkABJlK+EvYGsZUL6NLxb76HbsNgeSboP/V4Km7yUREOgIFoGZSAOr8NmUVMv/rnXy2Mad2i4Nh5s2Mt6QxyvIzHtgAsFmDMQ8cj2nITdCll3EFi4jIKSkANZMCkPvIPFLGT3uOsGbfEdbsy2d7bgldKWCsZTnXe3xNN9Mh174ZwUMpS5hI7DnX4OerUSERkfZGAaiZFIDcV2FZNWsz81m7L5+1ew8RuH851zq+5CJzuqt5OtcRzNc+ozjQayx9+pzJ4B4hxAT7YNKdZCIihlIAaiYFIKlTY7OzNaeYrVs3ErjpDZKPfEIohQDYHCa+tg/idVsq2/ySGRTXhcE9QhncI4SzogLx8jAbXL2IiHtRAGomBSA5oZoq8te+h331S3Q5tNq1OcPelTdtF/OO7UKOEIjVw0xit2AGx4UwuHsIg3qEEOrnZWDhIiKdnwJQMykASaMc3Aa/vIQj/U1Mlc7V6avx5CuG8lLlxfzs6AvUXxbr2dWPwd1DGNzD+ejV1R+zWZfNRERaigJQMykASZNUlcHG9+CXlyBrrWtzYUBvvvG/kpdLUlh38Nh/zYJ8PBnUPbg2EIWSGBuEr5dHW1YuItKpKAA1kwKQnLasX+Hn/8CG/0JNuXObpx+V/f7A+shrWF4cxZp9+aRnFlBRbW/wVovZxFlRga4RosE9QogO1t1mIiKNpQDUTApA0mzlBbD+beeo0MGt9dtjBkPyjVT3u5oth2pYsy/f9cgurDjmMFFB3gzqEUJybSDqFxWIp0XN1SIix6MA1EwKQNJiHA7Y9wP88h/Y/BHYq53bvYMgaTwMngxdzwAgq6C8QSDanF2Ezd7wX09vT2dzdXKcMxAN6h5CsK+aq0VEQAGo2RSApFWUHIRfX4M1L0NBRv32uPOc64/1vQI86sNMWVUN6zILWZuRzy97j7A2o4DC8upjDts73J/B3UM4t08YF/cLVx+RiLgtBaBmUgCSVmW3w640Z6/Qjs/BUdsL5BcOgybA4IkQ3P04b3Ow+1AJa/bl88vefNZk5LP7YGmDfXy9LFx6VgRjkmI4t0+YLpeJiFtRAGomBSBpMwWZsPYVWPsqlDgXaMVkhj6XOhdj7X0xmC0nfPuR0ip+zchn9Z4jLN2YTeaRctdrIb6eXJEQxVWJMST3CNEt9yLS6SkANZMCkLQ5WzVs/dTZK7Tn2/rtwd1h8CQY+D/gH37SQzgcDn7NLOCj9Cw+WZ/FoZIq12vRQd6MTopmTGIM/aICtGyHiHRKCkDNpAAkhjq0A355GdLfgIoC5zazJ/Qb7ewV6jECThFgamx2Vu0+zJL0LD7fmENxZY3rtT7h/lyVGM1VSdH06OLXih9ERKRtKQA1kwKQtAvV5bDpA2ev0IFf6reH9YXkGyHxOvAJPuVhKqptLN+Wx5L0LNK25lFVUz//UFJsMGOSorkiIYrwAO9W+BAiIm1HAaiZFICk3cle55xTaP27UF3b+OzhAwOucfYKxQxq1GGKKqr5fGMOH63LYuXOQ9TdZW82wfBeYVyVFM2osyMJ9PZspQ8iItJ6FICaSQFI2q2KovoJFvM212+PSnKOCPW8CLr2PeUlMoCDxZV8uj6LJeuy+DWjwLXdy8PM7/qGMyYpmovODMfb88RN2CIi7YkCUDMpAEm753BA5k/Oy2ObPwRbfcMz/pHQ8wKIv8D5NajbKQ+XcbiMj9YdYEl6FjvySuoPZfVgZP9IxiRFM7xXFzx0W72ItGMKQM2kACQdSulhWL8YdnwJGaug5jdLanTpDT0vdD7izgWfkBMeyuFwsDWnmCXpWXy8LosDBfW31Yf5e3FlQjSjE6MZ1D1Yd5KJSLujANRMCkDSYVVXwP7VsHs57F7hXJ3eUd/0jMnsvFzW8wJnIIpNAc/jL7hqtztYk5HPR+lZfLohmyOl9aNMsaE+XJUYzZikGM6ICGjVjyQi0lgKQM2kACSdRnkB7FtZH4gObWv4usUK3c+pD0RRScedeLHaZuf7nYf4KD2LLzblUFplc712ZmQAY5JiGJ0YRbcQ39b8NCIiJ6UA1EwKQNJpFWU5J1rcvdz5KM5u+Lp3kHNtsrpLZl16H9NQXV5lI21rLkvSs1i+LY9qW/1/QpJ7hDAmKZrLB0TRxd/a2p9GRKQBBaBmUgASt+BwOCdd3L0c9qyAPd9BZWHDfQKia8NQbVN1YFSDlwvLqvlsYzZL0rP4cc9h6v5rYjGbOLd3GGOSorm0fyT+Vi3QKiKtTwGomRSAxC3ZapzzDe3+xhmIMn5seHcZOCdhrAtEcec6R4xq5RRW8Mn6LD5al8X6/fVByuphJvWsCMYkRnNB365YPXRbvYi0DgWgZlIAEsE5E3XGj/UjRFnpwFH/uTCZIWZw7e32F0LsUPBwXvbafbCEj9Zl8VF6FrsP1a9YH+jtwWVnRzEmKZqUnl2waIFWEWlBHS4APfvsszzxxBPk5OSQmJjIM888w9ChQ0+4/7vvvssDDzzA3r176dOnD48//jiXX3656/VJkybxyiuvNHjPyJEjWbZsWaPqUQASOY6yI7D3O2cz9e7lcGRXw9c9fKDHsPpAFJmAw2RiU1YRS9IP8PG6bHKK6m/RDw+wMjoxmqsSo0noFqTb6kWk2TpUAHr77beZMGECCxYsICUlhblz5/Luu++ybds2wsOPXf36hx9+4Pzzz2fWrFlceeWVvPnmmzz++OOsXbuWs88+G3AGoNzcXF5++WXX+6xWKyEhJ57/5GgKQCKNUJDpHBmqC0SleQ1f9wmB+PNdgcgeHM/qffksSc9i6YZsCsurXbvGdfHlqqQYrkqMpne4f9t+DhHpNDpUAEpJSWHIkCHMnz8fALvdTmxsLHfeeSdTp049Zv9x48ZRWlrKJ5984tp2zjnnkJSUxIIFCwBnACooKODDDz9sVA2VlZVUVla6vi8qKiI2NlYBSKSxHA44uLX+dvu930NVccN9gmJrm6kvpKr7eXybZWLJuiy+2pxLeXX9bfVnxwQyJjGGKxOjiAo6/hxFIiLH05QAZOitGVVVVaxZs4Zp06a5tpnNZlJTU1m1atVx37Nq1SqmTJnSYNvIkSOPCTvLly8nPDyckJAQfve73/HII4/QpUuX4x5z1qxZPPTQQ837MCLuzGSC8H7Oxzm3gq0asn6tD0SZP0FhJvz6Ovz6Ol5AavhZpPa8kIrrziOtrA/vbSrk2+0H2XigiI0Hinj0sy0MjQtlTFIMlw+IJNjXy+hPKSKdiKEjQFlZWcTExPDDDz8wbNgw1/b77ruPFStW8NNPPx3zHi8vL1555RWuv/5617bnnnuOhx56iNzcXAAWL16Mr68v8fHx7Nq1i7///e/4+/uzatUqLJZj70DRCJBIK6sqhX2rYM9yZyjK2dDwdbMHxCRT3u1cVtgTeXlfF37aW+B62dNi4oIzunJVUgyp/cLx9dJt9SJyrA4zAtRarrvuOtfzAQMGkJCQQK9evVi+fDkXX3zxMftbrVasVk3aJtJqvPygT6rzAVB6yDkh457a/qH8vZD5Iz6ZPzIKGBUQRUnKZXxtOocX90awMaeUr7bk8dWWPHy9LFxyVgRjkqI5r09XPLVAq4icBkMDUFhYGBaLxTVyUyc3N5fIyMjjvicyMrJJ+wP07NmTsLAwdu7cedwAJCJtzC8Mzv6D8wHOALR7hXMOoh1fQXE2/ute4ipe4irfMAoHjeRLUnh2TzR78qtYkp7FkvQsQnw9uXxAFGOSYkjuEYJZt9WLSCMZ+r9OXl5eDB48mLS0NNc2u91OWlpag0tiRxs2bFiD/QG+/PLLE+4PsH//fg4fPkxUVNQJ9xERA4XEweCJ8MdFcN8uuOEdSBoP3sFQdoigzW9w7ea7+NrxZ9YMeJ9H+x8gys9Mflk1b/yUwdgXVnHu418za+kWNmUV0g5m9xCRds7wu8DefvttJk6cyAsvvMDQoUOZO3cu77zzDlu3biUiIoIJEyYQExPDrFmzAOdt8BdccAGPPfYYV1xxBYsXL+bRRx913QZfUlLCQw89xDXXXENkZCS7du3ivvvuo7i4mA0bNjTqUpdugxdpJ2zVzrmHNi+BLZ9A2SHXSw6vAA5GX8RntiHMz4jjYGV9f1/vcH/GJEZzVVI0Pbr4GVG5iBigQ90GDzB//nzXRIhJSUnMmzePlJQUAC688ELi4uJYtGiRa/93332X//u//3NNhDh79mzXRIjl5eVcffXV/PrrrxQUFBAdHc2ll17KzJkziYiIaFQ9CkAi7ZDdBhmrasPQxw0WcnV4+pIbfh6f1gxh/oFe5NfU/49OUmwwY5KiuSIhivAAbyMqF5E20uECUHujACTSztntcOAXZxja/BEUZrheclisZIcN56PqZBZk96HA4ZxY0WyC4b3CuCopmlFnRxLo7WlU9SLSShSAmkkBSKQDcTggO70+DB21RIfD7EFWyFA+rErmPwfP4gjOf5+9PMz8rm84Y5KiuejMcLw9tUCrSGegANRMCkAiHZTDAXmbnUFo8xI4uKX+JZOZA0GDeL8imdcKBnAQ59I4/lYPRvaPZExSNMN7dcFDt9WLdFgKQM2kACTSSRzaUTsytARy1rs2OzCx338A71ck83ZJElmEARDm78WVCc7m6YGxwVqgVaSDUQBqJgUgkU7oyB5n8/TmJc7+oaPs9+3He+WDea9iMBkO580SsaE+XJUYzZikGM6ICDCiYhFpIgWgZlIAEunkCvc7b6vfvMR5Zxn1/xk8YO3Nf8sH8VH1EHY5YgA4MzKAMUkxjE6MoluIr0FFi8ipKAA1kwKQiBspzoWttWFo7/fgqF+ZPsuzO+9XJPNpzRC2OLoDJobEhXBVUgyXnx1JF38toSPSnigANZMCkIibKj0M25bClo9g1zdgr3a9lGOJ4sPKZJbahrLe0ROL2cx5fcIYkxTNJWdF4m/tlEsrinQoCkDNpAAkIpQXwPbPnWFo51dQU+F6Kc/clY+rnGForaMPVk8PUvtFMCYphvPPCMPqodvqRYygANRMCkAi0kBlCez4whmGtn8B1aWulw6ZQlhancxn9qGstp+Jn7eVywdEcdmAKAb3CNHIkEgbUgBqJgUgETmh6nLYmeYMQ9s+g8oi10v5BLKsZjDL7EP5wd4fm8mDflGBJPcIITkulOS4EKKCfAwsXqRzUwBqJgUgEWmUmkrYvQK2LIGtn0J5vuulajzItIeR4YhgnyO89msEFf7diYw7k8SeUST3COGMiAAsZs03JNISFICaSQFIRJrMVu28i2zLR85b7EvzTrp7jiOEfY4Iss2R2ILi8I/qTXR8f3r3HYBPUBhoEkaRJlMAaiYFIBFpFrsdig5A/h7nBIy1X21H9uA4vBuP6uKTvr3E5EexTzcIjSco+gx8I3pDaDyExENgDJi1XIfI8SgANZMCkIi0GofDeansyB5sh3dxKGMbRdnbMeXvJbB8P+EcOfnbzV4QGocpJL4+FNV9DekBHpqbSNyXAlAzKQCJiBEcDgcH8g6zbetGsvdspixnJz6lGXQnl+6mXLqZDuFpsp3kCCbnCFFoPITEHRuQfILb6JOIGEMBqJkUgESkvSgsr2ZtRj5r9uazZs9B8g7sItKWTQ9THt1NufQw5RJnziPOnIePo/zkB/MJaRiIjv7qH6lLa9LhKQA1kwKQiLRXVTV2NmUVsmZfPj/vPcKaffkcKqkCHHShiB6mXLqb8hgYkM8AnyN0N+USUnkAS9nBkx/Yw9s5atQgGPWEoG7gG+oMTxbPtviIIqdNAaiZFIBEpKNwOBzsPVzGL3uP8MvefH7Zd4RdB0uP2a+7n51Lo8s5J7iQs7wPE2HLxlKw19mkXbi/wRpoJ+QV4AxCPsG1X3/zqAtKv32oL0naiAJQMykAiUhHdqS0ijX78p2haF8+G/YXUmWzN9jHx9NCUmwwyXEhDOkewMCgYgJKM2vvWttbf/da0QGoKAKa8afC0/f4wehUIcpTk0ZK0ygANZMCkIh0JhXVNjYcKHReMtubzy/78iksr26wj8kEZ0bWzVrtnLk6Jrg2gNhtUFHovHutMY+yI86vFQXgsB9bUGN5eB8nJAWDzwlGmuoeXn6aR8lNKQA1kwKQiHRmdruDXQdL+Ln2ktmaffnsO1x2zH5RQd4MiAkiLMBKFz8vQv286OJ/1HM/L0L8vPC0nKB52m53LhVy3KBUAOVHThyi7DWn/wHNnse5JBcKfmHgHw5+4Q2f+4aCWQvYdgYKQM2kACQi7iavqKK2sTqfNfuOsDGrCJu9cX8egnw8jwpIXoT6WRt838XPetRrJwlMdRwOqCo5/qhSgwD12+B0BGxVTf/wJjP4hoFfV/Dv6gxF/uG139d+Pfq5msHbLQWgZlIAEhF3V1ZVQ3pGATsPlnC4pIojpc7HoZJK1/P8sioamZEaCPT2oIu/1TWKVBeMQv2shLme1wcnL49G3p7vcDgXqz3eyFLZYSg5CKUHncuUlNR+LTtCk/ubfEJqQ1H4UYGp61HbjgpP6mNqUwpAzaQAJCJyaja7g4IyZxg6XBuKDpdU1j+v/b4uMB0pPb3AFODtcdxLcKF+XoTVBqmjR5isHk24nGWrcYaj0jwoyXMGpLqvrud59eGpMXfLHc3Lv+Ho0TEjSnWBKQysgepdaiYFoGZSABIRaXl2u4OC8mqOlFa6RpUOlVZxpKTKua20yrX9cO0IU2Mvwx0twOpBqH/9KFJdr1KQjydBPp4E+ngQ6F333Pk1wNvj1Jfm7HZnY7crFOVB6aHjhKfabTUVTSvcw/s34SjsOJfjwiEwSmHpBBSAmkkBSETEeHa7g8LyateI0pHSSg4ddTnu6BGmun1OJzDV8fOyuAJRoHdtUKp9fnRYCvT2cH1ft83Py4Lp6EDicEBl8W9GlI669NYgPB2EqpMvkHsMTz9nEAqIci5/EhgFAdFHfY12BiY3a+5WAGomBSARkY7HbndQVFF91OW4Kg6XVjpHmMqqKCqvobC8mqKKaorKax8VNZRUNuOOs1oWs4lAb48GAco12uQKVMcGqLp9vRyVxxlROurSW12QKsl1jkI1hskC/hGnCEpRzmkDOgkFoGZSABIRcR81NjvFFfXhqLC8mqLymqOe136t28cVnpzbq23N/zPq42k57qW54402dbXaCHMcIdR+CL+KPEzF2VCcDUVZzkdxNhTnNL5fyTvo2NGj3wYl3y4dYq24pvz99mijmkRERNolD4uZkNo+oaZyOBxUVNuPE5aqKSz7TWg6KlzV7VNc4Rx9Kq+2UV5tI7eosknn97QEEOrXhVC/wc476AK8CI30oqufB1EeJUSZjhDmOExIzSH8qw/iVZaDqSi7PihVlTgnuawohINbTnwis+fxL7M12BbVoZY9UQASERE5TSaTCR8vCz5eFiICvZv8fpvdQUlFw0tzvx2JOvo1ZxO58/JeSWUN1TYHuUWVpwhOAbWPeLws5vo75iK8iPGpoYdnIdGWfCI5TBf7EYJqDuJXmYe1PBdzcTam0oNgr4aCDOfjZHzDjn+ZLTC6/rl3cLto4NYlsOPQJTAREWnvKqptx8zP5Ox7atgcXtcHVVrVxFv4AS8PMxG+Znr7lhBvLaaHRz5R5nzCOUKI7TCB1QfxrcjFqywXk62Ro1cePs4gNHgyjLiryTWdjC6BiYiIdHLenhaig32IDm7cZIsV1TZXOKqfcqDSFZqO/Oa18mobVTV2MovsZBZ58w3eQNcTHN1BMCV09yygj3cx8dZCulkKiDTl11+Cq8rDWl0INeVwZDfVFSUYOae2ApCIiIgb8Pa0EBPsU7/I7SmUVdUcNS/Tb4NS7chS7fNDJZUU1ARQUB3A+mrgBHf1W6kiwpRPJEc4N38ALTv+0zQKQCIiInIMXy8PfEM9iA31PeW+DoeDsipbIy/HBZJeGsUFXXq1wac4MQUgERERaRaTyYSf1QM/a+MDU3MmrWwJCkAiIiLSpkwmEx4WY+8Ea/+zGomIiIi0MAUgERERcTsKQCIiIuJ2FIBERETE7SgAiYiIiNtRABIRERG3owAkIiIibkcBSERERNyOApCIiIi4HQUgERERcTsKQCIiIuJ2FIBERETE7SgAiYiIiNvRavDH4XA4ACgqKjK4EhEREWmsur/bdX/HT0YB6DiKi4sBiI2NNbgSERERaari4mKCgoJOuo/J0ZiY5GbsdjtZWVkEBARgMpla9NhFRUXExsaSmZlJYGBgix5bmk4/j/ZFP4/2RT+P9kU/j1NzOBwUFxcTHR2N2XzyLh+NAB2H2WymW7durXqOwMBA/QK3I/p5tC/6ebQv+nm0L/p5nNypRn7qqAlaRERE3I4CkIiIiLgdBaA2ZrVamTFjBlar1ehSBP082hv9PNoX/TzaF/08WpaaoEVERMTtaARIRERE3I4CkIiIiLgdBSARERFxOwpAIiIi4nYUgNrQs88+S1xcHN7e3qSkpLB69WqjS3JLs2bNYsiQIQQEBBAeHs7VV1/Ntm3bjC5Laj322GOYTCbuvvtuo0txawcOHOBPf/oTXbp0wcfHhwEDBvDLL78YXZZbstlsPPDAA8THx+Pj40OvXr2YOXNmo9a7khNTAGojb7/9NlOmTGHGjBmsXbuWxMRERo4cSV5entGluZ0VK1Zw++238+OPP/Lll19SXV3NpZdeSmlpqdGlub2ff/6ZF154gYSEBKNLcWv5+fmMGDECT09PPvvsMzZv3syTTz5JSEiI0aW5pccff5znn3+e+fPns2XLFh5//HFmz57NM888Y3RpHZpug28jKSkpDBkyhPnz5wPO9cZiY2O58847mTp1qsHVubeDBw8SHh7OihUrOP/8840ux22VlJQwaNAgnnvuOR555BGSkpKYO3eu0WW5palTp7Jy5Uq+++47o0sR4MorryQiIoL//Oc/rm3XXHMNPj4+vP766wZW1rFpBKgNVFVVsWbNGlJTU13bzGYzqamprFq1ysDKBKCwsBCA0NBQgytxb7fffjtXXHFFg39PxBgfffQRycnJ/PGPfyQ8PJyBAweycOFCo8tyW8OHDyctLY3t27cDsG7dOr7//nsuu+wygyvr2LQYahs4dOgQNpuNiIiIBtsjIiLYunWrQVUJOEfi7r77bkaMGMHZZ59tdDlua/Hixaxdu5aff/7Z6FIE2L17N88//zxTpkzh73//Oz///DN33XUXXl5eTJw40ejy3M7UqVMpKirizDPPxGKxYLPZ+Oc//8n48eONLq1DUwASt3b77bezceNGvv/+e6NLcVuZmZn85S9/4csvv8Tb29vocgTn/xgkJyfz6KOPAjBw4EA2btzIggULFIAM8M477/DGG2/w5ptv0r9/f9LT07n77ruJjo7Wz6MZFIDaQFhYGBaLhdzc3Abbc3NziYyMNKgqueOOO/jkk0/49ttv6datm9HluK01a9aQl5fHoEGDXNtsNhvffvst8+fPp7KyEovFYmCF7icqKoqzzjqrwbZ+/frx3nvvGVSRe7v33nuZOnUq1113HQADBgxg3759zJo1SwGoGdQD1Aa8vLwYPHgwaWlprm12u520tDSGDRtmYGXuyeFwcMcdd/DBBx/w9ddfEx8fb3RJbu3iiy9mw4YNpKenux7JycmMHz+e9PR0hR8DjBgx4pipIbZv306PHj0Mqsi9lZWVYTY3/HNtsViw2+0GVdQ5aASojUyZMoWJEyeSnJzM0KFDmTt3LqWlpUyePNno0tzO7bffzptvvsmSJUsICAggJycHgKCgIHx8fAyuzv0EBAQc03/l5+dHly5d1JdlkL/+9a8MHz6cRx99lLFjx7J69WpefPFFXnzxRaNLc0ujR4/mn//8J927d6d///78+uuvPPXUU9x4441Gl9ah6Tb4NjR//nyeeOIJcnJySEpKYt68eaSkpBhdltsxmUzH3f7yyy8zadKkti1GjuvCCy/UbfAG++STT5g2bRo7duwgPj6eKVOmcPPNNxtdllsqLi7mgQce4IMPPiAvL4/o6Giuv/56pk+fjpeXl9HldVgKQCIiIuJ21AMkIiIibkcBSERERNyOApCIiIi4HQUgERERcTsKQCIiIuJ2FIBERETE7SgAiYiIiNtRABIRERG3owAkItIIy5cvx2QyUVBQYHQpItICFIBERETE7SgAiYiIiNtRABKRDsFutzNr1izi4+Px8fEhMTGR//73v0D95alPP/2UhIQEvL29Oeecc9i4cWODY7z33nv0798fq9VKXFwcTz75ZIPXKysruf/++4mNjcVqtdK7d2/+85//NNhnzZo1JCcn4+vry/Dhw9m2bVvrfnARaRUKQCLSIcyaNYtXX32VBQsWsGnTJv7617/ypz/9iRUrVrj2uffee3nyySf5+eef6dq1K6NHj6a6uhpwBpexY8dy3XXXsWHDBh588EEeeOABFi1a5Hr/hAkTeOutt5g3bx5btmzhhRdewN/fv0Ed//jHP3jyySf55Zdf8PDw4MYbb2yTzy8iLUurwYtIu1dZWUloaChfffUVw4YNc23/85//TFlZGbfccgsXXXQRixcvZty4cQAcOXKEbt26sWjRIsaOHcv48eM5ePAgX3zxhev99913H59++imbNm1i+/bt9O3bly+//JLU1NRjali+fDkXXXQRX331FRdffDEAS5cu5YorrqC8vBxvb+9W/qcgIi1JI0Ai0u7t3LmTsrIyLrnkEvz9/V2PV199lV27drn2OzochYaG0rdvX7Zs2QLAli1bGDFiRIPjjhgxgh07dmCz2UhPT8disXDBBRectJaEhATX86ioKADy8vKa/RlFpG15GF2AiMiplJSUAPDpp58SExPT4DWr1dogBJ0uHx+fRu3n6enpem4ymQBnf5KIdCwaARKRdu+ss87CarWSkZFB7969GzxiY2Nd+/3444+u5/n5+Wzfvp1+/foB0K9fP1auXNnguCtXruSMM87AYrEwYMAA7HZ7g54iEem8NAIkIu1eQEAA99xzD3/961+x2+2ce+65FBYWsnLlSgIDA+nRowcADz/8MF26dCEiIoJ//OMfhIWFcfXVVwPwt7/9jSFDhjBz5kzGjRvHqlWrmD9/Ps899xwAcXFxTJw4kRtvvJF58+aRmJjIvn37yMvLY+zYsUZ9dBFpJQpAItIhzJw5k65duzJr1ix2795NcHAwgwYN4u9//7vrEtRjjz3GX/7yF3bs2EFSUhIff/wxXl5eAAwaNIh33nmH6dOnM3PmTKKionj44YeZNGmS6xzPP/88f//737nttts4fPgw3bt35+9//7sRH1dEWpnuAhORDq/uDq38/HyCg4ONLkdEOgD1AImIiIjbUQASERERt6NLYCIiIuJ2NAIkIiIibkcBSERERNyOApCIiIi4HQUgERERcTsKQCIiIuJ2FIBERETE7SgAiYiIiNtRABIRERG38/8ByRxqQ9sYbrkAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "from matplotlib import pyplot as plt\n",
    "plt.plot(history_simple.history['loss'], label='training loss')\n",
    "plt.plot(history_simple.history['val_loss'], label='validation loss')\n",
    "plt.xlabel(\"epoch\"); plt.ylabel(\"MSE\")\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "57b3cfe7-afeb-4125-b117-4519f8da8394",
   "metadata": {},
   "source": [
    "## Next steps ☕️"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0f204e8-bb47-4838-ae83-ccc433e5e22e",
   "metadata": {},
   "source": [
    "Now that you know how to train a rnn you should check the documentation to see how you can run sweeps using wandb.\n",
    "\n",
    "Happy coding 😊"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}