Simple_regression_task.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "Simple_regression_task.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "QjlEFdXpMgvw"
      },
      "source": [
        "# Deep learning programming I-A: Regression\n",
        "\n",
        "## Introduction\n",
        "This programming exercise is the first of a series of exercises, which are intended as a supplement to the theoretical part of the Deep Learning course. The goal is to introduce you to basic tasks and applications of methods you have encountered in the lecture. After completing the exercise you should be familiar with the basic ideas and one, possibly simple, way of solving the respective task. It is worth mentioning that most of the tasks can be solved in many different, not necessarily deep learning based, ways and the solution presented here is just one of them.\n",
        "\n",
        "## Regression\n",
        "\n",
        "In this exercise we consider the problem of regression, where we are interested in modeling a functional dependence between different variables with, possibly noisy, observations of input-output pairs. Mathematically such a dependence can be formulated as\n",
        "\n",
        "$\\mathbf{y}=f(\\mathbf{x})+\\boldsymbol{\\epsilon}$,\n",
        "\n",
        "where $\\mathbf{y}\\in\\mathbb{R}^{M}$ and $\\mathbf{x}\\in\\mathbb{R}^{N}$ are the input and output observations, $f:\\mathbb{R}^{N}\\rightarrow\\mathbb{R}^{M}$ is the function mapping inputs to outputs and $\\boldsymbol{\\epsilon}\\in\\mathbb{R}^{M}$ is a random vector, which models noise in our observations. Note that this assumes additive noise that only acts on the output and not on the input variable, which might not be true in all practical applications but is a reasonable approximation. For regression we are now interested in estimating the functional relationship $f$ between the inputs and outputs. This can be done in many different ways, not just with neural networks, but for this exercise we focus on approximating this relationship with a neural network $g_{\\boldsymbol{\\theta}}:\\mathbb{R}^{N}\\rightarrow\\mathbb{R}^{M}$ with parameter vector $\\boldsymbol{\\theta}$. The task is now to choose the parameters of the neural network in a way that results in a \"good\" approximation of $f$ with $g_{\\boldsymbol{\\theta}}$.\n",
        "\n",
        "In order to quantify how \"good\" our neural network can approximate $f$, we adopt a probabilistic view. For this we make the assumption that the noise $\\boldsymbol{\\epsilon}$ is a random vector drawn from a known dustribution, which enables us to derive a suitable cost function for training our neural network and also for quantifying a \"good\" approximation.\n",
        "\n",
        "### Mathematical formulation\n",
        "If we assume that the noise $\\boldsymbol{\\epsilon}$ is drawn from a gaussian distribution, e.g. $\\boldsymbol{\\epsilon}\\sim\\mathcal{N}(\\mathbf{0},\\sigma^{2}\\mathbf{I})$, we can use\n",
        "\n",
        "$\\mathbf{y}=g_{\\boldsymbol{\\theta}}(\\mathbf{x})+\\boldsymbol{\\epsilon}\\Rightarrow \\mathbf{y}-g_{\\boldsymbol{\\theta}}(\\mathbf{x})=\\boldsymbol{\\epsilon}$\n",
        "\n",
        "to derive a log likelihood. Since the probability density function (pdf) of a multivariate normal distribution is given by\n",
        "\n",
        "$p(\\mathbf{x})=\\dfrac{1}{\\sqrt{(2\\pi)^{D}\\vert\\mathbf{C}\\vert}}\\mathrm{e}^{-\\dfrac{1}{2}(\\mathbf{x}-\\boldsymbol{\\mu})\\mathbf{C}^{-1}(\\mathbf{x}-\\boldsymbol{\\mu})^{T}}$,\n",
        "\n",
        "we get\n",
        "\n",
        "$\\ln{p(\\boldsymbol{\\epsilon})}=\\ln{\\dfrac{1}{\\sqrt{(2\\pi)^{M}\\sigma^{2}}}\\mathrm{e}^{-\\dfrac{1}{2\\sigma^{2}}\\Vert\\boldsymbol{\\epsilon}\\Vert_{2}^{2}}}=-\\dfrac{1}{2\\sigma^{2}}\\Vert\\boldsymbol{\\epsilon}\\Vert_{2}^{2}-\\dfrac{1}{2}\\ln{(2\\pi)^{M}\\sigma^{2}}$.\n",
        "\n",
        "Replacing $\\boldsymbol{\\epsilon}$ by $\\mathbf{y}-g_{\\boldsymbol{\\theta}}(\\mathbf{x})$ yields the log likelihood for one particular input-output pair:\n",
        "\n",
        "$\\mathcal{L}(\\mathbf{x},\\mathbf{y},\\boldsymbol{\\theta})=\\ln {p(\\mathbf{y}\\vert\\mathbf{x},\\boldsymbol{\\theta})}=-\\dfrac{1}{2\\sigma^{2}}\\Vert\\boldsymbol{\\mathbf{y}-g_{\\boldsymbol{\\theta}}(\\mathbf{x})}\\Vert_{2}^{2}-\\dfrac{1}{2}\\ln{(2\\pi)^{M}\\sigma^{2}}$\n",
        "\n",
        "This log likelihood measures how likely the input-output pair is and we can use it to train our neural network. For this we maximize the expected log likelihood over all input-output pairs under the assumption that the noise is idependent and identically distributed (i.i.d.) over all input-output pairs. This corresponds to finding the parameters $\\boldsymbol{\\theta}^{\\star}$ of our neural network, which maximize the the expected probability for observing the corresponding input-output pairs. Mathematically the optimal parameters for our neural network are given by\n",
        "\n",
        "$\\boldsymbol{\\theta}^{\\star}=\\arg\\max_{\\boldsymbol{\\theta}}\\mathbb{E}\\left[\\mathcal{L}(\\mathbf{x},\\mathbf{y},\\boldsymbol{\\theta})\\right]=\\arg\\max_{\\boldsymbol{\\theta}}\\mathbb{E}\\left[-\\Vert\\boldsymbol{\\mathbf{y}-g_{\\boldsymbol{\\theta}}(\\mathbf{x})}\\Vert_{2}^{2}\\right]\\approx\\arg\\min_{\\boldsymbol{\\theta}}\\dfrac{1}{N_{D}}\\sum_{i=1}^{N_{D}}\\Vert\\boldsymbol{\\mathbf{y}_{i}-g_{\\boldsymbol{\\theta}}(\\mathbf{x}_{i})}\\Vert_{2}^{2}$,\n",
        "\n",
        "where all terms, which are independent of $\\boldsymbol{\\theta}$, are ignored and the expectation operator is approximated by the mean over all $N_{D}$ input-output pairs. In other words we are maximizing the log likelihood by minimizing the mean squared error loss over all input-output pairs in our dataset, hence this approach is called Maximum Likelihood (ML) estimation. For solving this optimization problem and obtaining the optimal network parameters, stochastic gradient descent (SGD) or one of it's many variants is typically used.\n",
        "\n",
        "It is worth noting, that choosing different distributions for the noise $\\boldsymbol{\\epsilon}$ leads to different log likelihoods and therefore different cost functions for training the neural network. Another commonly used distribution for modelling the noise in regression tasks is the laplace distribution. Deriving the log likelihood and the corresponding costfunction leads to the mean absolute error, which is given by the $l_{1}$-norm of the difference between observations predictions of the neural network. This cost function is considered more robust against outliers since these have less influence on the averall loss compared to the mean squared error.\n",
        "\n",
        "###  Implementation\n",
        "\n",
        "In the following we consider a simple regression task, implement a neural network and train it based on the mathematical fomrulation above. For this we first need to create a set of input-output pairs, which then needs to be partitioned into a training, validation and test set. We also define some constants to be used for partitioning the data and the hyperparameters for our neural network.\n",
        "\n",
        "But before we can start, we need to import the necessary packages tensorflow, numpy and matplotlib."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "CPuVp2lyNK2J",
        "colab": {}
      },
      "source": [
        "import tensorflow as tf\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "J84v9uucMgv5"
      },
      "source": [
        "Next we define our constants and set the random seeds of tensorflow and numpy in order to get reproducable results."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "xwC-1OnHMgv7",
        "colab": {}
      },
      "source": [
        "N_train_samples = 600\n",
        "N_validation_samples = 100\n",
        "N_test_samples = 100\n",
        "N_samples = N_train_samples + N_validation_samples + N_test_samples\n",
        "noise_sig = 0.1\n",
        "N_epochs = 150\n",
        "batch_size = 8\n",
        "learning_rate = 0.01\n",
        "\n",
        "tf.random.set_seed(0)\n",
        "np.random.seed(0)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ngFFSyG-MgwA"
      },
      "source": [
        "We create $600$ training samples, $100$ validation samples to optimize our hyperparameters and $100$ test samples, which are used to check if our model can generalize to unseen data. Furthermore we set the level of noise added to the observations. For training the model we plan to train it for $150$ epochs with a batch size of $8$ and a learning rate of $0.01$. Next we create the actual input-output pairs $\\mathbf{x},\\mathbf{y}$ for which we want to learn the regression model and plot them. In this simple example we choose scalar inputs as well as output but in general $\\mathbf{x}$ and $\\mathbf{y}$ can be vectors."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "4s8AtsdQMgwB",
        "outputId": "8c2a68b3-e1a4-48b2-e460-934e59bf04d5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        }
      },
      "source": [
        "x = np.linspace(0.0, 3.0, N_samples, dtype=np.float32)\n",
        "y = np.expand_dims(np.sin(1.0+x*x) + noise_sig*np.random.randn(N_samples).astype(np.float32), axis=-1)\n",
        "y_true = np.sin(1.0+x*x)\n",
        "\n",
        "plt.plot(x, y)\n",
        "plt.plot(x, y_true)\n",
        "plt.legend([\"Observation\", \"Ground truth\"])\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOydd3xUZfaHn/femUkl1NBLKEF6700UEBQRVpG1rQV7X9ddf1Zsu/ZdXfta1t51EZQmvUlP6C10EnpJLzNz7/v7Y0qmhQRIMpPkfT4fZebe9957ksx877nnPe85QkqJQqFQKKo/WrgNUCgUCkXloARfoVAoaghK8BUKhaKGoARfoVAoaghK8BUKhaKGYAm3ASXRoEEDmZSUFG4zFAqFokqxbt26E1LKxFD7Ilbwk5KSWLt2bbjNUCgUiiqFEGJ/SftUSEehUChqCErwFQqFooagBF+hUChqCErwFQqFooagBF+hUChqCErwFQqFooagBF+hUChqCErwFWcku9CBYaoS2oqqz57jufy+60S4zQgrSvBrOL9uPMS7i3aF3FdgN+j2zG/8Y8a2SrZKoSh/Lv7nYq77aFW4zQgrSvCrEFJKft14qFw97vu+TuWV2TtC7suzOwGYtj6j3K6nUEQau4/n8v3ag+E2o1JQgl+FmJqawX1fp/LJ8r2Vcj3T3Q1NiEq5nEJRYWw5lFXiviveWsYjP26kJnT/U4JfhTiRWwTA4azCoH35dif5bo/8XAj1YXcYnm1K8RVVm7FvLitxX57dAKDIaVaWOWFDCX4VQnO72qEcka7P/EaXp+ec87kLHEbQNof7C6A8fEVNoMAe/B2obijBr4JIghXfMCVlDe1/vmIfx3OK/LblFAY/HTgMt+CftYUKReRSUugmlNNT3VCCX4Xw9fA3Z2TxzeoDZ32OXcdymTJtC/d/k+K3PZTgex5xNeXiK6oRxaFKF7rm+nznKw9fEQ42Z2Sxdt+poO0e3ZVScvlby3jsf5vO+tx2t4hn5jv8tucUOoLGej38EHovpeTb1QcorAFekaJ64flce/AIvgrpKCoEh2FyLLuQQofBR0v3cOBkvt/+y99axsT3VwQd5/G0K2Id1PGcoqBHXfsZPPyZm47w6P828daCtPI3RqGoQAIF3+r18M896aGqELEdr6ojBXaDfLuT3n+f57f9sxX7WPrIxaUe7xFk8wzpY1JKXvttB9f0bUmLerHB+/GkWvqL+B1frGPK5Z2YPKS1d1vgo68vh7MKAMgrqv5ekaJ6MXfrUf7240YaJUTRq2Vdb5bO09O3MOOBoV6PvzqiPPxKQkpJxymzue3z4LaNWfnB4ZRQ2N2eia8Mn86ze8UXIO1YLu8s3M0dX6wrwQ7Xv4Lgyav/paYD8MPag5zOs58xpJNb5PKGakWfnc9gd5ocCZFWqlBUFm/Mcz2VHs0uYtbmI97t24/kkHrgdLjMqhRqrOAX2A2cRsXk3f57XhrP/bLV+/5IViGtH5sJQOqBzKDxZY3QeDzueVuPerf1e2EeA19c4DPG9TNtO5zNocwCAvEKvgjO58/Md7D/ZB5/+3EjPZ+f6520DSX4eW7Bj49yCX6hw2D5rhOlLl55fOomBrw4X8X+FRVOTqEjZFw+I8T3woPDkHyxcj8nc4tKHFOVqbaCP219BmlHc0rc33HKbO7+KqXE/efD6/N28l+f1bBpx0q2A4pF+HBWAYNfWlDiOI8AH/NJqQwMuzzy40bv60EhzuUwi+Pyv2054rcv/XSB3/kW7zzmHRuIx8OPcwv+t6sPcP1Hq5iaeuYyDPO3uW5W+XaDMW8sYfKna844XqE4V7o+8xtdn5nD3hN5ZT5mx5Fsnvp5Mw99v6ECLQsf1VbwH/x2PaNeX3LGMXN9PGVfjmUXkn46P+S+isDjFU9NzfDzPgJr5tjLsBJwy6Fsv/dJj87winOHp2bxweI93n3/CyHOu47lel97vKNQEU1PGqdVd+31xD2X7zp5Rvssuusj5zRMth/JYcH2Y2ccr1CcD05TctFri8o83hPPP5WnPPwqw/nWxOj3wnyGvLzwnI71zQDw2CFKWbrksdYI8NYDF4KURfBDcSizANOUFDpMZru9+k0ZWWxMD64vcjrf7n1teMM/ghkbD/s9Hnts+V9KBseyC71xn1CLwnzxZETYSwinZWQWcDrPHnKfQlFWAjNxyoonU6e6ltUpF8EXQvxXCHFMCLG5hP1CCPGmEGKXEGKjEKJXeVy3JEoSk8ogq6B4ArbQUbbSBEVOk+xCB5kF/pO3gfFHu3FucW8pXZ5OWVjo43Fnu+3ZeyKPe79O4blfi+clPOdbtfcUj/y0EcP9OzdLuY7Hwy/p5jX4pQUhQ1EKxdlw4hxj8O8s3F3OlkQW5eXhfwqMOcP+S4Fk9393AO+V03WDyMp3cO0HK884piKr4vmGYX5Yd5D2T8zyTnCe6Zhuz/zGx8v8q2AWOgymrc/g1Tnb3e/P7UYmkWUuqfybT5jr4Cn/sNahzAJyi5zM2nTYLxRjmNJ7AzCl6/ebdjQnpPhbSvHwoWYscVdULCdzz+8pccuhbD5auqf0gVWMcsnDl1IuEUIknWHIeOBz6VLalUKIOkKIJlLKw+Vx/UBSQmTC+OIrfpszsujSrHa5XdvXk54ybQsA2w6fedK2JFIOnOah79ZjSri0SxN+Skk/p/M4nBKn6RFYSQL5xFNAnCikFvnYhBNDapgInOhkEcdpWYu9J0x8fYLFO48z/NVF9E2q63f+xgnR3p975qbDtE2M5/V5O7nzwjY8dmlHv7EWd8y/yOfmtWjHMS5snxi0NkChOFfO5FDoGCSLDOLJZ79szHHqhBz39xnbuG1om4oyMSxU1sKrZoBvh4F09zY/wRdC3IHrCYCWLVue04WirMEPLUezC4mLsvDlyv0MS06kTWKcd9/lby3j/Rt6MaZLk3O63u+7T3A0u5A/9GwOBMfhwZW1cy48+O16PzvLig0HkzsYHNy5nmQtnZNffEhiXC6LbAdoIk4RJcqW929IwRHqsc9szF7ZmB2yBSl5yRjOvgHjip8gnKb0/rzLd53gy5X7uWFAK+9Yi+YO6fh8IW/+ZA1vXduTcd2blvlnVCjOhCNEyDCOAu6y/MIN+jzqiuLkhNXmBbzunMgKs3Pp5zVMTufbaVgrulztrSwiaqWtlPID4AOAPn36nFPcJcriL/hSSvq/MJ+W9WI5cCqff83dSepTo/zGpB7IZHTnxmQVOKgTazur6133oatl2h96Nsc0pXfxUmURSyFdxV66abvpru2mozhAkjiCvk+CzSXahwvrc6CgPkdkG2abfTkua5NLLLkyhjyiKcKKhomGxIJBbfKoJ3KoI3JoJk7QWhzhcm0lN4j5ABTtj2a1tR3zzV7MM3thmE1xhrjRbc7I5smMzQxNbkCr+q6brDWEhw9nzo1WKM6WwHTlDuIA71tfJ0k7ykyjH3OMPmRSi05iP9dZ5vON7R+85xzHy85r8M1Lk1L6PXk+8uNGpqZmkPaPS7HqVS/npbIEPwNo4fO+uXtbuRMYFvBEWA6449F2pxk0gWlKyaKdx7nlkzU8N770u3xJzNx82LuKr6JoLo4zQNtKX7GD7tpukkU6unD9PAfNRDbLJH41B9CyfU8+2GZlj2xCESXfxJ66vBPP+0zGAjwwIpk35wf+HJJmnKCXlsbIWvvpbKzjGevnPMPnHNzbjrSi8dShPZnUCrqGwzBdq4mFz6RtwAS0apSuKE98s3S6ij18ZXuBfKK4umgKa2QH777FdOd7/TJeiPmGuwt/oTa5PO68DY/oOwyJzVKsKTM2uYIShimx6pXzs5QnlSX404H7hBDfAv2BrIqK3wdSHLt2IUSwuDhNSbr7hrDhYMmt0AIJnJQM5eWeLx6BH6BtY4C2lebiBACnZDzrzXbMNvuy3mzLRrMtp0jwHvdgk2S2bS395jO6c6Mgwbf61BLRhOemKcggkQwzkV+yBgHX0kocYaS2jivMFVy895+sirLwizmI/zgvJ002955DSuj+3G/YdI1erVzx0kAP/8DJfBZsD70uQqE4Wzwhw2Yc5zPbS2TJOP5of4pDNAgaa1hiGP1/3/DOUzdyr2U66TKRd40JABQ5DWwWja2HsunQuJY3h7qqpm2Wi+ALIb4BhgMNhBDpwNOAFUBK+T4wE7gM2AXkA7eUx3XLQqE9oBSqEEE3gSKn6a2FfTZPaW8v3OX3vnaM9dyM9KEkgT8pa7HK7MjH8nKWGx1Jk82QZ0iy8pQ8KA2bRWPtkyPZfzKfq977HQCHz42sTWK832IsX/bLxnxsjOVjYywdxAGu0RcwSV/MxKglzDV68U/nJLbLlt7MfLtheoutBU6qfbf2IN/VkEbSiorhwyV7GH5BIu0axnPnF+uIws77ttexYPAnx6MhxR7ApmsgBK86/0gzcYK/Wn5gnXkBq2RH7E6TnUdzuOzNpdx/cTvvOpMzFTCMZMorS+faUvZL4N7yuNbZMvCl+X7vNU0wff0hv22FDsNH8Iu926RHZ/DZ5H5c2D4x5LlnbvJ/SDmXsERpAv+BOZaVZievwMfadPKdpactxkaV7XkzyqJTO8bqt9jJt8ZQ87oxJQq+L9tlS55x3swbzqu4UZ/LZMssZtoe4ydjKFlHiifgN2W4nqDKUkDN7jQ5cCqfdg3jy/SzKGouDsPkHzO38daCNGb9eRgAf7N8R1dtH7faH2afLDkpozgWL3jMcRvdbbv5p+09Rhe9jN0wvetRXLWiXCONmiz4kUxgFxtdCP4+Y5vftiKH6c39Dpzs+Tk1I6TgL9xxjO1HitMtv19zsEyVI88UollpdgoS+EDK2n2qrB6+Z5Lb4vNo06lpcWio7llOYmdSizeNK/nUuIR7LdO4WZ+DmDaKm/RJfGGMwnT/TC/O2l7quZ75ZQtfrzrA6idGVNmsCEXl4Pn+FjlNdhzJpofYxWR9Np87RzHf7O0dp2siyDGz+SR6FBDNw467+V/UM9xjmYbdeSlRFpfzlFXg8D6tyira77zaC34gRSG8Y5eH73TvD/5L2p0mr87ZzuQhrWlSOwaAWz7xL/r1yE8b+fc1PYKObcZxr7gP0LbRQjsO+Av8KrMjO2XzM4ZozpamdWLKNM7mFnpP9gzA5d2act/XqQDUifUPUzWIt3GiDItasonnRef1fGGM4h+W//Ks9TMm6Mv5q+NOdstmpR4/e/NhVu1x1eXJyncowVeckUK3Y2fTNTJzCnjR+iFHqMsrzj/6jYux6uQWOakfZ6PA/WQfa/N/Gk6R7fnJGMpt+kwOndqLI8r1ec0udHoXbVZVD7/q5RWdJ6GiLgU+IZ2igFWeU1MzaP/kLD5cupeBLy6gyGn4lU/wZdmOQ3QTu7lFn8Vb1jdZFvUAy6Mf5J+29xltXc/B6GSedtzE6KKX6F30Pvc4/sznxmh2yJZ8fuuAMtlfkn9vCWja0LBWlPf1K1d1K/F8mvu4wBSzK3u6PuRNa/vfOGwhJjkeHJEctK1DY1e2TrpsyE2O/+NB+z20Ekf4xfYkV+uLKK0o9F1fpnjDayqBR1EaHg8/p8hJ4p7/0VE7yHOOG8nFvwlQtHudzqB2DbyLqprUDnaOXnZcgxOdLZ//hc9/3wf4e/g1OoZf1fl990naN3LFiUurw5N2NJeJ7/+OwKSlOEYnsZ/u2m56aWl027qH6CjXzeCQrEeKmcxH5mU888BdJCR25IV3lrM5OzvonG9d25MmtUv2YK26OGP3KYAYm+7XiNw3RNOsbunefuAN47Wru/PyRNeNol6cjYd/cJWLDeXZ3H9xO/4dkMZ5QeNaPiEvwTRzCCuKOvOG9R1etX7AYG0zjzluo4CSf25P+EqlbCpKwyP40RTRfttbpLgz2ALxhGdirJo3G61Rgss5mvvQMG+F3WPU5WPjUu7Tp/HGhtVAc7/6T6XVjIpUqqWH/9TlnWiccHYhgJ1HXROTgemCAAnk0kPs4o/6Qhote5IvxBQ2Rd3G4qi/8J7t39yiz8aCwZfGSO6xP8CAwrcYVPQ29zke5FNjDDTqDJpWYipXt+a1vR/EQJrVieHDG/t43ycmFHvuqx4fwWtXdwcgOiAp2KoJfrlvCPP+MqzElm23Dy1uZ2gNWLCmaQKrrmHVNa7qXZxiGep+aAnw+v/xhy7eOvm+HKMuNzge55+OiYzTVvCj7VmaUHI55eIevlXzy6WoPDyFBm/W59CIU7zkuBYQvHRlV24cWLzS2+PhR7tDOwCN3FqR3KgWS/52EZMHu74XnzjHUICNey3Tgq5XRfW+enr4tw5pTcNaUdz/TWqZxkdhp5E4TRNO0S+3kJ76AVqLw7TWjtBaHKa+KJ6cde6KZy/N+NEYxlbZiq1mK9Jk8zMubvJQ0odESoiyFYvmX0a1JyHawtTUDKbdN4Rth4ufCr68tb+3mmSjhGi6NHNNsLZvFM9xn8Youibo2txVI+ho9gm/613TtwUWXfDE2E7ebVatbPf+soivVdeICbEqZWhyA5amneAt40o2yda8ZX2baVFPcbv9L2yQ7YLGe0xSHr6iNAocBrEUcqflVxYYPVgtXTWcJvVpwXuLiytgxtg8Hr7OVvf36oLGxYsFW9aP9aZeniaBL42R3KbP5HUxkQOykXdcVXVCqp/g2/Nhw9e0TT/J7fo+rBhYMLAKJ7XIp7bII8H7bx4NRBb1fOpqkA1Y4aisw17ZhDlGH/bKJuyRTdglm/H4xEu588uy3UgA/m9M8aq+kqp0GlL6efgPuGPiN7s9Dd9yEYGTsR0aJ/DJLX1Jqh/HqbwirnpvBVBcswagRwv/4lDPT+gSFLO36GXL/vGI728PDeOSMzSYCbUmYWLv5ixNc918Fpk9udL+LB9bX+Vb29+50/EQS8zufuM98yrnWttcUXModBj8UV9IXZHLW84/eLdrmvB7wvV8z2wWjXHdmrI07QT9kur5nauZz3fsY+dlTNZn8yd9Lv9w3uDdXlWdkOon+I58mPEwnYBOvpojdDLNaLJkHNnEki3j2EUzVpsdOCzrc5S6HJb1iK7bnJUno8kjdNz7bMQe4JbBSd7XgXrfq2UdUg5kEm3Vg2oA+VJaauRFFzQEoHWD4qJwuo+Ax0VZ6OATUw9VA8QTw+/VMnTlQA8PjUzmmV+2+l0rkAK7EVLwA8NOabI5f7A/x+e2l/jI+hoPOu5jltnfu3/PcVdrunNt/KKoGTgNk7yCQm61zGKV2YFU6Z9EoPukMntSp3MKnUzq24Kr+zQPKsdyy+DW9E2qx5TpW9hwEGabfZmkL+KfzqspxBVSraIOfjUU/Jh68PBOFu7K5L7vNuHAwuvX9mVs92b0eHRG6cefuUPfWeMrrr7doNZPGYVV10g5cJpmdWK83n9S/digcwSmRn5yc19v/DGQxy/rwAsztwfdQDzx8IdGtg95nBCCGQ8MoUW94OsD/Hr/EISAzk1re588SiLfbtC0TvAcSqgwz0lqc639ST62vcrb1jf5q+MupppD/cYUleLhH8osYHNGFpd0bnzGcYrqSbsnZjFeW8Y42wmmOG4O2u/r4bdNjGfRjuPePrehSnLrmqB7izp8f+cAxryxlM9OXsK4qJWM13/nO+MiQKVlRg6aBrUaYbfVJo8YmtavzdjurhTDafcODhr+35v7BG0rT3w/bL5PgZomiIuyMDTZtahLCMHnk/vx/V0Dg84R+KG8qEPDEksJ3zGsLfteGltiyObCC0KvGgaXmCdEhy4P0aVZbTo3LVvfgHy7M6SHH2MLPTGdTRw32h9lhdmJ16zvM1bzb2BzyydrOJJVyI/r0nnq5+CmahPeWc4dX6wrk22K6ojkNstM0sxmLDSL18J4Ehp8v4O3DW1Nk9rR3HVh21LPGmXR6dWyLmvlBWwzW3K9Ps+7r6rG8Kuf4LvxeMztGxVPyHRrHixYDeKLs158s1YqAt8PSaiVesPaJ5a4wOjxyzrwzLhOIfeVhcpKcRzYpj5/GtgqdEjHZ57i/ov9J2kLiOZ2x8Osk+15w/oOIzR/Af9+7UH++sMGvli5P+i8x9yT1VU1VU5xfnQTe+iq7eMz4xLv4sVeLesw0Z1d5llrcn3/ljSpHcOKx0YwsG39Mp3bldUj+N64kG7aXpKFq/y5aUqKnAY7jpxbc6NwUW0F3/Pd9y1FEOrxzTfMcEHjhKD9vqx+fIQ3K+Zc8Iht+0bxJMScXTTtjmFtSw2lnAmPl1NRgj95cGtuH9qab+4YQMNa0bQMERqK9mlOE6pERAHRTLb/jS2yFe9a/00/UVwC419zS28iU1UfsxXnx3X6fPJlFNOM4id436dJ/TycHc+803RjEE6pcaW+FHDpyxNTNzP6jSXn3D83HFRjwXf9cUvLNvTNjqkfZztjw/GGCdFM7NXcb9uVvVzhIt9sHIA5fx7GL/cN8dtWP841+frZ5H6V3s6vLHV+zocp4zr5pXnW93ly8uA7aeuZJE6sFcUFPk9hucRyk/1R0mUiH9j+RRtxKOg8JVFVMycUZafIafgV9zPyM7lCX8F0YyA5PqtqY6zFn3f9PNJ7PU7KSWqzyOzOBH05Gian8uys2XcKwG/BY6RTjQXf9W9pwurbEjHKqpVadCw+IMbtEXFNwN8ndPFub5QQ5c2D9/Dva3ry/Z0DQy7lrmhemdiNBy5uR59WdUsfXEZ6tKjDZV1Lnij93z2DmP3n4glYX8H3PGZP7N2cR8Zc4HdcFvHc7HgEJzqfWF+hHsGrk0OhBL/6c8GTs7nq/RXe9+aG74kVRXxtjPAb5+vhe8OZ5/AE6BuG/J8xlCbiFAO1LVz74UpvmZOqFM+vtoLvieGXVl3SN5slyqKXLvgBZYc9k6cjOjbkhgGtvMdrIVa3tqgXS7/W9YK2VwYNa0Xzl0suCGnXufLzvYN59/reJe7v1bIuHXzCZH6P2T4hplB/o4OyEbfZ/0ojcZqPbK8RRekF2wI7mSmqJxsOZrpeSImW8imbzSQ2Sv9m4zE+jpwnYeFc5nh8nZT5Zi+yZSxX6q7+0p7PbRXS++or+H3diymu63fmZug2P8HX/IqOeWjTIM47qx9r878hdGteh30vjaVdQ1dYol7c2ZUTrkn43lz94qpuvR/crr5fWup62Y4/O+6ll7aLZy2fereXuIBNCX6N4c35adz28n/Rj2/hW+MiLu7QyG+/7/fUI8zn6xAUYeM3sw+jtHVYcXo/tyV9HiORaiv4TevEsO+lsaXOxvtWf7RZNFq6m23H+XijC/46nEcvdcXoAxcPBfLVbf159orOJaY31mR8C7RpPh6+p7LmH/u29Hr+b1/XE4DZZj/eck7gGssi/qgvBIJ7FngI7GSmqL78a+5O+ufOQ2pWfjEGMqqTv+D7fk+Lq66evTB7irLde1FbGsRHMcvoS4LIZ5C2xXsjKa2wYSRRbQW/rPgW/oqyaLRwV5b0lE4NxHaGFbHgCtvcNCip3OyrDvxy3xAeHtXebz7FsxDYlJImtV035yu6N6XQXbyue/PiFb+vOyeyxOjKc5ZP6Sr2lCjsysOvOWiYjNd/50TT4WQRH1S22zf77nyydDyCH2PVOZFbxDKzKzkyhjHaam8MvyqV/qjxgu+LzaJ50wnTTxeEHOMblvj2jrLVsK/pdG1em/sDauaXlCbqaUTjG+830XjQcS/Hqc17tjdw5oReDl0RTeQVkckgbQsNRSaf57pKcQRWe/VtalL8NHn21/F83z1ZZ0XYWGD2ZLS+Bp2qV+up+pVWKCOfTe4X1EzEphcL/oFTeTw5tiMJAQuI2jeqxU0DW3HjoCTaJqpeq+eKVsJj9lvX9uL9xbuD6gedJoG77X/mJ9vTbPjoFuL/9A0dA1b+Kg+/5vAHfRnZMpYPjrgW8AV6+L49nT3h1QbxZz+/dteFbYmx6lzduzknc4t47bedzDL6MV7/ne7GFrbTptQeGpFEjfXwW9WLZVhAr9ooq06vVnUZ2KY+T13eiduGtmFSnxZ+Y3RN8Oz4LkrszxNPPD/QKx+S3IAvb+sfsob/gegLeM05ib4Fy/n5k5eD9s/cfDhom6L6sPu4q6ptNEWM0dcww+jvLUvu++Q9eXBrrvApPTKgTT1emdiNpy4/+5Xq0VadOy9si0XXvAsfF5ndKZA2ehcsB1QMv0oQSlBsuka0VeebOwbQrfmZq0Yqzo/mdV1PUm0bln7j9KwdiLPpfGiMZbnRmQfsH1F4ZCdr3YtfAF6ZvcP7OunRGfxjxtZytloRLkxTMuKfiwEYqaUQRyHTzOKVtb61o6aM60Qtn6QJIQST+rQI2ZTnbPAkchQSxXKzM/2d6wDptxAs0lGCD94wjrWMNeEV58/gdg344a6B3FHC5LgvnonyGJuORONhx104sLD93T9yzftL/cb6llL+cOne8jVaETZ8wyaX6qs4Jeqy2vTpNVFKj+TywDfpYKHZk1baMdqKQ1Uqhl/jBN+j874pgj/eNZBPb+lb6eUOajp9k+qVaSGYJ5PKk1t9hPo85riNHtoe7tGn+41dsP1Y+RuqCDuFPj1rL9I2sDp6EKaPfNWKtnJtv5Yh19GUJ55S5YsMVyXO4dp67CqkE7l4PHtfD79hQjTD3U1EFJHD45d14JWJ3bzNpn0zL2aZ/ZluDOQ+y1Tai4Pe7Xd9uU5VzaxmvDhzG1NTMwC4UNtArChiRXRxnarFfxtOjxZ1ePHKrqx+YmSF2vL55H4AZJDIDrM5F2nr+X7NwSrzmatxgu9ZLGEpYw9XRflTWvkKD3cMa8ukPi288VlPDLZFvRieuKwjzzhuIodYXrF+gEbxY7VDLcCqVvxnyR6e/cU1H3OpvppTMp6t1q7e/a3ql9x9rbzRAsI6/bTtpO46yPQNZS/yF05qnOp5PHtR437yyOD3Ry9m2f9ddFbHeGqheMJwTWrHUDvGyikSeNZxEz203UzWZ3nH5xcZ5WewIqz4zsnYcDBCS2WO0RcjTNLlJ/hGD2zCYIi2mT9/tz4s9pwtNU72WrizQ1S0Pjw0rRNDnVJ69AYypF0DADILHICryXRtdyx1ujmQuUYv/mr5nlbiCAA9n59bjhYrwsllbxZPyg/RNlFLFDDb7Bc2e3wDA+tkMtkyluGaS+yP5RQy8b3fOZZdGIL8lvQAACAASURBVCbrSqfGCf4Xt/bj39f08EvbUkQ2f+zbgp/uHsQUdx71tf1aUse7IE7wpGMyDiw8Y/kMKiFbQ1F57DqW6319qbaaLBnL72ZnAF6+qiv/+VPJ1VorAt+G6E4sLDW7MFzfAEi+XHmAtftP8+WqA5Vq09lQ4wS/YUI043s0C7cZirNACEHvVnXp0qw2+14aS7/W9Whcu7gV5FHq8YbzKi7SNzBaW+t3bFWqZKgoGQtORunrmGf2xoEFiavY3uhKblwfmMm3zOxKE3GKNuIwRe5MIms5liAvb2qc4CuqB4EtFD81RrPNbMFT1i+IofiRusipJnCrA320ndQRefxm9AmrHYEOxHLT1fRokLaF/yzZA/gXZIw0ItcyheIMCCGYes8g73sDnSmOW2guTnCvZZp3e1Wqc6IomRFaCkXSwlLTlZ0Trge3wOzLA7Ih6bIBQ7TN3m2RvICzXARfCDFGCLFDCLFLCPFoiP03CyGOCyHWu/+7rTyuq6jZdGzi31B+jezAT8ZQ7tB/9fbCLXIUC/7KPSc5lVd65yxFZOCb236xlspKsxP5uEJ54QrUBdfUFywzujBQ2+JNDbZU55COEEIH3gEuBToB1wohQlUp+k5K2cP930fne12FIlQzmhcd11GIjcctXwHFHr6Ukms+WMl1H66sVBsV505OkatUdmtxmLbaYeaZvXjxyq6lHFWxeAT/gka1vNt+N7tQW+TTWewDqn9Ipx+wS0q5R0ppB74FxpfDeRWKMnPL4CQATlCbd53jGamnMlDb4p1I85RO3n4kJ1wmKs6SbHca7ggtBYAFRk9qx4Q3u87j4Oua8K4n8WQNecI6kVymuzwEvxlw0Od9untbIFcJITYKIX4UQrQIsR8hxB1CiLVCiLXHjx8vB9MU1Z3HL+vAjQNbcc/wdt5tnxhjSJcNeNLyJXaHSzRUg/OqR5ZX8FPZZrYgg8Ti1qFhCuK3axhP9+a1eX5CZ5rXjWXtkyO5bkQftpktGOQW/EguplZZzx6/AElSym7AXOCzUIOklB9IKftIKfskJiaGGqJQ+HHHsLY8N76LXwOMImy87LiGztp+Yrf9wH+X7aXL03PCaKXiXMgucJBALn217cw3ewH+jU3CQbRVZ9p9Q+jdqh4ADeKjSKofy+9mF/pqO4jCHtGJAuUh+BmAr8fe3L3Ni5TypJSyyP32I6ByV0soqj1Wi/9E2S/mQFLNdjRe+yqv/pqiPPwqSFaBg+HaRizCZL7hEnxPaYNI+mtadI3fzU5ECwfdxW6+XLE/3CaVSHkI/hogWQjRWghhA64B/GrWCiGa+Ly9AthWDtdVKLxYgybKBM87bsBWcIw7Lb+GxSbF+ZFV4OBiPYUTMoENsi1QXBIlktbTWTTBGvMCTCnop23nUFYhpyM0G+y8BV9K6QTuA+bgEvLvpZRbhBDPCSGucA97QAixRQixAXgAuPl8r6tQ+BIqFS5Ftudoi0u5XZ9BfbLCYJXifDiVm89wbQNG21F+te8jDYsmyCaeHbIF/bTtABw4lR9mq0JTLr9FKeVMKWV7KWVbKeU/3NumSCmnu18/JqXsLKXsLqW8SEq5vTyuq1B4EELwylXdgravTLqbKBzcY5ke4ihFJGGYku/XHMRpmGw9lM3iub9SR+RR2HpUuE07I55qrqvMDvTWdmLByf7qLPgKRSTQo2VxH+Kk+rHomuDBubn8ZAzjBn0uTTkRRusUpfHtmgM88tNGPv19HxvSMxmur8chdfTkEd4xkdiUztNbY7XZgThRRGexj8z8ahrSUSgiBd9MHauukRDtapjypvMPANxvmRoWuxRlwxP3PpVnR0oYpm1knWxPs0aJ3DSwFY+MucBb5bZtYuU1PSkNT4+NNe4eu/21bX51/CMJJfiKaoPV4v9xjrK4UviO64342hjB1fpiksRhwLVsf//JPFVNMwLJKXTy9YK1dNb2s8TohhCCZ8d34Z7h7WjdII4vbu3HC2FecRuK49Rht9mEftr2iC3apwRfUW3w9fCFKG5Y/+71vXjHOQE7Vh6y/ETKgdO0eXwmF766iG9WHyzhbIpw8cXK/bTLWQPAEjNY2IcmJ3ob2kcCvj7DarMD/bQd3gV/kYYSfEW1webj4QuEt3Z5nVgrJ6jNJ8Zoxuu/c2J3qnfc6r0nK91ORekM1TdyUtZii0wKtyml4ltQbbXZgQSRT0J2WhgtKhkl+IpqQ1RASMfTji7BXX/lP87LyZExtN/xnneMoSI6EYPnBi0wGaZtYpnZFVkFJMr3I7TaHcdvkpkSHmNKIfJ/mwpFGfFdfDWuexPaNIgHINbmiuVnE8/nxihaHZlLW+FaDG6qFbgRRwdxkESRxRIjOM02EvH18DNI5DANaJmTeoYjwocSfEW1QfdZfHXvRe1485qevHt9L5rXLe6O9ZHzMhxaNPdZfgbAaUbm5FpNZpi2EcDb7CTSsWr+Mrpe60TjrA2RtRzYjRJ8RbVECEHtWCuXdW3it/00CaxN/ANXaL+TJA4TwXWuahyep62h2ka2mS04Rt0wW1Q2BrWtz93D23rfLy9qSwNOs2JdKjmFkTV5qwRfUSPwje8/sH8oDizcq08jtyiyvpA1GbthEkMhfbUdrNK689ilHXj3+l7hNqtUNE3w4Ihk7/sU0/X6m//9yJg3lobLrJAowVfUCDxxfHA1SfnaGMEf9GVomQfCaJXCF7vTpL+2nSjh5ESjodx5YdugJ7RIxRNOFAJ2yBbkymh6azvJyCzghZnbSHp0RpgtdKEEX1EjCMzb/o/zctB0Ls/+hv0n8yhyGqzeeypM1ikAipwmw7SNFEorO22dw23OWWHRBJP6NOeb2wdgoLPebEtvzZWa+cGSPWG2rhgl+IpqR3LD+KBtMTb/xhmDe3VlfeI4JupL+HTWcl6cuZ1J/1nB9iPZlWWmIoAip8lQbROrzI7kyfC2MjxbhBC8MrE7A9rUB2CdbE9HsZ9YCr1jIqH1oRJ8RbViy7Oj+fWBIUHbYwME/5JOjWkz/nE0JBdl/sQOd6/bk7mRWfSqJhBbcJhkLYMlZteIrUVTFp4b35kUsz26kHTXdnu3R0LrQyX4impFXJTFW0PHl9uHtvF7H23VqNcsmaVRQ+l7chpb9rhi+ZHghdVU2mSvAmCJ2b1KC37fpHqkmq4ey73FTu/2SOi6pgRfUSMY170p+14aS51YV6ggxuq6KSyufy0xsoAb9HkAGBGYO12d+XFdOm/Od8W6L8hdwzHqs5tm3HtRu1KOjFysukY2cewwm9NLKy6x4FQevkJRuXj0PNot+DTpxmKjG7dYZhOFnZT9pzkYoc0rqiN//WED/5q7E0yDDgUpbIzqxZ4XL+eSzo3Dbdo540kBXmcm00tLQ+ASekcE1PFQgq+okXgmcds2jOd9YxyJIos/6Mt4a8Euhr6ykK9WRW4j6urItpTFxJs5bIqO/Lz70vCU+EiR7akj8mjjLsmtYvgKRSXjqX8f7Y7z929djxVmJzaarbldn4Hm9saemLrZe0wkPIpXd2ZO/QoTwY7YPuE25byxeT389gD01lxxfKfy8BWKysXzlYu2uT767RvVAgT/cY6jrXaYUdpav/FLdh6n3ROz2JieWbmG1jCG6RvZpbfDHlU1yimcCY/g75WNyaQWvYUrju+IgLpNSvAVNQu34gdm8swy+7HfbMjdll/wLXi7eOdxALUoqwKpRT49xS7mFHVGi8CetWeLVff8EILNWgevh69COgpFJXPHMFd6ZmBevonGh8ZYemi76Se2e7d7lsxHQkpddWWQtgWLMFlqdOVIdmHpB0Q4vp3XtuodaKcdog45KqSjUFQ2949IZt9LY/1q53v40RjGKRnPZMts7zaP4Kv8/IpjqLaRXBlNikymwG6E25zzxtPIBWCN4Uov7antUh6+QhFJFBLF0oRxXKKtpYU4yj9mbEUXSvArCtcEumSYtpEVZmecWCh0hF8Uy4PJg1vz+eR+LMtvgVNq9NTSIuIpUQm+QuHDwoQrMNC4Wf+ND5fuVSGdCsRumLQSR2mpHWex6epulW93htmq8mHKuE4Ma59IAdFsky3pJdJwRMDqYSX4ihpPnE88/7TegF/NAUzSFxFPPha34KtWiOXLlkNZvLtwd1B3qwJH1Q/pBJJqJtND243DGf6bmRJ8RY3n1weGAlA7xorDMPmv81JqiQIm6YvRlIdfIYx9cxn/np/GMG0T+82GXDdmOAAjOjYKr2EVQIqZTLwoJOr0ztIHVzBK8BU1ntYN4tjx9zGseWIkDsNkk2zDGrM9N+uzsQrXY7gRATnU1Q0rTgZqW1hqdkXXBCseu5h/TeoebrPKnRTp6oAVfzz8jc2V4CsUuPLybRbNW+/kY+dltNSOY26bCaB631YAvUQa8aKQJe74fZPaMSErnVZ1DsiGnJS1SDihBF+hiCicbk9+rtmbdNmAnoe/AeCHtQcBSDuaw7/npXlLNCjOnaH6RpxSY4VZtbpbnQ1tE+MAQYqZTJ2T68NtjhJ8hcIXh9Ml5AY6nzhH01/bTmexl5wiJ6YpuezNpbw+byc5ReGfgKvqDNM2kiKTySHWL3e9OjHzwaGsnzKKVDOZhLy95Jw+FlZ7ykXwhRBjhBA7hBC7hBCPhtgfJYT4zr1/lRAiqTyuq1CUN76LY743LiJXRjPZMguAfIfhDflkFzjCYl91QEpJPbLpIvax1OgabnMqlCiLTp1YG6nStQArK21FWO05b8EXQujAO8ClQCfgWiFEp4BhtwKnpZTtgNeBl8/3ugpFReBb4CqHWH4wLmSctoJETrPvRJ53X5YS/HPGMCVDtM1oQnrj99WdDWZbDClYsXhOWO0oDw+/H7BLSrlHSmkHvgXGB4wZD3zmfv0jMEJU12c4RZXGE9Lx8KkxGpswuE5fwMb0LO92JfjnjsOQDNU2clrG07jDAACquxgIWxzbZUsaZW8Mqx3lIfjNgIM+79Pd20KOkVI6gSygfjlcW6EoVwLrneyXjVlodOd6y3xOZ+d6t2cXqBj+ueIwDIbqmzjdaBBN6saH25xK4dPJ/Ug129FD24U0Dfr8fR7vLdpd+oHlTERN2goh7hBCrBVCrD1+/Hi4zVHUQEIVuPrMuISGIpOGGXO921QM/9wxj2ylsTjN4QYDvdlO1f15P8aqk2ImkyAKsB/ZxoncIl6evb30A8uZ8hD8DKCFz/vm7m0hxwghLEBt4GTgiaSUH0gp+0gp+yQmJpaDaQrF2fHq1cELfxab3dlvNqTX0R+826pLzZdwoO1dCMDxhkPCbEnlEW3VvQuw8vesDJsd5SH4a4BkIURrIYQNuAaYHjBmOnCT+/VEYIFUicyKCGR058Zc0b2p3zaJxufGKNoWbKKT2AdAUQQUwqqq2PYuYKfZDEd8E67t3xJdE4zqVP1KKvgSa9PZJxtzSsZjHlgVNjvOW/DdMfn7gDnANuB7KeUWIcRzQogr3MM+BuoLIXYBfwGCUjcVikghlCcyU7+YAmnjT7orrLMxPYvMfHvlGlYNKMjLwZqxkiVmN2wWjQ6NE9j9wmU0rxsbbtMqlBirDghSzWSijqSEzY5yieFLKWdKKdtLKdtKKf/h3jZFSjnd/bpQSnm1lLKdlLKflHJPeVxXoagIGtaKCtrWuHETphqDmaAvpza5zNh0mPHvLA+DdVWbz7/5Ct20s8TshkWLqCnECiXGXZE1xUwmPnsXCeSVckTFUHN+4wpFGfnb6AuCtsVHWfjCuIQYYedqfTEA+0/mV7ZpVZ4Gx5ZRKK2sMjv69H6t/kS5G5t7FmD10HaFxQ4l+ApFANHW4AJeQgi2yVasMjtwk2UeGq4YvtOd1WOakoOn1A2gNAaa61lldqQIG1ZLzZEfIQSzHhzqXYDVUyjBVygiFk+OwefOS2ghjnKhtgGAdk/MIqfQwTsLdzH0lYV+q3EVAWQepKnzoHd1rS1EX+HqjM2ikUcMO2ULemlpYbGhZv3GFYrzZI7Zh2PU5Sb9N++2zHwHv+92ZRlnZBaEy7SIp3CHa8Lb087Q002spuAN65jt6KntQlD5mV5K8BWKM3Bh+0T+b0wHPEnETiz8KEYxXN9AkjgMQJ7diUVXnbHORIHdIGXBjxyS9dglXQvxa1JIB1wePrgaoiSIfNqIw2QXVu4Cvpr1G1cozpLPJvfj7uFtkT7JmlPFKOxS50/6PAByC53eZueq921o/vLtWroUprLE6Ianck5NC+lE6cWZOgC9tDTW7jtVqTbUrN+4QlFGvr69Pw+OSPa+9xTRfO/6XpwSdZll9udqfTGxFHLgVD7rD2YCkHYsh4+X7Q2HyRHL/pN5HNu2nASRz2KzeCWzpQZl6QBEWV1yu1c2JlPG0UukUeio3LCOpVKvplBUEQa1bcCgtg287z0efkKMFSHgM+cljI/6nQn6cv7yfbR33AszXfVRrunbgrgo9fUCOHAqn2H6RgwpWO7T3Soh2hpGqyofzxONRPPG8XdUcu9M5eErFGeBcP8/RSaz2UziRv03Qq3NzVMdsbzomuBCbSPrZTuyKa6OWTfWFkarKh/NZ5I6xUymvUhHFmZXrg2VejWFoorirfwkIMamAYLPjEvooB2kvwiueqhaIBYT7ciim9jjjt8X41l9WhNJlcloQvLzr9MxKnHeRwm+QlEGerWqC0DDWtHE2VyhmunGIE7LeG60BHcxUh5+MbUPLatR3a3KwnqzLaYUdJM72XM8t/QDygkl+ApFGXh4VHtm/3ko7RrGe2PzRdj4zhjOaG0tjQOqfecqwfdSK2MxmTKODbJtuE2JGHKJZadsTk8tjezCyvusKMFXKMqARXdVdgRXqVsPXxqj0JBcZ5nvNz63Er/EEY2U1D60lGVmV0wfufn1/ppTC78kPBO37y3YWWnXVIKvUJwlbROLJx7TZSLzzV5cqy/ARvEimjzVIMXFkY1EFRxjodHDb3OXZrXDZFB4WfX4CO/rFJlMHZHH3p0bKu36SvAVirPkscs6cGH74o5snxmXkCiyuVQrbmyhet7Cgu1H+eqLDzClYJEZ3EmsJtIooTiF17MAq6e2i5xKWnGrBF+hOEuiLDrjexR3xVpudma32YSbLMX1dY7lFIbDtIhi8qdr6ZS7ko2yDScp9uhvGZwUPqMiiD2yCVkyll4irdJKbSvBVyjOAd8Gna4WiJfQS9tFV+Hq7XM0uyhMlkUO9cimu9jNAqOn3/anx3Uu4YiahURjvdmOnloahQ6jUq6pBF+hOAc8eu9ZS/OTMZRcGe318o9mKw9/uLYeTUgWmMXx+1DNZWoaa58cyWp3LD/FTOYCkY6zMJtDmQUUOStW+JXgKxTnQe0YV3mAXGIp6jSJK/QV9G9okplfuVUQI42Dp/K5WE/lmKzDFpkEwOonRnDvRe3Ca1gE0CA+iobuWH6qbIcmJNFH1zPopQU8+M36Cr22EnyF4hzwNETxCL6uCepfdC82HEzSF+E0JSP/tZg7Pl8bTjPDQtrRHC56ZS7DtI0sMHog3TJT06pjloX1pusGGHPU1dh87rajFXo99RdQKM4BT0jHI/iGKaFhB2g9jOE505GGg13Hcvlta8V+gSORbUdy6KPtJEEUsNAsjt9blOAHkU0cO81m1DqRCnhqNVUc6i+gUJwDnkqPvjn5APS7k/rOY/S1rw6DVZGBaUou0lKxS50US3E6Zk1qWn42pJrJ1Du9EZCICv4VKcFXKM6B0Z0b8crEbsEZJ+3HcMrSiPH2GeExLAIwTMkILZVVZkeuHdLJu92qKbkJRYpMJtqRSWtxBFHBPr76CygU54AQgkl9WpAQE1DzXrewot54+pgbaSsywmNcGMm3O0nfu5V22iEWmj2xWTRv7F6rYT1sS+PvE7rw9LhOxQuwRFqFx3SU4CsU54FwP4Mn1orybkupfzlF0uptdG6aEruz8htWh4MHvllPZup0AOabPbHqGjMfHMpLV3YNs2WRxw0DWnFF96bskk3JljH00tJUDF+hiHR+e2gYsx4c6n1fYKvHL+ZArtKXUIt8npq2mfZPzvJm9lRX9p/MY+GOY4zW17LDbM5+2RinKWnXMJ5r+rUMt3kRidWieRdg9dJ2Vfj1lOArFOdJ+0a1aBBf7OFr7haIcaKI2xJW8tWqAwAUVNJqynBgmpILX11EgplFX7GdOWYfgEpbQVpV8YS7UmUyF4gDxImKXbCnBF+hKGekhE2yDalmOy4vnIHAFc7JKqiei7EyMguYufkwACP1FHQh+c1wCX6BXQn+mbB6BN9shy4kXdldoddTgq9QlDOejnWfOS+hrXaYwdoWADalZ1VLARz31jLu+9qVR36JtpZ02YDNsjUAhRVcKqCqo7snslPcC7B6irQKvZ4SfIWi3HEp/kyzP8dlgnfy9o4v1vHn71LDaViFcCrPDkAshQzTNjHX6I0n3WRs16ZnOFLhIZt4dplN6a4EX6GoWnjmZu1Y+ca4mBFaCs3FcQDmbDmKWYlNqyuTYdpGooSD39zx++3Pj2Fg2/phtqrqkGq2ozs7/UuxljNK8BWKcsb0+cJ+7RyBieBPenGt/K9XHwiHWRXOJfpaTsl4VpsdgOJwhaJspMhk6okcOLWnwq5xXoIvhKgnhJgrhEhz/1u3hHGGEGK9+7/p53NNhSLS8XXQjlCf2WY/rtUXEkcBAIezCsJkWcVhwckILYX5Ri8MXD1/9YquE1BN+OSWvtgsmncBVlbaigq71vl6+I8C86WUycB89/tQFEgpe7j/u+I8r6lQRDSBD+QfOi8jQeQzSV8EVE8hHKBto7bI94ZzQK2sLSsXXdCQ9o3iSZPNyZEx/DLj5wq71vkK/njgM/frz4AJ53k+haLKYwbEYDfIdqw2L2CyPhsdI6QQ/rgunamp6ZVlYrlzmbaSXBnNErNbuE2pkrSqH4eJxgazDT0qcOL2fAW/kZTysPv1EaBRCeOihRBrhRArhRAl3hSEEHe4x609fvz4eZqmUISJEHNuHzsvo4V2nNHaGkwJSY/O4MVZ27z7//rDBh76bkMlGll+WHByqb6GeWYvirCF25wqiaf0RIpMpoM4AEW5FXKdUgVfCDFPCLE5xH/jfcdJ17rxkqaXW0kp+wDXAW8IIdqGGiSl/EBK2UdK2ScxMfFsfxaFIiII9SWYa/Zmn9mI2y0zycxz9bv9z+KKm5yrDAxTIqVkiLaZuiKXX42B1I21htusKkmtaCsXd2jIGrMDFmHCwVUVcp1SBV9KOVJK2SXEf9OAo0KIJgDuf4+VcI4M9797gEVAz1DjFIrqQGBIB8BE42PjUnpqu7AdXhMGq8qfOz5fy8h/LeZyfSXZMpYlZjfevb53uM2qshQ6DNaZ7XFKDfYvr5BrnG9IZzpwk/v1TcC0wAFCiLpCiCj36wbAYGDreV5XoYhYSkqz/9EYRo6IZ+iJ70o9x+VvLeWLFfvK1a7yZv72Y6QfP80l2hrmGH2wY8VmURO150rdWBv5RLtWKe+LTMF/CRglhEgDRrrfI4ToI4T4yD2mI7BWCLEBWAi8JKVUgq+otpRUFbOAaBbEjWWocxUtxVFirHqJ59ickc1T07ZUlIlnzd1fruOBb4JXCQ/TNpIgCvjVHAgU14ZRnD0D2tQD4C3nBMzBf66Qa5zXX0dKeVJKOUJKmewO/Zxyb18rpbzN/fp3KWVXKWV3978fl4fhCkWk4ls5M5DVDSfiRGOyPosCh8GqPScr0bJzZ9bmI0zfcCho++X6Sk7JeJabrs5fSvDPnRsGtAJgvtmb9w4nV8g11F9HoShnHr20Ay9f1ZW+ScHrEC21m/KLOYhJ+mJqk8sLs7aT9Kh/O8SqUDff7jSJoZCR2jpmG/1w4ur8pfrWnjtCCDo2SQDgt61HK+QaSvAVinIm2qrzx74tMQKC+f1b16NeXBQfOMcSK4q4WZ/DhoOZQcdXhVI7U1PTGaOtIU4U8bMx2LvdqBmNvSqMInf/AFsF3TiV4CsUFYThI9x/GtCK7+4cSEKMhR2yJXON3tximU1y7eDjnGZkq+aK3Sf5v582cZW+hANmImvkBd59TepEh9Gyqk+mu2dCRYXGlOArFBWEpyrm1HsG8fyELgDERblCH+84x1NH5HGtPjfoOKcRuS5+gd3g2g9X0oSTDNK28j9zKNItI20T40iIVnn458O9F7nq4msVVH5DCb5CUUF48vEtWvHXrJZb8I/U6sIyozMTCqYShd3vOGcEx3S2Hs4C4A/6UjQh+cko7uXr+3Mqzo1bh7TmwvaJZBdWTHc09RdSKCoIj2776mB8tEvw68XZeMeYQD2ZydX6Yr/jAmP/kcTJXDsgmagvYZXZgYOyuJqKKodcPrx0VVc+vqlvhZxbCb5CUUF4Qjq+j+eekI6mwQqzE+vMZO6y/IIFp3dMJMfw008X0Euk0UY7wo/GML99FpWhUy40qR1DYq2SU3vPByX4CkUFYbhDOr6er809GefSdMHbzgk0FyeYoBevrIwkD7/AbviFF/acyOU6ywJyZTQzjf70S6rn3ac8/MhHCb5CUUGE8vBjba7VtS3rxQKw0OzBJjOJJ+N/xYITKWXQpO2ny/fS47nfCAcXvbaIbs8UXzvj0GEu11bwszGYPGJ4YmxHbhroWjBkUYIf8SjBVygqiJGdXPHtenHFJYPbJMbz7vW9eOVqT914wT+dV1OnKINJ+mIchgyatH3ml61k5jtwlJDkfvFri3htzo4K+RmOZBf6ve95aibRwsFXxkgAYmw6ozs3BoonbUd2bEjvViGb3ynCjBJ8haKC+L8xHVj9xAg/wQe4rGsTv/TFRWYPjiR0537LVF6fvREjIIbv8ZxzCp2EYs+JPN5euKucrQ+FZJxzNgdiu7BNurx6q67hcN+gPDH8j27qy093D6oEexRnixJ8haKC0DVBw1plWYgkWNfuPpqIU9hXfIjd6e/he4qs5VRQql5ZGahtpTWHWd/4Ku82qy7o06ouXZolDlOxLQAAF9VJREFU8OilHcJonaIsKMFXKMLEI2OKV6ger9+XpUYX7rZM59iJE37jot1x/zX7TgedoyLr7hS6l/l7uFmfw2kZz75Go7zbbLpGXJSFX+8fSuemIZYNKyIKJfgKRZi4Z3g772td13jNOYkGIpvolA/8xkVbXV/Tv/4Q3AKxyFm+KZxbD2VT5DSYsfEwHZ6a7d3eWhxmlLaOL4yRREXHerer6phVC0u4DVAoFGDVBBtkO2YbfRmy9xMS6cpx6gBnXsEa6IWfD3O2HOHOL9bROCE6aLL2dn0GDix87hzNg7biOv5WixL8qoT6aykUYeShke2Z0KOp11N+0XktNhw8bPneO6ZBfPGk7+zNRwA4ml1IocMoVw9/1zFX4+xAsW9AFlfpS/nJGMoJahNjK/YTVTnkqoUSfIUijDw4Mpk3runp9ZT3y8Z8aoxhkr6YzmIfAPXjildd3vXlOgD6vzCfP328qlw9/JK40TIHK04+NMYC+HXqsqr6OVUK9ddSKCIAm08s/G3nBE4Tz1PWL7jq3eUUBIj6pnRXAbM1+05T6KjYMgy1yOdGfS5zzd7slU2A4sVjAJpabFWlUIKvUEQAUT6x8GzieN05kQHaNpqkz2LfyTy/sePeXuZ9XdEe/q2WmdQRebzpvNK7LcZWci9eRWSjBF+hiAACs12+MS5mk5nEFOsXRDtzSzyurDH8rAIHX686cFZpnHXI4VZ9FjOMfmyRSd7tsUrwqyxK8BWKCMAWkO1ioPOY4zbqk8Udjs9LPC7PHnr1bSCP/rSRx6duYlNGFlJK0k/nY5qS3KKSj7/T8itxFPK6c6Lf9libzsK/DuejG/uU6dqKyEEJvkIRAQQKPsBm2YZPjDFcZf5GH7E95HFr9p4CIKl+bMj9Ho66M2/sTpP/Lt/HkJcXcu/XKXR5eg5Z+cEreJuLY0zWZzPVHMwu2dxvX4zNQusGcd5aQYqqgxJ8hSICCExvHNvNNUH6L+fVZMgGvGr9D7EUBh23+7gr3BNrC15SY3ea7DyaAxQ3YxFCMGeLK7VzljvFM6sgWPCftHyFE41XHNcE7Yu1qpBOVUUJvkIRAfhO2k69ZxAJ7s5Y+UTzF/vdtBLHmGIJDu14snQCa+jnFjlp/+QsLnl9Cafz7N7YvSbgdF5gS0XXOTxjhmibGKOv4R3nBI5Sj0DUpG3VpUqttHU4HKSnp1NYGOzpKCqf6OhomjdvjtWqGlefLzbdJaJWXdCzZV2+WLnfu2+V7Mj7xjjusUxnodmTOWZx+7sipytLxyPa3Z/9jcHt6jO2a1PvGIdpej18U8LpfH/B96R9OgxJDIU8b/kv+82GfGxcGtLWKLW6tspSpQQ/PT2dWrVqkZSUhKigru6KsiGl5OTJk6Snp9O6detwm1PlsVpcn2dPEs2Sncf99r/unMgQbRMvWz9gm70lB9y9ZAvcHr6nhn5WgYOZm45w48Ak77GFdpNNGa7cfYdhcjogZl/oMJBSciizgEcs39FaO8q19icowr+sswf13au6VKlbdWFhIfXr11cfuAhACEH9+vXV01Y54W196Fb858d38dvvwMJ9jgcQAj6w/ssbzz+RUwQQ1CXLt8vWop3His9jmEHhnwK7ydsLdpGeOodbLHP4xDmaFWZnAO6/uB2K6kOVEnxQ3kUkof4W5YentIJHiru1qBM05oBsxFOWh0kW6bxhfQcdg4zMAiA4hu/06Y6VV1S8OCtU16wCh8HitRt40/o2u80mvOwsnqj1Tdv/fHI/plze6ax/NkXkUOUEX6Gojng8fI/A2kooO5xi7clzzhu5RF/Hi5aP8NwinKbpJ+YOnxtAblFxCMceYqFWUWEeT+W/RCyF3OV4iEJctXv+2KcF0n3+izs0ZFj7RCYPUeG7qowS/HMgPT2d8ePHk5ycTNu2bXnwwQex2+18+umn3HfffeE2j59//pmtW7d630+ZMoV58+aF0SJFaXgEvn2jeACirP5fzX6tXdkyUsJnxmj+7bySSZbFPGv5FIGJ05R+cX9fD/9YdpH3tT0g9GPBSa+Vf6Yru3jYcTdpPjn3L13V1TvZq3rUVg+U4J8lUkquvPJKJkyYQFpaGjt37iQ3N5cnnniiQq7ndJZtJaUvgYL/3HPPMXLkyPI0S1HOaJrgq9v68/XtA4AQmTBu4fWEbl53XsV/nGO5yTKXt6xvoRtF3PrZWu9wh4+wH84qnmfx9fCjsPOm9W2aHlvMFOfNzDb7+V1SCOGdU1DRu+rBeWXpCCGuBp4BOgL9pJRrSxg3Bvg3oAMfSSlfOp/rAjz7yxa2Hso+39P40alpAk+P63zGMQsWLCA6OppbbrkFAF3Xef3112ndujXPP/88Bw8eZPjw4WRkZHDDDTfw9NNPk5eXx6RJk0hPT8cwDP6/vTMPjqrK9/jnl04nnRAgQEAStoQlbCEJBALKIovIMiEqoDA+kZiZYpRNNh+UwiCDxahQ8ywZHj5UVvXBkLAZQR8iYOXhCEkmgURcMOQpw4xmoCKbkIXz/uim05100o1ZOjc5n6quuvee0/f8fn2S7z33LL+zYsUKpk2bRmZmJosWLeLatWuEhISwdetWQkNDGTlyJLGxsaSnpzNp0iQ2b97M+fPn8fHx4fr16/Tq1Yv8/Hy2bt3Kpk2bKC4upnv37uzYsYPs7GwOHDjA8ePHeemll0hNTWX16tUkJCQwdepUjhw5wpIlSygtLWXQoEFs3LgRf39/wsPDmTlzJu+//z4lJSXs3r2bXr30HqX1ydDuIfZjxy4dXx+hW7sgThZcpjw4pfDH0n+jUAWz3PwuXVQh82QOBbaIlqUOG6HnF5bH4nn5kHXF7j1c5s9+rzPI52v+UDKDd8rKty10wvbc8NGK3yioaQs/F5gMfFpVBhExARuACUAf4NciYtiRn7y8POLi4pyutWjRgs6dO1NaWsrJkydJTU3l9OnT7N69m4yMDD788EPCwsLIyckhNzeX8ePHU1JSwrx580hJSSEzM5Pk5GSnt4Ti4mIyMjJYuXIlsbGxHD9+HIC0tDTGjRuH2Wxm8uTJnDp1ipycHHr37s3bb7/NfffdR2JiImvXriU7O5tu3brZ73nz5k2SkpLYtWsXZ86cobS0lI0bN9rTQ0JCyMrK4plnnmHdunV1/EtqqsNxQNzHR1g5qQ87fhNPSHN/p3xvlf2KWcUL6cQPHPR7nmdNqQRy02nWzkWHFn7RtetMN33C//j/O1FSwOzi+Wwum8DgiMoLrKB81pCW+8ZBjVr4Sqmz4Ha2RjxwTimVb8u7E3gI+KK6L7nDXUvcW4wdO5Y2bdoAMHnyZNLT05k4cSKLFy9m6dKlJCQkMHz4cHJzc8nNzWXsWGvLqqysjNDQUPt9pk2b5nS8a9cuRo0axc6dO5k9ezYAubm5LF++nKKiIq5du8a4ceOqte2rr74iIiKCyMhIAGbOnMmGDRtYsGCB3V6AuLg49uzZU0u/iKammESwmE0M79GWZalnKqX/rdkwxl/tyu/NO1hoTuU3vgcpzJ7ERJ/2FKj2FOPL1G5QVpBOos9ndPIp5NTtSJ4r+Z39jaBX++Z8bovL48ht3cJvVNTHwqsOwPcO5xeAwfVQbp3Qp08fUlJSnK5duXKF7777Dl9f30oPPxEhMjKSrKwsDh48yPLlyxkzZgyPPPIIffv25bPPPnNZTrNmzezHiYmJPP/881y+fJnMzExGjx4NQFJSEvv27SMmJoatW7dy7NixGvnm729tPZpMpl80dqCpG0wOm4xMievI60e+cUqPaNOMk1fbMLtkAf1Lv2GG72ESvz/Af/o5rJG4AKUmH/56uze/L07i6O1YHNvt3doFuSxb2WPw1Jo7Gi/itktHRD4WkVwXn4dq2xgRmSUiGSKSUVhY6P4LXmDMmDHcuHGD7dutcU3KyspYvHgxSUlJBAYGcvjwYS5fvszPP//Mvn37GDp0KBcvXiQwMJAnnniC5557jqysLHr27ElhYaFd8EtKSsjLy3NZZlBQEIMGDeLZZ58lISEBk20Z/tWrVwkNDaWkpIR3333Xnr958+ZcvXq10n169uxJQUEB586dA2DHjh3cf//9tfr7aGofR8Ff+EAPRvZs65TetW154+DHltEsKpnNW8OOkXhrNb8rXsD84rnsiXmLAbfe4ImSFzh6uz8VO2nC2zTDFQrl8rrGmLgVfKXUA0qpKBef/R6W8Xegk8N5R9s1V2VtUkoNVEoNbNu2rassXkdE2Lt3L7t376ZHjx5ERkZisVhYs2YNAPHx8UyZMoXo6GimTJnCwIEDOXPmDPHx8cTGxrJq1SqWL1+On58fKSkpLF26lJiYGGJjYzlx4kSV5U6bNo133nnHqatn9erVDB48mKFDhzoNsE6fPp21a9fSv39/vv32W/t1i8XCli1bePTRR+nXrx8+Pj48/fTTdfAraWqT4MDyWEUigm+FfWTDQ8rFenSvdgB89OVlTqtufHQ7ngO37+N6aDw3fVtUWcY9LSwuryvdpdOokLvZAafKm4gcA5a4mqUjIr7A18AYrEJ/CnhcKeW6OWtj4MCBKiPD+XZnz56ld+/eNbZXU3voOqk7wpd9AMCnz42is0O8+99uy+Djsz/Yz994Is6+ufl/zYhjwc7sSvvgvjYtlpc+OMu/rt3CFVkrxjJg9WGGdm/D/567BEDBy78i7fRF5r73N3bNGsLgrm1q1T9N3SAimUopl7vT1HRa5iPAeqAt8IGIZCulxolIGNbplxOVUqUiMhf4COu0zM3uxF6j0cCbTw6kucXXSeyhvD/d39eHW6W3ie7Y0p7WwmKmZ/vmZH9f5PSd5hZfWgb4Vin4LQPMfLxoBKEtA+i78iP79YToMOLDW9OuijcAjbGo6SydvcBeF9cvAhMdzg8CB2tSlkbT1BjrZkepP07uR2jLAMKCA+jVvjlf/vMqgX4m2lWYugkQ5O9Ly4Cqw1ibfITu7Zq7TNNi33jQK201GoNxpzc90M/Evd2s3SxB/ta2W5lSLjcZb+ZG8DVNAy34Go3BcDV++h/TYnl8cGeiO7Qk0L/yi7vF7GMX/JAg5zj3IyIb5gQJTe2jBV+jMSiO8y06tQ5kzSP98DX5uNxz1t/XZBf8+yPb2a8/HBvG9uT4Svk1jRMt+BpNI8NVl47FbCLItk9ui4DyNwC9p0HTQgv+XfLDDz/w+OOP07VrV+Li4rj33nvZu7fSuHWdUlBQQFRUlMvr77333i+652uvvcaNGzfs50FBrldearyPuIlsE+BnFXQ/h4ibFrMPJtv8fV8fLfJNFS34d4FSiocffpgRI0aQn59PZmYmO3fu5MKFC5XyeiM0QXWC786eioKvafhUtYLmTgu/vcPsGovZZI+06diqdyX9k2LCGNpdz7lvjBhqE3MnDi2Df1YOJFUj2veDCVVHbv7kk0/w8/NzWp3apUsX5s2bB8DWrVvZs2cP165do6ysjL1795KcnEx+fj6BgYFs2rSJ6OhoXnzxRYKCgliyZAkAUVFRpKWlATBhwgSGDRvGiRMn6NChA/v37ycgIMAeURPgwQcfdGnfsmXLOHv2LLGxscycOZNWrVo52bNq1SrWrVtnL2vu3LkMHDiQK1eucPHiRUaNGkVISAhHjx4F4IUXXiAtLY2AgAD279/PPfdUP01QUz+464W5E0s/LNjCd5etD3Gzyce+WtZd+379r/vX1ERNA0W38O+CvLw8BgwYUG2erKwsUlJSOH78OCtXrqR///6cPn2aNWvW8OSTT7ot45tvvmHOnDnk5eURHBxMamoqAE899RTr168nJyenyu++/PLLDB8+nOzsbBYuXFjJnqqYP38+YWFhHD161C72169fZ8iQIeTk5DBixAjefPNNt7ZrGgYdWgUAMDWuk9P1HrYAaT3bO8y31707TQrjtvCraYnXF3PmzCE9PR0/Pz9OnToFWMMjt25tjS2enp5uF+zRo0dz6dIlrlypftOWiIgIYmNjAWuY4oKCAoqKiigqKmLEiBEAzJgxg0OHDnlko6M9d4Ofnx8JCQl2Ow4fPnzX99DULVVFRRnWPYSsFWNp3cyPJbvLGwgT+oWyf85Qoju2ZNFfrNfdjQdoGhe6hX8X9O3bl6ysLPv5hg0bOHLkCI6RPR3DGleFr68vtx12JLp5szyM7Z0QxVA7YYod7amu3IqYzWZ7X68Ol9ywuLPIymxyLdYiQutmfi7TYjoFO/fha71vUmjBvwtGjx7NzZs3nXaJqm6gc/jw4fawxceOHSMkJIQWLVoQHh5uf3BkZWVx/vz5assNDg4mODiY9PR0AKdQyI5UFRb5Dl26dOGLL77g1q1bFBUVceTIEY+/q2k4rJjUhyUPRvJA75qPqSTGhNWCRRqjYNwuHS8gIuzbt4+FCxfy6quv0rZtW5o1a8Yrr7ziMv+LL75IcnIy0dHRBAYGsm3bNgCmTJnC9u3b6du3L4MHD7bvQFUdW7ZsITk5GRGpctA2Ojoak8lETEwMSUlJtGrVyim9U6dOPPbYY0RFRREREUH//uWDc7NmzWL8+PH2vnxNw6WFxczc0T08ypux/AF+Li6rMl2vsm1a1Ep45LpAh0c2BrpOjMm2EwXEdWlFVIeW7jNrDEWdhUfWaDTGZOZ94d42QeMFdB++RqPRNBEMJ/gNtQuqKaLrQqMxFoYSfIvFwqVLl7TQNACUUly6dAmLRW+OodEYBUP14Xfs2JELFy44zXvXeA+LxULHjh29bYZGo/EQQwm+2WwmIiLC22ZoNBqNITFUl45Go9Fofjla8DUajaaJoAVfo9FomggNdqWtiBQC/1eDW4QA/6olc7xJY/EDtC8NlcbiS2PxA2rmSxellMuYGQ1W8GuKiGRUtbzYSDQWP0D70lBpLL40Fj+g7nzRXToajUbTRNCCr9FoNE2Exiz4m7xtQC3RWPwA7UtDpbH40lj8gDrypdH24Ws0Go3GmcbcwtdoNBqNA1rwNRqNpolgaMEXkfEi8pWInBORZS7S/UVkly39cxEJr38rPcMDX5JEpFBEsm2f33rDTneIyGYR+VFEcqtIFxF53ebnaREZUN82eooHvowUkZ8c6uT39W2jJ4hIJxE5KiJfiEieiDzrIo8h6sVDX4xSLxYROSkiOTZfVrnIU7sappQy5AcwAd8CXQE/IAfoUyHPbOAN2/F0YJe37a6BL0nAn71tqwe+jAAGALlVpE8EDgECDAE+97bNNfBlJJDmbTs98CMUGGA7bg587eLvyxD14qEvRqkXAYJsx2bgc2BIhTy1qmFGbuHHA+eUUvlKqWJgJ/BQhTwPAdtsxynAGBGRerTRUzzxxRAopT4FLleT5SFgu7LyVyBYRELrx7q7wwNfDIFS6h9KqSzb8VXgLNChQjZD1IuHvhgC2299zXZqtn0qzqKpVQ0zsuB3AL53OL9A5Yq351FKlQI/AW3qxbq7wxNfAKbYXrdTRKRT/ZhW63jqq1G41/ZKfkhE+nrbGHfYugT6Y21NOmK4eqnGFzBIvYiISUSygR+Bw0qpKuulNjTMyILf1HgfCFdKRQOHKX/qa7xHFta4JTHAemCfl+2pFhEJAlKBBUqpK962pya48cUw9aKUKlNKxQIdgXgRiarL8ows+H8HHFu5HW3XXOYREV+gJXCpXqy7O9z6opS6pJS6ZTt9C4irJ9tqG0/qzRAopa7ceSVXSh0EzCIS4mWzXCIiZqwC+a5Sao+LLIapF3e+GKle7qCUKgKOAuMrJNWqhhlZ8E8BPUQkQkT8sA5oHKiQ5wAw03Y8FfhE2UY/GhhufanQn5qIte/SiBwAnrTNChkC/KSU+oe3jfoliEj7O/2pIhKP9f+pwTUobDa+DZxVSv2pimyGqBdPfDFQvbQVkWDbcQAwFviyQrZa1TBDbXHoiFKqVETmAh9hneWyWSmVJyJ/ADKUUgew/mHsEJFzWAffpnvP4qrx0Jf5IpIIlGL1JclrBleDiPw31lkSISJyAViJdTAKpdQbwEGsM0LOATeAp7xjqXs88GUq8IyIlAI/A9MbaINiKDADOGPrLwZ4HugMhqsXT3wxSr2EAttExIT1ofQXpVRaXWqYDq2g0Wg0TQQjd+loNBqN5i7Qgq/RaDRNBC34Go1G00TQgq/RaDRNBC34Go1G00TQgq/RaDRNBC34Go1G00T4fyTFclIRNUv4AAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Y3wx8vJlMgwI"
      },
      "source": [
        "With the input-output pairs created, your first task is now to partition the data in the training, validation and test sets. Keep in mind that we have created the data in a structured way, i.e. the input-output pairs are ordered. This means you need to shuffle the data before partitioning it."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "kDIMUZs0MgwK",
        "colab": {}
      },
      "source": [
        "\"\"\" Shuffle and partition the data set accordingly. you can use the predefined constants \"N_train_samples\", \"N_validation_samples\" and \"N_test_samples\". Use the variable names that are already in the below code \n",
        "to store the final shuffled and partitioned data. Hint: Shuffle the data and the labels in such a way that the pairing between an image and it's label is preserved.\"\"\"\n",
        "# Shuffle the data\n",
        "np.random.seed(0)\n",
        "np.random.shuffle(x)\n",
        "np.random.seed(0)\n",
        "np.random.shuffle(y)\n",
        "np.random.seed(0)\n",
        "np.random.shuffle(y_true)\n",
        "\n",
        "\n",
        "# Partition the data\n",
        "x_train = x[:N_train_samples].reshape([-1,1])\n",
        "y_train = y[:N_train_samples].reshape([-1,1])\n",
        "x_test = x[N_train_samples:N_train_samples+N_test_samples].reshape([-1,1])\n",
        "y_test = y[N_train_samples:N_train_samples+N_test_samples].reshape([-1,1])\n",
        "x_validation = x[N_train_samples+N_test_samples:N_samples].reshape([-1,1])\n",
        "y_validation = y[N_train_samples+N_test_samples:N_samples].reshape([-1,1])"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "-ucvKRhOMgwN"
      },
      "source": [
        "In order to feed the data to our model, we will use the Dataset class provided by Tensorflow. This class is simple to use and provides all the functionality we need for shuffling, batching and feeding the data to our model. It is also tightly integrated into the Tensorflow framework, which makes it very performant. Performance is not an aspect we need to worry about in this exercise, but it is important in more demanding applications.\n",
        "\n",
        "In this exercise we instantiate a separate Dataset object for the training, validation and test data sets, where we shuffle and repeat just the training data set. Shuffling the validation and test data sets is not necessary, since we only evaluate the loss on those data sets and do not perform SGD on it. Please fill in the missing part of the code."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "HifQ63iPMgwP",
        "colab": {}
      },
      "source": [
        "\"\"\" Create three tensorflow Dataset objects that can be used to feed the training test and validation data to a neural network. Hint: For the training data set use shuffling, batching with the size according to\n",
        "the predefined constant \"batch_size\" and repeat the data set indefinetly. For the validation and test data sets no shuffling or batching is needed.\"\"\"\n",
        "\n",
        "train_ds = tf.data.Dataset.from_tensor_slices((x_train,y_train)).shuffle(N_train_samples).batch(batch_size).repeat()  #Repeat indefinitely\n",
        "validation_ds = tf.data.Dataset.from_tensor_slices((x_validation,y_validation))\n",
        "test_ds = tf.data.Dataset.from_tensor_slices((x_test,y_test))"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FupUrcS6PGcT",
        "colab_type": "code",
        "outputId": "bdb45d21-cb06-4447-f793-130f1bbfe9b3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "train_ds"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<RepeatDataset shapes: ((None, 1), (None, 1)), types: (tf.float32, tf.float32)>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "5Crr6fIkMgwT"
      },
      "source": [
        "In this exercise we will create a a simple neural network with two hidden layers containing $10$ neurons. For creating a model and keeping track of its weights a class called MyModel is used. When initializing an instance of this class the necessary variables are created and stored in a list called \"trainable_variables\". This makes it easy to get all trainable variables of the model. We also override the \\__call__ method of this class in order to implement the forward pass of the neural network. This method should accept the inputs to the neural network and should return the result of the forward pass as an output. Please fill in the missing part of the code and select suitable activation functions for the different layers."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Nq8ri416MgwX",
        "colab": {}
      },
      "source": [
        "\"\"\" Implement a neural network with two hidden dense layers containing 10 neurons each. As an activation function use the tangens hyperbolicus (tf.nn.tanh()). Since we are not using Keras, we need to create and \n",
        "manage all the variables that we need ourselves. The varaibles are created in the constructor of our model class. Since we want to be able to just call the class with some inputs in order to make a prediction, \n",
        "we implement a __call__ method which computes the forward pass and returns the output of the network.\"\"\"\n",
        "\n",
        "class MyModel(object):\n",
        "    def __init__(self):\n",
        "        # Create model variables\n",
        "        input_dim = 1\n",
        "        hidden_dim = 10\n",
        "        output_dim = 1\n",
        "        \n",
        "        self.W0 = tf.Variable(tf.random.normal([input_dim,hidden_dim]))\n",
        "        self.b0 = tf.Variable(tf.random.normal([hidden_dim]))\n",
        "        self.W1 = tf.Variable(tf.random.normal([hidden_dim,hidden_dim]))\n",
        "        self.b1 = tf.Variable(tf.random.normal([hidden_dim]))\n",
        "        self.W2 = tf.Variable(tf.random.normal([hidden_dim,output_dim]))\n",
        "        self.b2 = tf.Variable(tf.random.normal([output_dim]))\n",
        "        self.trainable_variables = [self.W0, self.b0, self.W1, self.b1, self.W2, self.b2]\n",
        "\n",
        "    def __call__(self, inputs):\n",
        "        # Compute forward pass\n",
        "        output = tf.reshape(inputs, [-1, 1])\n",
        "        output = tf.nn.tanh(tf.linalg.matmul(output,self.W0) + self.b0)\n",
        "        output = tf.nn.tanh(tf.linalg.matmul(output,self.W1) + self.b1)\n",
        "        output = tf.linalg.matmul(output,self.W2) + self.b2\n",
        "        return output"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "uf6m8zXoMgwb"
      },
      "source": [
        "Now after the model class is defined we can instantiate a MyModel object by running"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "nSfI8wjLMgwb",
        "colab": {}
      },
      "source": [
        "mdl = MyModel()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "KraeH6MsMgwh"
      },
      "source": [
        "We can now use the model to make predictions by calling it. In the following we predict on the inputs an plot the result."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "K1at5RObMgwi",
        "outputId": "cdbb2c19-48d4-4b9f-8f06-65c01be5d27e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        }
      },
      "source": [
        "\"\"\" We want to plot a prediction on the complete data set with a model before training. For this make a prediction on the variable \"x\". \"\"\"\n",
        "\n",
        "y_pred = mdl.__call__(x)\n",
        "plt.plot(x, y,'b.')\n",
        "plt.plot(x, y_true,'r.')\n",
        "plt.plot(x, y_pred.numpy(),'g.')\n",
        "plt.legend([\"Observation\", \"Target\", \"Prediction\"])\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO29e3xU5bX//35mkknCRa2RiooIKlioSIAYGBHkiE20UkSsbS2ISCEELz89r9aJWltr1aDg69RzqMBERQU81W8F0eOlomgEnUEkgoJ3tChYFQxFQUKSmXl+f+zsuU8ySXYyl6z367VfM7P3nr2ffZnPrL2etdajtNYIgiAImYst1Q0QBEEQOoYIuSAIQoYjQi4IgpDhiJALgiBkOCLkgiAIGU5OKnZ6zDHH6AEDBqRi14IgCBlLbW3tN1rrPtHzUyLkAwYMYPPmzanYtSAIQsailPos3nxxrQiCIGQ4IuSCIAgZjgi5IAhChiNCLgiCkOGIkAuCIGQ4IuSCIAgZjgi5IAhCJ+Pd5eXixy5m6H1Dufjxi/Hu8lq6/ZTEkQuCIGQrlS9V8tCWh2jwNeAL+PBpH43+xuDy9795n2c/epZXZ76K80SnJfsUIRcEQWgH3l1ebnzpRt75+h0COkBAB2j0N9IYaGz1u02BJmp21qSXkCuldgIHAD/g01oXW7FdQRCEdKC6tpr/3vjffP3919Q31eML+JIS7ETk2nKZMGCCZe2z0iL/D631NxZuTxAEocupfKmSZVuWGa4Rv+EaaQo0Wbb94ccOZ8mFSyyzxkFcK4IgdGPC/dkBAjT4GiwTbYfNQY4tB42mILeAvj37ct2Y6ygfVW7J9sOxSsg1sFYppQG31ro6egWlVDlQDtC/f3+LdisIgpAc1bXVzN8wnz0H94Ay/NRWiXa+PZ8cWw42ZSMvJ48rR1zJ3efdbcm2k8EqIT9ba/2FUuqHwItKqQ+01uvDV2gW92qA4uJiGfFZEIROo7q2mqoNVew9uBetNAEdoMHfYMm2HXYHduwU5BZwbK9juX7M9Z1iZbcFS4Rca/1F8+sepdSTQAmwvuVvCYIgdBxTtOsO1WFTNg75DuEL+Dq8XYfNgV3ZUUpxRN4RjOk3BtdYl6W+bavosJArpXoCNq31geb3pcCfO9wyIesww7W2fLUFn99Hjj0HX8CHQmG32dFaY1M2mvxNoIye/aZAE2jIsecElztyHMwaMatLH12F9KC6tpr5r83nm++/waZs1PvqLXGPOGwOHDkOFIr8nPwud410FKV1x7wcSqmTgSebP+YA/6u1vrOl7xQXF2sZWKJ7UF1bzb0b7+WL777gu8bvLN12ri2XHJWD3WZn5HEjueu8u9LSWhLaR7Slfdh3uEMhfyZ2ZSfPnofdZs840VZK1cYL7+6wkLcHEfLsxrvLy4LXF7Dun+s40HigS/fdI6cHDruDM/qewV0TRdgzheraav77jf/mqwNfcbjpME3amo5IO3YcOQ5ybbkp6YS0mkRCLuGHgqVMXz2dR7c9mrL9H/Id4pDvEOs/W89Zy87CYXfQM7cnc0bNyegfcDZhPqXt+X4P9U31lkWP5NpyKcgpoMnfRF5uHuWjyrvNNReLXLCEypcqWbRxEfX++qTWz7Pn4bA72uwjTzYFOh45Kocj8o5Im0iD7kBEnLYO4Av4OOw/bMm2e+T0QGtNXk4e5cXdQ7TFtSJ0CtW11dz80s3UHa5rdd1ejl6M7NtxX3blS5U8vOVh6n31NPgbIgoStYU8ex4FOQWccewZ4l/vIN5dXhZ4FrBx10a+a/gOrTV+7bfEpw2GaNuUjd55vRl9wui0jR7pbETIBctJxo3Sw96D4hOKO1UozWiYbV9v45DvULvjhQtyCuiR24Mf9vyhWOwtYNYd+ergVxz2HY6p7tdRHDYHdpudo/KP6taiHQ8RcsEyvLu8XPXsVWz9emvCdQrzC6k6ryolYhjug/224dt2xxTn2fPIteVyTM9juOnsm7qdsJtRI/vq96FQBHSApkCTZYk1YJxjO3aOLDhSRDsJRMgFS6iurWbuM3MTLi8sKKRqYmoEPBFmavbXB79O2ocfD4fdQZ49D3/Aj8PuyAq/bLibSqNRKHwBo0iUFUk1Jg6bA4fdgU3ZsCmbuLPaiQi50GFaE3HXWFfaC5t3l5cb1xk1pOub6jtsXeaoHCNl22YPCmFBbgEzi2am/FwE62XveQd/wB+c7/f70UoHBdtqzPj+XHt2hPylEyLkQoeofKmSBa8vSLjcPcmdVlZ4sphuGLPOdL2v/RZ7NKagAeTacwnoQCgqB4KZrUCwSp5pEZuRPP6AH6118PtKKXx+XzCqxxfwodHG97U2lgd8ltYWaYnwuiN9e3VedT/BQIRcaDctdWoWHVvE4gsXZ80jcvioL43+Rg75DqW6SWmBGTWilMKu7OIaSRGSECS0i5ZEfNqwaaycurKLW9S5OE908uqVrwY/m1mqb3zxBgcaD6BQHPYd7hSXRCoxMx/NJwI0HFUgUSOZggi5kJCWRDwT/OFW4DzRyZO/ejJmvhnRsef7PaGEJrSlAxNYiZmABSEfebakrQviWhESkMgnrlAsnbRU/KAtEJ7NaPrDu9pHjsaIie8lMfHZhPjIhaRJJOJ9e/Zl9S9Xy2O2IKSIREJuS0VjhPSlpegUEXFBSE9EyIUg3l1eFr6+MO4y9yS3iLggpCki5EKQec/OQxPrasvUGHFB6C6IkAsAlK0o4+2v346ZLyIuCOmPCLlA2Yoy1n66Nma+a6xLRFwQMgAR8m7O9NXT44r4tGHTJLZYEDIEEfJuTOVLlXETfkpPLs26jE1ByGZEyLspiSJUSk8u5YXLX0hBiwRBaC8i5N2Uq567KiZCpejYIhFxQchALBNypZRdKbVFKfWMVdsUOofpq6ez9avY0X0WX7g4Ba0RBKGjWGmRXwe8b+H2hE4gkV9cKtwJQuZiiZArpfoBFwIPWLE9oXMwS7JGIxEqgpDZWGWR3wu4gECiFZRS5UqpzUqpzXv37rVot0JbuOq5q2LmFR1bJBEqgpDhdFjIlVKTgD1a69qW1tNaV2uti7XWxX369OnoboU2In5xQcherLDIxwKTlVI7gceAc5VSYuKlEeIXF4TspsNCrrW+SWvdT2s9APgV8LLWenqHWyZYQnVttfjFBSHLkTjyLMa7y8u8Z+fFzBe/uCBkF5aO2am1rgFqrNym0H7mPTuPgI7sf1Yo8YsLQpYhFnmWUvlSZdyytEsnLRW/uCBkGSLkWUii4dqkLK0gZCci5FlGIhEv6lsknZuCkKWIkGcRiSoaKhSLfyp+cUHIVkTIs4hEY26KX1wQshsR8ixh+urpMuamIHRTRMizgJYyN0XEBSH7ESHPcCRzUxAEEfIMprq2mrnPzI2ZX9RXMjcFoTshQp6heHd544q4RKgIQvdDhDxDiVdbHCRCRRC6IyLkGUjZirK4tcWlc1MQuici5BlG2Yoy1n66Nma+dG4KQvdFhDyDmL56ekIRl85NQei+iJBnCNNXT48bK156cqmIuCB0c0TIM4BEIn7qD07lhctfSEGLBEFIJ0TI05xEIm5XdpZfvDwFLRIEId0QIU9jEon4cb2OY8OVGyTMUBAEQIQ8bUlUP8Wu7Kz6xSoRcUEQgoiQpyGJBocQS1wQhHhYOviy0HESuVMAscQFQYiLWORpREsi7p7kFhEXBCEuHRZypVS+UmqTUuptpdS7SqnbrGhYd6M1EZfUe0EQEmGFa6UBOFdrfVAplQu8ppR6Xmu90YJtdwtExAVB6AgdFnKttQYONn/MbZ5iB44U4pKodopCsXTSUhFxQRBaxZLOTqWUHagFTgXu01q/EWedcqAcoH///lbsNuMZff9oNv1rU8z8vr36svoXq8UnLghCUljS2am19muti4B+QIlS6vQ461RrrYu11sV9+vSxYrcZTdmKsrgibld2EXFBENqEpVErWuv9wCvA+VZuN9tI5E459QenSpy4IAhtpsOuFaVUH6BJa71fKVUA/ASQwtgJSOROKTm+hDfmxHikBEEQWsUKi/w44BWl1DvAm8CLWutnLNhuVuHd5WXwosFxRbz05FIRcUEQ2o0VUSvvACMsaEvWUl1bTcUzFeg4wTylJ5dKKVpBEDqEpOh3MtW11XFHuwfDnSIiLghCR5EU/U7EtMTjIe4UQRCsQoS8kzAt8XjulGnDpoklLgiCZYhrpRNoyZ3iGuvK6tHum5qa2L17N4cPH051U7ol+fn59OvXj9zc3FQ3RehCRMgtpjuLOMDu3bvp3bs3AwYMQCmV6uZ0K7TW1NXVsXv3bgYOHJjq5ghdiLhWLMS7y5tQxKcNm5b1Ig5w+PBhCgsLRcRTgFKKwsJCeRrqhoiQW4R3l5epj0+Nu8w11sXKqSu7uEWpQ0Q8dci5756IkFtAdW01Y5eN5avvv4pZ1h3cKYIgpBYR8g7SWnSKiHhq2L17NxdddBGDBg3ilFNO4brrrqOxsZGHH36Ya665JtXNY82aNbz33nvBz3/84x956aWXUtgiIZMRIe8ALXVsThs2rVu5U9IJrTVTp05lypQpfPzxx3z00UccPHiQ3//+952yP5/P1+bvRAv5n//8Z8477zwrmyV0I0TI24l3lzdhso+IeNvwemH+fOPVCl5++WXy8/O58sorAbDb7fzlL39h2bJlHDp0iF27djFhwgQGDRrEbbcZIxN+//33XHjhhQwfPpzTTz+dxx9/HIDa2lrOOeccRo0aRVlZGV9++SUAEyZM4Prrr6e4uJg777yTk046iUAgENzWiSeeSFNTE/fffz9nnnkmw4cP55JLLuHQoUN4PB6efvppbrjhBoqKivjkk0+YOXMmTzzxBADr1q1jxIgRDBs2jFmzZtHQ0ADAgAEDuPXWWxk5ciTDhg3jgw8+sOaECRmPCHk7MDs247lTulvHZkfxemHiRPjDH4xXK8T83XffZdSoURHzjjjiCPr374/P52PTpk2sWrWKd955h7///e9s3ryZf/zjHxx//PG8/fbbbN++nfPPP5+mpiauvfZannjiCWpra5k1a1aEVd/Y2MjmzZu59dZbKSoq4tVXXwXgmWeeoaysjNzcXKZOncqbb77J22+/zZAhQ3jwwQc566yzmDx5MgsXLmTr1q2ccsopwW0ePnyYmTNn8vjjj7Nt2zZ8Ph9LliwJLj/mmGN46623mDdvHvfcc0/HT5aQFYiQtxHvLi9nLztbOjYtoqYGGhvB7zdea2o6f58/+clPKCwspKCggKlTp/Laa68xbNgwXnzxRSorK9mwYQNHHnkkH374Idu3b+cnP/kJRUVF3HHHHezevTu4nV/+8pcR700r/rHHHgsu2759O+PGjWPYsGE8+uijvPvuuy227cMPP2TgwIEMHjwYgCuuuIL169cHl0+dakRGjRo1ip07d1pyPoTMRxKC2shVz15FgEDMfOnYbB8TJoDDYYi4w2F87ihDhw4NuilMvvvuOz7//HNycnJiQvSUUgwePJi33nqL5557jltuuYWJEydy8cUX8+Mf/xhvgseEnj17Bt9PnjyZm2++mX379lFbW8u5554LwMyZM1mzZg3Dhw/n4YcfpqaD/1R5eXmA4S5qj29eyE7EIm8D01dPZ+vXW2Pmi0+8/TidsG4d3H678eq0YHCkiRMncujQIZYvXw6A3+/nt7/9LTNnzqRHjx68+OKL7Nu3j/r6etasWcPYsWP517/+RY8ePZg+fTo33HADb731Fqeddhp79+4NCnlTU1NCi7pXr16ceeaZXHfddUyaNAm73Q7AgQMHOO6442hqauLRRx8Nrt+7d28OHDgQs53TTjuNnTt3smPHDgBWrFjBOeec0/GTImQ1IuRJMn31dB7d9mjMfBHxjuN0wk03WSPiYFjYTz75JH//+98ZNGgQgwcPJj8/n6qqKgBKSkq45JJLOOOMM7jkkksoLi5m27ZtlJSUUFRUxG233cYtt9yCw+HgiSeeoLKykuHDh1NUVITH40m431/+8pesXLkywuVy++23M3r0aMaOHcuPfvSj4Pxf/epXLFy4kBEjRvDJJ58E5+fn5/PQQw9x6aWXMmzYMGw2GxUV8TvVBcFEaR3bYdfZFBcX682bN3f5fttLIhEvOraILRVbUtCi9OX9999nyJAhqW5Gt0auQfailKrVWhdHzxeLvBUSibgNG4svXJyCFgmCIEQiQt4ClS9VxhXxvj378tqs12S0e0EQ0gIR8gRU11az4PUFMfPtys7qX64WERcEIW0QIY9DotT743odx4YrN4iIC4KQVoiQR9FS6v2qX6wSERcEIe3osJArpU5USr2ilHpPKfWuUuo6KxqWKmY8OSNh6r2IuCAI6YgVFrkP+K3WeigwBrhaKTXUgu12OWUrytjx7x0x8yX1PnOoq6ujqKiIoqIi+vbtywknnBD83NjYaOm+9u/fz+LFErkkpJ4OC7nW+kut9VvN7w8A7wMndHS7Xc301dNZ++namPki4plFYWEhW7duZevWrVRUVPCf//mfwc8OhyPh99qT7i5CLqQLlvrIlVIDgBHAG1Zut7NJFCteenKpiHhXYHUd2yjilZIFow5KRUUFo0ePxuVy8cknnzBmzBiGDRvGLbfcQq9evYLbWLhwIWeeeSZnnHEGt956KwA33ngjn3zyCUVFRdxwww2d0nZBSAbLimYppXoBq4DrtdbfxVleDpQD9O/f36rddphEseKlJ5fywuUvpKBF3Qyzjq1ZNcuqgithTJ06lTlz5gBwyy238OCDD3LttdcCxkhCHo8Hu93OpEmTuO6667jssstYunRp8Ptr167l448/ZtOmTWitmTx5MuvXr+euu+5i+/btbN0aW39HELoSSyxypVQuhog/qrVeHW8drXW11rpYa13cp08fK3bbYRLFihf1LRIR7yq6oI5tS6VkL7300mCBK6/Xy6WXXgrAr3/96+A6a9euZe3atYwYMYKRI0fywQcf8PHHH1veTkFoLx22yJVRE/RB4H2t9X91vEldQ3VtdcIww8U/Fb9nl9EZdWyjaKmUbHgp2kRorbnpppuYOzcyt0DqgQvpghUW+VjgcuBcpdTW5umnFmy302hpwGT3JLeEGXYlnVHHNopEpWSjGTNmDKtWrQKMwSFMysrKWLZsGQcPHgTgiy++YM+ePQlL0QpCV9Nhi1xr/RqgWl0xTWgp4cc9yU35qPIubpGA09kpAm5ilpLt06cPo0ePTii+9957L9OnT+fOO+/k/PPP58gjjwSgtLSU999/H2dzG3v16sXKlSs55ZRTGDt2LKeffjoXXHABCxcu7LRjEISW6HZlbAf/z2A+/nesf1PCDK0hk0uoHjp0iIKCApRSPPbYY/ztb3/jqaeeSnWz2kwmXwOhZRKVse1WQ72VrSgTERcSUltbyzXXXIPWmqOOOoply5alukmCkBTdRsgl4UdojXHjxvH222+nuhmC0Ga6RdEsSfgRBCGbyXohl4QfQRCynawWckn4EQShO5C1Qp4o4UehJOFHEISsIiuFvKWEn6WTlkrCT5Zjt9spKiri9NNP59JLLw0WyWoPM2fO5IknngBg9uzZvPfeewnXrampwePxBD8vXbqU5cuXt3vfgpAsWSfkiYZpA0n46S4UFBSwdetWtm/fjsPhiCiABe0rWQvwwAMPMHRo4lL70UJeUVHBjBkz2rUvQWgLWSXk3l3ehCLuGusSEU9TvLu8zN8wH+8u68vYjhs3jh07dlBTU8O4ceOYPHkyQ4cOxe/3c8MNNwRL07rdbsCoq3LNNddw2mmncd5557Fnz57gtiZMmICZyPaPf/yDkSNHMnz4cCZOnMjOnTtZunQpf/nLXygqKmLDhg386U9/4p577gFg69atjBkzhjPOOIOLL76Yf//738FtVlZWUlJSwuDBg9mwYYPl50DIfrImjty7y8vU/zc17jKJFU9fvLu8TFw+kUZ/Iw67g3Uz1lnm+vL5fDz//POcf/75ALz11lts376dgQMHUl1dzZFHHsmbb75JQ0MDY8eOpbS0lC1btvDhhx/y3nvv8fXXXzN06FBmzZoVsd29e/cyZ84c1q9fz8CBA9m3bx9HH300FRUV9OrVi9/97ncArFu3LvidGTNmsGjRIs455xz++Mc/ctttt3HvvfcG27lp0yaee+45brvtNl566SVLjl/oPmSFRe7d5WXcQ+P46uBXMctExNObmp01NPob8Ws/jf5GanbWdHib9fX1FBUVUVxcTP/+/fnNb34DQElJCQMHDgSM0rTLly+nqKiI0aNHU1dXx8cff8z69eu57LLLsNvtHH/88Zx77rkx29+4cSPjx48Pbuvoo49usT3ffvst+/fv55xzzgHgiiuuYP369cHlU6caBsioUaOkoqLQLrLCIp/x5Az82h8zf9qwaSLiac6EARNw2B1Bi3zCgAkd3qbpI48mvGSt1ppFixZRVlYWsc5zzz3X4f23lby8PMDopG2v/17o3mS8RZ5owORpw6axcurKFLRIaAvOE52sm7GO2//jdkvdKq1RVlbGkiVLaGpqAuCjjz7i+++/Z/z48Tz++OP4/X6+/PJLXnnllZjvjhkzhvXr1/PPf/4TgH379gEkLGt75JFH8oMf/CDo/16xYkXQOhcEK8hoi7xsRVnc+iki4pmF80Rnl4eEzp49m507dzJy5Ei01vTp04c1a9Zw8cUX8/LLLzN06FD69+8fLF0bTp8+faiurmbq1KkEAgF++MMf8uKLL/Kzn/2Mn//85zz11FMsWrQo4juPPPIIFRUVHDp0iJNPPpmHHnqoqw5V6AZkbBnbRCIuqfepRUqoph65BtlLojK2GelaEREXBEEIkXFCPvr+0SLiSeL1wvz5xmtblglCppHofu4u93lG+cinr57Opn9tipmfDSLu9RoDyE+YEDnqWaL5yWxv4sTQmMbhw2G2tMwKtNYYY3ILXU0qXKWpxOuF5cvhoYfA54u8nzv7Pk8nMkbIvbu8/O+2/42Zny0iHn7D3Xsv1NVBYSFcf33yN2K46NfUGN/z+43XmprQd1ta1lHy8/Opq6ujsLBQxLyL0VpTV1dHfn5+qpvSJZi/m8OHwfz/Cr+fO/M+TzcyRshrdtbEFMEqOb4krUTctA6+as5L6tsXZsxo3cIOv+EaGuCaayAQAJvNmBcItH4jxvszsNuN74Lxp2AyYYKxjrnuhAmx22rPUwBAv3792L17N3v37m3bFwVLyM/Pp1+/fqluRpdg/m5MEVcKcnJg0yaYNw9GjGj5Ps8mMkbIJwyYQEFOAYd9hwH49bBfpyTEMFrkwsX72WehOSw5yLJlIQFO9KhnCmtDg/Edn8+4Oc0b1GYzpv37jRv0q69i/ySirY9HHzXaorUx79prjfW2bDFeTas/nisnuo3m9lsTduPc5DJhwsCstXyE9CHcILHZYNAg+PBDWLPGWJ6ba9yv33wDgwfDtm3tN1DSHq11l0+jRo3S7cHzuUdXrKzSFXd4tMeTYB2P1lVVut3LW9y/R+uCAq3tduPV7dba4TAlN/6kVGh/paVa22zGfLvdmG/ichnLlAp9r6Xtmttwu41tV1QYbbHbjSne+ua+Qeu8vPjnoKoqtJ7NZmw3L89oT6LvRJ+bvDzje+05x4LQGuZv2OXSuqRE6/HjjXs/md+MzWbcp5l6bwKbdRxNtUSYgWXAHmB7Muu3V8jdbq1zc40LlpNjfA4nWmijL1Zry1ujoiJSaE89tfUbR6nQjWYKpFLGZ5fLEHeXK1J8bTZj2+HC25KY5+WFXsePb/075lRaapzDqqrQ67RpketEH+P48aHvhVNVFXkMSkWe4478gQqCicfTuvGUjJiXlmbmvdjZQj4eGNmZQu7xGOIdLWIVFSERqqgIiYkptOGCEy420RZxon2a4uN2JyesyU4tWQ+mpV1QoPUc3HobQ/XXHK3300vv4wh9gAJ9gILg+4PN70PLe+gD9Ih4X0dv/W3z8oMR3+/R/L633k8vXUdvXcdR+hXG6zF4WjyGKVMihbqgIPK4zB+MeSzt/QMVBJOKitj7cDZuvYOTYn4fdfTWB+ih99Mr5n6OZ5lngrHRqUJubJ8BnSnk4Y/88QTRZjP+qU03QPg606aFLN9kBcXtNv44bLaQxdtRgQ6/8UxxNkS1R1BITaFtyO2hD9vydABSNvlAf48j+Kewn156F331KqYEfxThbpRwF0/4eQh/n8wfqCAkYsqU0L1UhUsfoEeb7udaioL3rmkImk+kmWBsJBJyy1L0lVIDgGe01qcnWF4OlAP0799/1Geffdam7ZudcPX1idex2+FnP4P166G5jlEMLhd8953xPjqixKS62uhUNCM+kmU21dzEfI7lazSKRuw48AGKJuzY0OTQRA8a2rTdVAXxtXRnaOAgPdDYqOMY7uImPhxfztFHG+f+tdfin7+8PDDrUGVtx5NgKWaAQWGhEdF1W1Ml1/JXehIawi+Z34gOe93KcP4zdwkblRO/PzJCzG6H22+Hm27qhIPpIIlS9DPGIte6dfdGSx195lRSEtkpN2WKMZlWZfQ+qnDpL+mj68Is5kjXRUGzeyO/TdZuW30xXW2Nt6c93+PQuzkuwmKPnsxzbXbMprP1I6Qe02Vns2ntxKN3ckLc+7Qt97M5z4/Sc3AHn+hzctL/niSBRZ4x4YdghM61ZCX7Y0uSM5tqbmy2kgF4Kxfl86HRNPpzsK8BP+DAj1qq+RE5/LrZirbjI5/GNrWxLdZzW56FAkA9PWgih1yMGMcmcsmlCQU0kouNAAFs5OJrXp4TfN+InRw0fmw4aEIHv+8DNE3kRDwxtNQ2Fec4NVBAIwV8ycWsYQprqKOQDYxjIS42YpjdzzxjXCfzQTDbEzWEjlFTY4Tl3hGopJIFwfvOfA2/TxvJ5RAFwd9HIznk00h+2BNw5L2rWcpczmY9VwRWctll8OMfZ+hTYjx1b89EF1jkd0/x6FcYr/fTK9iBt5/eeh+9ozo4TCvZGh9zZ1rOh5p90N/aQp2RB8L85t8k2fEYb1IqFJ5ltxtWR6InlnA/tunD30NhWFuO0IdwJH1+wpf7Qe/mOD0bd4zvvDXrJxM6oITOwexzeYRpLVrgByjQ83Hp4bpOENkAABtISURBVMPj39tOPLqW4S1a8s9SqnNz0/8+o5OjVv4GfAk0AbuB37S0fruE3OPRfru904XYCnFuwB78ozkQ1ZlZl0Cc7fbkOkrbMpkhmuajaU5OqMM32kUVvX+l4ot+KEKgt/6WHtqfxPkOX/YtPfQ2hurZuHVRUcjVYkYeRUcRZEIHlGA95rV/ltKE4ruPI/TsZtdIS9NJJxmvY/DoDzg14fYeYVqwIz5dDYhOFfK2Tu0S8qqqLhPi+JZzblwf+WF7gW6wF+gDOUfoLzlGV+Fqt/C2JOTRIltSYohgvJhapYx4e1Mco0MuzZt0ypTECUg2mxHtk5vbcpvH4NGrmKJ301d/H8dib+k6fM3RET/E6JCwtoaLCplLtHBWVWn9OiVxRdffLLpKaT1kSOsGkNttTEOGGJ+rcEUYIOH35gcl04JJcOloQGS+kHs8wV91e4TYsJIjY6dN14wZb3owwjVjCPWX9IkQ5+g4aTN0Kd4NFB4SabOFxNXtNsIho5N3xo831jEFzUwYMr9jhkOG31xud6TlnJsbmVXZklUbnY0ZHjZo/hkkEzs/YICxT5fLtNgHtEnU91AYFPRwwRaLvHsQL2N6W7/SuELrRwXvlfAnzkQhx9GJa263cb+OwaP3UJjQMjdnp5sBkflCrrXWHo/eXzRefxvmI6/P7a0POeL7yOMJsVVTuKCWlkYuU8oQNdPCSPSYZmapmZmeLa1rrh+9LNxqVcoQ1GS+F29ZdBkB0x0T/jmeuyX8XJh/UlOmGMlMOzhJ15OblKDvp5deYHdlXJKG0DGi7+Fod0rQWuZUPQZPhFGkdWTKfjzxjkdFhSHmjdji7su0+NPNgMgOIdeJXQXx/L6myLRmVSql9dCh8R/RlDKm8G0oFZniG22Ru1zJH09HhcpqqzWedRSewm/WlzHPi3nOoi3p8CxbMB5n6zgqoZUe4UfPO1rvcLnjnh8R9uzDfNpsScRfp0SfdJLxO43OKG7P/WD+Zsfg0bs5Lu69+HyRK+3us6wR8kTCFc/va7OFfMktiXlBQWQdFXOaMiUyRT+ea8PEtESTsQasxmpxS7bwWLxsuPDrY8blhv+ZjsGIPAqPgElkoX+Wf6oel+MJPgmMH5++vkuhfYQbYVW44or4s5TG/DbNYnHtNWKqqkK/9zF4dBP2mH37Ie1usqwRcq1bdxVE+31NMYkW8JycyPTyvLzQMtPVkex+uyvhoh5d7yZeCnT4dXiEaTFRL9GCbnZsxXtSiudGEjIHj8cwtJSK7ICMFvF4xldpacc6w6Of4hNZ5juPLkqrSp5ZJeSJaElUTKvO7MSLd3FMl0A6XbhMINGfZ7zO1XAfvPkDWsUUvY/eLVrodfSOCTVrqayukN54PKGIqGhL3LzuHzAo4VN0Ry1ysw2mTihl3It+VFx/eSLN6GqyXsjjXdRE/t5UX4xsI9oyMq3w1p6Yop+S4oWFRbtbzA6v9lhhQvpgVjGcjTvu9W5CRVxnsxiezRbZB2XFU3Iy7p0qXGnR+Zn1Qp7oMUvcIZ1PWy2j8P6MaEtrDB5dS1Grgv4I04IWuVzjzGPKFONa+8Is4JCIE8wCLinpmmtsunlMl1+0mPua/1jMkONUkfVCLjHHqaU9PzSzXyI6Nt8U9NYKJO3oU9JqJ7SQfphht7UUxRXxcuXu8t9xdB9ZvLatYoq229PTIs+oolkt4XQa40tKadTU4HS2/Zw7nUZJW7NEaV1dqFTpxiYnA9hNFaGSpZpQ0SMNnLx3E73nFvIm83mAchoapABXJlBTA7c1VjKCrcF5unm6WrkZtbScAXHGk+3sNvl8oc9Xs5jXGIsNHSyydRFrmOWvZtu28vS7x+Kpe2dPndXZKWQHHk9syGj44260/9KMbki1tSQkJvyJ7e+l7rgRKnfhiogR7+r2RUdVrWJKnCcGmx5rSzxmcGdDZw8s0RaKi4v15s2bu3y/Qmbh9cL118OmTcbnMXh5jEvpzxdAbCnTL3oMYvM1jzC/xsnxxxuDiKSd5dQNMQeFaWwEJ15e8Y/FHmbpamALwymxGxa6w2E8XXf1tQsfwKKuDobs9/KzBWdjIxDR1ieZwosVT7JkSde2D7pgYIm2TGKRC8ni8cSO1dpSRTx/c4QBGN8TCz31hAcixPM9+7DFVAJNm2gktzvmKdAPesX4FGT+6cQWua3r/1MEIXmcTrjvPmP4LZMLeYG7cAX9quG+cwXcyAKepQyfz7CwhNQyYYJhZc8nvl+8giVsxElOjnGdHQ7jO+mAd1g5nj5Tgp/Ne+yy9fPYVu1NWbuiESEX0p7ycpgzB1Tz861ScDN3MxYPH3EqEHKvmI/AF7CW1xnN/v3GI/P8+car0PU4nbDt2mpcLABCndUAzwxx8XlpOW63Mdbu7benxq0SD9MldMNeF35sEfeYjQA7KhZQXZ0m91Y8M72zJ3GtCG3F7Q6V1TVLA5thiy11hH7AID0uxyNhqanE49G+5iqD4S6VWop0aWmqG5eYqqpQ5+ds3NoXdX81ofRZqmvvLcS1ImQqZqenOdr5okVG2OLcuZCTA1ewknLc7OVoIORqARjMx9T4zuJ2fyWHD8Py5Sk7jG6H+SS0bcaCmA5DP4qrWcwll6SyhS0zYULIpfcA5TxFpIvFjua3egF+f2js2ZQRT907exKLXGgLLRVHMlO9zSnRqDJmNmh0oTTJCO0czASbObhj6pf4Ufq3vd0pqRTaVsITzs62Rz5ZhHfUptoiFyEX0p6WsnbNZaab5bjjtN7QK3FUi1kS1XTPiMulczAHbmiKk4K/iinpE5WSBOHF+J6ME1u+udf4Lrt/Egm5uFaEtMfM2o3XEWYumzsX8vJgzx4o9b/A6h7TgPidoO/rwYxq9NLYSHo8FmchX30F93FVTLy4Hxv/43ClTVRKMjidcNNNRqf7F9MiOz4BRh5cj3NbdcraBxK1ImQI5o8pXjSD0wn9+xsp1qYwryxdyV24CBAZogiG33w9Z1GuqtMu3C0bqKyE0Wvihxr+dcgS5tc40yIqpa14vXDDaidXsSTmnvrupqoUtkyEXMgSzFhlU5hdLnh32t2cHRWiaMYB24HFgbk8PLaaK64wtiFhih2nshL2LaimMk6o4d246HF9GtYpSZKaGsNIuJ9y1jM+YlnvfZ9BdQqt8nj+lrZOwPnAh8AO4MbW1hcfudAZRHdehg+KnSgb1A96Pq6IYenEZ94+PB6tz1Ie3ZQg1FCpNMrYbAfhfTVGx2dkOOK/CgZ0+n1DZ/nIlVJ24D7gAmAocJlSamhHtysIbSXa/RIe2nYhL7CSWL+5AipZwIO+6fj9EAgQrKIotI2aGvitXoA9TqjhtWox+fmZ7cIK76s5fY6TDVFWed/6nawfW8m8eV3/VGeFa6UE2KG1/lRr3Qg8BlxkwXYFoUOUl4PbDaWlxmu9eyXPHx2/E/RyHuURpge/m8mCkyomFXqZzJrgZ9OP/Pq0pUy605k2GZsdwTQWZsyAP9jvIoCK8JX/Ti/k7aVeJk7sWjG3QshPAHaFfd7dPC8CpVS5UmqzUmrz3r17LditILROeTm88AIMG2YkFU3698qIOi0QK+Zaw7ZtKWpwBjPwnquwEzqfAGuYwgWry7NujACn07DKF3ADEOp/saGp4sYuj4Tqss5OrXW11rpYa13cp0+frtqtIADGj6qhwXBo3szdzMUdjGiBSDF/WE/nqqsIPiJLJ2jr7CmbTs+PI6NUAigW4sra8M4ZM+BPjrv5lJMi5o9nPeVUd+lTnRVC/gVwYtjnfs3zBCFtCE+3BiPlugI3fmLDEy/nUZ72l7F0qfG9//gP+MMf6PLH5Uxh9/RK+qx9FIiMUvkv+w28aXdmbXin0wmzZsFd3AxERkUt8s/j6zVdeLPE6wFtywTkAJ8CAwEH8Dbw45a+I1ErQipwu0Op/uY0Bo/eRd+4WaCvUxKxbvhgwIJBzbT4o/283G9atyiBYJYieIXxcTNYrS5DQGem6AM/BT4CPgF+39r6IuRCqoiuzWKKuRkyF696YvigB2Ck92ezOCXLO+74Kfi1FOm8vO5zjjwerWcNiTwXZnXEWUOsPQmJhNwSH7nW+jmt9WCt9Sla6zut2KYgdAYzZhip/OFsxMk4XuMjBgGxWaCvMZbZhJI9mpqy0+fbVo6oujFOCr5R1bA7nSOnE0Zf7+T/woL1zOqIF76/oEvyhCSzU+hWOJ1GCdyKCpgyBUpKjNK4G3HyIz7CSwkQ6e+0oXEzNyjmdnt2+nyTxeuFBRd7OfGzDcF5Zj/DPJayESeBgDH2ZXfALLN8j3Lhbw5HNJnMUyyf5+30vhURcqHb4XTCkiXw5JNw772GhW63Q24unK3e4HlKgdjEITcVzKaaQKD7hid6vUbn73lr5qGarfHwFPwHKAeMP8e6upQ1s0sxU/c92snTcazyOwI3cv31ndtRLkIudGvCs/X++lfIz4fJ9kRZoJqlzGVWoJp581JbWiNVLFgA1Q3TGcHbEfM/5SRu5m7AGIovL6/7PLWYdX6UgoXEVkc8h/UM31TduVFP8RznnT1JZ6eQrng8RodoSUnkEHLR9VmqmuuzdJcOPa2NqJ/oc2K+n407OPCHOXBHd8K8b8zBNKLP0Q4GxAyK0h6QgSUEoXXMwkjmWI1VuCLC66JHHKqo6B4hdlVVWlcf7Yor4o8wLSjimTDqT2dijigUHo5onqcKu7vD94kIuSAkQfiwcuY0H1dcy9wUMaWMgaGzUcRMYTKGbIs9B7UUBWPsKypS3drUY94/Y4itjvhlwYAObz+RkIuPXBDCCPd3mtxivxvveBcQP6X/GV1GUxNZ5zf3euGaa6DY52UxFcFOXzBDDeFatRibzehbmDEjdW1NF8z75027k9eiqiMeW7/TKNjeCYiQC0IY0UPHmQNVqLvuRrndQKyYX8BaXmc0gYAhfNmSxl9TY4y4FG/INo3iKtycMdfJHXfEDsHXXTHvnzlz4Pe2u4L1fMxzF1iwoHNukHhmemdP4loRMoG4Kebu0KjwsVmgp+oxeHRpaeg7mZym7vFovdIW27npBz0HtwzA0QJVVYa7qYrYfoXvBhW1e7uIj1wQrOGRCo/+gEFxxdyHCoqc2x0aUcYUvXQT9njtMSMwXu4XP0JlPq5uGZnSFswaLKD1JwyIOYfa5WrXdkXIBcECTJHLydH6dUpaDE8cOtSwysLD8qKFPdXHEu+PxuGIb0manbtTpqS23ZmCx2OEsc6OE46obbZ23QCJhFx85IKQJF6vUcr2/vsN3/nVRYmzQG9kAZXvGYNUAOTkGK+NjYbfOR1qdJsZiWZ7li+H2bPhT42V3Bg2eLLJFoqY41iJy5WS5mYcTqeRObzMVs6KsAQzhaHoVt4AIuSCkCThwtfUBO+8Y4wFuiJOFigYES21jGAMXi64wIjqcDhCHaipznw0IyzsduOP5v77Yfp7sSJuDhJxrVrMokXSqdkWnE743e/gCoyRqQLY8GOjUVmb+ipCLghJEi58NhtBa/sKVnJ3guHjRrCV1xjLD58y4hLNcgBmlEeqRx+64gojwuKCC+BKfzWuOCKuUVSwlI3KyZYtMlpSW7n7bhg/3hiZ6mxe4xbuYCLr8GLhP2I8f0tnT+IjFzIVs3PQ5TKSgGw243XaNMMXGp00E+43f7ivK6ZTsSWfeWd2jIbvOzfXSHqKl/DjRwXT7x0OowMvXXz8mYTHE5loZrO1L12fBD7yHOv+EgQh+zHdChMnGi4Wm80otlVXB7fYyiEAS5iLOapceBLNjK8W8MJZW/F6XsDpDI0jGggYrzU1oe2b/vjGRuMpoKNx2l6vsf3CQtiyBd56K7TvZf7pXE5oqDYg+HQxl6U8QDlKwU9/Cv/3f5E+fnGzJIfTCYsXG3kGfr/1RcVEyAWhjZi+8kDAyACtqzN+lHl5sKyhnO2BYdzHVYxga0Rdcw2UsZb94wr55LfzKTylnEDA2GYgAPv3hwT3889jO0bbK5rmn8LhwyF3kMmzlHEBa4FIEQdYEFaWVino29f4UzH/XFLt4880ysth2DDjWk6YYPGfYDwzvbMnca0ImUwil4jHo3VpaSjkMLxSYHQYXwD0qz1KY4ady801tpuXZ7gyrHBjVFWFioCFTy2FT+6a5tJud8h9lK5x8N0NErhWlI7+i+4CiouL9ebNm7t8v4JgFablHG1ZmdZvfb3xeTbVVHEzx2CMshBt9X5DITdTFbR8TWw2OPts+Ne/YOpUo8MsmbZAyIViPikAnHOOEWmTqE1me/zA69PceH5cHrE9yy1IoV0opWq11sUx80XIBcFavF4jJvuhhwzxDARadmEAfMQgZvIIGxNEMrhccNRRIWFevtx4HTHCGGassdEIIdQafD5jnzab4e5Zt84Y0ejqq+EpX6gdZlvMdnzEIK7kETbnGkO1WeGbF6wlkZCLj1wQLMbpNKYZMwxrdv9++Nk9L3BHoBIXC4Ixv+GCPpiPeZ2z+JK+/InbYiz0e+4x/NR2u+E39/uN+WYYpNYE/e2mbRYIhBJ9Ln+vkkNqETnUx+wbwEsJY3kDu93wrZjflQ7NzEDiyAWhk3A64aabjEGei4vNOGIPHzEIiM0GVcDxfEU1czlAT6oIlTzVOtTxaYo4GIJrCnd4bDvAGLys85/Dfy3Nx7l+ATlN8UX8eUoZyxvGMmWMXZouSUtCcnRIyJVSlyql3lVKBZRSMea+IHR3TJ+56UnciJMf8RF3hSUQRQs6QE8OcSMLaCCHb+0/YD6VKGWIdSIGDzZeq6jkG47mNc5iPOvJpyFi++H7vQsXF/JCcBtaw5VXRiYtCelPR10r24GpgNuCtghC1hEeqmizwY9+BO+9Z1jnTzOF+7iKIrZGfCfcYs7FT65/Py4WcB1/oV7nk4sPgCZyycWHQtNIDvb3Nbk0kk9jzLbM7Zl8xKnMZHmET14pwwqfMUMEPNPokEWutX5fa/2hVY0RhGwjPK0/Lw8eeADcbhgyBN5QTkaxhbF4eJXx1OMAQtZyuMsFII8mjuIAPamnJ/UcxXf05BA9qOcoDtCbg0ERjx7NxxTxvRxNOW5Oz/kYx3hnsNyAw2EMpiFWeGbSZT5ypVS5UmqzUmrz3r17u2q3gpBSzBFjol0VO3YYr3Y71OY6OVe9Sk8auAsX31MARLpAVBunaLfNN80CXlZUR05FOffdB2++abhSbDZYtAiWLBERz1RaFXKl1EtKqe1xpovasiOtdbXWulhrXdynT5/2t1gQMgyz09MsknX11UZYohlt8pvfGNZwbi78Xt3ND3IOUY6bTzmJRnKBSGFOZgI4RD7vMoRy3PyQOh6yl/OLXxiCXVcXcvlobXwWMpdWfeRa6/O6oiGC0B2oqQmFCYJhkZs+aTNc8fPPobq6nAcCRgjibKq5ifkU8g0BVEIfuQ1NAEUj+fyr9ErG1NxNY7O73HSfFBYa1QsLCyXdPpuQOHJB6ELMmiwNDaGCW6Y7w4w/93rhkUcMkdUaHgiUBwtXxcvfize/lMhY8/POg0suCSUPORzGoAdm9qe4VDKbDgm5UupiYBHQB3hWKbVVa11mScsEIQsxfeaJ0t7NdHtTZAsLQ+Jrs4XS7MM55ZSQzx0MK79Hj9CoRA4H/OlPsSMC1dUZLh8h8+mQkGutnwSetKgtgtAtMC3vaBKVrjUr5q1ZA5s2xX7vn/+MtMoDAXj6aUPI58yJDCcUd0p2Iq4VQUgToi1mMz3enAoL4wu51oZo+/2hzkuz5kr//pGum5aeBoTMRYRcENIEM+Y8kcVc3lx+ZdUqKCoyQgbD/d2rVsGLL0am7EdvI9HTgJDZSPVDQUgjEpXHTWZd0zXT0GD4yf/615D4C9mBlLEVhG5AW/4IhMxDytgKQjdAXCfdEyljKwiCkOGIkAuCIGQ4IuSCIAgZjgi5IAhChiNCLgiCkOGIkAuCIGQ4KYkjV0rtBT5r59ePAb6xsDmpRI4l/ciW4wA5lnSlI8dyktY6ZkCHlAh5R1BKbY4XEJ+JyLGkH9lyHCDHkq50xrGIa0UQBCHDESEXBEHIcDJRyKtT3QALkWNJP7LlOECOJV2x/FgyzkcuCIIgRJKJFrkgCIIQhgi5IAhChpO2Qq6UOl8p9aFSaodS6sY4y/OUUo83L39DKTWg61uZHEkcy0yl1F6l1NbmaXYq2tkaSqllSqk9SqntCZYrpdT/NB/nO0qpkV3dxmRI4jgmKKW+Dbsef+zqNiaLUupEpdQrSqn3lFLvKqWui7NOplyXZI4l7a+NUipfKbVJKfV283HcFmcda/VLa512E2AHPgFOBhzA28DQqHWuApY2v/8V8Hiq292BY5kJ/DXVbU3iWMYDI4HtCZb/FHgeUMAY4I1Ut7mdxzEBeCbV7UzyWI4DRja/7w18FOf+ypTrksyxpP21aT7PvZrf5wJvAGOi1rFUv9LVIi8BdmitP9VaNwKPARdFrXMR8Ejz+yeAiUop1YVtTJZkjiUj0FqvB/a1sMpFwHJtsBE4Sil1XNe0LnmSOI6MQWv9pdb6reb3B4D3gROiVsuU65LMsaQ9zef5YPPH3OYpOqrEUv1KVyE/AdgV9nk3sRc0uI7W2gd8CxR2SevaRjLHAnBJ82PvE0qpE7umaZaT7LFmAs7mR+PnlVI/TnVjkqH58XwEhgUYTsZdlxaOBTLg2iil7EqprcAe4EWtdcJrYoV+pauQdzf+DxigtT4DeJHQP7WQGt7CqGkxHFgErElxe1pFKdULWAVcr7X+LtXt6QitHEtGXButtV9rXQT0A0qUUqd35v7SVci/AMKt0n7N8+Kuo5TKAY4E6rqkdW2j1WPRWtdprRuaPz4AjOqitllNMtct7dFaf2c+GmutnwNylVLHpLhZCVFK5WII36Na69VxVsmY69LasWTatdFa7wdeAc6PWmSpfqWrkL8JDFJKDVRKOTA6A56OWudp4Irm9z8HXtbNPQdpRqvHEuWvnIzhG8xEngZmNEdJjAG+1Vp/mepGtRWlVF/TX6mUKsH4naSjkUBzOx8E3tda/1eC1TLiuiRzLJlwbZRSfZRSRzW/LwB+AnwQtZql+pXT3i92Jlprn1LqGuAFjKiPZVrrd5VSfwY2a62fxrjgK5RSOzA6rn6VuhYnJslj+f+UUpMBH8axzExZg1tAKfU3jKiBY5RSu4FbMTpy0FovBZ7DiJDYARwCrkxNS1smieP4OTBPKeUD6oFfpamRADAWuBzY1uyTBbgZ6A+ZdV1I7lgy4docBzyilLJj/NH8P631M52pX5KiLwiCkOGkq2tFEARBSBIRckEQhAxHhFwQBCHDESEXBEHIcETIBUEQMhwRckEQhAxHhFwQBCHD+f8BUUP+YNW1bqwAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "utFkv4F3NULQ"
      },
      "source": [
        "Since we have initialized the variables of the neural network randomly, it's prediction is also random. In order to fit the model we need to minimize the expected mean squared error over all input-ouput pairs in our training data set. For this we need to create a function, that performs a training step when provided with the model, an optimizer and a batch of input-ouput pairs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "o4zFN0-kOdu7",
        "colab": {}
      },
      "source": [
        "\"\"\" For training we need to implement a function that executes one training step. Fill in the missing code pieces for this function.\"\"\"\n",
        "\n",
        "def train_step(model, optimizer, x, y):\n",
        "    with tf.GradientTape() as tape:\n",
        "        tape.watch(model.trainable_variables)\n",
        "        y_pred = model.__call__(x) # Compute a prediction with \"model\" on the input \"x\"\n",
        "        loss_val = tf.reduce_mean((y - y_pred)**2) # Compute the Mean Squared Error (MSE) for the prediction \"y_pred\" and the targets \"y\"\n",
        "    grads = tape.gradient(loss_val, model.trainable_variables)\n",
        "    optimizer.apply_gradients(zip(grads, model.trainable_variables))\n",
        "    return loss_val"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Ax-Kfd-tOm7I"
      },
      "source": [
        "This function uses the GradientTape to record the operations for which gradients have to be calculated. In our case this is the forward pass through our model and the computation of the loss function. After these operations are recoded we can get their gradients and apply these through the use of an optimizer. Finally we return the loss value in order to print it.\n",
        "\n",
        "With the training step function defined we now need to choose a suitable optimizer. Tensorflow offers a wide variety of optimizers but in this exercise we will use the RMSprop optimizer."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "PahsqMscPcKD",
        "colab": {}
      },
      "source": [
        "opt = tf.optimizers.RMSprop(learning_rate)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "8_WjHrAwQB9e"
      },
      "source": [
        "We now have everything we need to start training the model. For this we repeatedly sample a batch of input-output pairs from our training data set and use the train_step function to minimize the loss function over this batch. We repeat this until we have iterated over the complete training data set once. After this we compute the loss on the validation data set, print it and repeat with another epoch until we have reached $N\\_epochs$ epochs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "x3LqS3d_QrX6",
        "outputId": "7d7081b3-699b-4753-da95-206925c7b169",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "\"\"\" We can now use the train_step function to perform the training. Fill in the missing code parts.\"\"\"\n",
        "\n",
        "epoch = 0\n",
        "train_iters = 0\n",
        "train_loss = 0.0\n",
        "\n",
        "for x_t, y_t in train_ds:\n",
        "    train_loss += train_step(mdl, opt, x_t, y_t) # Perform a training step with the model \"mdl\" and the optimizer \"opt\" on the inputs \"x_t\" and the corresponding targets \"y_t\"\n",
        "    train_iters += 1\n",
        "    if train_iters==(N_train_samples/batch_size): # An epoch is completed\n",
        "        validation_loss = 0.0\n",
        "        for x_v, y_v in validation_ds:\n",
        "            y_pred = mdl.__call__(x_v)# Compute a prediction with \"mdl\" on the input \"x_v\"\n",
        "            validation_loss += ((y_v-y_pred)**2)# Compute the Mean Squared Error (MSE) for the prediction \"y_pred\" and the targets \"y_v\"\n",
        "        \n",
        "        #print(\"Epoch: {} Train loss: {:.5} Validation loss: {:.5}\".format(epoch, train_loss/train_iters, validation_loss/N_validation_samples))\n",
        "        print(\"------ Epoch: \",epoch)\n",
        "        print(\"Train loss: \",(train_loss/train_iters).numpy())\n",
        "        print(\"Validation loss: \",(validation_loss/N_validation_samples).numpy())\n",
        "        \n",
        "        train_iters = 0\n",
        "        train_loss = 0.0\n",
        "        epoch += 1    \n",
        "    if (epoch == N_epochs):\n",
        "        break"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "------ Epoch:  0\n",
            "Train loss:  0.5480865\n",
            "Validation loss:  [[0.16555005]]\n",
            "------ Epoch:  1\n",
            "Train loss:  0.14408645\n",
            "Validation loss:  [[0.08778594]]\n",
            "------ Epoch:  2\n",
            "Train loss:  0.11689215\n",
            "Validation loss:  [[0.09487397]]\n",
            "------ Epoch:  3\n",
            "Train loss:  0.114846244\n",
            "Validation loss:  [[0.07270131]]\n",
            "------ Epoch:  4\n",
            "Train loss:  0.10200283\n",
            "Validation loss:  [[0.06315949]]\n",
            "------ Epoch:  5\n",
            "Train loss:  0.088966414\n",
            "Validation loss:  [[0.08347025]]\n",
            "------ Epoch:  6\n",
            "Train loss:  0.07931683\n",
            "Validation loss:  [[0.07290609]]\n",
            "------ Epoch:  7\n",
            "Train loss:  0.06937425\n",
            "Validation loss:  [[0.04403754]]\n",
            "------ Epoch:  8\n",
            "Train loss:  0.058115058\n",
            "Validation loss:  [[0.04531762]]\n",
            "------ Epoch:  9\n",
            "Train loss:  0.04615892\n",
            "Validation loss:  [[0.04111396]]\n",
            "------ Epoch:  10\n",
            "Train loss:  0.039047368\n",
            "Validation loss:  [[0.04159105]]\n",
            "------ Epoch:  11\n",
            "Train loss:  0.036418837\n",
            "Validation loss:  [[0.01500163]]\n",
            "------ Epoch:  12\n",
            "Train loss:  0.03041168\n",
            "Validation loss:  [[0.04595648]]\n",
            "------ Epoch:  13\n",
            "Train loss:  0.03056726\n",
            "Validation loss:  [[0.01454838]]\n",
            "------ Epoch:  14\n",
            "Train loss:  0.027205124\n",
            "Validation loss:  [[0.0264213]]\n",
            "------ Epoch:  15\n",
            "Train loss:  0.028443454\n",
            "Validation loss:  [[0.02618033]]\n",
            "------ Epoch:  16\n",
            "Train loss:  0.026550619\n",
            "Validation loss:  [[0.06786799]]\n",
            "------ Epoch:  17\n",
            "Train loss:  0.026484208\n",
            "Validation loss:  [[0.01906609]]\n",
            "------ Epoch:  18\n",
            "Train loss:  0.026351469\n",
            "Validation loss:  [[0.04241015]]\n",
            "------ Epoch:  19\n",
            "Train loss:  0.027547918\n",
            "Validation loss:  [[0.02707507]]\n",
            "------ Epoch:  20\n",
            "Train loss:  0.024229506\n",
            "Validation loss:  [[0.04403067]]\n",
            "------ Epoch:  21\n",
            "Train loss:  0.027114354\n",
            "Validation loss:  [[0.01208668]]\n",
            "------ Epoch:  22\n",
            "Train loss:  0.026168779\n",
            "Validation loss:  [[0.02138912]]\n",
            "------ Epoch:  23\n",
            "Train loss:  0.022750309\n",
            "Validation loss:  [[0.01477526]]\n",
            "------ Epoch:  24\n",
            "Train loss:  0.025942568\n",
            "Validation loss:  [[0.01614882]]\n",
            "------ Epoch:  25\n",
            "Train loss:  0.02195125\n",
            "Validation loss:  [[0.01465194]]\n",
            "------ Epoch:  26\n",
            "Train loss:  0.023497215\n",
            "Validation loss:  [[0.02303959]]\n",
            "------ Epoch:  27\n",
            "Train loss:  0.022683654\n",
            "Validation loss:  [[0.03014282]]\n",
            "------ Epoch:  28\n",
            "Train loss:  0.024891526\n",
            "Validation loss:  [[0.04866017]]\n",
            "------ Epoch:  29\n",
            "Train loss:  0.023918385\n",
            "Validation loss:  [[0.02161884]]\n",
            "------ Epoch:  30\n",
            "Train loss:  0.025868358\n",
            "Validation loss:  [[0.0144715]]\n",
            "------ Epoch:  31\n",
            "Train loss:  0.02266684\n",
            "Validation loss:  [[0.03710939]]\n",
            "------ Epoch:  32\n",
            "Train loss:  0.023384143\n",
            "Validation loss:  [[0.01303839]]\n",
            "------ Epoch:  33\n",
            "Train loss:  0.022637231\n",
            "Validation loss:  [[0.0137868]]\n",
            "------ Epoch:  34\n",
            "Train loss:  0.022130616\n",
            "Validation loss:  [[0.01364521]]\n",
            "------ Epoch:  35\n",
            "Train loss:  0.02164642\n",
            "Validation loss:  [[0.01733869]]\n",
            "------ Epoch:  36\n",
            "Train loss:  0.023996288\n",
            "Validation loss:  [[0.04619719]]\n",
            "------ Epoch:  37\n",
            "Train loss:  0.024021773\n",
            "Validation loss:  [[0.03656345]]\n",
            "------ Epoch:  38\n",
            "Train loss:  0.023773655\n",
            "Validation loss:  [[0.02434142]]\n",
            "------ Epoch:  39\n",
            "Train loss:  0.022916455\n",
            "Validation loss:  [[0.01255531]]\n",
            "------ Epoch:  40\n",
            "Train loss:  0.02492857\n",
            "Validation loss:  [[0.02053205]]\n",
            "------ Epoch:  41\n",
            "Train loss:  0.023133982\n",
            "Validation loss:  [[0.02636188]]\n",
            "------ Epoch:  42\n",
            "Train loss:  0.022337854\n",
            "Validation loss:  [[0.0106892]]\n",
            "------ Epoch:  43\n",
            "Train loss:  0.022357272\n",
            "Validation loss:  [[0.01239729]]\n",
            "------ Epoch:  44\n",
            "Train loss:  0.025446074\n",
            "Validation loss:  [[0.01535382]]\n",
            "------ Epoch:  45\n",
            "Train loss:  0.022888279\n",
            "Validation loss:  [[0.02760892]]\n",
            "------ Epoch:  46\n",
            "Train loss:  0.024389036\n",
            "Validation loss:  [[0.03229044]]\n",
            "------ Epoch:  47\n",
            "Train loss:  0.020764587\n",
            "Validation loss:  [[0.01245769]]\n",
            "------ Epoch:  48\n",
            "Train loss:  0.022120457\n",
            "Validation loss:  [[0.02420781]]\n",
            "------ Epoch:  49\n",
            "Train loss:  0.023797015\n",
            "Validation loss:  [[0.01787792]]\n",
            "------ Epoch:  50\n",
            "Train loss:  0.020442052\n",
            "Validation loss:  [[0.04377156]]\n",
            "------ Epoch:  51\n",
            "Train loss:  0.022141803\n",
            "Validation loss:  [[0.03609345]]\n",
            "------ Epoch:  52\n",
            "Train loss:  0.02275333\n",
            "Validation loss:  [[0.0212937]]\n",
            "------ Epoch:  53\n",
            "Train loss:  0.023647467\n",
            "Validation loss:  [[0.01942215]]\n",
            "------ Epoch:  54\n",
            "Train loss:  0.0223476\n",
            "Validation loss:  [[0.0207578]]\n",
            "------ Epoch:  55\n",
            "Train loss:  0.021950258\n",
            "Validation loss:  [[0.01497361]]\n",
            "------ Epoch:  56\n",
            "Train loss:  0.02202189\n",
            "Validation loss:  [[0.0142233]]\n",
            "------ Epoch:  57\n",
            "Train loss:  0.022960844\n",
            "Validation loss:  [[0.01177481]]\n",
            "------ Epoch:  58\n",
            "Train loss:  0.021084957\n",
            "Validation loss:  [[0.01124904]]\n",
            "------ Epoch:  59\n",
            "Train loss:  0.022418581\n",
            "Validation loss:  [[0.01153344]]\n",
            "------ Epoch:  60\n",
            "Train loss:  0.022332327\n",
            "Validation loss:  [[0.02984803]]\n",
            "------ Epoch:  61\n",
            "Train loss:  0.021765344\n",
            "Validation loss:  [[0.01369493]]\n",
            "------ Epoch:  62\n",
            "Train loss:  0.024280556\n",
            "Validation loss:  [[0.02336219]]\n",
            "------ Epoch:  63\n",
            "Train loss:  0.022964971\n",
            "Validation loss:  [[0.04123324]]\n",
            "------ Epoch:  64\n",
            "Train loss:  0.024019377\n",
            "Validation loss:  [[0.02366165]]\n",
            "------ Epoch:  65\n",
            "Train loss:  0.021230645\n",
            "Validation loss:  [[0.01542195]]\n",
            "------ Epoch:  66\n",
            "Train loss:  0.019919615\n",
            "Validation loss:  [[0.05205909]]\n",
            "------ Epoch:  67\n",
            "Train loss:  0.02202274\n",
            "Validation loss:  [[0.01102713]]\n",
            "------ Epoch:  68\n",
            "Train loss:  0.02138336\n",
            "Validation loss:  [[0.03120021]]\n",
            "------ Epoch:  69\n",
            "Train loss:  0.021456433\n",
            "Validation loss:  [[0.01527438]]\n",
            "------ Epoch:  70\n",
            "Train loss:  0.02133151\n",
            "Validation loss:  [[0.01692958]]\n",
            "------ Epoch:  71\n",
            "Train loss:  0.022431567\n",
            "Validation loss:  [[0.02830462]]\n",
            "------ Epoch:  72\n",
            "Train loss:  0.02214402\n",
            "Validation loss:  [[0.0315087]]\n",
            "------ Epoch:  73\n",
            "Train loss:  0.020781348\n",
            "Validation loss:  [[0.01176434]]\n",
            "------ Epoch:  74\n",
            "Train loss:  0.021956945\n",
            "Validation loss:  [[0.04363904]]\n",
            "------ Epoch:  75\n",
            "Train loss:  0.021245213\n",
            "Validation loss:  [[0.0335107]]\n",
            "------ Epoch:  76\n",
            "Train loss:  0.021062663\n",
            "Validation loss:  [[0.01836992]]\n",
            "------ Epoch:  77\n",
            "Train loss:  0.020127214\n",
            "Validation loss:  [[0.01640853]]\n",
            "------ Epoch:  78\n",
            "Train loss:  0.022571087\n",
            "Validation loss:  [[0.02881464]]\n",
            "------ Epoch:  79\n",
            "Train loss:  0.022112299\n",
            "Validation loss:  [[0.01769202]]\n",
            "------ Epoch:  80\n",
            "Train loss:  0.019564312\n",
            "Validation loss:  [[0.02341166]]\n",
            "------ Epoch:  81\n",
            "Train loss:  0.02261153\n",
            "Validation loss:  [[0.02144204]]\n",
            "------ Epoch:  82\n",
            "Train loss:  0.020740177\n",
            "Validation loss:  [[0.03121405]]\n",
            "------ Epoch:  83\n",
            "Train loss:  0.021785589\n",
            "Validation loss:  [[0.01504779]]\n",
            "------ Epoch:  84\n",
            "Train loss:  0.01978653\n",
            "Validation loss:  [[0.02886174]]\n",
            "------ Epoch:  85\n",
            "Train loss:  0.022784056\n",
            "Validation loss:  [[0.0101482]]\n",
            "------ Epoch:  86\n",
            "Train loss:  0.021775091\n",
            "Validation loss:  [[0.02608629]]\n",
            "------ Epoch:  87\n",
            "Train loss:  0.020807646\n",
            "Validation loss:  [[0.01261684]]\n",
            "------ Epoch:  88\n",
            "Train loss:  0.022056356\n",
            "Validation loss:  [[0.02765949]]\n",
            "------ Epoch:  89\n",
            "Train loss:  0.021733057\n",
            "Validation loss:  [[0.02811474]]\n",
            "------ Epoch:  90\n",
            "Train loss:  0.022714144\n",
            "Validation loss:  [[0.02565669]]\n",
            "------ Epoch:  91\n",
            "Train loss:  0.01955325\n",
            "Validation loss:  [[0.01072596]]\n",
            "------ Epoch:  92\n",
            "Train loss:  0.020222722\n",
            "Validation loss:  [[0.0154796]]\n",
            "------ Epoch:  93\n",
            "Train loss:  0.021043777\n",
            "Validation loss:  [[0.02091678]]\n",
            "------ Epoch:  94\n",
            "Train loss:  0.020951312\n",
            "Validation loss:  [[0.05570944]]\n",
            "------ Epoch:  95\n",
            "Train loss:  0.02305933\n",
            "Validation loss:  [[0.01562858]]\n",
            "------ Epoch:  96\n",
            "Train loss:  0.019908767\n",
            "Validation loss:  [[0.01415233]]\n",
            "------ Epoch:  97\n",
            "Train loss:  0.021960447\n",
            "Validation loss:  [[0.03143197]]\n",
            "------ Epoch:  98\n",
            "Train loss:  0.02060386\n",
            "Validation loss:  [[0.02024469]]\n",
            "------ Epoch:  99\n",
            "Train loss:  0.020840546\n",
            "Validation loss:  [[0.01010519]]\n",
            "------ Epoch:  100\n",
            "Train loss:  0.019574802\n",
            "Validation loss:  [[0.02094823]]\n",
            "------ Epoch:  101\n",
            "Train loss:  0.020947563\n",
            "Validation loss:  [[0.04095791]]\n",
            "------ Epoch:  102\n",
            "Train loss:  0.023360193\n",
            "Validation loss:  [[0.01317217]]\n",
            "------ Epoch:  103\n",
            "Train loss:  0.020148419\n",
            "Validation loss:  [[0.01872642]]\n",
            "------ Epoch:  104\n",
            "Train loss:  0.02095313\n",
            "Validation loss:  [[0.02463569]]\n",
            "------ Epoch:  105\n",
            "Train loss:  0.01944168\n",
            "Validation loss:  [[0.01010959]]\n",
            "------ Epoch:  106\n",
            "Train loss:  0.021210242\n",
            "Validation loss:  [[0.0336307]]\n",
            "------ Epoch:  107\n",
            "Train loss:  0.023458067\n",
            "Validation loss:  [[0.01572553]]\n",
            "------ Epoch:  108\n",
            "Train loss:  0.019449674\n",
            "Validation loss:  [[0.01816366]]\n",
            "------ Epoch:  109\n",
            "Train loss:  0.019956186\n",
            "Validation loss:  [[0.01835212]]\n",
            "------ Epoch:  110\n",
            "Train loss:  0.019249273\n",
            "Validation loss:  [[0.01660836]]\n",
            "------ Epoch:  111\n",
            "Train loss:  0.021930413\n",
            "Validation loss:  [[0.01350294]]\n",
            "------ Epoch:  112\n",
            "Train loss:  0.018078247\n",
            "Validation loss:  [[0.01650085]]\n",
            "------ Epoch:  113\n",
            "Train loss:  0.020852529\n",
            "Validation loss:  [[0.03471498]]\n",
            "------ Epoch:  114\n",
            "Train loss:  0.020622425\n",
            "Validation loss:  [[0.01317852]]\n",
            "------ Epoch:  115\n",
            "Train loss:  0.022586895\n",
            "Validation loss:  [[0.02027737]]\n",
            "------ Epoch:  116\n",
            "Train loss:  0.020024851\n",
            "Validation loss:  [[0.01104768]]\n",
            "------ Epoch:  117\n",
            "Train loss:  0.019564116\n",
            "Validation loss:  [[0.05425089]]\n",
            "------ Epoch:  118\n",
            "Train loss:  0.020646283\n",
            "Validation loss:  [[0.01208784]]\n",
            "------ Epoch:  119\n",
            "Train loss:  0.020235196\n",
            "Validation loss:  [[0.01228718]]\n",
            "------ Epoch:  120\n",
            "Train loss:  0.018940652\n",
            "Validation loss:  [[0.00960836]]\n",
            "------ Epoch:  121\n",
            "Train loss:  0.02161523\n",
            "Validation loss:  [[0.01096756]]\n",
            "------ Epoch:  122\n",
            "Train loss:  0.02120794\n",
            "Validation loss:  [[0.01269315]]\n",
            "------ Epoch:  123\n",
            "Train loss:  0.020853763\n",
            "Validation loss:  [[0.01558759]]\n",
            "------ Epoch:  124\n",
            "Train loss:  0.020653412\n",
            "Validation loss:  [[0.03313897]]\n",
            "------ Epoch:  125\n",
            "Train loss:  0.018349888\n",
            "Validation loss:  [[0.02085792]]\n",
            "------ Epoch:  126\n",
            "Train loss:  0.023163313\n",
            "Validation loss:  [[0.00944085]]\n",
            "------ Epoch:  127\n",
            "Train loss:  0.019652205\n",
            "Validation loss:  [[0.01119555]]\n",
            "------ Epoch:  128\n",
            "Train loss:  0.02142303\n",
            "Validation loss:  [[0.01845533]]\n",
            "------ Epoch:  129\n",
            "Train loss:  0.019999467\n",
            "Validation loss:  [[0.00982338]]\n",
            "------ Epoch:  130\n",
            "Train loss:  0.021326965\n",
            "Validation loss:  [[0.01975048]]\n",
            "------ Epoch:  131\n",
            "Train loss:  0.019496761\n",
            "Validation loss:  [[0.03009328]]\n",
            "------ Epoch:  132\n",
            "Train loss:  0.021499656\n",
            "Validation loss:  [[0.01988059]]\n",
            "------ Epoch:  133\n",
            "Train loss:  0.019684827\n",
            "Validation loss:  [[0.01915742]]\n",
            "------ Epoch:  134\n",
            "Train loss:  0.019972675\n",
            "Validation loss:  [[0.03417294]]\n",
            "------ Epoch:  135\n",
            "Train loss:  0.019697968\n",
            "Validation loss:  [[0.02054704]]\n",
            "------ Epoch:  136\n",
            "Train loss:  0.02038511\n",
            "Validation loss:  [[0.0181012]]\n",
            "------ Epoch:  137\n",
            "Train loss:  0.020288862\n",
            "Validation loss:  [[0.01437132]]\n",
            "------ Epoch:  138\n",
            "Train loss:  0.020361198\n",
            "Validation loss:  [[0.02715241]]\n",
            "------ Epoch:  139\n",
            "Train loss:  0.01992552\n",
            "Validation loss:  [[0.00913886]]\n",
            "------ Epoch:  140\n",
            "Train loss:  0.020148337\n",
            "Validation loss:  [[0.013283]]\n",
            "------ Epoch:  141\n",
            "Train loss:  0.020003937\n",
            "Validation loss:  [[0.04064032]]\n",
            "------ Epoch:  142\n",
            "Train loss:  0.020244425\n",
            "Validation loss:  [[0.01289205]]\n",
            "------ Epoch:  143\n",
            "Train loss:  0.018619368\n",
            "Validation loss:  [[0.02503858]]\n",
            "------ Epoch:  144\n",
            "Train loss:  0.020342939\n",
            "Validation loss:  [[0.01408796]]\n",
            "------ Epoch:  145\n",
            "Train loss:  0.020708026\n",
            "Validation loss:  [[0.02148728]]\n",
            "------ Epoch:  146\n",
            "Train loss:  0.02017081\n",
            "Validation loss:  [[0.01108074]]\n",
            "------ Epoch:  147\n",
            "Train loss:  0.020076625\n",
            "Validation loss:  [[0.01403083]]\n",
            "------ Epoch:  148\n",
            "Train loss:  0.02082638\n",
            "Validation loss:  [[0.0204622]]\n",
            "------ Epoch:  149\n",
            "Train loss:  0.02051671\n",
            "Validation loss:  [[0.00945168]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "DduaR_pCQ0k-"
      },
      "source": [
        "After completion of the training process we use the test data set to test the models generalization to unseen data."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "3ofClNnHRAUy",
        "outputId": "edb0356c-554b-4e04-d494-3e21d2cfe34b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "test_loss = 0\n",
        "for x_t, y_t in test_ds:\n",
        "    y_pred = mdl.__call__(x_t) # Compute a prediction with \"mdl\" on the input \"x_t\"\n",
        "    test_loss += (y_t - y_pred)**2 # Compute the Mean Squared Error (MSE) for the prediction \"y_pred\" and the targets \"y_t\"\n",
        "test_loss = test_loss.numpy()\n",
        "print(\"Test loss: {:.5}\".format(test_loss[0][0]/N_test_samples))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Test loss: 0.01127\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "NzJ7wOZmRbez"
      },
      "source": [
        "After we have verified that our model achieves a similar loss on the test as on the validation and training data set, we can conclude that our model is not overfitting or underfitting and generalizes to unseen data. We can now predict on the inputs again and plot the results."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "tyvFN03bR8ez",
        "outputId": "e2578289-892e-417e-dc02-678331189192",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        }
      },
      "source": [
        "\"\"\" Now we want to plot the prediction after training. Predict on the variable \"x\" again. \"\"\"\n",
        "\n",
        "y_pred = mdl.__call__(x)# Compute a prediction on the variable \"x\"\n",
        "plt.plot(x, y,'b.')\n",
        "plt.plot(x, y_true,'r.')\n",
        "plt.plot(x, y_pred.numpy(),'g.')\n",
        "plt.legend([\"Observation\", \"Target\", \"Prediction\"])\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOyde3wU1fn/32c3Fy6CSKCCAsYqIGggUQoMN9fiDS9A5duvrfILinURhIqtBlBbtVWRqJW2KhIUJIV+a6st4B1BI5cMIlejIIgWIUoUgsg9l93z++Ps7M5sNgmQTbKbPe8X+2Jn5uzM7GbmM895nuc8R0gp0Wg0Gk3Tx9XYJ6DRaDSahkELvkaj0SQIWvA1Go0mQdCCr9FoNAmCFnyNRqNJEJIa+wSqo127djI9Pb2xT0Oj0WjiivXr1++TUraPtC1mBT89PZ1169Y19mloNBpNXCGE+Kq6bdqlo9FoNAmCFnyNRqNJELTgazQaTYKgBV+j0WgSBC34Go1GkyBowddoNJoEQQu+RqNJCEwTpk9X/ycqMZuHr2kYTBMKCsDjAcM4+e0aTTxgmjB0KJSXQ0oKLF+emNezFvw4I5oCXNtNoG8STVOhoEBdxz6f+r+gIHQtJ5JRowU/joi2ANd0E5zIdo0mXkhLA5cLpFT3jsej1ieaUaN9+HFEJAGuCx6PusjdbudNcKLbNZp4wDRh8mR137hcMHNmSNSjfU/FOtrCjyMsAbasEbsAn0q31DCURVPd52rbrtHEA5ao+/0gBJSWhrbVdE81RbTgxxHVCfDJdkvDHw41ta1tu0YT69Qk6olm1GjBjzMsAbZSzDyek/O1J5rPUqM5kZ5sotwDWvBjkBNJlbSL9syZJ94tDX845OefunWTSNkNmvgmkUS9JrTgNzLhonkiFni4aJeWnni31N69dbth3jyorDx5a1/3FDSa+EMLfiNgmsqy3rIFVq9WqWKpqSHRrs09E8knabdgarK87d3bXbtgzhx1rLIyeOgh9ToR4dYpm5p4xbo/0tJg40a1LitLGU5NvbeqBb8BsYT+xRehosK5rawsJNK1uWfsop2WFkolO9Eegj0OMH++OrbfD8uWwcqVykVU28WfaNkNmqZBXh5MnKh6tVI6t7lcIcOrqYq+Fvx6IJKFXdOFBupis9rX5p6xWyiTJyvRTUqCW29V2y0BP35cPWBqunivugrWrYOvv1afKSuDCRPU++Tk6i338POEUBD5ZDKENJqGwjThzjvVPRgJ6/qvS1wr5pFSxuTrkksukXWhsFDKxx5T/5/K9roct3lzKd1u9X9hoXolJUmppL7qy+2WcvbsEz9va//JyVK6XKH9CKHW2/edlBR5X4WFUqakVD0XIZzLd9xx4t/Z5VLnZH2X2n6b2bPr52+g0Uiprqs77lAv676y3y/VvZKTnfdvvAGsk9XoapO08OtaI6YuVmh1I/f8/lAblyu0zu2G554Dr/fE3DH5+cpyty5Pa7i49fL5nO0rK0NWvv17FRRUdStB5N5HJML3ZfUq/H5lRWVk1BxsLitT7ayh7k25G61peEwTLrtMXWcAc+fCX/+qXDZlZeq++c1v4OBB2LABPvpIXYtChHrhTTE21SQFvy41YuqafeLxKBG3xNxyd9gvtOuug9deC33GGvlX23mbprpwLVFOToa//EUFnubOVQIeSbDnzlVBKcv9k5ICkyZVL+5CqP9TUiA7O3Rsu/vG+o2SkmDYsNBnQH332oLNLpf6nn6/87tql48mGlj3kkV5ubpPahu46HaHRL8pxqaapOBXJ7r27dUFHKORfWKJn/V/JH/3O+9UPX5tgdCCgpAFL4Ty2Xu9ajk7W1ny8+aFLHerV+HzwauvOq3rf/9b7SOS6PfuHTp+QQEUFTkfFmPGhPbl88HixUr4IZRxdCLBZvs+PR6d6qmJHh6Puibtvdh589R9Mm2as234dWll7mRnN73rLyqCL4SYC1wHfCelvCjCdgH8GbgGOArcIqXcEI1jR6KoyCmM4dQUGK1r9klBQahLWFHhTHW0H2f5ciXQ1vla78MzZOwWb1qa+j5WNoFlfVvfyTBCwm9P+UxJgVGjVAaO5Xr54ouQSyg1VVn8BQWwfj1s2qT2uXat2u52O63xkhLn7yql2nb77dClS9XMofDf3lqXkeH8G0yfrlM9NfVHRYUzIAshkS8tdRohbnfoc03qGqzOuX8yL2AIcDHwSTXbrwHeAgTQH/iwtn2eatA2PEDqcqlgTXVtIwUN6xLQLSyUMjXVGQRNTQ3ty9r37NkqKBQeJE1NDQUzc3JCQVi3W72EqD0waj9+eEC4b1/nb3NX30K5b8hIKTt0kIdatpfTyakSxHK51G8qRCgwHH7eKSlq/7Nnq/Or7Twj/daRAt4azanw2GNVr1G3W90bbreUQ5IL5fOuO+Rz3CH7UxhMOAhPgojH65AagrZRy6oB0msQ/NnAL23L24CONe3vVAU/PBJviY4lsuHiUltmSW2EPxyqy34ZOdIpaJaAVpclUFM2QXUPscJCKa+8Uu23P4XyfYbIA5wmj7uby6MprWVZ6mlyP63kIZrLQ7SQP9BS+m079gdex3HLQzQPtG0hD9NC/uBqLQ9wmvyCdPkrZlfJ7LEyIcKzkVyuE8/asf4uOnNHU1ciGT4jR0r5ODnyAKcFr3XrVUxHeTuzq9x3bnf1BmOsUpPgN5QP/2xgt225OLBuj72REMILeAG6dOlySgfyeJwB0rvvVt00y5VhuTDGjDmxzJKaiORzzs+PnP3y2mvQoUPIZeFyVe9DD+bqdzJh6FT4UREIH7j8IF34XRU8VAmPT0+mwl8BlRUklVXil+DKhGaXwGYBw31Q6QYkJPuP4ReQdhTuWwW3R3CoCdWUFHykcMy5MRAPaM1h8hjHM9zJQU7nW37Ec+7J/L9sLwUFzmwkUMsTJ9aetWO5cKZNa2JdaE2jYBgqoeHFF+Gss+DJbnmc9cw0mrE/2MbySkrgLPYwm3GM9L/KcPc7uFzq2o1GGfJYIqaCtlLKPCAPoE+fPieYIOgk3D9vr4UNIT80hAK7oERn8mS4+OITD9ZEKkRmz6Kx4/cr37c9PjBpEjz9dOABMXQKZM0DdxnSVQEuH7jLQ1dlGOU+KPfZRDklQqPk0Fur5eFUGHc9TBwGzSuh0gUIcPtAChBSrb9lE8xYHvnY6qFQSTtKaUcpz1aOQ1x1D92yhvJOUg4rKgzHb+DznXiJCI0mGliTnpSXw/+sn8KPfbnBbeG3lF34h7GUT33dWHz9fHx9jTqVIY9FGkrwvwY625Y7BdbVC+EB0pQUp4VvpRtmZSnL3udTIr12rXrNnXtiAcNwwQJnsHjwYBU4tfb/1lvK6rCCsouOTCG11Rwqyg8qgY9ENYJfJyRUJKlXENvD4RCQOwie7g9JfnUKrcugXzHkFMKA4iq7Qx46RNsVi3ifRexrdQ4vdbiPaV96kYGsnbS0qiNxDUMFqV99FTIz1QMzP79pZkdoGhbLGLvVl8c9KLG3C3skrB5uNz7nt4sH4c5ZFbwQTVMlX1g6Eq9JBUJGMkdPZUdCpAOvy8hZOtcCE1HB237AX6SUfWvaX58+feS6detO6VwiVaC0R+PDn9oPPQTvvhuyzIWARx+tmr4V6ThWdo2VMRPJxTN7dmhQx4iJJvuzpvLh1x9S5isL7UxSvbhH508U4kQeItUdU0JKJaT44OISeHw5GMWRP7aPtrwzZDpHb/Y6UjAty8iymKybyCIlJT5vJk3jYr/vi4rgbxNM3vcNxI0MXvJThkLeJVDmBrdfUOFS21pWwo8Ow+QPwWu5OzMzYePGKtdprNfcEUKsl1L2ibgtGoIvhPg/wAO0A74FHiRgM0opnw+kZT4DXI1Ky7xVSlmjmp+q4IcPohg7tqrFGOmBYB+VZwkOVO+vq657l5enLNZRo1SOfF6eqk3j6xjwx5+zovqZhGv4U6RWQLIfXBIqAm6YpMqAjx5I9oHfpVwylYH4gFu6OCySkNIN/mRIPgbuCqfghx+ztodBhHNsXg4/+cYp/vZmpW27MuL7+RRKAyHgJz9RrjMIVet0nELggWu55OLVX6ppOCINniosyyKLTQiU0P+1LxyL5Pq0TPsAmXvg2TfVtSxmz2Z6qZff/S4Ue7v88hOvKtsY1Lvg1wenKvjTpxP844D6w6ekqEFK1VnhlujXZq3b/8Djx4csd+siGDXKOZho5kyYOMOkYvBUSF8R+nANgptaHngeCGheAR0Ow10fRg6yHqUZEkEZKeQxjvuYQWoqvP++Otfx4+H5552fEVdMIblfHqSU0ywpiUpfJT4J0p+E2+Wn3F+GT0aIOoefd6Tzl9DqOHQ6DJPXKEtJ2prNIIf7mBFsHp7jb5GSoobBR+oVaDSRsN/3QsCjcgpTyWXOxTBtKOxvYWtck1FjXbB+mP0GZO9JZ+Or/40r331CCb5pwpC786js+ipsGQUb1FBUIaBZM5Wdk5cX6po98khk1439AnK7Q4OK7KNi7UO3XS6CkX1rhG/7X06h5Me5oQusGqFP9sHZB2HaqlB3MvyvIoGjNMef0oLi8h8xk8nMdXvp3Bl27gy1s1vHQ4ZErgxo/RbV1Rg6NnAKZM5DJB8Htw8py1V3osqOIpykjbZHYPp7TuHfTlduYT5rUAe2ftusLGdt8ldfVeWard/yj3+s3cWmSVzsbpe+fpOVDOD+oSoWFeREnPg428x+DbyX52COnBE3vc2aBD+msnSiQVFKHpXDxqk/2HlLoe0XsGwGUoZGidozdtLSIu/HCshabp45c0Ilg6+5pmrqpT+Q0G5gcnOnCcy65mM+6Rg4UDVC3/YoTF9e1RK2KHel4mrRHPNIL+4Tj7Mh1WDMmJAbRPjhq6+c52EviRDuKgmegqy5xhDLZqiXCH3f/efmsb3tTEoOfgvJhyGp3LlTUfV77m+pMoKmDVXf8/YNKiBWyAD+xs2MYUEwbmIX+/A0Wp3Bo6kNKztv8mSYvnYqcy+G3IGBjXaht95XJkFlC3BVqJRn+/VsXcsSxl0H8vVczmQktIlxpT8BmpyF329OP9Z+sza0QgKrc2DZDFwuGDQIVq0KiYnXG7Lcw5/ceXmR62dblnx/TKYzlQyKcOEjmUruGHmUhb1tjSNYFR0OwcMFVd00x0nhMKfxLR34M3cxP9nLtdcGPtPB6WoKD3QC9O0Lt90WGiY+fryzTY8e8Pnn6kGQlKTa2uMblpVkVeMMfgVbjwAgNxe2HTX59qKp7E9dBylHnSdSzQOu4yF4yPa9Tfry09QP8flCv7HbHSrVEA/+Uk3sYJqQPziPrN7jGHcdTiMkcB02F22Z1GM6f73F6zAq3OkmFVdMgA6BuiK2z7n88Jd5mfz6m40xHay1SCiXzqXP/YwV3y2qKrSfXwl/fwcIlSdOTlbiYrltwgO8drfOr8jjLv7MmXxLc47iwk9zQlk2VXyFEUQvXOjLXamU+VPYS1tmcB/uO7yUlMCiRVW/l+WbBxVrWLFC1cuxcLlg1ixnLRC/PySkSUnw7LPw61+Hei2RXDtWLCO8+ma4W8XerrKDiRw6FdfZG/C5Dju/ey0PvEMpbbmnfDpzCLneLNGPB3+pJnaYPh26PteR/721BBlIbACC157LzGHVQzMcmXsHDqjaUS1awJIl4B8+GnovVB+wfX7EVvj2n4V85DZi3r2YUC6dnt/nsMK/WKWz2LpmdF0K96bB8un4N3iDZX2XLFHC6PPB5udNKmZPpbLZBpLwcY9IYpKvkiQqSCXyNDlzLoYHPVDSyrYyXOQk5KyGx1ak4mvekq2cyZ/FZF5K8jpKsc7MUoIcCWtg1/z5ztiBhd8PCxeGrHO7ZS+EKslsVcy0kDI0taI9N95ehM0+ybnlVrFnRCQlgfcag6ysD/j1r8HXPpCNdPYaSC4PdaNtv0lJK+XqefUCePvv+5nNOAaxgjEsICkJnnkmMeYX1USX/z2Qx/9cE1nsWZ2D9/wZjuu8qAh+/3t1ryQlBQoCLloAh86GQbkOF9CSC+C3nXLZ8N1/4tq92OQEP/unBnsnXcGr1y11io0EWuyH68dx2pXjuP0jNw8taYHLXwkI/MBpHEVIgsNSk3H+QNY1ZHaC3AHw3rlwsFmEBjZrNmOP4PdvnknWTQ/jXuYldzo88ECgK1npdClZlTYjkZSkJmqwd0PDSzOsWOFs73Kp/SUlwZtvVj+PZ3WljC3hD09htQ9AAXX+paWBcy82YP4H6txGBKwl65hhwr+0K7S71/LvK6vKvG1BcDKY6ipuajSRyNs5jU09bCtsYt989Qyy/+Ack2NNOQrq/+HD4Y03oPK9Gci07dAj0NUWaldLh76H+EfDfZ/6oMkJvrFoCv9av5Q5gYBLEJvQHE6Fpwf5eKb/IVJ90O6oypC5fUP1w66nDIV5WXAkGY4mE9ltEdg/QMrRFpQvf5qiDV5udMEjbWAa6kKzB41LSpxupPBRwULAwIHw4Ydq7lm/P1Qi+cYb4f/+r6ovH+DaayEnR13cu3aFgs4QKm1sWdN2d054JoJ9wvPx4wMunMrIAVVr1LGVB12+eAFy3Z2h8Qf23yvwEN7fwm7tL+TqL8E0F8RVGpym8cn75xRyewTq5Njy6m9Oz+HC1Bl4/qCWrevKMoYsXC51vwwbph4EFYU5cEFI8AGKzjnIRWfmUVDgjdvrscn58OnaFXbsAJQlPmYEfN4usK2mAUeoEaTNKm0DmwLFx8pd4HOHNa4uJVHCzefmMKR8BnfeqYTRHuiZbrPwoWpA1BoLkJUVcmsUFDhzjC3BturYP/10VevdPlrVNJ1ppMnJ1QdsI4lspGBupIBq+KxY+fmqeFVlJSSda5J+21Q+L1tRNU01sM/z90H+YmjZ7mYuLloQjK3U5jON94JWmrqR95bJ+DUD8If1IDtXdOXl67Y7eqZWqq9lNEmprrFnnlG9bXvcruPQ0ewZZPPnS+i1szXPj/shpq+zmnz41Y35jF9uuAFQGtK/GLY9CzmrAivsLxH2AsqTlIvmWAocS4ZDzdT/QbEPax8+6IivhpC8oJA7u8/A61UulkcecYqnVc3TEm0rRTI/X4nqnDnKT5+REaocac3eY12kEKrn0aYNfPCByr0fOTK0X6tgGah9jB0b2ub3KzdMbdM+hm+zl55ITa2aPWOdq/XZWbPUjXTFFfDMFIPt931ATvtC2NfV+fsFzmtHOxg4Fta4FjJfjg5+17Vr1Q1rmuqGNM3QMa2H0e9+p/63b9M0fUwTps2fWkXs8cPXC+YzdKjKths6NCT2EBow6fWq+8eaOc5KxxYC9ixfQI+dzrztj9MPUrRmdAN9u+jT9AR/xgzIyeFoi3YcoBVHaM7vljfn7bktuHarmxZWYo1d/KHqAyDSK/yhASTTAnYOgbmF8NIH+HYaDqGNVO53zBgYMUKJptsdKrxWneCCU2yTk0Ofs6zaadNUl7RZM+c2i+zs6rdB6EKPtN2+LTUVxo2LPGhr/HjV1hLfKVNUWuuyZSp7yDShzSED16ztsCrH+XcI/MZSKFfcx5ctZJ5/ND6fylq69FJV/iJc2Gt6UGmaPvnvmXzf3eYuDFxP576Rg3+XQXl5KFnBsuwtIhk+Vj7/uHGBONjyx5zXKDBz97+AyAZIrNP0XDoB7CPv3G5VF/+vfw24NS6fQqsh8zjmO0KZP5BDfqI/Q0WqqktzNI0rW94H670sXepsMnt21en7rHPyeFS6Y3KyOh/LbQPVu1RqGvUbqcZPTfV/anJ9nOpnq3P5QMiiskY1ezy2mifpJqePyWavT7ngwntON2+GCxc5yzGA083TFErWak6dnr/LYqt7k+Pa+enW03jvn4eAqmU6wicpr+l6sVKP1/lPZ13Hg0GjT0jIbz8bb443Jq+7hMrDt3MiVTPpZJK7Opfl29dw6NhhkK7A6DvAlwTuykBdgzNh1TRY7w3u3z5IyM7Ikc5Jyq2L4Wc/c+bY33GHcntUd7729XUVtfrwc1v7tILC9vpFVo0c6/JKSlIurvC/w+TJcOzq0dCrau4zRBb98Gqa2oefmOStz2Pca+PUQkCMXX64dl4OrxXPQAhlqc+aVTW+dDLXi/lWHoPWjAu5jSR4StJZ+cJ/TzjO1JAkrOBHIi9PReF9vlAwFapOZrxxo8qgef111TY5Gfr3d6Y+Vof9QWBdDB6Pqo9vL3cQLvg1URdRqw8rODwX3xrAlpSkCtW1bq1G5Frk5Chvmx1HobvLp8DAwAfCRP/Kz+Gyv+cwDedNrEk87PfBbavOZeuRnQ5360MFXXnU3O64v6Nxrf9ubkeWn10SerBIcC8oxP+VEVcWfpNLy7QTycK3l0ooK3MOZrIqXJaWhvLPX3tNiVlFBbRtq9pUVIREztqXFfG3XlZKpb22jf3Z6naHSiWcCOGTupwMkfzcdb047fuEqm6m6dNDJShcLhVcDseKDZSXg3vFDCp/OA//MJvFFri5lnYFeVMuA/8xkg2pxkn9bpqmg6MEcrpJxc07Hdlyg7+C98z5UR24Z7lhu/3oYVy3jsMfyODzAz/qmcvIK/8TVxP2NFnBj2TVFhQ43S/uQPaNJVxlZeqBYKVqXXONM5vmrbecfnf75CagHgJWF+/uu5XI2S86+1y7zz7bcBeJXVijVYgsfJ/hF72VjVTTMe3TUa5dC4sWecEHXHcHQfM+IPrvdoWR/3MNyyd/Hzc3lya62I0M36CpDqNASLhxeSYTKwx6boxeDzA/Xx3zk2IvPXfdx5b00uC209I+rBL0jXWarOBHsmotEbJE95lnVHDVsvBdrlBtdp8PFi92jmitrFRib/fV2XsHkybBn/6kPvvXvzq7eeFz7TbkRVIfx65tn/btaWnVj5q1lh9+OLBigxe+y6DViBEcarc3lB0hYVGPA7R8JR3D2Fn3L6CJOywj41g7E85Z6dh2wT5wFfev1+N32tfKIfhfdtlDWqYJxI/iN1nBj2TVVidSdmGaNCk0QMlyzyQlVZ3B3nIXWS4gu9umujkv6+KWqSv1ceza9mltqy1+EF5SohMG38z6jrTxZ1Da7oBaGRD9ha2+ggd6suCRLWgSC2sO5HHv5SqT3jaidtIa+BvZwd5mtMjOVvWkysqg1+ZMll+8E59Vq0dA6f58tODHANWJu71UgH1SbWudNZLVmoM2knvGKpscPooWou86iXdOJH5gfzgnJUGfPsp99v1rbyLGDFDFsCAk+klb4fF+LJj6YYN+F03js3GvCd2WONYN3gk3+Ydw4DGDpzzRN2xuvVUlcJhLcri7cBFPDgIk+AWkrd8CN0f3ePVJkxV8qN4CrS5rxbI0LbGHqu4Z03QWXbJXm2xMt02sciLxA+t3s8otv/aaetB6rzE4VFHI2+WDKW1l5XyiRP/4WlKeGM3cexc04LfRNDYlHfJht9+Rhvn4cvj6xz2D4zLGj1dtw0uHnOx9GT5PbiUGbZt1AELZOm/5t+OtbUcxRJMW/Oqozur0eKrmj4fPDlVQ4EytDK822Zhum1jkRB+C9t/W/vu+/KRB9w7P8f0t4/DbS1xImHdkIf3fuhPvMEPn4icApglrvtgCKaF1A3dB32IX95yVzY1hNaPmzQvNIXEqacl2nbBmtDtAW6Ak2GZJmxLM3SZG5/i46JpeaYUToLoyAuE1Z8CZWml9NjVVCX1ycsNm28Qr1ZWYCB+aHv53AXWjfVrs5dqXcuh4yPbhwN/osTdHBGul2MsuxOOwd031mCZ4/p9JSdIqx/qe+2AJw0n1qJIm9qlHLUPtVMtv2K/H5GR13x/4+C5ctlILfuDpf+TWvKMYIiEt/HCrE0L+/OzsUOaNNYjI3jXUbpvoUJ1bLfzvMn++cpu9uWcGDxR8zR+uX6h6XwHB/yptL0+83Y/jxz8MTugSPrYilgbFaE6NggIo75GvfDg2d84vN7uZSg7rn1axNivTDiKX7j6Z2Fqk6zE/30vrLx9k9XkhK3//9jVR+Y4NQUIKPjgDteHCU5uga7dN3anOrWb/bU1TFZoDK/NiATt/t5T5g5zpmjt6raXjwdHsWb4Av19N/RjtgWaaxsXjAT53ZmYN2gWlxdezBgMq4Mkn1Xq3G66/Xo3uDs/EO1kjLfx6nD8f7kzrz4fpi6gMuBhXdCzBfCsPY1jse/MT0qVjpzrhieSC0ESPmqpzQuhBbJWLBvW32b58MTdtDjSyWfp7Bi6ETsp/s3q16p1Vt29NfGGaqiomnZ3unB77nO38/tDAyr59q6ZE1/WetrRiVXEO12wPrBRq/ozn3/jzqe+4AUl4wa9NeDTRxfKtg7K6/vjH6nPzIw2cW+s2aLFoNr332BoHcqJbXKMSsKVUrrjq9q2JH0xTlcV+/uNcEE53TvZm+JYOwbbhZcOjjaUVH7kNx3EBPt53JC7iRQnr0rHQPvmGI5L7rLoKg+G5+bt2qUmnXS54weflsTcXsnlsoJJdwMo/2nEH/Gw0qW8vCM4Ypolv8vOhrL0J3Z2598O3wYBvXHyXk82Vm2DUqMglyaOJXSuKjnTjQ1u2TplMJj8/9vUj4QS/pnlbNfXLyRRxC8/Nz8tT660u+5Lix7l39QCeCAyCCY667LWQ7mcMYdIkLz6fDtrGOyUlQO+wYK2EnEL4odcgtrYxHDOv1fffORj7e7QnrxxfQUXAj7+j2w5SF+SRl+eNWuG2+iChXDp6OrzG5WTdZ4ahKnBak6bbC9+tweDqQzncHMGfv+W0B/UsWE0A01QjrmkXFqz9Sk1f+nJRz0a7l42fZnPN54EFARVuOO3CmUycGNv6klCCf6r5uJroYFntJ+Nb93hCs2fZEQKuXD+DXy0a4vTnA+VnlMDI0VXGUGjii4ICqDjThC6rHet77AOfK5l8md0o97Jpwvh8A39FumO9QAQHDsaqviSU4OsAbeNzstkShqEGtyUnh4Tf5QqVop7G4/z1TarMO0rvhXBxHjNnxmbXWlM7Hg+4zitwBGvdgWBtUedr2ZBqNPi9HAwiPw/7dw5UKwOj8luK0+o9cFxXEsqHrwO08YnXGwrIWbOSHTigZtRag8Gq4hxyVueSG+bPTx74IKWlsZ8brYmMYcCN2QdYuFMGRfW3hWAUw5YhHVj+fw1/L1teAttHqmoAACAASURBVICkFnsRUs2KioRlWWvp+XEeA67xxuykKAkl+KADtPFK+N9t+vRQVdP7mEHB8jW8c94KNncMtSk/o4QD5+VBoLyVrrcTX5i7Tf7vq6fUglBx29PLwIeLO9dkM52Gn0fW8hKUlcG+naNwy6VUBowMn4AzurxKly7emL2+ouLSEUJcLYTYJoTYIYSYGmH7LUKIvUKITYHXr6JxXE3i4vEot47FNB7nmTcDHh2ba+efXz4I6IB9PJL7Tj5+6XPMauXZCasYxCqf0Sg+csNQBdlGjoSt33gZUNhXbZAgBZxxVPVCY5U6C74Qwg08CwwDegK/FEL0jND0ZSllZuD1Ql2Pq0lsDEPNWGZNU2li8F3xSAbtdLbbebyE0f8erQP2ccj2b0ocywN3KXfOVno2uo/8m29U1pgoa6OKqQn1QGrf8V2abYxdayIaFn5fYIeU8kspZTnwD2BEFPar0dRIRobTyn+CHB5bLhBhAdyFRQs5cF6eDtjHGe3CLOUe+6CCZA6NzG60sRVWT3HtWrW8b+cokvwELfyXLpZUHMxv+BM7QaIh+GcDu23LxYF14YwSQnwshHhFCNE50o6EEF4hxDohxLq9e/dG4dQ0TZnwqRE/FAb7vhnBvVYWny2AO2/rn5k5U5dbiBfM3Sbm/jfUgoQkn8rOOTjkWnL+YzTa388etAVVuvv87eerhUBdnT8d3BKzLsOGSst8DUiXUvYC3gXmR2okpcyTUvaRUvZp3759A52aJl6xp9mmpsK4cXDerBxmvO8iMyw3f69vC5NyTR2wjRPyN+dT4a8I9tKu2w4DiqFdzw41f7CeCa+R73ZD2pFkR5u2ci/5MWrkR0PwvwbsFnunwLogUspSKWVZYPEF4JIoHFeT4NgHcr3/PsyaBRleA2bN4rk3cbp2XFA+aGrM3ogaJyWHS4KpmBAYZuFyR3eG8lPAfs198AGsXAlDUtuHThK4aE8q8+bFZmJANAT/I6CrEOJcIUQK8AvAUelICGFLlmM4sDUKx9VoHAO5rEqceXg5yz2EC8LK55K+grz1eTF5I2qc7N/vXO5wBMxBv42J7pn9mjMMaHlDT0fg1tdxM30qzJhMDKiz4EspK4GJwDsoIf+nlPJTIcQfhBDDA81+LYT4VAixGfg1cEtdj6vR2LGCafffr1w7v/jqce6yJiKyBXD9w8ar2uqamMQ0YfyjJiu/e12tkJAcGF3747YHG/fkqsEzJJskhCNwO/Ds3JhMz4yKD19K+aaUspuU8jwp5aOBdb+XUi4JvJ8mpbxQStlbSnmZlPKzaBxXo7EoKFCDYazJ59dg0H3DEIbstDUSgPCz5Yz4mYM0kbAe2s8X5SKpDD6kr92m0jE/+ihUNTWWMDobXFN+jloIBG439drE5Mmx59ZJqFo6mqZLpCJr03icR5erUix2f/DKvYswd8fYnahRD+32JnR7zbH+zCNQiZvHvs5m3DiYMiX2JqjvcF6mYzm15U4uLos9t44WfE2TwF5kTQiVPbEGg5XFOQzfZmsYKMeQuzo36POPJeFIZGoqlvYkv1Vz16Lmro21EdPZV+UE8/EBlnaDgZ1yY268hxZ8TZPB61WZE48+CrffrtbdxwxGFnZVQTWblb9462IG/cLkgQdiSzgSGcOAG69PU5HPsGJpbQj576WMvRHTRmeDEcU9g/GiCgHf9/kmFmLMDrTga5oUVgZFdrbKzQc4VjyUWa876+xIIfH3z8XvV77/WBGOROfzwxvVm4D//mAz9XbIELjySsjJgWbNYnPEdFaHrGDPxO+C885s09inVIWEq5apafpYVTH/8hfYuBEOlWRzz5I8Fmb4WZFua3jBYuhk4ioxYko4EplmaSXwXWhZArjdXPh4Nu8ErOWRI2Oz6qmrxV5EhSqX7PKD/9Nl6mKMoZPUgq9pUtgnSne7YexYGJxj4Bo2i55LxoUEP2Dui8x8nr2+8Ybqa0KYu03M0lA5hWQfjNkM/NaZfx+rJc7TumUityxVFr6AtCP+midubgS0S0fTpAivijl7tnoA5OHl+q/OqZKx063fcrx6jpSYILycwrXbof/XLmgTe66RcEwTXt3eRglqwK2zoQMxVytZC76mSWHVOhEB0ZBS+ejvvBMWf3ofI7Y522+r/JzRL01p8PPUVIPtYXzmEaiUgkUHPI12OieC1atcNseD8LmDcaIXs+Dfn2xs7NNzoAVf06Swap2MG6eCtm63evn9kCe9jLBn7Fjlk//7BHlv6TSdxqZ1s9bqTUD0L94Dr3E9//OUEdNZVFav0r/LoNP2DLVSQKUbZn23JqbOXQu+pslhqPppvP++KnL1zDOhuvnHiodyT3j5ZCF5cZ2uqtaYmLtN/mT+SS0EatLsawFvMQy/P7azqKxepcsFbY+kOLade3gzUy81GT8+NlJ/teBrmhT2wVRWiqbXq4K3APlk8+hyF73DyieXHlqjB2E1IgU7C6j0Vwb930kSBu+EdpSSnBxb6ZfhGAbMnKkE/4cSj1oZ7KVIBlQUBGNJjX19acHXNBlqmrc2O1uNwl2DwXhm0e9r52e/bL6J+583Y+KmTETSWqQFaiCr5bsLwSh2c8ZIT6wlukSktFTFi75s3sYx8c6mjrCPNKSMjYFiWvA1TYba5q297Tbo0AFewEuvzZmhjB2hqhyedtmEmLgpE5GNe0IDrlwSTi+DXZnXN+rsVidDMFlglwe3PxS4fSkL2nR6C1CxpMbuqWjB1zQZ7LMR2UdhWpb/nDnw/ffKn+8q7s8Fpc7PH0rfhOhiNvpNmWiYu03mrH9BLQTKIXt2QmHrYY16XieDlSxwxQUG7TZer/IBApUzW6cvwcBk7NjG76lowdc0GeyzEdnnrbVb/pWV8KtfwaGR2UyKUC/f11+XTm5ITBMmv5SPT4bKIQ/bDn2LBZ+tLo0r95phwEMPQem+YcozFRiAdeZRP7e48xt7si5AC76miWGfjcgi3PLPzoac/xhc5R/prJcPyPMXM/lJM66EJl6xel4frXWuP/MI+EmiQHrizr1mGHDd/5Y6/PgbOsL11ze+dQ9a8DUJQHWW/6r+OTyyXOCy+fJxSdaW5TN4cGxOttGUsHpe8ljV/PunuZsNqfFZ46jDMQ8uf1LwmpqXBasuaN3YpwVowdckCJEs/+NZBnuLRzBol7Nt27Nfw+eDCRN0xk594vGA6xwTMeAptUKoUvj7Wgr6XdnG8XCOJ7J/atB748WIgOBXuuCDrwoa+7QALfiaBKa0FJ4kp8pk5/s7fg1Dp+DzQX6+niSlvjAMyPpZAVL4QhOeAJd9JfA85IlLsQf1vfr18iCtUskCMravi4mLSAu+JqGwD8zyeGB9iqFSNMPKLTDwCdznmMybF3uzKzUlPH3TgmIP8JtCMC4eHp+mvY0uw9oELXzhh/3N/cp6aGS04GsShvCBWaD8yEf6PBex3EK3/82nsjL2ZldqShxsYcu/96v8+y+6xU86ZnWktUgLWvjSBWlHG/uMFFrwNQlDpIFZhqEydu4oHUlmWLmFFm3WRMzr10QHc7fJC+vnqAUJyYFyCkUFpTV+Lh7Y+FkpAkHgHxs6AllZjX1aWvA1iUN1A7NAZez0DSu3sLFsE5NyzSrZPZrokL85n0p8QTfa1duhX3ESa5p5GvW86oppwtwHPbh8LmXhA3Oz4O+vvdXYp6YFX5M4VJeeCdA12+DSj9MdpZP9Av62NDfmptJrUtjq33c4Ak/xG37oGd8/dkEB+HYatN4+UK0QUOGG9+USivIaNxCkBV+TUERKzzRNFU87kDaNgV8527dsu0b77usB04StbxxSCwHR770HDrvaxMSI1Lpg9SS/P9LTsd6Fn9JXCxrlnELnoNEkMFa2zvPPw52bvPxon3NKui/PKSEtU6fnRBPTBM//M1nZ/u9qRaBgWmkLQUl3T6OeWzSwepIj+wd89oEH2iV74Pz2BxrvxNCCr0lwCgqgoiK0fO7mwY6Rt34BC7dPaKSza5oUFED5WQX4hQzl30tw7xzMS9uMJpECaxjQ99JSXILghC4bOkKnfz7dqF9OC74mofF4VJ18izV7crg+bN7bFd9vIm+9rrMQLTwe4Kgz//7uQvhv8c34/U0nBdaT7iGJZBW4DZRKLuxQ2ahfTgu+JqExDHX/3XGHeuWuNLjj6JAqA7Ee/seDmKZz4Jbm1DAM+GmvjY6BSaeXCToklTatFNhiA9/G29T7QKnkgnMkXxxIq/lz9UhSox1Zo4kRDCMsC+e+xxk0ewAr00Or9jQvYcjNJu5vDCorlSjpVM1TwzTh+O4tyC4EBya1KxMYz3poXkqTyYoqKAB/cRZkEiyxcMZR2L2plPMa6Zy0ha/RhGMYPLArbEYsF6T1zA0O3Coraxpuh8Yg/z0T01ilFqyA7RWDyPAaVTKo4hmPB9ytQ6WShYTNHUG0bzwLPyqCL4S4WgixTQixQwgxNcL2VCHEy4HtHwoh0qNxXI2mvrjy0ecYHFZFs1W7NciAz9nvh7TGu2/jm/QCpPA7AraeQ20b+6yijmHAs/eGpjyUAuZeDJ9+1XgDsOos+EIIN/AsMAzoCfxSCNEzrNltwPdSyvOBp4EZdT2uRlOvGAY9kzs6Vn3RpQQ6hZz3Gzc29Ek1DbKSD1QJ2J67u0OjnlN94R1mcAm2AVgu2HT64kYbgBUNC78vsENK+aWUshz4BzAirM0IYH7g/SvAUCGEQKOJYbKb93OMvJUuOGNAlQ6s5iTZuK0gGLB1SWhVJljaIc5HW9VESvgALMn+P+c3SvA/GoJ/NrDbtlwcWBexjZSyEvgB0B1iTcxhz8Ixflk1RfOHC1ZAZxO3OyZqYcUd5m6TF33rgpUkk/xQsusmumY3Ecd9BDzdnQOwLt4D328taZSy2zEVtBVCeIUQ64QQ6/bu3dvYp6NJMMLLJ+cVGfzi6yFVBmJ16J2LlDB5sk7PPFnyl+ZSgd8xYfnlHS5s3JOqZ9p0LMUlhGMA1tXyLX7iMxt8zEE0BP9roLNtuVNgXcQ2Qogk4HSgSg1UKWWelLKPlLJP+/bto3BqGs2JYy+fXFYGEyfCM4WPMzAsePujtOVNaoBQg/LNN47FHx0RPLnO0yRG11aHJ91Dkts5AOujTuX8VBQ0+JiDaAj+R0BXIcS5QogU4BfAkrA2S4Axgff/A7wnpZXvoNHEBvbyyS6XEv7VfoMO+9o42n1yziEyOufhculMnZOl07Gu6k3g7t+35wpW+40m/fA0OhuMzRyLCJRZqHDBynTJwBFpDT6Wo86CH/DJTwTeAbYC/5RSfiqE+IMQYnig2YtAmhBiB/AbQEe+NDGHVfTq9tvhuuuU8AMc3+ytUja5hfEgPp9265wMpgkrP14d/B1dfpCnHWhao2urIatjlkrpDQzAOpAKF3YobfAxB1EZaSulfBN4M2zd723vjwM/j8axNJr6Zv585a4BEAJeK57BwM9eYHWP/cE2H11QQo+z8vhsjzc4c5amZt76vzzey9oZTMlMkjC201n0/WPTGV1bHRs/CwzAcgESZg6A5hsP8HADn0dMBW01msbG7seXMuTeOVA4vUrwtm3vPwctU11jp3a+aPYiFW4cAdvrJ+Y0qdG11bLTg0u6gtdPpYA9KQUNfhpa8DUaG3Y/fmoqPPssPPIIdO3j5cJdrR1tj51VzJgxUFTkzO7Roh+Z1u2bOZa/Pdobk6au9IrsnxqcteaXaiEQvL3kq3UNfrFowddobNinQZw5U42m3bULhg2DM0o7Odpu6HCQ1W/kceedKqvHPjm6pipZ1iTxgYDtj785PWF+K8OA68ZcqDo3Vv2g5n725OY36HnoapkaTRiWe8HjCfnyU1PhztF3sUqOww/KreOC03vNpHK3l+Rk5e9v6sHHU8U0YemeXYjWyrp1+eFoy+MJ9VtlXZCG/Jxg4PaHVFi8GFx54PU2zDloC1+jiUB+fkjsQb3/y3wvF33ldOscbKeGnNx9d+TJ0TVK7CfcnMfi83cq415Csh/O6nxbQv1WpUcDQ48CMYynB8CbZ7dm4sSG8+xowddowjBNmDfPuc7lUhUyW+9zVg35pMtBhneawqZNIcteB2+dFBTAj3pOp9IWsO29vQO9L20gszZG8KR7cIcFbkV6AT5fw7kBteBrNGEUFEBlZWi5b1947jnl1vmhaLIzW8cF3w/N5fC7JpddBkOGwP33hzJ3NNDjgEnzljsd61KPtE24MQxGZ4OrjzoDt1ceXc/gJLPBXFta8DWaMOyZOs2bq+Ct16vcNb+8w8vQEmcp35XpkJ41lfJy9aCQUrmA8hs2HhezfL0jn7e7BRYkJPtg/+a7EjLAnXxBK/Um0NP5uKOP+WMLGsy1pQVfownDnqlj98kbhnoY7F32cJU5b1f3W4dL300R+e8FJVS4UMXDgKyNvdn6jTchA9wdznQuf9sS3IcONNjx9SWq0UTAMHAMCLIGVuXnQ9EuLxd85ZyhaVf7o9x1Sx6pqSpbJzUVsptwifeToVvJTvyB0bUSOH/P6Vx+eWIGuLN7Z5MsXMHU1De7wZcfPNVgE6JowddoasFeNnnePOXqSXp/unPOWwE7yv7MrbfCuHHw/vuJJ2YRMU02frdJvQ/0hoo77uXHP268U2pMjM4G17YfpBYEVLhhYW8fpa8WNMjxteBrNLVgL7dQWQljxypfft+9rRztdrp3M2eOqsWjCVBQQEkL56rPXd2ZMycxRyWbJnxT6pwB69uWDTexuRZ8jaYW7EHclJTQTFfnVHZ2tPukyyEu6JhHWRk89FDiiVkkzIOf8oYtYOv2u/h2c05Cjkq2eorrXsjG5XO6deat3Ngg14sWfI2mFsLLLUyerNw7ny29q0qKZssBD+L3w7JliWnB2inKM3mpaKGjYFrf0u649xi4XIk3KtnqKfp3GXTe3kutDLh1PszY0iBZXVrwNZoTwArilpaG3DufFHvps72jo9267iVc2Ckv4WfEMk0oHJ/Pdy2d67/c1R2fTw1kmzkzseIc9p5iu6Mpjm3t2dcg56AFX6M5CcKraf7s3Ieck6O44Oze0xO+rk5+Puw4qyTMnSP4dlMOfr8aq1BaZZLTpo19gp1zfuRRKwNunb7Ht3JV6/rvDmrB12hOgvAc/anTvfQvdQZvj7XbyS/OMRPOgg1nRe//Otw5nbb1hmL1gyQlJe7DcP58+M+ONo5xHDMHSL4xc+u9LIcWfI3mJLHn6JsmnFbiDN6u7gIZlVP59a8T14d/VWuT3mx2rDvjiHJjCAG33pqYD0PLjy+/9OCSIij6Phe8fPr2ep9TQQu+RnOKWFkXX394VxW3zocDVnBxmZmQ2Tp5efDtE/mccTywIuC2OFjiweWCZs0Sd1Ba0CW4x6DL9t6ObXtlu3rPXtKCr9GcIpa19mmxl86fZTq2vXYBDOyUm3DZOqYJd94J+8/ewtMDAisF4IdeN7XhkUcSc4Sthd0l6D0rXa0MPBA7lx2s9wndteBrNKdIWppyT7hc8M2655xWvoDtvT9MuGydggJVRvrT9H1UCoITlrsQdDjmSYz5a2vBcglySQdVkyngx19ubOK+CXn1+kDUgq/RnAKmqfLx/X6VsfPMFINBFc4qmt+328NAl5lQ2ToeDwxOMhlw9DNkQOwBziy8iaz2Ca70YaS1zVYCbPPjF7WYXq8PRC34Gs0pEBxE41ev0lLo+eP+jjarzoGbBuQmnAvj6bNy+bijXy0ErNc9zVqxcWPjnVOsYZoweZTBRZ+lO9YvSd2Jubv+/H9a8DWaU8Cej+92q4nOs87OcZRNlgJmn/deI59pw2GaMM1jctHO1yhpWXX7vHmJE8uoDctgqCycVsUVmP9Obr0dVwu+RnMK2AfRCAFz5iiL7fyyNo52RekHefX+KQkx7WF+PgwoL2BNJ59jwBU+N2zOprIycWIZtWEZDFu/8ZIRZuWz55t6O64WfI3mFDEM6NJFVdC00ukGlQfmabWsfODz9rm8fr/ZpLN1rHmA95LGwt445q8VX1yPe4+RULGM2rAMhkcegX4pA9XKQLzDf7xN9R+sI1rwNZo6EF5J8/afz6D/987JUV7vDgPOzm2y2TqmqaqDVlTAMN7i2zB3zoihHarMHqYJZesk+z53ZOu8mPRuvfnxteBrNHUg0nSIV8jpVapobuu9BpdL+fqbkpVvDT5btgz6+k3ad1rM6zZ3TpJ0MeysbJ2OWQNXi7McfnyfkOSurh8/vhZ8jaaOhE+HOOyXXi7Y7ZzQ4kC7Evr6zCY38Yc9W2kM+SzsLR3unNafDWLyKKPJfN/64LqJOVy33bluybYl9WLla8HXaKKMYcCFPQc71q3qAsZZufh8cPw4DVL7vCGwu7Q6ukqquHP207bJurKihYlBd3Oks1eIn9zC6Fv5WvA1mnrg7l+EpWi64J2hywFVGrippCjaXVoZGcG4Y4gjHXSwthby8+H7XcMYuMu2UsLizxZH3crXgq/R1ANGZ4MePme2xSfph7iwUx5Ak0pRNAyVf//196/xZvfAykA65sgfZ+tgbQ1Y2U1plNLDPgeKAIkkf3N0u4J1EnwhRFshxLtCiM8D/59RTTufEGJT4LWkLsfUaOKFu3pVTdE8Y8B9TXN6v9xcFmb4qHAR9N/3bnk9//mLocW+BgoKAg9/PNy82R1y60CE7lLdqauFPxVYLqXsCiwPLEfimJQyM/AaXsdjajRxQUbnGfTc6UzRLOxeyqQBU6pYvaZJ/A7OMk1YsoQt7ZyrT28VubkmhBUDWesyKCz+LbPeALcfhIRkmUR27+jWka6r4I8A5gfezwdG1nF/Gk2TIT8fxPKqKZpfnP8cEBJ4K7Wxvie/qDfy8zHP8lPYxbn6+P4OkdtrghiGmtvX7YaDtGHsBsHKefDH5fCP73+F0Tm63aOkOn7+TCnlnsD7EuDMato1E0KsAyqBx6WUiyI1EkJ4AS9Aly5dIjXRaOKKT4u99Np1Lx+nHwyue6PzYYpvzqNol5eUFBgzJjQxupXREldukC1beD9d1YGxyiEj3dzWJ0FnOTlJSktVWute0nAj6V8M/YtBDGgd9WPVauELIZYJIT6J8BphbyeltHISInGOlLIPcBMwUwhxXqRGUso8KWUfKWWf9u3bn+x30WhiiuxsNdH5t/t+GloZyNhJ+cmDQYEH52jdePHt5+XB5H4mvpWr+SEVRznkzKO/xTssnp5ajYfl1mkvSvEhsJ6bPP101Lt7tQq+lPJyKeVFEV6LgW+FEB0BAv9/V80+vg78/yVQAGRF7RtoNDGKYcD778PPjJzQSMoA6y4oIaNzHikp6sEQPlo31n36eXkwbhw0X1uAebaPp+yzW0lB/8z6qwfT1LDcOitdHvy4Q5dJPaRy1dWHvwQYE3g/Blgc3kAIcYYQIjXwvh0wENhSx+NqNHGBYcCs+w0eaOPMy/cLOP3Sh4ICHz4xeqz59MMfQK++qv5vzQEW9FaTdwRnt3K5yB7iaaQzjU9KS9UArKf4DRCwDaRU06pFkboK/uPAFUKIz4HLA8sIIfoIIV4ItOkBrBNCbAbeR/nwteBrmiyRrPOH755B3++cPtnV5+5hwTN5VQTdKldQ3xNanyiRHkCjRkF/TO7hKbaGZedktBkY9WBjU8fjCQVufbjUNMDCpZ4EUaROQVspZSkwNML6dcCvAu8LgYy6HEejiRcscSwvV35Zu4umU9nZfEggeBvw5X/8w0wuvdTLM8+AN5C2b/l0rX00tk8/0gNo2jQYtDCfj770sSosvyL1YM/GOM24xjBg7Fj44HkP5aQiKadCpvBFmieq4qlH2mo0USSSOOblwZAhsH3ZZIR9YA2w5+xdVFTAnXeGegSRKnA2JuEloD0eda4td23hiQEq1TSYneN36eycUyQ7G9YlGwxlOb/nj1whlvN6aXT/+FrwNZooEi6OaWkwcaKKvxXt9vKTbc7c9C86HiFj6FX4/U7XTXgFzsYk/AFUVAQ5g012Va5kSXdnW/H5cDLaxMBJxyGGAXffrQZhzRDT2NjMiHrvTgu+RhNFwsWxtFRZ+xZHCh92BG8BPhm4lIs65UU7PhdVrAcQwIQJcJNPlUJ2WvcCUZjT6DGHeMU04a9/VbFaIWDSpOg/8LXgazRRxm6dezwqF9/lUlb/thIvI1Z3VQ1tk52L/tOZPDk2MnJqIj9fPcB6sKVKsJbdg0ndG32rNFEoKICyMiX4fn+9pOFrwddo6hP73KUrV8KKFXDOofn03uNst7n7Tsram3FhHffHxN1pJavOca4/O7VnTMQc4hWPRxkGFj5f9DO06lpaQaPR1IKVZx9kpsFnj6WzueNOtSwAF5xuTCUt7YNGOMMTJysLriRXuXPspRT8gpsvytZiXwcMA559VsV8fD7VM9Q+fI0mzjEMGH3VtCqjbw90X8HEGWbMunVME/4+yeR6qlbGZNdg2hzSal9XvF744APVI6yP3pIWfI2mnqiuPIJpwu5DXq788rTQyoAvv9ngCVW68Y1ZZsF+7IICuLE8n7Wd/KwOc+ewr2cw6BzrZSFinfrM0NIuHY2mHqhpAJa1/vqzJ+A6NzeU6QIcSt/Epx2nADNq3E/4sQoKVPc/miIRfuxJk8AgkHvvqIzpQhRlUzrwxM5X03jEleBXVFRQXFzM8ePHG/tUEpJmzZrRqVMnkpOTG/tUYp5IA7AMw7l+SfEMpi77D49d8bn6UEBA/74zlzt3j8TobFS7H4v6FNjwY5cVqGBteO4924aTXGKwa5fK4onrUs9NnLgS/OLiYlq1akV6ejpCiMY+nYRCSklpaSnFxcWce+65jX06MU915RGs9WVlKvXufXM+g7sNYGV64INCpeVNeHUqGyd/UGuZhdoeCNH8DjeW5zNjgDP33iUEw3+Uw1sC5sxRqadJAVWJhbIQGidx5cM/fvw4aWlpWuwbASEEaWlpund1glRXHsFa36ePWjYxuHF5ZmgwVoBNB1YwZdmUuiTd0wAAGc9JREFUYPvbb1cTpYQTqexBfXyHmTNhe+kyllzgbDPonMH07WhQWakeOj4f3Hpr7JSF0DiJKwsf0GLfiOjf/uSoko5pW3/xxbB2rVpeUPwcz70+gPHXBRoErOfc1bmM7D4SMJg/X1na8+dHfoBEy4dvmsotAyoFs7RU7VdOnUJ+rx1qkhPbZdCzXU883Z09gexsLfSxStwJvkbTFMjOhrlzoaICPnIZ5B0cyZCvFrEiPdAgIPoT3pjA/36/MegCKiur6rap7sFyspgmXHaZOoaFywWDk0yWlz/BBK+tsVQGQHbvbIzO0X3oaOqPuHLpxArFxcWMGDGCrl27ct5553HXXXdRXl7OSy+9xMSJExv79Fi0aBFbtoSmHPj973/PsmXLGvGMNOEYhqqbkpSkhNy7I4dHlgc22l07325ikRiN36+W/X44cKB+Uh+teIAdv1/57u8bKtnc0bltRPcRwbr3sVTsTVM9TV7wo31jSCm54YYbGDlyJJ9//jnbt2/n8OHD3H///dE5QBiVlZUn/Zlwwf/DH/7A5ZdfHs3T0kQBa/JqKWENBnuLR5KzKrDRKq4mYW3ZQrh8SvBzTz2lLPFoz4jl8YQCrhYuF8hOa8gdGFgROCchBDkDc6JzYE2D0aQFvz6minvvvfdo1qwZt956KwBut5unn36auXPncvToUXbv3o3H46Fr1648/PDDABw5coRrr72W3r17c9FFF/Hyyy8DsH79ei699FIuueQSrrrqKvbsUQVWPB4PkydPpk+fPjz66KOcc845+AMm3pEjR+jcuTMVFRXMmTOHn/zkJ/Tu3ZtRo0Zx9OhRCgsLWbJkCffeey+ZmZl88cUX3HLLLbzyyisALF++nKysLDIyMhg7dixlgf57eno6Dz74IBdffDEZGRl89tlndf+xNDViBVyt+ilPiRweW+7i5s2BBjbRZ0AudFIXsJWRE+0ZsQwDbrtNVWoE9f8fO+eRd80msOfdAzedc6+e1SoOadKCXx9TxX366adccskljnWtW7emS5cuVFZWsnbtWl599VU+/vhj/vWvf7Fu3TrefvttzjrrLDZv3swnn3zC1VdfTUVFBZMmTeKVV15h/fr1jB071tFLKC8vZ926dTz44INkZmbywQeqxsrrr7/OVVddRXJyMjfccAMfffQRmzdvpkePHrz44osMGDCA4cOH88QTT7Bp0ybOO++84D6PHz/OLbfcwssvv0xRURGVlZXMmjUruL1du3Zs2LCB8ePH8+STT9b9x9LUiL2w2uzZ0GucwWtiOH9bhLO4miW2104IrpJSPSiinZmTnQ3NmqmsnyHJJt+fP66KKyflQAf+PWGGHkkbhzRpwa/PlLXquOKKK0hLS6N58+bccMMNrFq1ioyMDN59912mTJnCypUrOf3009m2bRuffPIJV1xxBZmZmTzyyCMUFxcH93PjjTc63lu9gn/84x/BbZ988gmDBw8mIyODhQsX8umnn9Z4btu2bePcc8+lW7duAIwZM4YVK1YEt99www0AXHLJJezcuTMqv4emZizft9cLXbrAE+Tgw8Vzb1IlVZMOm+CmqwBlfV9++YmnPp6oa9Oeivnb/8nlyTBXDkD5yocpK4OHHtLlE+KNJi349TFVXM+ePVm/fr1j3cGDB9m1axdJSUlVUheFEHTr1o0NGzaQkZHBAw88wB/+8AeklFx44YVs2rSJTZs2UVRUxNKlS4Ofa9myZfD98OHDefvtt9m/fz/r16/npz/9KQC33HILzzzzDEVFRTz44IN1zpFPTU0FlJvqVGIHmrrh8cBat8F4ZtG/GO5dHdhguXYE0HUp3HQVUkJmptpcm5CfrGvTquM/pc1bVVw57YuuxLXJi98Py5ZFN4agqX+atOBD9LMHhg4dytGjR8kPJCv7fD5++9vfcsstt9CiRQveffdd9u/fz7Fjx1i0aBEDBw7km2++oUWLFowePZp7772XDRs20L17d/bu3YsZuFsqKiqqtdBPO+00fvKTn3DXXXdx3XXX4Xa7ATh06BAdO3akoqKChQsXBtu3atWKQ4cOVdlP9+7d2blzJzt27ADgb3/7G5deeml0fhhNnTEM+M1v4AW8rGAIjy/H6c+3CIj+U0+puXLvvz80z2wkwl2b+fnqIZGXV31xtzFP9GNr+zLH+rMPt+CRq97h8suVO8nvj24MQVP/NHnBjzZCCP7zn//wr3/9i65du9KtWzeaNWvGY489BkDfvn0ZNWoUvXr1YtSoUfTp04eioiL69u1LZmYmDz/8MA888AApKSm88sorTJkyhd69e5OZmUlhYWG1x73xxhtZsGCBw9Xzxz/+kX79+jFw4EAuuCA0BPIXv/gFTzzxBFlZWXzxxRfB9c2aNWPevHn8/Oc/JyMjA5fLxR133FEPv5LmVDl4ULlrpvE4EvjbIrgyUGrHPi0iXZfiG9OPykrlz7eE3I7lxklLC7k2k5JU/v8DD8C4cer/cCt9wruj+bxXYFSYzbpPLZjI5MkwapSq1d6QrlJNdBBSytpbNQJ9+vSR69atc6zbunUrPXr0aKQz0oD+G9Qn4QOfCsSlDJErEMBVN8HSwMyIdhFm3/mwOB+KDe64A6wYvGkqIa6ogORklfNfWgq7dqmaN/Z5dt1u5facNg1G/3s0Cz9eGDoO6ljn7WnJF3mHg209Hj3QKlYRQqyXUvaJtE1b+BpNjFBQAFboRAh4uffj+HEhgXf+Xo2l324H3DYA15VTyM4OWfW5ucrqt6z/jRuVoGdnK6vcnnppWemj/z2ahUU2sQ8Ejd0SWrz9J4dFrwdaxSda8DWaGMGeVZaUBM9vNriDWUhEUPT77g40tgdyAf+AXK4v6MiQu/P43e/gtdec+y4pUQ8CUHXthVCvpCSYlGuS/VFXJfb2hwnQ4RDcPvdmPtvj5fbbdUG0eEcLvkYTI9grY1ojcF/AyyJGBNt8ODfM0oegQJeWl1B59Th897bCd9mU4IAutxvefFNl6Vx2mRqp6/eDzMqjYlJHcksHsOP7HU6xl4Afpv+zK88XLwiWdigo0Fk58YwunqbRxBDWJCn20NoT5DCcJbjxI1CW/uiRsLA3VSxyAFIPw8Bc/P1mgnTjk+Dzq1vdB+Dygbsc3GGpt7bYgJDw3Bsw79v5wQfHiy+qB4WeySp+0Ra+RhNjeDwqC8YS2jWo3Hz7OKwFi1B1d8IHZ9nLFyeVQ/IxSDkGzQ6FXilHQ2Jvf1gE9nP+Plg1F3psGEKhNBBCBXkrKqJfzkHTsGjB12hiBCvgCqGSCyNHKl/7C3h5AmexsseXw+q5cOHONlV3Jk7wBaGHhlR5/9ufhb7Fgqk8js9H0J0DziCvJv7Qgn8SlJaWkpmZSWZmJh06dODss88OLpeH15WtIwcOHOC5556L6j41sUv4aFhQWTA5OaHaNg83n8HeK28O6rMAjGIomn+AV//TA9e2kVDWwmnx14RN6HvvgcK5Ku9fAuN5njUYJCc7y5OMG6fdOfFM0/fhm2bUEobT0tLYtGkTAA899BCnnXYa99xzT62fq6ysJCm87mwtWII/YcKE2htr4p7q5qa1ArnWoKpFWQtouQxu8i90FNP82eatlGz+jvt4mhcu/4IWg+biTimjrLKS8mOAPwkEJCeBTPLh8lfS5nAFRrFkSqF6cFjPiRnk8AJeRo5UDxzr/HTOfRNASnnKL+DnwKeAH+hTQ7urgW3ADmDqiez7kksukeFs2bKlyroaKSyUsnlzKd1u9X9h4cl9vgYefPBB+cQTT8i8vDzZp08f2atXL3nDDTfII0eOSCmlHDNmjBw3bpzs27evvPvuu+WOHTtkv3795EUXXSTvv/9+2bJly+C+cnNzZZ8+fWRGRob8/e9/L6WU8sYbb5TNmjWTvXv3lvfcc0/UzruunPTfQHNC1HSp2rclJ0vpckn5BldKP8hAMk/wvR/kBjLlx7PVDh57TEohpAw0kyDlfG6WPlt7++fnc7Ns317K2bMb6YfQ1BlgnaxGV+vq0vkEuAFYUV0DIYQbeBYYBvQEfimE6Pn/2zv/2KqqPIF/vn20FMJuEazrj9KlktXQ4TdaW0mhsQwFoyAsJOooMPxQxjSLMeM4bBwXNhEYNUKmRtoO1BSYOExwBXcWpIg0JWkFAR/aoat0tBtgiHRbqVswYMvZP+677evre+0rffS9e9/3k9xwf5x77vfLef3ec7/ne76nn88Nj5uRHzmAYCmKbc6dO0d1dTVvvvkmq1evZvXq1XzxxRekpaV1lKmoqODMmTMcO3YMr9fLiRMnqKqqYuPGjYwZMwav18vrr78ecbmV2KKnRH/+P2N7YtajCQc4ShbQLSSfSXgZ9+yDMHIkj/9pPg+YGtbzEhf4B66QxNP8oYsb3+7Z7+Bn/DxhJ3v3Wtk7FffRL5eOMaYOel3cOguoN8Z87Sv7R2AecLqnmyKCPZPFXl35Jow01dbW8vLLL3Pp0iVaW1spKCjouLZo0aKORGc1NTXs2bMHgCeffLLDFVRRUUFFRQWTJ08GoLW1lTNnzpCenh5xWZXYJtTatPbP2F7XFqwJU+//8ijfvVbAbCq6GX0DmOZmRjfvoZo93er0Lwewn1ks8+xky9vqtnEzAzFoexdw1u/4nO/czedm5EcOoKcUxf4pjkNhjGHNmjUdaZLr6+tZvnx5xOVUnIv9M7azVBpjGX6vFx5JOMBGfuU//gp0D8QJPOdfdgc/47HEA7z9tvbs3U6vBl9EPhKR2iDbvN7u7Ssi8oyIHBeR442NjZGp9CYn/QiVojiQ7Oxs3nvvPcBaxMSmoKCAsrIyWltbATh//jwXL14MmeJYiS/8QzXXru2apdLOWvlywm/JlWo+YyJASOMfeP77v08jN6GaJVgzaZuaQj9fZ9e6g15dOsaY/q5+fR4Y5Xec5jsX7FmlQClY2TL7+dwBwU5RnJqaygMPPBDSSG/evJmnnnqKV199ldmzZ5OSkgLArFmzqKurI8f3Qho2bBg7d+5kzJgxTJs2jXHjxjFnzhz148chpaVQWGj57gcPtnr5hw51jZgZP96K4Nm2LYepP3pZQSm/ZgN38DeGcK1LJA/AFQbTSgp/nb4U2fhbTuZbk26Tkqw0yhs2dNZth4raHlENx3QBoUZz+7IBlYSI0sF6qXwNZABJwCngJ73VGZEonRji8uXL5vr168YYY959910zd+7cKEt0Yzi5DZxEdbUxgwZ1RtYkJFgRN4Fl1q835rHHukbh2NtKSkzz8NHGDBtmrtw+2vxiUEm3KCC7jpKS7lFC69dbx2D9G/h8JTahhyidfg3aish8oAhIBf5LRLzGmAIRuRPYaox52BjTJiKFwAHAA5QZY3pefNWFnDhxgsLCQowxDB8+nLKysmiLpMQwlZVdZ7h6PF1jDvx736H4Pc9wyzPPMHy4de/TwKjKrvH09kDxhg3dA9oGIOZBGWD6G6XzPvB+kPN/Ax72O94H7OvPs5xObm4up06d6r2gotCZT+fqVWug9q23QodqJiR0LjloD+oaY6VB2LSpM+HZ5s3WAiivvQa3327lxrfrslfF8jfu9mCxTrpyD+6faasoDqQ3Y+vf+/Z4Oo28x0NHsjP73+vXrRfHc891Xelq27bOMvYLoakp+BeA4g7U4CtKjNKTsfV/IdjLFtrhmitXQnq61Wt//nnrpZCQYGW79OfHHy2Db6+K1dRkBbQp7kUNvqI4FPuFUFMD5eWd7hh/V43dax850lrpyt/nn5jYtYevPnr3owZfURxOoPsHOgd0PR5Ytqxz0fHt263lDgN9+P5unAjmG1RiDDX4fcTj8TB+/Hja2toYO3Ys5eXlDB069IbqWrp0KY888ggLFy5kxYoVvPDCC2RmBk8zVFlZSVJSEg8++CAAxcXFDB06lMX2X60S1/i7f/wjbtrboaTE+gI4dAi2bAl+r43G3rsb1+fDrzlbw4YjG6g5G5mpgkOGDMHr9VJbW0tSUhLFxcVdrre1tYW4s2e2bt0a0tiDZfCrq6s7jletWqXGXgnKyJFdj20ffTi5Awcg36ASRVxt8GvO1pC/PZ/fHP4N+dvzI2b0bXJzc6mvr6eyspLc3Fzmzp1LZmYm7e3tvPjii9x///1MmDCBkpISwJrkVlhYyL333svMmTO5ePFiR115eXkcP34cgA8//JApU6YwceJE8vPzaWhooLi4mE2bNjFp0iSOHDnC2rVreeONNwDwer1kZ2czYcIE5s+fz3fffddR50svvURWVhb33HMPR44ciaj+SuxRU2P56v2jcRISOn30vaVKsKN/7PQN6td3F6526VQ2VHKt/Rrtpp1r7deobKgkZ1Rkvk/b2trYv38/s2fPBuDkyZPU1taSkZFBaWkpKSkpfPrpp1y9epVp06Yxa9YsPvvsM7788ktOnz7Nt99+S2ZmJsuWLetSb2NjIytXrqSqqoqMjAyam5sZMWIEq1at6rLgyqFDhzruWbx4MUVFRcyYMYNXXnmFdevWsXnz5g45jx07xr59+1i3bh0fffRRRPRXYpPKyu7RODNnWnl4oHd3jcbeuxtXG/y80XkkeZK41n6NJE8SeaPz+l3nDz/8wKRJkwCrh798+XKqq6vJysoiIyMDsFIef/755+zevRuAlpYWzpw5Q1VVFU888QQej4c777yThx56qFv9n3zyCdOnT++oa8SIET3K09LSwqVLl5gxYwYAS5YsYdGiRR3XFyxYAMDUqVNpaGjon/JKTNDToGpenhV9Y0fjDB5sGftgs2m3bw9ej8beuxdXG/ycUTkcWnyIyoZK8kbnRaR3b/vwA/FPhWyMoaioqEtufIB9+wZ+svHgwYMBa7D5RscXlNiht0HVnJzOaByAyZM7/fD+k7UGDYKyss6QTB2cjQ9c7cMHy+ivyV0TMVdOOBQUFLBlyxZ+9H1bf/XVV1y+fJnp06eza9cu2tvbuXDhAocPH+52b3Z2NlVVVXzzzTcANDc3A4RMl5ySksItt9zS4Z/fsWNHR29fcR/hDKrm5FjROIsXWxOv/BdGt5eHmDPHcv3o4Gx84eoefrRYsWIFDQ0NTJkyBWMMqamp7Nmzh/nz5/Pxxx+TmZlJenp6R0pkf1JTUyktLWXBggVcv36d2267jYMHD/Loo4+ycOFC9u7dS1FRUZd7ysvLWbVqFVeuXOHuu+/mnXfeGShVlQGmLwnNgr0c7Jm069ZZ0TvQPTGb4l7EmNhMO3/fffcZO2rFpq6ujrFjx0ZJIgW0DWKBcCdGhXL/bNhg9frtfDvPPhs8Pl9xJiJywhhzX7Br2sNXFIcR7qBqqIibwK8Enc4RP6jBVxQXE+zloKGX8YvjDL4xBhHpvaAScWLV/af0HQ29jE8cFaWTnJxMU1OTGp4oYIyhqamJ5OTkaIuiKMoN4qgeflpaGufOnaOxsTHaosQlycnJpKWlRVsMRVFuEEcZ/MTExI4ZqIqiKErfcJRLR1EURblx1OAriqLECWrwFUVR4oSYnWkrIo3A//SjiluB/42QONHELXqA6hKruEUXt+gB/dPlH40xqcEuxKzB7y8icjzU9GIn4RY9QHWJVdyii1v0gJuni7p0FEVR4gQ1+IqiKHGCmw1+abQFiBBu0QNUl1jFLbq4RQ+4Sbq41oevKIqidMXNPXxFURTFDzX4iqIocYKjDb6IzBaRL0WkXkR+HeT6YBHZ5bt+VERGD7yU4RGGLktFpFFEvL5tRTTk7A0RKRORiyJSG+K6iMjvfHp+LiJTBlrGcAlDlzwRafFrk1cGWsZwEJFRInJYRE6LyF9EZHWQMo5olzB1cUq7JIvIMRE55dNlXZAykbVhxhhHboAH+CtwN5AEnAIyA8o8BxT79h8HdkVb7n7oshR4K9qyhqHLdGAKUBvi+sPAfkCAbOBotGXuhy55wJ+jLWcYetwBTPHt/x3wVZDflyPaJUxdnNIuAgzz7ScCR4HsgDIRtWFO7uFnAfXGmK+NMdeAPwLzAsrMA8p9+7uBfInN1VPC0cURGGOqgOYeiswDthuLT4DhInLHwEjXN8LQxREYYy4YY0769v8PqAPuCijmiHYJUxdH4Pu/bvUdJvq2wCiaiNowJxv8u4Czfsfn6N7wHWWMMW1ACzByQKTrG+HoAvDPvs/t3SIyamBEizjh6uoUcnyf5PtF5CfRFqY3fC6ByVi9SX8c1y496AIOaRcR8YiIF7gIHDTGhGyXSNgwJxv8eOM/gdHGmAnAQTrf+kr0OImVt2QiUATsibI8PSIiw4D3gOeNMd9HW57+0IsujmkXY0y7MWYSkAZkici4m/k8Jxv884B/LzfNdy5oGREZBKQATQMiXd/oVRdjTJMx5qrvcCswdYBkizThtJsjMMZ8b3+SG2P2AYkicmuUxQqKiCRiGcg/GGP+I0gRx7RLb7o4qV1sjDGXgMPA7IBLEbVhTjb4nwL/JCIZIpKENaDxQUCZD4Alvv2FwMfGN/oRY/SqS4A/dS6W79KJfAAs9kWFZAMtxpgL0RbqRhCR221/qohkYf09xVyHwifjNqDOGPNmiGKOaJdwdHFQu6SKyHDf/hDgp8B/BxSLqA1z1BKH/hhj2kSkEDiAFeVSZoz5i4j8O3DcGPMB1g9jh4jUYw2+PR49iUMTpi7/IiJzgTYsXZZGTeAeEJF3saIkbhWRc8C/YQ1GYYwpBvZhRYTUA1eAn0dH0t4JQ5eFwC9EpA34AXg8RjsU04CngS98/mKAfwXSwXHtEo4uTmmXO4ByEfFgvZT+ZIz58820YZpaQVEUJU5wsktHURRF6QNq8BVFUeIENfiKoihxghp8RVGUOEENvqIoSpygBl9RFCVOUIOvKIoSJ/w/Q+Tk6YR9OXgAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "WKmuAAmLSmiR"
      },
      "source": [
        "Now our model has learned to approximate the function mapping from the input to the output. The capability of neural networks to learn from input-ouput pairs alone and approximate an arbitrary function, see universal approximation theorem, can be very useful if the mapping between the input and output is too complex to be captured with model based approaches. But learning from input-ouput pairs alone implies that the model will only be able to make accurate predictions over input ranges it has seen during training. In order to demonstrate this we will predict on an interval that exeeds the $\\left[0,3\\right]$ interval the model was trained on, i.e. we will predict on the interval $\\left[-2,5\\right]$."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "ZRF8hAmmVFcH",
        "outputId": "9faa7db8-9155-4701-9bcc-d2086463f597",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        }
      },
      "source": [
        "x_generalize = np.linspace(-2.0, 5.0, N_samples, dtype=np.float32)\n",
        "y_generalize = np.sin(1.0+x_generalize*x_generalize) + noise_sig*np.random.randn(N_samples).astype(np.float32)\n",
        "y_truey_generalize = np.sin(1.0+x_generalize*x_generalize)\n",
        "y_pred = mdl(x_generalize)\n",
        "plt.plot(x_generalize, y_generalize)\n",
        "plt.plot(x_generalize, y_truey_generalize)\n",
        "plt.plot(x_generalize, y_pred.numpy())\n",
        "plt.legend([\"Observation\", \"Target\", \"Prediction\"])\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXIAAAD4CAYAAADxeG0DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdd1hVR/rA8e/cApcOUqUJFkAURcQeu0YTjYktfRPTzSbZbHbjpv5SN21TN82sKcb0HhOjiV3RaOyNogKKCCIgIL3cMr8/rhIbSLlwKfN5Hh/lcs6cl+J758yZeUdIKVEURVHaL429A1AURVGaRyVyRVGUdk4lckVRlHZOJXJFUZR2TiVyRVGUdk5nj4v6+PjIsLAwe1xaURSl3dqxY8cJKaXvua/bJZGHhYWxfft2e1xaURSl3RJCHLnQ62poRVEUpZ1TiVxRFKWdU4lcURSlnbPLGLmiKG2b0WgkKyuLqqoqe4fSKRkMBoKDg9Hr9Q06XiVyRVHOk5WVhZubG2FhYQgh7B1OpyKlpKCggKysLMLDwxt0jk2GVoQQnkKI74QQ+4UQKUKIYbZoV1EU+6iqqsLb21slcTsQQuDt7d2ouyFb9cj/C/wmpZwlhHAAnG3UrqIodqKSuP009nvf7B65EMIDGAV8CCClrJFSnmxuuxeyIWsDnyZ/Sl5FXks0ryiK0i7ZYmglHMgHFgohdgkhPhBCuJx7kBDiTiHEdiHE9vz8/CZdKCErgf9s+w+TvpvEf7b9h0pTZTNDVxSlLcvKyuLKK6+kV69e9OjRg/vvv5+amho+/vhj7r33XnuHx+LFi0lOTq79+IknnmDVqlWtHoctErkOiAPmSykHAOXAw+ceJKVcIKWMl1LG+/qet8K0QR4b+hg/X/UzV/a8ks+SP2POb3M4UXmiWcEritI2SSmZMWMGV111FampqRw8eJCysjIee+yxFrmeyWRq9DnnJvJnnnmGCRMm2DKsBrFFIs8CsqSUW059/B3WxN4iwj3CeWr4U7w17i0OFx9m7sq5lNaUttTlFEWxkzVr1mAwGLjlllsA0Gq1vP7663z00UdUVFRw9OhRxowZQ69evXj66acBKC8vZ8qUKfTv35++ffvy9ddfA7Bjxw5Gjx7NwIEDmTRpEjk5OQCMGTOGv//978THx/Pcc8/RrVs3LBZLbVshISEYjUbef/99Bg0aRP/+/Zk5cyYVFRVs2rSJn3/+mXnz5hEbG0t6ejpz5szhu+++A2D16tUMGDCAmJgYbr31VqqrqwFriZInn3ySuLg4YmJi2L9/f7O/V81+2CmlPC6EOCqEiJRSHgDGA8kXO6+5RoeM5vUxr3Pv6nv5v9//j9fHvK4ezihKC3h6SRLJx0ps2mZ0oDtPXtGn3mOSkpIYOHDgWa+5u7sTGhqKyWRi69atJCYm4uzszKBBg5gyZQpHjhwhMDCQpUuXAlBcXIzRaOS+++7jp59+wtfXl6+//prHHnuMjz76CICampra2k87d+5k/fr1jB07ll9++YVJkyah1+uZMWMGd9xxBwCPP/44H374Iffddx/Tpk1j6tSpzJo166w4q6qqmDNnDqtXryYiIoKbbrqJ+fPn8/e//x0AHx8fdu7cybvvvssrr7zCBx980Kzvp61Wdt4HfC6E2AvEAs/bqN16jQgawf1x97M6czWL0xa3xiUVRWkjJk6ciLe3N05OTsyYMYONGzcSExPDypUreeihh9iwYQMeHh4cOHCAxMREJk6cSGxsLP/+97/Jysqqbeeaa64569+ne/FfffVV7ecSExMZOXIkMTExfP755yQlJdUb24EDBwgPDyciIgKAm2++mYSEhNrPz5gxA4CBAweSkZHR7O+FTaYfSil3A/G2aKuxbupzE+uy1vHqjlcZFzoOD0cPe4ShKB3WxXrOLSU6Orp2mOK0kpISMjMz0el0592BCyGIiIhg586dLFu2jMcff5zx48czffp0+vTpw+bNmy94HReXP+dmTJs2jUcffZTCwkJ27NjBuHHjAJgzZw6LFy+mf//+fPzxx6xbt65ZX5ujoyNgHS5qytj8udp9rRWN0PDI4EcorSnlnd3v2DscRVFsZPz48VRUVPDJJ58AYDab+ec//8mcOXNwdnZm5cqVFBYWUllZyeLFixkxYgTHjh3D2dmZG2+8kXnz5rFz504iIyPJz8+vTeRGo7HOHrWrqyuDBg3i/vvvZ+rUqWi1WgBKS0vp2rUrRqORzz//vPZ4Nzc3SkvPf0YXGRlJRkYGaWlpAHz66aeMHj3apt+fM7X7RA4Q2SWSGb1m8O3Bbzleftze4SiKYgNCCH788Ue+/fZbevXqRUREBAaDgeeft47cDh48mJkzZ9KvXz9mzpxJfHw8+/btY/DgwcTGxvL000/z+OOP4+DgwHfffcdDDz1E//79iY2NZdOmTXVe95prruGzzz47a8jl2WefZciQIYwYMYKoqKja16+99lpefvllBgwYQHp6eu3rBoOBhQsXMnv2bGJiYtBoNMydO7cFvktWQkrZYo3XJT4+Xtp6Y4ljZceY8sMUZkfO5tEhj9q0bUXpbFJSUujdu7e9w+jULvQzEELskFKeN4zdIXrkAIGugUzrOY0fUn/gZFWLLCxVFEVpkzpMIge4ofcNVJur1QwWRVE6lQ6VyCO8Iojzi+PrA19jkRZ7h6MoitIqOlQiB7g26lqyyrL4Pft3e4eiKIrSKjpcIp8QOgEvRy9+Sv/J3qEoiqK0ig6XyPVaPZPDJ7M2c62qwaIoSqfQ4RI5wNTuU6mx1LDqSOuXk1QUpfkKCgqIjY0lNjaWgIAAgoKCaj+uqamx6bVOnjzJu+++a9M2W1uHTOQxPjF0c+/GkkNL7B2KoihN4O3tze7du9m9ezdz587lgQceqP3YwcGhzvOastxdJfI2SgjB5eGXs/34dlWvXFE6iAuVkgVrHZS5c+cyZMgQ/vWvf5Gens7QoUOJiYnh8ccfx9XVtbaNl19+mUGDBtGvXz+efPJJAB5++GHS09OJjY1l3rx5dvnamstWe3a2ORO7TWT+nvmsyVzD1ZFX2zscRWm/fn0Yju+zbZsBMXDZi406pa5SsmDdSWjTpk1otVqmTp3K/fffz3XXXcd7771Xe/6KFStITU1l69atSCmZNm0aCQkJvPjiiyQmJrJ7927bfX2trEP2yAF6evakm3s3VmeutncoiqLYQH2lZGfPnl1b4Grz5s3Mnj0bgOuvv772mBUrVrBixQoGDBhAXFwc+/fvJzU1tXW/iBbSYXvkQgjGh47nk6RPKK4uVuVtFaWpGtlzbin1lZI9sxRtXaSUPPLII9x1111nvW6LeuD21mF75GCdU26SJhKyEi5+sKIobVpdpWTPNXToUL7//nvAujnEaZMmTeKjjz6irKwMgOzsbPLy8uosRduedOhE3senD/7O/moaoqJ0AHWVkj3XG2+8wWuvvUa/fv1IS0vDw8N6N37ppZdy/fXXM2zYMGJiYpg1axalpaV4e3szYsQI+vbt224fdnaYMrZ1eWHLC3yf+j0J1yTgrHdulWsqSnvXnsvYVlRU4OTkhBCCr776ii+//JKffmp/K71bvYytECJDCLFPCLFbCNE6GbqBxoeOp9pczaZjdReSVxSl49ixYwexsbH069ePd999l1dffdXeIbU4Wz7sHCulbHOTtuP84/Bw9GDt0bVM6DbB3uEoitLCRo4cyZ49e+wdRqvq0GPkADqNjlFBo1iftR6TpfmbnCqKorQ1tkrkElghhNghhLjzQgcIIe4UQmwXQmzPz8+30WUbZmzoWIqri9mVt6tVr6soitIabJXIL5FSxgGXAfcIIUade4CUcoGUMl5KGe/r62ujyzbMiMAROGgcWJO5plWvqyiK0hpsksillNmn/s4DfgQG26JdW3HWOzOk6xDWHl2LPWbpKIqitKRmJ3IhhIsQwu30v4FLgcTmtmtr40LHkV2WTerJjrEkV1E6Oq1WS2xsLH379mX27Nm1RbKaYs6cOXz33XcA3H777SQnJ9d57Lp169i06c9Zbu+99x6ffPJJk6/dGmzRI/cHNgoh9gBbgaVSyt9s0K5NjQkZg0CwNnOtvUNRFKUBnJyc2L17N4mJiTg4OJxVAAuaVrIW4IMPPiA6OrrOz5+byOfOnctNN93UpGu1lmYncinlISll/1N/+kgpn7NFYLbm4+RDjG8Ma46qcXJFaW9GjhxJWloa69atY+TIkUybNo3o6GjMZjPz5s2rLU37v//9D7DWVbn33nuJjIxkwoQJ5OXl1bY1ZswYTi9I/O2334iLi6N///6MHz+ejIwM3nvvPV5//XViY2PZsGEDTz31FK+88goAu3fvZujQofTr14/p06dTVFRU2+ZDDz3E4MGDiYiIYMOGDa36/emwRbMuZGzIWP67878cLz9OgEuAvcNRlHbhpa0vsb9wv03bjOoSxUODH2rQsSaTiV9//ZXJkycDsHPnThITEwkPD2fBggV4eHiwbds2qqurGTFiBJdeeim7du3iwIEDJCcnk5ubS3R0NLfeeutZ7ebn53PHHXeQkJBAeHg4hYWFdOnShblz5+Lq6sqDDz4IwOrVf1ZQvemmm3jrrbcYPXo0TzzxBE8//TRvvPFGbZxbt25l2bJlPP3006xa1XqlQTr8PPIzjQsZB8C6o+vsG4iiKBdVWVlJbGws8fHxhIaGcttttwEwePBgwsPDAWtp2k8++YTY2FiGDBlCQUEBqampJCQkcN1116HVagkMDGTcuHHntf/HH38watSo2ra6dOlSbzzFxcWcPHmS0aNHA3DzzTeTkPBnQb4ZM2YAMHDgwFavqNipeuThHuF0c+/G2qNruTbqWnuHoyjtQkN7zrZ2eoz8XGeWrJVS8tZbbzFp0qSzjlm2bFmLx3cuR0dHwPqQtqnj903VqXrkQgjGhoxl6/GtlNa077KViqJYS9POnz8fo9EIwMGDBykvL2fUqFF8/fXXmM1mcnJyWLv2/EkOQ4cOJSEhgcOHDwNQWFgIUGdZWw8PD7y8vGrHvz/99NPa3rm9daoeOVjHyT9O+piN2Ru5LPwye4ejKEoz3H777WRkZBAXF4eUEl9fXxYvXsz06dNZs2YN0dHRhIaGMmzYsPPO9fX1ZcGCBcyYMQOLxYKfnx8rV67kiiuuYNasWfz000+89dZbZ52zaNEi5s6dS0VFBd27d2fhwoWt9aXWq8OXsT2X2WJm3LfjGBIwhP+M/o9dYlCUtq49l7HtKFq9jG17otVoGR08mg3ZGzCajfYOR1EUpdk6XSIH6/BKmbGMbbnb7B2KoihKs3XKRD40cCgGrUGt8lSUeqi6RPbT2O99p0zkTjonhgUOU0W0FKUOBoOBgoIC9f/DDqSUFBQUYDAYGnxOp5u1ctrYkLGsPbqW5MJk+nj3sXc4itKmBAcHk5WVRWvvHaBYGQwGgoODG3x8p03kY0LGoBM6lh9erhK5opxDr9fXrnhU2r5OObQC4GXwYkTQCJYeXorZYrZ3OIqiKE3WaRM5wNTuU8mryGNH7g57h6IoitJknTqRjw4ZjYvehV8O/WLvUBRFUZqsUydyJ50TE0InsPLISqpMVfYOR1EUpUk6dSIHmNpjKmXGMlZnrr74wYqiKG1Qp0/kgwMGE+wazHcHv7N3KIqiKE1is0QuhNAKIXYJIdrVgLNGaJgVMYvtuds5VHzI3uEoiqI0mi175PcDKTZsr9Vc1fMqdBqd6pUritIu2SSRCyGCgSnAB7Zor7V5O3kzPnQ8P6X9pB56KorS7tiqR/4G8C/AYqP2Wt01kddQUlOipiIqitLuNDuRCyGmAnlSynpX1Qgh7hRCbBdCbG+L9Rvi/ePp3aU3nyR/gkW22/cjRVE6IVv0yEcA04QQGcBXwDghxGfnHiSlXCCljJdSxvv6+trgsrYlhGBOnzkcLj7MhqwN9g5HURSlwZqdyKWUj0gpg6WUYcC1wBop5Y3NjswOJoZNpKtLVz5O+tjeoSiKojRYp59Hfia9Rs+NvW9ke+52Ek8k2jscRVGUBrFpIpdSrpNSTrVlm61tZsRM3PRufJT4kb1DURRFaRDVIz+Hi96Fa6OuZdWRVWqBkKIo7YJK5BdwY/SNOGod+Wif6pUritL2qUR+AV0MXZgZMZOlh5aSU5Zj73AURVHqpRJ5HW6OvhmARcmL7ByJoihK/VQir0NX165M7TGV7w9+T2FVob3DURRFqZNK5PW4te+tVJur+Sz5vPVNiqIobYZK5PUI9whnQrcJfLX/K8pqyuwdjqIoygWpRH4Rt8fcTqmxlK8PfG3vUBRFUS5IJfKLiPaOZnjgcD5J/kSVuFUUpU1SibwBbo+5ncKqQhanLbZ3KIqiKOdRibwB4v3j6e/bn4WJCzFajPYOR1EU5SwqkTeAEII7Yu7gWPkxfjv8m73DURRFOYtK5A00KngUvbx68cG+D9TGE4qitCkqkTeQEILb+t7GoeJDrD261t7hKIqi1FKJvBEmhU0i2DWYD/d9iJTS3uEoiqIAKpE3ik6j45a+t7DvxD62HN9i73AURVEAlcgb7cqeV9LF0IUvU760dyiKoiiASuSN5qh15MqeV7I+az15FXn2DkdRFEUl8qaY1WsWZmnmh9Qf7B2KoihK8xO5EMIghNgqhNgjhEgSQjxti8DaslD3UIZ2Hcr3qd9jtpjtHY6iKJ2cLXrk1cA4KWV/IBaYLIQYaoN227TZEbM5Xn6c34/9bu9QFEXp5JqdyKXV6Rqv+lN/OvzcvLGhY/E2ePPtgW/tHYqiKJ2cTcbIhRBaIcRuIA9YKaU8b26eEOJOIcR2IcT2/Px8W1zWrvQaPdN7TSchO4Hj5cftHY6iKJ2YTRK5lNIspYwFgoHBQoi+FzhmgZQyXkoZ7+vra4vL2t3MXjOxSAs/pv5o71AURenEbDprRUp5ElgLTLZlu21VsFswwwOHq4eeiqLYlS1mrfgKITxP/dsJmAjsb2677cWsiFnkVuSqh56KotiNLXrkXYG1Qoi9wDasY+S/2KDddmFMyBjrQ8+D6qGnoij2oWtuA1LKvcAAG8TSLp1+6PlR4kfklufi7+Jv75AURelk1MpOG5jRa4b1oWeaeuipKErrU4ncBkLcQhjWdRg/pP6gHnoqitLqVCK3kdmRs8kpz1EPPRVFaXUqkdvImJAx+Dj58HnK5/YORVGUTkYlchvRa/Tc0PsGNh3bREpBir3DURSlE1GJ3IaujrwaF70LCxMX2jsURVE6EZXIbcjdwZ2rI65m+ZHlHDp5yN7hKIrSSahEbmO39L0FZ50zr+943d6hKIrSSahEbmNeBi9ui7mNdVnrSMhKsHc4iqJ0AiqRt4Cbom+ip2dPntr0FMXVxfYOR1GUDk4l8hbgoHXguUueo6i6iPvX3k+NucbeISmK0oGpRN5Cor2jeW7Ec+zI3cHdq+7mZNVJe4ekKEoHpRJ5C7q8++U8f8nz7MrbxbTF0/g48WNOVJ6wd1iKonQwQsrW314zPj5ebt++vdWvay8HCg/w0raX2HZ8GwBBrkEEuwbj7eSNp6MnLnoXXB1ccdW7Wv99+m8HVzwdPfF39kcIYeevQlEUexNC7JBSxp/3ukrkrSetKI0N2RtIKkgitzyXE5UnKK4pptxYjkVa6jzPy9GLWL9YxoeOZ2K3iTjrnVsxakVR2gqVyNswKSWVpkrKjeWUGcv+/LumnPzKfJIKktiSs4Wc8hw8HD24sfeNzOkzB4POYO/QFUVpRXUl8mZvLKE0nxACZ70zznpnfLnwxtRSSnbl7WJh0kLe2f0OSw8t5YWRL9DX57x9rhVF6WTUw852QghBnH8cb417iwUTF1BtrmbOb3NYnrHc3qEpimJntth8OUQIsVYIkSyESBJC3G+LwJS6DQscxldTv6J3l97MWz+PZYeW2TskRVHsyBY9chPwTyllNDAUuEcIEW2DdpV6dDF04f1L3yfOP47HNj7GpuxN9g5JURQ7aXYil1LmSCl3nvp3KZACBDW3XeXiDDoDb417i+6e3fnXhn+RU5Zj75AURbEDm46RCyHCgAHAlgt87k4hxHYhxPb8/HxbXrZTc3Nw4/Uxr2OymHgw4UGMFqO9Q1IUpZXZLJELIVyB74G/SylLzv28lHKBlDJeShnv63vhmRlK04S6h/LUsKfYm7+XT5M/tXc4iqK0MpskciGEHmsS/1xK+YMt2lQaZ3L4ZMaHjmf+7vkcLTlq73AURWlFtpi1IoAPgRQp5WvND0lpqkcGP4JWo+WFrS/YOxRFUVqRLXrkI4C/AOOEELtP/bncBu0qjeTv4s+d/e5kQ/aG2rouiqJ0fLaYtbJRSimklP2klLGn/qiJzXZyfdT1+Dv789r217BH+QVFUVqfWtnZwRh0Bu6JvYfEgkRWHFlh73AURWkFKpF3QNN6TKO7R3fe2/NevVUVFUXpGFQi74C0Gi23x9xO2sk01h9db+9wFEVpYSqRd1CXhV9GkGsQ7+97X42VK0oHpxJ5B6XT6Li1763sO7GPrce32jscRVFakErkHdiVPa/Ex8mH9/e9b+9QFEVpQSqRd2COWkduir6JLTlbSCpIsnc4iqK0EJXIO7hZEbNw1buyMHGhvUNRFKWFqETewbk5uDE7cjYrj6xUNVgUpYNSibwTuLH3jWiFlkXJi+wdiqIoLUAl8k7Az9mPK3pcweK0xRRWFdo7HEVRbEwl8k7i5j43U22u5sv9X9o7FEVRbEwl8k6iu0d3xoaM5cv9X1JhrLB3OIqi2JBK5J3IrX1vpbi6mB/TfrR3KIqi2JBK5J1IrF8scX5xLEpapPb2VJQORCXyDuLXfTmEPbyUrKL6h01u6XsLOeU5rMhQJW4VpaNQibwNKiqv4XhxVaPO+WFXNgCJ2cX1HjcqeBTdPbrzwb4PMFvMTY7xTG+vSeW3xOM2aetcKTklrD+Y3yJtd0Tf7chi/cF80vJK+Xa7WjfQWejsHYDyJ6PZQrXJwvAX11BpNHPg35Nx1GnPO27HkSJOVtQwvrd/7Wt6rTjVRv2VDjVCw92xdzNv/Tx+Tv+Z6b2mNzvuV1YcBCDjxSnnfa6ovAYPJz0ajWhUm9UmMzqNhsv+uwGA6QOCEMBr18Q2O96O7MFv9wAQ0sWJo4WVXBodgIez3s5RKS3NJj1yIcRHQog8IUSiLdrrDGpMFg6fKOcf3+xmz9GTLE86zo0fbKHvk8upNFp7ypGP/8Yve4+dd+7M+Zu4bdH2s8rT6jTWH6XJcvGNJCZ1m0Q/n368vettKk2VNvqKzmcyWxjw7EqGv7iG9PwyAJ76OYl31qZd8HgpJRtS83l7TSqRj/9Wm5QAftyVXXvXoZwvr7SK11YerP04u8j6c92ZWWSvkBpk/rr02t8NpelsNbTyMTDZRm11WGEPL+X5ZSmsP5hPxOO/8ubqVH7Ymc2V7/zOXZ/uYMvh8xfrLNuXU2d76w/m88RPifxvfTo/77Em/HN75Ifyy6gynj2EIoTgn/H/JK8yj/l75l807hqThbs+3U7ysRLS8soaXN+8pMoEwPGSKsa/at3g4uNNGby8/EDtMVJKHv5+L99sP8qm9AL+8uHW2h7+jxdI3C8sS6G4Uj2oPde9n+/izdWptR9bTv2IjhVXsv5gPnM/3dHm6tIXVxp56bf93PD+FrvFUFRew9HCCn7anc2D3+5hU/oJu8XSHDYZWpFSJgghwmzRVkeScaKcfdnFjI70xVFnfc9ckHCIBQnpuFNByu7NXKIpxgEjesyY0FAiXSjBmRzZhRJccdRpKa4w0v8Z68PJByZE1LY/Z+G2865pNFvIK62i2mjB393AuFfX4+msZ9f/TUSIP4c34vzjmNlrJouSFjEhdAL9fPvV+XWk55exPCmX5Um5ADw0OYq7x/Rgc3oBRwv/fLgqpUQIQVpeGXM/28FLM89u8/r3/6j995DnV/H27GiKctI5uH072TuqiOuqZ6KmnAocKZdOHJW+FOAO/Bn3/xIOgYBHLutd+1puSRX+7oZ6fxYd1Y4jRRw7WUnisWKcqWKg5iBmNOywRFCNA+l55Tz2o/VGucZsueBQnb0cOV7ACM0+skt9WPj7Ya6KDcLLxaHVri+lZMCzK9FjYpgmiQwZwHc7slj1j9H09HNttThsQdjqXfpUIv9FStm3js/fCdwJEBoaOvDIkSM2uW5bsvvoSUK7ONPFxYEle45x35e7ALisbwD/HuvBO+++wSDNfuI0qfiLkxdtr0Q6U+jQlcQqH/ZbQkmRoaRYunEMb85Mbmd67PLevLk6ldJqEz/dM4Ir3/kdgNev6c/oCD+6nPEfpbSmlOk/TcegM/DFlC9wd3A/q631B/MZ1t2bXxNzuP+r3bWv+7o5suofo+n/9PkzX7Y+Op5nftvAb2nb8PYspNh4Ap2uBGdRipOmAgdRjQs1uEgjzhhxkOBmseBlNuNpseBltuBvMhFqMhFsNFElndhnCecPSzQJln7slj0Awf5nJ2PQa9mUfoLr39/CO9fHMaVf14t+TzuasIeXAjBTk8AT+k/wENY31nzpwb+Md7LWMqD22N1PTMTTufUS5YVsyyjk498zuLe/wO3bWQSLE5il4BXTNRTF3cMtI8JxNegI8nRq0TiklHyy+Qj//nkPnzs8x2DNAYxSy13GB1hjibvg8562QAixQ0oZf97rrZXIzxQfHy+3b99uk+u2FRaLpPujy4jwd+XHv46gz5PL0WJmkmYbt+h+Y5DGOlxwxOLHTtmLFEso2dKXPOlJNXpMaNFhxk1U4Ek5XUUBISKPEJFPD3GMbpq82mudlC6kWLqRJLuRbOlGkgwjXQZiqucGS68VGM2SFQ+MIsLfDYB7v9iJdDzExrLniA8YyLsT3uFEqYUPNhwi2MuZZ39Jpm+QO4nZJWe1ZdBr+P7u4Ux5c+OfLwoTOvc99AzfRnZ1hvUlCV5mSYDZiEFacJQSKbWU4EQ5eiqEjiqhRecEZZZKjKLm7KAldKlxIKa6hpHVhYyoqqSmxpevzGOZeecTVGudeXtNGiuSrXcKiU9PwtWxczy/f2ZJMh/9fhiAW7S/crXLV/zXLYRtBiecHSGkvJRbS/L5pvhOVliGALDp4XEEtnCCvJiwh5fiSBYJuYcAACAASURBVA0b3R9HU13M/xlv4XLtFqZqt/B+4LM8d6gHcOEH57b0x6ECrl3wBw/ovuV+3Y88abyZq7Xr8BMnGV/9MntfvKZFr99UdSXyzvFb3woKK6xJ6GBuGVsPF3CpZhsP676ku+Y4GRZ/XjJey1LLEDKl/1nnDe/hzbb0AvqHePL89L61yXFspC9R/QK59dQDP1cqiBRH6a3JJFocIVpzhBs1qzDorOPF1VLPARlMkiWMZNmNJEsY+2UoFViHHE6PnV/6egLjo/zwczfwy94cwAmdx3S28i23L7+Ljb9fhjT/eVt5bhIHqDJaqKyqIlzkEKY/iNlzF6keuZTpJI6lRh4qLSO+qgp9lRcHT8WSLLux19KN43Th3LuJl2bGkFVUyVtr9iN0Ffz4t94cOnmEh39eQ57hGGudj7L+1M1CULXgqoqlOHz6C4srp7DWfDmnf42fWJzYKWa1FFcYa5P4KM12nPyXcLVHV3ToGB06GJPJgXUZm3kgAK5w/oKgY2FkS38qamwz3bS55mqX4FuTxfXGR9lk6ctKSzw9xDGuyn2bV3iJalr+rkFKcKecW7W/8Yt5KIvMk9hh6cUvjo9zg3YNZsvVaBs508qeVCK3kdwS67xvfwqJXnMrCxw2kmoJ4q6av7PSEo+ljufK3X1d2JRegKujlj6BHvxtXE/eXJNGWbWJqf261s7cKMOZHTKSHebI2nO1mAkXOfQRGURrjtBHZDBZu43rxFoALFKQLX04Kn3JlH4clX4ck94UHXQjSboRIlwpl06YiqPw87iBfeJrXLrvR3diBC4lkbhYLHiLEvwowlcU01UUEiaOEypyyP22lPhgA6tcnDEDg6sEl1u6sy8jmCWWcF6S3SjF+YJf87T+gbUPZwG8nB04mFsG6JAmd2L9Yon1iyUpqhcfbjwMSITDCeZdZeGP4wm8k7+D+Z4wvmIV8wvX87/yO9gmo8g8Y7w+6Vgx/u4GfFwdm/FTbZsSj1nXCniIIgzBX/KpizvmwsEM8Z7DG2NHYjRb6PXYLwQF/sISj81MNL/ND8efoaLGZOfIrR2S23S/8qt5EJss1pv3+TcNZdXv93Ff9r+Ypt3Et+YxFFca8XBq2WmTM7UJuIlK5pumAZAou7PJHM0NulVsSs1lZGRAi17flmySyIUQXwJjAB8hRBbwpJTyQ1u03db9sDOLxOwSRkb4cKlmG//RL8Ah18STppsZ/5dH6J1VyvJVf84mMOg1VBn/nCIY5u0CgOl0j7lPAG+uSaO0yoRBryXjxSnkFFcy7IU1gLUHvym9gAh/V3r5u7F0r5Y0GcxPlktOtSjpSiFxjkfpYTpEuCaHUJHHeM0ufEU9i4Xy4GCRnmd9urA7YAUV/svpWV1NeI0JN4sFDZJDGgdW6ZxINThSrXFAZ9bhXNSbE4XjmThtIjMGhdLj6EkWnhqXB3Bz1FFabWJgNy/KqkwcyC0lzMflrEt3cXHgiv6BfLU1kyX3XVL7+j1je9LTz5Xxvf1IzC5mXJQ/f+VWHlmylu8Pfs8ar02sDDEyuewd+ueOIFHcVXvulDc34ufmyNbHJjTuB9oO5JdWA5LowHfZ7KKnT9WlaFxn88AE6wNgvVbDR3OG0N1nPK//djMrvQ4wsOw3KmqG2TXusmoTV2vX4y4qeMd0JV/cPgSNRjC0uzfL9g4mxRLCLdrlfGsew4Pf7uH9m84bQbCZapOZy7VbSLGEkiTDal//yjyWN7XvsHX/Joic0WLXtzVbzVq5zhbttEf/+GYPIJlw4hMWOMxnj6U79xvvIUN25eHufhwqOHuF5pd3DGX6u5tqP3Y/1etwOTW2G+BhHQoZ2t279piuHk5Ed3WnuNLIZ7cN4eklSVw/pBsB7gaW7s1BI/6cbgaCHLy5d+5VLEg4xJu7snnmyj7M/CkJJ6oIEEV4UYqnKMNblOBENXrM6DAzppsnkWkncTNUcdS1kDSnYvY4V2DWmEBYsJhckCYPzMVBmMt7YirrTZG0xu1walZO3yAProkP4crYQDyc9fQJ9Kj9Ot5Zm8bLyw9QbTr7Fl+n1RAb4knSM2fPYO3i4sB1g0MBGBf156yUF64YS+i6UF5aMRpX7zWs6rIRQ/hOZpU8jKz+jS92FwCQV1rd4J9je1JSZSTIazmJ7qUMK/Chwv1qPrt9yFnHjIuyDuFN6PMCKdunU+W3jvIq+34/ckuqmK1dT7pDFG/89ZazZoZEdHXni33jeVb/MRHiKNlF7nU3ZAMHUlO5Q6Tyumlm7WvXDQ7ll60V1EgtgcdXAbZL5FVGM+sO5DO5b8v08tXQSjNpsPCc7kOGH1nLj+YRPGy8o3aMz6DX1CZogBuGhBIT5MHsgcHcPDyMvkEerE6xPqg7/QP2cXUkYd7Y2oR+2rL7R9ZO73v6Sustqcls7dlfGRtUO+fa182RqAA3end15z+z+jFneBhl1dZb6koMHJZdOUxX3B119PRzZWfmSWKCPNiXXUyecxA/mrOZEOrHvpQ8GsNBa53WptUIXpp14amMLg7WYywWycJbBvHu2jRCujjTJ7Dx/2ndDDqwOFOWP5W5/a/n98wn+NjzBHmfj2ZF6t8An0a32V4cK82hym89Qypr2Jp3B90c6x4yCfDyxutEHMmBuziQuoDx0U+1XqDnqDiyixhNJmn9nz5vet/tl4Qz9NchPKVbxFTtZhIc6p4O2+w4akxkbf4GjV7Sddi19M9wYnhPH9wNekpxZrOlD1EFG2r/v9nCc0tT+H3LZsZ1eRWHmf+D7qNt0u5pqtZKc5hNvKx/j+t0a3nTdBWvuvzzrAc1QojaxSs3DevGc9Nj0Gk1vDy7P32DrD3VcVF+LLn3EmYPDK49L9TbubaHe6Zzf6l0Wg1bHx3PSzP78fO9I/jxr8PZ9tgEPr3N2jvTazX0D/G0Jr1T3r0hDoARPX1qe/Ez44KAPxfgnL5LeGJqNFseHV97rvOpRHzmNL/+IZ6nrnXxX/hrBoVy/ZBQ7hnbk7GRfnw7dzivXR2LXtv4X0OD/s/50ENCIhnm/zYhBX35TVNNUPjLeOqsY/DT3/29ribarT/y3kaDmb/6TOIkHpRW1b1AKrSLMzuLpxNSY2Zlwc+tGOX5XA/+QI3UYoy66rzP6bQaTuDB75a+TNNsxlHXcg8a1x/IZ5RmLxkWf0aNGMFP917CQ5OjuHZQCKMifNkiYvCvyeS9pZsu3lgDZRSUM0Ck4VCeg8XFz2btnqYSeSMcO1nJ9oxC8kqquOfzHex+bw4ztRtZ3OVWYm58mR/vuYQJvc/+IY2K8AXg6viQC7YphCAm2KPJ7/x+7gYcdBr6BXsyINTrgsc4O5watnE3cHlMV76/exivXR1bu9Kvm/fZY9a6U0/rHfWasx4Wrn1wDD/dM4J3ro+rfS3M2/pA82QDVls6OWh5fnqMTeYy686YUTCypw9dPZ1JzruR0Kzx5Okl3mFv4qY/yq7Mi8/Xb092HE8k3byXG4rL8RvyTwBKq+rukfu6OmLEgYCTPUjTGklJX95aoZ6lqsaE2L+UTZa+uHWpO5EtswwhTJNLd1quHMOalByGaPazyRJ9VofAy8WBT24dzEZjFADZu1fZ7JpCCGI1aZRIJ3ZW+Nqs3dNUIm+Eaxf8waz3NvP0L8n0TH6H2PwlvGm6CkY+yNgoP3zdHPng5kFnnRPh70bGi1Nqe+D2EObtzLWDQvjiDmtPfWC3Ljg5aPnr2J7Wj8O8+PX+kbXHX3tqXHpUL9+zpmD5uxtqe+Cf3TaEW0eEMybS+kvp28qzQ07HdXlMADqtpnYsfl/ZpQQfvYxqjRmvbu+i151gx5FCpJQUltfU12SbZjJb2JlZxJtbX8XdbMG1MBZvf+td3LAznqecS6MRzBoYzN6iK3GwSD75463WCvksG//4gzBNLqsscfW+ka839wegb8X5q5ZtRZO7F3dRwR+WPmcl8tMmjp9IqXRilMPBC5zdNAIYoEljr6U73m62n8uvxsgb4fT0Nqekr3hA/z3fmUexP+o+7osNPOu4zY+Mo7KNzNkF623rizPPH3Oc1CegduGFq7/1V+Gq2EDiQr0uuiDjkl4+XNLLOg7du6s7UQEt+3DqXN1O3Qn0D7a+sQwK8+LuMT2Yvy6dnRVjmXC0jKTQDQR2+y+zFjjy5OVDeGpJMgnzxhLqfeFpkW3Z66sO8u7GLbj23M5fS0qQA+7GyUHL2gfH0NWj/vIEr8zuT/m0Ptz7wXwSZAZGUzV6Xeu98RZXGCnYZR3WWWuO5VmHC5cJWHTrYPYcPcnRzaFElW9tsXi6lVpXXG+29K4tnXGm+yb0JnFnPyKrdp/3uaZylFVEiUzmW6ZxyTmztmxB9cgbIdjLib7iEM/pPmSjuQ8PG2/npuHh5w2LdPVwortv+6rVoNEIEp+exKtXN35BTWsncYB+wZ6seGAUd4zsDlhvXR+aHMXie0YAsKryCsZmR1GhqyYs+C1eXZkMQPbJlqv22JL2ZhXj4LUZnZT0Kg3ipius0yrDfVwu2Ks8l4ujDoMxnhKtYMfeRS0d7lluW7SN0BMbSLGEkI1vncOIoyN8+dv4XqQ4D6J3zT5MVbavinjsZCW9KveSbulKPl51Pp857jmAbpYsKkvOL2TXFGE1qeiEhSTRyybtnUsl8kawVBSx0OVtTuDBfcb7MKEjugkzLtoqV0fdBVez+bo5MuuMh7FtRYS/23l1zmNDPJkZZ431y7I5zMzz5IRzCe6e1uTVXodXaiyVOHluZWJ5BeZeNzap+FW500wcLZIVKd+22uKgKqOZg0eyGKTZTwJxrP7nxWdrZHsPwwEjNz/zts0rNv53VSr9NIdO1eypW1C0dc79nS+9z/hX1zX7ugEV1mGaebdc2+y2LkQl8npsTi9gypsbyCmupLSyhqctb9PFfAL9NYsowprA3Q0dv2j/tscm8Mrs/vYOo8HuHNUdf3dHNj8ykaqw/zDzpJESrxTcPDaRX9q4nZfaiiKxDbPWyJRiM/5DZjWpDS8PX8IrnFlXdYw7F222cYQX9ubqVOI1B9AJCynOg+jRgDvV6oB4zFIwWHOAjIL6ty5srEBdMf7iJKNHT2DTw+PqPC4qzvqG05dDpOeXN/u6AVWHKdO40yO8/jeQplKJvB63LdpG0rEShr2whvz17zNRu5OUvg/i1nO4vUNT6hEZ4MaWRycQ4GFg+vA+7Dp+FwMqq3EMWEJqQfusulmm+Z3QGiMpFcMJ96/74WZ9ogLcMZX2Jl+noezYUhtHeGG7Mk8yWLOfaqnjkGNUg87x8fElRXYjXhw4a3MRW/Astg6x+fQaUm8BMeHsRa4uiH6adJtcN9iUQZ4hHGw0L/1cKpHX43SRoWCRj9/mZ/jd3IequDsw6NW3rb2ICfJgj4ygW84wdJjZkveMzfYqbS255bmUatKZUl7Bj+ZRTd667ZYRYYyLvx2AENeWmxVypooaE0M0+9kjeyAcGvaQuYevC9sskQzQpFFZabs7qIwT5VRkbMeCgICYix5/3DWKfppDAJgtzRjikZIwy1EKXVqmNw4qkV+UwMLLuv8hEfzLeCcBns42W+2ltDyNRhDs5cQX1TOZW2ogV5/PUytebVe7DL2Q8BUIiCp1J9ep6cnAoNdy34SRhJj1lDjnUFnd8uPkwlhOjDjEVksUz1110QrXAPQJ9GCrJQpnUU1Qle2mAI55ZR09TOkc0waBo9tFjy/tEkOQKMCb4trV0U1xLDMdd1FBqXvLPOgElcgvqMpo5tdTW6z9RbuSYdpknjXdSDa+tTvRPD6lN1/fOdSeYSoN9PO9l/DxbcMIG/A6wyuqWJbzGe9t3GnvsBps3ZEl9KmuZrfxElb/o/lLuyMNkSQZBL9vOn9jEFsLKk1EJyxstUQ1eC2Fg07D/bf8BYCelXtt+sCzr+YwGfqGJVSHYOsMrihNZrMS+fOLfgTgiDa0yW1cjErkFzD+1fXc/flO/Chinu4b1pv78Y15DEDtdKXbR3ZnSD0LMZS2o4uLAyN7+RI/eDjD5UQEZnbkvGDvsBrkePlxzE65TCyvIDPocrxtsPCqe+BUqjUaVm56n9/TWm6PSrNFElWzFzMaHr1zTqPOjerVi2KnEOLY36wkeiZvigkUhaTpGnZX0zvW2lGLEpmU1bN6tj5SSgKqrbXjY+NaruOnEvkFnJ5r/Lj+Mxww8X+mW6hrazWl/XB11HHzXf9l1klBikwnIWOlvUO6qHWZ1tryPmUB5Arb1OjoHzQarQSNSxoHjpfapM0LKa8xMViznwL33kSFBV78hHOU+cYRq0njcL5t5pP31WQAkOvSsIeubl26UmXwJVqTWW89m/oUVRiJEFlUOHgTG9WzSW00hErkdRih2cc07WbeNU0jU/oT4e/Khze3XH1kpZVo9aQUzyG8xsi/1/8f1aa2Pa98bfoSuhmNbK0agcQ2QwwB7h74VLqT6lxDaWHL1TSpKC8nVqRT4N20/zdOYYPxFcWs29b8FZapuaX0FdaecUDU4AafV+3dmyiRSWkT7woO5pYSocmiyrPlxsdBJfILcsDIM7qPybD48575CgI9DKx4YDTje/tf/GSlzVtf1ZthJ7qSQznT3n24thxwW1NWU8bWgkRGlVexzDyU56dffKZFQ3g5O1BVFsUBRwcsmS1XRMuctR1HYaTEb9DFD74ArwjrUEROcvMrWG7LKKKv5jDHtIH8ZXTDS+SafaPpKbIpr2jaiuCfdx2ll8jCJbhhD3qbSiXyC7hJu4IemhyeNM0hIsiXr++y784qim39+6q+fF98GwMqayh1WcXvh1quV9ocSw6uxoTEsyyIa8cMsFnZB09nPTnl1l5ySXkC+4+fvy+rLegyN2ORgoqAhveAzyQCYjAJPd1r9jf7gWeNyUyM5jDePQedtxq4PpqAGByFCVHYtPnkojgbF1GNY6BK5K1q/Z6D3KtbzDpzf9Zb+nP7yHBCurS/IktK3W4c2o1VT1yNT95gSnWSZ5fMs3dIF7Rs71d4ms3sLR+Ft4vtNiTWazWYq4JxNmsoc8lh4YY0m7V9JsOxPzggQ3B0a+KkAJ0jBa6RxJDW7I2jzRUFBIsTaIIGNOo8hyBr791QkNKk6/pWnnoD8OvdpPMbSiXyM0gpSfn2adyp4EXTdYR2cWZslO2LwCv25+yg45eKmYwuNVPklcjKlGR7h3SWsuoq9lclM7yihtWWeEzNWZByAeseHMdwr0h2OmnJP7jFpm0DYDbieHw7WyxRODs2vchqiXc/YsQhCkubV+zMo8j689UFNa4onCEgkhqpxa14f5Ou639qxgq+DXvA2lQ2SeRCiMlCiANCiDQhxMO2aNMecjJTuUW7nB8sI9kvQ/nlb5d0iloqnZGDTkMNeiryJ2MR8MaKBykoazt7fH6xcw1VWgtuZcGM6B1a58YkTRXm48LIXlPJ0+kItWyw6Vztr7dl8sXiJRhklXVhTx1laxui2i8WF1FN8dHE8/Z6bYwup5bmi66Nqxmk0TtymGC8SlMvfvAFdK3JoFDrA06eTTq/oZqdyIUQWuAd4DIgGrhOCBHd3HbtoXL5MwC8ZpzFlH5dVRLvBNZUj2NkiZZjHpmsSNph73BqJWZ8g05KDpSO4YObB9HFhkMrpw0LmwiAg3Nyk2dlXMhD3+8jY6d1sdE2SxQOTdjK7zSncOtmKIu++4ERL65pcju+ZfvJwg+cuzT63CO6MHzKm5bIg41HOObQrUnnNoYteuSDgTQp5SEpZQ3wFXClDdptVabsPYRn/8JC8ySO4dOmemdKSxKMjpiHAL7f8hi5JW2jOuKByr3EVpp4bd6DLXaNrq5dCcSFTOcyik7k2rTtwZr9p2p+e9ZbnOpignv2oVg6EyvSOVHW9KmiXSsOkKppWnkDk18fvMwnWLI5sXF3LhYzIeaj5DqGN+m6jWGLRB4EHD3j46xTr51FCHGnEGK7EGJ7fn6+DS5rO8nHStj43n2U4My7pmncOao7z9loqpfS9k2dcB3xxa6kuefz1q9L7B0OmfnJHNMa8asOwdezZTcoifaIZYfBkYoDtluur8HCYM0BtlqiSHlm8gU3Em8oR72eDMdIYjXNeCBbWYSP8RiH9U1bkKPrap1x8sWSZexszB6wRRk4UkO+U/cmXbcxWu1hp5RygZQyXkoZ7+tr+81Hm2P+wg8Zo93D26arqNC48ejlvRtUN1lp3wx6DUGeTjjqtCTlX49OwvGiN22+mUFjSCn5dPUbAFRoJ7X49Yb1uIIqjYalW7602dc9wi0Xd1HBFktvnJoxPn6ad8RwIsVRDDTtLrkwfTsA60sbv7oUQBdo7dRFiUxqTA1fc7Bv1x8AHHMIa9J1G8MWiTwbOPNJTPCp19oHi4W7ahaRJX3wH38vqc9dZu+IlFay76lJrJs3BoBsUw/6FPuwzVDE8199bLeYErNL2F20lZAaC4dl0+ZfN8akXqPQSKh2OUKJjSpCDtdZKxbquo+wSXvBMSPRCQt9RAaWJszeSdyeYG0numnrQbz8QsiX7vQWmei1DZ+DvmWrdSHT7qqAJl23MWyRyLcBvYQQ4UIIB+Ba4GcbtNvinluazBcfvU5fcZh9Efdxx7hoVaK2E9FrNWft2bg7/y84SDhY8A5P/JTIP76x3ea7DbU1aSdpTiY8y7pyoqzlS+16GNzwrPTmD2cNH35nm/+2fYz7yNf68dwcG3WKggYC1l3oy5uwRZ17URLH8eG5G8c26fIBHgZSLN2I0mRiNDf8jaSb6QhZ0odCk+0fVJ+r2YlcSmkC7gWWAynAN1LKpOa22xoWbTjIyMz5JFm6cSz0CnuHo9hZsTmA7ieD2eVaxd6dX/HDzuzmbSjQSEcLK9i6cz4mIRjY81re+8vAVrnuFb2nk+GgJ+/w4uY3JiX9TIkcdhnQpH1FL8jVjzKnIGI1aU2qIx9Qvp/DDk2vdeLr6sh+GUqkyKKmpuHDO2GWTA5agjlZ0fJvyDYZI5dSLpNSRkgpe0gpn7NFmy2tssbMjdpVhGjyecF0Pe5OLf+uqbR9BaY7cJLQxe9nQLbqBhR//3o3etcUnM1w/5Q5xIa07Nzj0yZHzURIEO7J/G9987Y2Mx5PxpMSDrs2bgXlxZT5xDJAk0ZBY2euVJUQYMom1zmyydfWaTWkWEJxFEY0RYcbdI401dBNHuOgDCYq4OKbWDRXp13Z+c9P13Ov7kcSzDFstMTg0ozVZ0r798hlUfxjYgTV0ouAwl5sc5WMcvqNwvKWr45YZTRztLACkZ/MXhcz3uV+6DSt9/vYNyCYPhovklwr2Jt+9OIn1EFKyYvzPwAgxdG2m3XL4HiCRAFFuY3bc3Xfjg0AaBq5ovNc+6V1UwinwoatAF64ZA0OwoRbSF9ev6Z5126ITpnIy6tN9D28EE/KeVVeD1jHwZTO667RPfjb+F5YLLCv4HrczIDfGk6W2qYWdn0e/n4vI/+zlgjtLxRpteSUXNLi1zzXVT0ncdhBT6C56YtuluzNIU4mkiV9yBG2nZlmCLc+qJRHG7fXaMZeayK/dGLzxuuvnzIRo9TiUtSwpfo7t20EoGffobi1wsLCTpXIt2cU8t76dMY/+QW3an9lsWUEYX2HsfmRccSFetk7PKUNMFksWCzOXN5lIructWTs/E+LX3PL4UK0mJHuKThYoKis5Xtw55rc/y4cLZJM8/omt5FXXMkQzX62WHpT04iHgg3hER5HtdQjshqeyAvKqtEf302hPgCDZ/NmjkzuH0qaDCTn4Hb2ZRXXe6yUkihNJiapQfg1fUinMTpVIp/13mZe/HU//9R9iwBeM81mZlwwXT2avvJM6VjuGGldvHHjyMfxNAl+KPoVWdmIRSBNEOTpxBjNLja5aHEtCwbZ+s9rPFx8GFzlyk7HAsqqm7ZrkFvZIXxECX9YelPdiPnWDaHRO5Iow3DO38WG1IYtKFy2L4c+Mg1NUFyzr2/Qa0iR1pkrX2/PrPfYapOFKJFJugzExcWl2dduiE6VyAGiRQYztRtYaJ5MZGQfRkW0rcVJin3dPrI7GS9OIcy7C04nR7PPUcfaVQ+16DU9nR0Y5LqCQjsNq5zWUzOUKo3g0Z+fI6+08aUKPHKtC2BaIpEDpGgj6ScOkXS0YfuMlhUeJ0STj3vP5u+V6ajTst8SQqAoRFdVf4+8ymimtyaTFBnaavWaOk0if+HXFEDyqO5zTuLCu6ZpzVo6rHR8eoeZ+NToeDU3gZrCw1TUmFpkOqIfBWS656A3a6gpi+bK2KatQGwut65XMaKikt9LlzP4hWUNPq+ovIawh5eiP5JAtvTmqPRr1ArIhpp6+TQMwojIa9gDR+cTewFs0iPXawUp0lr8KiN5S72/B1WlBQSJAvZbQnEztM5D606Tyf63/hBjNHu4RJvEm6YZlOCCQW+jea5KhyTQUnF8Gpl6HYuW/43oJ5bzxE+JNr9O/5M/scrFCVHShz8evoz/XmvbqXsNdcPkS5hW6ECN1oSD99oGn5ecU4IOE4PlPjZY+gOiWSVn6+LZa7j1Hw184Kk7vgsLAgKb/8xBCEF1F+vmEOHmw/XesZiPW5fR7JehrTYbrsMn8l2ZRfzzmz1oMfOI7gsOW/z53DwBgDGRalhFqVsXF0dyywcTXubGB1WH6KlP4fMt9Y+PNtaW1GNUGJdTqdFQdHIUfm6ONm2/MZwddPTqPpUrS8px9F7P1uy9DTqvvNrEQJGKm6hku966hdw1g0JtH6BHMGV6H/xL9lBxkRWe6fllBJSlkGYJBEfbzOP+9O9XUOXgTW+RSU5x3Yn82H7rG02ypdtZK4dbUodO5EazhenvbuL7nVlcp11DpCaLbzxv4/1bhvHlHUOZ1t8+t7BK+/DSTGuxpMPH/4JRCIL9v8Cgt+1/mW8/fo3l7locqz2xVAU3aj/JlmDodxXzigpxNDtw5/K/UVhVWO/xRwrKWZGcy2jtHoxSy1pjFBkvTuG2S1qgdKsQAajbcAAAEYxJREFUnPAZxDBNMu+trb8a4r6jRQzUHKTcz3Z3N446LaaA/vTTHOJ4HYk8q6iCtH1bKJSuzJ0y3GbXvpgOncgrqq23d94UM0/3NZvM0WzSD2dMpB/DeniruipKvU7PZio1hhJR2I0dbkaiXZby6ooDZJwob3b7J8urGe7yG0mOjhQXjeLymK7NbrO5QqKHUSl9mJtjwCSKuXvV3fUm89Evr+O7HVmM0exhh4yg2NyyM8CKA4bhL06ybF390yQLD+/GU5QTM3yKTa+vCx1EhMgi6/iF67dnFVXST3OIJEsY/VppZS508ER+usDOw7ovcaKa/zPdQm5py6/UUzqOAHfrQrFdeXPoUW0h7//bu+/4qMp8j+Of35Q0IKEkoQUh1BCRFooQQIo0pSiKF1lgsXFFRBSRFbh6cRUV9a5eKWJZWQsWUFia9IAYpIai0YQqLQKhJrQkM3Oe/WMmEpAUSDk58Lxfr7wyJTPzDUx+c85Tw9bwwfebeaEQbeXxu09Q6/nFvPzuNFZVyMDpdtKsQlem/6Vk1lbJkwgHw7sw2JWMSunP3jN7Gbh4IAnH/rx7UnaHXxVOEm07wBpPE8oWc+fexQjvqJ5Y2y8kpqTluhpi+dRNADhqF+0ooIBarbGJIm3P1fc5PXr8BFFykK2qXpEs4VtQN2whT7vgot3kOFpIMv0da/nIczd7VXWOlpIdYDRrWPBkLDMGxZBJEI7fe3HeDnWrzSA4wE56xvWtwzJvWwqgaG18xtqgQC6e6kBGVulZIuLnkE74i4seF9KY3vlDlIKhS4cyMm4km49uxlDeESnHz3oXkOpp9xbNhp0eZPZ/X99SsQXlDK3NQSOMWFsivabE8/G6P699YhiKiPRtpNrCoHwRt9X7VmIMSt121eUbVMpW7KLYZtQlvFzJzRa/YQv56p2p+KlMJjs/JN2/ClPc9wAwY1ApOOrRLCM8OIAejbyzArdkdKDriYocLJvGsYuf0Hjicr5JOHzNz6lQdLJtZUHoeSrYggh2d2Ncz4ZFHf267Q2IZp9Rhf6O73n+izR2JgxjRNMRJBxL4OFlD9NlThfGfD+GjxM/wV42mZiA9WxVt9Cnyx3Ur1y8C0QFBzj40biV221J2PGw7dDlk7XOZrioM34RtS/sYE9g0a73AkBgeU4FRRLlTqb5yyvIcF0+OifkxDYAthn1qFQM+6zmpvQcBhSxs5lu/ub4ijq2I/za/lMml2tJxwZhekNlrVBWnHicrgETWVl2Df7B4SxNrMz9MREFeuz0NXtoVzcUUR6aVZjDhwH+vNRqDP0G31XMqa+Nv9PON54OjHXOxnVyH6jKPN7kcYZED2Ht4bXEHYpj67GtHLuwjKAaMM73OOfnMZT3L0+IfwiBjkCcNif+dn/87H5/fNnFjk1s2MSGIIgINmze7zluz3mZHF1Z6RdcbKpo43e7H9U9X/GbpwZvJ1xqLz+b4SYifDuf2YWkcn5sTHjb+xz5yK+/LOdznKtSBaffLgI8S5iw5idqVro0e/O4O5748tW56FnH1O2X2tFzPn+fOn2oGVy0GzKLGdtatWjRQm3ZsqVQz+H2GNhEcu3lf/v9D3jmyHPMdHenxeMfcltESKFeT7u5pZ7NICPLoMObq+luX0dmjTlsCwigNiOZN3RYvo83DEXt8d5JNv0CviW+5kb8L1bm++ErsUnpOjFOz3DRbeKX/Oj/FFM89/C2uz+7J/W8bCidUorXlm4mbctbxAbG84rqy4DO9UjLTCMtM40MTwZZnqzLvjI9mRjKwMDAUAZKKe93lPfyVW7PbsbJyeU2cKos3NhwY8dpF2y+QqkAw+3CjoHH5izQKCBFPjXwT3cbYHhw4/uwkUs/JobbO3ZdbDkyXf4E07pMo13162u7F5EEpVSLK2+37BF53QlL6NmoCu/5mkqSj6bj9igaVQ+BM4f465GX2WNUY7J7AIv99cQfrXCy2zvnPtGWftPhhd9/ITMiiUS/qXyVHMyAqAF5Pj674z3McYDEiA04DAeHf3+k1BVxgOAAJ4O7t2VNXBMG2lcx3d2XMxdchPnGuGe5DZ6f+xPztx7lB//NOAKb8FHf12gcUTKjNE6dzyLh9e5E2w7QLvP/qRVajtVjOnIu00387uPUmt2VU6ocO3t8wUOxxTAMMiMNNTmSaa7eZHYYz+A2NSnj52Dl8oX0TRjKiKynmPbqy0X/unkofe+ia7Ak8egfl3u88wO9psRz8tRJTs/8L5zKxTDXaDLwL7FpstqNL3uVzDeyhjI2xZ/YCxlM2jiJZ1aP5sRF7xogSUfS2bL/FHtSz/HcnB24PAbnMt04/FMIqjWDM3ZBHR5Az4ZRZv4qeQor58/77t6ESToP2Ndw+sKljr3hnycwd2sKd9oSqCanCO88osSKOHjbyb/ztKa6nKSZ7KFeuHej9HumreOtWQuIsh1ipRFDaNlimlwVEILUaEVn58+kpmfSatIqek+NZ//G+XiUcKRS6+J53TxYqsIlHUln/4nzOHKZLRVAJgen9uE2TxLDXKPZp7wTfsKK6z9Uu2ll4seIzDHMOfa/fF0e/qXiiE/5gQhHJ3YkR2JkVKdxREV+OnyaDo1c/HBkIWUi52LzeGhyqD3rPM2Z8mDh1wApLgFOOxtVFFuM+jzhWMC2Q09Ss1IQU1btYVVyKoLBKMc8fjMqE1m/R4lmc9htrDBiOKsCGeRYyc8VugGwJ/UcLzhWk6XszPe0JaY454nU7070wYlsTNgEVGXf8XP09lvPJqMh7w/rWnyvm4tCFXIR6Q9MBBoCrZRShWv4zsecDbuZu/0oZ3Jsm/fs7B0kHDhFKGm85/c2TTy7edr1BHHGpT8SPfFHK0obxnUh6Wg6a3cd55Ef/8YXZ16h4/lMRldsye6gZZSJNFDKxm9GEGUbZDJ+kwubIdx77hyVjzdjclYfwss5sJs8izMv1csHAMKrroF84/cSx/49ga5xwzl46gIAf7GvItp2gKeyRvCuveSPB8ff25o5C+9gkH0FUzds4njHulTEe/awzGjJeWcF2tapVHwBGg/As/LvDLCv4XX3g7SUndS2HWW6qy+vB5X8gIrCNq0kAv2AtUWQJV/9zn3FHPUcnWzbsOHtBFm8dS9tzixkmf9YGsl+nnSNZIERC0D/mAi+fKzwS1hqWk5VQgLo1CCcv7apRbK6hfuyXiI0y8mKY3E8uq8hzpQ+ZJ28g6z0aMqeiWbQsSDiDh0i6lhj3sh6CIAHWtQw+bfIW0zNiswc2pJ9gbey0P9uHnIsIzZtIaBoa0vkfxyf872nMQuMkpuGntPA1rfQdsjfycLJS7aPaDtpKS86PyWIDN5x38fqMR2pUJzD/4KrkhjckSH25UTIcZ5zfs0JFcx3nta5thgUp0J9lCqlkqDkjngDI1tj3z2fmX5vckaV4aQKJkJO4C8uEox6jHM9yi516Q8keyq+phWHSmW9heI3VZXeWZN40fkZT9mW8nSW4mBGGH7ipoqc5qwK5C33YD7xdOPb4W2pE1bWEsNgO0WFs/3FbsQnR7N21gFec/6TkY55VOE0e1U1nnE9waKR7U3LF1W/AWPdg3nD+SGbbE9QQc7xD9f97FXVCfIr/rOEjXVHUTdhHav8xuAvLsa6HqNHszrF/rpXU2LnRCIyDBgGcMst1zfbqmqre2i6yEE32xba2hIJkfOsNJoT52nORhUFV4wXvegq+qU0NS1buQAn4++K4tXvkkmjLM+6hjNV7uFu2wbq2w7jVna2G3VY6GnDaYIJDnBwa7UQyy2fXLtKJdq7xjLAWE1rWxK7jAhmenowvFsT7ygxE832dOKsCuJu+0bWG9HM8nQBvB2ixa18tToMWj+eIY7lbDaimO3pyKae5nRg5/vbishK4Gob3k1QSs0v6AsppT4APgDvOPICJ8whyM9BFk4WGW1YZHinAneoH8bGXZdv/dShfhiVyvhx122F26dP0/IzrEMdPl1/gMOnLwLeo/O9DYcz1TeiauZDLfl0pndZ0y8eu91yRRygWvlAhnWsz3tr7H8sAf3vEbE0LcFFofKyxGjNEqM1b/Vvwqw5O2gVWbFEWgnubx5BtZBBJB/twxeLkwCKb6RMPvIt5EqpO0siSEGtGdMREVjx6zEeio1k2uo9rPUV8om9o5m48FfGdm9g+pGCdvOYMSiGXlO8u6bf1zyC/3ugCbWeXwxAm9qXmvZKapOB4lCzYtBl1ysGlfy+onlZMqo9DasGUze8LHXCSmifTJvQrl4o7eqF8oqvkJu1DLHl3lm1Qr3/SY/6Nsk1csxMbV8/jP2vF+2ylZqWn5wHDdlrmH80pAXLfz162RF4mRJcDa+oRYZeXhxDy5WuQp69xotZZwkNqwaTaWJTbmGHH94LTAHCgMUisl0p1b1IkhVQ9fKX1j+uUMqOErSbR9WQALLcxh8jFu6Mrsyd0ZUv+xkrH5FnL3ERXTWYKQOblUhn4rUweyjnklHtMWO5k2yFHbUyD5hXRFmuy/0xEXy9+RBbDpwmJLD0jwTQbkxrx3bK9b6nOtfl3bg9BFqwfTxbkJ+Db4e3JTK0DBVLcFU/KzFzvkrp+li9DiLCN8PNGcuqadny2ptxdLcGjO7WoATTFI+YmhXMjqDlwvKFXNO0m9eXj91OypmLZscwnS7kmqZZlp7w52Xp1Q81TdM0Xcg1TdMsTxdyTdM0i9OFXNM0zeJ0Idc0TbM4Xcg1TdMsThdyTdM0i9OFXNM0zeLEjIVeROQ4cOA6Hx4KnCjCOMXNSnmtlBWslddKWcFaea2UFQqXt6ZSKuzKG00p5IUhIluUUi3MzlFQVsprpaxgrbxWygrWymulrFA8eXXTiqZpmsXpQq5pmmZxVizkH5gd4BpZKa+VsoK18lopK1grr5WyQjHktVwbuaZpmnY5Kx6Ra5qmaTnoQq5pmmZxlizkIvKmiCSLyE8iMk9EzNk6uwBEpL+I/CIihoiU2iFSItJDRHaKyB4Red7sPHkRkY9FJFVEEs3Okh8RqSEiq0XkV9/7YJTZmXIjIgEisklEdviyvmR2pvyIiF1EtonIIrOz5EdE9ovIzyKyXUS2FOVzW7KQAyuARkqpxsAuYJzJefKSCPQD1podJDciYgemAT2BaOBBEYk2N1We/gX0MDtEAbmBZ5VS0cDtwIhS/G+bCXRWSjUBmgI9ROR2kzPlZxSQZHaIa9BJKdVUjyMHlFLLlVJu39UNQISZefKilEpSSu00O0c+WgF7lFL7lFJZwFdAX5Mz5UoptRY4ZXaOglBKHVFKbfVdPou36FQ3N9XVKa9zvqtO31epHQ0hIhHA3cBHZmcxmyUL+RUeBpaYHcLiqgOHclw/TCktNlYmIrWAZsBGc5PkztdUsR1IBVYopUptVuAdYCxgmB2kgBSwXEQSRGRYUT5xqd18WURWAlWuctcEpdR8389MwHvqOqsks12pIFm1m5uIlAW+BZ5WSqWbnSc3SikP0NTX7zRPRBoppUpdX4SI9AJSlVIJItLR7DwF1E4plSIi4cAKEUn2nV0WWqkt5EqpO/O6X0SGAr2ALsrkwfD5ZbWAFKBGjusRvtu0IiAiTrxFfJZSaq7ZeQpCKXVGRFbj7YsodYUciAX6iMhdQAAQLCKfK6UGmZwrV0qpFN/3VBGZh7dJs0gKuSWbVkSkB95Tqj5KqQtm57kBbAbqiUikiPgBA4AFJme6IYiIAP8EkpRS/zA7T15EJCx7BJiIBAJdgWRzU12dUmqcUipCKVUL7/s1rjQXcREpIyLlsi8D3SjCD0hLFnJgKlAO7+nJdhGZYXag3IjIvSJyGGgDLBaRZWZnupKv4/hJYBnezrjZSqlfzE2VOxH5ElgPNBCRwyLyiNmZ8hALDAY6+96r231HkaVRVWC1iPyE98N9hVKq1A/rs4jKQLyI7AA2AYuVUkuL6sn1FH1N0zSLs+oRuaZpmuajC7mmaZrF6UKuaZpmcbqQa5qmWZwu5JqmaRanC7mmaZrF6UKuaZpmcf8BNX8kydDfHHYAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "J2yr-0UCVIli"
      },
      "source": [
        "As expected, the model is able to make farely accurate predictions on the interval it was trained on but makes unreliable predictions outside this interval.\n",
        "\n",
        "### Regularization\n",
        "In this section we will explore the concept of regularization. As there is no theorem that can be used to determine the required size and structure of a neural network given a certain task, one has to find a suitable neural architecture by trial and error. This can result in choosing a architecture with a capacity that is higher than required for solving the given task and hence overfitting might occur. A common way to prevent large neural networks networks from overfitting is to employ some sort of regularization, e.g. weight norm penalty, dropout, early stopping and data augmentation. In this section we will focus on weight norm penalty as a regularization and derive a probabilistic interpretation for some of those.\n",
        "\n",
        "We start by restating the conditional probability of the output $\\mathbf{y}$ given the input $\\mathbf{x}$ and the networks parameters $\\boldsymbol{\\theta}$\n",
        "\n",
        "$p(\\mathbf{y}\\vert\\mathbf{x},\\boldsymbol{\\theta})=\\dfrac{1}{\\sqrt{(2\\pi)^{M}\\sigma^{2}}}\\mathrm{e}^{-\\dfrac{1}{2\\sigma^{2}}\\Vert\\boldsymbol{\\mathbf{y}-g_{\\boldsymbol{\\theta}}(\\mathbf{x})}\\Vert_{2}^{2}}$,\n",
        "\n",
        "which we used to derive the log likelihood. If we have some prior knowledge about the parameters of the neural network, which is given by a pdf $p(\\boldsymbol{\\theta})$ over the weights, we can use Bayes theorem to derive a posterior distribution\n",
        "\n",
        "$p(\\boldsymbol{\\theta}\\vert\\mathbf{x},\\mathbf{y})=\\dfrac{p(\\boldsymbol{\\theta},\\mathbf{y}\\vert\\mathbf{x})}{p(\\mathbf{y})}=\\dfrac{p(\\mathbf{y}\\vert\\mathbf{x},\\boldsymbol{\\theta})p(\\boldsymbol{\\theta})}{p(\\mathbf{y})}$\n",
        "\n",
        "where $p(\\boldsymbol{\\theta})$ is the  prior over the networks parameters. Similar to the derivation at the beginning of the exercise, we can use this posterior distribution to derive a cost function for training the neural network. In this case, however, we are not maximizing the log likelihood but the posterior distribution over the weights, hence this approach is called Maximum A Posteori (MAP) estimation of the parameters. Mathematically we can formulate this as\n",
        "\n",
        "$\\boldsymbol{\\theta}^{\\star}=\\arg\\max_{\\boldsymbol{\\theta}}\\mathbb{E}\\left[p(\\boldsymbol{\\theta}\\vert\\mathbf{x},\\mathbf{y})\\right]=\\arg\\max_{\\boldsymbol{\\theta}}\\mathbb{E}\\left[\\ln{p(\\boldsymbol{\\theta}\\vert\\mathbf{x},\\mathbf{y})}\\right]=\\arg\\max_{\\boldsymbol{\\theta}}\\ln{p(\\boldsymbol{\\theta})}+\\mathbb{E}\\left[\\ln{p(\\mathbf{y}\\vert\\mathbf{x},\\boldsymbol{\\theta})}\\right]$,\n",
        "\n",
        "where we used the fact that applying a strictly increasing function, e.g. $\\ln{}$, does not change the position of the maximum of a cost function, ignored $p(\\mathbf{y})$, since it is independent of the network parameters and dropped the expectation operator for $\\ln{p(\\boldsymbol{\\theta})}$, since it is not depending on the random variable $\\mathbf{x}$. Comparing the MAP estimate of the parameters with the ML estimate we derived above, shows that the only difference is the addition of $\\ln{p(\\boldsymbol{\\theta})}$. This term is the regularization, i.e. the weight norm penalty. Depending on the distribution over $\\boldsymbol{\\theta}$ it can have different forms. If we choose a standard normal distribution, i.e. $\\boldsymbol{\\theta\\sim\\mathcal{N}(\\mathbf{0},\\mathbf{I})}$,  we get\n",
        "\n",
        "$\\boldsymbol{\\theta}^{\\star}=\\arg\\min_{\\boldsymbol{\\theta}}\\lambda\\Vert\\boldsymbol{\\theta}\\Vert_{2}^{2}+\\dfrac{1}{N_{D}}\\sum_{i=1}^{N_{D}}\\Vert\\boldsymbol{\\mathbf{y}_{i}-g_{\\boldsymbol{\\theta}}(\\mathbf{x}_{i})}\\Vert_{2}^{2}$,\n",
        "\n",
        "where we have made the same simplifications as for the ML estimation and also introduced the parameter $\\lambda=\\sigma^{2}$, which is used to control the strength of the regularization. This form of regularization is commonly known as $l_{2}$-norm or weight decay regularization. Choosing a prior where the weights follow an i.i.d laplacian distribution, i.e. $p(\\boldsymbol{\\theta})=\\prod_{j}\\dfrac{1}{2}\\mathrm{e}^{\\vert\\theta_{j}\\vert}$, leads to\n",
        "\n",
        "$\\boldsymbol{\\theta}^{\\star}=\\arg\\min_{\\boldsymbol{\\theta}}\\lambda\\Vert\\boldsymbol{\\theta}\\Vert_{1}+\\dfrac{1}{N_{D}}\\sum_{i=1}^{N_{D}}\\Vert\\boldsymbol{\\mathbf{y}_{i}-g_{\\boldsymbol{\\theta}}(\\mathbf{x}_{i})}\\Vert_{2}^{2}$,\n",
        "\n",
        "where the strength of the regularization is again controlled by $\\lambda=\\sigma^{2}$. This type of regularization is known as  $l_{1}$-norm and it has the property to induce sparsity in the parameters of the network.\n",
        "\n",
        "With this theoretical background on regularization we can now implement it and observe it's effects on the regression problem covered in this exercise. For this we will define a model with a high capacity and train it for a extended time to provoke overfitting. For this, we will increase the number of hidden neurons in both hidden layers to $100$ and $50$."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "T2Fv2J-VK_vw",
        "colab": {}
      },
      "source": [
        "\"\"\" Implement a bigger model with again two hidden layers contatining 100 and 50 neurons. As an activation use the tangens hyperbolicus function where it is appropiate. \"\"\"\n",
        "\n",
        "class MyBigModel(object):\n",
        "  \n",
        "    def __init__(self):\n",
        "        input_dim = 1\n",
        "        hidden_dim1 = 100\n",
        "        hidden_dim2 = 50\n",
        "        output_dim = 1\n",
        "        \n",
        "        \n",
        "        # Create model variables\n",
        "        self.W0 = tf.Variable(tf.random.normal([input_dim,hidden_dim1]))\n",
        "        self.b0 = tf.Variable(tf.random.normal([hidden_dim1]))\n",
        "        self.W1 = tf.Variable(tf.random.normal([hidden_dim1,hidden_dim2]))\n",
        "        self.b1 = tf.Variable(tf.random.normal([hidden_dim2]))\n",
        "        self.W2 = tf.Variable(tf.random.normal([hidden_dim2,output_dim]))\n",
        "        self.b2 = tf.Variable(tf.random.normal([output_dim]))\n",
        "        self.trainable_variables = [self.W0, self.b0, self.W1, self.b1, self.W2, self.b2]\n",
        "\n",
        "    def __call__(self, inputs):\n",
        "        # Compute forward pass\n",
        "        output = tf.reshape(inputs, [-1, 1])\n",
        "        output = tf.nn.tanh(tf.linalg.matmul(output,self.W0) + self.b0)\n",
        "        output = tf.nn.tanh(tf.linalg.matmul(output,self.W1) + self.b1)\n",
        "        output = tf.linalg.matmul(output,self.W2) +self.b2\n",
        "        return output"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Y1XpaZTNLV-p"
      },
      "source": [
        "After creating one instance of this class we can again train it on our data set. We will also create a new optimizer for training this bigger model, since some optimizers adapt the learning rates for individual parameters during a training process and we do not want to train our bigger model with learning rates adopted from an earlier training run."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "b7Yq-a1FLi0X",
        "colab": {}
      },
      "source": [
        "big_mdl = MyBigModel()\n",
        "big_opt = tf.optimizers.SGD(learning_rate)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "dBrf0I68MnMy"
      },
      "source": [
        "Now we are ready to train this bigger model using the same training step and training loop. In order to provoke overfitting we also reduce the number of samples in the training data set a lot, increase the batch size and train for a more epochs."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "Nz_lXM8LMwXo",
        "outputId": "1ffcc30c-4f74-40b0-cd95-b94ecc94b397",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "\"\"\" Implement the training for the bigger model similar to the training of the small model before. \"\"\"\n",
        "\n",
        "N_train_samples_overfit = 30\n",
        "N_epochs = 1000\n",
        "batch_size = 30\n",
        "\n",
        "sel_idx = np.arange(0, N_train_samples)\n",
        "sel_idx = np.random.choice(sel_idx, N_train_samples_overfit)\n",
        "x_train_overfit = x_train[sel_idx]\n",
        "y_train_overfit = y_train[sel_idx]\n",
        "\n",
        "train_overfit_ds = tf.data.Dataset.from_tensor_slices((x_train_overfit, y_train_overfit)).shuffle(N_train_samples_overfit).batch(batch_size).repeat()\n",
        "\n",
        "epoch = 0\n",
        "train_iters = 0\n",
        "train_loss = 0.0\n",
        "for x_t, y_t in train_overfit_ds:\n",
        "    train_loss +=  train_step(big_mdl,big_opt,x_t,y_t) # Perform a training step with the model \"big_mdl\" and the optimizer \"big_opt\" on the inputs \"x_t\" and the corresponding targets \"y_t\"\n",
        "    train_iters += 1\n",
        "    if train_iters == N_train_samples_overfit/batch_size: # An epoch is completed\n",
        "        for x_v, y_v in validation_ds:\n",
        "            y_pred = big_mdl.__call__(x_v) # Compute a prediction with \"big_mdl\" on the input \"x_v\"\n",
        "            validation_loss += (y_v - y_pred)**2 # Compute the Mean Squared Error (MSE) for the prediction \"y_pred\" and the targets \"y_v\"\n",
        "        print(\"------ Epoch: \",epoch)\n",
        "        print(\"Train loss: \",(train_loss/train_iters).numpy())\n",
        "        print(\"Validation loss: \",(validation_loss/N_validation_samples).numpy())\n",
        "        train_iters = 0\n",
        "        train_loss = 0.0\n",
        "        validation_loss = 0\n",
        "        epoch += 1\n",
        "    if (epoch == N_epochs):\n",
        "        break"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "------ Epoch:  0\n",
            "Train loss:  16.893576\n",
            "Validation loss:  [[8.999007]]\n",
            "------ Epoch:  1\n",
            "Train loss:  7.533063\n",
            "Validation loss:  [[2.5732632]]\n",
            "------ Epoch:  2\n",
            "Train loss:  2.2868123\n",
            "Validation loss:  [[1.5790274]]\n",
            "------ Epoch:  3\n",
            "Train loss:  1.227217\n",
            "Validation loss:  [[0.3881921]]\n",
            "------ Epoch:  4\n",
            "Train loss:  0.39196047\n",
            "Validation loss:  [[0.17469446]]\n",
            "------ Epoch:  5\n",
            "Train loss:  0.20710716\n",
            "Validation loss:  [[0.17775033]]\n",
            "------ Epoch:  6\n",
            "Train loss:  0.18036976\n",
            "Validation loss:  [[0.14620122]]\n",
            "------ Epoch:  7\n",
            "Train loss:  0.17881489\n",
            "Validation loss:  [[0.25744665]]\n",
            "------ Epoch:  8\n",
            "Train loss:  0.20349692\n",
            "Validation loss:  [[0.32442722]]\n",
            "------ Epoch:  9\n",
            "Train loss:  0.3247131\n",
            "Validation loss:  [[0.445038]]\n",
            "------ Epoch:  10\n",
            "Train loss:  0.32042238\n",
            "Validation loss:  [[0.6313446]]\n",
            "------ Epoch:  11\n",
            "Train loss:  0.5490793\n",
            "Validation loss:  [[0.21079601]]\n",
            "------ Epoch:  12\n",
            "Train loss:  0.17933218\n",
            "Validation loss:  [[0.14262655]]\n",
            "------ Epoch:  13\n",
            "Train loss:  0.13701019\n",
            "Validation loss:  [[0.22469378]]\n",
            "------ Epoch:  14\n",
            "Train loss:  0.13385099\n",
            "Validation loss:  [[0.17898569]]\n",
            "------ Epoch:  15\n",
            "Train loss:  0.16624816\n",
            "Validation loss:  [[0.2713014]]\n",
            "------ Epoch:  16\n",
            "Train loss:  0.1531869\n",
            "Validation loss:  [[0.22948426]]\n",
            "------ Epoch:  17\n",
            "Train loss:  0.20840873\n",
            "Validation loss:  [[0.27501592]]\n",
            "------ Epoch:  18\n",
            "Train loss:  0.15309516\n",
            "Validation loss:  [[0.22046179]]\n",
            "------ Epoch:  19\n",
            "Train loss:  0.19688326\n",
            "Validation loss:  [[0.260604]]\n",
            "------ Epoch:  20\n",
            "Train loss:  0.13919656\n",
            "Validation loss:  [[0.19356224]]\n",
            "------ Epoch:  21\n",
            "Train loss:  0.16948177\n",
            "Validation loss:  [[0.2510246]]\n",
            "------ Epoch:  22\n",
            "Train loss:  0.12713441\n",
            "Validation loss:  [[0.17732997]]\n",
            "------ Epoch:  23\n",
            "Train loss:  0.15187155\n",
            "Validation loss:  [[0.24346733]]\n",
            "------ Epoch:  24\n",
            "Train loss:  0.118021056\n",
            "Validation loss:  [[0.16571105]]\n",
            "------ Epoch:  25\n",
            "Train loss:  0.13880275\n",
            "Validation loss:  [[0.23642483]]\n",
            "------ Epoch:  26\n",
            "Train loss:  0.11037497\n",
            "Validation loss:  [[0.15629531]]\n",
            "------ Epoch:  27\n",
            "Train loss:  0.12800126\n",
            "Validation loss:  [[0.22955248]]\n",
            "------ Epoch:  28\n",
            "Train loss:  0.1036213\n",
            "Validation loss:  [[0.1482566]]\n",
            "------ Epoch:  29\n",
            "Train loss:  0.11864173\n",
            "Validation loss:  [[0.22270684]]\n",
            "------ Epoch:  30\n",
            "Train loss:  0.09749832\n",
            "Validation loss:  [[0.14117737]]\n",
            "------ Epoch:  31\n",
            "Train loss:  0.11029217\n",
            "Validation loss:  [[0.2158079]]\n",
            "------ Epoch:  32\n",
            "Train loss:  0.0918474\n",
            "Validation loss:  [[0.13481036]]\n",
            "------ Epoch:  33\n",
            "Train loss:  0.10269126\n",
            "Validation loss:  [[0.20883103]]\n",
            "------ Epoch:  34\n",
            "Train loss:  0.08656733\n",
            "Validation loss:  [[0.12901376]]\n",
            "------ Epoch:  35\n",
            "Train loss:  0.09568501\n",
            "Validation loss:  [[0.20181634]]\n",
            "------ Epoch:  36\n",
            "Train loss:  0.08160389\n",
            "Validation loss:  [[0.12372153]]\n",
            "------ Epoch:  37\n",
            "Train loss:  0.08920228\n",
            "Validation loss:  [[0.19485813]]\n",
            "------ Epoch:  38\n",
            "Train loss:  0.076938726\n",
            "Validation loss:  [[0.11891358]]\n",
            "------ Epoch:  39\n",
            "Train loss:  0.08322568\n",
            "Validation loss:  [[0.18809149]]\n",
            "------ Epoch:  40\n",
            "Train loss:  0.07258083\n",
            "Validation loss:  [[0.11459242]]\n",
            "------ Epoch:  41\n",
            "Train loss:  0.077767484\n",
            "Validation loss:  [[0.18166202]]\n",
            "------ Epoch:  42\n",
            "Train loss:  0.06855248\n",
            "Validation loss:  [[0.11076184]]\n",
            "------ Epoch:  43\n",
            "Train loss:  0.072847776\n",
            "Validation loss:  [[0.17570454]]\n",
            "------ Epoch:  44\n",
            "Train loss:  0.06487986\n",
            "Validation loss:  [[0.10741501]]\n",
            "------ Epoch:  45\n",
            "Train loss:  0.068479136\n",
            "Validation loss:  [[0.17032114]]\n",
            "------ Epoch:  46\n",
            "Train loss:  0.061580785\n",
            "Validation loss:  [[0.10452846]]\n",
            "------ Epoch:  47\n",
            "Train loss:  0.064658836\n",
            "Validation loss:  [[0.1655762]]\n",
            "------ Epoch:  48\n",
            "Train loss:  0.05866344\n",
            "Validation loss:  [[0.10206516]]\n",
            "------ Epoch:  49\n",
            "Train loss:  0.06136789\n",
            "Validation loss:  [[0.16149338]]\n",
            "------ Epoch:  50\n",
            "Train loss:  0.056121584\n",
            "Validation loss:  [[0.09997821]]\n",
            "------ Epoch:  51\n",
            "Train loss:  0.05857377\n",
            "Validation loss:  [[0.15806504]]\n",
            "------ Epoch:  52\n",
            "Train loss:  0.053939342\n",
            "Validation loss:  [[0.09821875]]\n",
            "------ Epoch:  53\n",
            "Train loss:  0.056235768\n",
            "Validation loss:  [[0.15526025]]\n",
            "------ Epoch:  54\n",
            "Train loss:  0.052093778\n",
            "Validation loss:  [[0.09673903]]\n",
            "------ Epoch:  55\n",
            "Train loss:  0.05431002\n",
            "Validation loss:  [[0.1530303]]\n",
            "------ Epoch:  56\n",
            "Train loss:  0.05055589\n",
            "Validation loss:  [[0.09549596]]\n",
            "------ Epoch:  57\n",
            "Train loss:  0.052750356\n",
            "Validation loss:  [[0.15131567]]\n",
            "------ Epoch:  58\n",
            "Train loss:  0.049294893\n",
            "Validation loss:  [[0.0944528]]\n",
            "------ Epoch:  59\n",
            "Train loss:  0.05151259\n",
            "Validation loss:  [[0.15005276]]\n",
            "------ Epoch:  60\n",
            "Train loss:  0.04827905\n",
            "Validation loss:  [[0.09357837]]\n",
            "------ Epoch:  61\n",
            "Train loss:  0.05055256\n",
            "Validation loss:  [[0.14917073]]\n",
            "------ Epoch:  62\n",
            "Train loss:  0.04747492\n",
            "Validation loss:  [[0.09284635]]\n",
            "------ Epoch:  63\n",
            "Train loss:  0.049827423\n",
            "Validation loss:  [[0.14859864]]\n",
            "------ Epoch:  64\n",
            "Train loss:  0.046849165\n",
            "Validation loss:  [[0.09223406]]\n",
            "------ Epoch:  65\n",
            "Train loss:  0.049294725\n",
            "Validation loss:  [[0.14826316]]\n",
            "------ Epoch:  66\n",
            "Train loss:  0.046367623\n",
            "Validation loss:  [[0.09172021]]\n",
            "------ Epoch:  67\n",
            "Train loss:  0.048910346\n",
            "Validation loss:  [[0.14809236]]\n",
            "------ Epoch:  68\n",
            "Train loss:  0.045995515\n",
            "Validation loss:  [[0.0912853]]\n",
            "------ Epoch:  69\n",
            "Train loss:  0.04863221\n",
            "Validation loss:  [[0.14801879]]\n",
            "------ Epoch:  70\n",
            "Train loss:  0.04569912\n",
            "Validation loss:  [[0.09090958]]\n",
            "------ Epoch:  71\n",
            "Train loss:  0.048418686\n",
            "Validation loss:  [[0.1479801]]\n",
            "------ Epoch:  72\n",
            "Train loss:  0.04544599\n",
            "Validation loss:  [[0.09057497]]\n",
            "------ Epoch:  73\n",
            "Train loss:  0.048231672\n",
            "Validation loss:  [[0.14792813]]\n",
            "------ Epoch:  74\n",
            "Train loss:  0.045208883\n",
            "Validation loss:  [[0.09026606]]\n",
            "------ Epoch:  75\n",
            "Train loss:  0.048040558\n",
            "Validation loss:  [[0.14782567]]\n",
            "------ Epoch:  76\n",
            "Train loss:  0.04496507\n",
            "Validation loss:  [[0.08996991]]\n",
            "------ Epoch:  77\n",
            "Train loss:  0.04782165\n",
            "Validation loss:  [[0.14765245]]\n",
            "------ Epoch:  78\n",
            "Train loss:  0.044700384\n",
            "Validation loss:  [[0.08967835]]\n",
            "------ Epoch:  79\n",
            "Train loss:  0.0475617\n",
            "Validation loss:  [[0.14740272]]\n",
            "------ Epoch:  80\n",
            "Train loss:  0.044406746\n",
            "Validation loss:  [[0.08938749]]\n",
            "------ Epoch:  81\n",
            "Train loss:  0.04725642\n",
            "Validation loss:  [[0.14708145]]\n",
            "------ Epoch:  82\n",
            "Train loss:  0.044083867\n",
            "Validation loss:  [[0.0890969]]\n",
            "------ Epoch:  83\n",
            "Train loss:  0.04690949\n",
            "Validation loss:  [[0.14670354]]\n",
            "------ Epoch:  84\n",
            "Train loss:  0.04373617\n",
            "Validation loss:  [[0.08880888]]\n",
            "------ Epoch:  85\n",
            "Train loss:  0.046530247\n",
            "Validation loss:  [[0.14628695]]\n",
            "------ Epoch:  86\n",
            "Train loss:  0.043371342\n",
            "Validation loss:  [[0.08852656]]\n",
            "------ Epoch:  87\n",
            "Train loss:  0.0461307\n",
            "Validation loss:  [[0.14585213]]\n",
            "------ Epoch:  88\n",
            "Train loss:  0.042999275\n",
            "Validation loss:  [[0.08825377]]\n",
            "------ Epoch:  89\n",
            "Train loss:  0.045724016\n",
            "Validation loss:  [[0.14541559]]\n",
            "------ Epoch:  90\n",
            "Train loss:  0.04262851\n",
            "Validation loss:  [[0.08799283]]\n",
            "------ Epoch:  91\n",
            "Train loss:  0.04532067\n",
            "Validation loss:  [[0.14499049]]\n",
            "------ Epoch:  92\n",
            "Train loss:  0.0422671\n",
            "Validation loss:  [[0.0877458]]\n",
            "------ Epoch:  93\n",
            "Train loss:  0.044930436\n",
            "Validation loss:  [[0.14458726]]\n",
            "------ Epoch:  94\n",
            "Train loss:  0.041921075\n",
            "Validation loss:  [[0.08751304]]\n",
            "------ Epoch:  95\n",
            "Train loss:  0.04456013\n",
            "Validation loss:  [[0.1442104]]\n",
            "------ Epoch:  96\n",
            "Train loss:  0.041594267\n",
            "Validation loss:  [[0.08729374]]\n",
            "------ Epoch:  97\n",
            "Train loss:  0.04421275\n",
            "Validation loss:  [[0.14385961]]\n",
            "------ Epoch:  98\n",
            "Train loss:  0.04128785\n",
            "Validation loss:  [[0.08708672]]\n",
            "------ Epoch:  99\n",
            "Train loss:  0.043889012\n",
            "Validation loss:  [[0.14353314]]\n",
            "------ Epoch:  100\n",
            "Train loss:  0.041001853\n",
            "Validation loss:  [[0.08688963]]\n",
            "------ Epoch:  101\n",
            "Train loss:  0.043588176\n",
            "Validation loss:  [[0.1432259]]\n",
            "------ Epoch:  102\n",
            "Train loss:  0.040734638\n",
            "Validation loss:  [[0.0866999]]\n",
            "------ Epoch:  103\n",
            "Train loss:  0.043307044\n",
            "Validation loss:  [[0.14293128]]\n",
            "------ Epoch:  104\n",
            "Train loss:  0.04048346\n",
            "Validation loss:  [[0.08651513]]\n",
            "------ Epoch:  105\n",
            "Train loss:  0.043042205\n",
            "Validation loss:  [[0.14264226]]\n",
            "------ Epoch:  106\n",
            "Train loss:  0.040244937\n",
            "Validation loss:  [[0.08633274]]\n",
            "------ Epoch:  107\n",
            "Train loss:  0.042789362\n",
            "Validation loss:  [[0.14235294]]\n",
            "------ Epoch:  108\n",
            "Train loss:  0.04001608\n",
            "Validation loss:  [[0.08615051]]\n",
            "------ Epoch:  109\n",
            "Train loss:  0.042545285\n",
            "Validation loss:  [[0.14205778]]\n",
            "------ Epoch:  110\n",
            "Train loss:  0.039793838\n",
            "Validation loss:  [[0.08596687]]\n",
            "------ Epoch:  111\n",
            "Train loss:  0.042305816\n",
            "Validation loss:  [[0.14175133]]\n",
            "------ Epoch:  112\n",
            "Train loss:  0.0395754\n",
            "Validation loss:  [[0.08578]]\n",
            "------ Epoch:  113\n",
            "Train loss:  0.04206837\n",
            "Validation loss:  [[0.14143085]]\n",
            "------ Epoch:  114\n",
            "Train loss:  0.039358784\n",
            "Validation loss:  [[0.08558914]]\n",
            "------ Epoch:  115\n",
            "Train loss:  0.041830707\n",
            "Validation loss:  [[0.14109457]]\n",
            "------ Epoch:  116\n",
            "Train loss:  0.03914244\n",
            "Validation loss:  [[0.08539352]]\n",
            "------ Epoch:  117\n",
            "Train loss:  0.04159185\n",
            "Validation loss:  [[0.14074141]]\n",
            "------ Epoch:  118\n",
            "Train loss:  0.038925696\n",
            "Validation loss:  [[0.08519266]]\n",
            "------ Epoch:  119\n",
            "Train loss:  0.04135086\n",
            "Validation loss:  [[0.14037155]]\n",
            "------ Epoch:  120\n",
            "Train loss:  0.03870808\n",
            "Validation loss:  [[0.08498697]]\n",
            "------ Epoch:  121\n",
            "Train loss:  0.041108124\n",
            "Validation loss:  [[0.13998711]]\n",
            "------ Epoch:  122\n",
            "Train loss:  0.038489994\n",
            "Validation loss:  [[0.08477613]]\n",
            "------ Epoch:  123\n",
            "Train loss:  0.040864073\n",
            "Validation loss:  [[0.13958861]]\n",
            "------ Epoch:  124\n",
            "Train loss:  0.038271666\n",
            "Validation loss:  [[0.08456048]]\n",
            "------ Epoch:  125\n",
            "Train loss:  0.040619936\n",
            "Validation loss:  [[0.13917932]]\n",
            "------ Epoch:  126\n",
            "Train loss:  0.038054593\n",
            "Validation loss:  [[0.08434082]]\n",
            "------ Epoch:  127\n",
            "Train loss:  0.040377036\n",
            "Validation loss:  [[0.13876171]]\n",
            "------ Epoch:  128\n",
            "Train loss:  0.037839167\n",
            "Validation loss:  [[0.08411679]]\n",
            "------ Epoch:  129\n",
            "Train loss:  0.040136438\n",
            "Validation loss:  [[0.13833678]]\n",
            "------ Epoch:  130\n",
            "Train loss:  0.03762646\n",
            "Validation loss:  [[0.08388921]]\n",
            "------ Epoch:  131\n",
            "Train loss:  0.039899554\n",
            "Validation loss:  [[0.1379077]]\n",
            "------ Epoch:  132\n",
            "Train loss:  0.037417676\n",
            "Validation loss:  [[0.08365812]]\n",
            "------ Epoch:  133\n",
            "Train loss:  0.039667584\n",
            "Validation loss:  [[0.13747595]]\n",
            "------ Epoch:  134\n",
            "Train loss:  0.037213366\n",
            "Validation loss:  [[0.08342393]]\n",
            "------ Epoch:  135\n",
            "Train loss:  0.03944123\n",
            "Validation loss:  [[0.13704227]]\n",
            "------ Epoch:  136\n",
            "Train loss:  0.037014134\n",
            "Validation loss:  [[0.08318629]]\n",
            "------ Epoch:  137\n",
            "Train loss:  0.039221175\n",
            "Validation loss:  [[0.13660811]]\n",
            "------ Epoch:  138\n",
            "Train loss:  0.036820475\n",
            "Validation loss:  [[0.08294532]]\n",
            "------ Epoch:  139\n",
            "Train loss:  0.03900786\n",
            "Validation loss:  [[0.13617356]]\n",
            "------ Epoch:  140\n",
            "Train loss:  0.036632523\n",
            "Validation loss:  [[0.08270112]]\n",
            "------ Epoch:  141\n",
            "Train loss:  0.038801927\n",
            "Validation loss:  [[0.13573995]]\n",
            "------ Epoch:  142\n",
            "Train loss:  0.036450706\n",
            "Validation loss:  [[0.08245349]]\n",
            "------ Epoch:  143\n",
            "Train loss:  0.038603235\n",
            "Validation loss:  [[0.13530695]]\n",
            "------ Epoch:  144\n",
            "Train loss:  0.036274727\n",
            "Validation loss:  [[0.0822022]]\n",
            "------ Epoch:  145\n",
            "Train loss:  0.038411576\n",
            "Validation loss:  [[0.13487396]]\n",
            "------ Epoch:  146\n",
            "Train loss:  0.036104545\n",
            "Validation loss:  [[0.08194724]]\n",
            "------ Epoch:  147\n",
            "Train loss:  0.03822681\n",
            "Validation loss:  [[0.13444084]]\n",
            "------ Epoch:  148\n",
            "Train loss:  0.035939828\n",
            "Validation loss:  [[0.0816882]]\n",
            "------ Epoch:  149\n",
            "Train loss:  0.038048968\n",
            "Validation loss:  [[0.13400798]]\n",
            "------ Epoch:  150\n",
            "Train loss:  0.035780855\n",
            "Validation loss:  [[0.08142509]]\n",
            "------ Epoch:  151\n",
            "Train loss:  0.03787763\n",
            "Validation loss:  [[0.13357447]]\n",
            "------ Epoch:  152\n",
            "Train loss:  0.035626728\n",
            "Validation loss:  [[0.08115759]]\n",
            "------ Epoch:  153\n",
            "Train loss:  0.037712052\n",
            "Validation loss:  [[0.13314058]]\n",
            "------ Epoch:  154\n",
            "Train loss:  0.0354779\n",
            "Validation loss:  [[0.08088585]]\n",
            "------ Epoch:  155\n",
            "Train loss:  0.037553094\n",
            "Validation loss:  [[0.13270544]]\n",
            "------ Epoch:  156\n",
            "Train loss:  0.035333846\n",
            "Validation loss:  [[0.08060936]]\n",
            "------ Epoch:  157\n",
            "Train loss:  0.037399616\n",
            "Validation loss:  [[0.13226926]]\n",
            "------ Epoch:  158\n",
            "Train loss:  0.035194293\n",
            "Validation loss:  [[0.08032835]]\n",
            "------ Epoch:  159\n",
            "Train loss:  0.037251998\n",
            "Validation loss:  [[0.13183214]]\n",
            "------ Epoch:  160\n",
            "Train loss:  0.035059333\n",
            "Validation loss:  [[0.0800423]]\n",
            "------ Epoch:  161\n",
            "Train loss:  0.037109733\n",
            "Validation loss:  [[0.13139312]]\n",
            "------ Epoch:  162\n",
            "Train loss:  0.034928583\n",
            "Validation loss:  [[0.07975112]]\n",
            "------ Epoch:  163\n",
            "Train loss:  0.03697244\n",
            "Validation loss:  [[0.13095203]]\n",
            "------ Epoch:  164\n",
            "Train loss:  0.034801923\n",
            "Validation loss:  [[0.07945497]]\n",
            "------ Epoch:  165\n",
            "Train loss:  0.03684036\n",
            "Validation loss:  [[0.1305089]]\n",
            "------ Epoch:  166\n",
            "Train loss:  0.034679048\n",
            "Validation loss:  [[0.07915351]]\n",
            "------ Epoch:  167\n",
            "Train loss:  0.036713038\n",
            "Validation loss:  [[0.130064]]\n",
            "------ Epoch:  168\n",
            "Train loss:  0.03456042\n",
            "Validation loss:  [[0.0788466]]\n",
            "------ Epoch:  169\n",
            "Train loss:  0.036590714\n",
            "Validation loss:  [[0.12961727]]\n",
            "------ Epoch:  170\n",
            "Train loss:  0.034445617\n",
            "Validation loss:  [[0.07853436]]\n",
            "------ Epoch:  171\n",
            "Train loss:  0.036473416\n",
            "Validation loss:  [[0.1291681]]\n",
            "------ Epoch:  172\n",
            "Train loss:  0.034334764\n",
            "Validation loss:  [[0.07821667]]\n",
            "------ Epoch:  173\n",
            "Train loss:  0.036361292\n",
            "Validation loss:  [[0.12871718]]\n",
            "------ Epoch:  174\n",
            "Train loss:  0.03422799\n",
            "Validation loss:  [[0.0778933]]\n",
            "------ Epoch:  175\n",
            "Train loss:  0.036253944\n",
            "Validation loss:  [[0.1282641]]\n",
            "------ Epoch:  176\n",
            "Train loss:  0.034124948\n",
            "Validation loss:  [[0.07756417]]\n",
            "------ Epoch:  177\n",
            "Train loss:  0.036151513\n",
            "Validation loss:  [[0.12780903]]\n",
            "------ Epoch:  178\n",
            "Train loss:  0.034025934\n",
            "Validation loss:  [[0.07722947]]\n",
            "------ Epoch:  179\n",
            "Train loss:  0.036054324\n",
            "Validation loss:  [[0.12735172]]\n",
            "------ Epoch:  180\n",
            "Train loss:  0.03393108\n",
            "Validation loss:  [[0.07688887]]\n",
            "------ Epoch:  181\n",
            "Train loss:  0.035962448\n",
            "Validation loss:  [[0.12689261]]\n",
            "------ Epoch:  182\n",
            "Train loss:  0.033840377\n",
            "Validation loss:  [[0.07654244]]\n",
            "------ Epoch:  183\n",
            "Train loss:  0.03587557\n",
            "Validation loss:  [[0.12643066]]\n",
            "------ Epoch:  184\n",
            "Train loss:  0.033753283\n",
            "Validation loss:  [[0.07618997]]\n",
            "------ Epoch:  185\n",
            "Train loss:  0.035793774\n",
            "Validation loss:  [[0.12596667]]\n",
            "------ Epoch:  186\n",
            "Train loss:  0.03367047\n",
            "Validation loss:  [[0.07583149]]\n",
            "------ Epoch:  187\n",
            "Train loss:  0.035717573\n",
            "Validation loss:  [[0.12550013]]\n",
            "------ Epoch:  188\n",
            "Train loss:  0.033591986\n",
            "Validation loss:  [[0.0754669]]\n",
            "------ Epoch:  189\n",
            "Train loss:  0.035646595\n",
            "Validation loss:  [[0.125031]]\n",
            "------ Epoch:  190\n",
            "Train loss:  0.0335174\n",
            "Validation loss:  [[0.07509635]]\n",
            "------ Epoch:  191\n",
            "Train loss:  0.03558085\n",
            "Validation loss:  [[0.12455868]]\n",
            "------ Epoch:  192\n",
            "Train loss:  0.033446833\n",
            "Validation loss:  [[0.07471953]]\n",
            "------ Epoch:  193\n",
            "Train loss:  0.035520367\n",
            "Validation loss:  [[0.12408367]]\n",
            "------ Epoch:  194\n",
            "Train loss:  0.03338057\n",
            "Validation loss:  [[0.0743366]]\n",
            "------ Epoch:  195\n",
            "Train loss:  0.035465464\n",
            "Validation loss:  [[0.12360572]]\n",
            "------ Epoch:  196\n",
            "Train loss:  0.033318482\n",
            "Validation loss:  [[0.07394759]]\n",
            "------ Epoch:  197\n",
            "Train loss:  0.03541595\n",
            "Validation loss:  [[0.12312427]]\n",
            "------ Epoch:  198\n",
            "Train loss:  0.033260483\n",
            "Validation loss:  [[0.07355258]]\n",
            "------ Epoch:  199\n",
            "Train loss:  0.03537209\n",
            "Validation loss:  [[0.12263932]]\n",
            "------ Epoch:  200\n",
            "Train loss:  0.03320667\n",
            "Validation loss:  [[0.07315134]]\n",
            "------ Epoch:  201\n",
            "Train loss:  0.035333656\n",
            "Validation loss:  [[0.1221509]]\n",
            "------ Epoch:  202\n",
            "Train loss:  0.033157237\n",
            "Validation loss:  [[0.0727442]]\n",
            "------ Epoch:  203\n",
            "Train loss:  0.035300907\n",
            "Validation loss:  [[0.12165897]]\n",
            "------ Epoch:  204\n",
            "Train loss:  0.03311189\n",
            "Validation loss:  [[0.07233118]]\n",
            "------ Epoch:  205\n",
            "Train loss:  0.035273597\n",
            "Validation loss:  [[0.12116271]]\n",
            "------ Epoch:  206\n",
            "Train loss:  0.033070944\n",
            "Validation loss:  [[0.07191274]]\n",
            "------ Epoch:  207\n",
            "Train loss:  0.0352523\n",
            "Validation loss:  [[0.12066296]]\n",
            "------ Epoch:  208\n",
            "Train loss:  0.033034228\n",
            "Validation loss:  [[0.07148864]]\n",
            "------ Epoch:  209\n",
            "Train loss:  0.035236668\n",
            "Validation loss:  [[0.12015844]]\n",
            "------ Epoch:  210\n",
            "Train loss:  0.033001643\n",
            "Validation loss:  [[0.07105925]]\n",
            "------ Epoch:  211\n",
            "Train loss:  0.03522635\n",
            "Validation loss:  [[0.11964995]]\n",
            "------ Epoch:  212\n",
            "Train loss:  0.03297331\n",
            "Validation loss:  [[0.07062519]]\n",
            "------ Epoch:  213\n",
            "Train loss:  0.03522219\n",
            "Validation loss:  [[0.11913748]]\n",
            "------ Epoch:  214\n",
            "Train loss:  0.03294948\n",
            "Validation loss:  [[0.0701865]]\n",
            "------ Epoch:  215\n",
            "Train loss:  0.03522386\n",
            "Validation loss:  [[0.11862122]]\n",
            "------ Epoch:  216\n",
            "Train loss:  0.03293036\n",
            "Validation loss:  [[0.06974385]]\n",
            "------ Epoch:  217\n",
            "Train loss:  0.03523167\n",
            "Validation loss:  [[0.11810057]]\n",
            "------ Epoch:  218\n",
            "Train loss:  0.03291528\n",
            "Validation loss:  [[0.06929747]]\n",
            "------ Epoch:  219\n",
            "Train loss:  0.035245333\n",
            "Validation loss:  [[0.11757599]]\n",
            "------ Epoch:  220\n",
            "Train loss:  0.03290488\n",
            "Validation loss:  [[0.06884809]]\n",
            "------ Epoch:  221\n",
            "Train loss:  0.03526548\n",
            "Validation loss:  [[0.1170482]]\n",
            "------ Epoch:  222\n",
            "Train loss:  0.032899253\n",
            "Validation loss:  [[0.06839647]]\n",
            "------ Epoch:  223\n",
            "Train loss:  0.03529163\n",
            "Validation loss:  [[0.11651643]]\n",
            "------ Epoch:  224\n",
            "Train loss:  0.032897998\n",
            "Validation loss:  [[0.06794284]]\n",
            "------ Epoch:  225\n",
            "Train loss:  0.03532405\n",
            "Validation loss:  [[0.11598162]]\n",
            "------ Epoch:  226\n",
            "Train loss:  0.0329012\n",
            "Validation loss:  [[0.06748822]]\n",
            "------ Epoch:  227\n",
            "Train loss:  0.035362132\n",
            "Validation loss:  [[0.11544304]]\n",
            "------ Epoch:  228\n",
            "Train loss:  0.032908846\n",
            "Validation loss:  [[0.06703348]]\n",
            "------ Epoch:  229\n",
            "Train loss:  0.03540655\n",
            "Validation loss:  [[0.11490168]]\n",
            "------ Epoch:  230\n",
            "Train loss:  0.032920767\n",
            "Validation loss:  [[0.06657913]]\n",
            "------ Epoch:  231\n",
            "Train loss:  0.035456292\n",
            "Validation loss:  [[0.11435702]]\n",
            "------ Epoch:  232\n",
            "Train loss:  0.032936614\n",
            "Validation loss:  [[0.06612623]]\n",
            "------ Epoch:  233\n",
            "Train loss:  0.035511773\n",
            "Validation loss:  [[0.11381043]]\n",
            "------ Epoch:  234\n",
            "Train loss:  0.03295677\n",
            "Validation loss:  [[0.06567591]]\n",
            "------ Epoch:  235\n",
            "Train loss:  0.035572868\n",
            "Validation loss:  [[0.11326186]]\n",
            "------ Epoch:  236\n",
            "Train loss:  0.032980736\n",
            "Validation loss:  [[0.06522869]]\n",
            "------ Epoch:  237\n",
            "Train loss:  0.035638623\n",
            "Validation loss:  [[0.11271122]]\n",
            "------ Epoch:  238\n",
            "Train loss:  0.033008102\n",
            "Validation loss:  [[0.06478599]]\n",
            "------ Epoch:  239\n",
            "Train loss:  0.035709254\n",
            "Validation loss:  [[0.11215901]]\n",
            "------ Epoch:  240\n",
            "Train loss:  0.033038523\n",
            "Validation loss:  [[0.06434829]]\n",
            "------ Epoch:  241\n",
            "Train loss:  0.03578381\n",
            "Validation loss:  [[0.11160602]]\n",
            "------ Epoch:  242\n",
            "Train loss:  0.033071894\n",
            "Validation loss:  [[0.06391672]]\n",
            "------ Epoch:  243\n",
            "Train loss:  0.035861857\n",
            "Validation loss:  [[0.11105139]]\n",
            "------ Epoch:  244\n",
            "Train loss:  0.03310713\n",
            "Validation loss:  [[0.06349152]]\n",
            "------ Epoch:  245\n",
            "Train loss:  0.03594179\n",
            "Validation loss:  [[0.11049498]]\n",
            "------ Epoch:  246\n",
            "Train loss:  0.033143297\n",
            "Validation loss:  [[0.06307361]]\n",
            "------ Epoch:  247\n",
            "Train loss:  0.03602314\n",
            "Validation loss:  [[0.10993741]]\n",
            "------ Epoch:  248\n",
            "Train loss:  0.033180293\n",
            "Validation loss:  [[0.06266371]]\n",
            "------ Epoch:  249\n",
            "Train loss:  0.036105078\n",
            "Validation loss:  [[0.10937922]]\n",
            "------ Epoch:  250\n",
            "Train loss:  0.03321729\n",
            "Validation loss:  [[0.06226231]]\n",
            "------ Epoch:  251\n",
            "Train loss:  0.03618669\n",
            "Validation loss:  [[0.1088199]]\n",
            "------ Epoch:  252\n",
            "Train loss:  0.033253178\n",
            "Validation loss:  [[0.06186952]]\n",
            "------ Epoch:  253\n",
            "Train loss:  0.036266174\n",
            "Validation loss:  [[0.10826018]]\n",
            "------ Epoch:  254\n",
            "Train loss:  0.033287413\n",
            "Validation loss:  [[0.06148582]]\n",
            "------ Epoch:  255\n",
            "Train loss:  0.036343046\n",
            "Validation loss:  [[0.10769919]]\n",
            "------ Epoch:  256\n",
            "Train loss:  0.033318687\n",
            "Validation loss:  [[0.06111099]]\n",
            "------ Epoch:  257\n",
            "Train loss:  0.036415588\n",
            "Validation loss:  [[0.10713711]]\n",
            "------ Epoch:  258\n",
            "Train loss:  0.03334627\n",
            "Validation loss:  [[0.0607449]]\n",
            "------ Epoch:  259\n",
            "Train loss:  0.03648223\n",
            "Validation loss:  [[0.10657379]]\n",
            "------ Epoch:  260\n",
            "Train loss:  0.033369143\n",
            "Validation loss:  [[0.0603874]]\n",
            "------ Epoch:  261\n",
            "Train loss:  0.03654225\n",
            "Validation loss:  [[0.10600951]]\n",
            "------ Epoch:  262\n",
            "Train loss:  0.033386495\n",
            "Validation loss:  [[0.06003828]]\n",
            "------ Epoch:  263\n",
            "Train loss:  0.036594424\n",
            "Validation loss:  [[0.10544369]]\n",
            "------ Epoch:  264\n",
            "Train loss:  0.0333971\n",
            "Validation loss:  [[0.05969661]]\n",
            "------ Epoch:  265\n",
            "Train loss:  0.036637027\n",
            "Validation loss:  [[0.10487612]]\n",
            "------ Epoch:  266\n",
            "Train loss:  0.033400476\n",
            "Validation loss:  [[0.05936212]]\n",
            "------ Epoch:  267\n",
            "Train loss:  0.03666936\n",
            "Validation loss:  [[0.10430802]]\n",
            "------ Epoch:  268\n",
            "Train loss:  0.033395708\n",
            "Validation loss:  [[0.05903411]]\n",
            "------ Epoch:  269\n",
            "Train loss:  0.03669042\n",
            "Validation loss:  [[0.10373791]]\n",
            "------ Epoch:  270\n",
            "Train loss:  0.033382002\n",
            "Validation loss:  [[0.05871174]]\n",
            "------ Epoch:  271\n",
            "Train loss:  0.036699213\n",
            "Validation loss:  [[0.10316662]]\n",
            "------ Epoch:  272\n",
            "Train loss:  0.033359166\n",
            "Validation loss:  [[0.0583947]]\n",
            "------ Epoch:  273\n",
            "Train loss:  0.036695514\n",
            "Validation loss:  [[0.10259487]]\n",
            "------ Epoch:  274\n",
            "Train loss:  0.03332704\n",
            "Validation loss:  [[0.05808255]]\n",
            "------ Epoch:  275\n",
            "Train loss:  0.036679167\n",
            "Validation loss:  [[0.1020223]]\n",
            "------ Epoch:  276\n",
            "Train loss:  0.033284936\n",
            "Validation loss:  [[0.0577741]]\n",
            "------ Epoch:  277\n",
            "Train loss:  0.036649168\n",
            "Validation loss:  [[0.10144904]]\n",
            "------ Epoch:  278\n",
            "Train loss:  0.033232857\n",
            "Validation loss:  [[0.05746913]]\n",
            "------ Epoch:  279\n",
            "Train loss:  0.036605995\n",
            "Validation loss:  [[0.10087633]]\n",
            "------ Epoch:  280\n",
            "Train loss:  0.03317105\n",
            "Validation loss:  [[0.05716719]]\n",
            "------ Epoch:  281\n",
            "Train loss:  0.036549345\n",
            "Validation loss:  [[0.10030294]]\n",
            "------ Epoch:  282\n",
            "Train loss:  0.03309901\n",
            "Validation loss:  [[0.05686753]]\n",
            "------ Epoch:  283\n",
            "Train loss:  0.036479488\n",
            "Validation loss:  [[0.09973077]]\n",
            "------ Epoch:  284\n",
            "Train loss:  0.033017736\n",
            "Validation loss:  [[0.05657048]]\n",
            "------ Epoch:  285\n",
            "Train loss:  0.036397103\n",
            "Validation loss:  [[0.09915978]]\n",
            "------ Epoch:  286\n",
            "Train loss:  0.032927167\n",
            "Validation loss:  [[0.05627552]]\n",
            "------ Epoch:  287\n",
            "Train loss:  0.03630281\n",
            "Validation loss:  [[0.09859104]]\n",
            "------ Epoch:  288\n",
            "Train loss:  0.03282794\n",
            "Validation loss:  [[0.0559823]]\n",
            "------ Epoch:  289\n",
            "Train loss:  0.036196638\n",
            "Validation loss:  [[0.09802392]]\n",
            "------ Epoch:  290\n",
            "Train loss:  0.032720204\n",
            "Validation loss:  [[0.05569071]]\n",
            "------ Epoch:  291\n",
            "Train loss:  0.036079373\n",
            "Validation loss:  [[0.09745986]]\n",
            "------ Epoch:  292\n",
            "Train loss:  0.03260459\n",
            "Validation loss:  [[0.0554008]]\n",
            "------ Epoch:  293\n",
            "Train loss:  0.035951998\n",
            "Validation loss:  [[0.09689889]]\n",
            "------ Epoch:  294\n",
            "Train loss:  0.032482002\n",
            "Validation loss:  [[0.05511299]]\n",
            "------ Epoch:  295\n",
            "Train loss:  0.035815466\n",
            "Validation loss:  [[0.09634157]]\n",
            "------ Epoch:  296\n",
            "Train loss:  0.032352723\n",
            "Validation loss:  [[0.05482684]]\n",
            "------ Epoch:  297\n",
            "Train loss:  0.035670117\n",
            "Validation loss:  [[0.09578861]]\n",
            "------ Epoch:  298\n",
            "Train loss:  0.032217503\n",
            "Validation loss:  [[0.05454225]]\n",
            "------ Epoch:  299\n",
            "Train loss:  0.035516605\n",
            "Validation loss:  [[0.09523927]]\n",
            "------ Epoch:  300\n",
            "Train loss:  0.032076143\n",
            "Validation loss:  [[0.05425938]]\n",
            "------ Epoch:  301\n",
            "Train loss:  0.035355687\n",
            "Validation loss:  [[0.09469438]]\n",
            "------ Epoch:  302\n",
            "Train loss:  0.031929836\n",
            "Validation loss:  [[0.05397812]]\n",
            "------ Epoch:  303\n",
            "Train loss:  0.035187893\n",
            "Validation loss:  [[0.09415392]]\n",
            "------ Epoch:  304\n",
            "Train loss:  0.031778835\n",
            "Validation loss:  [[0.05369898]]\n",
            "------ Epoch:  305\n",
            "Train loss:  0.03501444\n",
            "Validation loss:  [[0.09361843]]\n",
            "------ Epoch:  306\n",
            "Train loss:  0.031623926\n",
            "Validation loss:  [[0.05342177]]\n",
            "------ Epoch:  307\n",
            "Train loss:  0.03483564\n",
            "Validation loss:  [[0.0930885]]\n",
            "------ Epoch:  308\n",
            "Train loss:  0.031465445\n",
            "Validation loss:  [[0.05314672]]\n",
            "------ Epoch:  309\n",
            "Train loss:  0.034652114\n",
            "Validation loss:  [[0.09256302]]\n",
            "------ Epoch:  310\n",
            "Train loss:  0.031303696\n",
            "Validation loss:  [[0.05287365]]\n",
            "------ Epoch:  311\n",
            "Train loss:  0.034464445\n",
            "Validation loss:  [[0.0920433]]\n",
            "------ Epoch:  312\n",
            "Train loss:  0.031139312\n",
            "Validation loss:  [[0.05260287]]\n",
            "------ Epoch:  313\n",
            "Train loss:  0.034273263\n",
            "Validation loss:  [[0.09152934]]\n",
            "------ Epoch:  314\n",
            "Train loss:  0.030972684\n",
            "Validation loss:  [[0.05233455]]\n",
            "------ Epoch:  315\n",
            "Train loss:  0.03407904\n",
            "Validation loss:  [[0.0910201]]\n",
            "------ Epoch:  316\n",
            "Train loss:  0.030803973\n",
            "Validation loss:  [[0.05206853]]\n",
            "------ Epoch:  317\n",
            "Train loss:  0.033882312\n",
            "Validation loss:  [[0.09051654]]\n",
            "------ Epoch:  318\n",
            "Train loss:  0.03063351\n",
            "Validation loss:  [[0.05180493]]\n",
            "------ Epoch:  319\n",
            "Train loss:  0.033683077\n",
            "Validation loss:  [[0.09001816]]\n",
            "------ Epoch:  320\n",
            "Train loss:  0.030461617\n",
            "Validation loss:  [[0.05154328]]\n",
            "------ Epoch:  321\n",
            "Train loss:  0.033481684\n",
            "Validation loss:  [[0.08952431]]\n",
            "------ Epoch:  322\n",
            "Train loss:  0.030288251\n",
            "Validation loss:  [[0.05128422]]\n",
            "------ Epoch:  323\n",
            "Train loss:  0.03327878\n",
            "Validation loss:  [[0.08903552]]\n",
            "------ Epoch:  324\n",
            "Train loss:  0.030113947\n",
            "Validation loss:  [[0.05102731]]\n",
            "------ Epoch:  325\n",
            "Train loss:  0.033074334\n",
            "Validation loss:  [[0.08855221]]\n",
            "------ Epoch:  326\n",
            "Train loss:  0.029938992\n",
            "Validation loss:  [[0.05077311]]\n",
            "------ Epoch:  327\n",
            "Train loss:  0.032869246\n",
            "Validation loss:  [[0.08807383]]\n",
            "------ Epoch:  328\n",
            "Train loss:  0.029763639\n",
            "Validation loss:  [[0.05052145]]\n",
            "------ Epoch:  329\n",
            "Train loss:  0.03266337\n",
            "Validation loss:  [[0.0876006]]\n",
            "------ Epoch:  330\n",
            "Train loss:  0.0295878\n",
            "Validation loss:  [[0.05027219]]\n",
            "------ Epoch:  331\n",
            "Train loss:  0.032456826\n",
            "Validation loss:  [[0.08713243]]\n",
            "------ Epoch:  332\n",
            "Train loss:  0.02941193\n",
            "Validation loss:  [[0.0500256]]\n",
            "------ Epoch:  333\n",
            "Train loss:  0.0322502\n",
            "Validation loss:  [[0.0866688]]\n",
            "------ Epoch:  334\n",
            "Train loss:  0.029236041\n",
            "Validation loss:  [[0.04978155]]\n",
            "------ Epoch:  335\n",
            "Train loss:  0.032043386\n",
            "Validation loss:  [[0.0862098]]\n",
            "------ Epoch:  336\n",
            "Train loss:  0.029060122\n",
            "Validation loss:  [[0.04953997]]\n",
            "------ Epoch:  337\n",
            "Train loss:  0.031836938\n",
            "Validation loss:  [[0.08575639]]\n",
            "------ Epoch:  338\n",
            "Train loss:  0.028884836\n",
            "Validation loss:  [[0.04930116]]\n",
            "------ Epoch:  339\n",
            "Train loss:  0.03163085\n",
            "Validation loss:  [[0.08530712]]\n",
            "------ Epoch:  340\n",
            "Train loss:  0.028709808\n",
            "Validation loss:  [[0.04906498]]\n",
            "------ Epoch:  341\n",
            "Train loss:  0.0314252\n",
            "Validation loss:  [[0.08486267]]\n",
            "------ Epoch:  342\n",
            "Train loss:  0.028535457\n",
            "Validation loss:  [[0.04883128]]\n",
            "------ Epoch:  343\n",
            "Train loss:  0.031220168\n",
            "Validation loss:  [[0.08442294]]\n",
            "------ Epoch:  344\n",
            "Train loss:  0.028361695\n",
            "Validation loss:  [[0.04860036]]\n",
            "------ Epoch:  345\n",
            "Train loss:  0.031015905\n",
            "Validation loss:  [[0.08398736]]\n",
            "------ Epoch:  346\n",
            "Train loss:  0.02818857\n",
            "Validation loss:  [[0.04837198]]\n",
            "------ Epoch:  347\n",
            "Train loss:  0.03081268\n",
            "Validation loss:  [[0.08355671]]\n",
            "------ Epoch:  348\n",
            "Train loss:  0.028016554\n",
            "Validation loss:  [[0.04814658]]\n",
            "------ Epoch:  349\n",
            "Train loss:  0.030610777\n",
            "Validation loss:  [[0.08313096]]\n",
            "------ Epoch:  350\n",
            "Train loss:  0.027845684\n",
            "Validation loss:  [[0.04792397]]\n",
            "------ Epoch:  351\n",
            "Train loss:  0.030410077\n",
            "Validation loss:  [[0.08270958]]\n",
            "------ Epoch:  352\n",
            "Train loss:  0.027675632\n",
            "Validation loss:  [[0.04770372]]\n",
            "------ Epoch:  353\n",
            "Train loss:  0.030210314\n",
            "Validation loss:  [[0.08229229]]\n",
            "------ Epoch:  354\n",
            "Train loss:  0.027506594\n",
            "Validation loss:  [[0.04748621]]\n",
            "------ Epoch:  355\n",
            "Train loss:  0.030012038\n",
            "Validation loss:  [[0.08187959]]\n",
            "------ Epoch:  356\n",
            "Train loss:  0.027338775\n",
            "Validation loss:  [[0.04727116]]\n",
            "------ Epoch:  357\n",
            "Train loss:  0.02981467\n",
            "Validation loss:  [[0.08147015]]\n",
            "------ Epoch:  358\n",
            "Train loss:  0.02717163\n",
            "Validation loss:  [[0.04705842]]\n",
            "------ Epoch:  359\n",
            "Train loss:  0.029618554\n",
            "Validation loss:  [[0.08106475]]\n",
            "------ Epoch:  360\n",
            "Train loss:  0.027005412\n",
            "Validation loss:  [[0.04684781]]\n",
            "------ Epoch:  361\n",
            "Train loss:  0.029423308\n",
            "Validation loss:  [[0.08066288]]\n",
            "------ Epoch:  362\n",
            "Train loss:  0.02684017\n",
            "Validation loss:  [[0.04663992]]\n",
            "------ Epoch:  363\n",
            "Train loss:  0.02922963\n",
            "Validation loss:  [[0.08026543]]\n",
            "------ Epoch:  364\n",
            "Train loss:  0.02667617\n",
            "Validation loss:  [[0.04643432]]\n",
            "------ Epoch:  365\n",
            "Train loss:  0.029037308\n",
            "Validation loss:  [[0.07987137]]\n",
            "------ Epoch:  366\n",
            "Train loss:  0.026513258\n",
            "Validation loss:  [[0.04623123]]\n",
            "------ Epoch:  367\n",
            "Train loss:  0.02884639\n",
            "Validation loss:  [[0.07948136]]\n",
            "------ Epoch:  368\n",
            "Train loss:  0.026351612\n",
            "Validation loss:  [[0.04603044]]\n",
            "------ Epoch:  369\n",
            "Train loss:  0.028657027\n",
            "Validation loss:  [[0.07909521]]\n",
            "------ Epoch:  370\n",
            "Train loss:  0.026191374\n",
            "Validation loss:  [[0.04583203]]\n",
            "------ Epoch:  371\n",
            "Train loss:  0.028469164\n",
            "Validation loss:  [[0.07871276]]\n",
            "------ Epoch:  372\n",
            "Train loss:  0.026032273\n",
            "Validation loss:  [[0.04563638]]\n",
            "------ Epoch:  373\n",
            "Train loss:  0.028283114\n",
            "Validation loss:  [[0.07833396]]\n",
            "------ Epoch:  374\n",
            "Train loss:  0.025874374\n",
            "Validation loss:  [[0.04544277]]\n",
            "------ Epoch:  375\n",
            "Train loss:  0.028098345\n",
            "Validation loss:  [[0.07795904]]\n",
            "------ Epoch:  376\n",
            "Train loss:  0.02571797\n",
            "Validation loss:  [[0.04525167]]\n",
            "------ Epoch:  377\n",
            "Train loss:  0.027915288\n",
            "Validation loss:  [[0.07758772]]\n",
            "------ Epoch:  378\n",
            "Train loss:  0.025562746\n",
            "Validation loss:  [[0.04506304]]\n",
            "------ Epoch:  379\n",
            "Train loss:  0.027733915\n",
            "Validation loss:  [[0.07721937]]\n",
            "------ Epoch:  380\n",
            "Train loss:  0.025408665\n",
            "Validation loss:  [[0.04487614]]\n",
            "------ Epoch:  381\n",
            "Train loss:  0.027553713\n",
            "Validation loss:  [[0.07685445]]\n",
            "------ Epoch:  382\n",
            "Train loss:  0.025255775\n",
            "Validation loss:  [[0.04469185]]\n",
            "------ Epoch:  383\n",
            "Train loss:  0.027375398\n",
            "Validation loss:  [[0.07649364]]\n",
            "------ Epoch:  384\n",
            "Train loss:  0.025104497\n",
            "Validation loss:  [[0.04450969]]\n",
            "------ Epoch:  385\n",
            "Train loss:  0.027198521\n",
            "Validation loss:  [[0.07613558]]\n",
            "------ Epoch:  386\n",
            "Train loss:  0.024954252\n",
            "Validation loss:  [[0.04432956]]\n",
            "------ Epoch:  387\n",
            "Train loss:  0.027023166\n",
            "Validation loss:  [[0.07578059]]\n",
            "------ Epoch:  388\n",
            "Train loss:  0.024805093\n",
            "Validation loss:  [[0.04415151]]\n",
            "------ Epoch:  389\n",
            "Train loss:  0.02684924\n",
            "Validation loss:  [[0.07542878]]\n",
            "------ Epoch:  390\n",
            "Train loss:  0.02465724\n",
            "Validation loss:  [[0.04397545]]\n",
            "------ Epoch:  391\n",
            "Train loss:  0.026676835\n",
            "Validation loss:  [[0.07508066]]\n",
            "------ Epoch:  392\n",
            "Train loss:  0.024510825\n",
            "Validation loss:  [[0.04380203]]\n",
            "------ Epoch:  393\n",
            "Train loss:  0.026506435\n",
            "Validation loss:  [[0.07473601]]\n",
            "------ Epoch:  394\n",
            "Train loss:  0.02436575\n",
            "Validation loss:  [[0.04363033]]\n",
            "------ Epoch:  395\n",
            "Train loss:  0.02633735\n",
            "Validation loss:  [[0.07439425]]\n",
            "------ Epoch:  396\n",
            "Train loss:  0.024221817\n",
            "Validation loss:  [[0.04346072]]\n",
            "------ Epoch:  397\n",
            "Train loss:  0.026169725\n",
            "Validation loss:  [[0.07405476]]\n",
            "------ Epoch:  398\n",
            "Train loss:  0.024078941\n",
            "Validation loss:  [[0.04329299]]\n",
            "------ Epoch:  399\n",
            "Train loss:  0.026003543\n",
            "Validation loss:  [[0.07371889]]\n",
            "------ Epoch:  400\n",
            "Train loss:  0.023937412\n",
            "Validation loss:  [[0.0431272]]\n",
            "------ Epoch:  401\n",
            "Train loss:  0.02583882\n",
            "Validation loss:  [[0.07338554]]\n",
            "------ Epoch:  402\n",
            "Train loss:  0.023796927\n",
            "Validation loss:  [[0.04296325]]\n",
            "------ Epoch:  403\n",
            "Train loss:  0.025675487\n",
            "Validation loss:  [[0.07305504]]\n",
            "------ Epoch:  404\n",
            "Train loss:  0.023657655\n",
            "Validation loss:  [[0.04280142]]\n",
            "------ Epoch:  405\n",
            "Train loss:  0.025513869\n",
            "Validation loss:  [[0.07272772]]\n",
            "------ Epoch:  406\n",
            "Train loss:  0.023519712\n",
            "Validation loss:  [[0.04264142]]\n",
            "------ Epoch:  407\n",
            "Train loss:  0.025353746\n",
            "Validation loss:  [[0.07240355]]\n",
            "------ Epoch:  408\n",
            "Train loss:  0.023383113\n",
            "Validation loss:  [[0.04248352]]\n",
            "------ Epoch:  409\n",
            "Train loss:  0.025195312\n",
            "Validation loss:  [[0.0720821]]\n",
            "------ Epoch:  410\n",
            "Train loss:  0.023247689\n",
            "Validation loss:  [[0.04232738]]\n",
            "------ Epoch:  411\n",
            "Train loss:  0.025038278\n",
            "Validation loss:  [[0.07176373]]\n",
            "------ Epoch:  412\n",
            "Train loss:  0.023113454\n",
            "Validation loss:  [[0.04217287]]\n",
            "------ Epoch:  413\n",
            "Train loss:  0.024882449\n",
            "Validation loss:  [[0.07144751]]\n",
            "------ Epoch:  414\n",
            "Train loss:  0.022980237\n",
            "Validation loss:  [[0.0420204]]\n",
            "------ Epoch:  415\n",
            "Train loss:  0.024728257\n",
            "Validation loss:  [[0.0711342]]\n",
            "------ Epoch:  416\n",
            "Train loss:  0.022848245\n",
            "Validation loss:  [[0.04186961]]\n",
            "------ Epoch:  417\n",
            "Train loss:  0.024575435\n",
            "Validation loss:  [[0.07082374]]\n",
            "------ Epoch:  418\n",
            "Train loss:  0.022717465\n",
            "Validation loss:  [[0.04172054]]\n",
            "------ Epoch:  419\n",
            "Train loss:  0.024424054\n",
            "Validation loss:  [[0.07051599]]\n",
            "------ Epoch:  420\n",
            "Train loss:  0.022587877\n",
            "Validation loss:  [[0.04157332]]\n",
            "------ Epoch:  421\n",
            "Train loss:  0.02427425\n",
            "Validation loss:  [[0.07021082]]\n",
            "------ Epoch:  422\n",
            "Train loss:  0.02245938\n",
            "Validation loss:  [[0.04142775]]\n",
            "------ Epoch:  423\n",
            "Train loss:  0.024125744\n",
            "Validation loss:  [[0.06990822]]\n",
            "------ Epoch:  424\n",
            "Train loss:  0.022332003\n",
            "Validation loss:  [[0.04128368]]\n",
            "------ Epoch:  425\n",
            "Train loss:  0.023978386\n",
            "Validation loss:  [[0.0696077]]\n",
            "------ Epoch:  426\n",
            "Train loss:  0.022205548\n",
            "Validation loss:  [[0.04114132]]\n",
            "------ Epoch:  427\n",
            "Train loss:  0.023832543\n",
            "Validation loss:  [[0.0693102]]\n",
            "------ Epoch:  428\n",
            "Train loss:  0.022080373\n",
            "Validation loss:  [[0.04100066]]\n",
            "------ Epoch:  429\n",
            "Train loss:  0.023688104\n",
            "Validation loss:  [[0.0690152]]\n",
            "------ Epoch:  430\n",
            "Train loss:  0.021956464\n",
            "Validation loss:  [[0.04086166]]\n",
            "------ Epoch:  431\n",
            "Train loss:  0.02354507\n",
            "Validation loss:  [[0.06872258]]\n",
            "------ Epoch:  432\n",
            "Train loss:  0.02183334\n",
            "Validation loss:  [[0.04072419]]\n",
            "------ Epoch:  433\n",
            "Train loss:  0.02340322\n",
            "Validation loss:  [[0.06843268]]\n",
            "------ Epoch:  434\n",
            "Train loss:  0.021711618\n",
            "Validation loss:  [[0.04058846]]\n",
            "------ Epoch:  435\n",
            "Train loss:  0.02326301\n",
            "Validation loss:  [[0.06814495]]\n",
            "------ Epoch:  436\n",
            "Train loss:  0.021590767\n",
            "Validation loss:  [[0.0404542]]\n",
            "------ Epoch:  437\n",
            "Train loss:  0.023124017\n",
            "Validation loss:  [[0.06786032]]\n",
            "------ Epoch:  438\n",
            "Train loss:  0.021471353\n",
            "Validation loss:  [[0.04032161]]\n",
            "------ Epoch:  439\n",
            "Train loss:  0.022986472\n",
            "Validation loss:  [[0.06757797]]\n",
            "------ Epoch:  440\n",
            "Train loss:  0.021352911\n",
            "Validation loss:  [[0.04019073]]\n",
            "------ Epoch:  441\n",
            "Train loss:  0.022850363\n",
            "Validation loss:  [[0.0672979]]\n",
            "------ Epoch:  442\n",
            "Train loss:  0.021235578\n",
            "Validation loss:  [[0.04006137]]\n",
            "------ Epoch:  443\n",
            "Train loss:  0.02271553\n",
            "Validation loss:  [[0.06702027]]\n",
            "------ Epoch:  444\n",
            "Train loss:  0.021119231\n",
            "Validation loss:  [[0.03993304]]\n",
            "------ Epoch:  445\n",
            "Train loss:  0.022581624\n",
            "Validation loss:  [[0.06674466]]\n",
            "------ Epoch:  446\n",
            "Train loss:  0.021003762\n",
            "Validation loss:  [[0.03980638]]\n",
            "------ Epoch:  447\n",
            "Train loss:  0.022449158\n",
            "Validation loss:  [[0.06647129]]\n",
            "------ Epoch:  448\n",
            "Train loss:  0.020889401\n",
            "Validation loss:  [[0.03968117]]\n",
            "------ Epoch:  449\n",
            "Train loss:  0.022317804\n",
            "Validation loss:  [[0.06620006]]\n",
            "------ Epoch:  450\n",
            "Train loss:  0.020775987\n",
            "Validation loss:  [[0.0395572]]\n",
            "------ Epoch:  451\n",
            "Train loss:  0.022187777\n",
            "Validation loss:  [[0.06593129]]\n",
            "------ Epoch:  452\n",
            "Train loss:  0.0206636\n",
            "Validation loss:  [[0.03943481]]\n",
            "------ Epoch:  453\n",
            "Train loss:  0.022059005\n",
            "Validation loss:  [[0.06566468]]\n",
            "------ Epoch:  454\n",
            "Train loss:  0.020552248\n",
            "Validation loss:  [[0.03931371]]\n",
            "------ Epoch:  455\n",
            "Train loss:  0.021931317\n",
            "Validation loss:  [[0.06540027]]\n",
            "------ Epoch:  456\n",
            "Train loss:  0.020441802\n",
            "Validation loss:  [[0.03919397]]\n",
            "------ Epoch:  457\n",
            "Train loss:  0.021804962\n",
            "Validation loss:  [[0.06513819]]\n",
            "------ Epoch:  458\n",
            "Train loss:  0.020332433\n",
            "Validation loss:  [[0.03907554]]\n",
            "------ Epoch:  459\n",
            "Train loss:  0.021679701\n",
            "Validation loss:  [[0.0648779]]\n",
            "------ Epoch:  460\n",
            "Train loss:  0.020223832\n",
            "Validation loss:  [[0.03895839]]\n",
            "------ Epoch:  461\n",
            "Train loss:  0.021555476\n",
            "Validation loss:  [[0.06461953]]\n",
            "------ Epoch:  462\n",
            "Train loss:  0.020116147\n",
            "Validation loss:  [[0.03884234]]\n",
            "------ Epoch:  463\n",
            "Train loss:  0.021432368\n",
            "Validation loss:  [[0.06436359]]\n",
            "------ Epoch:  464\n",
            "Train loss:  0.020009618\n",
            "Validation loss:  [[0.038728]]\n",
            "------ Epoch:  465\n",
            "Train loss:  0.021310795\n",
            "Validation loss:  [[0.0641099]]\n",
            "------ Epoch:  466\n",
            "Train loss:  0.01990406\n",
            "Validation loss:  [[0.03861476]]\n",
            "------ Epoch:  467\n",
            "Train loss:  0.021190085\n",
            "Validation loss:  [[0.06385814]]\n",
            "------ Epoch:  468\n",
            "Train loss:  0.01979936\n",
            "Validation loss:  [[0.03850284]]\n",
            "------ Epoch:  469\n",
            "Train loss:  0.021070657\n",
            "Validation loss:  [[0.06360848]]\n",
            "------ Epoch:  470\n",
            "Train loss:  0.019695582\n",
            "Validation loss:  [[0.03839208]]\n",
            "------ Epoch:  471\n",
            "Train loss:  0.020952309\n",
            "Validation loss:  [[0.06336097]]\n",
            "------ Epoch:  472\n",
            "Train loss:  0.019592803\n",
            "Validation loss:  [[0.03828249]]\n",
            "------ Epoch:  473\n",
            "Train loss:  0.020835115\n",
            "Validation loss:  [[0.06311548]]\n",
            "------ Epoch:  474\n",
            "Train loss:  0.019491084\n",
            "Validation loss:  [[0.03817439]]\n",
            "------ Epoch:  475\n",
            "Train loss:  0.02071923\n",
            "Validation loss:  [[0.06287216]]\n",
            "------ Epoch:  476\n",
            "Train loss:  0.019390043\n",
            "Validation loss:  [[0.03806743]]\n",
            "------ Epoch:  477\n",
            "Train loss:  0.020604325\n",
            "Validation loss:  [[0.06263077]]\n",
            "------ Epoch:  478\n",
            "Train loss:  0.019290077\n",
            "Validation loss:  [[0.03796139]]\n",
            "------ Epoch:  479\n",
            "Train loss:  0.020490343\n",
            "Validation loss:  [[0.06239106]]\n",
            "------ Epoch:  480\n",
            "Train loss:  0.019190956\n",
            "Validation loss:  [[0.03785681]]\n",
            "------ Epoch:  481\n",
            "Train loss:  0.020377675\n",
            "Validation loss:  [[0.06215392]]\n",
            "------ Epoch:  482\n",
            "Train loss:  0.019092817\n",
            "Validation loss:  [[0.03775331]]\n",
            "------ Epoch:  483\n",
            "Train loss:  0.02026618\n",
            "Validation loss:  [[0.06191853]]\n",
            "------ Epoch:  484\n",
            "Train loss:  0.01899554\n",
            "Validation loss:  [[0.03765093]]\n",
            "------ Epoch:  485\n",
            "Train loss:  0.020155536\n",
            "Validation loss:  [[0.06168479]]\n",
            "------ Epoch:  486\n",
            "Train loss:  0.018899027\n",
            "Validation loss:  [[0.03754948]]\n",
            "------ Epoch:  487\n",
            "Train loss:  0.020045834\n",
            "Validation loss:  [[0.06145287]]\n",
            "------ Epoch:  488\n",
            "Train loss:  0.018803386\n",
            "Validation loss:  [[0.03744924]]\n",
            "------ Epoch:  489\n",
            "Train loss:  0.01993732\n",
            "Validation loss:  [[0.06122353]]\n",
            "------ Epoch:  490\n",
            "Train loss:  0.01870894\n",
            "Validation loss:  [[0.03735033]]\n",
            "------ Epoch:  491\n",
            "Train loss:  0.019830072\n",
            "Validation loss:  [[0.06099551]]\n",
            "------ Epoch:  492\n",
            "Train loss:  0.01861501\n",
            "Validation loss:  [[0.03725214]]\n",
            "------ Epoch:  493\n",
            "Train loss:  0.019723503\n",
            "Validation loss:  [[0.06076908]]\n",
            "------ Epoch:  494\n",
            "Train loss:  0.018521719\n",
            "Validation loss:  [[0.03715481]]\n",
            "------ Epoch:  495\n",
            "Train loss:  0.019617712\n",
            "Validation loss:  [[0.06054427]]\n",
            "------ Epoch:  496\n",
            "Train loss:  0.018429272\n",
            "Validation loss:  [[0.03705846]]\n",
            "------ Epoch:  497\n",
            "Train loss:  0.019512793\n",
            "Validation loss:  [[0.06032111]]\n",
            "------ Epoch:  498\n",
            "Train loss:  0.018337488\n",
            "Validation loss:  [[0.03696303]]\n",
            "------ Epoch:  499\n",
            "Train loss:  0.01940886\n",
            "Validation loss:  [[0.06009984]]\n",
            "------ Epoch:  500\n",
            "Train loss:  0.018246628\n",
            "Validation loss:  [[0.03686887]]\n",
            "------ Epoch:  501\n",
            "Train loss:  0.01930617\n",
            "Validation loss:  [[0.05988078]]\n",
            "------ Epoch:  502\n",
            "Train loss:  0.018156702\n",
            "Validation loss:  [[0.03677569]]\n",
            "------ Epoch:  503\n",
            "Train loss:  0.019204434\n",
            "Validation loss:  [[0.05966315]]\n",
            "------ Epoch:  504\n",
            "Train loss:  0.018067535\n",
            "Validation loss:  [[0.03668343]]\n",
            "------ Epoch:  505\n",
            "Train loss:  0.019103456\n",
            "Validation loss:  [[0.05944725]]\n",
            "------ Epoch:  506\n",
            "Train loss:  0.017979134\n",
            "Validation loss:  [[0.03659204]]\n",
            "------ Epoch:  507\n",
            "Train loss:  0.01900347\n",
            "Validation loss:  [[0.05923295]]\n",
            "------ Epoch:  508\n",
            "Train loss:  0.017891387\n",
            "Validation loss:  [[0.03650147]]\n",
            "------ Epoch:  509\n",
            "Train loss:  0.018904392\n",
            "Validation loss:  [[0.05902026]]\n",
            "------ Epoch:  510\n",
            "Train loss:  0.017804436\n",
            "Validation loss:  [[0.0364118]]\n",
            "------ Epoch:  511\n",
            "Train loss:  0.01880622\n",
            "Validation loss:  [[0.05880928]]\n",
            "------ Epoch:  512\n",
            "Train loss:  0.017718278\n",
            "Validation loss:  [[0.03632327]]\n",
            "------ Epoch:  513\n",
            "Train loss:  0.018709075\n",
            "Validation loss:  [[0.05860046]]\n",
            "------ Epoch:  514\n",
            "Train loss:  0.017633168\n",
            "Validation loss:  [[0.03623572]]\n",
            "------ Epoch:  515\n",
            "Train loss:  0.018612878\n",
            "Validation loss:  [[0.05839287]]\n",
            "------ Epoch:  516\n",
            "Train loss:  0.017548466\n",
            "Validation loss:  [[0.03614882]]\n",
            "------ Epoch:  517\n",
            "Train loss:  0.018517414\n",
            "Validation loss:  [[0.05818688]]\n",
            "------ Epoch:  518\n",
            "Train loss:  0.01746458\n",
            "Validation loss:  [[0.03606279]]\n",
            "------ Epoch:  519\n",
            "Train loss:  0.018422881\n",
            "Validation loss:  [[0.0579827]]\n",
            "------ Epoch:  520\n",
            "Train loss:  0.01738151\n",
            "Validation loss:  [[0.03597779]]\n",
            "------ Epoch:  521\n",
            "Train loss:  0.018329442\n",
            "Validation loss:  [[0.05778034]]\n",
            "------ Epoch:  522\n",
            "Train loss:  0.01729939\n",
            "Validation loss:  [[0.03589366]]\n",
            "------ Epoch:  523\n",
            "Train loss:  0.018236963\n",
            "Validation loss:  [[0.05757968]]\n",
            "------ Epoch:  524\n",
            "Train loss:  0.01721795\n",
            "Validation loss:  [[0.03581053]]\n",
            "------ Epoch:  525\n",
            "Train loss:  0.018145414\n",
            "Validation loss:  [[0.05738064]]\n",
            "------ Epoch:  526\n",
            "Train loss:  0.017137289\n",
            "Validation loss:  [[0.03572797]]\n",
            "------ Epoch:  527\n",
            "Train loss:  0.018054605\n",
            "Validation loss:  [[0.05718294]]\n",
            "------ Epoch:  528\n",
            "Train loss:  0.017057259\n",
            "Validation loss:  [[0.03564627]]\n",
            "------ Epoch:  529\n",
            "Train loss:  0.0179646\n",
            "Validation loss:  [[0.05698698]]\n",
            "------ Epoch:  530\n",
            "Train loss:  0.016978025\n",
            "Validation loss:  [[0.03556541]]\n",
            "------ Epoch:  531\n",
            "Train loss:  0.017875496\n",
            "Validation loss:  [[0.05679239]]\n",
            "------ Epoch:  532\n",
            "Train loss:  0.016899353\n",
            "Validation loss:  [[0.03548509]]\n",
            "------ Epoch:  533\n",
            "Train loss:  0.017787093\n",
            "Validation loss:  [[0.05659912]]\n",
            "------ Epoch:  534\n",
            "Train loss:  0.016821397\n",
            "Validation loss:  [[0.03540566]]\n",
            "------ Epoch:  535\n",
            "Train loss:  0.017699404\n",
            "Validation loss:  [[0.05640742]]\n",
            "------ Epoch:  536\n",
            "Train loss:  0.016744154\n",
            "Validation loss:  [[0.03532707]]\n",
            "------ Epoch:  537\n",
            "Train loss:  0.017612811\n",
            "Validation loss:  [[0.05621748]]\n",
            "------ Epoch:  538\n",
            "Train loss:  0.016667664\n",
            "Validation loss:  [[0.03524932]]\n",
            "------ Epoch:  539\n",
            "Train loss:  0.01752707\n",
            "Validation loss:  [[0.05602904]]\n",
            "------ Epoch:  540\n",
            "Train loss:  0.016591886\n",
            "Validation loss:  [[0.03517206]]\n",
            "------ Epoch:  541\n",
            "Train loss:  0.017442057\n",
            "Validation loss:  [[0.05584193]]\n",
            "------ Epoch:  542\n",
            "Train loss:  0.01651669\n",
            "Validation loss:  [[0.03509562]]\n",
            "------ Epoch:  543\n",
            "Train loss:  0.017357713\n",
            "Validation loss:  [[0.05565615]]\n",
            "------ Epoch:  544\n",
            "Train loss:  0.016442152\n",
            "Validation loss:  [[0.03501977]]\n",
            "------ Epoch:  545\n",
            "Train loss:  0.017274145\n",
            "Validation loss:  [[0.05547184]]\n",
            "------ Epoch:  546\n",
            "Train loss:  0.016368316\n",
            "Validation loss:  [[0.0349448]]\n",
            "------ Epoch:  547\n",
            "Train loss:  0.01719145\n",
            "Validation loss:  [[0.0552889]]\n",
            "------ Epoch:  548\n",
            "Train loss:  0.016295157\n",
            "Validation loss:  [[0.03487032]]\n",
            "------ Epoch:  549\n",
            "Train loss:  0.017109506\n",
            "Validation loss:  [[0.05510743]]\n",
            "------ Epoch:  550\n",
            "Train loss:  0.016222414\n",
            "Validation loss:  [[0.03479638]]\n",
            "------ Epoch:  551\n",
            "Train loss:  0.017028062\n",
            "Validation loss:  [[0.05492708]]\n",
            "------ Epoch:  552\n",
            "Train loss:  0.016150374\n",
            "Validation loss:  [[0.03472317]]\n",
            "------ Epoch:  553\n",
            "Train loss:  0.016947532\n",
            "Validation loss:  [[0.0547481]]\n",
            "------ Epoch:  554\n",
            "Train loss:  0.016079025\n",
            "Validation loss:  [[0.03465078]]\n",
            "------ Epoch:  555\n",
            "Train loss:  0.016867813\n",
            "Validation loss:  [[0.05457041]]\n",
            "------ Epoch:  556\n",
            "Train loss:  0.016008323\n",
            "Validation loss:  [[0.03457881]]\n",
            "------ Epoch:  557\n",
            "Train loss:  0.016788756\n",
            "Validation loss:  [[0.05439428]]\n",
            "------ Epoch:  558\n",
            "Train loss:  0.015938198\n",
            "Validation loss:  [[0.03450753]]\n",
            "------ Epoch:  559\n",
            "Train loss:  0.016710337\n",
            "Validation loss:  [[0.05421926]]\n",
            "------ Epoch:  560\n",
            "Train loss:  0.01586859\n",
            "Validation loss:  [[0.03443671]]\n",
            "------ Epoch:  561\n",
            "Train loss:  0.016632635\n",
            "Validation loss:  [[0.05404568]]\n",
            "------ Epoch:  562\n",
            "Train loss:  0.015799653\n",
            "Validation loss:  [[0.03436675]]\n",
            "------ Epoch:  563\n",
            "Train loss:  0.016555684\n",
            "Validation loss:  [[0.053873]]\n",
            "------ Epoch:  564\n",
            "Train loss:  0.015731245\n",
            "Validation loss:  [[0.03429713]]\n",
            "------ Epoch:  565\n",
            "Train loss:  0.016479425\n",
            "Validation loss:  [[0.05370196]]\n",
            "------ Epoch:  566\n",
            "Train loss:  0.015663512\n",
            "Validation loss:  [[0.03422817]]\n",
            "------ Epoch:  567\n",
            "Train loss:  0.016403768\n",
            "Validation loss:  [[0.05353226]]\n",
            "------ Epoch:  568\n",
            "Train loss:  0.0155963935\n",
            "Validation loss:  [[0.03415983]]\n",
            "------ Epoch:  569\n",
            "Train loss:  0.016328987\n",
            "Validation loss:  [[0.05336349]]\n",
            "------ Epoch:  570\n",
            "Train loss:  0.0155298095\n",
            "Validation loss:  [[0.03409192]]\n",
            "------ Epoch:  571\n",
            "Train loss:  0.016254764\n",
            "Validation loss:  [[0.05319638]]\n",
            "------ Epoch:  572\n",
            "Train loss:  0.015463916\n",
            "Validation loss:  [[0.03402467]]\n",
            "------ Epoch:  573\n",
            "Train loss:  0.0161812\n",
            "Validation loss:  [[0.0530301]]\n",
            "------ Epoch:  574\n",
            "Train loss:  0.015398477\n",
            "Validation loss:  [[0.03395798]]\n",
            "------ Epoch:  575\n",
            "Train loss:  0.016108723\n",
            "Validation loss:  [[0.05286576]]\n",
            "------ Epoch:  576\n",
            "Train loss:  0.015333867\n",
            "Validation loss:  [[0.03389189]]\n",
            "------ Epoch:  577\n",
            "Train loss:  0.016036497\n",
            "Validation loss:  [[0.05270214]]\n",
            "------ Epoch:  578\n",
            "Train loss:  0.015269752\n",
            "Validation loss:  [[0.03382639]]\n",
            "------ Epoch:  579\n",
            "Train loss:  0.015965242\n",
            "Validation loss:  [[0.05253977]]\n",
            "------ Epoch:  580\n",
            "Train loss:  0.01520619\n",
            "Validation loss:  [[0.03376136]]\n",
            "------ Epoch:  581\n",
            "Train loss:  0.015894627\n",
            "Validation loss:  [[0.05237923]]\n",
            "------ Epoch:  582\n",
            "Train loss:  0.015143393\n",
            "Validation loss:  [[0.03369697]]\n",
            "------ Epoch:  583\n",
            "Train loss:  0.015824808\n",
            "Validation loss:  [[0.05221947]]\n",
            "------ Epoch:  584\n",
            "Train loss:  0.0150810145\n",
            "Validation loss:  [[0.03363294]]\n",
            "------ Epoch:  585\n",
            "Train loss:  0.015755367\n",
            "Validation loss:  [[0.05206072]]\n",
            "------ Epoch:  586\n",
            "Train loss:  0.015019152\n",
            "Validation loss:  [[0.03356943]]\n",
            "------ Epoch:  587\n",
            "Train loss:  0.015686736\n",
            "Validation loss:  [[0.05190332]]\n",
            "------ Epoch:  588\n",
            "Train loss:  0.014957881\n",
            "Validation loss:  [[0.03350643]]\n",
            "------ Epoch:  589\n",
            "Train loss:  0.015618728\n",
            "Validation loss:  [[0.05174688]]\n",
            "------ Epoch:  590\n",
            "Train loss:  0.014897051\n",
            "Validation loss:  [[0.03344382]]\n",
            "------ Epoch:  591\n",
            "Train loss:  0.015551171\n",
            "Validation loss:  [[0.05159185]]\n",
            "------ Epoch:  592\n",
            "Train loss:  0.014836886\n",
            "Validation loss:  [[0.03338172]]\n",
            "------ Epoch:  593\n",
            "Train loss:  0.015484306\n",
            "Validation loss:  [[0.05143748]]\n",
            "------ Epoch:  594\n",
            "Train loss:  0.014777037\n",
            "Validation loss:  [[0.03331995]]\n",
            "------ Epoch:  595\n",
            "Train loss:  0.0154179735\n",
            "Validation loss:  [[0.05128428]]\n",
            "------ Epoch:  596\n",
            "Train loss:  0.014717758\n",
            "Validation loss:  [[0.03325863]]\n",
            "------ Epoch:  597\n",
            "Train loss:  0.015352142\n",
            "Validation loss:  [[0.05113217]]\n",
            "------ Epoch:  598\n",
            "Train loss:  0.014658961\n",
            "Validation loss:  [[0.03319776]]\n",
            "------ Epoch:  599\n",
            "Train loss:  0.01528709\n",
            "Validation loss:  [[0.05098125]]\n",
            "------ Epoch:  600\n",
            "Train loss:  0.014600766\n",
            "Validation loss:  [[0.03313742]]\n",
            "------ Epoch:  601\n",
            "Train loss:  0.015222539\n",
            "Validation loss:  [[0.05083118]]\n",
            "------ Epoch:  602\n",
            "Train loss:  0.014543066\n",
            "Validation loss:  [[0.03307743]]\n",
            "------ Epoch:  603\n",
            "Train loss:  0.015158617\n",
            "Validation loss:  [[0.05068256]]\n",
            "------ Epoch:  604\n",
            "Train loss:  0.014485904\n",
            "Validation loss:  [[0.03301803]]\n",
            "------ Epoch:  605\n",
            "Train loss:  0.01509554\n",
            "Validation loss:  [[0.050535]]\n",
            "------ Epoch:  606\n",
            "Train loss:  0.014429316\n",
            "Validation loss:  [[0.03295893]]\n",
            "------ Epoch:  607\n",
            "Train loss:  0.01503284\n",
            "Validation loss:  [[0.05038843]]\n",
            "------ Epoch:  608\n",
            "Train loss:  0.014373106\n",
            "Validation loss:  [[0.03290027]]\n",
            "------ Epoch:  609\n",
            "Train loss:  0.01497065\n",
            "Validation loss:  [[0.0502428]]\n",
            "------ Epoch:  610\n",
            "Train loss:  0.014317518\n",
            "Validation loss:  [[0.0328421]]\n",
            "------ Epoch:  611\n",
            "Train loss:  0.014909176\n",
            "Validation loss:  [[0.05009821]]\n",
            "------ Epoch:  612\n",
            "Train loss:  0.0142624015\n",
            "Validation loss:  [[0.03278416]]\n",
            "------ Epoch:  613\n",
            "Train loss:  0.014848105\n",
            "Validation loss:  [[0.04995457]]\n",
            "------ Epoch:  614\n",
            "Train loss:  0.014207667\n",
            "Validation loss:  [[0.03272666]]\n",
            "------ Epoch:  615\n",
            "Train loss:  0.014787715\n",
            "Validation loss:  [[0.04981215]]\n",
            "------ Epoch:  616\n",
            "Train loss:  0.014153571\n",
            "Validation loss:  [[0.0326697]]\n",
            "------ Epoch:  617\n",
            "Train loss:  0.014728085\n",
            "Validation loss:  [[0.04967083]]\n",
            "------ Epoch:  618\n",
            "Train loss:  0.014099922\n",
            "Validation loss:  [[0.03261301]]\n",
            "------ Epoch:  619\n",
            "Train loss:  0.014668828\n",
            "Validation loss:  [[0.04953061]]\n",
            "------ Epoch:  620\n",
            "Train loss:  0.014046826\n",
            "Validation loss:  [[0.03255675]]\n",
            "------ Epoch:  621\n",
            "Train loss:  0.014610122\n",
            "Validation loss:  [[0.04939108]]\n",
            "------ Epoch:  622\n",
            "Train loss:  0.0139941415\n",
            "Validation loss:  [[0.03250089]]\n",
            "------ Epoch:  623\n",
            "Train loss:  0.014552025\n",
            "Validation loss:  [[0.0492528]]\n",
            "------ Epoch:  624\n",
            "Train loss:  0.013941913\n",
            "Validation loss:  [[0.03244542]]\n",
            "------ Epoch:  625\n",
            "Train loss:  0.014494469\n",
            "Validation loss:  [[0.04911544]]\n",
            "------ Epoch:  626\n",
            "Train loss:  0.0138903195\n",
            "Validation loss:  [[0.03239027]]\n",
            "------ Epoch:  627\n",
            "Train loss:  0.014437537\n",
            "Validation loss:  [[0.04897901]]\n",
            "------ Epoch:  628\n",
            "Train loss:  0.013839039\n",
            "Validation loss:  [[0.0323355]]\n",
            "------ Epoch:  629\n",
            "Train loss:  0.014381078\n",
            "Validation loss:  [[0.04884362]]\n",
            "------ Epoch:  630\n",
            "Train loss:  0.0137883825\n",
            "Validation loss:  [[0.0322811]]\n",
            "------ Epoch:  631\n",
            "Train loss:  0.014325165\n",
            "Validation loss:  [[0.04870924]]\n",
            "------ Epoch:  632\n",
            "Train loss:  0.013738018\n",
            "Validation loss:  [[0.03222699]]\n",
            "------ Epoch:  633\n",
            "Train loss:  0.014269742\n",
            "Validation loss:  [[0.04857564]]\n",
            "------ Epoch:  634\n",
            "Train loss:  0.013688161\n",
            "Validation loss:  [[0.03217326]]\n",
            "------ Epoch:  635\n",
            "Train loss:  0.014214767\n",
            "Validation loss:  [[0.04844294]]\n",
            "------ Epoch:  636\n",
            "Train loss:  0.013638709\n",
            "Validation loss:  [[0.03211991]]\n",
            "------ Epoch:  637\n",
            "Train loss:  0.014160475\n",
            "Validation loss:  [[0.04831126]]\n",
            "------ Epoch:  638\n",
            "Train loss:  0.013589765\n",
            "Validation loss:  [[0.03206688]]\n",
            "------ Epoch:  639\n",
            "Train loss:  0.014106615\n",
            "Validation loss:  [[0.04818067]]\n",
            "------ Epoch:  640\n",
            "Train loss:  0.013541293\n",
            "Validation loss:  [[0.03201403]]\n",
            "------ Epoch:  641\n",
            "Train loss:  0.01405311\n",
            "Validation loss:  [[0.0480506]]\n",
            "------ Epoch:  642\n",
            "Train loss:  0.013493152\n",
            "Validation loss:  [[0.03196165]]\n",
            "------ Epoch:  643\n",
            "Train loss:  0.014000312\n",
            "Validation loss:  [[0.04792155]]\n",
            "------ Epoch:  644\n",
            "Train loss:  0.013445368\n",
            "Validation loss:  [[0.03190948]]\n",
            "------ Epoch:  645\n",
            "Train loss:  0.013947804\n",
            "Validation loss:  [[0.04779324]]\n",
            "------ Epoch:  646\n",
            "Train loss:  0.013398153\n",
            "Validation loss:  [[0.03185754]]\n",
            "------ Epoch:  647\n",
            "Train loss:  0.013895795\n",
            "Validation loss:  [[0.04766585]]\n",
            "------ Epoch:  648\n",
            "Train loss:  0.013351248\n",
            "Validation loss:  [[0.03180603]]\n",
            "------ Epoch:  649\n",
            "Train loss:  0.013844369\n",
            "Validation loss:  [[0.04753935]]\n",
            "------ Epoch:  650\n",
            "Train loss:  0.013304758\n",
            "Validation loss:  [[0.03175462]]\n",
            "------ Epoch:  651\n",
            "Train loss:  0.0137931835\n",
            "Validation loss:  [[0.04741341]]\n",
            "------ Epoch:  652\n",
            "Train loss:  0.013258594\n",
            "Validation loss:  [[0.03170359]]\n",
            "------ Epoch:  653\n",
            "Train loss:  0.013742643\n",
            "Validation loss:  [[0.04728862]]\n",
            "------ Epoch:  654\n",
            "Train loss:  0.013212952\n",
            "Validation loss:  [[0.03165291]]\n",
            "------ Epoch:  655\n",
            "Train loss:  0.013692522\n",
            "Validation loss:  [[0.04716446]]\n",
            "------ Epoch:  656\n",
            "Train loss:  0.013167577\n",
            "Validation loss:  [[0.03160246]]\n",
            "------ Epoch:  657\n",
            "Train loss:  0.013642762\n",
            "Validation loss:  [[0.0470411]]\n",
            "------ Epoch:  658\n",
            "Train loss:  0.013122707\n",
            "Validation loss:  [[0.03155219]]\n",
            "------ Epoch:  659\n",
            "Train loss:  0.013593448\n",
            "Validation loss:  [[0.04691871]]\n",
            "------ Epoch:  660\n",
            "Train loss:  0.013078169\n",
            "Validation loss:  [[0.03150244]]\n",
            "------ Epoch:  661\n",
            "Train loss:  0.013544863\n",
            "Validation loss:  [[0.04679756]]\n",
            "------ Epoch:  662\n",
            "Train loss:  0.013034288\n",
            "Validation loss:  [[0.03145299]]\n",
            "------ Epoch:  663\n",
            "Train loss:  0.013496721\n",
            "Validation loss:  [[0.04667667]]\n",
            "------ Epoch:  664\n",
            "Train loss:  0.01299059\n",
            "Validation loss:  [[0.03140361]]\n",
            "------ Epoch:  665\n",
            "Train loss:  0.013448746\n",
            "Validation loss:  [[0.04655645]]\n",
            "------ Epoch:  666\n",
            "Train loss:  0.012947204\n",
            "Validation loss:  [[0.03135456]]\n",
            "------ Epoch:  667\n",
            "Train loss:  0.013401196\n",
            "Validation loss:  [[0.04643711]]\n",
            "------ Epoch:  668\n",
            "Train loss:  0.012904194\n",
            "Validation loss:  [[0.03130568]]\n",
            "------ Epoch:  669\n",
            "Train loss:  0.0133541925\n",
            "Validation loss:  [[0.04631905]]\n",
            "------ Epoch:  670\n",
            "Train loss:  0.012861793\n",
            "Validation loss:  [[0.03125732]]\n",
            "------ Epoch:  671\n",
            "Train loss:  0.013307903\n",
            "Validation loss:  [[0.04620157]]\n",
            "------ Epoch:  672\n",
            "Train loss:  0.012819711\n",
            "Validation loss:  [[0.03120909]]\n",
            "------ Epoch:  673\n",
            "Train loss:  0.01326173\n",
            "Validation loss:  [[0.04608467]]\n",
            "------ Epoch:  674\n",
            "Train loss:  0.012778006\n",
            "Validation loss:  [[0.03116111]]\n",
            "------ Epoch:  675\n",
            "Train loss:  0.013216159\n",
            "Validation loss:  [[0.04596886]]\n",
            "------ Epoch:  676\n",
            "Train loss:  0.012736659\n",
            "Validation loss:  [[0.0311134]]\n",
            "------ Epoch:  677\n",
            "Train loss:  0.013170911\n",
            "Validation loss:  [[0.04585365]]\n",
            "------ Epoch:  678\n",
            "Train loss:  0.012695749\n",
            "Validation loss:  [[0.03106607]]\n",
            "------ Epoch:  679\n",
            "Train loss:  0.013126267\n",
            "Validation loss:  [[0.04573943]]\n",
            "------ Epoch:  680\n",
            "Train loss:  0.012655186\n",
            "Validation loss:  [[0.03101891]]\n",
            "------ Epoch:  681\n",
            "Train loss:  0.013082037\n",
            "Validation loss:  [[0.04562599]]\n",
            "------ Epoch:  682\n",
            "Train loss:  0.01261506\n",
            "Validation loss:  [[0.03097206]]\n",
            "------ Epoch:  683\n",
            "Train loss:  0.013038206\n",
            "Validation loss:  [[0.04551358]]\n",
            "------ Epoch:  684\n",
            "Train loss:  0.01257538\n",
            "Validation loss:  [[0.03092548]]\n",
            "------ Epoch:  685\n",
            "Train loss:  0.0129948845\n",
            "Validation loss:  [[0.04540151]]\n",
            "------ Epoch:  686\n",
            "Train loss:  0.012535995\n",
            "Validation loss:  [[0.03087916]]\n",
            "------ Epoch:  687\n",
            "Train loss:  0.012951949\n",
            "Validation loss:  [[0.04529056]]\n",
            "------ Epoch:  688\n",
            "Train loss:  0.012497036\n",
            "Validation loss:  [[0.03083302]]\n",
            "------ Epoch:  689\n",
            "Train loss:  0.012909322\n",
            "Validation loss:  [[0.04518013]]\n",
            "------ Epoch:  690\n",
            "Train loss:  0.012458337\n",
            "Validation loss:  [[0.03078704]]\n",
            "------ Epoch:  691\n",
            "Train loss:  0.012867093\n",
            "Validation loss:  [[0.04507029]]\n",
            "------ Epoch:  692\n",
            "Train loss:  0.012419979\n",
            "Validation loss:  [[0.03074142]]\n",
            "------ Epoch:  693\n",
            "Train loss:  0.012825334\n",
            "Validation loss:  [[0.04496114]]\n",
            "------ Epoch:  694\n",
            "Train loss:  0.012381939\n",
            "Validation loss:  [[0.03069593]]\n",
            "------ Epoch:  695\n",
            "Train loss:  0.0127839\n",
            "Validation loss:  [[0.0448528]]\n",
            "------ Epoch:  696\n",
            "Train loss:  0.012344272\n",
            "Validation loss:  [[0.03065073]]\n",
            "------ Epoch:  697\n",
            "Train loss:  0.012742732\n",
            "Validation loss:  [[0.04474481]]\n",
            "------ Epoch:  698\n",
            "Train loss:  0.012306799\n",
            "Validation loss:  [[0.03060552]]\n",
            "------ Epoch:  699\n",
            "Train loss:  0.012701963\n",
            "Validation loss:  [[0.044638]]\n",
            "------ Epoch:  700\n",
            "Train loss:  0.012269861\n",
            "Validation loss:  [[0.03056072]]\n",
            "------ Epoch:  701\n",
            "Train loss:  0.012661745\n",
            "Validation loss:  [[0.04453159]]\n",
            "------ Epoch:  702\n",
            "Train loss:  0.012233117\n",
            "Validation loss:  [[0.03051603]]\n",
            "------ Epoch:  703\n",
            "Train loss:  0.012621653\n",
            "Validation loss:  [[0.0444258]]\n",
            "------ Epoch:  704\n",
            "Train loss:  0.01219671\n",
            "Validation loss:  [[0.0304717]]\n",
            "------ Epoch:  705\n",
            "Train loss:  0.012582169\n",
            "Validation loss:  [[0.04432113]]\n",
            "------ Epoch:  706\n",
            "Train loss:  0.012160768\n",
            "Validation loss:  [[0.03042746]]\n",
            "------ Epoch:  707\n",
            "Train loss:  0.012543042\n",
            "Validation loss:  [[0.04421677]]\n",
            "------ Epoch:  708\n",
            "Train loss:  0.012125099\n",
            "Validation loss:  [[0.03038357]]\n",
            "------ Epoch:  709\n",
            "Train loss:  0.012504237\n",
            "Validation loss:  [[0.04411311]]\n",
            "------ Epoch:  710\n",
            "Train loss:  0.012089746\n",
            "Validation loss:  [[0.03033976]]\n",
            "------ Epoch:  711\n",
            "Train loss:  0.0124657545\n",
            "Validation loss:  [[0.04401006]]\n",
            "------ Epoch:  712\n",
            "Train loss:  0.012054636\n",
            "Validation loss:  [[0.03029604]]\n",
            "------ Epoch:  713\n",
            "Train loss:  0.012427491\n",
            "Validation loss:  [[0.0439075]]\n",
            "------ Epoch:  714\n",
            "Train loss:  0.012019764\n",
            "Validation loss:  [[0.03025256]]\n",
            "------ Epoch:  715\n",
            "Train loss:  0.012389614\n",
            "Validation loss:  [[0.04380583]]\n",
            "------ Epoch:  716\n",
            "Train loss:  0.01198526\n",
            "Validation loss:  [[0.03020948]]\n",
            "------ Epoch:  717\n",
            "Train loss:  0.012352228\n",
            "Validation loss:  [[0.04370473]]\n",
            "------ Epoch:  718\n",
            "Train loss:  0.011951113\n",
            "Validation loss:  [[0.03016645]]\n",
            "------ Epoch:  719\n",
            "Train loss:  0.012315156\n",
            "Validation loss:  [[0.04360458]]\n",
            "------ Epoch:  720\n",
            "Train loss:  0.011917425\n",
            "Validation loss:  [[0.03012375]]\n",
            "------ Epoch:  721\n",
            "Train loss:  0.012278551\n",
            "Validation loss:  [[0.04350489]]\n",
            "------ Epoch:  722\n",
            "Train loss:  0.011883927\n",
            "Validation loss:  [[0.03008119]]\n",
            "------ Epoch:  723\n",
            "Train loss:  0.012242279\n",
            "Validation loss:  [[0.04340591]]\n",
            "------ Epoch:  724\n",
            "Train loss:  0.01185076\n",
            "Validation loss:  [[0.03003882]]\n",
            "------ Epoch:  725\n",
            "Train loss:  0.012206163\n",
            "Validation loss:  [[0.04330722]]\n",
            "------ Epoch:  726\n",
            "Train loss:  0.011817845\n",
            "Validation loss:  [[0.02999659]]\n",
            "------ Epoch:  727\n",
            "Train loss:  0.012170674\n",
            "Validation loss:  [[0.04320984]]\n",
            "------ Epoch:  728\n",
            "Train loss:  0.0117854895\n",
            "Validation loss:  [[0.02995471]]\n",
            "------ Epoch:  729\n",
            "Train loss:  0.012135534\n",
            "Validation loss:  [[0.04311286]]\n",
            "------ Epoch:  730\n",
            "Train loss:  0.011753332\n",
            "Validation loss:  [[0.02991302]]\n",
            "------ Epoch:  731\n",
            "Train loss:  0.012100636\n",
            "Validation loss:  [[0.04301635]]\n",
            "------ Epoch:  732\n",
            "Train loss:  0.011721383\n",
            "Validation loss:  [[0.02987142]]\n",
            "------ Epoch:  733\n",
            "Train loss:  0.012065939\n",
            "Validation loss:  [[0.04292035]]\n",
            "------ Epoch:  734\n",
            "Train loss:  0.011689651\n",
            "Validation loss:  [[0.02982995]]\n",
            "------ Epoch:  735\n",
            "Train loss:  0.012031601\n",
            "Validation loss:  [[0.04282485]]\n",
            "------ Epoch:  736\n",
            "Train loss:  0.011658192\n",
            "Validation loss:  [[0.02978855]]\n",
            "------ Epoch:  737\n",
            "Train loss:  0.011997398\n",
            "Validation loss:  [[0.04272977]]\n",
            "------ Epoch:  738\n",
            "Train loss:  0.0116270445\n",
            "Validation loss:  [[0.02974738]]\n",
            "------ Epoch:  739\n",
            "Train loss:  0.0119636925\n",
            "Validation loss:  [[0.04263548]]\n",
            "------ Epoch:  740\n",
            "Train loss:  0.0115961265\n",
            "Validation loss:  [[0.02970649]]\n",
            "------ Epoch:  741\n",
            "Train loss:  0.011930343\n",
            "Validation loss:  [[0.04254195]]\n",
            "------ Epoch:  742\n",
            "Train loss:  0.011565608\n",
            "Validation loss:  [[0.02966576]]\n",
            "------ Epoch:  743\n",
            "Train loss:  0.011897377\n",
            "Validation loss:  [[0.04244924]]\n",
            "------ Epoch:  744\n",
            "Train loss:  0.0115355365\n",
            "Validation loss:  [[0.02962528]]\n",
            "------ Epoch:  745\n",
            "Train loss:  0.0118647795\n",
            "Validation loss:  [[0.04235706]]\n",
            "------ Epoch:  746\n",
            "Train loss:  0.011505664\n",
            "Validation loss:  [[0.02958501]]\n",
            "------ Epoch:  747\n",
            "Train loss:  0.011832458\n",
            "Validation loss:  [[0.04226539]]\n",
            "------ Epoch:  748\n",
            "Train loss:  0.011476044\n",
            "Validation loss:  [[0.0295449]]\n",
            "------ Epoch:  749\n",
            "Train loss:  0.011800451\n",
            "Validation loss:  [[0.04217404]]\n",
            "------ Epoch:  750\n",
            "Train loss:  0.011446578\n",
            "Validation loss:  [[0.02950487]]\n",
            "------ Epoch:  751\n",
            "Train loss:  0.011768669\n",
            "Validation loss:  [[0.04208369]]\n",
            "------ Epoch:  752\n",
            "Train loss:  0.011417526\n",
            "Validation loss:  [[0.029465]]\n",
            "------ Epoch:  753\n",
            "Train loss:  0.011737176\n",
            "Validation loss:  [[0.04199352]]\n",
            "------ Epoch:  754\n",
            "Train loss:  0.011388659\n",
            "Validation loss:  [[0.02942538]]\n",
            "------ Epoch:  755\n",
            "Train loss:  0.011706044\n",
            "Validation loss:  [[0.04190411]]\n",
            "------ Epoch:  756\n",
            "Train loss:  0.011360102\n",
            "Validation loss:  [[0.02938584]]\n",
            "------ Epoch:  757\n",
            "Train loss:  0.011675169\n",
            "Validation loss:  [[0.04181511]]\n",
            "------ Epoch:  758\n",
            "Train loss:  0.011331788\n",
            "Validation loss:  [[0.02934648]]\n",
            "------ Epoch:  759\n",
            "Train loss:  0.0116446735\n",
            "Validation loss:  [[0.04172669]]\n",
            "------ Epoch:  760\n",
            "Train loss:  0.011303709\n",
            "Validation loss:  [[0.02930726]]\n",
            "------ Epoch:  761\n",
            "Train loss:  0.011614278\n",
            "Validation loss:  [[0.04163879]]\n",
            "------ Epoch:  762\n",
            "Train loss:  0.011275839\n",
            "Validation loss:  [[0.02926819]]\n",
            "------ Epoch:  763\n",
            "Train loss:  0.011584221\n",
            "Validation loss:  [[0.0415514]]\n",
            "------ Epoch:  764\n",
            "Train loss:  0.0112483\n",
            "Validation loss:  [[0.0292293]]\n",
            "------ Epoch:  765\n",
            "Train loss:  0.011554476\n",
            "Validation loss:  [[0.04146441]]\n",
            "------ Epoch:  766\n",
            "Train loss:  0.01122086\n",
            "Validation loss:  [[0.02919044]]\n",
            "------ Epoch:  767\n",
            "Train loss:  0.011524895\n",
            "Validation loss:  [[0.04137829]]\n",
            "------ Epoch:  768\n",
            "Train loss:  0.011193861\n",
            "Validation loss:  [[0.02915199]]\n",
            "------ Epoch:  769\n",
            "Train loss:  0.0114958165\n",
            "Validation loss:  [[0.04129238]]\n",
            "------ Epoch:  770\n",
            "Train loss:  0.011167015\n",
            "Validation loss:  [[0.02911351]]\n",
            "------ Epoch:  771\n",
            "Train loss:  0.011466848\n",
            "Validation loss:  [[0.04120684]]\n",
            "------ Epoch:  772\n",
            "Train loss:  0.011140319\n",
            "Validation loss:  [[0.02907507]]\n",
            "------ Epoch:  773\n",
            "Train loss:  0.011438039\n",
            "Validation loss:  [[0.04112193]]\n",
            "------ Epoch:  774\n",
            "Train loss:  0.011113893\n",
            "Validation loss:  [[0.02903693]]\n",
            "------ Epoch:  775\n",
            "Train loss:  0.011409617\n",
            "Validation loss:  [[0.04103751]]\n",
            "------ Epoch:  776\n",
            "Train loss:  0.011087734\n",
            "Validation loss:  [[0.02899892]]\n",
            "------ Epoch:  777\n",
            "Train loss:  0.011381521\n",
            "Validation loss:  [[0.04095387]]\n",
            "------ Epoch:  778\n",
            "Train loss:  0.0110619115\n",
            "Validation loss:  [[0.02896109]]\n",
            "------ Epoch:  779\n",
            "Train loss:  0.011353693\n",
            "Validation loss:  [[0.04087054]]\n",
            "------ Epoch:  780\n",
            "Train loss:  0.011036267\n",
            "Validation loss:  [[0.02892333]]\n",
            "------ Epoch:  781\n",
            "Train loss:  0.0113261035\n",
            "Validation loss:  [[0.04078784]]\n",
            "------ Epoch:  782\n",
            "Train loss:  0.011010906\n",
            "Validation loss:  [[0.02888581]]\n",
            "------ Epoch:  783\n",
            "Train loss:  0.011298856\n",
            "Validation loss:  [[0.04070554]]\n",
            "------ Epoch:  784\n",
            "Train loss:  0.010985718\n",
            "Validation loss:  [[0.02884837]]\n",
            "------ Epoch:  785\n",
            "Train loss:  0.011271632\n",
            "Validation loss:  [[0.04062352]]\n",
            "------ Epoch:  786\n",
            "Train loss:  0.010960729\n",
            "Validation loss:  [[0.02881103]]\n",
            "------ Epoch:  787\n",
            "Train loss:  0.01124473\n",
            "Validation loss:  [[0.0405419]]\n",
            "------ Epoch:  788\n",
            "Train loss:  0.010935943\n",
            "Validation loss:  [[0.02877382]]\n",
            "------ Epoch:  789\n",
            "Train loss:  0.011218171\n",
            "Validation loss:  [[0.04046106]]\n",
            "------ Epoch:  790\n",
            "Train loss:  0.010911311\n",
            "Validation loss:  [[0.02873675]]\n",
            "------ Epoch:  791\n",
            "Train loss:  0.011191666\n",
            "Validation loss:  [[0.04038051]]\n",
            "------ Epoch:  792\n",
            "Train loss:  0.010886946\n",
            "Validation loss:  [[0.02869978]]\n",
            "------ Epoch:  793\n",
            "Train loss:  0.01116554\n",
            "Validation loss:  [[0.04030034]]\n",
            "------ Epoch:  794\n",
            "Train loss:  0.010862903\n",
            "Validation loss:  [[0.028663]]\n",
            "------ Epoch:  795\n",
            "Train loss:  0.011139659\n",
            "Validation loss:  [[0.04022095]]\n",
            "------ Epoch:  796\n",
            "Train loss:  0.0108390665\n",
            "Validation loss:  [[0.02862642]]\n",
            "------ Epoch:  797\n",
            "Train loss:  0.011114073\n",
            "Validation loss:  [[0.04014179]]\n",
            "------ Epoch:  798\n",
            "Train loss:  0.010815423\n",
            "Validation loss:  [[0.02858995]]\n",
            "------ Epoch:  799\n",
            "Train loss:  0.011088767\n",
            "Validation loss:  [[0.04006349]]\n",
            "------ Epoch:  800\n",
            "Train loss:  0.010792049\n",
            "Validation loss:  [[0.02855374]]\n",
            "------ Epoch:  801\n",
            "Train loss:  0.011063695\n",
            "Validation loss:  [[0.03998541]]\n",
            "------ Epoch:  802\n",
            "Train loss:  0.010768938\n",
            "Validation loss:  [[0.02851742]]\n",
            "------ Epoch:  803\n",
            "Train loss:  0.0110387625\n",
            "Validation loss:  [[0.03990769]]\n",
            "------ Epoch:  804\n",
            "Train loss:  0.010745953\n",
            "Validation loss:  [[0.02848128]]\n",
            "------ Epoch:  805\n",
            "Train loss:  0.011014135\n",
            "Validation loss:  [[0.03983069]]\n",
            "------ Epoch:  806\n",
            "Train loss:  0.010723235\n",
            "Validation loss:  [[0.02844536]]\n",
            "------ Epoch:  807\n",
            "Train loss:  0.010989721\n",
            "Validation loss:  [[0.03975392]]\n",
            "------ Epoch:  808\n",
            "Train loss:  0.010700663\n",
            "Validation loss:  [[0.02840958]]\n",
            "------ Epoch:  809\n",
            "Train loss:  0.010965609\n",
            "Validation loss:  [[0.03967799]]\n",
            "------ Epoch:  810\n",
            "Train loss:  0.01067848\n",
            "Validation loss:  [[0.02837411]]\n",
            "------ Epoch:  811\n",
            "Train loss:  0.010941963\n",
            "Validation loss:  [[0.03960242]]\n",
            "------ Epoch:  812\n",
            "Train loss:  0.010656539\n",
            "Validation loss:  [[0.02833866]]\n",
            "------ Epoch:  813\n",
            "Train loss:  0.0109184\n",
            "Validation loss:  [[0.03952734]]\n",
            "------ Epoch:  814\n",
            "Train loss:  0.0106347455\n",
            "Validation loss:  [[0.02830327]]\n",
            "------ Epoch:  815\n",
            "Train loss:  0.010895011\n",
            "Validation loss:  [[0.03945275]]\n",
            "------ Epoch:  816\n",
            "Train loss:  0.010613221\n",
            "Validation loss:  [[0.02826811]]\n",
            "------ Epoch:  817\n",
            "Train loss:  0.010871877\n",
            "Validation loss:  [[0.03937837]]\n",
            "------ Epoch:  818\n",
            "Train loss:  0.010591684\n",
            "Validation loss:  [[0.02823292]]\n",
            "------ Epoch:  819\n",
            "Train loss:  0.010848876\n",
            "Validation loss:  [[0.03930444]]\n",
            "------ Epoch:  820\n",
            "Train loss:  0.010570453\n",
            "Validation loss:  [[0.02819799]]\n",
            "------ Epoch:  821\n",
            "Train loss:  0.010826234\n",
            "Validation loss:  [[0.03923113]]\n",
            "------ Epoch:  822\n",
            "Train loss:  0.010549542\n",
            "Validation loss:  [[0.02816315]]\n",
            "------ Epoch:  823\n",
            "Train loss:  0.010803779\n",
            "Validation loss:  [[0.03915807]]\n",
            "------ Epoch:  824\n",
            "Train loss:  0.010528713\n",
            "Validation loss:  [[0.02812846]]\n",
            "------ Epoch:  825\n",
            "Train loss:  0.010781518\n",
            "Validation loss:  [[0.03908537]]\n",
            "------ Epoch:  826\n",
            "Train loss:  0.010507979\n",
            "Validation loss:  [[0.02809371]]\n",
            "------ Epoch:  827\n",
            "Train loss:  0.010759306\n",
            "Validation loss:  [[0.03901294]]\n",
            "------ Epoch:  828\n",
            "Train loss:  0.010487484\n",
            "Validation loss:  [[0.02805916]]\n",
            "------ Epoch:  829\n",
            "Train loss:  0.010737465\n",
            "Validation loss:  [[0.03894143]]\n",
            "------ Epoch:  830\n",
            "Train loss:  0.01046726\n",
            "Validation loss:  [[0.02802485]]\n",
            "------ Epoch:  831\n",
            "Train loss:  0.010715741\n",
            "Validation loss:  [[0.03886977]]\n",
            "------ Epoch:  832\n",
            "Train loss:  0.010447115\n",
            "Validation loss:  [[0.02799044]]\n",
            "------ Epoch:  833\n",
            "Train loss:  0.010694178\n",
            "Validation loss:  [[0.03879841]]\n",
            "------ Epoch:  834\n",
            "Train loss:  0.010427074\n",
            "Validation loss:  [[0.0279562]]\n",
            "------ Epoch:  835\n",
            "Train loss:  0.010672776\n",
            "Validation loss:  [[0.03872763]]\n",
            "------ Epoch:  836\n",
            "Train loss:  0.010407272\n",
            "Validation loss:  [[0.02792206]]\n",
            "------ Epoch:  837\n",
            "Train loss:  0.010651608\n",
            "Validation loss:  [[0.03865715]]\n",
            "------ Epoch:  838\n",
            "Train loss:  0.010387554\n",
            "Validation loss:  [[0.02788801]]\n",
            "------ Epoch:  839\n",
            "Train loss:  0.010630622\n",
            "Validation loss:  [[0.03858714]]\n",
            "------ Epoch:  840\n",
            "Train loss:  0.010368087\n",
            "Validation loss:  [[0.02785407]]\n",
            "------ Epoch:  841\n",
            "Train loss:  0.010609759\n",
            "Validation loss:  [[0.03851718]]\n",
            "------ Epoch:  842\n",
            "Train loss:  0.010348699\n",
            "Validation loss:  [[0.02782024]]\n",
            "------ Epoch:  843\n",
            "Train loss:  0.010589211\n",
            "Validation loss:  [[0.03844795]]\n",
            "------ Epoch:  844\n",
            "Train loss:  0.010329585\n",
            "Validation loss:  [[0.02778646]]\n",
            "------ Epoch:  845\n",
            "Train loss:  0.010568676\n",
            "Validation loss:  [[0.0383789]]\n",
            "------ Epoch:  846\n",
            "Train loss:  0.010310573\n",
            "Validation loss:  [[0.02775279]]\n",
            "------ Epoch:  847\n",
            "Train loss:  0.010548411\n",
            "Validation loss:  [[0.03831042]]\n",
            "------ Epoch:  848\n",
            "Train loss:  0.010291878\n",
            "Validation loss:  [[0.02771946]]\n",
            "------ Epoch:  849\n",
            "Train loss:  0.010528541\n",
            "Validation loss:  [[0.03824237]]\n",
            "------ Epoch:  850\n",
            "Train loss:  0.010273337\n",
            "Validation loss:  [[0.02768612]]\n",
            "------ Epoch:  851\n",
            "Train loss:  0.010508683\n",
            "Validation loss:  [[0.03817472]]\n",
            "------ Epoch:  852\n",
            "Train loss:  0.010254937\n",
            "Validation loss:  [[0.02765294]]\n",
            "------ Epoch:  853\n",
            "Train loss:  0.010489087\n",
            "Validation loss:  [[0.03810736]]\n",
            "------ Epoch:  854\n",
            "Train loss:  0.010236727\n",
            "Validation loss:  [[0.02761979]]\n",
            "------ Epoch:  855\n",
            "Train loss:  0.010469673\n",
            "Validation loss:  [[0.0380404]]\n",
            "------ Epoch:  856\n",
            "Train loss:  0.010218679\n",
            "Validation loss:  [[0.02758674]]\n",
            "------ Epoch:  857\n",
            "Train loss:  0.010450504\n",
            "Validation loss:  [[0.03797365]]\n",
            "------ Epoch:  858\n",
            "Train loss:  0.010200784\n",
            "Validation loss:  [[0.02755384]]\n",
            "------ Epoch:  859\n",
            "Train loss:  0.010431388\n",
            "Validation loss:  [[0.0379076]]\n",
            "------ Epoch:  860\n",
            "Train loss:  0.010182986\n",
            "Validation loss:  [[0.02752098]]\n",
            "------ Epoch:  861\n",
            "Train loss:  0.010412363\n",
            "Validation loss:  [[0.03784141]]\n",
            "------ Epoch:  862\n",
            "Train loss:  0.0101653235\n",
            "Validation loss:  [[0.02748817]]\n",
            "------ Epoch:  863\n",
            "Train loss:  0.010393553\n",
            "Validation loss:  [[0.03777573]]\n",
            "------ Epoch:  864\n",
            "Train loss:  0.010147812\n",
            "Validation loss:  [[0.02745562]]\n",
            "------ Epoch:  865\n",
            "Train loss:  0.010374955\n",
            "Validation loss:  [[0.03771018]]\n",
            "------ Epoch:  866\n",
            "Train loss:  0.010130409\n",
            "Validation loss:  [[0.02742291]]\n",
            "------ Epoch:  867\n",
            "Train loss:  0.010356438\n",
            "Validation loss:  [[0.03764494]]\n",
            "------ Epoch:  868\n",
            "Train loss:  0.01011314\n",
            "Validation loss:  [[0.02739041]]\n",
            "------ Epoch:  869\n",
            "Train loss:  0.010338029\n",
            "Validation loss:  [[0.03758028]]\n",
            "------ Epoch:  870\n",
            "Train loss:  0.010096142\n",
            "Validation loss:  [[0.02735802]]\n",
            "------ Epoch:  871\n",
            "Train loss:  0.010319919\n",
            "Validation loss:  [[0.03751587]]\n",
            "------ Epoch:  872\n",
            "Train loss:  0.010079266\n",
            "Validation loss:  [[0.0273257]]\n",
            "------ Epoch:  873\n",
            "Train loss:  0.010301997\n",
            "Validation loss:  [[0.03745192]]\n",
            "------ Epoch:  874\n",
            "Train loss:  0.010062536\n",
            "Validation loss:  [[0.02729356]]\n",
            "------ Epoch:  875\n",
            "Train loss:  0.010284253\n",
            "Validation loss:  [[0.03738808]]\n",
            "------ Epoch:  876\n",
            "Train loss:  0.010045896\n",
            "Validation loss:  [[0.02726141]]\n",
            "------ Epoch:  877\n",
            "Train loss:  0.010266497\n",
            "Validation loss:  [[0.0373247]]\n",
            "------ Epoch:  878\n",
            "Train loss:  0.010029416\n",
            "Validation loss:  [[0.02722942]]\n",
            "------ Epoch:  879\n",
            "Train loss:  0.010248994\n",
            "Validation loss:  [[0.03726169]]\n",
            "------ Epoch:  880\n",
            "Train loss:  0.010013181\n",
            "Validation loss:  [[0.02719759]]\n",
            "------ Epoch:  881\n",
            "Train loss:  0.010231869\n",
            "Validation loss:  [[0.0371995]]\n",
            "------ Epoch:  882\n",
            "Train loss:  0.009997205\n",
            "Validation loss:  [[0.02716603]]\n",
            "------ Epoch:  883\n",
            "Train loss:  0.010214772\n",
            "Validation loss:  [[0.03713709]]\n",
            "------ Epoch:  884\n",
            "Train loss:  0.009981233\n",
            "Validation loss:  [[0.02713435]]\n",
            "------ Epoch:  885\n",
            "Train loss:  0.010197874\n",
            "Validation loss:  [[0.03707532]]\n",
            "------ Epoch:  886\n",
            "Train loss:  0.009965481\n",
            "Validation loss:  [[0.02710279]]\n",
            "------ Epoch:  887\n",
            "Train loss:  0.010181116\n",
            "Validation loss:  [[0.03701375]]\n",
            "------ Epoch:  888\n",
            "Train loss:  0.00994979\n",
            "Validation loss:  [[0.02707145]]\n",
            "------ Epoch:  889\n",
            "Train loss:  0.010164552\n",
            "Validation loss:  [[0.03695261]]\n",
            "------ Epoch:  890\n",
            "Train loss:  0.0099343425\n",
            "Validation loss:  [[0.02704009]]\n",
            "------ Epoch:  891\n",
            "Train loss:  0.01014807\n",
            "Validation loss:  [[0.03689135]]\n",
            "------ Epoch:  892\n",
            "Train loss:  0.009918926\n",
            "Validation loss:  [[0.02700875]]\n",
            "------ Epoch:  893\n",
            "Train loss:  0.010131758\n",
            "Validation loss:  [[0.03683077]]\n",
            "------ Epoch:  894\n",
            "Train loss:  0.009903673\n",
            "Validation loss:  [[0.02697767]]\n",
            "------ Epoch:  895\n",
            "Train loss:  0.010115597\n",
            "Validation loss:  [[0.0367706]]\n",
            "------ Epoch:  896\n",
            "Train loss:  0.009888663\n",
            "Validation loss:  [[0.02694665]]\n",
            "------ Epoch:  897\n",
            "Train loss:  0.010099647\n",
            "Validation loss:  [[0.03671056]]\n",
            "------ Epoch:  898\n",
            "Train loss:  0.009873742\n",
            "Validation loss:  [[0.02691568]]\n",
            "------ Epoch:  899\n",
            "Train loss:  0.0100839\n",
            "Validation loss:  [[0.03665109]]\n",
            "------ Epoch:  900\n",
            "Train loss:  0.009859027\n",
            "Validation loss:  [[0.02688492]]\n",
            "------ Epoch:  901\n",
            "Train loss:  0.010068313\n",
            "Validation loss:  [[0.03659201]]\n",
            "------ Epoch:  902\n",
            "Train loss:  0.009844464\n",
            "Validation loss:  [[0.02685423]]\n",
            "------ Epoch:  903\n",
            "Train loss:  0.01005286\n",
            "Validation loss:  [[0.03653293]]\n",
            "------ Epoch:  904\n",
            "Train loss:  0.009829966\n",
            "Validation loss:  [[0.0268235]]\n",
            "------ Epoch:  905\n",
            "Train loss:  0.01003749\n",
            "Validation loss:  [[0.03647442]]\n",
            "------ Epoch:  906\n",
            "Train loss:  0.009815653\n",
            "Validation loss:  [[0.02679301]]\n",
            "------ Epoch:  907\n",
            "Train loss:  0.010022338\n",
            "Validation loss:  [[0.03641598]]\n",
            "------ Epoch:  908\n",
            "Train loss:  0.0098014325\n",
            "Validation loss:  [[0.02676262]]\n",
            "------ Epoch:  909\n",
            "Train loss:  0.010007241\n",
            "Validation loss:  [[0.03635778]]\n",
            "------ Epoch:  910\n",
            "Train loss:  0.009787283\n",
            "Validation loss:  [[0.02673227]]\n",
            "------ Epoch:  911\n",
            "Train loss:  0.009992333\n",
            "Validation loss:  [[0.03630029]]\n",
            "------ Epoch:  912\n",
            "Train loss:  0.009773394\n",
            "Validation loss:  [[0.02670204]]\n",
            "------ Epoch:  913\n",
            "Train loss:  0.009977558\n",
            "Validation loss:  [[0.03624277]]\n",
            "------ Epoch:  914\n",
            "Train loss:  0.009759554\n",
            "Validation loss:  [[0.02667177]]\n",
            "------ Epoch:  915\n",
            "Train loss:  0.009962914\n",
            "Validation loss:  [[0.03618567]]\n",
            "------ Epoch:  916\n",
            "Train loss:  0.009745744\n",
            "Validation loss:  [[0.02664166]]\n",
            "------ Epoch:  917\n",
            "Train loss:  0.009948339\n",
            "Validation loss:  [[0.03612852]]\n",
            "------ Epoch:  918\n",
            "Train loss:  0.009732112\n",
            "Validation loss:  [[0.02661159]]\n",
            "------ Epoch:  919\n",
            "Train loss:  0.009933787\n",
            "Validation loss:  [[0.03607153]]\n",
            "------ Epoch:  920\n",
            "Train loss:  0.009718496\n",
            "Validation loss:  [[0.02658167]]\n",
            "------ Epoch:  921\n",
            "Train loss:  0.0099195\n",
            "Validation loss:  [[0.036015]]\n",
            "------ Epoch:  922\n",
            "Train loss:  0.009705084\n",
            "Validation loss:  [[0.02655174]]\n",
            "------ Epoch:  923\n",
            "Train loss:  0.009905283\n",
            "Validation loss:  [[0.03595876]]\n",
            "------ Epoch:  924\n",
            "Train loss:  0.009691735\n",
            "Validation loss:  [[0.02652196]]\n",
            "------ Epoch:  925\n",
            "Train loss:  0.009891113\n",
            "Validation loss:  [[0.03590255]]\n",
            "------ Epoch:  926\n",
            "Train loss:  0.009678337\n",
            "Validation loss:  [[0.02649201]]\n",
            "------ Epoch:  927\n",
            "Train loss:  0.009876975\n",
            "Validation loss:  [[0.03584664]]\n",
            "------ Epoch:  928\n",
            "Train loss:  0.009665167\n",
            "Validation loss:  [[0.02646235]]\n",
            "------ Epoch:  929\n",
            "Train loss:  0.0098630665\n",
            "Validation loss:  [[0.03579096]]\n",
            "------ Epoch:  930\n",
            "Train loss:  0.009652044\n",
            "Validation loss:  [[0.02643271]]\n",
            "------ Epoch:  931\n",
            "Train loss:  0.009849239\n",
            "Validation loss:  [[0.03573574]]\n",
            "------ Epoch:  932\n",
            "Train loss:  0.009639145\n",
            "Validation loss:  [[0.02640326]]\n",
            "------ Epoch:  933\n",
            "Train loss:  0.009835641\n",
            "Validation loss:  [[0.03568065]]\n",
            "------ Epoch:  934\n",
            "Train loss:  0.009626282\n",
            "Validation loss:  [[0.02637378]]\n",
            "------ Epoch:  935\n",
            "Train loss:  0.00982192\n",
            "Validation loss:  [[0.03562572]]\n",
            "------ Epoch:  936\n",
            "Train loss:  0.009613506\n",
            "Validation loss:  [[0.02634436]]\n",
            "------ Epoch:  937\n",
            "Train loss:  0.00980856\n",
            "Validation loss:  [[0.03557117]]\n",
            "------ Epoch:  938\n",
            "Train loss:  0.009600862\n",
            "Validation loss:  [[0.02631508]]\n",
            "------ Epoch:  939\n",
            "Train loss:  0.009795145\n",
            "Validation loss:  [[0.035517]]\n",
            "------ Epoch:  940\n",
            "Train loss:  0.009588392\n",
            "Validation loss:  [[0.02628594]]\n",
            "------ Epoch:  941\n",
            "Train loss:  0.009781935\n",
            "Validation loss:  [[0.03546282]]\n",
            "------ Epoch:  942\n",
            "Train loss:  0.009575907\n",
            "Validation loss:  [[0.02625676]]\n",
            "------ Epoch:  943\n",
            "Train loss:  0.00976878\n",
            "Validation loss:  [[0.03540891]]\n",
            "------ Epoch:  944\n",
            "Train loss:  0.009563517\n",
            "Validation loss:  [[0.02622772]]\n",
            "------ Epoch:  945\n",
            "Train loss:  0.009755663\n",
            "Validation loss:  [[0.03535531]]\n",
            "------ Epoch:  946\n",
            "Train loss:  0.009551292\n",
            "Validation loss:  [[0.02619869]]\n",
            "------ Epoch:  947\n",
            "Train loss:  0.00974277\n",
            "Validation loss:  [[0.03530179]]\n",
            "------ Epoch:  948\n",
            "Train loss:  0.009539083\n",
            "Validation loss:  [[0.02616983]]\n",
            "------ Epoch:  949\n",
            "Train loss:  0.0097299805\n",
            "Validation loss:  [[0.03524876]]\n",
            "------ Epoch:  950\n",
            "Train loss:  0.009527093\n",
            "Validation loss:  [[0.0261411]]\n",
            "------ Epoch:  951\n",
            "Train loss:  0.009717287\n",
            "Validation loss:  [[0.03519585]]\n",
            "------ Epoch:  952\n",
            "Train loss:  0.009515103\n",
            "Validation loss:  [[0.02611235]]\n",
            "------ Epoch:  953\n",
            "Train loss:  0.009704713\n",
            "Validation loss:  [[0.03514323]]\n",
            "------ Epoch:  954\n",
            "Train loss:  0.009503186\n",
            "Validation loss:  [[0.02608358]]\n",
            "------ Epoch:  955\n",
            "Train loss:  0.0096921045\n",
            "Validation loss:  [[0.03509085]]\n",
            "------ Epoch:  956\n",
            "Train loss:  0.009491507\n",
            "Validation loss:  [[0.02605513]]\n",
            "------ Epoch:  957\n",
            "Train loss:  0.009679816\n",
            "Validation loss:  [[0.03503872]]\n",
            "------ Epoch:  958\n",
            "Train loss:  0.00947987\n",
            "Validation loss:  [[0.02602662]]\n",
            "------ Epoch:  959\n",
            "Train loss:  0.009667595\n",
            "Validation loss:  [[0.03498686]]\n",
            "------ Epoch:  960\n",
            "Train loss:  0.009468357\n",
            "Validation loss:  [[0.02599826]]\n",
            "------ Epoch:  961\n",
            "Train loss:  0.009655427\n",
            "Validation loss:  [[0.03493521]]\n",
            "------ Epoch:  962\n",
            "Train loss:  0.009456913\n",
            "Validation loss:  [[0.02596997]]\n",
            "------ Epoch:  963\n",
            "Train loss:  0.009643453\n",
            "Validation loss:  [[0.0348839]]\n",
            "------ Epoch:  964\n",
            "Train loss:  0.009445597\n",
            "Validation loss:  [[0.02594173]]\n",
            "------ Epoch:  965\n",
            "Train loss:  0.009631511\n",
            "Validation loss:  [[0.03483274]]\n",
            "------ Epoch:  966\n",
            "Train loss:  0.009434334\n",
            "Validation loss:  [[0.02591361]]\n",
            "------ Epoch:  967\n",
            "Train loss:  0.009619647\n",
            "Validation loss:  [[0.03478188]]\n",
            "------ Epoch:  968\n",
            "Train loss:  0.009423298\n",
            "Validation loss:  [[0.02588569]]\n",
            "------ Epoch:  969\n",
            "Train loss:  0.0096081095\n",
            "Validation loss:  [[0.03473158]]\n",
            "------ Epoch:  970\n",
            "Train loss:  0.009412363\n",
            "Validation loss:  [[0.02585774]]\n",
            "------ Epoch:  971\n",
            "Train loss:  0.009596453\n",
            "Validation loss:  [[0.03468112]]\n",
            "------ Epoch:  972\n",
            "Train loss:  0.009401385\n",
            "Validation loss:  [[0.02582983]]\n",
            "------ Epoch:  973\n",
            "Train loss:  0.009584993\n",
            "Validation loss:  [[0.03463084]]\n",
            "------ Epoch:  974\n",
            "Train loss:  0.0093904855\n",
            "Validation loss:  [[0.02580197]]\n",
            "------ Epoch:  975\n",
            "Train loss:  0.009573529\n",
            "Validation loss:  [[0.0345809]]\n",
            "------ Epoch:  976\n",
            "Train loss:  0.0093797585\n",
            "Validation loss:  [[0.02577433]]\n",
            "------ Epoch:  977\n",
            "Train loss:  0.009562224\n",
            "Validation loss:  [[0.03453127]]\n",
            "------ Epoch:  978\n",
            "Train loss:  0.009369145\n",
            "Validation loss:  [[0.02574672]]\n",
            "------ Epoch:  979\n",
            "Train loss:  0.009551126\n",
            "Validation loss:  [[0.03448194]]\n",
            "------ Epoch:  980\n",
            "Train loss:  0.009358614\n",
            "Validation loss:  [[0.02571918]]\n",
            "------ Epoch:  981\n",
            "Train loss:  0.009539994\n",
            "Validation loss:  [[0.03443256]]\n",
            "------ Epoch:  982\n",
            "Train loss:  0.00934803\n",
            "Validation loss:  [[0.02569159]]\n",
            "------ Epoch:  983\n",
            "Train loss:  0.009528827\n",
            "Validation loss:  [[0.03438334]]\n",
            "------ Epoch:  984\n",
            "Train loss:  0.009337558\n",
            "Validation loss:  [[0.02566423]]\n",
            "------ Epoch:  985\n",
            "Train loss:  0.009517909\n",
            "Validation loss:  [[0.03433436]]\n",
            "------ Epoch:  986\n",
            "Train loss:  0.009327177\n",
            "Validation loss:  [[0.02563689]]\n",
            "------ Epoch:  987\n",
            "Train loss:  0.009506946\n",
            "Validation loss:  [[0.03428553]]\n",
            "------ Epoch:  988\n",
            "Train loss:  0.009316895\n",
            "Validation loss:  [[0.02560963]]\n",
            "------ Epoch:  989\n",
            "Train loss:  0.009496168\n",
            "Validation loss:  [[0.03423718]]\n",
            "------ Epoch:  990\n",
            "Train loss:  0.009306645\n",
            "Validation loss:  [[0.02558237]]\n",
            "------ Epoch:  991\n",
            "Train loss:  0.009485387\n",
            "Validation loss:  [[0.03418866]]\n",
            "------ Epoch:  992\n",
            "Train loss:  0.009296439\n",
            "Validation loss:  [[0.02555535]]\n",
            "------ Epoch:  993\n",
            "Train loss:  0.009474835\n",
            "Validation loss:  [[0.03414098]]\n",
            "------ Epoch:  994\n",
            "Train loss:  0.009286541\n",
            "Validation loss:  [[0.02552835]]\n",
            "------ Epoch:  995\n",
            "Train loss:  0.009464344\n",
            "Validation loss:  [[0.03409304]]\n",
            "------ Epoch:  996\n",
            "Train loss:  0.009276608\n",
            "Validation loss:  [[0.02550142]]\n",
            "------ Epoch:  997\n",
            "Train loss:  0.009453931\n",
            "Validation loss:  [[0.03404555]]\n",
            "------ Epoch:  998\n",
            "Train loss:  0.009266757\n",
            "Validation loss:  [[0.02547452]]\n",
            "------ Epoch:  999\n",
            "Train loss:  0.009443646\n",
            "Validation loss:  [[0.03399832]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "gkaUDmA8Ps6w"
      },
      "source": [
        "Predicting with this model shows overfitting. For recognizing overfitting a comparison of the validation and training loss is very useful. If the training loss decreases during training while the validation loss consistently increases, the model you are training is probably overfitting. Plotting the models prediction and the target also shows that there is a significant discrepancy between the target and the prediction of the model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "3Sq-9xhWPxI4",
        "outputId": "f6e78562-420d-4ab4-df11-c16990ef17b2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        }
      },
      "source": [
        "y_pred =big_mdl.__call__(x) # Predict on x with \"big_mdl\"\n",
        "plt.scatter(x_train_overfit, y_train_overfit)\n",
        "plt.plot(x, y_true,'r.')\n",
        "plt.plot(x, y_pred.numpy(),'g.')\n",
        "plt.legend([\"Target\", \"Prediction\", \"Training samples\"])\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2deXxU1d3/32eyQFgEBSwaUNAqgiIJIOQSwFSsqCAg6NOqPNFWG8D2J1VLAF/tU9unLUmo1taWJSpqCk/VVgwBpOCWhmUQWYILuwgmURSC7MGQzPn9cWcmc2cJWSaz5ft+vfJizpkz957LzHzm3O/5LkprjSAIghD72MI9AUEQBCE0iOALgiC0EkTwBUEQWgki+IIgCK0EEXxBEIRWQny4JxCIrl276l69eoV7GoIgCFHFli1bjmitu/l7LmIFv1evXmzevDnc0xAEQYgqlFIHAz0nJh1BEIRWggi+IAhCK0EEXxAEoZUggi8IgtBKEMEXBEFoJYjgC4IgtBJE8CMce5mdO1+5k6HPDSV/S364pyMIQhQTsX74AuRvyWfqiqlozBTWm77YxOs7Xmf1f68O88wEQYhGZIUfodjL7Baxd7Fm/xomL50cplkJghDNiOBHKHnr83zE3sWSj5ZgL7OHeEaCIEQ7IvgRiL3MzrLdy+odk7c+L0SzEQQhVhDBj0D8re7bJ7S3tIv2FMkqXxCERiGCH2HYy+wU7Smy9NmUjadHP41N1b1dDu2gYHtBqKcnCEIUI4IfYcx6ZxYO7bD0zR8zn6xBWQzvOdzSv+PwjlBOTRCEKEcEP4Kwl9kpOVhi6evbtS9Zg7IA6Netn+W59WXrxawjCEKDEcGPIPxtxP487efux5kDMolTce621priA8WhmJogCDGACH6E4M8zJ6V7int1D2D0NHh82OPutgMHXdp1CdkcBUGIbkTwIwR/njlpyWk+406cPWFpr9q3qkXnJQhC7CCCHwH4W93blI3MAZnnfe3y3cvFji8IQoMQwY8AHn7zYZ/V/fwx8zF6Gj5jve34Du0QO74gCA1CBD/MTF46mdJDpZa+fl37WWz3nnjb8TWaY98ea9E5CkKsULitgvScd+k9ayXpOe9SuK0i3FMKKSL4YWTm2zNZ8tESn/7padPrfV3nNp1RKHf7qQ1PiVlHEM5D4bYKZi/9iIpjVWig4lgVs5d+1KpEXwQ/TORvyffrhnlf//sCru5dZPTKsETd1upaiboVhPMwd/Vuqs7VWvqqztUyd/XuMM0o9Ijgh4E7F/+UKSum4J0MM+U7KSyeuPi8rzd6GqT3TLf0SdStINTPF8eqGtUfi4jgh5gbn59E4b55pti7rDJO4Z83Zl6Dj+MddbuubJ2YdQShHi7tnNSo/lhEBD+EzHx7JiUVS82Gl9hfavuBX6+cQGQOyPRJpuZpIsrfkk/vZ3rT7vftuCj3Ima+PbO50xeEqGbG6D4kJcRZ+pIS4pgxuk+YZhR6lNb+i2yEm8GDB+vNmzeHexpBI39Lfp0Zx0vsO9ZMokvNj/gsZ0yjjnnjizdS8rk19067+HZUO6qpcdT4jE+KS+L/pf0/cm/ObcIVCEL0U7itgrmrd/PFsSou7ZzEjNF9mJCaHO5pBRWl1Bat9WB/z0lN2xBgL7MzbeU0s+El9hed+ykda29r0m1lv279fAT/TM2ZgOOraqvIW5/HGzve4OU7X27UHYUgxAITUpNjTuAbg5h0QoBPymMvsW/qbWXmgExsTXgL936zl2GLhpG/Jb/RrxUEIXoJygpfKbUIGAt8rbW+zs/zCvgzcDtwBnhAa701GOeOVFy3jvtPbOVQG+sqHAVXxD2K4+yoZt1WGj0N5o+db5qKAqBQAWvjul53PjdQQYglZr49k0XbFlFdU805xznQEB8XT5yK4/ru15MzKidm736DYsNXSo0ETgEFAQT/duD/YQr+UODPWuuh9R0zmm34rgCPqnO1fJ3wO6riNuIRJ0V2enZQ7ej2MjsPFT7K7qOfUOtwoGw1tItvy0+HTiX3LcjfOI8nhp6isp3zBR5mJQU8uzKOn+xsR6LSoBTUOO3/8fFQ6/Rb7tQJhg6F7Gwwmv9lyN+Sz5x1c/jq5FcAfKfDd5g9Yrb8+Agthr3Mzt0v3k6FI0Bkusd39L7+9zXIRToSqc+GH7RNW6VUL2BFAMFfCBRrrf/hbO8GMrTWXwY6XjQLfnrOu1Qcq+Jb204OJc4EHO4P08jLR/KfB/4T1PMVbqvg1b+8xv3rXyOlYjcdvz2NUpCIJr7mnHvc5AmwZICz4bmXoGHhCvjJVstnPjCJidCmDaSmQk5Og34A8rfk88zGZ/jq1Fec+PYENdp3UxnMTedeF/Zi+tDpIv5C0Jj50mTyDnhEtTfggz7k0iG8/5P3W25SLUR9gh8qG34yUObRLnf2WVBKZSmlNiulNh8+fDhEUws+rkCOU3Hv4hZ7DWgbOaNygnOS/Hzo3RvatePWG3qx+KXHGL13I9858w3taqtJqqkmzkPsARYXwn3bnQ3X77wy/6bcAc8NbOC5q6vh5EkoKYFhw6BdO7jxRrD7xgHMfHsmHf/QkSkrprDzyE6Onj0aUOzB3HTecXgHU1ZMYfTfRzdwQoIQmMkLR9eJvfPz7lro+Px5sOmLTVz9l6tjKr4lojZttdb5WuvBWuvB3bp1C/d0mozL46aazy39HdW1TbcNzpwJF19smlYSE2HKFDhwAKqqaFNbg426z7LnnzeLCyF7HdYPuHPglLHmXUCjqaqqE/9LLiH/T5Pp97d+JP0uibz1eZw6d6oJB4U1+9fE3BdOCC2TF45myZdrzIaXh1xiDSRVm3/tqvEr/Hu/2cvwRcNj5jMYKsGvAHp6tHs4+4JO4bYKrvzdYyT9z0Cu/N1jYUmMNGN0H0jYQ3XcTkv/sMtTGn4QjxU88fGQlweHD8OJE3DOunL3FPeAC5aEBOjQATp25MniRB7dYKt7gcdBlgyALjMgf2g8dOxYd/6GTHkgdJl8iCnHl7Dz8E7O1p4972viVBxxxAV8Pta+cELomPnSZP9ir8073W+fac+ZpxI5MwdOz4ENi6DHMY9xThw4eHjlwyGcecsRKsEvAjKVSRpwvD77fVMp3FbBtKV57K/5E2dt29hf8yceXPpoyEV/Qmoy11y5DVSd7V4pG78eNS3wi/Lz4dproUsXU5w9VvDujdMAeAt8tYqjKj6R6nYdoFs3c6PVZYY5cYLVH+xnVdpK2tVm1B0A3KJ/tD1Mua2Gq3/fHfuut80fmIULoW9fzna6kLNx8TicL5s5Ci7+BbR9wjQLHW3vcawAJNoS6dy2M9np2dT8Tw01v64hOz2bzm07+x3vwMG9/3yo3v8DQfDEXma3mnFcXxAHLNyezOLsDXDqFHz7rfnZBoxyKPsz3LLXeRAP0S/9qjQmotWD5aXzDyAD6Ap8BfwaSADQWi9wumX+FbgV0y3zR1rrendkm7Jpm57zLpvPTKPatqfuTQYujfsBFb96pVHHai7eUbAjLxvJf37ksVlrt8OsWbB1q/mh81q1N5i4OEhM5NvEtlQkXkD+wLGszZh4XldPl9voR6fmcTLh9XpPkRiXSNu4tnxbW0P1OdDUYHM4sFFDjffi3FPoPT5a7Wpt3HLdOLKHZddr1rKX2Zn4yr0cOn2g7njO40z47sO8Mflv9c5VEABufCaVkmOllpV995Ow9PgtGM+v9n2B3Q4TJ8KhQ0AABwdg4diFEe9MEBIvnWDTFMHvPWslX3m7QTovb+EdoXuj7GV2hi8ajoO6YKupPSYwfwWwcSMcPWquuJtC27am62SbNpCVBbnNd++0l9nJfCOTfd/sO/9gfx+XACKPhs5nIWsL5L6Deffyhz+Y866H9Jx3KT39v5yJK647vjMlRTR84YTwkr8lnylFUywbtErD+ooAYu/Cbjf3oZykZkHpJVg+3zZsrPvxuoj2048EL52QcGnnJDrVTgKUj5lizto5DT5O/pZ8rp13Lb2e6cWdr94Z0H4cqHpO3vq8OrHXYKuFzCcLobDQXEE0Ruzj4iApCXr1Mm89q6rgzBn45pugiD2YAVx7H9lLdnr2+Qf72xn2siklnoORB0yb6Dd5TrEHqKw0TVWpqX49elx8cayKbud+QYKjt/W8GqatnCb2fCEg9jI705ZbxR5gxscd6xd7MN2LJ9R5Lcx707TKetvz8zb41rGIFmJK8GeM7kPnuOu46Jxzg8XjjTpw/MB5bXD2MjtXP3s1U1ZMYcfhHRw8fpDCXYUMWzTM57Xe1XNGFC/l2lsM/tP3Aop2FlrOPW63aR9sMG3aQOfOpu29psYU+M8+O+/KuLnk3pzLhh9vYMB3BgQeFMiVTcdh0524qPZu1ly1jP98NjLwNZeWQnq6uW/hB5eXU5eah/H+8XZoB7PemdXYSxNaCXnLZ5lLLY9V+cgDkDv6jw07QHY22ExZNMphwUrfIYW7CqM2LUlMmXSgzja96+RSjib62nsDmQRmvj3TbwUqNxq62kbx3B0vM+Hs5xQ/OINrDu6g47enide1JDrMjdVpY2DhYNyrC5uGdYvOI/iJiXDRRZCWFrRI1uZiL7Mz6+1ZbDu0jVpHLfG2eIsNH0DpeJRKoIttNO2rMn3TRNjtcP/9sHdv4BNlZ/vcqXhGKp+MW8XRBOf7GGW2VCG02MvspD8/DO2xurdpWHc+U443+fnmnaiTO/8LCvsSNaadVmPD98YnfbDTDjz1+lzm31lnvpi8dLLf2rIWnP9NP/hQ8Y83/P+fKUzBX+Ah+BN2whuveQ2MjzfNNI2IVI0UmpReduZM0600ELfcAqutX0jP81S3/ztfOF61PB/JXzghPNw570YKvy6x7N9N+OIC3sg/3viD3XijGVsC2HvA8B+Dw8seMqHPBN744RvNm3QL0GoF315mZ9iidNDaZxN3ZI+J3Jc6midWPkal47TZ6TXGjVd/9joPu7QXlt19YOFyyPow3gyYuvhi+PnPW9w0E2yCkkPcboeHHzbNOf4YMgTeDxzGfuNLN1Jy0JqEbuqgqcwfO79x8xBiEnuZneEvDKsz57hW999MwPhLE0TZawM3f6AZmOhpBFco1v94fcQtOlrNpq2bmTOhWzeMa0fz7Aqb36jSkvKlTFk+xb/YaxjwpTMNgaet2jkmb7j5AfA2ZecPtLpyKQ2V900wXS6PHIEdO6JS7D33KiqOVTF76UeNj20wDNi2De67z//zmzbB6MCpFHJG5aC8nPuljq/gIm9Dno/tftwuMO5pgCOCPwwD+tWVEc3aChN2WYdoNAXbC5p2/DARe4LvMh8cOQInT/LTzbVkr3c+5yX67sfegRkroDTfTEOwYZHpv2sZj/lr/9xAOGeL40xcIkc6XsQfb2pvDXu1KTLubuIHLkKYu3o3VeesgV9V52qZu3p30w64eLHpbdSli+9za9YEFH2jp8H4a8Zb+qSOrwDm6r5o17K6Dg02B2THj2yeuXT6dEsze4PTa8eDjeUbm378MBB7gr90qaWpMM0vlvwxrtW6l6nmqiOw4UXz19zVnVYOr79m3h76JBwbB1sOrKV9zbe88d4c9nY4bTn3iMtGRNztXmNxJYJraH+DyMoyf5CHDPF9bs0auOoqv26b2cOyfer4iseOkLd8Fg6X2db5Pf3rm/DiBXc1L8o+KwtGjnQ3jXIY77XOibYI3NgT/IkTLU2XTue8A+sXQfIxryc8cmvsWZSEceICc0M1KQnVsSPfXtiFfVf8gM41P/W9MwAmvjqR/C35TF0x1Wcqe/beYfHPj0YClV5sSklGH95/39yw9WbfPhgxwkf0jZ4GfbpYK4OtPbhWVvmtGHuZnaKvrXs7Iw5C7xNprOl0RdPMj57k5LjdNAGy15umWk/+uP6PUfMZjD3Bz801Xf26doWOHVFJSdS2acvJxHZc91Vbdsxvxy/scXQ7Dd1PwYRP49mwNcXMrXHmDBw/bv575gycOEHbo0fI/M8rVP7+r34Dkw6dPsSUFVOsVaU0JNWk0cbRt+k27whhxug+JCVY8yc0tSSjX1av9i/6tbUwzTf30M/Tfm5pa3RUB8IIzcOf7b7vEcgfOglopvkRTJPQ8OF1zXKYcbCHZYgDR/0u3RFE7Ak+mKLvyix55gzxZ6u44NvTXFBt/jt3dQ1fz9V8+UfNG38/h7F8W4Nsfbk353Jf/wCbjl6YEb8mzf7QhZEJqcnMmdif5M5JKCC5cxJzJvYPbiHoQKK/fTtMnmzpyhqUxcjLR1r6inYVRc0KSwgegWz31xy9lq3Jfd3dzTI/gmXzFiC34AtS2l9l6Vu2e1lUfAZjU/BbkMUTF9cv+ho61kyijaOvpbvZH7owMiE1mfWzbuKznDGsn3VTcMXexerV/j14lizxEf2cUTmW4u0OHFHnLSE0n4LtBX5t92uvfMAyrtnmx8xMi1kHh4O0A9Zkh9HisSOC3wTqE/2utlFcVPMjn/6g2LxjncWLIcVPzYAlS0zvKydGT4Phlw+3DNlxRFw0Wxs7PrV6yIw8CFeeNCyr+6CYHw0Dxo2zdGUu/9zHTXhjReR77IjgN5HFExezcOxCenXqRYeEDlze6XIWjl3Ic3e83LI271hn3jwzYZw3eXmWTdx+Xa232esOiotma8JeZmfdN9Ygvn6V0Gb2rJYxP2ZnWz6XRplmvLrGMqT0UOR77MR0pG24CEpkamvGbodJk+BLrxo5KSlm8BYBUlBL5G2rYdqKaSzYvMAaVbs1xdyPaym8UoTYn80mvXKuxWEjElJ+tL5I2xYmUFpkFyGxeccyhgGv+ynKUlrqtueLWad1s+NT693c8INg9Ehr2ZOeOGFpGu/sYUb6DEtfpO8nieA3kqClGhDqxzDM22hvPOz5YtZpnZjmnO2Wvn5HMDdXQ0lREbntJ5DyHeu+UyRH34rgN5KgpxoQApOb699zJy8P8vPJHJAp3jqtEIvvvdMVM1OltHzWWT/eOhQUkOZ1ZxHJ0bci+I2kRVINCIEJ5LkzdSpGOYzrY/WeOHTqUIgmJoQD0/e+yNI3bncIzDngE4QFwKFDZA7I9PHYmbt+bkTebYrgN5IWTTUg+GfePLOOrydaw8MPk52eTbwt3t29cu/KiPyiCcGhYHuBuVHvsbrPfj8udOYcryAsMPeTvG35keqXL4LfSFo81YDgi2HAggW+/aWlGKs+YuzVY91d5xznIvKLJgSHHYd3WOpVDP8cjNQ7QldEKDMTEhLq2itXgt1O7s25PqVB7eWRt/AQwW8kIUk1IPiSleV/E3fOHJ+CNeKtE5vYy+ys+3ytpa/vEVh84NvQJSk0DBgzpq597pzbVdPoYf3R2f7V9oirfSuC3wTE7TJM5Ob62vMPHKD7+x9busRbJzbxTqVgc5hZbpded1NoveW6d7e2ly8Hu92vLf/PG//c8vNpBCL4QnQxb55PV+a/9pn1CpxEU/ZCoeF437kN/xwSqEuUFjJvucxMazS41lBcjNHTYMRlIyxDdx7ZGVGLDxF8Ibrw459vlJvl7Dwp2iMZNGMJe5mddQd9zTn7ul5m6QuJt5xhwOOP17UdDncFt5ybraU4NTqiivSI4AvRhx/TTvZ6rKt8LT75sUTe+jwceJlzPlQsve4my7iQect5Rd2yahXgLMXZx1qKs+RgScTY8kXwhejEy7RjlMO4nWGai9Ci2MvsFO3x9b3/6NKJlsyYCXEqfN5yTjs+4LdQ0jMbnwn1jPwigi9EJ4YBEyZYum7bh8Vj54K2F4R2TkKLYG7WOpPkOROlzdgAF3x7xjKufWJ86BwovO34DgcUFwPmKt+7SM/uyt0RYWIUwReil+xsS6h7ZTtrvdGnNjwVEV8yoXn4bNYeNO/ovDledc63s6XwtuNr7bbjg7NIj/JI+6EdEVGKUwRfiF4MA+bXpUPOOOC04ztFv1bXih0/yjE3a9dZ7tz6HoFawmi/d+Ftx99Wl5rZ6GkwvKc1DcOyXeEvgyiCL0Q3WVkw0rx9Nsoh/XPr05JbJ7rxl0ohczu8d3Va8CtbNZdD1s9av27WNAwaHXZ3YRF8IfrJyXHn2ul3JMxzEYLKodNWER23G4Z9lUBiS1W2agyZmRAf7yqnS3XRcrKmPesO/vLO5grhdxcWwReiH49cO5nbIaEWd1FrSaYWvdjL7KzYs8L9XiY4IHsDMGYMNz4wLvzR7oZBxfCbAfMGJMFRy8iNq9wRv0ZPg/lj51v88h06vEGBQRF8pdStSqndSql9SimfKAOl1ANKqcNKqVLn30PBOK8guHGadoxyGLPH2afMZGrhvo0WmkbB9gJqHDW49HLMbv+bteHk/TPxlnbX099YIn6zBmX5RN+Gc5XfbMFXSsUBfwNuA/oB9yilfHOIwqta6xTn3/PNPa8g+OA07XQ/7dGnw38bLTSNgEnwvHPZhJHFV9/IOVuce0/5pk83M7BipyXi19uWH06PnWCs8IcA+7TW+7XW1cArwPjzvEYQgo9hwIgRZG738NZREnUbjbi9czzofhrT9z3UpQzr4atrB/LulTcALrNODRM/ftfiMeTPlh8uj51gCH4yUObRLnf2eTNJKfWhUupfSqme/g6klMpSSm1WSm0+fPhwEKYmtDpycjAqFMMPevRpOHRQUiZHE27vHLB45/D446HLfd8AZozuwzcdL7L0xdmsEb8uW74nGs2st0OfYydUm7bLgV5a6+uBt4CX/Q3SWudrrQdrrQd369YtRFMTYgrDgPHjfb119u0Ny3SE4DBuNxhf2KBz53BPxcKE1GSuue1GoC5UYGD/Xj6byFmDsujX1WraWfv52pCv8oMh+BWA54q9h7PPjda6Umv9rbP5PDAoCOcVBP9kZ5P5oenV4foWrujwJfZVkZHASjg/F7S9wO2dA860GUpBRkYYZ+Wf1Pa1KGX64iigz/89586r48n0tOmWdjj88oMh+B8AVymleiulEoEfApZMR0qpSzya4wBJcyW0HIaBMWgCY1yp0RXUxEHBe5GRwEqoH3uZnac2PGU2lLkfU9kOuCOEpQwbQ0aGNa9Oba07r44nWYOyfMogLtsdWlt+swVfa10D/AxYjSnkr2mtP1FK/VYpNc457BGl1CdKqe3AI8ADzT2vINRLdjZexYfYceag/7FCRJG3Po9aXeuOrlXaTJvBbbeFeWYBMAx47LG6tkd+fJ+hXmUQQ73KD4oNX2v9ptb6aq31lVrr3zv7/kdrXeR8PFtrfa3WeoDW+nta6131H1EQmolh0L1rL0vXui5nsM+eHJ75CA3CXyrkO3aDUaGgsjJMs2oAnTvXJfJTypJXxxO/HjshXOVLpK0Qs2SOmY3NZcdX4FBQ8PESv/ZVITLwlwo5ewOmmEag/d5NRgbEO4OwtIYXXvD7OQu3x44IvhCzGLdlMU5/19J3qB2QJ5G30cK4Xc7o2ki137swDLj99rr2uXNQ4D/2I2tQlk8wVqg8dkTwhZgmO6uAeFduHWBlH7BvKZRVfoTiLlrj6Z0DkWu/96QREcDTh/p67ISi9q0IvhDTGD0Nxtb0MhsKztmg4HpklR+BeHvnKJd3js0W2fZ7F6mp1vYFgSuuZQ3K4vJOl1v6Sg6WtPgqXwRfiH16XGZp7ugKLFsmq/wIo2B7gemdA277fcYBTJfHSLbfu6isdKfpBuDpp+v9jD0x4gmfvpZe5YvgCzFP98v7WVw0110G9mQNs0If2i40nDtc2TEffTSy7fcuPDduIaA/vousQVk+tW9LDpaQv6XlAgRF8IWYx+0K5/LWsUHeMGDtWlnlRxB+7fe2yEunEBDDMH+cXHjVufVHzqgcS758gD+s/UNLzA4QwRdaAUZPg3HXjLOs8ov6OFf5ATwphNDitt87f5Td9vsITacQkM6d68w6Ddh7MHoajLjcmi//4PGDTF7aMvEiIvhCqyB7WDY2ZV3lFwwANm4M99QEfKNr3fb7SHfH9KZLF3NlD/VG3Hrib5W/5KMlLWLaEcEXWgVGT4NxV4/zfaK0FGbODP2EBDf2MjvL9yy39Lnt99HgjulJZWVdxC0EjLj1xOhpMCN9hk//nLVzgjkzQARfaEXcdtVt7hUkwAVnnU/MnSu2/DBSfKDYEl0b56pdqyI8nYI/vDduA0TcepN7cy4p3VMsfQeOHwj6Kl8EX2g1VJ6pNG+dnXfPTw0Dew/MW3Cx5YeNLu26oN3Z5OHxDc7VfbS4Y3rSiIhbb+bdPs/HtPP6jteDOTsRfKH1kNErw7TjAyioddnxAXZIRaxwsWrvKkv7RFvng8ceiy77vQvviNtDhxr0Mn+mnUn9JgVrVoAIvtCKMHoapPdMt/Tt6Op8sG6dmHXCgD/7vZsTJ0I7mWCRmQkJCXXtN99s8Gcr9+ZcFo5dyC1X3MLCsQvJGpQV1KmJ4AutCu+kVesuc5p1HA4JxAoD/uz3mdvDO6dmYxjw4IN17pk1NfUGYHmTNSiL1f+9OuhiDyL4Qisjc0CmxazjDsICCcQKA8e+Pebffp+QYK6Uo5XU1Ea7Z4YCEXyhVeHPPXNZH4/NW0mqFjIsydIwg606uypfjxkTnfZ7F555dSLI20gEX2h1ZKdnW1b52gazRjmfLCqSVX6ICJgsDRqVajgi8QzA0hqOHQvvfJyI4AutDqOnQZ8ufSx9a3t52PLFRTMsuIOt4uKi25wDjc6cGSpE8IVWyc/Tfl7XUGYslttFU9IthAR3sjQn7mInjz8e3eYcaHTmzFAhgi+0SrIGZflENrpdNCXdQovjbb+3uZKlQfS6Y3rShMyZoUAEX2i1pCWnWdrrLneadUDSLbQw3vZ75Wm/jxU8M2dGyMatCL7QavFx0VQem7eSbiGkuO338fHRb7934b1xKyt8QQgf/lw0S3pB/kBnQ9IttBgBi5WPHRv99nsXnpkzlWpQ5syWRgRfaNVkp2fXNZx338+4LD2SbqFF8C5WbrHfR7s7pieeG7daNzhzZksigi+0aoyeBiMvs9YV3dXNucoXF80WIaD9Ptqja71pRubMlkIEX2j15NzsUXHI6aI5bYxzA1fMOkHn0Clr9ki3/T7ao2v90cTMmS2FCL7Q6jF6GozvM76uw7vQeX7wS821VuxldlbuXWk2NCTUOoudQGyZc1xkZlr98VetCvNUE5gAACAASURBVKtZRwRfEDBt+d7FJ5Zd4yx0Pm1a2G2vsULB9gLOOc6522P2OFf3YCYcizUMAx56qM4989y5sAZgieALAs5V/jXWVb5WzlW+2PJbjO6nnQ9stojwU28RIihzpgi+IDjJHuaRVM3JsmucG7hiyw8K3ukUUr90PojGcoYNJYIyZ4rgC4ITH7985yp/2liwH5Bc+c3FJx2yw8Md89FHY2/D1kUEBWAFRfCVUrcqpXYrpfYppXzKBiml2iilXnU+/75SqlcwzisIwSY7PRub59fCGYGblyaRt83FJx0yHukUYiF/TiAiKACr2YKvlIoD/gbcBvQD7lFK9fMa9iDwjdb6u8CfgNzmnlcQWgKjp8H8sfPNRl0hJgr7gL1csmgGE7c7ZqzjHYD14othu1sMxgp/CLBPa71fa10NvAKM9xozHnjZ+fhfwCillEIQIpCsQVlM6DOhrkMBNpjVtVTMOs0gYDrkWMh/Xx+GAT/+cUR46gRD8JOBMo92ubPP7xitdQ1wHAh/JiFBCEB2erb5/fRY5ZdcBvnPPxy2OUUz9aZDjoX89+fD21MnTBWwImrTVimVpZTarJTafPjw4XBPR2jFGD0NZrjy7GjMVb6CqT1Ksa+SQKzGEjCdglJmGuFYx7sC1p/+FJa7xWAIfgXQ06Pdw9nnd4xSKh7oBPj4Jmmt87XWg7XWg7t16xaEqQlC08m9OZeRF3oUSXF67cxa++vwTSpKCZhOIZbdMT3JyDCv1UVNTVjMOsEQ/A+Aq5RSvZVSicAPgSKvMUXA/c7HdwHvaq01ghDh5Eych9JYTTsJh8jfIqv8hmJJp4BXOoXHHot9cw6Y1/jYY3XtMLlnNlvwnTb5nwGrgZ3Aa1rrT5RSv1VKuZyaXwC6KKX2AY8BPq6bghCJfHXkMsbuu8JseJh2pi2fir1MNnAbgiWdgvZKpxDL7pjeeFbAClNkcVBs+FrrN7XWV2utr9Ra/97Z9z9a6yLn47Na67u11t/VWg/RWu8PxnkFoaWZu3o38bZpjDjg0anAgSZvuaxbmoI7nUJrwzMAK0wpFiJq01YQIo0vjlWxNbkvQw5loBxYTDvLvi6RVX4DCJhOIZbKGTaECAjAEsEXhHq4tHMSAP9K+QXfO9Cu7gln3vyC7RJ9Wx8+6RQ83TFjqZxhQ4iAACwRfEGohxmj+5CUYHpXDC+/HpvXBu6OT2WFXx8+6RS0RzqF1oYrAMtFGAKwRPAFoR4mpCYzZ2J/kjsn8VHyJO7YZX2+5Jvt4rFTD4dOB3DHhNgseHI+PHP+h8GOH3/+IYLQupmQmsyE1GTgJux3LGG5oxSHDdNjR8O0FdPof3F/jJ6tyDzRAOxldlbu8XDHdHi4Y8Za/dqG4grA0josnjqywheERmD0SGPcbo8OBQ4cYsv3g487pufqPhbr1zYEb0+dEKdYEMEXhMaQmUn2RmX12NGw44gUSDkfFnfM1mjOAd8UC08/HdKNWxF8QWgMhoExcDzjd1u71x1cKy6aXgR0x2yt5hyweuoA1NaGdONWBF8QGkt2NtkbFTbXKl+BQ2sx63hQb3Wr1mrOAfO6H320rh3iFAsi+ILQWAwDo9cIhn9u7d5xWMw6LuqtbtVazTkuOncOWwCWCL4gNIV+/eh3xNq17nMx6wTC4o7p6ZrYGgljAJYIviA0hcxMMj/yNevkrc8L98wigoDVrZQKS9KwiCKMAVgi+ILQFJxmnXFem7dFe4pa/SrfXmbnafvT7rYlnYLN1jry35+PMAVgieALQlPp14/sDdSlW1Dg0OKTX3ygmFpHnf0+3jOdwh13tN4NW0883TNDGIAlgi8ITSUzE+MLG8MPevRp2LFvY9imFAkc+/YY2iPh0KMbnPb7hATIzg7fxCKJMAVgieALQlMxDBg3znfz9lhpqzXr+CtW3vlbZ6M1u2N6E6YatyL4gtAcsrPNzVtPsw602s3bgMXKBSveNW5DFIAlgi8IzcEZeTvOK4tm0W7ZvAXJjhkQ7xq3Idq4FcEXhObiiryVzVsOHXNKitM87XbHbG3VrRqCZ43bELmriuALQnNxrvItm7fAjk9b1+ZtzjtFFO5b4P7Rc6VT0ND6qls1BM+N2xClWBDBF4RgkJ3tu3n7TevavH1mw3OAw10nwJJOQfAlDDVuRfAFIRgYBpkqxRp5S+uqeXu6utbSHiv2+/rJyDBdVcFc4T//fIt76ojgC0KQMHqk+SZUa0VmnQsSnekUnFaK2/eZD2tscWK/94dhwG231bVraqCgZRcIIviCECwyM33MOuu/2d4qzDr2Mjtf6dfNhjL9713pFL4e+X2x3wfC+87n0CH/44KECL4gBAunWSfOw6yj0RQfKA7zxFqe4gPFOHSt234f5+F/n3xNrzDOLMLJzKwz6wC8+WaLmnVE8AUhiBg90njcVahbm3b8Y59+Es4phYRA6RRUnJhz6sUw4MEH69wza2paNABLBF8QgklmJp2rFcq5wgd46rMlMW3WqTedwuOPiznnfKSmWvPqtKB7pgi+IAQTwyCj5whLEFatgoLVsZtqod50CidOhGta0YNnXp0WDsASwReEIGP8LIf0Mmvfjo/fDc9kwoAlnYJwfrwDsFowc6YIviAEG8OgX82Flq51F5zAvio/TBNqWQJWtxL7fcMIYeZMEXxBaAEyh/7EGoRlg4L3ngn3tIKOt/3elU4BEPt9Qwlh5kwRfEFoAYyf5TL8aDtL345vK8I0m5bD235vSacg9vuGEcLMmSL4gtBC9GvXy9Je1zn2zDqHTlkDhcR+30Q6dw5JXp1mCb5S6iKl1FtKqb3Ofy8MMK5WKVXq/CtqzjkFIVrI/N50a8pkBXlrngzzrIKHvczOyr0rzYaG+FrIdsUg2Gxiv28MGRlmCmkwN25feKFF7PjNXeHPAt7RWl8FvONs+6NKa53i/BvXzHMKQlRg3JbFuOPW0Pmijl/GzCq/YHsB5xzn3O2xezxW9+PGif2+MRgG3H57XfvcuRbJq9NcwR8PvOx8/DIwoZnHE4SYInv0b3w2b5/85x8o3Bb99vxDpwPkfYmLk2LlTSEEGUWbK/jf0Vp/6Xx8CPhOgHFtlVKblVIblVLyoyC0GozbshhytIOl7+uEw8xe+lH0i762Nrufdj4Q75ymkZpafzsInFfwlVJvK6U+9vM33nOc1tplrfTH5VrrwcC9wDNKqSsDnCvL+cOw+fDhw429FkGISC44Y/W42N79DJcfzGXu6t1hmlHz8We/z9zufFK8c5pGCAqinFfwtdY3a62v8/O3DPhKKXWJOT91CfB1gGNUOP/dDxQDfn+6tNb5WuvBWuvB3bp1a+IlCUJkcTZurMWsoxW81buY+E+XhntqTaZe+73QNEKwcdtck04RcL/z8f3AMu8BSqkLlVJtnI+7AunAjmaeVxCihporJzK0wsOBzemx07Z2Rfgm1Uy83THdiHdO0wnBxm1zBT8H+L5Sai9ws7ONUmqwUup555i+wGal1HbgPSBHay2CL7QaZozuQ/uE/zYzaHoYPSsu8HtDHJW47ffindM8WnjjtlmCr7Wu1FqP0lpf5TT9HHX2b9ZaP+R8vEFr3V9rPcD57wvBmLggRAsTUpP56ZRs0r+4yNL/Secq8l+bGaZZNR23/d75A5bgst/bbOKd01wyM6FNG9OG36ZN0O+W4oN6tBbm3LlzlJeXc/bs2XBPRWgB2rZtS48ePUjwrAAUI0xITeY7E+eQvnGKuch3Voaa89E8sv4rN8yzaxx5G/JM+73zGsa47Pcjh8vqvrkYBrz3nplLJyMj6P+fUSX45eXldOzYkV69eqE8s8sJUY/WmsrKSsrLy+ndu3e4p9MiGLdlMeLfMyi5sM6L5UDcKWa+PZPcm6ND9O1ldop2WYPl3eacfv1CP6FYxDBa7IczqnLpnD17li5duojYxyBKKbp06RLzd285+ia3KcRVEStvbR6XPPFH0nPejXjf/ILtBThwmA0NNofTnBMfL5u1UUBUCT4gYh/DtIb31rgnuy7fjEv0FZzRT1NxrCrqArLGuZKljR0r5pwoIOoEXxCiGsMgd2A2vb6xdp9o8yVH41+k6lxtRAdkuYudOL2N3MVOhKhABL8RVFZWkpKSQkpKCt27dyc5Odndrq6uDuq5jh07xrx584J6TCFCyM1l9sEe5mMP087J+Nc5GbeKL45VhW1q9WEpdqK8ip2EIA+M0HxiX/DtdpgzJygRa126dKG0tJTS0lKmTp3Ko48+6m4nJiYGfF1NTU2jzyWCH9tkXTKWAV96dDhF/2jC37B1eCcsczof/oqd3HgAapSNLPpFlSmqtRLbgm+3w6hR8Ktfmf+2QH7p5557jhtuuIEBAwYwadIkzpw5A8ADDzzA1KlTGTp0KNnZ2Xz66aekpaXRv39/fvnLX9KhQ11Crblz53LDDTdw/fXX8+tf/xqAWbNm8emnn5KSksKMGTOCPm8hzGRmMm+VuUp2B2M5RX9/7Z/I3xL5KZRdxU7e/u4Q1nS6Iur2H1ojsS34xcVQXW3WiKyubpE6kRMnTuSDDz5g+/bt9O3blxdeqIsrKy8vZ8OGDTz99NNMnz6d6dOn89FHH9GjRw/3mDVr1rB37142bdpEaWkpW7ZsoaSkhJycHK688kpKS0uZO3du0OcthBnDYFjvkSxYSZ3XDrhFf+qKqREn+he0vcAy19v2QS2K/KGTACJ+/0GIdcHPyIDERDM/d2Ki2Q4yH3/8MSNGjKB///4sWbKETz75xP3c3XffTZyzOLHdbufuu+8G4N5773WPWbNmDWvWrCE1NZWBAweya9cu9u7dG/R5ChFITg5Z2xTZ651tj7QLGs3UFVOxlwX/rrQp+NjvNRxpB5t79GNrcl/3uEjdfxBMoirwqtEYBrzzTotFrYFpuiksLGTAgAG89NJLFHvcRbRv3/68r9daM3v2bKZMmWLpP3DgQJBnKkQShdsqmPufKn713aHkvLMRgLx055POVb5GM/HViSz9wVKMnuF1ecxbn2fa753RtTZtFivf1/Uyy7hLOyeFZX5Cw4jtFT6YIj97dov5CJ88eZJLLrmEc+fOsWTJkoDj0tLSeP311wF45ZVX3P2jR49m0aJFnDp1CoCKigq+/vprOnbsyMmTJ1tkzkJ4KdxWweylH1FxrIr8oZNwADnv4Helf+j0IYYvGh7Wlb69zE7R7iLLvO7YDUPKFUuvu8ndl5QQx4zRfcIwQ6GhxL7gtzD/+7//y9ChQ0lPT+eaa64JOO6ZZ57h6aef5vrrr2ffvn106tQJgFtuuYV7770XwzDo378/d911FydPnqRLly6kp6dz3XXXyaZtjDF39W6qzpneLluT+/JBj2sByH0H7tvuO96Bg4fffDiUU7Tgjq51re4dZrHybwYO4atrB6KA5M5JzJnYnwmpyWGbp3B+lFmoKvIYPHiw3rx5s6Vv586d9O3bN8ArIpszZ86QlJSEUopXXnmFf/zjHyxb5lM+oNUTze9xQ+k9a6WlNNzAip38c3E2NjQKmDwBlgzAbdpxkZ2eHZacO9NWTGPBlgVmQ8OEnfDGa8DUqTB/fsjnI9SPUmqLs8KgD7LCDxFbtmwhJSWF66+/nnnz5vHUU0+Fe0pCmPC2c29N7stbVw11txcXOlf6XmuxvPV5YfHcueDISR/vHCl0Ep2I4IeIESNGsH37dj788ENKSkr47ne/G+4pCWFixug+JCXEWfpeTv8vtEcuocWFMOBYW5/XTlsxLaT2fHuZnacO/J/ZUOZmbWU7pNBJlCKCLwghZkJqMnMm9ie5c5Lb/v2DR/4L2/jxlnHzXz+Lzcuu48BB3vq8kM01b30etWi3/V5pyDiIFDqJUmLbLVMQIpQJqcm+G5zZ2bBsmVnAGjOKdf66C5k6/Bu0h32ncHch+VvyyRqU1aJzNL1zrPtMd+wGY9AEWd1HKbLCF4RIwTBgxAhLV9bbRxlfc6XP0GkrW960U7C9AIfWVu+cjUpW91GMrPAFIZLIyYH0dPcqHyD7pX0UPWSrKzwCOLSD37wzn5OHqvjiWBWXdk5ixug+QXWL3PHh25b28M/BGDheVvdRjKzwG0lcXBwpKSlcd9113H333e5kaU3hgQce4F//+hcADz30EDt27Ag4tri4mA0bNrjbCxYsoKCgoMnnFiIUwwCvuAujHObv9F3lv7d/PRXHqtAQ9OIp9jI7685ak933O4Ks7qOcmBd8e5mdOWvnBO32NykpidLSUj7++GMSExNZsGCB5fmmpEIGeP755+lXT01Qb8GfOnUqmeIWF5vk5kKvXpaurFf3MqHbSEtftW0/J+NWudvBTF5WsDoPhw1LKoVMlSKr+ygnpgXfXmZnVMEofvXerxhVMCroNs8RI0awb98+iouLGTFiBOPGjaNfv37U1tYyY8YMd8rjhQsXAmbenJ/97Gf06dOHm2++ma+//tp9rIyMDFyBZv/+978ZOHAgAwYMYNSoURw4cIAFCxbwpz/9iZSUFNauXcuTTz7JH//4RwBKS0tJS0vj+uuv58477+Sbb75xH3PmzJkMGTKEq6++mrVr1wb1+oUWZPZsn67sf32J8vLaORFnLSgerORlOz5619IefhCMJ6Q+Q7QT04JffKCY6tpqanUt1bXVFB8oDtqxa2pqWLVqFf379wdg69at/PnPf2bPnj288MILdOrUiQ8++IAPPviA5557js8++4w33niD3bt3s2PHDgoKCiwrdheHDx/mJz/5Ca+//jrbt2/nn//8J7169bIUXBnhtbGXmZlJbm4uH374If379+c3v/mNZZ6bNm3imWeesfQLEU5Wls8q33hvLyPOdLX01djKLKv8YCQvs/91Jus7nbD09avtLKv7GCCmBT+jVwaJcYnEqTgS4xLJ6JXR7GNWVVWRkpLC4MGDueyyy3jwwQcBGDJkCL179wbMlMcFBQWkpKQwdOhQKisr2bt3LyUlJdxzzz3ExcVx6aWXctNNN/kcf+PGjYwcOdJ9rIsuuqje+Rw/fpxjx45x4403AnD//fdTUlLifn7ixIkADBo0SDJwRht+Vvk5/3e4zjffXSVrPt/adgYneZndTvFrc3E4i6ujIc4BmUNb1gVUCA0x7aVj9DR4J/Mdig8Uk9ErIygpZl02fG88UyFrrXn22WcZPXq0Zcybb77Z7PM3ljZt2gDmZnNT9xeEMJGVBSUl4JGF1SiHcQfaUtjLabpRgHagk/7DnLH3NMlLp3BbBXNX7+aLY1W8WDSHT/potFPsAR4v64Hx29Dn8BGCT0yv8MEU/dkjZoc0n/jo0aOZP38+586dA2DPnj2cPn2akSNH8uqrr1JbW8uXX37Je++95/PatLQ0SkpK+OyzzwA4evQoQMB0yZ06deLCCy902+f//ve/u1f7QgyweDGkpFi6st+uMksjulDQp+eJJou9K1VzasVOdiatNxO3OY+LAzrfPLbJ0xcii5he4YeLhx56iAMHDjBw4EC01nTr1o3CwkLuvPNO3n33Xfr168dll12G4ccm2q1bN/Lz85k4cSIOh4OLL76Yt956izvuuIO77rqLZcuW8eyzz1pe8/LLLzN16lTOnDnDFVdcwYsvvhiqSxVCwbx5Ft98oxzG74LCvrjNOusOrsNeZm/0wsYzVfNvV8/jB3c5n3ClUlCQMVK8wWIFSY8sRBTyHgdg5kzIq8uhY+8Bw38EDo8cbBP6TOCNH77RqMO6UjU/tfyPXHKimOE/Au3hjjnywhT+M31bUC5BCA2SHlkQop3cXItpxyg3I189UygX7SlqtOvxpZ2TyC5+kYk7iikYYBV7BeRMFFfMWEIEXxCihXlW8e13xPq0QzvI29C4TJp/PbWZae+bpTd3Wj0+GdFrZNhr6QrBRQRfEKIFw7CkNsjcbiY0s6zydzVilZ+fT+rvZwKwsQes86xHrqBf18CR30J0IoIvCNFEbq5b9I1ymL8SSzUqh25gvvz8fJgyBTBNN3nDqEulANiwkTlANmtjjWYJvlLqbqXUJ0oph1LK7yaBc9ytSqndSql9SqlZzTmnILR6PEQ/ayuMPGh9etmOQux/nen/tXY7pKa6xR7MDeBlXvFa4/qME3NODNLcFf7HwESgJNAApVQc8DfgNqAfcI9SSu4VBaE55ObCffcBXrZ8ZW685q3Ng6QkaNfO/LvgAmjfHoYNA6/AwVmjPDZrAYUiO12yYsYizRJ8rfVOrfX50vMNAfZprfdrrauBV4Dx53lNRFJZWUlKSgopKSl0796d5ORkd7u6urre127evJlHHnnkvOcYNmxYsKYbUjp06BDuKbQ+Fi+G7GzTlu9h1gFYdg3Yu56Fqirz7+RJ8JPKO38glPSy9vXt2ldW9zFKKAKvkoEyj3Y5MNTfQKVUFpAFcNlll/kb0ig8Q8aDUSCiS5cu7rQKTz75JB06dOAXv/iF+/mamhri4/3/lw4ePJjBgwNavdz4S6gmCAHJzcW48krm509hyhhnnzK1f9Yo+M/LgV9q7wHTxta9xsX0tOktNFkh3Jx3ha+Uelsp9bGfv6Cv0rXW+VrrwVrrwd26dWvWsTxDxluiQISLBx54gKlTpzJ06FCys7PZtGkThmGQmprKsGHD2L3bvAEqLi5m7Fjz2/Xkk0/y4x//mIyMDK644gr+8pe/uI/nWikXFxeTkZHBXXfdxTXXXMN9992HK0juzTff5JprrmHQoEE88sgj7uN68sknnzBkyBBSUlK4/vrr2bt3LwATJkxg0KBBXHvtteTn51vOO2PGDK699lpuvvlmNm3a5J5fUZGZgvell15i/PjxZGRkcNVVVwXMvjl37lx3auhf//rXAJw+fZoxY8YwYMAArrvuOl599dVm/b8LdRTeMIbNN/yFXscSLf0ll5uiHoi8W9pZNmoBEmuu5eW3vhv074kQGZx3ha+1vrmZ56gAenq0ezj7WhTPkHEXrgIRwSwDB1BeXs6GDRuIi4vjxIkTrF27lvj4eN5++22eeOIJXn/9dZ/X7Nq1i/fee4+TJ0/Sp08fpk2bRkJCgmXMtm3b+OSTT7j00ktJT09n/fr1DB48mClTplBSUkLv3r255557/M5pwYIFTJ8+nfvuu4/q6mpqa83/i0WLFnHRRRdRVVXFDTfcwKRJk+jSpQunT5/mpptuYu7cudx555388pe/5K233mLHjh3cf//9jBs3DoBNmzbx8ccf065dO2644QbGjBljuXNZs2YNe/fuZdOmTWitGTduHCUlJRw+fJhLL72UlStXAmaWT6HpuO5eK45VmXFSna7gZNxPgL+Zy3uniD88BrYVJIHrzjMhAbp3xz5tLIWVHt48TnPQRbUPuBdHQNC/K0J4CYVb5gfAVUqp3kqpROCHQNF5XtNsAhWCCFaBCE/uvvtu4uLMGPfjx49z9913c9111/Hoo4/yySef+H3NmDFjaNOmDV27duXiiy/mq6++8hkzZMgQevTogc1mIyUlhQMHDrBr1y6uuOIKd/rkQIJvGAZ/+MMfyM3N5eDBgyQlmXnS//KXvzBgwADS0tIoKytzr/wTExO59dZbAejfvz833ngjCQkJ9O/f35JW+fvf/z5dunQhKSmJiRMnsm7dOst516xZw5o1a0hNTWXgwIHs2rWLvXv30r9/f9566y1mzpzJ2rVr6dSpUyP+hwVPPO9eoc5037H2NuIcF9cNVFB6CYxeOAJOnDD/Kiux//t5Jp71LY+ZVJtGG4eZ1iKY1bOEyKG5bpl3KqXKAQNYqZRa7ey/VCn1JoDWugb4GbAa2Am8prX2r4JBJFAhiGAUiPDGMzXyr371K773ve/x8ccfs3z5cs6ePev3Na60xRA4dXFDxgTi3nvvpaioiKSkJG6//XbeffddiouLefvtt7Hb7Wzfvp3U1FT3/BISElDKXBbabDb3uW02m+W8rjGB2lprZs+eTWlpKaWlpezbt48HH3yQq6++mq1bt9K/f39++ctf8tvf/rbB1yJY8Xf36qJT7d3mA48N3DX71zD0OXPbLH9LPsMWDePQ6UN1AzSAolPtJMuxWmJxJISX5nrpvKG17qG1bqO1/o7WerSz/wut9e0e497UWl+ttb5Sa/375k66IcwY3YekhDhLX1AKRJyH48ePk5xs3ga/9NJLQT9+nz592L9/v3vVHcgWvn//fq644goeeeQRxo8fz4cffsjx48e58MILadeuHbt27WLjxo2NPv9bb73F0aNHqaqqorCwkPT0dMvzo0ePZtGiRZw6dQqAiooKvv76a7744gvatWvH5MmTmTFjBlu3bm30uQWT+oS4Y+1ttKvNwKsSIpu+2ET8b+OZsmKK74sUdKyZ6F7du2iJxZEQXmI2PbLL9hhML52GkJ2dzf3338/vfvc7xowZc/4XNJKkpCTmzZvHrbfeSvv27bnhhhv8jnvttdf4+9//TkJCAt27d+eJJ56gffv2LFiwgL59+9KnTx/S0tIaff4hQ4YwadIkysvLmTx5so/n0S233MLOnTvdqZ87dOjA4sWL2bdvHzNmzMBms5GQkMD8+fMbf/ECYApxRT2ifxkzadtVU3rkP5b+Wu3/rmBk8kS+PvgQVdQ9H4rFkRB6JD1yFHLq1Ck6dOiA1pqf/vSnXHXVVTz66KMtft6XXnqJzZs389e//rXFziHv8flx2fA9zTquAlXJHguboc8NZdMXm+o91n3972PxxMVBd2EWwkd96ZFjdoUfyzz33HO8/PLLVFdXk5qaypQpfm7ThZiloXev7//kfUb/fTRr9q/xexyX2LuOKQIf+8gKX4go5D0OPvlb8pmzbg6VZypJsCXQvWN3pg+dTtYgKUwei8TUCl9r7eMZIsQGkbr4iHayBmWJuAtAlKVHbtu2LZWVlSIMMYjWmsrKStq2bRvuqQhCzBJVK/wePXpQXl7O4cOHwz0VoQVo27YtPXrUkwtAEIRmEVWCn5CQ4I4wFQRBEBpHVJl0px/S8QAABDVJREFUBEEQhKYjgi8IgtBKEMEXBEFoJUSsH75S6jBw8LwDA9MVOHLeUZFPrFwHyLVEKrFyLbFyHdC8a7lca+23oEjECn5zUUptDhR8EE3EynWAXEukEivXEivXAS13LWLSEQRBaCWI4AuCILQSYlnw888/JCqIlesAuZZIJVauJVauA1roWmLWhi8IgiBYieUVviAIguCBCL4gCEIrIaoFXyl1q1Jqt1Jqn1Jqlp/n2yilXnU+/75SqlfoZ9kwGnAtDyilDiulSp1/D4VjnudDKbVIKfW1UurjAM8rpdRfnNf5oVJqYKjn2FAacC0ZSqnjHu/J/4R6jg1BKdVTKfWeUmqHUuoTpdR0P2Oi4n1p4LVEy/vSVim1SSm13Xktv/EzJrgaprWOyj8gDvgUuAJIBLYD/bzGPAwscD7+IfBquOfdjGt5APhruOfagGsZCQwEPg7w/O3AKsyqfGnA++GeczOuJQNYEe55NuA6LgEGOh93BPb4+XxFxfvSwGuJlvdFAR2cjxOA94E0rzFB1bBoXuEPAfZprfdrrauBV4DxXmPGAy87H/8LGKUis3pKQ64lKtBalwBH6xkyHijQJhuBzkqpS0Izu8bRgGuJCrTWX2qttzofnwR2At71DKPifWngtUQFzv/rU85mgvPP24smqBoWzYKfDJR5tMvxfePdY7TWNcBxoEtIZtc4GnItAJOct9v/Ukr1DM3Ugk5DrzVaMJy35KuUUteGezLnw2kSSMVcTXoSde9LPdcCUfK+KKXilFKlwNfAW1rrgO9LMDQsmgW/tbEc6KW1vh54i7pffSF8bMXMWzIAeBYoDPN86kUp1QF4Hfi51vpEuOfTHM5zLVHzvmita7XWKUAPYIhS6rqWPF80C34F4LnK7eHs8ztGKRUPdAIqQzK7xnHea9FaV2qtv3U2nwcGhWhuwaYh71tUoLU+4bol11q/CSQopbqGeVp+UUolYArkEq31Uj9DouZ9Od+1RNP74kJrfQx4D7jV66mgalg0C/4HwFVKqd5KqUTMDY0irzFFwP3Ox3cB72rn7keEcd5r8bKnjsO0XUYjRUCm0yskDTiutf4y3JNqCkqp7i57qlJqCOb3KeIWFM45vgDs1Fo/HWBYVLwvDbmWKHpfuimlOjsfJwHfB3Z5DQuqhkVViUNPtNY1SqmfAasxvVwWaa0/UUr9FtistS7C/GD8XSm1D3Pz7Yfhm3FgGngtjyilxgE1mNfyQNgmXA9KqX9gekl0VUqVA7/G3IxCa70AeBPTI2QfcAb4UXhmen4acC13AdOUUjVAFfDDCF1QpAP/DXzktBcDPAFcBlH3vjTkWqLlfbkEeFkpFYf5o/Sa1npFS2qYpFYQBEFoJUSzSUcQBEFoBCL4giAIrQQRfEEQhFaCCL4gCEIrQQRfEAShlSCCLwiC0EoQwRcEQWgl/H8lYp8Iunp70wAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "lZ36J2EN85b9"
      },
      "source": [
        "In order to implement a regularization we need to modify the loss function. Since the loss function in this exercise is computed during the training step, we define a new training step with a regularization."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "sLbcWwlt9Jwl",
        "colab": {}
      },
      "source": [
        "\"\"\" In order to avoid overfitting we implement a training step that also includes a regularization on the weights of our big model. For this we use the Frobenius/squared l2-norm of each weight matrix/vector. \n",
        "Hint: Use the tf.reduce_sum() function on a list of individual regularization terms for each matrix/vector of the network.\"\"\"\n",
        "\n",
        "def regularized_train_step(model, optimizer, x, y, lmbd):\n",
        "    with tf.GradientTape() as tape:\n",
        "        tape.watch(model.trainable_variables)\n",
        "        y_pred = model.__call__(x) # Compute a prediction with \"model\" on the input \"x\"\n",
        "        loss_val = tf.reduce_mean((y-y_pred)**2) # Compute the Mean Squared Error (MSE) for the prediction \"y_pred\" and the targets \"y\"\n",
        "        regul_val = tf.reduce_sum([tf.norm(param) for param in model.trainable_variables])# Compute the regularization based on the list \"model.trainable_variables\"\n",
        "        total_loss = loss_val + lmbd * regul_val # Add the loss with a the regularization term weighted by \"lmbd\"\n",
        "    grads = tape.gradient(total_loss, model.trainable_variables)\n",
        "    optimizer.apply_gradients(zip(grads, model.trainable_variables))\n",
        "    return loss_val"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ExK4FIw89s0M"
      },
      "source": [
        "We can now set the strength of the regularization and retrain the big model with a regularization. We create another instance of the big model in order to compare the big model with and without regularization."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "_PUUBGi496Vt",
        "outputId": "6354b156-4bba-4d27-fb04-8cfead1fda04",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        }
      },
      "source": [
        "\"\"\" Implement the training for the bigger model with the regularized_train_step function. Note: We are plotting the MSE loss without the regularization in order to compare it with the unregularized model. \"\"\"\n",
        "\n",
        "lmbd = 0.005\n",
        "\n",
        "big_reg_mdl = MyBigModel()\n",
        "big_opt = tf.optimizers.SGD(learning_rate)\n",
        "\n",
        "epoch = 0\n",
        "train_iters = 0\n",
        "train_loss = 0.0\n",
        "for x_t, y_t in train_overfit_ds:\n",
        "    train_loss += regularized_train_step(big_reg_mdl, big_opt, x_t, y_t, lmbd) # Perform a regularized training step with the model \"big_mdl\" and the optimizer \"big_opt\" on the inputs \"x_t\" and the corresponding targets \"y_t\" with the regularization parameter being \"lmbd\"\n",
        "    train_iters += 1\n",
        "    if train_iters == N_train_samples_overfit/batch_size: # An epoch is completed\n",
        "        for x_v, y_v in validation_ds:\n",
        "            y_pred = big_reg_mdl.__call__(x_v)# Compute a prediction with \"big_mdl\" on the input \"x_v\"\n",
        "            validation_loss += (y_v- y_pred)**2 # Compute the Mean Squared Error (MSE) for the prediction \"y_pred\" and the targets \"y_v\"\n",
        "        print(\"------ Epoch: \",epoch)\n",
        "        print(\"Train loss: \",(train_loss/train_iters).numpy())\n",
        "        print(\"Validation loss: \",(validation_loss/N_validation_samples).numpy())\n",
        "        train_iters = 0\n",
        "        train_loss = 0.0\n",
        "        train_reg = 0.0\n",
        "        validation_loss = 0\n",
        "        epoch += 1\n",
        "    if (epoch == N_epochs):\n",
        "        break"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "------ Epoch:  0\n",
            "Train loss:  7.7333536\n",
            "Validation loss:  [[0.8449731]]\n",
            "------ Epoch:  1\n",
            "Train loss:  1.1618569\n",
            "Validation loss:  [[0.7813591]]\n",
            "------ Epoch:  2\n",
            "Train loss:  0.5991075\n",
            "Validation loss:  [[0.3799633]]\n",
            "------ Epoch:  3\n",
            "Train loss:  0.3798367\n",
            "Validation loss:  [[0.33906606]]\n",
            "------ Epoch:  4\n",
            "Train loss:  0.33673593\n",
            "Validation loss:  [[0.3058833]]\n",
            "------ Epoch:  5\n",
            "Train loss:  0.30479118\n",
            "Validation loss:  [[0.2812339]]\n",
            "------ Epoch:  6\n",
            "Train loss:  0.2776755\n",
            "Validation loss:  [[0.2605023]]\n",
            "------ Epoch:  7\n",
            "Train loss:  0.25378102\n",
            "Validation loss:  [[0.2422978]]\n",
            "------ Epoch:  8\n",
            "Train loss:  0.23234558\n",
            "Validation loss:  [[0.22595085]]\n",
            "------ Epoch:  9\n",
            "Train loss:  0.21286795\n",
            "Validation loss:  [[0.21112785]]\n",
            "------ Epoch:  10\n",
            "Train loss:  0.19499749\n",
            "Validation loss:  [[0.19765961]]\n",
            "------ Epoch:  11\n",
            "Train loss:  0.17850879\n",
            "Validation loss:  [[0.18549114]]\n",
            "------ Epoch:  12\n",
            "Train loss:  0.16328834\n",
            "Validation loss:  [[0.17464872]]\n",
            "------ Epoch:  13\n",
            "Train loss:  0.14931141\n",
            "Validation loss:  [[0.165205]]\n",
            "------ Epoch:  14\n",
            "Train loss:  0.13660608\n",
            "Validation loss:  [[0.1572214]]\n",
            "------ Epoch:  15\n",
            "Train loss:  0.12521204\n",
            "Validation loss:  [[0.1506918]]\n",
            "------ Epoch:  16\n",
            "Train loss:  0.11514417\n",
            "Validation loss:  [[0.1455156]]\n",
            "------ Epoch:  17\n",
            "Train loss:  0.106369816\n",
            "Validation loss:  [[0.14151177]]\n",
            "------ Epoch:  18\n",
            "Train loss:  0.09880558\n",
            "Validation loss:  [[0.13846079]]\n",
            "------ Epoch:  19\n",
            "Train loss:  0.09232964\n",
            "Validation loss:  [[0.1361439]]\n",
            "------ Epoch:  20\n",
            "Train loss:  0.08680272\n",
            "Validation loss:  [[0.13437048]]\n",
            "------ Epoch:  21\n",
            "Train loss:  0.082084276\n",
            "Validation loss:  [[0.13298881]]\n",
            "------ Epoch:  22\n",
            "Train loss:  0.07804577\n",
            "Validation loss:  [[0.13188514]]\n",
            "------ Epoch:  23\n",
            "Train loss:  0.07457681\n",
            "Validation loss:  [[0.13097863]]\n",
            "------ Epoch:  24\n",
            "Train loss:  0.07158451\n",
            "Validation loss:  [[0.13021345]]\n",
            "------ Epoch:  25\n",
            "Train loss:  0.06899354\n",
            "Validation loss:  [[0.12955156]]\n",
            "------ Epoch:  26\n",
            "Train loss:  0.066742055\n",
            "Validation loss:  [[0.12896854]]\n",
            "------ Epoch:  27\n",
            "Train loss:  0.06477944\n",
            "Validation loss:  [[0.12844704]]\n",
            "------ Epoch:  28\n",
            "Train loss:  0.06306414\n",
            "Validation loss:  [[0.12797642]]\n",
            "------ Epoch:  29\n",
            "Train loss:  0.06156135\n",
            "Validation loss:  [[0.12754902]]\n",
            "------ Epoch:  30\n",
            "Train loss:  0.06024166\n",
            "Validation loss:  [[0.12715907]]\n",
            "------ Epoch:  31\n",
            "Train loss:  0.05907979\n",
            "Validation loss:  [[0.12680386]]\n",
            "------ Epoch:  32\n",
            "Train loss:  0.058054194\n",
            "Validation loss:  [[0.12647775]]\n",
            "------ Epoch:  33\n",
            "Train loss:  0.05714625\n",
            "Validation loss:  [[0.12618178]]\n",
            "------ Epoch:  34\n",
            "Train loss:  0.056339715\n",
            "Validation loss:  [[0.1259073]]\n",
            "------ Epoch:  35\n",
            "Train loss:  0.055620443\n",
            "Validation loss:  [[0.12566377]]\n",
            "------ Epoch:  36\n",
            "Train loss:  0.054976504\n",
            "Validation loss:  [[0.1254271]]\n",
            "------ Epoch:  37\n",
            "Train loss:  0.054397\n",
            "Validation loss:  [[0.12523736]]\n",
            "------ Epoch:  38\n",
            "Train loss:  0.053873308\n",
            "Validation loss:  [[0.12501393]]\n",
            "------ Epoch:  39\n",
            "Train loss:  0.053397622\n",
            "Validation loss:  [[0.12490472]]\n",
            "------ Epoch:  40\n",
            "Train loss:  0.05296392\n",
            "Validation loss:  [[0.12462984]]\n",
            "------ Epoch:  41\n",
            "Train loss:  0.05256825\n",
            "Validation loss:  [[0.12472115]]\n",
            "------ Epoch:  42\n",
            "Train loss:  0.052212182\n",
            "Validation loss:  [[0.1242079]]\n",
            "------ Epoch:  43\n",
            "Train loss:  0.051911134\n",
            "Validation loss:  [[0.12502484]]\n",
            "------ Epoch:  44\n",
            "Train loss:  0.051731583\n",
            "Validation loss:  [[0.12391428]]\n",
            "------ Epoch:  45\n",
            "Train loss:  0.051906493\n",
            "Validation loss:  [[0.12827694]]\n",
            "------ Epoch:  46\n",
            "Train loss:  0.053214226\n",
            "Validation loss:  [[0.12875417]]\n",
            "------ Epoch:  47\n",
            "Train loss:  0.05846978\n",
            "Validation loss:  [[0.15647402]]\n",
            "------ Epoch:  48\n",
            "Train loss:  0.075783096\n",
            "Validation loss:  [[0.20509568]]\n",
            "------ Epoch:  49\n",
            "Train loss:  0.1361087\n",
            "Validation loss:  [[0.3614281]]\n",
            "------ Epoch:  50\n",
            "Train loss:  0.26630223\n",
            "Validation loss:  [[0.7282415]]\n",
            "------ Epoch:  51\n",
            "Train loss:  0.5996614\n",
            "Validation loss:  [[0.5242144]]\n",
            "------ Epoch:  52\n",
            "Train loss:  0.41929683\n",
            "Validation loss:  [[0.8340921]]\n",
            "------ Epoch:  53\n",
            "Train loss:  0.65708315\n",
            "Validation loss:  [[0.20082937]]\n",
            "------ Epoch:  54\n",
            "Train loss:  0.15541205\n",
            "Validation loss:  [[0.16459551]]\n",
            "------ Epoch:  55\n",
            "Train loss:  0.10263567\n",
            "Validation loss:  [[0.13338071]]\n",
            "------ Epoch:  56\n",
            "Train loss:  0.081476934\n",
            "Validation loss:  [[0.12931685]]\n",
            "------ Epoch:  57\n",
            "Train loss:  0.07383963\n",
            "Validation loss:  [[0.12483892]]\n",
            "------ Epoch:  58\n",
            "Train loss:  0.06900744\n",
            "Validation loss:  [[0.12291996]]\n",
            "------ Epoch:  59\n",
            "Train loss:  0.06546281\n",
            "Validation loss:  [[0.12212525]]\n",
            "------ Epoch:  60\n",
            "Train loss:  0.0626486\n",
            "Validation loss:  [[0.12090725]]\n",
            "------ Epoch:  61\n",
            "Train loss:  0.06031521\n",
            "Validation loss:  [[0.12135502]]\n",
            "------ Epoch:  62\n",
            "Train loss:  0.05840016\n",
            "Validation loss:  [[0.12040398]]\n",
            "------ Epoch:  63\n",
            "Train loss:  0.0567957\n",
            "Validation loss:  [[0.12157866]]\n",
            "------ Epoch:  64\n",
            "Train loss:  0.055479955\n",
            "Validation loss:  [[0.12049881]]\n",
            "------ Epoch:  65\n",
            "Train loss:  0.054392014\n",
            "Validation loss:  [[0.12241173]]\n",
            "------ Epoch:  66\n",
            "Train loss:  0.053523805\n",
            "Validation loss:  [[0.12079179]]\n",
            "------ Epoch:  67\n",
            "Train loss:  0.052861672\n",
            "Validation loss:  [[0.12390693]]\n",
            "------ Epoch:  68\n",
            "Train loss:  0.052418068\n",
            "Validation loss:  [[0.12124514]]\n",
            "------ Epoch:  69\n",
            "Train loss:  0.052294977\n",
            "Validation loss:  [[0.12683578]]\n",
            "------ Epoch:  70\n",
            "Train loss:  0.052544475\n",
            "Validation loss:  [[0.12265304]]\n",
            "------ Epoch:  71\n",
            "Train loss:  0.05371244\n",
            "Validation loss:  [[0.13393117]]\n",
            "------ Epoch:  72\n",
            "Train loss:  0.05585248\n",
            "Validation loss:  [[0.12916045]]\n",
            "------ Epoch:  73\n",
            "Train loss:  0.061533555\n",
            "Validation loss:  [[0.15378004]]\n",
            "------ Epoch:  74\n",
            "Train loss:  0.06948335\n",
            "Validation loss:  [[0.15762281]]\n",
            "------ Epoch:  75\n",
            "Train loss:  0.09173918\n",
            "Validation loss:  [[0.20427375]]\n",
            "------ Epoch:  76\n",
            "Train loss:  0.110194616\n",
            "Validation loss:  [[0.24521165]]\n",
            "------ Epoch:  77\n",
            "Train loss:  0.17490393\n",
            "Validation loss:  [[0.26216522]]\n",
            "------ Epoch:  78\n",
            "Train loss:  0.16170026\n",
            "Validation loss:  [[0.32141244]]\n",
            "------ Epoch:  79\n",
            "Train loss:  0.24059305\n",
            "Validation loss:  [[0.22780655]]\n",
            "------ Epoch:  80\n",
            "Train loss:  0.13970211\n",
            "Validation loss:  [[0.2231386]]\n",
            "------ Epoch:  81\n",
            "Train loss:  0.15250857\n",
            "Validation loss:  [[0.1730015]]\n",
            "------ Epoch:  82\n",
            "Train loss:  0.096810885\n",
            "Validation loss:  [[0.15553454]]\n",
            "------ Epoch:  83\n",
            "Train loss:  0.09208121\n",
            "Validation loss:  [[0.14681199]]\n",
            "------ Epoch:  84\n",
            "Train loss:  0.074188225\n",
            "Validation loss:  [[0.1340616]]\n",
            "------ Epoch:  85\n",
            "Train loss:  0.07150534\n",
            "Validation loss:  [[0.13756494]]\n",
            "------ Epoch:  86\n",
            "Train loss:  0.06488566\n",
            "Validation loss:  [[0.1275372]]\n",
            "------ Epoch:  87\n",
            "Train loss:  0.064126246\n",
            "Validation loss:  [[0.13522555]]\n",
            "------ Epoch:  88\n",
            "Train loss:  0.06113062\n",
            "Validation loss:  [[0.12605992]]\n",
            "------ Epoch:  89\n",
            "Train loss:  0.061684087\n",
            "Validation loss:  [[0.13640146]]\n",
            "------ Epoch:  90\n",
            "Train loss:  0.060258906\n",
            "Validation loss:  [[0.12715863]]\n",
            "------ Epoch:  91\n",
            "Train loss:  0.062199943\n",
            "Validation loss:  [[0.14037794]]\n",
            "------ Epoch:  92\n",
            "Train loss:  0.061734255\n",
            "Validation loss:  [[0.13076656]]\n",
            "------ Epoch:  93\n",
            "Train loss:  0.06572069\n",
            "Validation loss:  [[0.1474796]]\n",
            "------ Epoch:  94\n",
            "Train loss:  0.06590187\n",
            "Validation loss:  [[0.13791646]]\n",
            "------ Epoch:  95\n",
            "Train loss:  0.07316377\n",
            "Validation loss:  [[0.15795772]]\n",
            "------ Epoch:  96\n",
            "Train loss:  0.07317536\n",
            "Validation loss:  [[0.14947458]]\n",
            "------ Epoch:  97\n",
            "Train loss:  0.08504103\n",
            "Validation loss:  [[0.17005146]]\n",
            "------ Epoch:  98\n",
            "Train loss:  0.08245231\n",
            "Validation loss:  [[0.16299509]]\n",
            "------ Epoch:  99\n",
            "Train loss:  0.0985373\n",
            "Validation loss:  [[0.17839022]]\n",
            "------ Epoch:  100\n",
            "Train loss:  0.08956097\n",
            "Validation loss:  [[0.17052352]]\n",
            "------ Epoch:  101\n",
            "Train loss:  0.105911344\n",
            "Validation loss:  [[0.17781524]]\n",
            "------ Epoch:  102\n",
            "Train loss:  0.09003736\n",
            "Validation loss:  [[0.1663001]]\n",
            "------ Epoch:  103\n",
            "Train loss:  0.10196562\n",
            "Validation loss:  [[0.1699459]]\n",
            "------ Epoch:  104\n",
            "Train loss:  0.08458207\n",
            "Validation loss:  [[0.15528736]]\n",
            "------ Epoch:  105\n",
            "Train loss:  0.09147227\n",
            "Validation loss:  [[0.16058034]]\n",
            "------ Epoch:  106\n",
            "Train loss:  0.07748413\n",
            "Validation loss:  [[0.14505549]]\n",
            "------ Epoch:  107\n",
            "Train loss:  0.08149498\n",
            "Validation loss:  [[0.1533673]]\n",
            "------ Epoch:  108\n",
            "Train loss:  0.071677126\n",
            "Validation loss:  [[0.13827786]]\n",
            "------ Epoch:  109\n",
            "Train loss:  0.07463758\n",
            "Validation loss:  [[0.14901064]]\n",
            "------ Epoch:  110\n",
            "Train loss:  0.067864336\n",
            "Validation loss:  [[0.13456653]]\n",
            "------ Epoch:  111\n",
            "Train loss:  0.07069715\n",
            "Validation loss:  [[0.14703959]]\n",
            "------ Epoch:  112\n",
            "Train loss:  0.065795116\n",
            "Validation loss:  [[0.1330162]]\n",
            "------ Epoch:  113\n",
            "Train loss:  0.06893789\n",
            "Validation loss:  [[0.14685316]]\n",
            "------ Epoch:  114\n",
            "Train loss:  0.06508262\n",
            "Validation loss:  [[0.13296603]]\n",
            "------ Epoch:  115\n",
            "Train loss:  0.06878702\n",
            "Validation loss:  [[0.14795946]]\n",
            "------ Epoch:  116\n",
            "Train loss:  0.06540635\n",
            "Validation loss:  [[0.1339905]]\n",
            "------ Epoch:  117\n",
            "Train loss:  0.06983684\n",
            "Validation loss:  [[0.14991647]]\n",
            "------ Epoch:  118\n",
            "Train loss:  0.06647519\n",
            "Validation loss:  [[0.13572717]]\n",
            "------ Epoch:  119\n",
            "Train loss:  0.07169792\n",
            "Validation loss:  [[0.15222868]]\n",
            "------ Epoch:  120\n",
            "Train loss:  0.06794443\n",
            "Validation loss:  [[0.1377271]]\n",
            "------ Epoch:  121\n",
            "Train loss:  0.07387246\n",
            "Validation loss:  [[0.15431783]]\n",
            "------ Epoch:  122\n",
            "Train loss:  0.069384985\n",
            "Validation loss:  [[0.13942066]]\n",
            "------ Epoch:  123\n",
            "Train loss:  0.07574303\n",
            "Validation loss:  [[0.15563013]]\n",
            "------ Epoch:  124\n",
            "Train loss:  0.07035585\n",
            "Validation loss:  [[0.1402659]]\n",
            "------ Epoch:  125\n",
            "Train loss:  0.07673745\n",
            "Validation loss:  [[0.15583912]]\n",
            "------ Epoch:  126\n",
            "Train loss:  0.07056344\n",
            "Validation loss:  [[0.14000073]]\n",
            "------ Epoch:  127\n",
            "Train loss:  0.076579675\n",
            "Validation loss:  [[0.15498565]]\n",
            "------ Epoch:  128\n",
            "Train loss:  0.06998508\n",
            "Validation loss:  [[0.1387702]]\n",
            "------ Epoch:  129\n",
            "Train loss:  0.075409204\n",
            "Validation loss:  [[0.1534184]]\n",
            "------ Epoch:  130\n",
            "Train loss:  0.06884265\n",
            "Validation loss:  [[0.1369977]]\n",
            "------ Epoch:  131\n",
            "Train loss:  0.073650375\n",
            "Validation loss:  [[0.15158921]]\n",
            "------ Epoch:  132\n",
            "Train loss:  0.06745716\n",
            "Validation loss:  [[0.13513339]]\n",
            "------ Epoch:  133\n",
            "Train loss:  0.07176272\n",
            "Validation loss:  [[0.14987057]]\n",
            "------ Epoch:  134\n",
            "Train loss:  0.066107504\n",
            "Validation loss:  [[0.1334879]]\n",
            "------ Epoch:  135\n",
            "Train loss:  0.070075706\n",
            "Validation loss:  [[0.14848201]]\n",
            "------ Epoch:  136\n",
            "Train loss:  0.064967886\n",
            "Validation loss:  [[0.13220651]]\n",
            "------ Epoch:  137\n",
            "Train loss:  0.068755515\n",
            "Validation loss:  [[0.14750405]]\n",
            "------ Epoch:  138\n",
            "Train loss:  0.06411283\n",
            "Validation loss:  [[0.13131283]]\n",
            "------ Epoch:  139\n",
            "Train loss:  0.0678441\n",
            "Validation loss:  [[0.14692234]]\n",
            "------ Epoch:  140\n",
            "Train loss:  0.06354542\n",
            "Validation loss:  [[0.13076311]]\n",
            "------ Epoch:  141\n",
            "Train loss:  0.06730712\n",
            "Validation loss:  [[0.14666586]]\n",
            "------ Epoch:  142\n",
            "Train loss:  0.06322614\n",
            "Validation loss:  [[0.13047932]]\n",
            "------ Epoch:  143\n",
            "Train loss:  0.06706894\n",
            "Validation loss:  [[0.14663576]]\n",
            "------ Epoch:  144\n",
            "Train loss:  0.06309092\n",
            "Validation loss:  [[0.13036972]]\n",
            "------ Epoch:  145\n",
            "Train loss:  0.06703289\n",
            "Validation loss:  [[0.14672269]]\n",
            "------ Epoch:  146\n",
            "Train loss:  0.0630648\n",
            "Validation loss:  [[0.13033986]]\n",
            "------ Epoch:  147\n",
            "Train loss:  0.06709583\n",
            "Validation loss:  [[0.14682227]]\n",
            "------ Epoch:  148\n",
            "Train loss:  0.063072965\n",
            "Validation loss:  [[0.13030319]]\n",
            "------ Epoch:  149\n",
            "Train loss:  0.067160994\n",
            "Validation loss:  [[0.14684817]]\n",
            "------ Epoch:  150\n",
            "Train loss:  0.06305016\n",
            "Validation loss:  [[0.13019052]]\n",
            "------ Epoch:  151\n",
            "Train loss:  0.06714985\n",
            "Validation loss:  [[0.14674358]]\n",
            "------ Epoch:  152\n",
            "Train loss:  0.06294999\n",
            "Validation loss:  [[0.12996054]]\n",
            "------ Epoch:  153\n",
            "Train loss:  0.06701403\n",
            "Validation loss:  [[0.1464864]]\n",
            "------ Epoch:  154\n",
            "Train loss:  0.06275038\n",
            "Validation loss:  [[0.12960255]]\n",
            "------ Epoch:  155\n",
            "Train loss:  0.06673945\n",
            "Validation loss:  [[0.14608766]]\n",
            "------ Epoch:  156\n",
            "Train loss:  0.062453568\n",
            "Validation loss:  [[0.12913352]]\n",
            "------ Epoch:  157\n",
            "Train loss:  0.066341765\n",
            "Validation loss:  [[0.14558157]]\n",
            "------ Epoch:  158\n",
            "Train loss:  0.062080044\n",
            "Validation loss:  [[0.1285888]]\n",
            "------ Epoch:  159\n",
            "Train loss:  0.06585852\n",
            "Validation loss:  [[0.14501412]]\n",
            "------ Epoch:  160\n",
            "Train loss:  0.061660536\n",
            "Validation loss:  [[0.1280091]]\n",
            "------ Epoch:  161\n",
            "Train loss:  0.06533349\n",
            "Validation loss:  [[0.144431]]\n",
            "------ Epoch:  162\n",
            "Train loss:  0.061226748\n",
            "Validation loss:  [[0.12743106]]\n",
            "------ Epoch:  163\n",
            "Train loss:  0.06480785\n",
            "Validation loss:  [[0.14386837]]\n",
            "------ Epoch:  164\n",
            "Train loss:  0.060805503\n",
            "Validation loss:  [[0.12688254]]\n",
            "------ Epoch:  165\n",
            "Train loss:  0.06431315\n",
            "Validation loss:  [[0.14335085]]\n",
            "------ Epoch:  166\n",
            "Train loss:  0.06041531\n",
            "Validation loss:  [[0.12637967]]\n",
            "------ Epoch:  167\n",
            "Train loss:  0.06386865\n",
            "Validation loss:  [[0.14288878]]\n",
            "------ Epoch:  168\n",
            "Train loss:  0.060065594\n",
            "Validation loss:  [[0.12592693]]\n",
            "------ Epoch:  169\n",
            "Train loss:  0.06348088\n",
            "Validation loss:  [[0.14248203]]\n",
            "------ Epoch:  170\n",
            "Train loss:  0.059757832\n",
            "Validation loss:  [[0.1255213]]\n",
            "------ Epoch:  171\n",
            "Train loss:  0.06314785\n",
            "Validation loss:  [[0.14212264]]\n",
            "------ Epoch:  172\n",
            "Train loss:  0.059487347\n",
            "Validation loss:  [[0.12515406]]\n",
            "------ Epoch:  173\n",
            "Train loss:  0.06286014\n",
            "Validation loss:  [[0.14179724]]\n",
            "------ Epoch:  174\n",
            "Train loss:  0.05924568\n",
            "Validation loss:  [[0.12481327]]\n",
            "------ Epoch:  175\n",
            "Train loss:  0.06260439\n",
            "Validation loss:  [[0.14149067]]\n",
            "------ Epoch:  176\n",
            "Train loss:  0.059022658\n",
            "Validation loss:  [[0.12448671]]\n",
            "------ Epoch:  177\n",
            "Train loss:  0.06236713\n",
            "Validation loss:  [[0.14118928]]\n",
            "------ Epoch:  178\n",
            "Train loss:  0.058808304\n",
            "Validation loss:  [[0.12416333]]\n",
            "------ Epoch:  179\n",
            "Train loss:  0.062135484\n",
            "Validation loss:  [[0.14088139]]\n",
            "------ Epoch:  180\n",
            "Train loss:  0.05859422\n",
            "Validation loss:  [[0.12383474]]\n",
            "------ Epoch:  181\n",
            "Train loss:  0.061899576\n",
            "Validation loss:  [[0.14055972]]\n",
            "------ Epoch:  182\n",
            "Train loss:  0.058373928\n",
            "Validation loss:  [[0.12349547]]\n",
            "------ Epoch:  183\n",
            "Train loss:  0.06165267\n",
            "Validation loss:  [[0.1402197]]\n",
            "------ Epoch:  184\n",
            "Train loss:  0.058144193\n",
            "Validation loss:  [[0.12314329]]\n",
            "------ Epoch:  185\n",
            "Train loss:  0.061391745\n",
            "Validation loss:  [[0.13986138]]\n",
            "------ Epoch:  186\n",
            "Train loss:  0.057904013\n",
            "Validation loss:  [[0.12277885]]\n",
            "------ Epoch:  187\n",
            "Train loss:  0.061117236\n",
            "Validation loss:  [[0.13948748]]\n",
            "------ Epoch:  188\n",
            "Train loss:  0.057654522\n",
            "Validation loss:  [[0.12240446]]\n",
            "------ Epoch:  189\n",
            "Train loss:  0.06083168\n",
            "Validation loss:  [[0.13910186]]\n",
            "------ Epoch:  190\n",
            "Train loss:  0.057398506\n",
            "Validation loss:  [[0.12202422]]\n",
            "------ Epoch:  191\n",
            "Train loss:  0.06053921\n",
            "Validation loss:  [[0.13871035]]\n",
            "------ Epoch:  192\n",
            "Train loss:  0.057139244\n",
            "Validation loss:  [[0.12164205]]\n",
            "------ Epoch:  193\n",
            "Train loss:  0.06024437\n",
            "Validation loss:  [[0.13831681]]\n",
            "------ Epoch:  194\n",
            "Train loss:  0.056879506\n",
            "Validation loss:  [[0.12126075]]\n",
            "------ Epoch:  195\n",
            "Train loss:  0.059950445\n",
            "Validation loss:  [[0.13792485]]\n",
            "------ Epoch:  196\n",
            "Train loss:  0.05662198\n",
            "Validation loss:  [[0.12088338]]\n",
            "------ Epoch:  197\n",
            "Train loss:  0.059661087\n",
            "Validation loss:  [[0.13753812]]\n",
            "------ Epoch:  198\n",
            "Train loss:  0.056369033\n",
            "Validation loss:  [[0.12051178]]\n",
            "------ Epoch:  199\n",
            "Train loss:  0.05937833\n",
            "Validation loss:  [[0.13715735]]\n",
            "------ Epoch:  200\n",
            "Train loss:  0.056121405\n",
            "Validation loss:  [[0.12014645]]\n",
            "------ Epoch:  201\n",
            "Train loss:  0.059102926\n",
            "Validation loss:  [[0.13678306]]\n",
            "------ Epoch:  202\n",
            "Train loss:  0.05587931\n",
            "Validation loss:  [[0.11978722]]\n",
            "------ Epoch:  203\n",
            "Train loss:  0.05883465\n",
            "Validation loss:  [[0.13641444]]\n",
            "------ Epoch:  204\n",
            "Train loss:  0.05564242\n",
            "Validation loss:  [[0.11943348]]\n",
            "------ Epoch:  205\n",
            "Train loss:  0.05857298\n",
            "Validation loss:  [[0.13605031]]\n",
            "------ Epoch:  206\n",
            "Train loss:  0.055410147\n",
            "Validation loss:  [[0.11908413]]\n",
            "------ Epoch:  207\n",
            "Train loss:  0.05831654\n",
            "Validation loss:  [[0.13568987]]\n",
            "------ Epoch:  208\n",
            "Train loss:  0.055181388\n",
            "Validation loss:  [[0.11873832]]\n",
            "------ Epoch:  209\n",
            "Train loss:  0.058064245\n",
            "Validation loss:  [[0.13533102]]\n",
            "------ Epoch:  210\n",
            "Train loss:  0.054954942\n",
            "Validation loss:  [[0.11839428]]\n",
            "------ Epoch:  211\n",
            "Train loss:  0.05781377\n",
            "Validation loss:  [[0.13497238]]\n",
            "------ Epoch:  212\n",
            "Train loss:  0.054729797\n",
            "Validation loss:  [[0.11805147]]\n",
            "------ Epoch:  213\n",
            "Train loss:  0.057564612\n",
            "Validation loss:  [[0.13461329]]\n",
            "------ Epoch:  214\n",
            "Train loss:  0.054505076\n",
            "Validation loss:  [[0.11770876]]\n",
            "------ Epoch:  215\n",
            "Train loss:  0.057315722\n",
            "Validation loss:  [[0.1342532]]\n",
            "------ Epoch:  216\n",
            "Train loss:  0.054280583\n",
            "Validation loss:  [[0.11736645]]\n",
            "------ Epoch:  217\n",
            "Train loss:  0.057066713\n",
            "Validation loss:  [[0.13389158]]\n",
            "------ Epoch:  218\n",
            "Train loss:  0.054055844\n",
            "Validation loss:  [[0.11702392]]\n",
            "------ Epoch:  219\n",
            "Train loss:  0.05681687\n",
            "Validation loss:  [[0.1335287]]\n",
            "------ Epoch:  220\n",
            "Train loss:  0.053830504\n",
            "Validation loss:  [[0.11668112]]\n",
            "------ Epoch:  221\n",
            "Train loss:  0.05656648\n",
            "Validation loss:  [[0.13316493]]\n",
            "------ Epoch:  222\n",
            "Train loss:  0.053605225\n",
            "Validation loss:  [[0.11633898]]\n",
            "------ Epoch:  223\n",
            "Train loss:  0.056316208\n",
            "Validation loss:  [[0.13280028]]\n",
            "------ Epoch:  224\n",
            "Train loss:  0.05337973\n",
            "Validation loss:  [[0.1159968]]\n",
            "------ Epoch:  225\n",
            "Train loss:  0.056065574\n",
            "Validation loss:  [[0.13243538]]\n",
            "------ Epoch:  226\n",
            "Train loss:  0.053154245\n",
            "Validation loss:  [[0.1156556]]\n",
            "------ Epoch:  227\n",
            "Train loss:  0.05581534\n",
            "Validation loss:  [[0.13207074]]\n",
            "------ Epoch:  228\n",
            "Train loss:  0.052929297\n",
            "Validation loss:  [[0.1153154]]\n",
            "------ Epoch:  229\n",
            "Train loss:  0.055565562\n",
            "Validation loss:  [[0.13170637]]\n",
            "------ Epoch:  230\n",
            "Train loss:  0.052704964\n",
            "Validation loss:  [[0.11497647]]\n",
            "------ Epoch:  231\n",
            "Train loss:  0.055316713\n",
            "Validation loss:  [[0.1313431]]\n",
            "------ Epoch:  232\n",
            "Train loss:  0.052481268\n",
            "Validation loss:  [[0.11463907]]\n",
            "------ Epoch:  233\n",
            "Train loss:  0.05506873\n",
            "Validation loss:  [[0.13098085]]\n",
            "------ Epoch:  234\n",
            "Train loss:  0.052258443\n",
            "Validation loss:  [[0.11430317]]\n",
            "------ Epoch:  235\n",
            "Train loss:  0.054821946\n",
            "Validation loss:  [[0.13062009]]\n",
            "------ Epoch:  236\n",
            "Train loss:  0.05203674\n",
            "Validation loss:  [[0.11396915]]\n",
            "------ Epoch:  237\n",
            "Train loss:  0.05457632\n",
            "Validation loss:  [[0.13026047]]\n",
            "------ Epoch:  238\n",
            "Train loss:  0.05181609\n",
            "Validation loss:  [[0.11363664]]\n",
            "------ Epoch:  239\n",
            "Train loss:  0.05433209\n",
            "Validation loss:  [[0.12990203]]\n",
            "------ Epoch:  240\n",
            "Train loss:  0.051596243\n",
            "Validation loss:  [[0.11330585]]\n",
            "------ Epoch:  241\n",
            "Train loss:  0.054088794\n",
            "Validation loss:  [[0.12954476]]\n",
            "------ Epoch:  242\n",
            "Train loss:  0.0513772\n",
            "Validation loss:  [[0.11297628]]\n",
            "------ Epoch:  243\n",
            "Train loss:  0.053846326\n",
            "Validation loss:  [[0.12918818]]\n",
            "------ Epoch:  244\n",
            "Train loss:  0.05115879\n",
            "Validation loss:  [[0.11264825]]\n",
            "------ Epoch:  245\n",
            "Train loss:  0.053604532\n",
            "Validation loss:  [[0.12883277]]\n",
            "------ Epoch:  246\n",
            "Train loss:  0.05094083\n",
            "Validation loss:  [[0.11232147]]\n",
            "------ Epoch:  247\n",
            "Train loss:  0.05336346\n",
            "Validation loss:  [[0.12847804]]\n",
            "------ Epoch:  248\n",
            "Train loss:  0.05072353\n",
            "Validation loss:  [[0.11199608]]\n",
            "------ Epoch:  249\n",
            "Train loss:  0.053122934\n",
            "Validation loss:  [[0.12812394]]\n",
            "------ Epoch:  250\n",
            "Train loss:  0.050506473\n",
            "Validation loss:  [[0.11167164]]\n",
            "------ Epoch:  251\n",
            "Train loss:  0.052882824\n",
            "Validation loss:  [[0.12777112]]\n",
            "------ Epoch:  252\n",
            "Train loss:  0.050290283\n",
            "Validation loss:  [[0.11134882]]\n",
            "------ Epoch:  253\n",
            "Train loss:  0.052643467\n",
            "Validation loss:  [[0.1274189]]\n",
            "------ Epoch:  254\n",
            "Train loss:  0.050074354\n",
            "Validation loss:  [[0.11102707]]\n",
            "------ Epoch:  255\n",
            "Train loss:  0.05240436\n",
            "Validation loss:  [[0.12706722]]\n",
            "------ Epoch:  256\n",
            "Train loss:  0.049858347\n",
            "Validation loss:  [[0.1107062]]\n",
            "------ Epoch:  257\n",
            "Train loss:  0.05216522\n",
            "Validation loss:  [[0.1267161]]\n",
            "------ Epoch:  258\n",
            "Train loss:  0.049642768\n",
            "Validation loss:  [[0.11038663]]\n",
            "------ Epoch:  259\n",
            "Train loss:  0.051926408\n",
            "Validation loss:  [[0.12636581]]\n",
            "------ Epoch:  260\n",
            "Train loss:  0.04942758\n",
            "Validation loss:  [[0.11006854]]\n",
            "------ Epoch:  261\n",
            "Train loss:  0.051688265\n",
            "Validation loss:  [[0.12601648]]\n",
            "------ Epoch:  262\n",
            "Train loss:  0.04921255\n",
            "Validation loss:  [[0.1097513]]\n",
            "------ Epoch:  263\n",
            "Train loss:  0.05145014\n",
            "Validation loss:  [[0.12566818]]\n",
            "------ Epoch:  264\n",
            "Train loss:  0.04899803\n",
            "Validation loss:  [[0.10943572]]\n",
            "------ Epoch:  265\n",
            "Train loss:  0.051212747\n",
            "Validation loss:  [[0.12532085]]\n",
            "------ Epoch:  266\n",
            "Train loss:  0.048783936\n",
            "Validation loss:  [[0.10912145]]\n",
            "------ Epoch:  267\n",
            "Train loss:  0.050975744\n",
            "Validation loss:  [[0.12497436]]\n",
            "------ Epoch:  268\n",
            "Train loss:  0.04857029\n",
            "Validation loss:  [[0.1088085]]\n",
            "------ Epoch:  269\n",
            "Train loss:  0.050739218\n",
            "Validation loss:  [[0.12462887]]\n",
            "------ Epoch:  270\n",
            "Train loss:  0.048356906\n",
            "Validation loss:  [[0.10849677]]\n",
            "------ Epoch:  271\n",
            "Train loss:  0.05050296\n",
            "Validation loss:  [[0.12428425]]\n",
            "------ Epoch:  272\n",
            "Train loss:  0.048143856\n",
            "Validation loss:  [[0.10818654]]\n",
            "------ Epoch:  273\n",
            "Train loss:  0.05026729\n",
            "Validation loss:  [[0.12394088]]\n",
            "------ Epoch:  274\n",
            "Train loss:  0.047931265\n",
            "Validation loss:  [[0.10787757]]\n",
            "------ Epoch:  275\n",
            "Train loss:  0.050031908\n",
            "Validation loss:  [[0.12359825]]\n",
            "------ Epoch:  276\n",
            "Train loss:  0.04771908\n",
            "Validation loss:  [[0.1075703]]\n",
            "------ Epoch:  277\n",
            "Train loss:  0.049797263\n",
            "Validation loss:  [[0.12325694]]\n",
            "------ Epoch:  278\n",
            "Train loss:  0.04750732\n",
            "Validation loss:  [[0.1072641]]\n",
            "------ Epoch:  279\n",
            "Train loss:  0.04956281\n",
            "Validation loss:  [[0.12291666]]\n",
            "------ Epoch:  280\n",
            "Train loss:  0.04729588\n",
            "Validation loss:  [[0.1069594]]\n",
            "------ Epoch:  281\n",
            "Train loss:  0.049329087\n",
            "Validation loss:  [[0.12257734]]\n",
            "------ Epoch:  282\n",
            "Train loss:  0.04708499\n",
            "Validation loss:  [[0.10665607]]\n",
            "------ Epoch:  283\n",
            "Train loss:  0.049095623\n",
            "Validation loss:  [[0.12223911]]\n",
            "------ Epoch:  284\n",
            "Train loss:  0.046874404\n",
            "Validation loss:  [[0.10635419]]\n",
            "------ Epoch:  285\n",
            "Train loss:  0.04886274\n",
            "Validation loss:  [[0.12190206]]\n",
            "------ Epoch:  286\n",
            "Train loss:  0.046664063\n",
            "Validation loss:  [[0.10605387]]\n",
            "------ Epoch:  287\n",
            "Train loss:  0.048630055\n",
            "Validation loss:  [[0.12156558]]\n",
            "------ Epoch:  288\n",
            "Train loss:  0.046454094\n",
            "Validation loss:  [[0.10575458]]\n",
            "------ Epoch:  289\n",
            "Train loss:  0.048397694\n",
            "Validation loss:  [[0.1212306]]\n",
            "------ Epoch:  290\n",
            "Train loss:  0.046244606\n",
            "Validation loss:  [[0.10545696]]\n",
            "------ Epoch:  291\n",
            "Train loss:  0.048165932\n",
            "Validation loss:  [[0.12089644]]\n",
            "------ Epoch:  292\n",
            "Train loss:  0.046035413\n",
            "Validation loss:  [[0.10516058]]\n",
            "------ Epoch:  293\n",
            "Train loss:  0.04793463\n",
            "Validation loss:  [[0.1205635]]\n",
            "------ Epoch:  294\n",
            "Train loss:  0.045826755\n",
            "Validation loss:  [[0.10486586]]\n",
            "------ Epoch:  295\n",
            "Train loss:  0.04770388\n",
            "Validation loss:  [[0.12023145]]\n",
            "------ Epoch:  296\n",
            "Train loss:  0.045618277\n",
            "Validation loss:  [[0.10457234]]\n",
            "------ Epoch:  297\n",
            "Train loss:  0.04747341\n",
            "Validation loss:  [[0.11990047]]\n",
            "------ Epoch:  298\n",
            "Train loss:  0.04541019\n",
            "Validation loss:  [[0.10428027]]\n",
            "------ Epoch:  299\n",
            "Train loss:  0.047243204\n",
            "Validation loss:  [[0.11957045]]\n",
            "------ Epoch:  300\n",
            "Train loss:  0.045202646\n",
            "Validation loss:  [[0.10398968]]\n",
            "------ Epoch:  301\n",
            "Train loss:  0.047013585\n",
            "Validation loss:  [[0.11924145]]\n",
            "------ Epoch:  302\n",
            "Train loss:  0.044995315\n",
            "Validation loss:  [[0.10370043]]\n",
            "------ Epoch:  303\n",
            "Train loss:  0.04678452\n",
            "Validation loss:  [[0.11891346]]\n",
            "------ Epoch:  304\n",
            "Train loss:  0.0447882\n",
            "Validation loss:  [[0.10341246]]\n",
            "------ Epoch:  305\n",
            "Train loss:  0.04655554\n",
            "Validation loss:  [[0.11858664]]\n",
            "------ Epoch:  306\n",
            "Train loss:  0.044581555\n",
            "Validation loss:  [[0.10312615]]\n",
            "------ Epoch:  307\n",
            "Train loss:  0.046327133\n",
            "Validation loss:  [[0.11826087]]\n",
            "------ Epoch:  308\n",
            "Train loss:  0.044375453\n",
            "Validation loss:  [[0.10284121]]\n",
            "------ Epoch:  309\n",
            "Train loss:  0.046099402\n",
            "Validation loss:  [[0.11793607]]\n",
            "------ Epoch:  310\n",
            "Train loss:  0.04416959\n",
            "Validation loss:  [[0.10255766]]\n",
            "------ Epoch:  311\n",
            "Train loss:  0.04587201\n",
            "Validation loss:  [[0.11761226]]\n",
            "------ Epoch:  312\n",
            "Train loss:  0.043964107\n",
            "Validation loss:  [[0.10227558]]\n",
            "------ Epoch:  313\n",
            "Train loss:  0.045645054\n",
            "Validation loss:  [[0.11728951]]\n",
            "------ Epoch:  314\n",
            "Train loss:  0.04375918\n",
            "Validation loss:  [[0.10199498]]\n",
            "------ Epoch:  315\n",
            "Train loss:  0.045418683\n",
            "Validation loss:  [[0.1169683]]\n",
            "------ Epoch:  316\n",
            "Train loss:  0.0435547\n",
            "Validation loss:  [[0.10171606]]\n",
            "------ Epoch:  317\n",
            "Train loss:  0.04519287\n",
            "Validation loss:  [[0.11664762]]\n",
            "------ Epoch:  318\n",
            "Train loss:  0.0433505\n",
            "Validation loss:  [[0.10143831]]\n",
            "------ Epoch:  319\n",
            "Train loss:  0.044967532\n",
            "Validation loss:  [[0.11632814]]\n",
            "------ Epoch:  320\n",
            "Train loss:  0.0431468\n",
            "Validation loss:  [[0.10116187]]\n",
            "------ Epoch:  321\n",
            "Train loss:  0.044742424\n",
            "Validation loss:  [[0.11600937]]\n",
            "------ Epoch:  322\n",
            "Train loss:  0.042943288\n",
            "Validation loss:  [[0.10088702]]\n",
            "------ Epoch:  323\n",
            "Train loss:  0.044517983\n",
            "Validation loss:  [[0.11569213]]\n",
            "------ Epoch:  324\n",
            "Train loss:  0.042740464\n",
            "Validation loss:  [[0.10061371]]\n",
            "------ Epoch:  325\n",
            "Train loss:  0.044294067\n",
            "Validation loss:  [[0.11537549]]\n",
            "------ Epoch:  326\n",
            "Train loss:  0.04253792\n",
            "Validation loss:  [[0.10034173]]\n",
            "------ Epoch:  327\n",
            "Train loss:  0.044070568\n",
            "Validation loss:  [[0.11505995]]\n",
            "------ Epoch:  328\n",
            "Train loss:  0.042335648\n",
            "Validation loss:  [[0.10007113]]\n",
            "------ Epoch:  329\n",
            "Train loss:  0.0438478\n",
            "Validation loss:  [[0.11474581]]\n",
            "------ Epoch:  330\n",
            "Train loss:  0.042134326\n",
            "Validation loss:  [[0.09980238]]\n",
            "------ Epoch:  331\n",
            "Train loss:  0.04362562\n",
            "Validation loss:  [[0.11443265]]\n",
            "------ Epoch:  332\n",
            "Train loss:  0.041933194\n",
            "Validation loss:  [[0.09953482]]\n",
            "------ Epoch:  333\n",
            "Train loss:  0.043403875\n",
            "Validation loss:  [[0.11412057]]\n",
            "------ Epoch:  334\n",
            "Train loss:  0.041732527\n",
            "Validation loss:  [[0.09926885]]\n",
            "------ Epoch:  335\n",
            "Train loss:  0.043182917\n",
            "Validation loss:  [[0.11380966]]\n",
            "------ Epoch:  336\n",
            "Train loss:  0.041532587\n",
            "Validation loss:  [[0.09900454]]\n",
            "------ Epoch:  337\n",
            "Train loss:  0.04296272\n",
            "Validation loss:  [[0.11349977]]\n",
            "------ Epoch:  338\n",
            "Train loss:  0.041333135\n",
            "Validation loss:  [[0.09874151]]\n",
            "------ Epoch:  339\n",
            "Train loss:  0.042742826\n",
            "Validation loss:  [[0.11319092]]\n",
            "------ Epoch:  340\n",
            "Train loss:  0.041134007\n",
            "Validation loss:  [[0.09847996]]\n",
            "------ Epoch:  341\n",
            "Train loss:  0.042523663\n",
            "Validation loss:  [[0.11288317]]\n",
            "------ Epoch:  342\n",
            "Train loss:  0.040935382\n",
            "Validation loss:  [[0.09821995]]\n",
            "------ Epoch:  343\n",
            "Train loss:  0.042304974\n",
            "Validation loss:  [[0.11257605]]\n",
            "------ Epoch:  344\n",
            "Train loss:  0.040737156\n",
            "Validation loss:  [[0.09796119]]\n",
            "------ Epoch:  345\n",
            "Train loss:  0.042086914\n",
            "Validation loss:  [[0.11227039]]\n",
            "------ Epoch:  346\n",
            "Train loss:  0.04053954\n",
            "Validation loss:  [[0.097704]]\n",
            "------ Epoch:  347\n",
            "Train loss:  0.04186953\n",
            "Validation loss:  [[0.11196563]]\n",
            "------ Epoch:  348\n",
            "Train loss:  0.040342398\n",
            "Validation loss:  [[0.09744824]]\n",
            "------ Epoch:  349\n",
            "Train loss:  0.041652575\n",
            "Validation loss:  [[0.1116613]]\n",
            "------ Epoch:  350\n",
            "Train loss:  0.040145595\n",
            "Validation loss:  [[0.09719387]]\n",
            "------ Epoch:  351\n",
            "Train loss:  0.04143628\n",
            "Validation loss:  [[0.11135866]]\n",
            "------ Epoch:  352\n",
            "Train loss:  0.039949488\n",
            "Validation loss:  [[0.09694109]]\n",
            "------ Epoch:  353\n",
            "Train loss:  0.04122083\n",
            "Validation loss:  [[0.11105704]]\n",
            "------ Epoch:  354\n",
            "Train loss:  0.039753877\n",
            "Validation loss:  [[0.09668973]]\n",
            "------ Epoch:  355\n",
            "Train loss:  0.041005857\n",
            "Validation loss:  [[0.11075625]]\n",
            "------ Epoch:  356\n",
            "Train loss:  0.039558817\n",
            "Validation loss:  [[0.09643987]]\n",
            "------ Epoch:  357\n",
            "Train loss:  0.040791593\n",
            "Validation loss:  [[0.1104565]]\n",
            "------ Epoch:  358\n",
            "Train loss:  0.039364316\n",
            "Validation loss:  [[0.09619127]]\n",
            "------ Epoch:  359\n",
            "Train loss:  0.040577993\n",
            "Validation loss:  [[0.11015779]]\n",
            "------ Epoch:  360\n",
            "Train loss:  0.039170224\n",
            "Validation loss:  [[0.09594446]]\n",
            "------ Epoch:  361\n",
            "Train loss:  0.040365074\n",
            "Validation loss:  [[0.10985992]]\n",
            "------ Epoch:  362\n",
            "Train loss:  0.038976837\n",
            "Validation loss:  [[0.09569891]]\n",
            "------ Epoch:  363\n",
            "Train loss:  0.04015294\n",
            "Validation loss:  [[0.10956348]]\n",
            "------ Epoch:  364\n",
            "Train loss:  0.03878392\n",
            "Validation loss:  [[0.09545483]]\n",
            "------ Epoch:  365\n",
            "Train loss:  0.039941393\n",
            "Validation loss:  [[0.10926793]]\n",
            "------ Epoch:  366\n",
            "Train loss:  0.03859166\n",
            "Validation loss:  [[0.09521233]]\n",
            "------ Epoch:  367\n",
            "Train loss:  0.039730668\n",
            "Validation loss:  [[0.10897359]]\n",
            "------ Epoch:  368\n",
            "Train loss:  0.038400095\n",
            "Validation loss:  [[0.09497125]]\n",
            "------ Epoch:  369\n",
            "Train loss:  0.039520726\n",
            "Validation loss:  [[0.1086799]]\n",
            "------ Epoch:  370\n",
            "Train loss:  0.03820888\n",
            "Validation loss:  [[0.09473154]]\n",
            "------ Epoch:  371\n",
            "Train loss:  0.039311334\n",
            "Validation loss:  [[0.1083874]]\n",
            "------ Epoch:  372\n",
            "Train loss:  0.038018372\n",
            "Validation loss:  [[0.09449335]]\n",
            "------ Epoch:  373\n",
            "Train loss:  0.039102774\n",
            "Validation loss:  [[0.10809589]]\n",
            "------ Epoch:  374\n",
            "Train loss:  0.03782853\n",
            "Validation loss:  [[0.0942567]]\n",
            "------ Epoch:  375\n",
            "Train loss:  0.038895063\n",
            "Validation loss:  [[0.10780551]]\n",
            "------ Epoch:  376\n",
            "Train loss:  0.037639182\n",
            "Validation loss:  [[0.09402139]]\n",
            "------ Epoch:  377\n",
            "Train loss:  0.038687754\n",
            "Validation loss:  [[0.10751608]]\n",
            "------ Epoch:  378\n",
            "Train loss:  0.037450273\n",
            "Validation loss:  [[0.09378745]]\n",
            "------ Epoch:  379\n",
            "Train loss:  0.038481362\n",
            "Validation loss:  [[0.10722739]]\n",
            "------ Epoch:  380\n",
            "Train loss:  0.037262216\n",
            "Validation loss:  [[0.093555]]\n",
            "------ Epoch:  381\n",
            "Train loss:  0.038275614\n",
            "Validation loss:  [[0.10693986]]\n",
            "------ Epoch:  382\n",
            "Train loss:  0.03707464\n",
            "Validation loss:  [[0.09332398]]\n",
            "------ Epoch:  383\n",
            "Train loss:  0.03807082\n",
            "Validation loss:  [[0.10665348]]\n",
            "------ Epoch:  384\n",
            "Train loss:  0.036887772\n",
            "Validation loss:  [[0.09309442]]\n",
            "------ Epoch:  385\n",
            "Train loss:  0.03786678\n",
            "Validation loss:  [[0.10636794]]\n",
            "------ Epoch:  386\n",
            "Train loss:  0.036701564\n",
            "Validation loss:  [[0.09286635]]\n",
            "------ Epoch:  387\n",
            "Train loss:  0.03766328\n",
            "Validation loss:  [[0.10608348]]\n",
            "------ Epoch:  388\n",
            "Train loss:  0.036515832\n",
            "Validation loss:  [[0.09263957]]\n",
            "------ Epoch:  389\n",
            "Train loss:  0.037460797\n",
            "Validation loss:  [[0.10579998]]\n",
            "------ Epoch:  390\n",
            "Train loss:  0.03633082\n",
            "Validation loss:  [[0.09241437]]\n",
            "------ Epoch:  391\n",
            "Train loss:  0.03725903\n",
            "Validation loss:  [[0.10551739]]\n",
            "------ Epoch:  392\n",
            "Train loss:  0.036146346\n",
            "Validation loss:  [[0.09219037]]\n",
            "------ Epoch:  393\n",
            "Train loss:  0.037057932\n",
            "Validation loss:  [[0.10523593]]\n",
            "------ Epoch:  394\n",
            "Train loss:  0.035962615\n",
            "Validation loss:  [[0.09196803]]\n",
            "------ Epoch:  395\n",
            "Train loss:  0.036857776\n",
            "Validation loss:  [[0.10495531]]\n",
            "------ Epoch:  396\n",
            "Train loss:  0.035779413\n",
            "Validation loss:  [[0.09174695]]\n",
            "------ Epoch:  397\n",
            "Train loss:  0.03665825\n",
            "Validation loss:  [[0.10467581]]\n",
            "------ Epoch:  398\n",
            "Train loss:  0.035596814\n",
            "Validation loss:  [[0.09152739]]\n",
            "------ Epoch:  399\n",
            "Train loss:  0.03645964\n",
            "Validation loss:  [[0.1043973]]\n",
            "------ Epoch:  400\n",
            "Train loss:  0.035415042\n",
            "Validation loss:  [[0.0913092]]\n",
            "------ Epoch:  401\n",
            "Train loss:  0.036261782\n",
            "Validation loss:  [[0.10411984]]\n",
            "------ Epoch:  402\n",
            "Train loss:  0.03523399\n",
            "Validation loss:  [[0.09109244]]\n",
            "------ Epoch:  403\n",
            "Train loss:  0.0360649\n",
            "Validation loss:  [[0.10384303]]\n",
            "------ Epoch:  404\n",
            "Train loss:  0.035053395\n",
            "Validation loss:  [[0.09087685]]\n",
            "------ Epoch:  405\n",
            "Train loss:  0.0358685\n",
            "Validation loss:  [[0.10356736]]\n",
            "------ Epoch:  406\n",
            "Train loss:  0.034873378\n",
            "Validation loss:  [[0.09066279]]\n",
            "------ Epoch:  407\n",
            "Train loss:  0.035672974\n",
            "Validation loss:  [[0.10329256]]\n",
            "------ Epoch:  408\n",
            "Train loss:  0.034694083\n",
            "Validation loss:  [[0.09045015]]\n",
            "------ Epoch:  409\n",
            "Train loss:  0.035478428\n",
            "Validation loss:  [[0.10301899]]\n",
            "------ Epoch:  410\n",
            "Train loss:  0.034515675\n",
            "Validation loss:  [[0.09023898]]\n",
            "------ Epoch:  411\n",
            "Train loss:  0.03528468\n",
            "Validation loss:  [[0.10274628]]\n",
            "------ Epoch:  412\n",
            "Train loss:  0.034337837\n",
            "Validation loss:  [[0.09002917]]\n",
            "------ Epoch:  413\n",
            "Train loss:  0.035091903\n",
            "Validation loss:  [[0.10247478]]\n",
            "------ Epoch:  414\n",
            "Train loss:  0.034160633\n",
            "Validation loss:  [[0.08982082]]\n",
            "------ Epoch:  415\n",
            "Train loss:  0.034899816\n",
            "Validation loss:  [[0.10220392]]\n",
            "------ Epoch:  416\n",
            "Train loss:  0.033984184\n",
            "Validation loss:  [[0.08961377]]\n",
            "------ Epoch:  417\n",
            "Train loss:  0.034708638\n",
            "Validation loss:  [[0.10193413]]\n",
            "------ Epoch:  418\n",
            "Train loss:  0.03380852\n",
            "Validation loss:  [[0.0894081]]\n",
            "------ Epoch:  419\n",
            "Train loss:  0.034518346\n",
            "Validation loss:  [[0.10166543]]\n",
            "------ Epoch:  420\n",
            "Train loss:  0.033633493\n",
            "Validation loss:  [[0.08920377]]\n",
            "------ Epoch:  421\n",
            "Train loss:  0.03432885\n",
            "Validation loss:  [[0.10139751]]\n",
            "------ Epoch:  422\n",
            "Train loss:  0.033459015\n",
            "Validation loss:  [[0.08900063]]\n",
            "------ Epoch:  423\n",
            "Train loss:  0.03414007\n",
            "Validation loss:  [[0.10113066]]\n",
            "------ Epoch:  424\n",
            "Train loss:  0.03328544\n",
            "Validation loss:  [[0.08879916]]\n",
            "------ Epoch:  425\n",
            "Train loss:  0.033952538\n",
            "Validation loss:  [[0.10086485]]\n",
            "------ Epoch:  426\n",
            "Train loss:  0.03311258\n",
            "Validation loss:  [[0.08859909]]\n",
            "------ Epoch:  427\n",
            "Train loss:  0.03376586\n",
            "Validation loss:  [[0.10060021]]\n",
            "------ Epoch:  428\n",
            "Train loss:  0.032940485\n",
            "Validation loss:  [[0.08840013]]\n",
            "------ Epoch:  429\n",
            "Train loss:  0.03357986\n",
            "Validation loss:  [[0.10033611]]\n",
            "------ Epoch:  430\n",
            "Train loss:  0.03276902\n",
            "Validation loss:  [[0.08820255]]\n",
            "------ Epoch:  431\n",
            "Train loss:  0.033394847\n",
            "Validation loss:  [[0.10007338]]\n",
            "------ Epoch:  432\n",
            "Train loss:  0.03259831\n",
            "Validation loss:  [[0.0880065]]\n",
            "------ Epoch:  433\n",
            "Train loss:  0.033210844\n",
            "Validation loss:  [[0.09981158]]\n",
            "------ Epoch:  434\n",
            "Train loss:  0.032428395\n",
            "Validation loss:  [[0.08781165]]\n",
            "------ Epoch:  435\n",
            "Train loss:  0.03302757\n",
            "Validation loss:  [[0.09955048]]\n",
            "------ Epoch:  436\n",
            "Train loss:  0.032259084\n",
            "Validation loss:  [[0.08761816]]\n",
            "------ Epoch:  437\n",
            "Train loss:  0.03284508\n",
            "Validation loss:  [[0.09929042]]\n",
            "------ Epoch:  438\n",
            "Train loss:  0.03209041\n",
            "Validation loss:  [[0.08742597]]\n",
            "------ Epoch:  439\n",
            "Train loss:  0.032663483\n",
            "Validation loss:  [[0.09903118]]\n",
            "------ Epoch:  440\n",
            "Train loss:  0.03192257\n",
            "Validation loss:  [[0.08723508]]\n",
            "------ Epoch:  441\n",
            "Train loss:  0.03248281\n",
            "Validation loss:  [[0.09877281]]\n",
            "------ Epoch:  442\n",
            "Train loss:  0.031755365\n",
            "Validation loss:  [[0.08704541]]\n",
            "------ Epoch:  443\n",
            "Train loss:  0.032302957\n",
            "Validation loss:  [[0.09851526]]\n",
            "------ Epoch:  444\n",
            "Train loss:  0.03158893\n",
            "Validation loss:  [[0.08685717]]\n",
            "------ Epoch:  445\n",
            "Train loss:  0.032124016\n",
            "Validation loss:  [[0.09825901]]\n",
            "------ Epoch:  446\n",
            "Train loss:  0.031423256\n",
            "Validation loss:  [[0.08667018]]\n",
            "------ Epoch:  447\n",
            "Train loss:  0.03194597\n",
            "Validation loss:  [[0.09800333]]\n",
            "------ Epoch:  448\n",
            "Train loss:  0.031258285\n",
            "Validation loss:  [[0.08648456]]\n",
            "------ Epoch:  449\n",
            "Train loss:  0.03176874\n",
            "Validation loss:  [[0.09774871]]\n",
            "------ Epoch:  450\n",
            "Train loss:  0.031094074\n",
            "Validation loss:  [[0.08630013]]\n",
            "------ Epoch:  451\n",
            "Train loss:  0.03159256\n",
            "Validation loss:  [[0.09749517]]\n",
            "------ Epoch:  452\n",
            "Train loss:  0.030930532\n",
            "Validation loss:  [[0.0861171]]\n",
            "------ Epoch:  453\n",
            "Train loss:  0.031417243\n",
            "Validation loss:  [[0.09724239]]\n",
            "------ Epoch:  454\n",
            "Train loss:  0.030767849\n",
            "Validation loss:  [[0.08593532]]\n",
            "------ Epoch:  455\n",
            "Train loss:  0.031242741\n",
            "Validation loss:  [[0.09699041]]\n",
            "------ Epoch:  456\n",
            "Train loss:  0.030605717\n",
            "Validation loss:  [[0.08575474]]\n",
            "------ Epoch:  457\n",
            "Train loss:  0.031069059\n",
            "Validation loss:  [[0.09673949]]\n",
            "------ Epoch:  458\n",
            "Train loss:  0.030444395\n",
            "Validation loss:  [[0.08557548]]\n",
            "------ Epoch:  459\n",
            "Train loss:  0.030896544\n",
            "Validation loss:  [[0.09648968]]\n",
            "------ Epoch:  460\n",
            "Train loss:  0.030283993\n",
            "Validation loss:  [[0.08539745]]\n",
            "------ Epoch:  461\n",
            "Train loss:  0.03072464\n",
            "Validation loss:  [[0.09624049]]\n",
            "------ Epoch:  462\n",
            "Train loss:  0.030124191\n",
            "Validation loss:  [[0.08522087]]\n",
            "------ Epoch:  463\n",
            "Train loss:  0.030553883\n",
            "Validation loss:  [[0.09599233]]\n",
            "------ Epoch:  464\n",
            "Train loss:  0.029965185\n",
            "Validation loss:  [[0.08504531]]\n",
            "------ Epoch:  465\n",
            "Train loss:  0.030383939\n",
            "Validation loss:  [[0.09574485]]\n",
            "------ Epoch:  466\n",
            "Train loss:  0.029806942\n",
            "Validation loss:  [[0.08487106]]\n",
            "------ Epoch:  467\n",
            "Train loss:  0.030214878\n",
            "Validation loss:  [[0.09549867]]\n",
            "------ Epoch:  468\n",
            "Train loss:  0.029649375\n",
            "Validation loss:  [[0.08469813]]\n",
            "------ Epoch:  469\n",
            "Train loss:  0.030046618\n",
            "Validation loss:  [[0.09525308]]\n",
            "------ Epoch:  470\n",
            "Train loss:  0.029492596\n",
            "Validation loss:  [[0.08452646]]\n",
            "------ Epoch:  471\n",
            "Train loss:  0.029879507\n",
            "Validation loss:  [[0.0950086]]\n",
            "------ Epoch:  472\n",
            "Train loss:  0.029336529\n",
            "Validation loss:  [[0.08435598]]\n",
            "------ Epoch:  473\n",
            "Train loss:  0.029713115\n",
            "Validation loss:  [[0.09476478]]\n",
            "------ Epoch:  474\n",
            "Train loss:  0.029181337\n",
            "Validation loss:  [[0.08418676]]\n",
            "------ Epoch:  475\n",
            "Train loss:  0.029547695\n",
            "Validation loss:  [[0.09452189]]\n",
            "------ Epoch:  476\n",
            "Train loss:  0.029026905\n",
            "Validation loss:  [[0.08401875]]\n",
            "------ Epoch:  477\n",
            "Train loss:  0.029383099\n",
            "Validation loss:  [[0.09428018]]\n",
            "------ Epoch:  478\n",
            "Train loss:  0.02887311\n",
            "Validation loss:  [[0.08385198]]\n",
            "------ Epoch:  479\n",
            "Train loss:  0.029219618\n",
            "Validation loss:  [[0.09403913]]\n",
            "------ Epoch:  480\n",
            "Train loss:  0.028720265\n",
            "Validation loss:  [[0.08368646]]\n",
            "------ Epoch:  481\n",
            "Train loss:  0.02905701\n",
            "Validation loss:  [[0.09379919]]\n",
            "------ Epoch:  482\n",
            "Train loss:  0.028568115\n",
            "Validation loss:  [[0.08352215]]\n",
            "------ Epoch:  483\n",
            "Train loss:  0.028895231\n",
            "Validation loss:  [[0.0935597]]\n",
            "------ Epoch:  484\n",
            "Train loss:  0.028416654\n",
            "Validation loss:  [[0.08335904]]\n",
            "------ Epoch:  485\n",
            "Train loss:  0.028734414\n",
            "Validation loss:  [[0.0933214]]\n",
            "------ Epoch:  486\n",
            "Train loss:  0.028266089\n",
            "Validation loss:  [[0.08319719]]\n",
            "------ Epoch:  487\n",
            "Train loss:  0.028574526\n",
            "Validation loss:  [[0.09308397]]\n",
            "------ Epoch:  488\n",
            "Train loss:  0.028116265\n",
            "Validation loss:  [[0.08303642]]\n",
            "------ Epoch:  489\n",
            "Train loss:  0.028415505\n",
            "Validation loss:  [[0.09284712]]\n",
            "------ Epoch:  490\n",
            "Train loss:  0.027967095\n",
            "Validation loss:  [[0.08287679]]\n",
            "------ Epoch:  491\n",
            "Train loss:  0.028257394\n",
            "Validation loss:  [[0.09261132]]\n",
            "------ Epoch:  492\n",
            "Train loss:  0.027818771\n",
            "Validation loss:  [[0.08271841]]\n",
            "------ Epoch:  493\n",
            "Train loss:  0.0281002\n",
            "Validation loss:  [[0.09237622]]\n",
            "------ Epoch:  494\n",
            "Train loss:  0.027671091\n",
            "Validation loss:  [[0.08256115]]\n",
            "------ Epoch:  495\n",
            "Train loss:  0.027943704\n",
            "Validation loss:  [[0.0921418]]\n",
            "------ Epoch:  496\n",
            "Train loss:  0.02752413\n",
            "Validation loss:  [[0.08240507]]\n",
            "------ Epoch:  497\n",
            "Train loss:  0.027788278\n",
            "Validation loss:  [[0.09190845]]\n",
            "------ Epoch:  498\n",
            "Train loss:  0.027377976\n",
            "Validation loss:  [[0.08225016]]\n",
            "------ Epoch:  499\n",
            "Train loss:  0.02763369\n",
            "Validation loss:  [[0.09167586]]\n",
            "------ Epoch:  500\n",
            "Train loss:  0.027232675\n",
            "Validation loss:  [[0.08209652]]\n",
            "------ Epoch:  501\n",
            "Train loss:  0.027480168\n",
            "Validation loss:  [[0.0914443]]\n",
            "------ Epoch:  502\n",
            "Train loss:  0.027088083\n",
            "Validation loss:  [[0.0819439]]\n",
            "------ Epoch:  503\n",
            "Train loss:  0.027327491\n",
            "Validation loss:  [[0.09121346]]\n",
            "------ Epoch:  504\n",
            "Train loss:  0.026944403\n",
            "Validation loss:  [[0.0817926]]\n",
            "------ Epoch:  505\n",
            "Train loss:  0.02717576\n",
            "Validation loss:  [[0.09098358]]\n",
            "------ Epoch:  506\n",
            "Train loss:  0.026801376\n",
            "Validation loss:  [[0.08164241]]\n",
            "------ Epoch:  507\n",
            "Train loss:  0.027024973\n",
            "Validation loss:  [[0.09075458]]\n",
            "------ Epoch:  508\n",
            "Train loss:  0.026659263\n",
            "Validation loss:  [[0.0814934]]\n",
            "------ Epoch:  509\n",
            "Train loss:  0.026875172\n",
            "Validation loss:  [[0.09052631]]\n",
            "------ Epoch:  510\n",
            "Train loss:  0.026517894\n",
            "Validation loss:  [[0.08134539]]\n",
            "------ Epoch:  511\n",
            "Train loss:  0.026726007\n",
            "Validation loss:  [[0.09029872]]\n",
            "------ Epoch:  512\n",
            "Train loss:  0.026377143\n",
            "Validation loss:  [[0.08119876]]\n",
            "------ Epoch:  513\n",
            "Train loss:  0.026578007\n",
            "Validation loss:  [[0.09007216]]\n",
            "------ Epoch:  514\n",
            "Train loss:  0.02623726\n",
            "Validation loss:  [[0.08105302]]\n",
            "------ Epoch:  515\n",
            "Train loss:  0.026430769\n",
            "Validation loss:  [[0.08984613]]\n",
            "------ Epoch:  516\n",
            "Train loss:  0.026098136\n",
            "Validation loss:  [[0.08090848]]\n",
            "------ Epoch:  517\n",
            "Train loss:  0.026284432\n",
            "Validation loss:  [[0.08962106]]\n",
            "------ Epoch:  518\n",
            "Train loss:  0.025959782\n",
            "Validation loss:  [[0.08076505]]\n",
            "------ Epoch:  519\n",
            "Train loss:  0.02613902\n",
            "Validation loss:  [[0.08939689]]\n",
            "------ Epoch:  520\n",
            "Train loss:  0.02582229\n",
            "Validation loss:  [[0.08062281]]\n",
            "------ Epoch:  521\n",
            "Train loss:  0.02599463\n",
            "Validation loss:  [[0.08917356]]\n",
            "------ Epoch:  522\n",
            "Train loss:  0.0256855\n",
            "Validation loss:  [[0.08048173]]\n",
            "------ Epoch:  523\n",
            "Train loss:  0.02585105\n",
            "Validation loss:  [[0.08895075]]\n",
            "------ Epoch:  524\n",
            "Train loss:  0.02554946\n",
            "Validation loss:  [[0.08034169]]\n",
            "------ Epoch:  525\n",
            "Train loss:  0.025708389\n",
            "Validation loss:  [[0.08872886]]\n",
            "------ Epoch:  526\n",
            "Train loss:  0.025414236\n",
            "Validation loss:  [[0.08020269]]\n",
            "------ Epoch:  527\n",
            "Train loss:  0.025566697\n",
            "Validation loss:  [[0.08850786]]\n",
            "------ Epoch:  528\n",
            "Train loss:  0.025279827\n",
            "Validation loss:  [[0.08006487]]\n",
            "------ Epoch:  529\n",
            "Train loss:  0.025425844\n",
            "Validation loss:  [[0.08828747]]\n",
            "------ Epoch:  530\n",
            "Train loss:  0.025146151\n",
            "Validation loss:  [[0.07992816]]\n",
            "------ Epoch:  531\n",
            "Train loss:  0.02528603\n",
            "Validation loss:  [[0.08806816]]\n",
            "------ Epoch:  532\n",
            "Train loss:  0.025013415\n",
            "Validation loss:  [[0.07979248]]\n",
            "------ Epoch:  533\n",
            "Train loss:  0.025147188\n",
            "Validation loss:  [[0.08784952]]\n",
            "------ Epoch:  534\n",
            "Train loss:  0.024881324\n",
            "Validation loss:  [[0.07965802]]\n",
            "------ Epoch:  535\n",
            "Train loss:  0.025009036\n",
            "Validation loss:  [[0.08763159]]\n",
            "------ Epoch:  536\n",
            "Train loss:  0.024749974\n",
            "Validation loss:  [[0.07952443]]\n",
            "------ Epoch:  537\n",
            "Train loss:  0.024871834\n",
            "Validation loss:  [[0.08741427]]\n",
            "------ Epoch:  538\n",
            "Train loss:  0.024619488\n",
            "Validation loss:  [[0.07939216]]\n",
            "------ Epoch:  539\n",
            "Train loss:  0.024735497\n",
            "Validation loss:  [[0.08719775]]\n",
            "------ Epoch:  540\n",
            "Train loss:  0.0244897\n",
            "Validation loss:  [[0.07926086]]\n",
            "------ Epoch:  541\n",
            "Train loss:  0.024600178\n",
            "Validation loss:  [[0.08698219]]\n",
            "------ Epoch:  542\n",
            "Train loss:  0.024360808\n",
            "Validation loss:  [[0.07913058]]\n",
            "------ Epoch:  543\n",
            "Train loss:  0.024465792\n",
            "Validation loss:  [[0.08676734]]\n",
            "------ Epoch:  544\n",
            "Train loss:  0.02423268\n",
            "Validation loss:  [[0.0790014]]\n",
            "------ Epoch:  545\n",
            "Train loss:  0.024332292\n",
            "Validation loss:  [[0.08655339]]\n",
            "------ Epoch:  546\n",
            "Train loss:  0.024105402\n",
            "Validation loss:  [[0.07887342]]\n",
            "------ Epoch:  547\n",
            "Train loss:  0.02419983\n",
            "Validation loss:  [[0.08634014]]\n",
            "------ Epoch:  548\n",
            "Train loss:  0.02397898\n",
            "Validation loss:  [[0.07874648]]\n",
            "------ Epoch:  549\n",
            "Train loss:  0.024068289\n",
            "Validation loss:  [[0.08612769]]\n",
            "------ Epoch:  550\n",
            "Train loss:  0.023853203\n",
            "Validation loss:  [[0.07862043]]\n",
            "------ Epoch:  551\n",
            "Train loss:  0.023937484\n",
            "Validation loss:  [[0.08591586]]\n",
            "------ Epoch:  552\n",
            "Train loss:  0.023728255\n",
            "Validation loss:  [[0.07849561]]\n",
            "------ Epoch:  553\n",
            "Train loss:  0.023807641\n",
            "Validation loss:  [[0.08570483]]\n",
            "------ Epoch:  554\n",
            "Train loss:  0.023604246\n",
            "Validation loss:  [[0.07837185]]\n",
            "------ Epoch:  555\n",
            "Train loss:  0.023678755\n",
            "Validation loss:  [[0.08549465]]\n",
            "------ Epoch:  556\n",
            "Train loss:  0.023480885\n",
            "Validation loss:  [[0.07824913]]\n",
            "------ Epoch:  557\n",
            "Train loss:  0.023550833\n",
            "Validation loss:  [[0.08528506]]\n",
            "------ Epoch:  558\n",
            "Train loss:  0.023358393\n",
            "Validation loss:  [[0.07812732]]\n",
            "------ Epoch:  559\n",
            "Train loss:  0.0234237\n",
            "Validation loss:  [[0.08507647]]\n",
            "------ Epoch:  560\n",
            "Train loss:  0.02323664\n",
            "Validation loss:  [[0.07800672]]\n",
            "------ Epoch:  561\n",
            "Train loss:  0.023297692\n",
            "Validation loss:  [[0.08486833]]\n",
            "------ Epoch:  562\n",
            "Train loss:  0.02311579\n",
            "Validation loss:  [[0.07788713]]\n",
            "------ Epoch:  563\n",
            "Train loss:  0.023172416\n",
            "Validation loss:  [[0.08466104]]\n",
            "------ Epoch:  564\n",
            "Train loss:  0.02299562\n",
            "Validation loss:  [[0.07776865]]\n",
            "------ Epoch:  565\n",
            "Train loss:  0.023048142\n",
            "Validation loss:  [[0.08445444]]\n",
            "------ Epoch:  566\n",
            "Train loss:  0.022876363\n",
            "Validation loss:  [[0.07765109]]\n",
            "------ Epoch:  567\n",
            "Train loss:  0.022924758\n",
            "Validation loss:  [[0.08424863]]\n",
            "------ Epoch:  568\n",
            "Train loss:  0.022757802\n",
            "Validation loss:  [[0.07753456]]\n",
            "------ Epoch:  569\n",
            "Train loss:  0.022802325\n",
            "Validation loss:  [[0.0840437]]\n",
            "------ Epoch:  570\n",
            "Train loss:  0.022640323\n",
            "Validation loss:  [[0.07741918]]\n",
            "------ Epoch:  571\n",
            "Train loss:  0.022680948\n",
            "Validation loss:  [[0.08383949]]\n",
            "------ Epoch:  572\n",
            "Train loss:  0.022523532\n",
            "Validation loss:  [[0.07730468]]\n",
            "------ Epoch:  573\n",
            "Train loss:  0.022560434\n",
            "Validation loss:  [[0.08363605]]\n",
            "------ Epoch:  574\n",
            "Train loss:  0.022407608\n",
            "Validation loss:  [[0.07719129]]\n",
            "------ Epoch:  575\n",
            "Train loss:  0.022440927\n",
            "Validation loss:  [[0.08343329]]\n",
            "------ Epoch:  576\n",
            "Train loss:  0.022292595\n",
            "Validation loss:  [[0.07707901]]\n",
            "------ Epoch:  577\n",
            "Train loss:  0.022322254\n",
            "Validation loss:  [[0.08323138]]\n",
            "------ Epoch:  578\n",
            "Train loss:  0.022178328\n",
            "Validation loss:  [[0.07696759]]\n",
            "------ Epoch:  579\n",
            "Train loss:  0.022204548\n",
            "Validation loss:  [[0.0830301]]\n",
            "------ Epoch:  580\n",
            "Train loss:  0.022064844\n",
            "Validation loss:  [[0.07685728]]\n",
            "------ Epoch:  581\n",
            "Train loss:  0.022087818\n",
            "Validation loss:  [[0.08282942]]\n",
            "------ Epoch:  582\n",
            "Train loss:  0.021952203\n",
            "Validation loss:  [[0.07674804]]\n",
            "------ Epoch:  583\n",
            "Train loss:  0.02197196\n",
            "Validation loss:  [[0.08262982]]\n",
            "------ Epoch:  584\n",
            "Train loss:  0.021840481\n",
            "Validation loss:  [[0.07663987]]\n",
            "------ Epoch:  585\n",
            "Train loss:  0.021857195\n",
            "Validation loss:  [[0.08243068]]\n",
            "------ Epoch:  586\n",
            "Train loss:  0.021729598\n",
            "Validation loss:  [[0.07653259]]\n",
            "------ Epoch:  587\n",
            "Train loss:  0.02174334\n",
            "Validation loss:  [[0.08223258]]\n",
            "------ Epoch:  588\n",
            "Train loss:  0.021619622\n",
            "Validation loss:  [[0.07642619]]\n",
            "------ Epoch:  589\n",
            "Train loss:  0.021630473\n",
            "Validation loss:  [[0.0820352]]\n",
            "------ Epoch:  590\n",
            "Train loss:  0.02151047\n",
            "Validation loss:  [[0.07632098]]\n",
            "------ Epoch:  591\n",
            "Train loss:  0.021518636\n",
            "Validation loss:  [[0.08183876]]\n",
            "------ Epoch:  592\n",
            "Train loss:  0.02140232\n",
            "Validation loss:  [[0.07621673]]\n",
            "------ Epoch:  593\n",
            "Train loss:  0.021407835\n",
            "Validation loss:  [[0.08164287]]\n",
            "------ Epoch:  594\n",
            "Train loss:  0.021294912\n",
            "Validation loss:  [[0.07611339]]\n",
            "------ Epoch:  595\n",
            "Train loss:  0.02129775\n",
            "Validation loss:  [[0.08144764]]\n",
            "------ Epoch:  596\n",
            "Train loss:  0.021188345\n",
            "Validation loss:  [[0.07601115]]\n",
            "------ Epoch:  597\n",
            "Train loss:  0.021188796\n",
            "Validation loss:  [[0.08125353]]\n",
            "------ Epoch:  598\n",
            "Train loss:  0.021082783\n",
            "Validation loss:  [[0.0759099]]\n",
            "------ Epoch:  599\n",
            "Train loss:  0.021080883\n",
            "Validation loss:  [[0.08106]]\n",
            "------ Epoch:  600\n",
            "Train loss:  0.02097804\n",
            "Validation loss:  [[0.07580952]]\n",
            "------ Epoch:  601\n",
            "Train loss:  0.02097388\n",
            "Validation loss:  [[0.08086732]]\n",
            "------ Epoch:  602\n",
            "Train loss:  0.020874286\n",
            "Validation loss:  [[0.07571019]]\n",
            "------ Epoch:  603\n",
            "Train loss:  0.020867953\n",
            "Validation loss:  [[0.08067551]]\n",
            "------ Epoch:  604\n",
            "Train loss:  0.020771354\n",
            "Validation loss:  [[0.07561186]]\n",
            "------ Epoch:  605\n",
            "Train loss:  0.020762922\n",
            "Validation loss:  [[0.08048456]]\n",
            "------ Epoch:  606\n",
            "Train loss:  0.020669453\n",
            "Validation loss:  [[0.07551435]]\n",
            "------ Epoch:  607\n",
            "Train loss:  0.020659056\n",
            "Validation loss:  [[0.08029439]]\n",
            "------ Epoch:  608\n",
            "Train loss:  0.02056837\n",
            "Validation loss:  [[0.07541791]]\n",
            "------ Epoch:  609\n",
            "Train loss:  0.020556096\n",
            "Validation loss:  [[0.08010516]]\n",
            "------ Epoch:  610\n",
            "Train loss:  0.020468295\n",
            "Validation loss:  [[0.07532232]]\n",
            "------ Epoch:  611\n",
            "Train loss:  0.020454248\n",
            "Validation loss:  [[0.07991672]]\n",
            "------ Epoch:  612\n",
            "Train loss:  0.020369178\n",
            "Validation loss:  [[0.07522771]]\n",
            "------ Epoch:  613\n",
            "Train loss:  0.020353409\n",
            "Validation loss:  [[0.07972899]]\n",
            "------ Epoch:  614\n",
            "Train loss:  0.020270916\n",
            "Validation loss:  [[0.07513405]]\n",
            "------ Epoch:  615\n",
            "Train loss:  0.020253567\n",
            "Validation loss:  [[0.07954232]]\n",
            "------ Epoch:  616\n",
            "Train loss:  0.020173669\n",
            "Validation loss:  [[0.07504126]]\n",
            "------ Epoch:  617\n",
            "Train loss:  0.020154817\n",
            "Validation loss:  [[0.07935646]]\n",
            "------ Epoch:  618\n",
            "Train loss:  0.020077376\n",
            "Validation loss:  [[0.07494937]]\n",
            "------ Epoch:  619\n",
            "Train loss:  0.02005708\n",
            "Validation loss:  [[0.07917152]]\n",
            "------ Epoch:  620\n",
            "Train loss:  0.019981973\n",
            "Validation loss:  [[0.07485844]]\n",
            "------ Epoch:  621\n",
            "Train loss:  0.019960325\n",
            "Validation loss:  [[0.0789875]]\n",
            "------ Epoch:  622\n",
            "Train loss:  0.019887527\n",
            "Validation loss:  [[0.07476821]]\n",
            "------ Epoch:  623\n",
            "Train loss:  0.019864721\n",
            "Validation loss:  [[0.07880445]]\n",
            "------ Epoch:  624\n",
            "Train loss:  0.0197942\n",
            "Validation loss:  [[0.07467909]]\n",
            "------ Epoch:  625\n",
            "Train loss:  0.019770186\n",
            "Validation loss:  [[0.07862235]]\n",
            "------ Epoch:  626\n",
            "Train loss:  0.019701805\n",
            "Validation loss:  [[0.07459069]]\n",
            "------ Epoch:  627\n",
            "Train loss:  0.019676618\n",
            "Validation loss:  [[0.07844105]]\n",
            "------ Epoch:  628\n",
            "Train loss:  0.019610353\n",
            "Validation loss:  [[0.07450303]]\n",
            "------ Epoch:  629\n",
            "Train loss:  0.019584214\n",
            "Validation loss:  [[0.07826071]]\n",
            "------ Epoch:  630\n",
            "Train loss:  0.019519918\n",
            "Validation loss:  [[0.07441625]]\n",
            "------ Epoch:  631\n",
            "Train loss:  0.019492745\n",
            "Validation loss:  [[0.07808134]]\n",
            "------ Epoch:  632\n",
            "Train loss:  0.019430442\n",
            "Validation loss:  [[0.07433037]]\n",
            "------ Epoch:  633\n",
            "Train loss:  0.019402456\n",
            "Validation loss:  [[0.07790306]]\n",
            "------ Epoch:  634\n",
            "Train loss:  0.019341966\n",
            "Validation loss:  [[0.07424518]]\n",
            "------ Epoch:  635\n",
            "Train loss:  0.019313196\n",
            "Validation loss:  [[0.0777258]]\n",
            "------ Epoch:  636\n",
            "Train loss:  0.01925451\n",
            "Validation loss:  [[0.07416077]]\n",
            "------ Epoch:  637\n",
            "Train loss:  0.019225089\n",
            "Validation loss:  [[0.07754959]]\n",
            "------ Epoch:  638\n",
            "Train loss:  0.019168075\n",
            "Validation loss:  [[0.07407706]]\n",
            "------ Epoch:  639\n",
            "Train loss:  0.019137999\n",
            "Validation loss:  [[0.07737441]]\n",
            "------ Epoch:  640\n",
            "Train loss:  0.019082682\n",
            "Validation loss:  [[0.07399403]]\n",
            "------ Epoch:  641\n",
            "Train loss:  0.019052032\n",
            "Validation loss:  [[0.07720031]]\n",
            "------ Epoch:  642\n",
            "Train loss:  0.018998316\n",
            "Validation loss:  [[0.07391166]]\n",
            "------ Epoch:  643\n",
            "Train loss:  0.018967109\n",
            "Validation loss:  [[0.07702738]]\n",
            "------ Epoch:  644\n",
            "Train loss:  0.01891497\n",
            "Validation loss:  [[0.07382998]]\n",
            "------ Epoch:  645\n",
            "Train loss:  0.018883256\n",
            "Validation loss:  [[0.07685543]]\n",
            "------ Epoch:  646\n",
            "Train loss:  0.018832548\n",
            "Validation loss:  [[0.0737489]]\n",
            "------ Epoch:  647\n",
            "Train loss:  0.018800613\n",
            "Validation loss:  [[0.07668455]]\n",
            "------ Epoch:  648\n",
            "Train loss:  0.01875121\n",
            "Validation loss:  [[0.07366841]]\n",
            "------ Epoch:  649\n",
            "Train loss:  0.01871894\n",
            "Validation loss:  [[0.07651494]]\n",
            "------ Epoch:  650\n",
            "Train loss:  0.018670868\n",
            "Validation loss:  [[0.07358854]]\n",
            "------ Epoch:  651\n",
            "Train loss:  0.018638352\n",
            "Validation loss:  [[0.07634646]]\n",
            "------ Epoch:  652\n",
            "Train loss:  0.018591572\n",
            "Validation loss:  [[0.07350922]]\n",
            "------ Epoch:  653\n",
            "Train loss:  0.018558852\n",
            "Validation loss:  [[0.07617924]]\n",
            "------ Epoch:  654\n",
            "Train loss:  0.018513348\n",
            "Validation loss:  [[0.07343014]]\n",
            "------ Epoch:  655\n",
            "Train loss:  0.018480431\n",
            "Validation loss:  [[0.07601304]]\n",
            "------ Epoch:  656\n",
            "Train loss:  0.018436145\n",
            "Validation loss:  [[0.0733516]]\n",
            "------ Epoch:  657\n",
            "Train loss:  0.018403016\n",
            "Validation loss:  [[0.07584821]]\n",
            "------ Epoch:  658\n",
            "Train loss:  0.018359829\n",
            "Validation loss:  [[0.07327353]]\n",
            "------ Epoch:  659\n",
            "Train loss:  0.018326683\n",
            "Validation loss:  [[0.07568435]]\n",
            "------ Epoch:  660\n",
            "Train loss:  0.01828457\n",
            "Validation loss:  [[0.07319576]]\n",
            "------ Epoch:  661\n",
            "Train loss:  0.018251399\n",
            "Validation loss:  [[0.07552188]]\n",
            "------ Epoch:  662\n",
            "Train loss:  0.018210283\n",
            "Validation loss:  [[0.07311836]]\n",
            "------ Epoch:  663\n",
            "Train loss:  0.018177172\n",
            "Validation loss:  [[0.07536051]]\n",
            "------ Epoch:  664\n",
            "Train loss:  0.018137079\n",
            "Validation loss:  [[0.07304128]]\n",
            "------ Epoch:  665\n",
            "Train loss:  0.018103963\n",
            "Validation loss:  [[0.07520063]]\n",
            "------ Epoch:  666\n",
            "Train loss:  0.018064713\n",
            "Validation loss:  [[0.07296437]]\n",
            "------ Epoch:  667\n",
            "Train loss:  0.018031744\n",
            "Validation loss:  [[0.07504203]]\n",
            "------ Epoch:  668\n",
            "Train loss:  0.017993404\n",
            "Validation loss:  [[0.07288765]]\n",
            "------ Epoch:  669\n",
            "Train loss:  0.017960524\n",
            "Validation loss:  [[0.07488453]]\n",
            "------ Epoch:  670\n",
            "Train loss:  0.017923065\n",
            "Validation loss:  [[0.07281105]]\n",
            "------ Epoch:  671\n",
            "Train loss:  0.017890258\n",
            "Validation loss:  [[0.0747285]]\n",
            "------ Epoch:  672\n",
            "Train loss:  0.017853642\n",
            "Validation loss:  [[0.07273456]]\n",
            "------ Epoch:  673\n",
            "Train loss:  0.017821008\n",
            "Validation loss:  [[0.07457357]]\n",
            "------ Epoch:  674\n",
            "Train loss:  0.017785143\n",
            "Validation loss:  [[0.0726581]]\n",
            "------ Epoch:  675\n",
            "Train loss:  0.017752698\n",
            "Validation loss:  [[0.07442003]]\n",
            "------ Epoch:  676\n",
            "Train loss:  0.017717559\n",
            "Validation loss:  [[0.0725816]]\n",
            "------ Epoch:  677\n",
            "Train loss:  0.017685283\n",
            "Validation loss:  [[0.0742678]]\n",
            "------ Epoch:  678\n",
            "Train loss:  0.017650835\n",
            "Validation loss:  [[0.07250507]]\n",
            "------ Epoch:  679\n",
            "Train loss:  0.017618863\n",
            "Validation loss:  [[0.07411686]]\n",
            "------ Epoch:  680\n",
            "Train loss:  0.017584976\n",
            "Validation loss:  [[0.07242843]]\n",
            "------ Epoch:  681\n",
            "Train loss:  0.017553218\n",
            "Validation loss:  [[0.07396714]]\n",
            "------ Epoch:  682\n",
            "Train loss:  0.017520003\n",
            "Validation loss:  [[0.07235166]]\n",
            "------ Epoch:  683\n",
            "Train loss:  0.017488515\n",
            "Validation loss:  [[0.0738188]]\n",
            "------ Epoch:  684\n",
            "Train loss:  0.017455917\n",
            "Validation loss:  [[0.07227449]]\n",
            "------ Epoch:  685\n",
            "Train loss:  0.017424667\n",
            "Validation loss:  [[0.07367192]]\n",
            "------ Epoch:  686\n",
            "Train loss:  0.017392615\n",
            "Validation loss:  [[0.07219715]]\n",
            "------ Epoch:  687\n",
            "Train loss:  0.017361598\n",
            "Validation loss:  [[0.07352619]]\n",
            "------ Epoch:  688\n",
            "Train loss:  0.017330078\n",
            "Validation loss:  [[0.07211939]]\n",
            "------ Epoch:  689\n",
            "Train loss:  0.017299293\n",
            "Validation loss:  [[0.07338183]]\n",
            "------ Epoch:  690\n",
            "Train loss:  0.017268373\n",
            "Validation loss:  [[0.07204136]]\n",
            "------ Epoch:  691\n",
            "Train loss:  0.017237904\n",
            "Validation loss:  [[0.07323872]]\n",
            "------ Epoch:  692\n",
            "Train loss:  0.017207326\n",
            "Validation loss:  [[0.07196283]]\n",
            "------ Epoch:  693\n",
            "Train loss:  0.017177157\n",
            "Validation loss:  [[0.07309697]]\n",
            "------ Epoch:  694\n",
            "Train loss:  0.017147016\n",
            "Validation loss:  [[0.07188392]]\n",
            "------ Epoch:  695\n",
            "Train loss:  0.017117223\n",
            "Validation loss:  [[0.07295658]]\n",
            "------ Epoch:  696\n",
            "Train loss:  0.01708749\n",
            "Validation loss:  [[0.07180426]]\n",
            "------ Epoch:  697\n",
            "Train loss:  0.017057907\n",
            "Validation loss:  [[0.07281724]]\n",
            "------ Epoch:  698\n",
            "Train loss:  0.017028594\n",
            "Validation loss:  [[0.07172414]]\n",
            "------ Epoch:  699\n",
            "Train loss:  0.016999269\n",
            "Validation loss:  [[0.07267918]]\n",
            "------ Epoch:  700\n",
            "Train loss:  0.016970297\n",
            "Validation loss:  [[0.07164341]]\n",
            "------ Epoch:  701\n",
            "Train loss:  0.016941288\n",
            "Validation loss:  [[0.07254253]]\n",
            "------ Epoch:  702\n",
            "Train loss:  0.016912673\n",
            "Validation loss:  [[0.07156197]]\n",
            "------ Epoch:  703\n",
            "Train loss:  0.01688389\n",
            "Validation loss:  [[0.07240677]]\n",
            "------ Epoch:  704\n",
            "Train loss:  0.016855618\n",
            "Validation loss:  [[0.07147977]]\n",
            "------ Epoch:  705\n",
            "Train loss:  0.016827162\n",
            "Validation loss:  [[0.07227243]]\n",
            "------ Epoch:  706\n",
            "Train loss:  0.016799161\n",
            "Validation loss:  [[0.07139694]]\n",
            "------ Epoch:  707\n",
            "Train loss:  0.01677094\n",
            "Validation loss:  [[0.07213922]]\n",
            "------ Epoch:  708\n",
            "Train loss:  0.016743183\n",
            "Validation loss:  [[0.07131324]]\n",
            "------ Epoch:  709\n",
            "Train loss:  0.016715277\n",
            "Validation loss:  [[0.07200707]]\n",
            "------ Epoch:  710\n",
            "Train loss:  0.016687836\n",
            "Validation loss:  [[0.07122879]]\n",
            "------ Epoch:  711\n",
            "Train loss:  0.01666007\n",
            "Validation loss:  [[0.07187603]]\n",
            "------ Epoch:  712\n",
            "Train loss:  0.016632875\n",
            "Validation loss:  [[0.07114349]]\n",
            "------ Epoch:  713\n",
            "Train loss:  0.01660537\n",
            "Validation loss:  [[0.07174595]]\n",
            "------ Epoch:  714\n",
            "Train loss:  0.016578415\n",
            "Validation loss:  [[0.07105742]]\n",
            "------ Epoch:  715\n",
            "Train loss:  0.016551133\n",
            "Validation loss:  [[0.07161704]]\n",
            "------ Epoch:  716\n",
            "Train loss:  0.016524388\n",
            "Validation loss:  [[0.07097032]]\n",
            "------ Epoch:  717\n",
            "Train loss:  0.01649736\n",
            "Validation loss:  [[0.07148912]]\n",
            "------ Epoch:  718\n",
            "Train loss:  0.016470797\n",
            "Validation loss:  [[0.07088247]]\n",
            "------ Epoch:  719\n",
            "Train loss:  0.016443985\n",
            "Validation loss:  [[0.07136214]]\n",
            "------ Epoch:  720\n",
            "Train loss:  0.01641766\n",
            "Validation loss:  [[0.07079362]]\n",
            "------ Epoch:  721\n",
            "Train loss:  0.016390996\n",
            "Validation loss:  [[0.07123619]]\n",
            "------ Epoch:  722\n",
            "Train loss:  0.016364861\n",
            "Validation loss:  [[0.07070372]]\n",
            "------ Epoch:  723\n",
            "Train loss:  0.016338408\n",
            "Validation loss:  [[0.07111102]]\n",
            "------ Epoch:  724\n",
            "Train loss:  0.016312461\n",
            "Validation loss:  [[0.07061294]]\n",
            "------ Epoch:  725\n",
            "Train loss:  0.016286222\n",
            "Validation loss:  [[0.07098672]]\n",
            "------ Epoch:  726\n",
            "Train loss:  0.016260372\n",
            "Validation loss:  [[0.07052135]]\n",
            "------ Epoch:  727\n",
            "Train loss:  0.016234308\n",
            "Validation loss:  [[0.07086334]]\n",
            "------ Epoch:  728\n",
            "Train loss:  0.016208606\n",
            "Validation loss:  [[0.07042874]]\n",
            "------ Epoch:  729\n",
            "Train loss:  0.01618272\n",
            "Validation loss:  [[0.07074068]]\n",
            "------ Epoch:  730\n",
            "Train loss:  0.016157145\n",
            "Validation loss:  [[0.07033508]]\n",
            "------ Epoch:  731\n",
            "Train loss:  0.0161314\n",
            "Validation loss:  [[0.0706187]]\n",
            "------ Epoch:  732\n",
            "Train loss:  0.01610605\n",
            "Validation loss:  [[0.07024059]]\n",
            "------ Epoch:  733\n",
            "Train loss:  0.016080383\n",
            "Validation loss:  [[0.07049756]]\n",
            "------ Epoch:  734\n",
            "Train loss:  0.016055148\n",
            "Validation loss:  [[0.07014509]]\n",
            "------ Epoch:  735\n",
            "Train loss:  0.016029635\n",
            "Validation loss:  [[0.07037688]]\n",
            "------ Epoch:  736\n",
            "Train loss:  0.016004514\n",
            "Validation loss:  [[0.07004867]]\n",
            "------ Epoch:  737\n",
            "Train loss:  0.015979184\n",
            "Validation loss:  [[0.07025681]]\n",
            "------ Epoch:  738\n",
            "Train loss:  0.015954198\n",
            "Validation loss:  [[0.06995125]]\n",
            "------ Epoch:  739\n",
            "Train loss:  0.015928976\n",
            "Validation loss:  [[0.07013755]]\n",
            "------ Epoch:  740\n",
            "Train loss:  0.015904095\n",
            "Validation loss:  [[0.06985299]]\n",
            "------ Epoch:  741\n",
            "Train loss:  0.015879009\n",
            "Validation loss:  [[0.07001863]]\n",
            "------ Epoch:  742\n",
            "Train loss:  0.015854185\n",
            "Validation loss:  [[0.0697539]]\n",
            "------ Epoch:  743\n",
            "Train loss:  0.015829287\n",
            "Validation loss:  [[0.06990023]]\n",
            "------ Epoch:  744\n",
            "Train loss:  0.01580458\n",
            "Validation loss:  [[0.06965383]]\n",
            "------ Epoch:  745\n",
            "Train loss:  0.015779711\n",
            "Validation loss:  [[0.06978214]]\n",
            "------ Epoch:  746\n",
            "Train loss:  0.015755137\n",
            "Validation loss:  [[0.06955284]]\n",
            "------ Epoch:  747\n",
            "Train loss:  0.015730392\n",
            "Validation loss:  [[0.06966457]]\n",
            "------ Epoch:  748\n",
            "Train loss:  0.015705872\n",
            "Validation loss:  [[0.06945103]]\n",
            "------ Epoch:  749\n",
            "Train loss:  0.015681285\n",
            "Validation loss:  [[0.06954733]]\n",
            "------ Epoch:  750\n",
            "Train loss:  0.015656833\n",
            "Validation loss:  [[0.06934845]]\n",
            "------ Epoch:  751\n",
            "Train loss:  0.015632322\n",
            "Validation loss:  [[0.06943038]]\n",
            "------ Epoch:  752\n",
            "Train loss:  0.015607993\n",
            "Validation loss:  [[0.06924505]]\n",
            "------ Epoch:  753\n",
            "Train loss:  0.015583617\n",
            "Validation loss:  [[0.06931384]]\n",
            "------ Epoch:  754\n",
            "Train loss:  0.015559409\n",
            "Validation loss:  [[0.0691408]]\n",
            "------ Epoch:  755\n",
            "Train loss:  0.015535083\n",
            "Validation loss:  [[0.06919747]]\n",
            "------ Epoch:  756\n",
            "Train loss:  0.015510908\n",
            "Validation loss:  [[0.06903576]]\n",
            "------ Epoch:  757\n",
            "Train loss:  0.015486679\n",
            "Validation loss:  [[0.06908125]]\n",
            "------ Epoch:  758\n",
            "Train loss:  0.0154626565\n",
            "Validation loss:  [[0.06893004]]\n",
            "------ Epoch:  759\n",
            "Train loss:  0.0154384915\n",
            "Validation loss:  [[0.06896517]]\n",
            "------ Epoch:  760\n",
            "Train loss:  0.015414526\n",
            "Validation loss:  [[0.06882351]]\n",
            "------ Epoch:  761\n",
            "Train loss:  0.015390458\n",
            "Validation loss:  [[0.0688493]]\n",
            "------ Epoch:  762\n",
            "Train loss:  0.01536654\n",
            "Validation loss:  [[0.06871628]]\n",
            "------ Epoch:  763\n",
            "Train loss:  0.015342639\n",
            "Validation loss:  [[0.06873348]]\n",
            "------ Epoch:  764\n",
            "Train loss:  0.015318809\n",
            "Validation loss:  [[0.06860836]]\n",
            "------ Epoch:  765\n",
            "Train loss:  0.015294956\n",
            "Validation loss:  [[0.06861767]]\n",
            "------ Epoch:  766\n",
            "Train loss:  0.015271217\n",
            "Validation loss:  [[0.06849986]]\n",
            "------ Epoch:  767\n",
            "Train loss:  0.015247438\n",
            "Validation loss:  [[0.06850185]]\n",
            "------ Epoch:  768\n",
            "Train loss:  0.015223806\n",
            "Validation loss:  [[0.06839055]]\n",
            "------ Epoch:  769\n",
            "Train loss:  0.015200049\n",
            "Validation loss:  [[0.06838604]]\n",
            "------ Epoch:  770\n",
            "Train loss:  0.015176469\n",
            "Validation loss:  [[0.06828076]]\n",
            "------ Epoch:  771\n",
            "Train loss:  0.015152891\n",
            "Validation loss:  [[0.06827021]]\n",
            "------ Epoch:  772\n",
            "Train loss:  0.015129388\n",
            "Validation loss:  [[0.06817014]]\n",
            "------ Epoch:  773\n",
            "Train loss:  0.015105854\n",
            "Validation loss:  [[0.06815434]]\n",
            "------ Epoch:  774\n",
            "Train loss:  0.01508239\n",
            "Validation loss:  [[0.06805912]]\n",
            "------ Epoch:  775\n",
            "Train loss:  0.01505897\n",
            "Validation loss:  [[0.06803835]]\n",
            "------ Epoch:  776\n",
            "Train loss:  0.015035606\n",
            "Validation loss:  [[0.06794739]]\n",
            "------ Epoch:  777\n",
            "Train loss:  0.01501228\n",
            "Validation loss:  [[0.06792223]]\n",
            "------ Epoch:  778\n",
            "Train loss:  0.01498898\n",
            "Validation loss:  [[0.06783506]]\n",
            "------ Epoch:  779\n",
            "Train loss:  0.014965717\n",
            "Validation loss:  [[0.06780592]]\n",
            "------ Epoch:  780\n",
            "Train loss:  0.014942474\n",
            "Validation loss:  [[0.0677222]]\n",
            "------ Epoch:  781\n",
            "Train loss:  0.014919269\n",
            "Validation loss:  [[0.06768953]]\n",
            "------ Epoch:  782\n",
            "Train loss:  0.014896184\n",
            "Validation loss:  [[0.0676088]]\n",
            "------ Epoch:  783\n",
            "Train loss:  0.014873049\n",
            "Validation loss:  [[0.06757298]]\n",
            "------ Epoch:  784\n",
            "Train loss:  0.014850049\n",
            "Validation loss:  [[0.06749489]]\n",
            "------ Epoch:  785\n",
            "Train loss:  0.014826992\n",
            "Validation loss:  [[0.06745612]]\n",
            "------ Epoch:  786\n",
            "Train loss:  0.014804006\n",
            "Validation loss:  [[0.06738044]]\n",
            "------ Epoch:  787\n",
            "Train loss:  0.01478107\n",
            "Validation loss:  [[0.06733909]]\n",
            "------ Epoch:  788\n",
            "Train loss:  0.014758178\n",
            "Validation loss:  [[0.06726549]]\n",
            "------ Epoch:  789\n",
            "Train loss:  0.014735297\n",
            "Validation loss:  [[0.06722176]]\n",
            "------ Epoch:  790\n",
            "Train loss:  0.014712539\n",
            "Validation loss:  [[0.06714995]]\n",
            "------ Epoch:  791\n",
            "Train loss:  0.014689699\n",
            "Validation loss:  [[0.06710419]]\n",
            "------ Epoch:  792\n",
            "Train loss:  0.014666911\n",
            "Validation loss:  [[0.06703401]]\n",
            "------ Epoch:  793\n",
            "Train loss:  0.014644246\n",
            "Validation loss:  [[0.06698637]]\n",
            "------ Epoch:  794\n",
            "Train loss:  0.014621534\n",
            "Validation loss:  [[0.06691757]]\n",
            "------ Epoch:  795\n",
            "Train loss:  0.014599017\n",
            "Validation loss:  [[0.06686822]]\n",
            "------ Epoch:  796\n",
            "Train loss:  0.014576359\n",
            "Validation loss:  [[0.06680062]]\n",
            "------ Epoch:  797\n",
            "Train loss:  0.0145537825\n",
            "Validation loss:  [[0.0667498]]\n",
            "------ Epoch:  798\n",
            "Train loss:  0.0145313\n",
            "Validation loss:  [[0.0666832]]\n",
            "------ Epoch:  799\n",
            "Train loss:  0.0145088425\n",
            "Validation loss:  [[0.06663102]]\n",
            "------ Epoch:  800\n",
            "Train loss:  0.014486444\n",
            "Validation loss:  [[0.06656542]]\n",
            "------ Epoch:  801\n",
            "Train loss:  0.014464023\n",
            "Validation loss:  [[0.06651201]]\n",
            "------ Epoch:  802\n",
            "Train loss:  0.014441664\n",
            "Validation loss:  [[0.066447]]\n",
            "------ Epoch:  803\n",
            "Train loss:  0.014419373\n",
            "Validation loss:  [[0.06639259]]\n",
            "------ Epoch:  804\n",
            "Train loss:  0.014397088\n",
            "Validation loss:  [[0.06632823]]\n",
            "------ Epoch:  805\n",
            "Train loss:  0.01437484\n",
            "Validation loss:  [[0.0662728]]\n",
            "------ Epoch:  806\n",
            "Train loss:  0.014352674\n",
            "Validation loss:  [[0.06620903]]\n",
            "------ Epoch:  807\n",
            "Train loss:  0.01433052\n",
            "Validation loss:  [[0.06615276]]\n",
            "------ Epoch:  808\n",
            "Train loss:  0.014308414\n",
            "Validation loss:  [[0.06608939]]\n",
            "------ Epoch:  809\n",
            "Train loss:  0.014286415\n",
            "Validation loss:  [[0.06603242]]\n",
            "------ Epoch:  810\n",
            "Train loss:  0.014264344\n",
            "Validation loss:  [[0.06596937]]\n",
            "------ Epoch:  811\n",
            "Train loss:  0.014242359\n",
            "Validation loss:  [[0.06591168]]\n",
            "------ Epoch:  812\n",
            "Train loss:  0.014220432\n",
            "Validation loss:  [[0.06584872]]\n",
            "------ Epoch:  813\n",
            "Train loss:  0.014198452\n",
            "Validation loss:  [[0.06579061]]\n",
            "------ Epoch:  814\n",
            "Train loss:  0.014176647\n",
            "Validation loss:  [[0.06572772]]\n",
            "------ Epoch:  815\n",
            "Train loss:  0.014154794\n",
            "Validation loss:  [[0.06566907]]\n",
            "------ Epoch:  816\n",
            "Train loss:  0.014133048\n",
            "Validation loss:  [[0.0656064]]\n",
            "------ Epoch:  817\n",
            "Train loss:  0.01411134\n",
            "Validation loss:  [[0.0655472]]\n",
            "------ Epoch:  818\n",
            "Train loss:  0.01408964\n",
            "Validation loss:  [[0.06548467]]\n",
            "------ Epoch:  819\n",
            "Train loss:  0.014068026\n",
            "Validation loss:  [[0.06542488]]\n",
            "------ Epoch:  820\n",
            "Train loss:  0.014046354\n",
            "Validation loss:  [[0.06536248]]\n",
            "------ Epoch:  821\n",
            "Train loss:  0.014024814\n",
            "Validation loss:  [[0.0653023]]\n",
            "------ Epoch:  822\n",
            "Train loss:  0.014003259\n",
            "Validation loss:  [[0.06523985]]\n",
            "------ Epoch:  823\n",
            "Train loss:  0.013981823\n",
            "Validation loss:  [[0.06517928]]\n",
            "------ Epoch:  824\n",
            "Train loss:  0.013960365\n",
            "Validation loss:  [[0.0651169]]\n",
            "------ Epoch:  825\n",
            "Train loss:  0.013938987\n",
            "Validation loss:  [[0.06505608]]\n",
            "------ Epoch:  826\n",
            "Train loss:  0.013917651\n",
            "Validation loss:  [[0.06499349]]\n",
            "------ Epoch:  827\n",
            "Train loss:  0.0138963405\n",
            "Validation loss:  [[0.06493227]]\n",
            "------ Epoch:  828\n",
            "Train loss:  0.013875091\n",
            "Validation loss:  [[0.06486973]]\n",
            "------ Epoch:  829\n",
            "Train loss:  0.013853878\n",
            "Validation loss:  [[0.06480829]]\n",
            "------ Epoch:  830\n",
            "Train loss:  0.01383264\n",
            "Validation loss:  [[0.06474562]]\n",
            "------ Epoch:  831\n",
            "Train loss:  0.013811546\n",
            "Validation loss:  [[0.06468384]]\n",
            "------ Epoch:  832\n",
            "Train loss:  0.013790428\n",
            "Validation loss:  [[0.06462113]]\n",
            "------ Epoch:  833\n",
            "Train loss:  0.013769428\n",
            "Validation loss:  [[0.06455909]]\n",
            "------ Epoch:  834\n",
            "Train loss:  0.013748421\n",
            "Validation loss:  [[0.06449623]]\n",
            "------ Epoch:  835\n",
            "Train loss:  0.013727472\n",
            "Validation loss:  [[0.06443404]]\n",
            "------ Epoch:  836\n",
            "Train loss:  0.0137065565\n",
            "Validation loss:  [[0.06437097]]\n",
            "------ Epoch:  837\n",
            "Train loss:  0.013685695\n",
            "Validation loss:  [[0.06430845]]\n",
            "------ Epoch:  838\n",
            "Train loss:  0.0136648575\n",
            "Validation loss:  [[0.06424525]]\n",
            "------ Epoch:  839\n",
            "Train loss:  0.013644068\n",
            "Validation loss:  [[0.06418265]]\n",
            "------ Epoch:  840\n",
            "Train loss:  0.013623381\n",
            "Validation loss:  [[0.06411933]]\n",
            "------ Epoch:  841\n",
            "Train loss:  0.013602668\n",
            "Validation loss:  [[0.06405637]]\n",
            "------ Epoch:  842\n",
            "Train loss:  0.013582013\n",
            "Validation loss:  [[0.06399297]]\n",
            "------ Epoch:  843\n",
            "Train loss:  0.0135614285\n",
            "Validation loss:  [[0.06392986]]\n",
            "------ Epoch:  844\n",
            "Train loss:  0.013540909\n",
            "Validation loss:  [[0.06386633]]\n",
            "------ Epoch:  845\n",
            "Train loss:  0.013520386\n",
            "Validation loss:  [[0.06380302]]\n",
            "------ Epoch:  846\n",
            "Train loss:  0.0134999305\n",
            "Validation loss:  [[0.06373929]]\n",
            "------ Epoch:  847\n",
            "Train loss:  0.0134795485\n",
            "Validation loss:  [[0.06367577]]\n",
            "------ Epoch:  848\n",
            "Train loss:  0.01345917\n",
            "Validation loss:  [[0.06361188]]\n",
            "------ Epoch:  849\n",
            "Train loss:  0.013438895\n",
            "Validation loss:  [[0.06354827]]\n",
            "------ Epoch:  850\n",
            "Train loss:  0.013418606\n",
            "Validation loss:  [[0.06348429]]\n",
            "------ Epoch:  851\n",
            "Train loss:  0.013398363\n",
            "Validation loss:  [[0.0634203]]\n",
            "------ Epoch:  852\n",
            "Train loss:  0.013378202\n",
            "Validation loss:  [[0.06335633]]\n",
            "------ Epoch:  853\n",
            "Train loss:  0.013358127\n",
            "Validation loss:  [[0.06329217]]\n",
            "------ Epoch:  854\n",
            "Train loss:  0.013338005\n",
            "Validation loss:  [[0.063228]]\n",
            "------ Epoch:  855\n",
            "Train loss:  0.013317984\n",
            "Validation loss:  [[0.06316372]]\n",
            "------ Epoch:  856\n",
            "Train loss:  0.013297983\n",
            "Validation loss:  [[0.06309943]]\n",
            "------ Epoch:  857\n",
            "Train loss:  0.013278046\n",
            "Validation loss:  [[0.06303509]]\n",
            "------ Epoch:  858\n",
            "Train loss:  0.013258171\n",
            "Validation loss:  [[0.06297047]]\n",
            "------ Epoch:  859\n",
            "Train loss:  0.013238298\n",
            "Validation loss:  [[0.06290596]]\n",
            "------ Epoch:  860\n",
            "Train loss:  0.01321853\n",
            "Validation loss:  [[0.06284133]]\n",
            "------ Epoch:  861\n",
            "Train loss:  0.0131987985\n",
            "Validation loss:  [[0.06277663]]\n",
            "------ Epoch:  862\n",
            "Train loss:  0.0131790815\n",
            "Validation loss:  [[0.06271188]]\n",
            "------ Epoch:  863\n",
            "Train loss:  0.013159436\n",
            "Validation loss:  [[0.06264701]]\n",
            "------ Epoch:  864\n",
            "Train loss:  0.013139893\n",
            "Validation loss:  [[0.06258214]]\n",
            "------ Epoch:  865\n",
            "Train loss:  0.013120365\n",
            "Validation loss:  [[0.06251714]]\n",
            "------ Epoch:  866\n",
            "Train loss:  0.013100814\n",
            "Validation loss:  [[0.06245207]]\n",
            "------ Epoch:  867\n",
            "Train loss:  0.013081368\n",
            "Validation loss:  [[0.06238697]]\n",
            "------ Epoch:  868\n",
            "Train loss:  0.013061936\n",
            "Validation loss:  [[0.06232187]]\n",
            "------ Epoch:  869\n",
            "Train loss:  0.013042615\n",
            "Validation loss:  [[0.06225659]]\n",
            "------ Epoch:  870\n",
            "Train loss:  0.013023323\n",
            "Validation loss:  [[0.06219122]]\n",
            "------ Epoch:  871\n",
            "Train loss:  0.013004052\n",
            "Validation loss:  [[0.06212594]]\n",
            "------ Epoch:  872\n",
            "Train loss:  0.012984865\n",
            "Validation loss:  [[0.06206055]]\n",
            "------ Epoch:  873\n",
            "Train loss:  0.012965697\n",
            "Validation loss:  [[0.06199504]]\n",
            "------ Epoch:  874\n",
            "Train loss:  0.012946562\n",
            "Validation loss:  [[0.06192942]]\n",
            "------ Epoch:  875\n",
            "Train loss:  0.012927539\n",
            "Validation loss:  [[0.06186393]]\n",
            "------ Epoch:  876\n",
            "Train loss:  0.012908543\n",
            "Validation loss:  [[0.06179833]]\n",
            "------ Epoch:  877\n",
            "Train loss:  0.01288959\n",
            "Validation loss:  [[0.06173261]]\n",
            "------ Epoch:  878\n",
            "Train loss:  0.012870711\n",
            "Validation loss:  [[0.06166695]]\n",
            "------ Epoch:  879\n",
            "Train loss:  0.0128518045\n",
            "Validation loss:  [[0.06160111]]\n",
            "------ Epoch:  880\n",
            "Train loss:  0.01283303\n",
            "Validation loss:  [[0.06153519]]\n",
            "------ Epoch:  881\n",
            "Train loss:  0.01281426\n",
            "Validation loss:  [[0.06146942]]\n",
            "------ Epoch:  882\n",
            "Train loss:  0.012795606\n",
            "Validation loss:  [[0.06140342]]\n",
            "------ Epoch:  883\n",
            "Train loss:  0.012776967\n",
            "Validation loss:  [[0.06133752]]\n",
            "------ Epoch:  884\n",
            "Train loss:  0.012758306\n",
            "Validation loss:  [[0.06127141]]\n",
            "------ Epoch:  885\n",
            "Train loss:  0.012739793\n",
            "Validation loss:  [[0.06120554]]\n",
            "------ Epoch:  886\n",
            "Train loss:  0.012721251\n",
            "Validation loss:  [[0.06113923]]\n",
            "------ Epoch:  887\n",
            "Train loss:  0.012702818\n",
            "Validation loss:  [[0.06107321]]\n",
            "------ Epoch:  888\n",
            "Train loss:  0.012684417\n",
            "Validation loss:  [[0.06100691]]\n",
            "------ Epoch:  889\n",
            "Train loss:  0.012666048\n",
            "Validation loss:  [[0.06094076]]\n",
            "------ Epoch:  890\n",
            "Train loss:  0.012647763\n",
            "Validation loss:  [[0.06087438]]\n",
            "------ Epoch:  891\n",
            "Train loss:  0.012629505\n",
            "Validation loss:  [[0.0608082]]\n",
            "------ Epoch:  892\n",
            "Train loss:  0.012611306\n",
            "Validation loss:  [[0.06074178]]\n",
            "------ Epoch:  893\n",
            "Train loss:  0.012593163\n",
            "Validation loss:  [[0.06067541]]\n",
            "------ Epoch:  894\n",
            "Train loss:  0.01257506\n",
            "Validation loss:  [[0.06060897]]\n",
            "------ Epoch:  895\n",
            "Train loss:  0.0125570595\n",
            "Validation loss:  [[0.06054256]]\n",
            "------ Epoch:  896\n",
            "Train loss:  0.012539056\n",
            "Validation loss:  [[0.06047602]]\n",
            "------ Epoch:  897\n",
            "Train loss:  0.0125210835\n",
            "Validation loss:  [[0.0604095]]\n",
            "------ Epoch:  898\n",
            "Train loss:  0.012503176\n",
            "Validation loss:  [[0.06034296]]\n",
            "------ Epoch:  899\n",
            "Train loss:  0.012485355\n",
            "Validation loss:  [[0.06027643]]\n",
            "------ Epoch:  900\n",
            "Train loss:  0.012467514\n",
            "Validation loss:  [[0.06020982]]\n",
            "------ Epoch:  901\n",
            "Train loss:  0.01244978\n",
            "Validation loss:  [[0.06014314]]\n",
            "------ Epoch:  902\n",
            "Train loss:  0.012432099\n",
            "Validation loss:  [[0.06007645]]\n",
            "------ Epoch:  903\n",
            "Train loss:  0.012414451\n",
            "Validation loss:  [[0.06000975]]\n",
            "------ Epoch:  904\n",
            "Train loss:  0.01239688\n",
            "Validation loss:  [[0.05994307]]\n",
            "------ Epoch:  905\n",
            "Train loss:  0.012379324\n",
            "Validation loss:  [[0.05987636]]\n",
            "------ Epoch:  906\n",
            "Train loss:  0.012361788\n",
            "Validation loss:  [[0.05980961]]\n",
            "------ Epoch:  907\n",
            "Train loss:  0.012344397\n",
            "Validation loss:  [[0.05974273]]\n",
            "------ Epoch:  908\n",
            "Train loss:  0.012326984\n",
            "Validation loss:  [[0.05967603]]\n",
            "------ Epoch:  909\n",
            "Train loss:  0.01230966\n",
            "Validation loss:  [[0.05960904]]\n",
            "------ Epoch:  910\n",
            "Train loss:  0.012292331\n",
            "Validation loss:  [[0.05954232]]\n",
            "------ Epoch:  911\n",
            "Train loss:  0.012275115\n",
            "Validation loss:  [[0.05947527]]\n",
            "------ Epoch:  912\n",
            "Train loss:  0.01225791\n",
            "Validation loss:  [[0.05940866]]\n",
            "------ Epoch:  913\n",
            "Train loss:  0.012240772\n",
            "Validation loss:  [[0.05934165]]\n",
            "------ Epoch:  914\n",
            "Train loss:  0.012223695\n",
            "Validation loss:  [[0.05927476]]\n",
            "------ Epoch:  915\n",
            "Train loss:  0.012206613\n",
            "Validation loss:  [[0.05920776]]\n",
            "------ Epoch:  916\n",
            "Train loss:  0.01218966\n",
            "Validation loss:  [[0.05914082]]\n",
            "------ Epoch:  917\n",
            "Train loss:  0.0121727055\n",
            "Validation loss:  [[0.05907385]]\n",
            "------ Epoch:  918\n",
            "Train loss:  0.012155815\n",
            "Validation loss:  [[0.05900683]]\n",
            "------ Epoch:  919\n",
            "Train loss:  0.012138969\n",
            "Validation loss:  [[0.05893978]]\n",
            "------ Epoch:  920\n",
            "Train loss:  0.012122149\n",
            "Validation loss:  [[0.05887283]]\n",
            "------ Epoch:  921\n",
            "Train loss:  0.012105402\n",
            "Validation loss:  [[0.0588057]]\n",
            "------ Epoch:  922\n",
            "Train loss:  0.012088714\n",
            "Validation loss:  [[0.05873876]]\n",
            "------ Epoch:  923\n",
            "Train loss:  0.0120720295\n",
            "Validation loss:  [[0.0586715]]\n",
            "------ Epoch:  924\n",
            "Train loss:  0.012055457\n",
            "Validation loss:  [[0.05860471]]\n",
            "------ Epoch:  925\n",
            "Train loss:  0.012038865\n",
            "Validation loss:  [[0.05853751]]\n",
            "------ Epoch:  926\n",
            "Train loss:  0.012022406\n",
            "Validation loss:  [[0.05847047]]\n",
            "------ Epoch:  927\n",
            "Train loss:  0.012005916\n",
            "Validation loss:  [[0.05840328]]\n",
            "------ Epoch:  928\n",
            "Train loss:  0.011989488\n",
            "Validation loss:  [[0.05833634]]\n",
            "------ Epoch:  929\n",
            "Train loss:  0.011973104\n",
            "Validation loss:  [[0.05826911]]\n",
            "------ Epoch:  930\n",
            "Train loss:  0.011956799\n",
            "Validation loss:  [[0.05820206]]\n",
            "------ Epoch:  931\n",
            "Train loss:  0.011940539\n",
            "Validation loss:  [[0.0581349]]\n",
            "------ Epoch:  932\n",
            "Train loss:  0.011924302\n",
            "Validation loss:  [[0.05806771]]\n",
            "------ Epoch:  933\n",
            "Train loss:  0.01190816\n",
            "Validation loss:  [[0.05800067]]\n",
            "------ Epoch:  934\n",
            "Train loss:  0.011892004\n",
            "Validation loss:  [[0.05793342]]\n",
            "------ Epoch:  935\n",
            "Train loss:  0.011875915\n",
            "Validation loss:  [[0.05786633]]\n",
            "------ Epoch:  936\n",
            "Train loss:  0.011859917\n",
            "Validation loss:  [[0.05779911]]\n",
            "------ Epoch:  937\n",
            "Train loss:  0.011843874\n",
            "Validation loss:  [[0.05773202]]\n",
            "------ Epoch:  938\n",
            "Train loss:  0.011827951\n",
            "Validation loss:  [[0.05766476]]\n",
            "------ Epoch:  939\n",
            "Train loss:  0.011812017\n",
            "Validation loss:  [[0.05759771]]\n",
            "------ Epoch:  940\n",
            "Train loss:  0.01179619\n",
            "Validation loss:  [[0.05753049]]\n",
            "------ Epoch:  941\n",
            "Train loss:  0.011780396\n",
            "Validation loss:  [[0.05746339]]\n",
            "------ Epoch:  942\n",
            "Train loss:  0.011764615\n",
            "Validation loss:  [[0.05739608]]\n",
            "------ Epoch:  943\n",
            "Train loss:  0.011748864\n",
            "Validation loss:  [[0.05732901]]\n",
            "------ Epoch:  944\n",
            "Train loss:  0.011733244\n",
            "Validation loss:  [[0.05726179]]\n",
            "------ Epoch:  945\n",
            "Train loss:  0.0117175765\n",
            "Validation loss:  [[0.05719462]]\n",
            "------ Epoch:  946\n",
            "Train loss:  0.011701999\n",
            "Validation loss:  [[0.05712748]]\n",
            "------ Epoch:  947\n",
            "Train loss:  0.011686462\n",
            "Validation loss:  [[0.05706024]]\n",
            "------ Epoch:  948\n",
            "Train loss:  0.011670975\n",
            "Validation loss:  [[0.0569931]]\n",
            "------ Epoch:  949\n",
            "Train loss:  0.011655527\n",
            "Validation loss:  [[0.05692596]]\n",
            "------ Epoch:  950\n",
            "Train loss:  0.011640134\n",
            "Validation loss:  [[0.05685868]]\n",
            "------ Epoch:  951\n",
            "Train loss:  0.011624759\n",
            "Validation loss:  [[0.05679153]]\n",
            "------ Epoch:  952\n",
            "Train loss:  0.011609421\n",
            "Validation loss:  [[0.05672435]]\n",
            "------ Epoch:  953\n",
            "Train loss:  0.011594135\n",
            "Validation loss:  [[0.05665711]]\n",
            "------ Epoch:  954\n",
            "Train loss:  0.0115789035\n",
            "Validation loss:  [[0.05659003]]\n",
            "------ Epoch:  955\n",
            "Train loss:  0.011563718\n",
            "Validation loss:  [[0.05652276]]\n",
            "------ Epoch:  956\n",
            "Train loss:  0.011548608\n",
            "Validation loss:  [[0.05645565]]\n",
            "------ Epoch:  957\n",
            "Train loss:  0.011533464\n",
            "Validation loss:  [[0.05638837]]\n",
            "------ Epoch:  958\n",
            "Train loss:  0.011518397\n",
            "Validation loss:  [[0.05632131]]\n",
            "------ Epoch:  959\n",
            "Train loss:  0.011503367\n",
            "Validation loss:  [[0.05625406]]\n",
            "------ Epoch:  960\n",
            "Train loss:  0.0114883585\n",
            "Validation loss:  [[0.05618691]]\n",
            "------ Epoch:  961\n",
            "Train loss:  0.011473428\n",
            "Validation loss:  [[0.0561197]]\n",
            "------ Epoch:  962\n",
            "Train loss:  0.011458532\n",
            "Validation loss:  [[0.05605258]]\n",
            "------ Epoch:  963\n",
            "Train loss:  0.011443676\n",
            "Validation loss:  [[0.05598529]]\n",
            "------ Epoch:  964\n",
            "Train loss:  0.011428829\n",
            "Validation loss:  [[0.05591817]]\n",
            "------ Epoch:  965\n",
            "Train loss:  0.011414051\n",
            "Validation loss:  [[0.05585093]]\n",
            "------ Epoch:  966\n",
            "Train loss:  0.0113993045\n",
            "Validation loss:  [[0.05578389]]\n",
            "------ Epoch:  967\n",
            "Train loss:  0.01138459\n",
            "Validation loss:  [[0.05571662]]\n",
            "------ Epoch:  968\n",
            "Train loss:  0.011369982\n",
            "Validation loss:  [[0.05564955]]\n",
            "------ Epoch:  969\n",
            "Train loss:  0.01135532\n",
            "Validation loss:  [[0.0555823]]\n",
            "------ Epoch:  970\n",
            "Train loss:  0.011340709\n",
            "Validation loss:  [[0.05551517]]\n",
            "------ Epoch:  971\n",
            "Train loss:  0.011326134\n",
            "Validation loss:  [[0.05544787]]\n",
            "------ Epoch:  972\n",
            "Train loss:  0.011311659\n",
            "Validation loss:  [[0.05538072]]\n",
            "------ Epoch:  973\n",
            "Train loss:  0.011297179\n",
            "Validation loss:  [[0.05531348]]\n",
            "------ Epoch:  974\n",
            "Train loss:  0.011282806\n",
            "Validation loss:  [[0.05524641]]\n",
            "------ Epoch:  975\n",
            "Train loss:  0.011268391\n",
            "Validation loss:  [[0.0551791]]\n",
            "------ Epoch:  976\n",
            "Train loss:  0.011254009\n",
            "Validation loss:  [[0.05511199]]\n",
            "------ Epoch:  977\n",
            "Train loss:  0.011239656\n",
            "Validation loss:  [[0.05504476]]\n",
            "------ Epoch:  978\n",
            "Train loss:  0.011225422\n",
            "Validation loss:  [[0.05497756]]\n",
            "------ Epoch:  979\n",
            "Train loss:  0.011211146\n",
            "Validation loss:  [[0.05491038]]\n",
            "------ Epoch:  980\n",
            "Train loss:  0.011196963\n",
            "Validation loss:  [[0.05484332]]\n",
            "------ Epoch:  981\n",
            "Train loss:  0.011182794\n",
            "Validation loss:  [[0.05477605]]\n",
            "------ Epoch:  982\n",
            "Train loss:  0.011168649\n",
            "Validation loss:  [[0.05470886]]\n",
            "------ Epoch:  983\n",
            "Train loss:  0.011154535\n",
            "Validation loss:  [[0.05464168]]\n",
            "------ Epoch:  984\n",
            "Train loss:  0.011140468\n",
            "Validation loss:  [[0.05457447]]\n",
            "------ Epoch:  985\n",
            "Train loss:  0.011126437\n",
            "Validation loss:  [[0.05450727]]\n",
            "------ Epoch:  986\n",
            "Train loss:  0.01111245\n",
            "Validation loss:  [[0.05444006]]\n",
            "------ Epoch:  987\n",
            "Train loss:  0.011098488\n",
            "Validation loss:  [[0.0543729]]\n",
            "------ Epoch:  988\n",
            "Train loss:  0.011084528\n",
            "Validation loss:  [[0.05430571]]\n",
            "------ Epoch:  989\n",
            "Train loss:  0.011070625\n",
            "Validation loss:  [[0.05423852]]\n",
            "------ Epoch:  990\n",
            "Train loss:  0.01105677\n",
            "Validation loss:  [[0.05417133]]\n",
            "------ Epoch:  991\n",
            "Train loss:  0.011042974\n",
            "Validation loss:  [[0.0541041]]\n",
            "------ Epoch:  992\n",
            "Train loss:  0.011029151\n",
            "Validation loss:  [[0.05403692]]\n",
            "------ Epoch:  993\n",
            "Train loss:  0.01101543\n",
            "Validation loss:  [[0.05396964]]\n",
            "------ Epoch:  994\n",
            "Train loss:  0.011001659\n",
            "Validation loss:  [[0.05390242]]\n",
            "------ Epoch:  995\n",
            "Train loss:  0.010988005\n",
            "Validation loss:  [[0.0538353]]\n",
            "------ Epoch:  996\n",
            "Train loss:  0.010974296\n",
            "Validation loss:  [[0.05376803]]\n",
            "------ Epoch:  997\n",
            "Train loss:  0.01096067\n",
            "Validation loss:  [[0.05370078]]\n",
            "------ Epoch:  998\n",
            "Train loss:  0.010947091\n",
            "Validation loss:  [[0.05363354]]\n",
            "------ Epoch:  999\n",
            "Train loss:  0.010933554\n",
            "Validation loss:  [[0.05356625]]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ylr1C2ovWXbO"
      },
      "source": [
        "During the training of the regularized model we can already notice, that, although there is still a difference between training and validation loss, the validation loss decreases as the training loss dreases. The effect of the regularization becomes even more evident if we plot the predictions of the regularized model and the overfitting model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab_type": "code",
        "id": "UdMRV0vAWLmm",
        "outputId": "e100c956-4d43-42a2-a391-b627da9f37af",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        }
      },
      "source": [
        "\"\"\" We now want to plot the prediction of the regularized and unregularized big model. \"\"\"\n",
        "\n",
        "y_pred = big_reg_mdl.__call__(x) # Predict with \"big_reg_mdl\" on \"x\"\n",
        "y_pred_overfit = big_mdl.__call__(x)# Predict with \"reg_mdl\" on \"x\"\n",
        "plt.scatter(x_train_overfit, y_train_overfit)\n",
        "plt.plot(x, y_true,'r.')\n",
        "plt.plot(x, y_pred.numpy(),'g.')\n",
        "plt.plot(x, y_pred_overfit.numpy(),'b.')\n",
        "plt.legend([\"Target\", \"Regularization\", \"No regularization\", \"Training samples\"])\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9eXyU1dn//z6TEAhrgFDDEggiOyEJ+50ApoKiBUOEYm3RYEUDWCtaSwL+qtL6VEji3keWuEf4ubQiINQHChpDyEQMEmQTwYoCArKIbIEsc75/3LPds4Qsk8wkOe/Xa16Zc+7tTGbmM9d9netcl5BSolAoFIrGj8nfA1AoFApF/aAEX6FQKJoISvAVCoWiiaAEX6FQKJoISvAVCoWiiRDs7wF4Izw8XEZFRfl7GAqFQtGg2L59+ykpZSdP2wJW8KOioigqKvL3MBQKhaJBIYT4zts25dJRKBSKJoISfIVCoWgiKMFXKBSKJoISfIVCoWgiKMFXKBSKJoISfIVCoWgiKMEPcMxmuO02GDkSsrP9PRqFQtGQCdg4fIUu8LNngy2D9bZt8P77sGGDf8elUCgaJsrCD1DMZqPY29i4Ee680z9jUigUDRsl+AFKZqa72NtYuVL/QVAoFIrqoAQ/ADGbYc2ayvfJzKyfsSgUisaDEvwAxJN136qVsb12rbLyFQpF9VCCH2CYzbqYO2MywbPP6n9tWCyQk1O/Y1MoFA0bJfgBxvz5upg7s3QppKbC6NHG/r17629cvsB82Mz1b1xP66da02ZRG65//XrMh9VtikJRXyjBDyDMZsjLM/b176+LPcCAAcZtW7c2HLdO+qZ04l+LJ++7PC6WXeRC6QXyvs8j/rV40jel+3t4CkWTQAl+AOFpIvahhxzPU1IgKMjRlhJyc+t8WLUmfVM6mVu9zzJnbs1Uoq9Q1ANK8AMET5E5sbEO6x5A0+CRRxxtiwU6dqyf8dWU7O3ZlYq9jcytmcq9o1DUMUrwAwRPkTmjRrnvd+6csf3RR3U3ptpiPmxmzvo5Vd5//ub5dTgahUKhBD8A8GTdm0y6C+dqfPhh4PrxMwsysUjjDLRAsHzScpZPWu62f953eWRvVwmDFIq6Qgl+AHD//e7W/dKlugvHFVc/vsUSmH5882Eza75yXz22bNIyUoemkjo0lbSENLftc9bPUa4dhaKOUILvZ+68E4qLjX0DBhh99864+vGlhLNn6258NSVnZw4S469Ycr9kUoc6XljG+AzGdh9r2MciLeTsVAsMFHXD6h1HSVj8MT3nrydh8ces3nHU30OqV5Tg+5H0dD0vjitz51Z+XFgYCOFoP/NM4Ll19p40LhIQCNLi3S36xeMXIxCGPtdjFQpfsHrHURas2sXRsyVI4OjZEhas2tWkRF8Jvp/IzobMTAvYrWAJSPrfUOTVureRmGhcdVtREVirbrO3Z7Pl+y2Gvsn9JqNFuvuotEiNyf0mG/ryD+crt47C52Rt2E9JWYWhr6SsgqwN+/00ovpHCb4fuP6Or5k1ywII68NKxA72jR1OeGZ4pZOXmgYJCca+QFl1a4vMcXbnmITJo3VvIy0+DZNwfBQt0lKlUE6Fojr8cLakWv2NESX49Uzc+O/Je7c3RrGXgAUm/gGA0yWnmbVuFneu8p743nXVbX5+YLh1cnbmuEXmJPVJ8mjd29AiNZL6JBn61n69Vln5Cp/SJSy0Wv2NESX49Uh6OhRvjrS2nMUeSMiEyELD/it3rSR2WaxH4UtJcU+m5rxSNzsbevaEli2hQwf92vXB3lMefPceonFcSUtwt/LV5K3Cl8yb0JfQZkGGvtBmQcyb0NdPI6p/lODXEw6fPRjFXkJ8JtzwhMfjdp7Y6THfjKa5J1NbvVpPo9ysGcyaBYcOQUkJ/PST/mPQsmXdCr/5sJn87/INfZP7evbdu6JFaozubnxBrj8eCkVtSI7ryqIp0XQNC0UAXcNCWTQlmuS4rv4eWr2hBL8eMJthzhxwd+NImv3qWXqMGUR8y/+r1BLO3JpJ8/9pTpdnunDbO7dhPmy2unWcQx8lly5JysulS79OSYku/H361I37J2dnDhYc7hwTpipZ9zYGhBv9VFu/36rcOgqfkhzXla3zb+DbxRPZOv+GJiX2oAS/XtBTHktcxV5MnEeX6P7228qM8Rksn7TcLUzRRmlFKccuHGP1/tXEvxbPy+VjgAoc4i5cHtLp4eDAAYiP1+86fIlrOOXo7qOrZN3bSIlJIUg4brkt0kLuoVxfDU+haPL4RPCFEK8JIX4UQuz2sl0IIV4UQhwUQnwphBjii+t6w3zYzKIti/xqHdoWeHS+ayt5ec6TmFYBnjSL9jFX3G4rU4emsvWercRcE3PVa1R0y4dJc3CIvusD6zbwJPyzZvlO9M2HzeQfNrpzBnQa4GVvz2iRGo/EO1aVSSQdWwZ4djhFgyM9HTp1gnbtdDdny5bQti20bw/XXx8YwQ91ha8s/DeAmyvZfgvQ2/pIBZb66LpumA+bGZczjsc+eYxxOeP8IvrOCzx+/uxajBY3kJDB2Klfc/rv/+vxtlKL1CieXVw1d8iwV2DmGOjxKTQ/AyFnIegitDgDCYthYTOYlAqhp6wHGF1As2dZeCloDqWt20KbNvon3/lb0KqV/ujSBW67zeu3wTVvjkmYSInxngzIeVK5ZUv9eXY2nLtszA730YEAzg6naFCYzdCt/TkyMyWnTknOnZOUlOiP8+clZ89K8vL0u987vQfINWiEdE3iUtMTCREFrJNSDvKwbTmQK6V829reDyRKKY95O9+wYcNkUVFRtcexaMsiHvvkMSpkBUEiiCd/+SQLxiyo9nlqQ8Lijzl6toQrR8M4vlID6ST4PT7FdM848n+fXyV3h/mwmcytmRQeKeTM5TOUVpR63E9UmGheDhUmC2XBrhutf99/E3bd5dKpW/7LmcV9vOLFmeRCSAg0bw5xcbB4MeZukPBagiH2PrlvMh/c8YG9nZ0Nzz8PJ07oGT/Lyz2fOqh5CRVtv4GRL8CwVwgSQWz5/ZZquYYUClfS7zxC5souOD73V/+kjxgBn31Wp8OqE4QQ26WUwzxtqy8fflfgsFP7iLXPgBAiVQhRJIQoOnnyZI0ulBiVSLApGIEg2BRMYlRijc5TG2wLOS7s7moUe1EB4xdUy7etRWp8cMcHHPvzMa785QrLJy2nR7setCaEtleg00X4U76g7EkLlxZZuPJ3SMuHThegeZn1JDZvztQZEP2WUyfY7j5msZyXubdqL7C0FM6fx2YOzX8iAWfDwTkUMz1dv3GYNQv27YMzZ7yLPUDFlRZwciCsy4ac9cqPr6g1d074kcyVXfE+v+V5rmvbtroLcPAXrragX5FSZgPZoFv4NT6P9Y1zTd5VX3QJC+Xo2RJKT7U2buieD5GFDAifXf2TpqfD66+TeuUKqSUlUFZm32R7nTabJWOz/gDIHgILE+FYGxyi3/YobLXFZzoih2axnDzGsIIZVR6WuRtsiXT6P0u4ZsefmPmuxrffwuXL1X2hTnce/70F+eI+9rTfBGOqex6FQhf7lRs7WVvGtS8hXCHIOv8lsHAJ5++rvu+BA3r4c36+5+y1DY36svCPApFO7W7WPp+zxLyW0s9mIN/6N6WfzWCJeW2l+6dvSqdTVifaLmpLm6fa0Pnpzvawx5oyb0JfONGR0qMdjBs67cNE5b5tO85O7uBgPZ7y5EndH+Ik9uAp2NPxSP0CfngW0gqEo/PGRyEhw+kIx1lWchcd+ZHs4Dm6aW67vhcy451uYoruhYwfOb42i337JJcvX/0HNyjImO7Z+KqAM31YOTeV7NW7rnouhcKZ9DuPeBF7yXTe4kqrcC6FdOASbbhIOwpIoBvfO+2nY7HoKcwbBVJKnzyAKGC3l20TgY/Q/+ujgG1XO9/QoUNldfngiyOyza/elmCxP1pqG+UHXxzxuP9NOTdJFuLxIRYKubxoebXHYGPC1PPWMUj9ryiTzBwlk99J9nzA8uVSDhggZYcOUgYHS+uBVXpYXB6XRZC8GBwiL7dsLWWnTlKmpemXKFpufJ3Rbzr9r9xP3bu3lAUFTuPr31+WtGsvS4KCZQXIrd2QxD8laXlMEnTB8H83nsvYDgmRMizMPiwppf48LMx5f4vh+NaR/63xe6FoehQUSAnlxu8gFgllcnnXvzp9sKX+2Xb6gN7Eeo+fY+fPayADFElvOu1tQ3UewNvAMaAM3T8/E5gNzLZuF8BLwDfALmDY1c5ZE8GPX7RZBnU+4PYmdxn7rdu+af9J8yr2zqJf8H2B27FVYexYaRxHj0+k6a8mx/kKCvSdWreWslmzagm84REUJGVoqLzcrr38plMPmT7hDzJ+0WavP3Juop/wlISKSoU/JETKtm2lbN6yTIqQK5KgMmkyXZFClLiIvCeht/7wmi7K5GTj98wTBQVSRkRecTmf/jx5xs81ei8UTY+xsT+5GQ0RHJEFNz3h+YCCAikjIuwf3ul4NoaW19wGrDfqXPDr4lETwY9KXydDeu93e6PBYnij3ESvkkfM0phqj6OgQEqTyUXwhy2RY//UQf9QhYTUXOBbtJAyNNTdRK7O+L4vkBFZEY7XOXOUpP1XlYq+q4BXReShQoZxSqbxlL6xY8cqfWPiF22WzQYWehT9hvCFU/gX3WCvMHx2BBXexd6Gfltgf8Sy3e3zbTJd3WjxN5UJfqNaadslLJQOI08Cznnmdd/dokV6y1th7SARREhQiN4ouhde2gXPHWTnC48zYdFCj9fzVj0nM9N5Za0EKmBwDgP2n4Hjx/Uol6oSFAShoRAVBcuX6/kRLl3SE+RkZFz1cE9okRqrfrPK0RFZCHP76XH7gON/54rrSl7PK3pDKGEsn1JAAj8RTgaP6oefPq2H68TFVRr68MPZElolL4FrdrhcW09R0ZiiJhS+xWyGObMqcF37Mq/NMrQNCys/WNMgOdneXMIfEIaV7O5JChsajUrw503oS1jUeVre8i7OAgR6IrH0dO+Ftbf8fgu5467QetlRPSTw5ED4+Vr46jY2Pvo4I3/zseEY1+o5Y3JXMfAmjU9bT2Dtape4w35rEd0KSdlZxRfSvLle1iotTY9hvHQJvv3We93DGqBFau4Lu258FGbGE9R5N55FX3p5AKbLiJYX6aD9l42vb+LTsU+gUejhHOg1HRMSvC7z7RIWSgtLNEz8I66pIywWPVWFQuGJzPmnsWDCOc5+LJ+S8XQVAxLT0uxpaDUKWcYcXLVk9WrfpyWpN7yZ/v5+1MSlI6U+cdv9yZmSSfe63daBRTLpPje3zfKi5TItzdUt4e6maBezR/eNFxTIT/onyGMt28sLQSHysinIPmE6iyXGa4oyyT2j5NgZlbhpQkJ0V09VnNw+xutcxkxN9o47Itu0kbJlS1cf/gX9EXJGn7BNeEqG/mWU+9xBQYE+81uZi8qDW+qDL47Ifn/5SIb+ZZT1fSx3e0+Ua0fhSkGBlIIyw/fPRPnVXTmuuEziJvN+g3Lt0FR8+DYKvi+QQX8NkvT4xOWN0v3KTLrXLmzJbyfL6dO96ZH7XMBvWOEWFWN7SJCzXQW/7/tSPIYs6OZ04uBgKdu00SdtA+BTU9kEtmukUuz/Xq9ve8Kxj2Ey2uMF0rz9g/XHTTe5HfLBF0fktU8+rF8n4Sm3+YJA/sIp/EPy2FNu39nktptqdjI96kJKkAWMkibDD4n+SPYScOdvKhP8RuXSsaFFaiREJsD4BXjMJrkuW08zUHQfH898i5Urjbdsxls441zAu/yO+Tzl0ZMN8N/QVsbjen/E5K8FWklH6N9f98OXlenx9J9+GhCrOTLGZzA9errHbbPWzaLnswPp+uQdBD3egeJTn+obnFamL524tPKVwxkZUFAAsbGet2/cCCNHGrqS47py7/WdEELorqYenxq2WyyBVcdX4V/MZlib196pR2LCQtoN22t2wsWL7U81Cllqd+04WLOm4c0nNUrBJz2dAes/g26F1mySzgJuVapdd8G65Zy71MrYb903hh30j3gLxwSwtO+TSTrZ3Ovmyc7mXjaWOOeqsSBKOpGWtRVOndILz/rQD+9LVkxZ4VX0D53byw8V72Ix/eTm2p8ePZ3UoVV4TZoGO3bAdM/XYNs2mDDB0JUYlUiQyboqy+3HO3Dq+Cr8T2YmWFxqRCexBi2t+ku00zel84v8ybRbIGifBtfPgOhur5DMapw/f1I2PKOj8Ql+ejpkZpLy2RWEBIa+4mVVKU7PnSNNKljOLIoZyt7jM7hp7Gho9YPb/rNYxsvcS5kpiEtBIZxq04EnW8932ke/VveYbxtM4i+vou+cb8rpX3dd++tYMWVFNS+yQr/L6egh7fHGjQbR1yI1/qT9SW9EFkI/46rpQKnjq/AvZjOsXWNMQW6igrSx26p0B529PZuez/ekzVNtaP5kczK3ZnLy0knONZecDYW8KIifCf9OycLV6Cj0EpcQqDQ+wV+lhxtqR6C/Lf/ajY9aQw5drXVjbo3efE0BY0jlFXvv/+UVEvGrX4NwDfU0MYtlbM8vp1X5FT54+jRHLvQ0jqXHFh793S/r5nXWESumrDBG77j+RtpvaUzk3FZD8yY1Vb/jGTHCfdvGjdC7t13Jw5qHOQrCJGQZ3gcVsaMAa2SOc5JCLPwv9/P6gK72UGlPZG/PJjwznFnrZnHo50NcKLtAqcUlZNpJJkp7uhodkuJiyZ1zjvj2BdUhjU/wp0yxP51rS20qgfGPwj2joY1zrgzHYzpv8XVoHFrbvXrce2gook0brrTvyC2mcpg422l/sIn++Ek/k774G2bNdo39rcB0QxZv/ue6Sj90gUjG+AwK7imgd/veHgtnhVQMJDrk+drfuXz2Gdx0k3v/wYMwZgyYzSRGJTqKm0cWQsd9hl23bFFWflPG3XcPY9hCz9672NjuWhas2uX2/TMfNhO3LI5Z62ZxuuR01S5k+2onZOEwHPVfgpXLO9MnfUaDKMfZ+AQ/I0OPpQ0PJ/VAG5b/XzD9TwkizkPf0EJe6N+bP5uephM/EsExkoPXURD7ACsKeuvx7j//rP+9dAnOnaPFmVO89s7npM3t4ME1BJfOtCVzQU+Qxthf+q4lrHNPjp4t8fihC3S0SI2vH/ya2YMzCJGRBFl+QWjFKCJKs7iWp/nbLVOufpKqsGGDZ9GvqIA5c9AiNW7tc6ujf9QLht2kbNgLYRS1w5Pvvj/7yB45FYCSsgqyNuy3b0vflE78a/EUnyi+6rmDsM4fORs83QohwfaBs4q+NHFg7WTiX4snbnlcQAu/zwqg+JqaFkCpS9I3pZP5p4FeiojY2votZfO7HyTiml/Zj+0aFsrW+TfU32B9yOodR8nasJ8fzpbQJSyUeRP6+r7484QJujvHlenTyX54LLPWzbJ3dX3/IEd39bK3TabGk75WUXXMZhidYDG4c0xU8HS3Obww3bFiVgDfLp7InavuZOWulV7P18zUjNDgUJoHN+f3cb8nY3wG2duzeeGDdI6XnuVyM7gUbD3h8u1wPA7nO3pmjtHvQoHYa2JZMnGJX+bvKiuAElD58AOdjPEZHF14JysXYhV9z3MBJu1Ng9iDoyhKQyQ5rqvvBd6VDRv0unIrXb6QK1dyuv1uRLhAWv+/x0fejdjzKdKi36DaQjSV4DctcnJw8d1X8L/cz38S+xr26xIWelWxnx493WMAQurQVFJLo/Wk+BYL5m4wfxxs7bKNiuNx1r0EEAQ7U+yCX3yimPjX4klLSCNjfM1SoNQFjc+lU8esmLKC5a9fplnse9Yeo4M7JPo/RI69xu24LmGh9TbGBsuKFR5j9RNX7yRYOm7ZLd220iP6sGEfFaLZ9Nhb+LOhPZZ8evXZzRdd+9v7QpsFERz+klexj70mloJ7CiqPNtM0SErSnx6BT9+ELcUrEIZADuDISLdDM7dm0jGjI9nbAyMXgxL8GpA6NJXSHb/hpofeR4R9ByHnCen4A2mL/su7bw4ktJmxokdosyC9KIri6ixZ4lYRRTsCD+c7wu4kkrDIHwz7qBDNpoXZDPnFxopyA/iK5gvm0zUsFIHuRv1Fj5fJO7rK4znSEtLYMXtH1dwuaWmGz6UmC5jc7yCGebvjcfCfp9wOPXP5DLPWzSJ2Wazf/ftK8GvBhud+jeWnKOSVtlw51ZWM+b1IjuvKoinRhg/doinRde8SaSxomh5607mzoTvsCvq6CitfRsxDmBw/AmrlbdMiJwenJGm67z4l9kuuvzuJrfNv4NvFExk9bINHse/cujMF9xRUz9WiafDII462lKTdug9h13vrk4J0ODzK4yl2nthJ/GvxpG9K97i9PlCCXwO8pUW2kRzX1f6h2zr/BiX21UXT4P33DV2Jh8DkdAdt6baViP4HDfsot07TYa/Z6M4ZTT7aKIdFYD5sJnOre/iWCRPv3/5+zSZTz50zNLWv32TePOcePWJn7NlX6BjqYWGhlcytmfR+sbdfrH0l+NXENS1yQw27DHg0Tb+NtjWPwK37jbuEdT9maCu3TtPAbIb8na7unH2Q4qgVff969yK0Ea0iyL8n33eRM2vXkpFsdpt2OvfNQE6lnfKaqgTg4E8HiX8tnuvfuL5ehV8JfjXJ2rCfkrIKQ59rrK/CR2RkGHLv3GIz6K2G3JCbd9tSlwPKrdNU0GPvXd05u+xhWneuutMtzl4gWPWbVbUT+5QUPH3gRrl4cIqL9QwvK6asoOCeAmKv8ZI0EMj7Lq9e3TxK8KuJt/DKhhx2GdA4Re6cbmn141uj8N45+yCjx58y7H78eP0PUVF/mM2w1phSiSTW2t052duzPUbkzEuYV3vLXtP08Exnjh8nJQUnX75OVpY+Vi1SY8fsHSyftPyqbp64ZXW/aEsJfjXxFl6pwi7rkCVLQAjdj++UCqnCYqHDja8R7LSaZP165dZpzOTkGMuHmqggLeg5uztnUf4it2OmR0/3XSz8gAFuXZqGiy/fPZNm6tBUTqWd4qZrPawqt2KL3a9La18JfjWZN6GvCrusbzQNli3z6MePkOuZNMnRLitTbp3GjOvE/Gjy0W4NB00je3s2h84eMmyPjYitfkbXykhJgWbNHG2rhZGRATExxl09GR4b7trA8knLiWgd4fUSdRm7rwS/mqiwSz+RmgppaW5+/LaFO9x2VdE6jROzGfK3GOtR92cfKw5dIWHxx/xl85OGbQLBkl8t8e0gNA0mTnS0y8rsyZxcV3rv3Om59m3q0FSOPXLMvaa0E7bYfV+LvhL8GqDCLv1ERgane3V2+PGBZwaeh68+NOymonUaJ66pFExUMJ23WDXoBnZdWMLJS8Y0xWO6j6mbXDYRLtb5hx+C2ezRl/+CMdefAVtW2tgI75O67+993+u2mqAEX9GgSPz9QqMf3wRnujyFCUfklMWiMmg2Rjy5c5p1O4c5Es4HGxdYCQSLxy+mTkhJMa4GlxJyc9E0Pau3M/v2VW58aJEaO2bpk7odQju4bZ86YKqPBq2jBF/RoNBuSeVWeZ2hLz+hkNEt1uCc12TtWmXlNya8uXMOhnfnTNAbgHQuNOebqBxvuK66tVjsFdwWLzZa+VJWrUhP6tBUTqedJi0hjU4tOxEVFsXyScurVj60GijBVzQ40lJzDFa+xQQdhmRhQqVaaKxkZnpy56zg5WHXUhrkZPoL6NGuR91nqHRZdctHHwH6b8HkycZNeXmeffmeyBifwY/zfuTbud/6XOxBCb6iAaJFaozu4BIS0a2QJIxWvqJx4C32ftfIzhRG5OJq3T865tF6HiF2Pz4YFojbef75eh6PF5TgKxokA3q5367fwkeGdtu29TUaRV2ix96Ds3U/jyyKIr6iNGiPY0cBY7uPrRPL2A1XP77FArm5gG7ljx1r3H3//sBwMSrBVzRIUmJSCBZBdqtufR/Y0TYc4eTWeeaZwPiSKWqHPlnruHMbTT4ahWzodUjvcKq7XGcTta54yJ5p8+OD7st3zcIQCIEESvAVDRItUmNS31vtBcfKguD4oFyrH18Xh4oK5cdv6JjNepitM/3ZR343+KH1RUN/G1N0/ZYUdPXj73CsCfGUhWHNGv8bIErwFQ0Xl5jnM70LSWCroU/l1mnYeHLnpJDD/JvaO6qLSn37own/48eR4vZhc83CIKX/rXwl+IoGS0Qr4wKY/O7QodU+px41gdvQ0TXU8T4msRYR9Tnm7mcdP/gC+nccwfxxSfU7uJQUCA62FzgtXfshqXP+YU+V7ppcE/wfLqwEX9FgSYlJwWT7CFvDM8+MzKEZpdhEQiVTa7iYzbBuna0laUYZaWSRmfwLe0F7G9f3jHM7vs7RNI6OHg/ovz3NLBWMLfzIXh9D02DpUmNcvr99+T4RfCHEzUKI/UKIg0IIt2UGQoi7hRAnhRDF1se9vriuommjRWok9TVadfmjC9FarLe2BGVl0u+30YqakZMD5eVgM+Unsg66FbK2nbHwjQkTKTEp7ieoBz67FGxoh1/8yVAfIzXVffWtP638Wgu+ECIIeAm4BRgA/FYI4Z5DFN6VUsZaH6/U9roKBeiFqE3CycoXcOoXJwz7+Ps2WlEzXKNzAHJiwOKy39JJS+t3staJFX2up8wUZB/lDd8UMeToPkN9DFdfvj+tfF9Y+COAg1LK/0opS4F3gMlXOUah8AlapMbo7sZwiJCBOdbcOvpSXItFqmidBoan6JwITrC3E4bJ+nqLu/fCiYFD+LjXcMDm1ilnyu6PDfUxPPny/RWx4wvB7wocdmofsfa5MlUI8aUQ4l9CiEhPJxJCpAohioQQRSdPnvTB0BRNgQHhTiaUgOLhhfRqZVSL43tPoWg4eIrOieuUQ34P434DOnlyJtQf8yb05ac2xqRnQSZhqI9h8+U7U9UcO76mviZtPwSipJSDgf8Ab3raSUqZLaUcJqUc1qlTp3oamqKhY5i8BRBwoN8+404HDqJoaBijc3aMKMTiZN3703dvIzmuK/1uuR5wjHZIdJRbyvTUVHfXzpYt9W/l+0LwjwLOFns3a58dKeVpKeUVa/MVYKgPrqtQANbJ235Ok7cCiMnBJMqwfQ3XHYvDnL3LL+NTVB9HWgz9/buFj3R3jhOje4z2m+/emaLz0CsAACAASURBVLhWFQgh7MsC+v7/L3tU8rlzjW1/xOX7QvA/B3oLIXoKIUKAOwBDqiMhRGenZhLgYn4pFLUjLT4N4ezcjSwkrJstpk9QTgg5z5/xy9gU1cNs1tNi6AhMWNjRNpz87sb9DK48f5KYaMyrU1Fhz6vjTGqqexnE+vbl11rwpZTlwAPABnQhf09KuUcI8TchhM3selAIsUcIsRN4ELi7ttdVKJzRIjUm93OKFRBwpqVxn73fuXQoApLMTF0zbf57gYW9w3KxOKlVILhz7Gga/OlPjrZTfnxPuzpT31a+T3z4Usp/Syn7SCl7SSn/bu17XEq51vp8gZRyoJQyRkr5SynlV764rkLhTFq8U4gmQGtjeGb+pTjMd75Uz6NSVAdHKmSH/z6hxYfkJxQa9kvqlxQQ7hw7YWGOUBwhDHl1nPF3xI5aaatoNGiRGqMjnUI0Y3PAOTyTIHJWBqmg/ADGU3ROhyFZRutemEiL914A3C8kJkKwdRGWlPDqqx4/Z/6O2FGCr2hUGML0Iguhr7FyxnF+4f8MVooqk8RazvQ2WvejIwNjstaApsGvfuVol5V5TdXqz4gdJfiKRkVKTIpx8nZ0FpiccuswCfPqE8rKD1Bco3P6hH2kT9Y6vaX+jr33SkTE1fex4ilipz6sfCX4ikaFFqkxL2GeoyOyEPo45dahGTncpaz8AMQRnaO74AQVrIsMD9zJWlfiXBK4VVJyLTUVergsIsvLq3s7RAm+otGRMT6D2GtivW7fS//AqEahMJCT4xqdI9k3LNewT8BN1jpz+rQxNeazz1b6GXvUQ+ndurbyleArGiWjuo1yNFyjdRiNWY70z9p2RZXpfs2HyO6FdneOiQCcrHXGeeIWvMbj20hNda99m5cH2dl1MjpACb6ikZISk+II0Yxxj9bJZJ5/1rYrvKJ7QGzlRKBZT2NR+kBZWesVTYOHH3a0XercemLxYuNNAcBTT9XB2KwowVc0SrRIjaQ+1nV/kYXQzxits5Yk3cpXaTQDAtfVtYIKDoaGG/YJmJW1lREW5lBwk0l381SCprnny//uO7jzzroZnhJ8RaMlLcEp3UJCFgiXmHxSoLCwslMo6gnX1bUgkT1zDe6cgJ2sdaZjR92yh0pX3DrjycpfubJuXDtK8BWNFkPEjoeYfACKiyE9vX4HpjBgNsOHHxr7OkR+qL9nVkZ3D3B3jo3Tp41Lab2suHVG02DePPf+RYt8OC4rSvAVjRpDxM51Np+wboG15azezMpSvnw/kptrW10L+uracs6MzzLsE7Cx9664Ttx6WXHrSkYGxLoElh065HsrXwm+otFjj9gpCUefvNXvn5/hz5gZpd+CK1++33B4QfQf4l59n3aLzmkQ7hyo1opbV5YscXftvP++D8eGEnxFE8AesROVC8IWBSKoIFj344OtgKrCD3xkD8bR1e5g6zDD9oCOvfeE64rb48erdJgn187UqT4akxUl+IpGjxapsXTiUkTkZ9B9q2HbXvrrT/LzlVvHD3jy30uwz90GiaDAjr33REoKNGvmaP/731X+bGVkwPLlcNNN+t9UH5frVYKvaBKkDk2lX3g/6GSsvbOF0bpbx2JRC7H8gKv/XlCur5uwBro8Ev9Iw7LuQTfVZ850+GfKyytdgOVKaips2OB7sQcl+IomxEOjHtLFxCk8U9oWYYFaiOUHzp41+u+v6/u0Hp1jrRd47vI5fw6v5sTFVTs8sz5Qgq9oMqQOTSV22GW38Mw1JDkmb1VStXrD42Krrg1U4F1xzqsjxFUXYNUXSvAVTYpR3UZZF2FZcLby52MNel67Vln59YQjWRrYF1tF5dr99ybRgKJzXHFegCWlfisTACjBVzQpUmJSMHXfBh1dffljHb58FaLpFwyLrUQDWmzliWpmzqwvlOArmhS2iB1GvWjtsVn5whGiqdIt1AuuxU7ODDYmS2sQuXO8Uc3MmfWFEnxFkyN1aCr9b86HCOOy98IQa4imSrdQ57j678GCLAk3hGM2WHcO1ChzZn2gBF/RJHlo1EPQ7TNDX3GpNUQTVLqFOsbVfy+ERV8Y15DDMV1xzpwZIBO3SvAVTZLUoan0vqHQEKIJQcwJsU7eqnQL9Yrs/aE9HNMkBGHNw65+UKDjOnGrLHyFwn+MG9PSLURzZ+n1ZHOv3lDpFuoMV/89fT6yu3OEMJEYleifgfkS58yZQlQpc2ZdowRf0WRJiUlBJDwDOEI0ARYFzdV3UOkW6gRP/nsuOYqdJHRPaPjuHDBO3EpZ5cyZdYkSfEWTRYvUWDZnBvTYYug/VDFAt/JViGad4BZ/b/PfA4gGHp3jTC0yZ9YVSvAVTZrUoan0mLoc55q3IJjNUn0CV7l1fI5b8sg+H0L3wsYRneNKDTNn1hVK8BVNnkd/90uXmrdOOXa2bKmbWnNNFLMZ1q+3tSQmSvWVz40pOseZlBRjPP5HH/nVraMEX9HkSR2aSuyUTTisfJ01tkLnc+b43ffaWMjJ0T0bNix919ujc4SgcUTnOKNpcO+9jvDMsjK/LsBSgq9QAEtm3+Xdyle+/Lqj9QlH7hxE44jOcSWAMmcqwVco0CdwY6duckqqprOaZH0CV/nyfYJbOGbnL+zbEsJiGpc7x0YAZc5Ugq9QWBk1SkLfNU49+gTuHJZi3lKu3Dq1xD0cs8IQjjmg1yh/DKvuCaAFWD4RfCHEzUKI/UKIg0IIt7JBQojmQoh3rds/E0JE+eK6CoUvSYlJQYx+BqMvX2AhiEz5iHLr1BL3cExpT6fQrCEVKq8uAbQAq9aCL4QIAl4CbgEGAL8VQrgG0s4EfpJSXgc8B2TU9roKha/RIjUmj/sFTJqDYzGWzmqSMBcKr8cqakBfazgmMPFy98bpzgH3BVivv+63u0VfWPgjgINSyv9KKUuBd4DJLvtMBt60Pv8XME4Iob49ioAjLSEN0/DXoJ+rayeI+cW/UW6dWuDmv7/OKR1yZPf6Hk79oWlwzz0BEanjC8HvChx2ah+x9nncR0pZDvwM+D+TkELhghapkdQnSY8Nd7Hy8xhD9v3+z4fSELlaOoWIHo1kda03XCN1/FQBK6AmbYUQqUKIIiFE0cmTJ/09HEUTJS0hDRH5GSTY6ts6rcAtTsWcvcuPo2uYeEyn0DPXurq2EfvvbbhWwHruOb/cLfpC8I8CkU7tbtY+j/sIIYKBdoBbbJKUMltKOUxKOaxTp04+GJpCUX20SI0x3cfAjY9Cj0+dtljr3z4R4rexNVQ8plPopvvvH+nx28brv7eRmAhBQY52eblf3Dq+EPzPgd5CiJ5CiBDgDmCtyz5rgRnW578GPpZSShSKAGXx+MUIBIxfgJtr53hvlW2hGrimU8BkTacg9PumsF4D/Ti6ekLT4E9/crT9FJ5Za8G3+uQfADYA+4D3pJR7hBB/E0IkWXd7FegohDgI/AlwC91UKAIJLVJjTI8x+rJ/D66dObMtav62irimU6DPenuytEa7utYTzhWwTCa/LMDyiQ9fSvlvKWUfKWUvKeXfrX2PSynXWp9fllJOk1JeJ6UcIaX8ry+uq1DUJfY0vTc+Ct2Nrh2LFGTO93/JugZJqxP2pwml1zR+d44N5wVYfkqxEFCTtgpFIJESkwLSpBv2Ny7ALblaXjtl5VeBytIpDLi2ka6u9UQALMBSgq9QeEGL1GhvslqfkYUek6upxbeV4zWdgoRgCSkT0vw4unomABZgKcFXKCohLcEqSBLrRKNxAnev+We/jKuh4DGdQs9cACb9HNF03DngWIBlww8LsJTgKxSVMH9cEt3a9NaN08hC6LPGsD1vZxsVsVMJbuGYfT/U/48ALVrU+3j8Tlyc47kf/PhK8BWKq/BY4p8djdFZuJZDVPVRPOMejlkG8Vn27RG9Y/0yLr/ivADLD5E6SvAViquQOjSVsd3H6o3IQuhr9OVbLFL58j3gFo7Ze52jdi2ClJuakP/ehmukTj2nWFCCr1BUgcXjF2MS1q+LwcrXUfVRqkDrE47atRcbabGTq+GaYuHZZ+v19lAJvkJRBbRIjdGRo/WGm5UP+Vukcuu44DEcU+jztmGxTSgc0xnnSB3QZ7TrceJWCb5CUUUGdHLK6Jhg9OVbpKqP4ozHcMyScOvqWkgc28iTpXlD0+Dhhx3tek6xoARfoagiKTEpDrdO90Lonm/Yrtw6DrxWtwISyppYOKYrYWF+W4ClBF+hqCJapMbo7qMdHZ32GbYrt04lOIVjDmjfx8+D8TN+XIClBF+hqAb2/DoAMTm4unUyM70c2MTwWN1KQrMKSClt5MVOroYfF2ApwVcoqkFKTAomnNw6/YyTt2vXqph8s1kPPtFx8t8DEw+AdkMT9d8746cFWErwFYpqoEVqJPVNcnTEZ4FwsvJVTD65uS7+e5PF7r+PCI/SLdymjp8WYCnBVyiqSVpCmtHKj3SZvC30T73SQOHsWdvaIqs7R3sGuhVikpAycYE/hxY4+GkBlhJ8haKaaJEao3tUMnlb3KbJunU8FitvcQ6ApJ8j0G5J9dfQAgs/1bhVgq9Q1AC3yVtntw6mJjt567FYeY9cAG45H+GvYQUerjVu62kBlhJ8haIGpMQ4TTx2L4Q+rpO3KkQT0MMxuxciLHC6vSr+bse1xm09TdwqwVcoaoAWqTG2x1hHR4Lr5G3TXHl7vOQc+v/AGI5pAhLHzfTjyAIQ5xq3QtTLxK0SfIWihiwetxiB9QvrcfK2aRVHWfzGSVbntLa2jOGYCZc6Kv+9K84Tt/WUYkEJvkJRQ7RIjcn9Jjs63CZvWzcpt87zS6+A1GsEuKZTGHC5jT+HFpj4ocatEnyFohakxac58uvEuqy8xdSk3DoXS8uNHX0/hG6FBFkgxdQEi51cjcREaNZMfy4lvPJKnUfqKMFXKGqBW9rkHk3XreMxnQLwsBm03zbBYidXQ9Pgllsc7fLyOp/4UYKvUNQSQ9rkcKNbZ+vOpuHWMZvhxNbu1pY1/r4kHJOE4C691epab0S4hKq6FQH2LUrwFYpaYkibHJsDohybW0fKeq1v4Tdyc8FiMWH335sqoEcuQkJSu4F+Hl0Ak5LicOsA/PvfderWUYKvUNQSLVIjqY81v05kIcQ/bd0isWDi7J4jfhtbfeExnUJkIQmHlTunUjQNZs50hGeWl9ephaAEX6HwAWkJTpO3Lc6hT97qX+JnVkY0areOI52CflfjnE5hQNdY5c65GnFxxrw6dRieGXz1XQKHsrIyjhw5wuXLl/09FEUd0aJFC7p160Yz59vcBoBt8jbv+zw9FFFI65dYUEEQOZnH0D7o7O9h1gmOdAq2cEw9nYLJAinnovw7uIaALa+OlHW+AKtBCf6RI0do06YNUVFRCOfEQ4pGgZSS06dPc+TIEXr27Onv4VSbAZ0G6IIfWQjdt8J319u37f34BNA4Bd8Na3WrpH2gdVD5c66K6wKsOsyc2aBcOpcvX6Zjx45K7BspQgg6duzYYO/gDMVRXBdhnRuEOXuXH0ZV93gLx7zlG6FPSioqpx4zZzYowQeU2DdyGvL7a0ib7Fr+kCBynj/jx9HVDe7++wp7OObpCWOU/74q1GPmzAYn+ApFIGNPm+xpEdbRth6OaNi4+++lPRwz8XwHP4+ugVCPmTOV4FeD06dPExsbS2xsLBEREXTt2tXeLi0t9em1zp49y5IlS3x6TkXdkxKT4kio5ubWiW50bh19nZB0dFj997fuB61M+e+rTFhYveTVqZXgCyE6CCH+I4Q4YP3b3st+FUKIYutjrad96gyzGRYt8olPrGPHjhQXF1NcXMzs2bN5+OGH7e2QEO+5vsvLy71u84YS/IaJFqkxL2Ge3nArjBJE5sKL/hyeTzGbYf16W0uCqRTiszBZIK1Q+e+rRWIiBFtjaKSEV1+tEz9+bS38+cBmKWVvYLO17YkSKWWs9ZHkZR/fYzbDuHHw2GP63zr4B7788ssMHz6cmJgYpk6dyqVLlwC4++67mT17NiNHjiQtLY1vvvmGUaNGER0dzV/+8hdat25tP0dWVhbDhw9n8ODBPPHEEwDMnz+fb775htjYWObNm+fzcSvqjozxGcRGxOpunb4uhVGODWs0Vn5ODpSVgW29AX3WQ2Qho78Hbchk5b+vDpoGv/qVo11WVid5dWor+JOBN63P3wSSa3k+35KbC6WlupOxtLROJkKmTJnC559/zs6dO+nfvz+vvvqqfduRI0coKCjg2WefZe7cucydO5ddu3bRrVs3+z4bN27kwIEDbNu2jeLiYrZv305eXh6LFy+mV69eFBcXk5WV5fNxK+qWUV1H6U8SsnCdvF34l8Os3nHUj6PzDW7uHCsDTgNpanVttXHNq1MH1Fbwr5FSHrM+Pw5c42W/FkKIIiFEoRCi/n4UEhMhJESfAQ8J0ds+Zvfu3YwZM4bo6GhWrlzJnj177NumTZtGkHX23Ww2M23aNAB+97vf2ffZuHEjGzduJC4ujiFDhvDVV19x4MABn49TUb/YQzQ9TN7++HMnFqza1ShE30CrE/piq0HTlXVfE+LiKm/7gKsuvBJCbAI8/fT8f84NKaUUQrj/3Ov0kFIeFUJcC3wshNglpfzGw7VSgVSA7t27u26uPpoGmzfrln1iYp18CO+++25Wr15NTEwMb7zxBrlOdxGtWrW66vFSShYsWMCsWbMM/YcOHfLxSBX1iS1EM++7PH3y1mkR1s7SIdy06l2yWu0nOa6rH0dZc8xmWLfegiNZWinE5JC0H7QOqthJjbAVRLFY6mzi9qoWvpRyvJRykIfHGuCEEKIzgPXvj17OcdT6979ALuDxp0tKmS2lHCalHNapU6caviQXNA0WLKgzi+P8+fN07tyZsrIyVq5c6XW/UaNG8f777wPwzjvv2PsnTJjAa6+9xoULFwA4evQoP/74I23atOH8+fN1MmZF/WAP0XSJyZeY+M/e2wnOPVbJ0YFNTg6Ul9mqWwF91mPqWkhagV+H1bCph4nb2rp01gIzrM9nAGtcdxBCtBdCNLc+DwcSgL21vG7A8OSTTzJy5EgSEhLo16+f1/2ef/55nn32WQYPHszBgwdp164dADfddBO/+93v0DSN6Ohofv3rX3P+/Hk6duxIQkICgwYNUpO2DRR72uTIQujnPHmr+/JbfOExqK1B4Ja2XaJP1v5gUtE5NaU+Jm6llDV+AB3Ro3MOAJuADtb+YcAr1ufxwC5gp/XvzKqce+jQodKVvXv3uvU1FC5evCgtFouUUsq3335bJiUl+XlEgUtDfp9dSX47WbIQycxREsokWKxZ1SxyYIt9/h5ejRk74ZThtTB0iZw9ESmTk/09tIbN7NnS+k/VH7NnV/sUQJH0oqu1svCllKellOOklL2l7vo5Y+0vklLea31eIKWMllLGWP++WvlZGyfbt28nNjaWwYMHs2TJEp7R16MrGjlpCdZoFTcrH/Zc7kt2uttUVsBjNkP+5jBrS/ffmwbnkLJLqOic2pKSAs2b6z785s19frfUoLJlNmTGjBnDzp07/T0MRT2jRWqM7T5Wz6KZkAVfTUb3pOqTnYuWtCU1w8+DrCaZmWApd6pu1Wc9f/6+EC1qrIrOqS2aBp98UmeBJiq1gkJRxywev1hPtxBZCD22GLYduhBOerqfBlYDzGZYs9Zi7Gx1gnMtgAEDPB6jqCZ1GGiiBF+hqGO0SI3J/SbrjfELAAuO7JKQmVlB57u2krD444CPzc/JAWmxRedIoMKaQkKlUmgIKMFXKOqBtPg0R8ROQqa11yb6Ji6t68XRsyUBvyDr+AWX8Jx+azF1LSSlrUqF3BBQgq9Q1ANapEbfjn31xo2PQrtvDdvPnb2GM7l9KSmrIGvDfj+MsGocKim2PrOusez1EUvXg3ZJpUJuCCjBryZBQUHExsYyaNAgbr31Vs7WQTmyxMREioqKqnXM448/zqZNm6p9rdWrV7N3r2NZRE3Po7g6D416yNEYs8j6xOHaOf/ZtZwvjuSHsyX1PraqYDZD8QfjrS292EnUsXBSv6Be8sAoak+jF3zzYTOLtizCfNg3K9ZCQ0MpLi5m9+7ddOjQgZdeeskn560NFRUV/O1vf2P8+PFX39kFV8Gv6XkUVyd1aCox18TojWGvwDXOS+d1v/iZDYMw7b/WH8O7KpkvHQNLEM7FTmJa5FIuTKQyIKBdUQqdRi345sNmxuWM47FPHmNczjifib4NTdM4elT/kH/zzTfcfPPNDB06lDFjxvDVV1/Z+z2lRc7NzWXSpEn2cz3wwAO88cYbbteYM2cOw4YNY+DAgfbUyQBRUVGkp6czZMgQ/vnPf3L33Xfzr3/9i6KiIntRlujoaHvJQE9pnAsKCli7di3z5s0jNjaWb775xn4egM2bNxMXF0d0dDT33HMPV65csV/7iSeeYMiQIURHR9tfq+LqaN2c/NwT/4Aj5QLYRP+/q/uRnV3/Y7saX5/52tjR50PSvi9k03Uj2Nju2oCff1A0csHPPZRLaUUpFbKC0opScg/l+uzcFRUVbN68maQkPb1/amoq//jHP9i+fTtPP/00999/P4DXtMhV5e9//ztFRUV8+eWXfPrpp3z55Zf2bR07duSLL77gjjvusPcNGzbMXpTl5ptv5s9//jPgOY1zfHw8SUlJZGVlUVxcTK9eveznuXz5MnfffTfvvvsuu3btory8nKVLl9q3h4eH88UXXzBnzhyefvrpar+upoqhIlb3Qpg0B13wjaI/ezYBJ/rnOGJ9po+1d/uPGHFEkD1yKkDAzz8oGrngJ0YlEhIUQpAIIiQohMSoxFqfs6SkxF7i8MSJE9x4441cuHCBgoICpk2bRmxsLLNmzeLYMT0xlre0yFXlvffeY8iQIcTFxbFnzx6D++U3v/mN1+PeffddvvjiCxYvXgxUnsbZE/v376dnz5706dMHgBkzZpCXl2ffPmXKFACGDh2qMntWA0NFLNBdOwm2lVeOZLNSwuzZdVKzp0Zkr97Fkf+73drS/fftfwqnqNsAvuja375foM4/KHQateBrkRqbUzbz5C+fZHPKZrTI2oeN2Xz43333HVJKXnrpJSwWC2FhYXbLuri4mH379lV6nuDgYCwWxwKWy5cvu+3z7bff8vTTT7N582a+/PJLJk6caNjPW/rl3bt3s3DhQt555x17Pv67776b//3f/2XXrl088cQTHq9XHZo3bw7ok9g1KeHYlMkYn0FUG6dFSuMfhfjFGC19XfSnTAkM0X/0bz+BDMbZfz/zTC4Hw41pzLuEhfplfIqq0agFH3TRXzBmgU/E3pmWLVvy4osv8swzz9CyZUt69uzJP//5T0BPSGdLo+AtLXKPHj3Yu3cvV65c4ezZs2zevNntGufOnaNVq1a0a9eOEydO8NFHH111XGfPnuW3v/0tOTk5OKeY9pbG2Vsa5r59+3Lo0CEOHjwIwFtvvcX111/vtp+i+qzecZSzZ6OMXpwbHyX2WndL//hxGD3av6Kf/vpqTu+IN/R17vwh95z8jFWDbrD3hTYLYt6EvvU9PEU1aPSCX5fExcUxePBg3n77bVauXMmrr75KTEwMAwcOZM0aPVO0t7TIkZGR3H777QwaNIjbb7+dOA/VbWJiYoiLi6Nfv3787ne/IyEh4apjWrNmDd999x333XefffIWvKdxvuOOO8jKyiIuLo5vvnEk8mrRogWvv/4606ZNIzo6GpPJxOzZs2v1/1LoZG3YT4vyXwLCUCFQjn+U6bxla9n7LRawTgn5hexXLwNO0TlUsLAsi5+GjODEwCEIoGtYKIumRDfYgi5NBaFn0ww8hg0bJl1j0fft20f//v29HBGYXLp0idDQUIQQvPPOO7z99tv2HwOFZxri+1wdes5fjwR+bPY/lAQVOnQUSMuHo5vfZCV3WfcW9uPS0iCjnhOtmQ+biZ9SDEWzsQ20Y+QHnDo8VZ9kcJrIVwQGQojtUsphnrapbJl1zPbt23nggQeQUhIWFsZrr73m7yEp/EyXsFCOni2hXcVUXfBta68kZCZAwf4ZcAQn0dfJzIRevSA1tf7GOn/TfGh+s7Wl/yrNPP+RXopP5c5pcCiXTh1jS4v85ZdfkpeXx3XXXefvISn8zLwJfQltFkRzS3/alOshjXbRFzB/HKxgBjEt3Nc3zJlTf/787O3Z5G0thYI/W3sEYCHsbDgkJancOQ0QJfgKRT2THNeVRVOi6RoWSsfy3xMirnH23JDXA7KHwNLL92ASxlTEFotu6dcHzxc+D1vnGaJzTFhI5FNV6KSBogRfofADyXFd2Tr/Br5dPJF/TPqbY4PVyp8zCehWyNL2jyKE8djVq+t+UZb5sJl9O9rBV0mG/iQ+REu+Rln3DRQl+AqFn0kdmsrY7mMdHQIsVtdO6pkMJvfa5XZMXbt25m+aDztTcI7OEVSQJp5R1n0DRgm+QhEALB6/2K0vrweYu0HawVRMJnfXzl+fuUDC4o/pOX+9T4unZG/P1ksynjRGSo0hH23yL5R134BRgl9NhBA88sgj9vbTTz/NwoUL/TcgJ5wTn1WVZcuWkZOTU+1r5ebmUlBQUOvzKHS0SM1R8NxpAvf+X4FGIUt7uRa9l3yypYKjZ0uQ4LPiKebDZmavmw2HR8H3ow3bBrBPWfcNnEYv+GYzLFrku9vf5s2bs2rVKk6dOlXrc/k7JUF5eTmzZ88mpQbhda6CX9PzKBxkjM8gKizKMIFb3BnuTIbUA2kkjz1t2L/0x7acL460t32RvGz+5vlIpO7OkQ53jokKUmJ3Keu+gdOoBd9shnHj4LHH9L++EP3g4GBSU1N57rnn3LYdOnSIG264gcGDBzNu3Di+//57t30WLlzIXXfdRUJCAnfddRcnT55k6tSpDB8+nOHDh7N161YATp48yY033sjAgQO599576dGjB6dOneLQoUMMGjTIfj5vdxh/+9vfGD58OIMGDSI1NRXbArvExEQeeughMyjy9QAAEzVJREFUhg0bxgsvvMDChQt5+umn+eGHH+wrc2NjYwkKCuK7777jww8/ZOTIkcTFxTF+/HhOnDjBoUOHWLZsGc899xyxsbFs2bLFfh6A4uJiRo0axeDBg7ntttv46aef7NdOT09nxIgR9OnThy1btriNu6mzYPQCR8Mq/CtjIH0cpB37k9MErv7k3OdRhuNrk7zMfNhM3nfWBHku7pzR5KMtucvDUYqGRKMW/NxcKC2Figr9b26ub877hz/8gZUrV/Lzzz8b+v/4xz8yY8YMvvzyS6ZPn86DDz7o8fi9e/eyadMm3n77bebOncvDDz/M559/zvvvv8+9994LwF//+lduuOEG9uzZw69//WuPPx6V8cADD/D555+ze/duSkpKWLdunX1baWkpRUVFBtdUly5d7Inf7rvvPqZOnUqPHj0YPXo0hYWF7NixgzvuuIPMzEyioqKYPXs2Dz/8MMXFxYwZM8Zw7ZSUFDIyMvjyyy+Jjo7mr3/9q31beXk527Zt4/nnnzf0K3RSh6YyPXq63nAUwyIzASjJYUz4XsP+5WfaGKz82iQvu3+9NX/D4VHwvTGNx4Cw48q6bwQ06pW2iYkQEqKLfUiI3vYFbdu2JSUlhRdffJHQUMcXzGw2s2rVKgDuuusu0rz4O5OSkuzHbdq0yZDy+Ny5c1y4cIH8/Hw++OADAG6++Wbat29frTF+8sknZGZmcunSJc6cOcPAgQO59dZbgcrTKm/dupWXX36Z/Px8AI4cOcJvfvMbjh07RmlpKT179qz0uj///DNnz561J1qbMWOGPT00qLTKVWHFlBXs/nE3O0/oCfhson//r2BJ9kxGi61YpNXJj+TMxkGEdDpPWNT5Gicvu3PVnRSfsNarPZQITucPopyU1Ba1e1GKgKBRW/iaBps3w5NP6n99aaA89NBDvPrqq1y8eLHaxzqnNbZYLBQWFtqt66NHj9qrYnmiKmmVL1++zP3338+//vUvdu3axX333VeltMrHjh1j5syZvPfee/Yx/PGPf+SBBx5g165dLF++XKVVrieWTlyqF0pxSnVV3BleSi4kqcVGHI5+AVIgv+5R4+Rlt634Ayt3rXRc68f+6OGYescj3d5Dy0iuxatRBAqNWvBBF/kFC3x/N9qhQwduv/12Xn31VXtffHy8PQXyypUr3Vwdnrjpppv4xz/+YW8XF+tWVkJCAu+99x4AGzdutPvBr7nmGn788UdOnz7NlStXDK4aGzZRDg8P58KFC1WK3CkrK2PatGlkZGTYi56AbrF37aqLyJtvvmnv95ZWuV27drRv397un1dplWuGvVCKLbGakz//0vC/Iqhw2lvQt0W3Gon9nA8yWX1wieMaRffCLufEbRWETRrt/QSKBkWjF/y65JFHHjFE6/zjH//g9ddfZ/Dgwbz11lu88MILVz3Hiy++SFFREYMHD2bAgAEsW7YMgCeeeIKNGzcyaNAg/vnPfxIREUGbNm1o1qwZjz/+OCNGjODGG280pDq2ERYWxn333cegQYOYMGECw4cPv+o4CgoKKCoq4oknnrBP3P7www8sXLiQadOmMXToUMLDw+3733rrrXzwwQf2SVtn3nzzTebNm8fgwYMpLi7m8ccfv+r1Fe5kjM/Q/fkuor/xl4V07LoGZ/M/P7/6QQnZ27NZ9mW63rDdMHw2F0eHRCBITOlR49egCCxUeuQA5cqVKwQFBREcHIzZbGbOnDl267+x05Te56oQtzyO4mPFDlGW6BOrr23BsRIWkpPBOu1zVbK3ZzNr3SzDDwnfu55TMjb2HJ/uaOe7F6OocypLj6ws/ADl+++/Z/jw4cTExPDggw/y8ssv+3tICj+x5FdLCDIFOSogCiCyEHrkG/Zbu7ZqVn76pnRd7MH4I+KWSsHC4iVK7BsTjTpKpyHTu3dvduzY4e9hKAIALVJjy++3MOPt2zlw6YhD9MP3wXeO+RFbJs3KrPwJb01g4383GjutN/ndvunPEafuMWODVCRmI0NZ+ApFA0CL1Pg67TAjKiL0DgnE5gAVOHz5ktVryslebUy2lr09m54v9KTZk828iv1Nn4zih7POk7OCAQNQNDKUha9QNCA+e/IYEx6LYmPQd9CtECbNgXVLsbtiZBCzFhzgj7uH0TyoOZfLL1NmKfN8Mqeyil/nzcPi5M4xmYQqaNUIqZWFL4SYJoTYI4SwCCE8ThJY97tZCLFfCHFQCDG/NtdUKJo6G548xPRy66T20FfcfPl8lUTpoSGcLz3vWeytcwFCwvIPIXnzKNbgnPdeqIJWjZTaunR2A1OAPG87CCGCgJeAW4ABwG+FEOpmUaGoBSv+Zy/LfxxBx0vovnw7AgjSK1U5I50ewHWnYOtrkPoFzGcR0inaRwiVFLOxUivBl1Luk1JeLT3fCOCglPK/UspS4B1gcm2u6y9Onz5tj1GPiIiga9eu9nZpaWmlxxYVFXnNreNMfHy8r4Zbr1S2OlhRN6Qu+YxTIo208zkgnH356JWqvh9lEPngcujxk27VH3gJtCOQzb3kYVwY17+/su4bK/Xhw+8KHHZqHwFGetpRCJEKpAJ079691hdeveMoWRv288PZErqEhTJvQt8arUa00bFjR3ss/MKFC2ndujV//vOf7dvLy8sJDvb8Lx02bBjDhnn1etlxTjmsUFyVjAwyemXTa9b9zGIpug2nW/mmDYtoPuOXNC+H1O2Qsdl4qJlRzGGpteXIyTx3LopGylUtfCHEJiHEbg8Pn1vpUspsKeUwKeWwTp061epcq3ccZcGqXT4vEOHK3XffzezZsxk5ciRpaWls27YNTdOIi4sjPj6e/fv1G6Dc3FwmTZoE6D8W99xzD4mJiVx77bW8+OKL9vPZLOXc3FwSExP59a9/Tb9+/Zg+fbo9xfG///1v+vXrx9ChQ3nwwQft53Vmz549jBgxgtjYWAYPHsyBAwcASE5OZujQoQwcOJBsp8KorVu3Zt68eQwcOJDx48ezbds2+/jWrl0LwBtvvMHkyZNJTEykd+/eXrNdZmVlMXz4cAYPHswTTzwBwMWLF5k4cSIxMTEMGjSId999t1b/d4WD1cMnUjQ7mqjgbw39lqNj2PzUKH7KdBd7gMyWC50magEkId1O8eYZ31XPUgQWV7XwpZTja3mNo0CkU7ubta9Oydqwn5KyCkOfrUBEbax8Txw5coSCggKCgoI4d+4cW7ZsITg4mE2bNvHoo4/y/vvvux3z1Vdf8cknn3D+/Hn69u3LnDlzaNasmWGfHTt2sGfPHrp06UJCQgJbt25l2LBhzJo1i7y8PHr27Mlvf/tbj2NatmwZc+fOZfr06ZSWllJRof8vXnvtNTp06EBJSQnDhw9n6tSpdOzYkYsXL3LDDTeQlZXFbbfdxl/+8hf+85//sHfvXmbMmEFSkj6pt23bNnbv3k3Lli0ZPnw4EydONNy5bNy4kQMHDrBt2zaklCQlJZGXl8fJkyfp0qUL69evB3BLLa2oHra716NnS/S4mnbXcn7cJdgAjkB9E/ezhB2hCWC782zWDCIiME/6O6szJzidUff9dEjcbzeOAJ9/VxT+pT7i8D8HegshegohQoA7gLV1fVFvhSBqUyDCG9OmTSMoKAjQhWzatGkMGjSIhx9+mD179ng8ZuLEiTRv3pzw8HB+8YtfcOLECbd9RowYQbf/1979xTZ1XwEc/x5nhjgJBQTToKRdlwkiunkkqRSSopWiCZKB1GjQB0SzwcQE2qhgPAQGgk1DjIqXPTAECDoEhWkr6jYUOqomiKLxwJ+hCFhYshEiUALtwowaFoGGqM4e7o1xgp3cECf2tc9HsnRt/3LvOfzCye/+fP27hYUEAgFKSkq4efMmra2tFBUVRZcpTlTwKysr2bFjBzt37uTWrVvR5Zh37drFrFmzqKiooKOjIzryHzNmDNXV1QCEw2Hmzp1LMBgkHA73WcZ4/vz5TJo0iVAoxOLFi6PLKPdqaGigoaGB0tJSysrKaG1t5fr164TDYRobG9m4cSNnz55l/Hj7Buezij17hScz9+NKOsh57kFMS+EypVR9+wHcv+88IhHOvXuNxe89vfplaPq/GTvtcyA5d88y6We4l2V+T0Q6gUrgLyLysfv68yJyEkBVHwNv44w9WoBjqhq/CiZRohtBDOcGEYnELje8detW5s2bR3NzMydOnEi4nHDvMsGQeKlgL20SWbZsGfX19YRCIRYuXMjp06c5c+YMp06d4ty5c1y5coXS0tJofMFgEHFvpxQIBKLHDgQCfY4rT265FPe5qrJp06bocs9tbW2sXLmSGTNm0NTURDgcZsuWLWzbts1zLqaveGevvcZX3nC3nnyA29AAs91Pzfbvh1dfhc8+i/0pp+342e199jUSgyOTWsO9SufPqlqoqmNV9SuqWuW+fkdVF8a0O6mqM1T166r6q+EG7UVdVTGhYE6f10LBnGe+QYRXscsJHzp0KOn7Ly4upr29PTrqTjQX3t7eTlFREWvXrqWmpoarV6/S3d3NxIkTycvLo7W1lfPnzw/5+I2Njdy7d4+HDx9y/Phx5szpe2ekqqoqDh48SE9PDwC3b9+mq6uLO3fukJeXR21tLXV1dTQ1NQ352MYxUCEeV9JB3stPz5hevOjM6qxeneDnZt+Iju57jcTgyKRWxn7TtnfuMZlX6XixYcMGli9fzvbt21m0aFHS9x8KhdizZw/V1dXk5+cnXPr42LFjHDlyhGAwyJQpU9i8eTP5+fns27ePmTNnUlxcTEVFxZCPX15ezpIlS+js7KS2tvapK48WLFhAS0sLle51fQUFBRw9epS2tjbq6uoIBAIEg0H27t0bb/fGg+cnhKLTOfG8uLiZ3Ocmcfl834L9RfyTAl777gO6XmnjYcx3tEZjcGRGny2P7EM9PT0UFBSgqqxZs4bp06ezfv36ET/uoUOHuHTpErt37x7R41g/D6x3Dj92Wqd3yfxpMQOb2bOdkf1A3noLjh5N/iXMJnUGWh45Y0f4mezAgQMcPnyYR48eUVpayupE5+kmI3k9e71wAaqqnDn8eHqLfe8+rcBnPhvhm7Rj/Zxc+/fDO+9AJBK9KpN162DVqlRHZkZCRo3wVfWpK0NM5kjXAYifrVplxd04fLUefm5uLpFIxIpChlJVIpEIubm5qQ7FmIzkqxF+YWEhnZ2d3L17N9WhmBGSm5tLYWFhqsMwJiP5quAHg8HoN0yNMcYMja+mdIwxxjw7K/jGGJMlrOAbY0yWSNvr8EXkLnBrGLuYDPwnSeGkUqbkAZZLusqUXDIlDxheLl9V1bg3FEnbgj9cInIp0ZcP/CRT8gDLJV1lSi6ZkgeMXC42pWOMMVnCCr4xxmSJTC74+wdv4guZkgdYLukqU3LJlDxghHLJ2Dl8Y4wxfWXyCN8YY0wMK/jGGJMlfF3wRaRaRP4pIm0i8rM4748Vkffd9y+IyEujH6U3HnJZISJ3ReSy+/hRKuIcjIgcFJEuEWlO8L6IyC43z6siUjbaMXrlIZfXRaQ7pk9+PtoxeiEiL4jIJyLyDxG5JiLr4rTxRb94zMUv/ZIrIhdF5Iqbyy/jtEluDVNVXz6AHOAGUASMAa4AL/dr8xNgn7u9FHg/1XEPI5cVwO5Ux+ohl9eAMqA5wfsLgY9w7spXAVxIdczDyOV14MNUx+khj6lAmbs9DvhXnN8vX/SLx1z80i8CFLjbQeACUNGvTVJrmJ9H+OVAm6q2q+oj4A9ATb82NcBhd/sD4DuSnndP8ZKLL6jqX4F7AzSpAd5Tx3lggohMHZ3ohsZDLr6gqp+qapO7/V+gBeh/P0Nf9IvHXHzB/bfucZ8G3Uf/q2iSWsP8XPCnAR0xzzt5uuOjbVT1MdANTBqV6IbGSy4AS9zT7Q9E5IXRCS3pvObqF5XuKflHIvKNVAczGHdKoBRnNBnLd/0yQC7gk34RkRwRuQx0AY2qmrBfklHD/Fzws80J4CVV/RbQyJO/+iZ1mnDWLZkF/AY4nuJ4BiQiBcAfgZ+q6v1UxzMcg+Tim35R1S9UtQQoBMpF5JsjeTw/F/zbQOwot9B9LW4bEfkSMB6IjEp0QzNoLqoaUdX/uU/fBV4ZpdiSzUu/+YKq3u89JVfVk0BQRCanOKy4RCSIUyB/p6p/itPEN/0yWC5+6pdeqvo58AlQ3e+tpNYwPxf8vwHTReRrIjIG5wON+n5t6oHl7vabwGl1P/1IM4Pm0m8+9Q2cuUs/qgd+4F4VUgF0q+qnqQ7qWYjIlN75VBEpx/n/lHYDCjfG3wItqvrrBM180S9ecvFRv3xZRCa42yFgPtDar1lSa5ivbnEYS1Ufi8jbwMc4V7kcVNVrIrINuKSq9Ti/GEdEpA3nw7elqYs4MY+5rBWRN4DHOLmsSFnAAxCR3+NcJTFZRDqBX+B8GIWq7gNO4lwR0gY8AH6YmkgH5yGXN4Efi8hj4CGwNE0HFHOA7wN/d+eLATYDL4Lv+sVLLn7pl6nAYRHJwfmjdExVPxzJGmZLKxhjTJbw85SOMcaYIbCCb4wxWcIKvjHGZAkr+MYYkyWs4BtjTJawgm+MMVnCCr4xxmSJ/wORS/vmgdeBeQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "sePQBaGqZzT6"
      },
      "source": [
        "The model with regularization seems to follow the overall trend of the data, while the model without any regularization very precisely fits the training samples. This is espacilly evident in the interval $\\left[0,0.5\\right]$, where the prediction of the unregularized model shows an oscillating behavior. Such oscillations are however not present in the ground truth and therefore undesirable. The regularized model on the other hand is not as flexible as the unregularized model and therefore does not fit the target function well in the interval $\\left[2.25, 3.0\\right]$.\n",
        "\n",
        "## Conclusion\n",
        "In this exercise we revisited the mathematical background for a simple regression task and covered it's practical implementation in Tensorflow 2. We also explored the phenomenon of overfitting and derived different regularizations from a probabilistic perspective. This exercise covers a very simple task with a very basic neural architecture and is intended as a primer for the second part of the regression exercise, which is dealing with a bigger and more realistic problem. In this second part we will consider the problem of estimating the age of a person from a potrait picture."
      ]
    }
  ]
}