21_Recurent_neural_network.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What do we need an RNN?\n",
    "\n",
    "The structure of an Artificial Neural Network is relatively simple and\n",
    "is mainly about matrice multiplication. During the first step, inputs\n",
    "are multiplied by initially random weights, and bias, transformed with\n",
    "an activation function and the output values are used to make a\n",
    "prediction. This step gives an idea of how far the network is from the\n",
    "reality.\n",
    "\n",
    "The metric applied is the loss. The higher the loss function, the dumber\n",
    "the model is. To improve the knowledge of the network, some optimization\n",
    "is required by adjusting the weights of the net. The stochastic gradient\n",
    "descent is the method employed to change the values of the weights in\n",
    "the rights direction. Once the adjustment is made, the network can use\n",
    "another batch of data to test its new knowledge.\n",
    "\n",
    "The error, fortunately, is lower than before, yet not small enough. The\n",
    "optimization step is done iteratively until the error is minimized,\n",
    "i.e., no more information can be extracted.\n",
    "\n",
    "The problem with this type of model is, it does not have any memory. It\n",
    "means the input and output are independent. In other words, the model\n",
    "does not care about what came before. It raises some question when you\n",
    "need to predict time series or sentences because the network needs to\n",
    "have information about the historical data or past words.\n",
    "\n",
    "To overcome this issue, a new type of architecture has been developed:\n",
    "Recurrent Neural network (RNN hereafter)\n",
    "\n",
    "## What is a Recurrent Neural Network (RNN)?\n",
    "\n",
    "A recurrent neural network looks quite similar to a traditional neural\n",
    "network except that a memory-state is added to the neurons. The\n",
    "computation to include a memory is simple.\n",
    "\n",
    "Imagine a simple model with only one neuron feeds by a batch of data. In\n",
    "a traditional neural net, the model produces the output by multiplying\n",
    "the input with the weight and the activation function. With an RNN, this\n",
    "output is sent back to itself $n$ number of time. We call **timestep**\n",
    "the amount of time the output becomes the input of the next matrice\n",
    "multiplication.\n",
    "\n",
    "For instance, in the picture below, you can see the network is composed\n",
    "of one neuron. The network computes the matrices multiplication between\n",
    "the input and the weight and adds non-linearity with the activation\n",
    "function. It becomes the output at t-1. This output is the input of the\n",
    "second matrices multiplication.\n",
    "\n",
    "![](https://github.com/thomaspernet/Tensorflow/blob/master/tensorflow/21_RNN_V3_files/image002.png)\n",
    "\n",
    "Below, we code a simple RNN in tensorflow to understand the step and\n",
    "also the shape of the output.\n",
    "\n",
    "The network is composed of:\n",
    "\n",
    "-   Four inputs\n",
    "\n",
    "-   Six neurons\n",
    "\n",
    "-   2-time steps\n",
    "\n",
    "The network will proceed as depicted by the picture below.\n",
    "\n",
    "![](https://github.com/thomaspernet/Tensorflow/blob/master/tensorflow/21_RNN_V3_files/image004.png)\n",
    "\n",
    "The network is called ‘recurrent’ because it performs the same operation\n",
    "in each *activate square.* The network computed the weights of the\n",
    "inputs and the previous output before to use an activation function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/Thomas/anaconda3/envs/hello-tf/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6\n",
      "  return f(*args, **kwds)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "n_inputs = 4\n",
    "n_neurons = 6\n",
    "n_timesteps = 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data is a sequence of a number from 0 to 9 and divided into three\n",
    "batches of data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "## Data \n",
    "X_batch = np.array([\n",
    "        [[0, 1, 2, 5], [9, 8, 7, 4]], # Batch 1\n",
    "        [[3, 4, 5, 2], [0, 0, 0, 0]], # Batch 2\n",
    "        [[6, 7, 8, 5], [6, 5, 4, 2]], # Batch 3\n",
    "    ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can build the network with a placeholder for the data, the recurrent\n",
    "stage and the output.\n",
    "\n",
    "1)  Define the placeholder for the data\n",
    "\n",
    "`X = tf.placeholder(tf.float32, [None, n_timesteps, n_inputs])`\n",
    "\n",
    "Here:\n",
    "\n",
    "-   `None`: Unknown and will take the size of the batch\n",
    "\n",
    "-   `n_timesteps`: Number of time the network will send the output back\n",
    "    to the neuron\n",
    "\n",
    "-   `n_inputs`: Number of input per batch\n",
    "\n",
    "1)  Define the recurrent network\n",
    "\n",
    "As mentioned in the picture above, the network is composed of 6 neurons.\n",
    "The network will compute two dot product:\n",
    "\n",
    "-   Input data with the first set of weights (i.e., 6: equal to the\n",
    "    number of neurons)\n",
    "\n",
    "-   Previous output with a second set of weights (i.e., 6: corresponding\n",
    "    to the number of output)\n",
    "\n",
    "Note that, during the first feedforward, the values of the previous\n",
    "output are equal to zeroes because we don't have any value available.\n",
    "\n",
    "The object to build an RNN is `tf.contrib.rnn.BasicRNNCell` with the\n",
    "argument `num_units` to define the number of input\n",
    "\n",
    "`basic_cell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)`\n",
    "\n",
    "Now that the network is defined, you can compute the\n",
    "`outputs ``and`` states`\n",
    "\n",
    "`outputs, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32)`\n",
    "\n",
    "This object uses an internal loop to multiply the matrices the\n",
    "appropriate number of times.\n",
    "\n",
    "Note that the recurent neuron is a function of all the inputs of the\n",
    "previous time steps. This is how the network build its own memory. The\n",
    "information from the previous time can propagate in future time. This is\n",
    "the magic of Recurrent neural network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[-0.9999238   0.99430686 -0.9638557  -0.9994302  -0.997174   -0.86753696]\n",
      " [-0.02186426  0.42749134  0.12737474  0.16807137  0.5695009   0.5968522 ]\n",
      " [-0.9978742   0.9293986  -0.89790034 -0.98720515 -0.90166324 -0.291719  ]]\n"
     ]
    }
   ],
   "source": [
    "## Define the shape of the tensor\n",
    "X = tf.placeholder(tf.float32, [None, n_timesteps, n_inputs])\n",
    "## Define the network\n",
    "basic_cell = tf.contrib.rnn.BasicRNNCell(num_units=n_neurons)\n",
    "outputs, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32)\n",
    "\n",
    "init = tf.global_variables_initializer()\n",
    "\n",
    "with tf.Session() as sess:\n",
    "    init.run()\n",
    "    outputs_val = outputs.eval(feed_dict={X: X_batch})\n",
    "    print(states.eval(feed_dict={X: X_batch}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For explanatory purposes, you print the values of the previous state. The output printed above shows the output from the last state. Now print all the output, you can notice the states are the previous output of each batch. That is, the previous output contains the information about the entire sequence.e"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[-0.9192994   0.9710009  -0.35473573  0.4106098  -0.8282092\n",
      "    0.9195943 ]\n",
      "  [-0.9999238   0.99430686 -0.9638557  -0.9994302  -0.997174\n",
      "   -0.86753696]]\n",
      "\n",
      " [[-0.9969728   0.95827335 -0.24309447 -0.9053337  -0.17600021\n",
      "    0.3927361 ]\n",
      "  [-0.02186426  0.42749134  0.12737474  0.16807137  0.5695009\n",
      "    0.5968522 ]]\n",
      "\n",
      " [[-0.9999831   0.9979963  -0.82032    -0.9925979  -0.93228835\n",
      "    0.46669725]\n",
      "  [-0.9978742   0.9293986  -0.89790034 -0.98720515 -0.90166324\n",
      "   -0.291719  ]]]\n",
      "(3, 2, 6)\n"
     ]
    }
   ],
   "source": [
    "print(outputs_val)    \n",
    "print(outputs_val.shape)    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://github.com/thomaspernet/Tensorflow/blob/master/tensorflow/21_RNN_V3_files/image006.png)\n",
    "\n",
    "The output has the shape of (3, 2, 6):\n",
    "\n",
    "-   3: Number of batches\n",
    "\n",
    "-   2: Number of the timestep\n",
    "\n",
    "-   6: Number of neurons\n",
    "\n",
    "The optimization of a recurrent neural network is identical to a\n",
    "traditional neural network. You will see in more detail how to code\n",
    "optimization in the next part of this tutorial.\n",
    "\n",
    "## Applications of RNN\n",
    "\n",
    "RNN has multiple uses, especially when it comes to predicting the\n",
    "future. In the financial industry, RNN can be helpful in predicting\n",
    "stock prices or the sign of the stock market direction (i.e., positive\n",
    "or negative).\n",
    "\n",
    "RNN is useful for an autonomous car as it can avoid a car accident by\n",
    "anticipating the trajectory of the vehicle.\n",
    "\n",
    "RNN is widely used in text analysis, image captioning, sentiment\n",
    "analysis and machine translation. For example, one can use a movie\n",
    "review to understand the feeling the spectator perceived after watching\n",
    "the movie. Automating this task is very useful when the movie company\n",
    "does not have enough time to review, label, consolidate and analyze the\n",
    "reviews. The machine can do the job with a higher level of accuracy.\n",
    "\n",
    "## Limitations of RNN\n",
    "\n",
    "In theory, RNN is supposed to carry the information up to time $t$.\n",
    "However, it is quite challenging to propagate all this information when\n",
    "the time step is too long. When a network has too many deep layers, it\n",
    "becomes untrainable. This problem is called: **vanishing gradient\n",
    "problem**. If you remember, the neural network updates the weight using\n",
    "the gradient descent algorithm. The gradients grow smaller when the\n",
    "network progress down to lower layers.\n",
    "\n",
    "In conclusion, the gradients stay constant meaning there is no space for\n",
    "improvement. The model learns from a change in the gradient; this change\n",
    "affects the network's output. However, if the difference in the gradient\n",
    "is too small (i.e., the weights change a little), the network can't\n",
    "learn anything and so the output. Therefore, a network facing a\n",
    "vanishing gradient problem cannot converge toward a good solution.\n",
    "\n",
    "## Improvement LSMT\n",
    "\n",
    "To overcome the potential issue of vanishing gradient faced by RNN,\n",
    "three researchers, Hochreiter, Schmidhuber and Bengio improved the RNN\n",
    "with an architecture called Long Short-Term Memory (LSTM). In brief,\n",
    "LSMT provides to the network relevant past information to more recent\n",
    "time. The machine uses a better architecture to select and carry\n",
    "information back to later time.\n",
    "\n",
    "LSTM architecture is available in TensorFlow, tf.contrib.rnn.LSTMCell.\n",
    "LSTM is out of the scope of the tutorial. You can refer to the official\n",
    "[documentation](https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/LSTMCell)\n",
    "for further information\n",
    "\n",
    "## RNN in time series\n",
    "\n",
    "In this tutorial, you will use an RNN with time series data. Time series\n",
    "are dependent to previous time which means past values includes relevant\n",
    "information that the network can learn from. The idea behind time series\n",
    "prediction is to estimate the future value of a series, let's say, stock\n",
    "price, temperature, GDP and so on.\n",
    "\n",
    "The data preparation for RNN and time series can be a little bit tricky.\n",
    "First of all, the objective is to predict the next value of the series,\n",
    "meaning, you will use the past information to estimate the value at\n",
    "$t + 1$. The label is equal to the input sequence and shifted one period\n",
    "ahead. Secondly, the number of input is set to 1, i.e., one observation\n",
    "per time. Lastly, the time step is equal to the sequence of the\n",
    "numerical value. For instance, if you set the time step to 10, the input\n",
    "sequence will return ten consecutive times.\n",
    "\n",
    "Look at the graph below, we have represented the time series data on the\n",
    "left and a fictive input sequence on the right. You create a function to\n",
    "return a dataset with random value for each day from January 2001 to\n",
    "December 2016"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2016-08-31    29.244005\n",
       "2016-09-30    18.769957\n",
       "2016-10-31    16.676111\n",
       "2016-11-30    27.457955\n",
       "2016-12-31    24.407237\n",
       "Freq: M, dtype: float64"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# To plot pretty figures\n",
    "%matplotlib inline\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "\n",
    "def create_ts(start = '2001', n = 201, freq = 'M'):\n",
    "    rng = pd.date_range(start=start, periods=n, freq=freq)\n",
    "    ts = pd.Series(np.random.uniform(-18, 18, size=len(rng)), rng).cumsum()\n",
    "    return ts\n",
    "\n",
    "ts= create_ts(start = '2001', n = 192, freq = 'M')\n",
    "ts.tail(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAApoAAAE3CAYAAAAKZuFSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzsnXd4m9X5v+9HlrxlO87eiwwgw4QkBAIlQAwpNTu0FFoKZRQolKbs0S/8SqFQAimbsguFtpSWWVIgQFhpCIHsEJI4O3ES7y1Zss/vj/O+smzLW8Pj3Nely9K7zvFr6+ijZ4pSCoPBYDAYDAaDIdw4Yj0Bg8FgMBgMBkPPxAhNg8FgMBgMBkNEMELTYDAYDAaDwRARjNA0GAwGg8FgMEQEIzQNBoPBYDAYDBHBCE2DwWAwGAwGQ0QwQrObIyJKRObHeh7hQkTuFJH1URjnZyLyUaTH6a6ISD/rf2uO9XqyiOwVkZQYT81g6NGIyAsi8k47z1kqIo9Gak5B40RlfTb0LMTU0YwNIjIN+ApYrpSa3YbjXwD6KaVyGm0fBBQrpbwRmWiUEZFUIEEpVRjBMeKBbcAFSqlPIjVOtLFE4cdAf6VUQSev1Q/IB05QSi21tv0LWK2UuquTUzUYuizhWps7MX46+rO5pB3nZAI+pVR5OObQwjhhXZ9F5CLgUaVUajiuZ+iaGItm7LgUeByYJCKHdvQiSqn9PUFkiohDROKUUhWRFJkW84Hq7iIyLWHcFXgeuFJEnLGeiMEQQcKyNjdGRFxtOU4pVdoekWmdUxRpkWmNE4312dDDMEIzBohIEnA+8BTwGnBJK8ffCfwM+IHlzgx2aQZc5yIyynp9noh8IiLVIrJKRKaIyCQRWSYilSLyuYiMbjTGaSLytYh4RGS7iNzdksARkXQReUlEDlrnbBORXzfa/5S1v9yaz/Sg/ReJSIWInGq5YmqAQ0O5ZkTkYhHZaI2zWUQWiIgjaP8vrO0eESkQkfdaEUPnAw1cUyLiFJFFIlJsPRaJyBMisjToGBGRG0Uk17q360TkJ0H77ft/joh8ICJV1ryzG411mIj8x7ovB0Xkb5Zl2t7/goi8IyI3icgeYI+1/Sci8lXQef8UkaH22GhrJkC+NY8X2jJv65gZQX//VcBRIe7b+0AmMKeFe2swdFvCtTYHrQU/FpGPRKQa+IWI9LXe73us9+IGEbm40TUbuM5Fu8UfF5F7rPXtoIgsbLQGNnCdi8gOEbldRP4sImXWeDc0Gme8tS57ROQ7ay2uEG1lbPb3DV6fg9aqa0WH1hSLyPMikhx0zPdEZLl17VIRWSH682gO+strStC9u9M6p9m1zto/xzr+JBH50lprV4q2RgfPd5Z1/yutsT8SkSHWvlbXRUOYUEqZR5QfwE+BNdbzOcBBwNXC8anAP4APgEHWI97ap4D51vNR1uvvgFOBiWjxscH6eQJwOLASeDvo+qcAZcDFwFjruO+AhS3M6RFgNTATGGn9Huda+wT4HPiPtf8Q4C5rjMHWMRcBfuB/wGxgPOAG7gTWB41zGZCHtkKOBk4D9gNXW/unW9e5wJrHVGAB4Gxh7iVot3nwtpuBYuAcYALwEFAKLA065m7rvsyz5nI+UAn8oNH932TNcxzwF6AQSLWOGQwUAPcBhwJTgLeBLwGHdcwLQDnwMjAJmGxt/7n1dx1j3dePgU+tfXHA2db4h1n/I+ltnHcq+n/wn9Z4pwDfWtea0+g+LQfuivV7yDzMIxIPwrQ2B60FO4LWrmHAUOAGIMt6H1+O/pJ9UtA1XwDeCXq91FqLfodeJ39orXk/bnTMo0Gvd1jrztXo9fcaaz5HW/sd6M+FD625HG2tQT7gohZ+3ztpuD6/YM3taWs9Oxm9vt5i7Xei19WF6M+Widb6c6h1n6611iL73tnrZLNrXdDfRgEr0J9XE4H3rHXLDgmcClSjvzRkWWP+Ahhh7W9xXTSPML6vYj2B3viwFoXrrediL0atnNNg8QnaHkpo/iJof4617eygbRcBFUGvPwV+2+i6ZwIV9ps2xLhvAc81s+9E69ykRttXAzcGzUEBRzY6pvFCtgv4aaNjfg1stJ6fbS107jbe+wxr3BMabc8Dbg56LdYitNR6nWItWsc1Ou9PwLst3P+h1rZjrde/Az5sdI0+1jEzg/7W+ehYqJZ+l4nWecOs13Os1/2CjmnLvC9HfzikBu3/CaGF5r+Bl2L9HjIP84jEgzCtzUFrwXVtGPPvwDPNXc+a0/8anfNBo3OW0lRo/q3ROVuA263np6DF6tCg/cdYc76ohbneSVOhuRuIC9r2NLDEep5pXfP4Zq53EUGfRS2M29xad0rQMbMbHfNy4/sWdGyr66J5hO9hYq2ijIgcAhyL/vaEUkqJyMtoF81rYRpmbdDzA9bPdY22pYhIslKqCjgSmCkiNwUd4wCS0N8y80KM8QTwmogciV703lb1MY9HAsloF27wOYnob7U2frT4DImI9AeGA38WkSeCdjnRHwJYY+8EtovIe2j37r9V8/FKSdZPT9A46dbvucLeZv1dVljjg7YSJgL/FREVdD0XelEPJvj+77N+DrB+Hgl8T0QqQsxtbNAc1qtGsbeWW+gO9LfzTOrvwQgs93oI2jLvQ4G1SqngOf2vmetVU38PDYYeQ4TW5pWNxohDe09+hP4SmoC27C1t5TprG73eR/2a0pFzJgL7lFJ7g/Z/BdS1cs1QbFRK1TYa5yjQsaNWCM97IvIh2oL6mlJqV0sXbMda19xauwc4Ani9mSHas54bOokRmtHnUrSbc1eQCBMAERmulNodhjF8Qc9VC9scQT//H9p12pj8UAMopRaLyEjg+8BJwH9E5J9KqYut6x0AjgtxalnQc2+jBaox9vyuAJY1M49ya1H6HpAN3ALcIyIzlFL7QpxSiP79+7QwbktzOQ1tZQ3G19xr68Mq+HwHOqTg+hBjHAh6Xhm8Q3RZofeAJWj33kGgH/AZ+oMqHPNuC5mYhdjQM4nE2lzZ6PX1wHVol/E6tOfnHloXjY3fq4rWcyw6ck5HaHEcpdTFIvIntIv6dOBuETlTKfVeqIu1c61r6XOtJcK9LhpawAjNKCI6QeVnaDHUuE7aS+gYyd81c3oNehGMBN8AE5VSW9tzktIldF4CXhKRxcDfROQK63oDgTql1LaOTkopdUBE9gFjlVIvtnCcH/gI+EhE7kAvTDno2JzGx9aIyEb0N9p3rW2lIrIfmGFdB9GfNDPQ8aAAGwEvMFIp1Zn6m9+gY6x2KqXas6BNRC+2tyqltltzPLvRMTXWz+D/k7bM+1vgIhFJUUrZH4yzmjl2Etp9bjD0GKK4Nh+L9v68ZI0r6LjLdmWZh4FNwBARGRL0hXw6EUoQVkqtAdYA91mfFT9Di8lQ964ta11bWIUO4wpFuNZzQxswQjO6/AD9BnpaNSoRISJ/B64QkbuUUirEuTuA74vIBLRVrrSdQqUlfge8IyI7gVfRLu1J6JjBG0OdICK/Q4umDej/o7OBbUopr4gsAb4A3hSRG9GL2iD0N9olSqnP2jG3O4BHRKQELQxdwDR0bNEfRCQH7XL+FChCB4a70eKpOd5DL/gLg7Y9BNwoIpvRi9Av0Ik7eRCwnC4EFlofDp+iEwFmoQV1E1HbDI+hE5z+ISL3oS3GY9Di87oWXP670Avj1SLyGNrd3bie5U70t/ofiMjb6BJObZn3K+jA+Oesv+sQ4LbGExCd2T4UHZ5gMPQkwro2tzDOZuBHInIsOinwGnQiyqrO/wrt4gN0DPpfROR6dDjMg+i1P9Tv2CFEVzf5BTqmfy96rZuCDr0Cfe8SRVfmWAVU0ba1ri3cDywXkafQ664H7WV7Xym1K0zruaENmPJG0eUS4OPGC5nFP9EB5Nkh9oEOsP4WHfOTjw58DguWC+MHaJG2wnrcTFOXQjBetDhZgxaVbrQbAmsxPhVtHXwavaC9is7mDuXObmluz6AzEH9qjfUZOnllu3VICTpxaQla0F4PXNqKmH0amCe6yLHNQrTl4nl0ZjXo+B5P0DG/RQfDX48W2B+gs9S300Ys68FsdCzUf63rPIa+n83WQ1VK5aOtAGeihfAdwG8aHbPX2n432g1vlztpcd5WbGYOOkv+G/S9CI7XtfkxepHe2dbf12DoJkRrbf49en1djBY3leiklaiilKoDzkLHiK5AV8e4Gy0yPS2c2l6q0Bbbf6JF9l/Qv+991jyWAU8Cf0Pfuxvbsta1BaXUamAu2kK6HJ1Vfx71rvFOr+eGtmE6Axl6JZaVYoNqocuN6HqSnyulronezLomIpKAzlr9sVLqi1jPx2AwhBcRmYpOzpyulPo61vMx9ByM69zQW7kR/Y0eACux6RTgE7R7/jK0i+eymMyu6zESuNuITIOhZyAiZ6EtqlvQFtsH0V6jb2I4LUMPxFg0DQZ0VinafTMZHVKyEV1b1MQjGgyGHoeIXAjcji7hVowusbRAKXWgpfMMhvYSUaFpfXi/iM5AVsBTSqmHrDZTl1FfOudWpdS7EZuIwWAwGAwGgyHqRFpoDka3HPxGRNzA1+gA3x+iuwEsbPECBoPBYDAYDIZuS0RjNJVSeTQsD/MtujyKwWAwGAwGg6GHE7UYTasG36fo+oy/Qfc4LUOXhLhOKVXc0vn9+vVTo0aNiugcDQZD7+Prr78uUEr1j/U8IoFZNw0GQyTo168f77333ntKqXmtHRsVoSkiqehs3ruVUv8WkYHoYrUKXYh1sFLq5yHOuxxdM5ERI0YcuXOnKd9nMBjCi4h8rZSaHut5RILp06erlStXtn6gwWAwtJO2rp0RL9guIi7gX8DLSql/g24tqJSqtYrGPg3MDHWuUuoppdR0pdT0/v17pMHBYDAYDAaDoccSUaFptXZ6FvhWKfVg0PbBQYedBayP5DwMBoPBYDAYDNEn0gXbZ6NbB64TkdXWtluBH4tIFtp1vgPdC9VgMBgMBoPB0IOIdNb554CE2GVqZhoMBkMM8Pl87NmzB48nnC2tez6JiYkMGzYMl8sV66kYDN0K04LSYDAYehF79uzB7XYzatQodHSToTWUUhQWFrJnzx5Gjx4d6+kYDN2KiCcDGQwGg6H9iMh1IqJEpJ/1eo6IlIrIauvxfx25rsfjoW/fvkZktgMRoW/fvsYKbDB0AGPRNBgMhi6G1b73ZGBXo12fKaVywnD9Zvf5/eA0nwxNMMLcYOgYxqJpMBgiwtc7izjmDx/y4bcHYj2V7sgi4EZ0wmTU2LULNmyASJdXPuaYYzp03htvvMHGjRvDPBuDwRBJjNA0GAxhp65OcedbG9lX6uGql7/h27yyWE+p2yAiZwB7lVJrQuw+WkTWiMhiETk83GMnJ4PPB9XV4b5yQ5YtW9ah84zQNBi6H0ZoGgyGsPPhpoOs21vKjfMm4PXX8cXWglhPqUshIktEZH2IxxnoEnCh4i+/AUYqpaYCjwBvNHPty0VkpYiszM/Pb9e80tP1z5KSdp3WblJTUwFYunQpc+bMYf78+UycOJELLrgAu1vdzTffzGGHHcaUKVO4/vrrWbZsGW+99RY33HADWVlZ5Obm8vTTTzNjxgymTp3KOeecQ1VVFQAXXXQRv/rVrzjmmGMYM2YMr732WmDs++67j8mTJzN16lRuvvlmAHJzc5k3bx5HHnkkxx13HJs2bYrsDTAYehEmEsdgMISdHQWVAPxk1kgWfbCZwsqaGM+oa6GUmhtqu4hMBkYDa6yYwGHANyIyUym1P+j8d0XkcRHpp5RqoOKVUk8BT4FuQdnSPH79a1i9uuE2S6uRnNy+38kmKwv+9Ke2H79q1So2bNjAkCFDmD17Nl988QWHHnoor7/+Ops2bUJEKCkpISMjg9NPP52cnBzmz58PQEZGBpdddhkAt99+O88++yzXXHMNAHl5eXz++eds2rSJ008/nfnz57N48WLefPNNvvzyS5KTkykqKgLg8ssv58knn2TcuHF8+eWXXHXVVXz00UcduwEGg6EBRmgaDIawU+bx4RBIjXeSmRJPYYU31lPqFiil1gED7NcisgOYrpQqEJFBwAGllBKRmWiPVGG45+B0gter4zSjkf8yc+ZMhg0bBkBWVhY7duxg1qxZJCYmcskll5CTk0NOTuj8p/Xr13P77bdTUlJCRUUFp5xySmDfmWeeicPh4LDDDuPAAR0nvGTJEi6++GKSLRWdmZlJRUUFy5Yt49xzzw2c6/Wa/1eDIVwYoWkwGMJOWbUPd6ILh0PITEmgyFg0w8F84EoR8QPVwHlKdS5tJ5TlsaoKNm6EUaOgX7/OXL1tJCQkBJ7HxcXh9/txOp2sWLGCDz/8kNdee41HH300pIXxoosu4o033mDq1Km88MILLF26NOR1W7pNdXV1ZGRksLqxaddgMIQFE6NpMBjCTpnHT1qS/h7bLzXeuM47iFJqlO0aV0o9qpQ6XCk1VSk1SynVsYyaVkhKgvj4yMdptkRFRQWlpaWceuqpLFq0iDVrdF6U2+2mvLw8cFx5eTmDBw/G5/Px8ssvt3rd7Oxsnn/++UAsZ1FREWlpaYwePZp//vOfgBal9ngGg6HzGKFpMBjCTlm1j7RE3apPu86N0OwuiOikoLIyqKuLzRzKy8vJyclhypQpHHvssTz44IMAnHfeedx///0cccQR5Obmctddd3HUUUcxe/ZsJk6c2Op1582bx+mnn8706dPJyspi4cKFALz88ss8++yzTJ06lcMPP5w333wzor+fwdCbkE56XqLG9OnT1cqVK2M9DYPB0AbmP7EMV5yDv10+i9+9vZFXV+5m/f87pfUTY4CIfK2Umh7reUSCUOvmt99+y6GHHtrieaWlsGULjBtXn4luaNu9Mxh6C21dO3ukRfPLbYX86m+rKKkyVhSDIRaUeXykJ2mLZt/UeCq8fjy+2hjPytBW3G5wOLTgNBjCRW4uLLjKy8C0auIcdQxMq2bBVV5yc2M9M0Mk6ZFCc3+Zh7fW7KPAuOsMhqhRVFlDVY0fgLLq+hjNzJT4wH5D98Dh0GKzpCTyXYIMvYPFi2HWlEqSnnmYZeWT8Kp4lpVPIumZh5k1pZLFi2M9Q0Ok6JFC07aklFabDzaDIRoopZh21wdc9PxXgLZo2jGafS2haeI0uxcZGVBTAx5PrGdi6O7k5sKF8yt5q2ou9/huZCzbcFLLWLZxj+9G3qqay4XzK41ls4fSw4WmL8YzMRh6B1sPVgCwYnsRvto6qmpqSQtynQMUVprahN2JaHUJMvR8Hn3Ay2W+xzma5SH3H81yLvU9wWOLzBrRE+mRQjMjWX+wGaFpMESHT7fo5jRDM5Io92j3eVqidp33TdH1DI1Fs3sRH6+7A5k4TUNneeWvdVzie7LFYy71PcErL5k47p5IjxSaAYtmlRGaBkM0+HyL7qmdkhBHmfUFz7ZoZqaaGM3uSnEx3PVbk7xh6BwFFQmMZGeLx4xgFwUViVGakSGa9EihaVtSSoxF02CICit3FgPai1DmsYSmFaPpTnASH+egwLjOuxWLF8OZJ1cy/I3wJ2+UlJTw+OOPh2+yzbB06VKWLYtIXXtDO+iX6mUnI1s8Zhcj6JdqAoJ7Ij1SaDrjHKQmOI3r3GCIAhVef8BdXlrto6zacp1bFk0Rob87gQOl5kOkuxCcvHGvP/zJG+0Vmkop6jpQPd4Iza7B+T9x8KzrihaPecZ1Jef/NC5KMzJEk5gJTRGZJyLfichWEbk53NdPT3IZoWkwRIH9loAcNyAVj6+OggptubRDWACG9Ulib0l1TOZnaD+RTt64+eabyc3NJSsriwULFnDSSScxbdo0Jk+eHOjKs2PHDiZMmMCFF17IpEmT2L17N88++yzjx49n5syZXHbZZVx99dUA5Ofnc8455zBjxgxmzJjBF198wY4dO3jyySdZtGgRWVlZfPbZZx27GYZOc/V1CTztuor/MSvk/v8xi2dcV/LLBQkh9xu6N85YDCoiccBjQDawB/hKRN5SSm0M1xjpSa5ArJjBYIgcttCcMMjNloMV7CnWfaTtOpoAQ/sksTy3MCbzM7SfV/5ax7I2JG/MfukqHny0/de/9957Wb9+PatXr8bv91NVVUVaWhoFBQXMmjWL008/HYAtW7bwl7/8hVmzZrFv3z7uuusuvvnmG9xuNyeeeCJTp04F4Nprr2XBggUce+yx7Nq1i1NOOYVvv/2WK664gtTUVK6//vr2T9IQNsaOhe/NS2Huv5dwtfMJLvc/wQh2sYsRPOO6kmdcV/LiaymMHRvrmRoiQUyEJjAT2KqU2gYgIn8HzgDCKjRLTDKQwRBx9pdZQnOgm3fIY1eRJTQTgyyaGUnsL/Pgq63DFdfQkfL2mn3MmdAfd9DxhtgSzeQNpRS33norn376KQ6Hg71793LgwAEARo4cyaxZ2gq2YsUKjj/+eDIzMwE499xz2bx5MwBLlixh48b6j4+ysjIqKio6PTdDeNi0Cd56C846NwXfgGuY/dJV5JclkuTwcPnlcSxfkGBEZg8mVq7zocDuoNd7rG1hw7jODYbocMASmuMHuQHYXVRNnENIjq+PtxraJ4k6VW/9tNlf6uGav63izdX7ojdhQ6tEM3nj5ZdfJj8/n6+//prVq1czcOBAPFaV+JSUlDZdo66ujuXLl7N69WpWr17N3r17SU1N7fTcDJ1HKViwQJfKevRRePDRBPaXJvPHhQ4q65L5zc1GZPZ0unQykIhcLiIrRWRlfn5+u87NSDZC02CIBnml1WQkuxjg1vFV2wsq6ZPsQkQCxwzNSAZgT3HDOE37PVpsSh91KSKdvOF2uykvLwegtLSUAQMG4HK5+Pjjj9m5M7QldcaMGXzyyScUFxfj9/v517/+Fdh38skn88gjjwRer169usk4htjwn//Af/8Ld94JAwbUb8/O1j8/+CAm0zJEkVgJzb3A8KDXw6xtDVBKPaWUmq6Umt6/f/92DZCe5DLljQyGTrC9oJILn1tBhdff4nH7S70MSksMZJnvL/MwIjO5wTFD+yQBBBKCthdU8rPnVgTc7ua92rWIdPJG3759mT17NpMmTWL16tWsXLmSyZMn8+KLLzJx4sSQ5wwdOpRbb72VmTNnMnv2bEaNGkW61b7o4YcfZuXKlUyZMoXDDjuMJ5/U8aWnnXYar7/+ukkGihFeL/z613DooWDlbQWYPBkGDjRCszcQqxjNr4BxIjIaLTDPA84P5wBpSS5q/HV4fLUkukzJBIMhFPtLPfzg4c94+bKjmDgorcG+L7cV8unmfDYfKGfaiD7NX6OsmkHpiQ2yzEf1bejyHJyuY/n2WhbNd9fl8cnmfI4YkQFg4qm7GGPHwouvpXD6/CVc6nuCS33hT9545ZVXWj1m/fr1DV6ff/75XH755fj9fs466yzOPPNMAPr168c//vGPJuePHz+etWvXdnyShk6xaJEulfX+++BqFIItoq2a770HdXXg6NL+VUNniMmfVinlB64G3gO+BV5VSm0I5xgZyabfucHQGrn5FRRW1rBxX1mTfUVV2p3dWuvI/aVeBjcWmv0aCs1EVxz93QnsLdGJQqt26QbatvAsrTau867G978Py9em4L38GmanrSNJvMxKWUf1pdewfG0K3/9+9Od05513kpWVxaRJkxg9enRAaBq6Hnv3wu9/D2eeWe8mb0x2NuTng/ku0LOJlUUTpdS7wLuRun6gDWW1j4Fppq2VwRAK+4tYqPaQdtykXRczFDX+OgorvQxMS8QV5yA5Po6qmtomQhNgRGYyOwurUEqxerclNC1XurFodk3GjtXJGw8+ChUVsGlTMmPGgJX4HXUWLlwYm4EN7eamm8DvhwceaP6YuXP1zw8+gKys6MzLEH16rLHaFprmA8xgaB77/VEQwmpZVKn3FbYgNA+UeVCq3jVuv+9G9U1ucuwh/VPZcrCCvSXVAfFqJwe1FqPpr62jrk619usYIkhKCjidUFIS65kYujpffAEvvwzXXw9jxjR/3JAhcPjhJk6zp9PjhaZxnRsMzVNiuayLQvQhL6myLZrNu7VtoTisjxaWAaEZwqI5bmAqRZU1fPjtwcC2vNK2WTTPf/pL7ntvU4vHGNqOUu0X7SKQng5lZbpkTW+jI/esN1JbC7/6FQwdCrfc0vrx2dnw2WdQbRqH9Vh6rNDMSIoHjNA0GFqitKp513lRVeuuc7sL0DArqzwtyUXflPgGxdptxg/UdTZf+XIXiS4HSa44fLX6w7u0uqbFD/Lc/ApyD1a25Vfq9ojInSKyV0RWW49Tg/bdYrXt/U5ETunI9RMTEyksLOyQcEpP1+7Q3lYLXSlFYWEhiYkmDKs1nnsOvvkGFi7UVvDWyM4Gjwc+/zzyczPEhpjFaEYaY9E0GFqnJde5HaPZUjLQnuJqRGBwuhaaR47sw/A+Td3mUC80vztQzkkTB7Bpf3kgRtNXq6iqqSUlIfSSVO71U+bpVe/lRUqpBgGJInIYukLH4cAQYImIjFdK1bbnwsOGDWPPnj20tzYx6OzgggKoqYE+zRci6JEkJiYybNiwWE+jS1NcDLfeCscdBz/6UdvOOf54nZH+wQfNJw0Zujc9Vmi6E52IQGlV9LJZ//X1HlbuLOYPZ0+O2pgGQ2eod52HitFs3aK5u7iKQWmJxDu1c+SmeaFrIAIMTEvAneik3ONnzsQB5JV6AkJTz8UXUmh6/bXU+OsoM18azwD+rpTyAttFZCu6ne//2nMRl8vF6NGjOzyJa6+FvDzYENY6IYaewJ13QlERPPywDrVoCykpcMwxJk6zJ9NjXecOh+BOcEbFolnh9eP11/LJ5nzeWWNa6Rm6DyXNuM59tXWUeXSh9sIWuvbsKa4OuM1bQ0QCVs054/sHSpDVzyX0OBXWPMo9LReO72FcLSJrReQ5EbFthxFv3dsWcnJg40bYti3aIxu6Mhs2wGOPweWXtz+DPDsbVq+GgwdbP9bQ/eixQhMgIzk+4kLT669l0h3v8au/raLM46Pc66fGXxfRMQ2GcGG/Pyq8fjy+eg+sLUD7psRTXFWDvzb0//Te4upmXeWhOH58f44Z25fhmcmB8BZXnDZ9lDaTEGQLzJ7kOheRJSKyPsTjDOAJYCyQBeQBLRQ9LEAYAAAgAElEQVSICXntDrfubQunnaZ/vvNO2C9t6KYopS3daWlw113tP//kk/XPDz8M77wMXYMeLTTTkyLf7/xvX+4C4L0NBwKuveYsMwZDV6OkykecQwu9YKtmsfU/fMiAVJSqTwwKxldbR15p2y2aAL86aRyvXKbbGtoWzSEZ+vzmShzZQrPC6+8xJY6UUnOVUpNCPN5USh1QStUqpeqAp9HucYhC6962MHYsTJxohKahntdf1yLxrrugX7/2nz9tmo75Ne7znkmPF5rh7qFcWuXj083aSlBXp3j041wApg7PCHwghvpQNhi6IiXVNQy3hGKw0LSf267uwooaNuwrZV9QTOX+Ug91qr60UXuxe6PbQrWkyscXWws498llnP/08kBWdLlXv4eV0klBPR0RGRz08izA7sP4FnCeiCRY7XvHASuiPT/Q7vOlS6G8PBajG7oS1dXwm9/o3uW/+EXHrhEXByedpIWmqSLV8+jxQjPcFs0/fbiZi55fQbnHR0GlN5AoUe7xBVx7oRIrDIauhsdXi8dXx5j+qUDDpB8743zcwPp9P3vuKxa+/13gmB2FutxQeyyawdglyIZlaKFaUl3Dk5/k8tWOYpblFlJVo135wbGZ5c24z/21ddT2EGsn8EcRWScia4ETgAUAVpveV4GNwH+BX7Y34zxcnHYa+HzGAmWA+++HnTt1ApCzE+nF2dmwZw98913rxxq6Fz1baCa7wpqpqpTi/Q0HqFOwr8TDvhIPAP3dCZRV+wMfiMWVPSeWzNBzsd8bY6zi6g0smpZV/vAh6QB8vrWAggovB8o8gWO+2l6EQ2DysPQOjW+7zvumxpPoclBUUcO6vaWB/YH40SChWVYd2qJ59L0fcf7Tyzs0j66GUuqnSqnJSqkpSqnTlVJ5QfvuVkqNVUpNUEotjtUcjzkGMjLg7bdjNQNDV2DXLrj3Xjj3XJgzp3PXsksbvf9+p6dl6GL0bKGZ5KKkyhe2jg7Bdf/2lVQH3IgTB7kpra4JWGCM69zQHbDDSsYPcpPgdARCQgC+3FZEgtPB5KHpDExLCMQiB9fUXL6tiMlD03GHKM7eFuxkoNREJ+MHulm8fj8lVT5mH9IXqE/+CbZiLsst4K/Ldza5Vn65ly+3F1FV0/Nd610BpxO+/334z390bU1D7+SGG/TP++/v/LVGj9bxv8ZK3vPo8ULTX6cCArCzLNl4IPB8b5DQPHRwWqDDCdS7HQ2GroydWT4kPYmfHzuaN1bvY/3eUr7ZVcxba/Zx2XFjiHc6mD4qM1DqqKDCy+J1eVz18tes3l3CrDF9Ozx+hiU03QlOjh/fP/Al7thDdAKLnYVeERSX+fCHW7jrnY0NrhP8RfK9Dfs7PB9D+8jJgfx8+OqrWM/EEAuWLoVXX4Wbb4aRI8NzzexsfV2fcQr2KHq00MwIc3egNXtKGNs/BadD2FdSzd6SalITnAzNaBijZmI0Dd0BuzpCRrKLK+eMxZ3g5K/Ld/Lish30sbYBzByVGTinqLKGl5bv5N11+6mprWPW2I4LzXTLdZ5iCU3QpY5mjtZlI+33bXCMZpnHj9df16AUU3XQ839/0yQJ2xAh5s3TSRzGfd778Pt1P/ORI+utmuEgO1u3N13eM6JgDBY9WmiGuw3l1oMVTBjkZlB6YsB1PiQjkbSkhhHQxcZ1bugG2K7z9CQXaYkuxg9ys6Owkp1FVRw2JC3QpWeGJTTjnQ7qFIE4zTkT+nPU6MzQF28DEwel8eu54zhx4gCyhmfgTnQycVAa/VITAMgr9XDx8ytYt7c0UGvTpjxE3Oa0ERmcPnVIh+djaB+ZmTB7tilz1Bt56ilYtw4eeACSOpYLGJITTwSHw7jPexq9QmiWNFMIuj14/bXsKqrikP6pDMlIYl+pTgYakpFEWqMYteYsmp9uzue219d1ei4GQziwk2zciVpQDu+TxO6iavYWVzew0k8Y5OZH04dz4SztH9tWUMmZWUN44eKZJMd3PM00ziH8eu54MpLjccY5uOO0w7nmxEMC79sV24v4+Lt8luUWMjAtscG5wXGb9vOLZ4/m3OnDMUSPnBxYswZ27279WEPPoLAQbr9di8Kzzw7vtTMyYOZMIzR7Gj1aaKaF0aK5o6CKOgVjB6QyNCMpyKKZ1CAZYmBaQrMWzQ82HuDlL3dRHaaY0ca8t2E/03+/hMpeUGvQ0Hlsq2CqZbkckZlMXmk1B8u9DM2or40Z5xDumz+Fkw4dCOg6d4MzwmjGsJh/5DBOPnxQ4P209WBFYF+f5HiS4+MCr8sauNP1+9t+vxuih+kS1Pv47W+hrAweeqjt/czbQ3Y2rFgBJSXhv7YhNvRooWmXTymt7rwrOzdff+iN7Z/KkIxE9hRXU1hZw9CMpIBFCGBk35RmyxvZ2eh5pdUh93eWb/PKKKjwsiXoAzocHCzzsL2gMqzXNMSeCq+PJFcczji9DAzLTMYuRRmqNmZ/d3zg+ZAICE2bOIfgTnSyraD+/9id6GzgOQguW2a7zoPfh4boMGGCzhQ2QrN3sHo1/PnP8MtfwqRJkRkjO1tXMvj448hc3xB9erTQ7JOsPxiLw+A6t60rYy3Xuc2wPkkNLCmj+iY3a9G0s9HzSj0h93cWO0Rgy4Hwtuu4+91vuerlb8J6TUPsqfD6SQ0SZyMy662YQ0MIzb4pCYHnQ9ITm+wPJ2mJrgaVHFITnA1ioYP7ngcsmh0ss2ToOCLaff7hh1Bpvov2aJTSCUCZmXDnnZEbZ9YsSE019TR7Ej1aaCbHxxHvdIQlOSc3v4KhGUkkxceRNTyD5Pg4zj5iKPMmDWpgSRndL5WqmtqQ/c7t2M3gNn7hxP49tx6sYFt+Bf7a8BS4O1jmJb+8dXFc5vGx8L3vqPGbwnrdgTKPv8H/7vBgoRnCYpme5Ar0RY+kRdMeKxh3ok5Yst3nwYXbbTd646Q8Q3Q47TTweuGjj2I9E0MkefVV+OwzuPtu3Zc8Urhcuvi7idPsOURMaIrI/SKySUTWisjrIpJhbR8lItUistp6PBnBOdAn2RWWupabD1RwyADdju/wIels/N08HvxRFgnOOFLjnYjoeoBTrS4pa/aUNrlGcVVkLZq25Xbx+v2c9OAn/ObVNdSFoS1fmcfXpsL37284wKMfb2XNHhNc0x2o8PhxJ9SLs0FpibjihDiHMDiExdLhEDJTtJdgSHq0haaT06YO4fLvjQEaWTSrjUUzlhx3HLjdpsxRT6ayEq6/Ho44Ai65JPLjZWdDbi5s3x75sQyRJ5IWzQ+ASUqpKcBm4JagfblKqSzrcUUE50Cf5PhOu869/lq2HCjn8CFpIfc7HEJqghN3opPJw9IRgdW7GootpVTELZq2FXVXURVKwVtr9vH3rzqfDlrm8eGvU1S2ksRku+yb60dt6Fo0dp3HOYShGUkMSksMxG02pl9qAinxcRG3HtpC026PmZrg5GfHjOLak8bhipMGMZrlHj/xcQ4SXXEhr2WILPHxcMopOk4zTE3YDF2Me+/VfcgfeUTXTo00djvKrmjVzM2FBVd5GZhWTZyjjoFp1Sy4yktubqxn1nWJmNBUSr2vlLL9W8uBYZEaqyUykl0h3djtYfP+Cvx1iklDm+/pnJboIi3JhTvRxbgBqazaXdxgf4XXH4g529eMRfO7/eU8sbTj/63BZZyOG9eP9CQXm/aXdfh6NnaHltbu43cBoWmy3rsD2qLZ0Ao4dXgGk1v4P+/vTmBonyQkEummQdhCdvqoPhwyIDXwJU9ESEt0UebxUe7xccEzy1m9u9i4zWNMTg7k5cGqVbGeiSHcbNumW0xecIGumxoNJk6EoUO7ntBcvBhmTakk6ZmHWVY+Ca+KZ1n5JJKeeZhZUypZvDjWM+yaRGt1/jnwj6DXo0VkFVAG3K6U+ixSA2emxLP5QOeysNfv027wSUOa/wB2JzoD8W5HDO/D+xv3o5QKfCAHZ6LnNWPR/PeqPfz5k22cM20oA9Lan2xRXFXDwLQEDpR5yZkymO0FlQ3a93WEujpFuXWNkiofw1qIzdli3ecyIzS7BeUeXwOLJsAD505t8Zyb5k2IWHmuYGyL5tCMZP44v+Gc0pJclFX7Wb+3jC+2FgL1lk9DbDj1VJ0Y9PbbMG1arGdjCCfXXad72993X/TGFIGTT4Y33oDa2uhYUVsjNxcunF/JW1VzOZr61kVj2cY9vhs5zfdvTp+/hOVrUxg7NoYT7YJ0yqIpIktEZH2IxxlBx9wG+IGXrU15wAil1BHAb4BXRCSkT1pELheRlSKyMj8/v0NzzEiO73SM5vq9pbgTnQzPbD4u7bSpQ/j+pMEAZI3IoLjKx87CqsB+u7SRrlUY2qJpz9MWtu3BX1tHucfPWUcM46Z5EzkjayipCc5AUe6OUlHjD7jDWqpHWu7xBXpVl4WpE5MhspR7/YEamjbOOEezbnPQ8cnTR3W8G1BbsYXmwLSEJvvSEp2UeXzsKa5/f5nSRrGlf3+dLWzKHPUsPvhAi73bbtMWxmiSnQ3FxfBNFyl48ugDXi7zPd5AZAZzNMu51PcEjy3yRnlmXZ9OCU2l1Fyl1KQQjzcBROQiIAe4QFmZJEopr1Kq0Hr+NZALjG/m+k8ppaYrpab379+/Q3Psk+yipLr1RJaWWL+vjMOHpLXoLvzlCYfw82NHA5A1PAOggfvcFpGHD0mjwutvkMxgU2RZPdfvbb+7224nODg9kSvnjCXRFaeFZictmqVB7viWOiwF1+40rvOuj1KKCq+ftC4q0GyhOSCU0ExyUVbtY09xdYNthtiSkwMrV2oXuqH74/PBtdfqOqkLFkR//JNO0j+7ivv8lb/WcYmv5dzlS31P8MpLkff4dDcimXU+D7gROF0pVRW0vb+IxFnPxwDjgG2Rmkef5Hhq61Sn3Lm5ByuYOCh0IlAoxg90kxwf1yAhyE4EGjfQrV9XNLWy2lnp6/a236Jpi0C7SD1AamLnhWawIC5pofD9VsttLmKSgboDlTW1KEUT13lXoY+V3T4orakXQcdo+hsKTZNxHnPsLkHvvhvbeRjCw2OPwbffwqJFkBjZsrkhGTAAsrK6jtAsqEhgJDtbPGYEuyioiMHN6uJEMuv8UcANfNCojNH3gLUishp4DbhCKVUUqUlkWEXbO5oQVFunLT/BAq414hzClGHprN5dLzRtETm2v44lKwnhXratnhs6JDT1uXaReiAsFs3geoUtWTTtcIAh6UkmRrMbUBFoP9k1BdrcQwfy0HlZHDrY3WRfWpKTsmofe0uqGmwzxJZJk2DECFPmqCdw8CDccQfMm6ct1bEiOxu++AIqwtvsrkP0S/Wyk5EtHrOLEfRLjUz5wu5MJLPOD1FKDW9cxkgp9S+l1OHWtmlKqYguS5kp+oO0oyWOKhr1g24rWcP7sDGvDI9Pm9GLKmtwOiTQ2i9kQfcqfcy+Uk/AAtpWikNZNMMQoxkcl9lSjGZ+hYeMZBd9U+ONRbMbUOHVf6OuGtuY6IrjjKyhIcNV7KzzPcXVgXqfbmPRjDl2l6APPgCP+aztVjQu2TNuWDXeci+/+U1k+pm3lexs7cL/9NPYzcHm/J84eNbVcjXGZ1xXcv5Pu0DmUhejR3cGgnqLZkcTgsq9HSsGfcSIDHy1ig37dLxlcVUNfVLiA/NpLNr8tXWUVvsYZWXPHmxDJ55giiNl0bREowjsKKjkuc+3hywCX1BeQ//UBNyJTpMM1A2wrc5d1XXeEmlJLjy+OvYUVzNnwgCAdnkcDJEjJweqqmDp0ljPxNBWQpXs+cY3iV/Jw5x/ZmxL9hx7LCQkdA33+dXXJfC06yr+x6yQ+//HLJ5xXckvFzSNK+/t9HihWd/vvGNC0xZq7f1AtrsI7S7S7r2iyhr6JLsCSQ6NhWZptQ+lYKTVBrC9CTW2hbRxjGZVTS21negOZIvGIelJvL/xAL97ZyMb85omKxVUeOmXmoA7wWWSgboBtqXb3U5LfVcgOIFp6rB0nrlwOj+aPjyGMzLYnHACJCcb93l3Ibhkzz2+GxnLNpzUMpZt/LHuRt6qmsuF8ytjVow8KUl3nuoKQnPsWPjDQynMZQk3yP3kMgYfTnIZww1yP6cnL+HF10xpo1D0AqHZOdd5eQdd57agtC2CJVU+MpLjA9sbxzvaQnhEXy002+vyLqny4bQ6FNnYz/cUVwVKD7WXsmofIg17X285WN7kuIIKL/3cCaQlOY3Q7AbYX6C6o8v5+PEDAs+H9kli7mED6ZtqrAhdgcRE7e40XYK6B92hZM/JJ8OGDbBvX8ymEGDJEqhLTKH8Z9cwO20dSQ4vMxLX8ZC6hidfTOH734/1DLsmPV5opiW6cEjHk4EClp92WjRtV7ttESz36FIyrjgHqQnOJkLTLm00wrJohip/1BLFlpANjmmzheaNr63lkhe+atf1bMqsftjBltItIQrgF1TU0C81HrcVP2fo2thxtN3RdT6ibzJ/PGcK6UmudlWD6A6IyJ0istdKoFwtIqda20eJSHXQ9pbrrMSQnBzYtQvWr4/1TAyt0R1K9tjtKJcsidkUAF3P8x//0D3fn3w+gf2lyfhrHezKT8admcBf/hLb+XVlerzQdDiEjOT4difX2Niiqb1CM97pIMkVF4iFK/P4AuIzPcnVpFSQPb+RfTvuOm8cp2aLiPV7S9lRWNnuWqJ/WrKZF5btwJ3oanD/gmtmAlTX1FLh9dMvNYG0RBdVNbX4a+vaNZYhcvhq6/jjfzc1iPvtqKW+q/DDGcNZ/X/Z9Hf3SEvmoqAkyuBiQbmNkyu7Ij/4gf5p3Oddn+5QsmfKFN0QINbu81tugcxMLTSDSU3V9UbffhvWro3N3Lo6PV5ogt3vvINZ551wMaYlOQMFz8uqfYGi0hnJriYJMwHXeaZOBrKFwJKNB9hyoKmrujHFVTWBMAEbW0RU1tTi8dW1+x78ackWQItg2/U+fmAqWxsJzYIK7Vaxk4GATichGcLHNzuLeXxpLks2HgxsC8Qed1OhCUS837qhYwweDNOnmy5B3YHuULLH4YC5c7XQjFU4xkcfwfvvw623QnqITtTXXANuN9xzT/Tn1h3oFUKzT3J8h5OBOmP5scuw1Fm1OG0Rlp7UVPjaFsOhGUnEOSTg2rzhtTU8+vHWVseyY0CDaWyF3Vfa9jjNqhr9e4/ITOYPZ0/mwR9mcfYRQznl8EHsLKwMlG0CyLeEZj93fGDM4PqbhthiNwAoqqyPsyr3+EmJjyPOYcRaF+RqEVkrIs+JSJ+g7aNFZJWIfCIix8Vsdm0gJweWL4cOdg42RInuUrInOxsOHIB166I/tlLamjlsGPzyl6GP6dMHrroKXn0VNm+O7vy6A71EaLo6lAy0r6SaCo8fh0ByfPvfaGlJWmhW1vipU/VxmxlWW0ybP/53E/e/9x0JTgdJ8XG4E3VCja+2juIqHzuCeqY3R0mVr4lFM6WRON7fTI/1UBSUa+F7zYmHcOYRQzl6bF8e/FEW4wa6qVOwvaAy6FjbopkYsNqaOM2ug11iqyCoG1WFx98t4zN7AiKyRETWh3icATwBjAWygDzgAeu0PGCEUuoI4DfAKyISMkBVRC4XkZUisjI/RkrvtNP0B3QsS+MYWqe7lOyx4zRj4T5/4w1YsQL+3/9ruUPSggW6FNN990Vvbt2FXiI049udDLRhXynH3PsRn28tIDXB2SE3XVqik7Jqf8AqancvSU+KD1g0fbV1PP/FDgC8fh3XqIWmL1D7c1dhJa2hXecNLZqNrbD72iE08yv0sY1j4Eb31a79nUHi1xYwwRZNk3nedai3aAYJTa+/W2ac9wSUUnOVUpNCPN5USh1QStUqpeqAp4GZ1jlepVSh9fxrIBcY38z1n1JKTVdKTe/fv3+0fq0GHHEEDBli4jS7OmPHwmk/SuEklnCTs2HJnltcXadkz7BhMHFi9IWm36/d5RMnwoUXtnzswIFw2WXw4os6Gc5QT+8QmintTwbaXaTdzGv3lHT4Azk9yUVptS8ooajeollaXYNSivV7S6n21XLU6EwuPXa0Ps6qRWkLuOIqX4sWwuqaWrz+OtIbWTTdjdoL7m+H6zzfsmg2Fpp2wlHwfOwYzb4pCfXZ9sai2SWo9PrJzdcxtYVBrvMyj69bx2f2VERkcNDLs4D11vb+IhJnPR8DjAO2RX+GbUNEJwW99x7UdCxqyRAFysvhzTdhxvdS8P2ivmTP7LR1eC+/huVru07Jnuxs3SEoml2nXnwRNm2Cu+8GZxuWSztR6P77Izuv7kavEJoZyS68/jqqa9peosG2gNapjrfps13ndrxiwHWe5MJXq6j21bJiu27z/sj5R3B7zmEAAdd5sDDY1YL7PFRXIICUhHp3f0ayi7yS9lg06xN8Gv9OQINkpvxyL+lJLuKdjiZlnQyxY8O+Us5/5kuUgiRXHIUVjS2aRmh2Qf4oIutEZC1wArDA2v49YK2IrAZeA65QShXFapJtISdHC5nPPov1TAzN8cgjUFQECxfCg4/Wl+zZX5rMg48mxNySGUx2NlRXw7Jl0RnP49H93mfOhLPOats5I0Zoy+czz+iYUoOmVwjN4O5A2wsqeXtN65Vfg2MoO2r5SUvU2eW26Kp3ndcXbf9qRxGj+6UwwF0f/OFOdFHu9Tewwu5sk9BsaMF0xjlIdDlwJzg5pH8qee1xnZd7EYHMlEYJRglORBoKybxST6DndEZK6M5Hhujz6le72ZRXxi++N4Z5kwY1idE0QrProZT6qVJqslJqilLqdKVUnrX9X0qpw63SRtOUUl3eKT13ro5pM+7zrklpqRaYp50GM2bEejatM2eOtipGy33++OOwZw/ce2/7+r3ffLO24i9aFLm5dTd6ndD8y7IdXPfqmlZrSgZnhXfcoumkTkFemce6Tr3r3J7Pyp3FzBjVp+F5VoxmsDDY0UKcpl1CqXHWOUBqgovBGYkMSk8kr12ucy99U+JxxjX8F3E4BHeCM1AfFHTS1BCrc5A7wYnTIR2uW2oIH1/tKGb6qD7ccuqhDMlIpLiqJtCnvtzjN65zQ0RJToYTT9RC03QJ6no89BAUF8Odd8Z6Jm3D7Yajj46O0Cwt1aWKTj5Zt1VtD+PGwQ9/qIVqcXFk5tfd6CVCs96CmF/upaa2LpB405jqmlrySqspDSqontrBGE3bjbynuMp6rT/YbUG4YV8ZJVU+Jg/LaHBewHVe4cXpEPqmxLfiOtdCs7Hr3L7WoPQkBqcnklfqaXPR9vxy3bs85O9lxZ7a5JVWMyRDWzRFhD4pHS8nZQgPZR4fm/aXMWNUJqDjZ2vrVODvVuH1k5pgkoEMkSUnB7Ztg+++i/VMDMGUlMCDD8KZZ8K0abGeTdvJztYdegoLIzvOwoV6jI7WxbzlFh028sgj4Z1Xd6V3CE3L/VtUWRNIXGmuoPifP80l5+HPKa7svEXTdpHvKa62rqNfjxuQCsAbq/YCMGGgu8F57kQXFV4/BRVeMlPiGdk3uUWLpi3qGncGArhp3kSuPuEQ+qUm4PXXUdnGONWCCm+zXVfSk+oLzlfV+Cmu8jE4vb4XemYnOjEZwsOqXSXUKeqFZqp+D+wursLjqzUxmoaokJOjf5ri7V2LRYu01a67WDNtsrO1dfzDDyM3xoEDWoT/8Idw5JEdu8aUKTok4aGHoKJpx+ZeR68QmhkBi2a90Gyu/M6e4moKK2saCDt3R2M0g4RmostBvFPf7r6pCYzITGZZrv5aNn5gaoPz3IlOausUe4qr6ZuawJj+qQ3qVjampAWhOW/SIGaOzgyI3rbGTuaXe5skAgV+r6B+5vusBKOhGfVCs0+Kq4FQN0SftbtLAMgarq3ltnX69Ee/4KZ/6T5pRmgaIs3w4TB1qonT7EoUFWmhec45+m/TnZg+XXfmef/9yI3x+9+D1wt33dW569x2m77XT7bcSr5X0CuEZn2MZn3cY0UzQtMWYlsOVjDAsuh1OEbTsmDuLa4KPLexBcAAd0KIjj762B0FlfRLjWds/1QOlnubLRlUXOUjOT6OBGfzReUDQrMNheuVUuS3YNFMS3IGMun3Wa0phwQJzcyUeIqacZ17vbB/P2zcqGuUGSJDYWUNaYnOQNF+26IJ8PVOHThkYjQN0SAnB774Qn/oGmLPAw9oK1t3s2aCTgY68cTItaPctg3+/Ge49FIYH7JKbds56ig46SR9v6NZkqkr0iuEpitOZ14fLPcEhGS5N7TgsoVYbZ0ia3gGiS5HAxHVHuws84KKmiZi1RaaEwa5m5xnH7uv1ENmSjyHWK723IOhbfC6K1DT+Mxg2mPRLPP4qfHXNRujmR4Uo2knGNkxmqBjUIubcZ2PHKl7IR9+OBw8GPIQQxgorfY1qKvaN6X+b9k4lMNgiCQ5OVBbq2tqGmJLQQE8/LB2C0+aFOvZdIzsbF0QfcuW8F/7//4P4uL0z3Bw223asPL88+G5XnelVwhN0GV3thyoF2q26/yl5Ts5WFb/daMkKAloZN9kPr3hBM7IGtqhMYOtmLYb3SZrhBaa4wc2FZrBrQH7piQwtr/uxrO1WaFZE9Jt3mAu7RCaB6z7MSg9dL+tYNf53hIPIjAwrf7YTKu3vJ3hHEx6Rv02Y+GIHCVVNYEvF9C09BVgWlAaosLMmdC/v3GfdwUWLoTKyvAJqVgQqXaUa9bAK6/AtdfqrlbhYM4cnSl/333g68XRZL1GaI7MTGHNnpLA6wqPn4NlHn77xnpeXbk7sD24rFFGcjwD0hKJc7S//SRocWe7JxuLrsOHpHHcuH6cfNjApucFCdQhGYmMyEzGFSfk5oeO0yyt9jVxzTcmPUSh9eaw6202KzSTXFTV1OKrrWNfSTUD3Ym4gsog9UmJp06F7g6Uml6fjNRYaJaWwj/+AftaL3PaKlsPVjT4AtHbKK32kZFUb+V2xjn404+y+NnRIwPbjOvcEA0cDt0laPFiEy4TSw4e1FnQP/4xHEVebN0AACAASURBVHZYrGfTccaOhdGjwy80b7tNx3/edFP4rimir7tzpxaxvZVeIzTHDUzF46svaVTh9Qe63+ywSgcppRoUak9P6pxrMc4hPPuz6aQnuTjmkH4N9iU443jpkqM4akzfJudNGprGpceOZuG5U/np0SNxxjkY1TelWYtmWzKIbTdqmyyattBMCy007fvyf29u4PVVe5u4/zOtou2hMs+TU+v/Bo1rjF34M8V558Gso+s6/YF0xV+/5r7/9t6aKqXVvib/v2ceMbTB/1uasWgaokROji6pE62uLoam3H+/jhXsztZM0OItOxs+/jh8X1w++wz+8x9dbL1Pn9aPbw+nnqqTrv7wBx1C0huJmNAUkTtFZK+IrLYepwbtu0VEtorIdyJySqTmEEzjEkLlHl+gJd9OK8Pc46ujJqi+Zmvu6LZw1Ji+rLnjZG6aN7HN5yQ447g95zDmHzkskOBzyIDUQM/qxpR7/K3G27kTnMQ5pE1Cc79lCRyQ1nwyEMDfVuzihAn9eeCHDVMXgwvkNyYpNbRFs64O3npTW45373KweXOr02yRA2UeDpb3botm43ANaBjiYFznhmhx8sngckXHfZ6bCwuu8jIwrZo4Rx0D06pZcJWX3NzIj91V2b8fHnsMLrgAJkyI9Ww6T3Y2lJXBihWdv5ZSuu7l4MFwzTWdv15jRODWW3Ut2X//O/zX7w5E2qK5yGqZlqWUehdARA4DzgMOB+YBj4tI8+nSYWJ8kNVNhAYtHm2Lph2f6bRc5cGux1gzbqCbnYWVgX7t/to6ch75jA82HqDc42vVoikipCU6WbOnhOPv/zhQRD4U+8t0ElJzWezBbvoLZo1skjSUGahb2lTUxqfUfwUNFpqN3eWdydLz19ZR7vH32jaYSunC7KG+KA0OCocwrnNDtHC7dbxapOtpLl4Ms6ZUkvTMwywrn4RXxbOsfBJJzzzMrCmVLF4c2fG7Kvfdp9si/va3sZ5JeDjxRP05Hg73+Tvv6KoId9yhu1lFgnPO0QL/nnt6Z5esWLjOzwD+rpTyKqW2A1uBmZEe1C6SnuB0kJkcT4XHH6ipmV/updJbL0zsLO9wWDTDxWGD3dQp2HygHNCZ7Ov3lrFqV3Gbi2+nJ7n4fGsBOwurWLWrpNnjDpR6Gli+Ql0H9Bt92oimfoaARTOE69yVHFpoNrY2lDQ/vVax22OWtKGUU0+k2leLr1aFDP3o705ARP/tUuKN0DREj5wc2LQJtm6NzPVzc+HC+ZW8VTWXe3w3MpZtOKllLNu4x3cjb1XN5cL5lb3Osrlvn67leOGFuj1iTyAzU9fU7Gw9zdpabW0cNw5+/vPwzC0UcXHaLb96Nb3yy06khebVIrJWRJ4TEVuRDAV2Bx2zx9rWBBG5XERWisjK/Pz8Tk3EnehiaEYS/VIT6ls8BgmhnYVVAWEya0xf4p2OBtafWDNxUBoA3+aVAQRE8v5SD3WqbbU+05NcgW9Tu4patmi29LvbLtkJA90hxUzAohnCde5Iqt8WHKMZTqFpu+x7q0XT/j/OCPG3ccU56JeaQGq8E0cHk9wMho5gdwn6z38ic/1HH/Byme9xjmZ5yP1Hs5xLfU/w2CJvZCbQRbn3Xh3LePvtsZ5JeMnOhi+/1EmkHeWVV2D9el2k3RVhu9IFF8CIEXD33b3PqtkpoSkiS0RkfYjHGcATwFggC8gDHmjv9ZVSTymlpiulpvfv378zUwXgyJF9OGRAaqDFY2FF/YKzs7Ay8AE9/8hhrLj1JPo2U0cyFozITCYlPo531uZx5mNfsH6vfnfZNRHb0rc6OGavpd7pB8patmjarvMjR4aOmk6OjyM+zhG6lmZ8vfgLt0VTKcX7G/YHkpnKPD5qQ5RY6unYAru5ZLbB6YkmPtMQdcaM0dnOkYrTfOWvdVzia7kNy6W+J3jlpd6TkbFnjy5AftFF+v73JLKztUVy6dKOne/16sSoadNg/vywTi0kLhfceKNOiPvkk8iP15Xo1KeNUmpuW44TkacBOzpnLzA8aPcwa1vE+eP8KQBc/PxXVHj8CLpW5s7CKrYXVtLXssRlJLuadOuJNQ6HMGGQm8+3FgD1Fkw71rKtFk2b5iyaXn8tBRU1zWacg3a//mDKYM6dPjzkfhHR3YNCdF9S8fXivqhIAdqq1lhoFhTob33SDqPbqyt3c9O/1nHcOJ3hr5RO+upqf8tI05rQHNk3BWPLNMSCnBzdR7qsDNLSwnddnw8KKhIYyc4WjxvBLgoquo6nKtL84Q96HbzttljPJPwcfbSOqfzgAzjjjPaf/+c/w44d+qcjSkGEP/+5bm159906Zrm3EMms88FBL88C1lvP3wLOE5EEERkNjAPCkDvWOomuOBJdcaQmOinz+CisrGFEZjIZyS7ySjz1LscuKkwOHVy/MtvF5+0M8bZYqNoiNA+WaSE4KL15a26cQ3js/GmB7kahcCe6KA9RR9PvqheaBYX127dsbWh5vOkmmHZkHdXVzQ7RAKUUz32+A6i38kLvdJ/b/8fpzcQY33HaYTx2wbRoTslgALTQ9Ps7H1vn8Wir0F13actWRgbEKy87GdniebsYQb/U3lGNYtcuePppuOQSGDUq1rMJPwkJcPzxHUsIKi/X7vITTqgvAB8NkpLguutgyZLwZMx3FyKp4/8oIutEZC1wArAAQCm1AXgV2Aj8F/ilUiqqvgx3glO7ziu99EtNIM0SRSXVPpwOISU+4knwHeKYsf0CCUq2wLQ9w22piWifO3V4Bnml1Q1KOdnYcavN9TlvK3YcbGNq4uoXeW3R1IQK0F+9ysHjj7dtvG92FfOdlShll6uC3pkQVNaKRbNfagLD+kQovdJgaIEBA8Cd4OXn57ev9FBFhRant98O3/ueLqw9Z47OFM7P172ps0928Izzihav84zrSs7/addc38PN3XfXl9bpqZx8MmzerEV1e1i0SP/f3Htv+7xm4eCKK3Stznvuie64sSRiQlMp9VOl1GSl1BSl1OlKqbygfXcrpcYqpSYopaKeg+VOtIRmRQ2ZKfEBN29JlS4JI9H+z2sjP5gymG9uz2ZYn6a919sSo5lp9bs++bCB1CnYW9LUXNiaSGkrWmg2FXnVjvoxi4v1fS4pgdKS0Pe8rd/6tlldk1Li4wgOyyzphRbN1lznBkMsWLwYjp1WyRU1D7PK13LpoeJiHct5ww1w1FHaYnnKKVoYeL26TeDbb0Nhoc7kfeghWPR4As/EX8X/mBVy/P8xi2dcV/LLBV0n9j5SbN8Ozz0Hl10Gw0NHOPUIOtKOMj9ft+I8+2zdHjXauN36//fNN2HduuiPHwt6TWegYFITnZRU+aiqqaVvajzuBG3RLAvRTaWr4XBIyIzwtsRozj9yGM9dNJ2ZozOB0O5zu21kay0tW0Pf06YWzSrqhWZZqVBb2/K30bg2Gh9scdW4S9HidXncu3hT2y7SQyipriHOIaZO5v9v787j46rLxY9/niSTfWuTtIWWljYUEApUKFBEuCwFqVfBhUUUaO9VKiDLRRThp1cBRUUUvBUFEVSqIiCLcMWClsumUPbaFqFCSoGWliZNm2adTJLn98f3TDJJJ8kkc2Z/3q/XeXVyzuQ83+9kmjzzXU3aiFx66Acafemhz32ynUWLYO5cqKmBk0+GpUtdF+mVV7oWzR073EzjH/zAdcNH7uJSXw/L7i3j5NIVXBm4ngZmEaKABmZxef71nFy6gmX3llFfn7rXIVmuvdb97rzyylSXJLH228/tSz6WRPN733P7vX/nO4kr12guugjKy11ZckFuJpoRrX+1ZUWuRbOzhx2d3Wk7PjNStBnhsU4GOm7fyexe7VpEN0dt0XTJYbRdZcZiuK7zncEQhRFrae7YMXKiGesYzZ2dIURg9iSXaIaHP9z1wrvc8mRD/0L3uSC8/WS6tsyb3BPL0kP/EbyZe38XpK4OrrnGjcFsaYGnnhoYi1lePnKchQth5eoygksu4sjKNZTkBTkobw13113EytVlLFyYgMqlmYYG+PWv4YtfhKlRFw7MHiKwYIEb89i360iwXbz9ttshafFi+MAHEl68YU2cCOefD3ffnbh1ZdNJbiaaEUnZxLJCKosD7OwKua7zNG/RhIHdXUoCLpka6+Lb4VbbaBNlfGvR9Ma9PrhqE7/821sAdIV6Cfb0UVoxkPQtXz5yovnOu7EtT9TSGaKiqKA/iZ5RUzbo+oaIcZvZ7O1t7Ty3vjmtNhswJpalhy7gZirKevnrXwfGYhaPY4J4fT3ccFMRW1pK6enN45vfK+WdLUUxJSLZ4NvfdkvpXHFFqkuSHCec4IZQvPLK6M+96ir39/KqqxJdqtF9+cvu5/T976e6JImXk4nmIdMncNC0KhbOmcIhMyZ4SZEbozncTN10MqXKJVOz6lwyNdbFt8sK8ykYZt/znZ0hAvlCcSC+t0ZFcQHt3b38duXb3LjiX7z09nY+9pO/AVAVMVn97LPhd78b/j7vvBN7ollVGuhPwidVFvUn4gBvNeVGonnhna+wpaWLr56YBRsa5zARuUhEXheRV0XkBxHnrxSRN0VknYh8JJVlHItULj101llu+Zo77vD91mnnjTfgN7+BCy5we3fnggXeIoujdZ+/+iosWwYXXpge41anTHGT2JYtg3ffHf35mSwnE839dq/kwQs/zM1nHcIEbzJQm7f3ebqP0YSBFs36OtePNNbFt0WEqpLAsC2alcXxd7uGu/IbGttp7erhmj/9kze3uiWZSooG3/uZZ4a/T1NjHt1R1n0fKtxdPNl7bapLAoNa9dY3to2xBplHVXljaytnHLoHCw/Ikb8yWUhEjsVt1XuQqu4P/NA7vx/wGWB/4CTgZyKSEVOoa8tTt/TQ7ru72cnLlsXWvZrJrrnGjWm9/PJUlyR5pkyBAw4YPdH8xjfc0It0Grd6+eVundMf/jDVJUmsnEw0h6rwuok7Q71Ul6T/GM09vKVpwutqxjI+c6iqkkDUGdk7O3viHp8JA13vzd5ySf94d2Crn30P2HXs5kg2xbCcfzjRDCfh1aWF/R8aKosLWJ8DLZrbO0J0hfr6hw+YjHU+8H1VDQKo6lbv/CnAXaoaVNW3gDeBFMybHbvPnpXH7YHULT20eLFrNXr88YTcPi28/rrbUvHCC2Hy5FSXJrlOOAH+9jfoGGbDu2efhT/+0a1iUFOT3LKNZPp016v3i1/A1q2jPz9TWaLJ4DUoM2Fs25yplfzuC4dz8tzdgYFEeSwqSwL9SxlFci2a8c9Wjpb8XrFwXz53+HSuuyZAVVXsW0Nu3Dj6c1o6Q1SXFDKlP9F0LZrTJ5ay/+5VOdF1/p43ucsSzYy3N3CUiDwnIk+KyKHe+alAZCfbRu9c2rvwsiJ+EUjd0kOnnOLW3vz1rxNy+7RwzTVuQfCvfjXVJUm+E0+E7m54+uldr6m68aqTJ8N//Vfyyzaar33NbUBw442pLkniWKLJ4BnWmZBoighH7lXLBK+s41nGprp0mK7zzpAvLZqRyW9RQR4ibnmlaz95AHvPCvDgg7t2zYtETz5jGb/S4pW7sjjADacfxBmH7sElx+/N1afsz8y6spxINMProk61RDPticgKEVkb5TgFtzXwRGA+8FXgHhnDWBYRWSIiL4rIi42NjQmqwdiMtPTQlYHELz1UXAxnngn33ee2v8w2r74Kd90FF18MdXWpLk3yHXUUFBZG7z5/9FG3csF///foqxakwj77wGmnudnw27enujSJYYkmg1vfMmGMZlhJIJ/C/Lxxd51HH6PZE/eMcxj8ml65cF++/6kDqC0faK2YNGnX76mbpJx33q7J5mgtmqra33UO8KmDp7FbVQlH1Ndw7D6TmFVbxo6OENvbYxjsmcEGWjRzZy/nTKWqC1R1TpTjQVxL5f3qPA/0AbXAJiByGsM079zQe9+qqvNUdV5dGmUd0ZYeOrJyDcElyVl6aPFit1zaH/6Q2DipcPXVLom67LJUlyQ1Skvhwx/eNdHs63NjMmfOdIvXp6v/9//ctpg//WmqS5IYlmgyeCmfTFhHM0xEmDqhJOoC7qOpKglE3Z7RtWj623X+b/tM4oxDpw+6Hi3RnDFD+MlPhMcei9g2TZSWlpFjdYZ6CfXqsB8SwuuONrUFo17PFu/t6KSoII+JZZnzHjZR/RG3bS8isjdQCDQBDwGfEZEiEZkJzAYyasfkoUsPbWkp5YabipKyiPphh7nWo2zrPl+92iXPl1ySXuMPk+2EE9xrsWXLwLm773Y7R337267FM10ddJDbgODHP3bbrWYbSzQZkmhmUIsmwF1L5nPJgr3H/H1VJW7t0L6+wS2I4Vnn8YrsOp9cueu4qwkTIL9gcOzZewkFBXDccW6Nsc/duJpjr3uaa68dOdZoWy6OtG5oNnmvpYup1SW2UHvm+yUwS0TWAncBi7zWzVeBe4B/Ao8AX1LV3NmJIE4irlXzb3/LrkWyr74aKivd78xcFt6OcsUK9293t5tpfuCBbthEuvv61916oLfemuqS+M8STRjUgpcJYzQjTa4sHtcYzaqSAKrQGhyYAR7s6aUr1OfTGM2C/n9Loywmn5cHNTVDEs3ZA49ramDaHtAWw9pGoyWa4Z9ptBbcbPLejk6bCJQFVLVbVc/yutIPVtX/i7h2rarWq+o+qrp8pPuYXZ19tvvds2xZqkvij1degfvvh0svHbwdZy6qrITKoiAX/Gcn+Xl9TKnqZOP6IBdf7H7m6W7+fDj2WLfUUZf/q3ylVAa8/IkXTtRExjeDOxP1t/JFJF/hLSP9mHVe7I0fnRJlu8ywyZMHt7zttdfg65XDjCMdKlyH0Vo0oy3nlE1comnjM40ZztSpruXrjjuyY03Nq66C6ur0nE2dTMuXw4fmtvPF7qW8EppDUAt5oWsOF7OUKy5uZ3mGfCT7+tdh8+bsG95hiSZQkJ9HWWE+FUUF5I9hh51MFq07ObzckR8tmuBaM6Ptyx42Zchab5EtmuAS3mBPH12hkXsHR23R9NZGzeau8+6ePra2Bq1F05hRLF7str194olUlyQ+L70EDz3kJgBVV4/+/GzV0ADnnNrOQx0L+IFeTj3rKaCXetZzPZfzUMcCzjm1nYaGVJd0dMcdB4cfDtddB6Es+nNliaansiSQUROB4hU10exv0fQn0dxrUjn7T60c9vqkSSO3aIbLGG5pHc5oiWZFcQEi0NKRvbPO39/ZhaqtoWnMaLJlTc1vfQsmTnRLGuWym34U5NzQzziClVGvH8FKvhC6mZ/emP6TQUVcq+aGDW65qmxhiaanorgg48ZnxiOcVEdv0Yy/6xzcRKUrTtp32Ou9Qxoqh86YrIxxEk947OWEsug/v7w8obI4tm74TBVeQ3P3Kks0jRlJSQmccYZbU7O1NdWlGZ/nnoOHH4avfMWNTcxld/62j8+HbhnxOV8I3cydv8mMeXMf+5hbHeGrFweZXOnGm06u7OTSC4IZ0SobjSWantmTKpg9qSLVxUiagXGLrpVPVXn5HbdarF/jVEVkxBnQoy2cHE40d3aNnCA2d3QTyJcRJ0UNt+VmtrA1NI2J3eLFbrvCe+9NdUnG56qroLbWbTeZ65raipjB2yM+Zzrv0NSWGb8bH3kEtm5o56wdS3mm1Y03faZ1DiW3LWX+gZkz3jSSJZqen5z5Qa4/9cBUFyNphnad3/rUen684g2O3KuGWbVlSSnD8ccPPN5nn12vh7vwR2uJ3N7ezYTSwhGT2uF2QsoWtv2kMbGbPx/23jszu8+fecYlI5dfDhW50zYyrNryIG8zY8TnvMN0asvTfyp3eLzpw8EF/JDB402/G8qs8aaRLNH05OUJeTkyEQigOJBHYUEeLR0hGluDLH3sDY7fdxLL/vNwCvKT87Y4/3w4+pg+6vfq4557dr1e5XXhR9uTPVJze/eoi5QPt0B9tnivpYuaskKKA/mpLooxaU8EFi1yWxNm2h/tb33LbXhxwQWpLkl6+OxZedweOG/E59wWOJ/Pnp3+vxuzabxpJEs0c5SIUFNWyLb2bm59qoFgTx9f//cPJHXWfUkJPPl4Hm++kceBURqT+7vOR2vR7OgedXxtVUlg1PtkMltD05ixOftsl3Bm0pqaTz/tFiT/2tegLDkdT2nvwsuK+EXgAp5lftTrzzKf2wLn86VLd904JN1k23jTsIQlmiJyt4is8o4NIrLKO7+niHRGXBv5VTUJU1NeSHN7N//cvJMDplUxq6481UUaJNx1vnOUWeextGhWl2b/GE0bn2lM7PbYAxYscIlmuq6p2dAAl14wMCnko8d2Ul0S5MQTU12y9FFfD8vuLePk0hVcGbieBmYRooAGZnFl4HpOLl3BsnvLkrLNabyybbxpWMISTVU9Q1Xnqupc4D7g/ojLDeFrqjpym7dJmIllRWxrC7J1Z5BJFen3aa84kE9RQd6oLZE7OkJMGGVpqipv8XdVHfF5mUhV2bTdWjSNGavFi91SMk89leqS7Gr5cph/YDsltw1MClnVO4clwaUce3hmTgpJlIULYeXqMoJLLuLIyjWU5AU5snINwSUXsXJ1GQsXprqEscmm8aaREt51Lm6GxunA7xMdy4xNbVkhTW3dbG0NMqkiPT8hjbY7UF+fsr0jhhbNkkJ6+5S24Mito5loZ1cP7d29TLVE05gx+cQn3PJA6TYpKHIR8u+GBk8Kua4vcyeFJFJ9PdxwUxFbWkrp6c1jS0spN9xUlBEtmWHZNN40UjLGaB4FvK+qb0Scmykir4jIkyJyVBLKYKKoKS+ksTVIS2coLVs0wRtbOcLyRju7QvQpoy6237+cUxZOCGpsdZ9u69L0Z2hMuiothdNPd8sctbWlujQDsnVSiBlZNo03jRRXoikiK0RkbZTjlIinncng1szNwHRV/SDwZeBOEYm65KyILBGRF0XkxcbGxniKaqKoKS+iu9cNTkrXJKWyuGDEFs3mdrcO6MRhFmsPqyqNbamkTNS/o5NPW4cak0sWL4b2dreAe7rI1kkhZmTZNN40UlyJpqouUNU5UY4HAUSkAPgUcHfE9wRVdZv3+CWgAdh7mPvfqqrzVHVeXV1dPEU1UUR2N0+qTNNEsyTAzs7hu7u3e9tKxjJGE7Iz0Wzt3zrUnx2djMklH/qQ2/42nbrPs3VSiBldtow3jZTorvMFwOuqujF8QkTqRCTfezwLmA2sT3A5TBS15RGJZpqO0awqCfQnk9Fsb3eJYyyzziFbE01XJ792dDIml4i4Vs0nnoC33kp1aZxsnRRiYpMN400jJTrR/Ay7TgI6GljtLXd0L3CeqjYnuBwmipqygVbMdB2jOau2nE07OoedxNMcY4tmdYm7no1jNMMtmhXWomnMuKTbmprZOinE5KaEJpqqulhVbxly7j5V3d9b2uhgVf3fRJbBDK/Ga9HMEzdeMx0dOK0KVVi7qSXq9e3eGM0JMewMBNaiaYzZ1fTpcNxxcMcd6bGm5oWXFfHz/OybFGJyk+0MlMPCLZo15UVJ3RFoLA6YVgXAmo3RE83mjm4K8/MoKxz5k314y80dncN3w2eq1q4e8oRRXwNjzPAWL3Zd53/7W6pL4haTL6sr4wRZwRVZNCnE5CZLNHNYSWE+pYX5adttDlBbXsTU6hJWj9CiOaEsgFuudXgikrXbULZ29VBeVDDqa2CMGd4nPwkVFekxKejHP4Z334Ubf15GdxZNCjG5yQZ15bia8sK0TjTBdZ+v3rgj6rXtMewKFFZdEsjKMZo7u0LWbW5MnMrK3Jqad90FS5dCeYp25H37bbj6ajjlFDj3XIAibrgpfLU0NYUyJg7WopnjvnbSviw5Or37X+ZMreLtbR1RJwRtb++OOdGsGmWXoUzV2tVjE4GM8UF4Tc377x/1qQlzySXu36VLU1cGY/xkiWaO+9iBu3NEfU2qizGi+royADY0tQPwyNotNLa6HTGaY9h+Mqy6NDtbNFu7QlRai6YxcTvySLe0TKq6z//3f+HBB+Fb33ITlIzJBpZomrQ3s9b1Ya1vaqcr1Mt5v32Jj//EjdgPj9GMxWj7pmcqa9HMLiJykYi8LiKvisgPvHN7ikiniKzyjpG3jTHjIgKLFsHjj7su7GTq6ICLLoL994dLL01ubGMSyRJNk/Zm1JQiAm81tvfve75lZxe9fcqOzhATYx6jWWiJpklrInIscApwkKruD/ww4nKDtyzcXFUdeZFFM27nnOP+Tfaamt/5jktub74ZAtZBYbKIJZom7RUH8tm9qoT1TW20dQ2M03xu/TZUR19DM6y6NEBbsIdQb+oXynvp7WYee+39uO7RFuzhhQ3NtNpkoGxyPvB9VQ0CqOrWFJcn58yY4dbU/PWvQTU5MV97DX74Q9eaetRRyYlpTLJYomkywqy6Mt5qau/fBQfgDy+5nU3HMhkISIsljpY+9ibX/vk1evuUd7Z1jOse97zwLqfd8izbO0LWopk99gaOEpHnRORJETk04tpMEXnFOz9sOiIiS0TkRRF5sbGxMfElzkKLFsH69clZU1MVLrjAzXK//vrExzMm2SzRNBlhZm0ZbzUOTjRXeC2CY2nRBNiRBolmY2uQlo4Qj766hWN/9ATv7egc8z3ebx3Y59haNDOHiKwQkbVRjlNwS85NBOYDXwXuEbdA6mZguqp+EPgycKeIVEa7v6reqqrzVHVeXV1dkmqVXT79aZf4JWNS0G9/6/ZZ//73wX5cJhtZomkywszaMlqDPby1zc0833dKRX/SGesYzco02oayqS1IS2eId5s76O1TXt+yc8z3CG+/CbbPeSZR1QWqOifK8SCwEbhfneeBPqBWVYOqus37/peABlzrp0mAsjI47TT4wx/cckeJsn07fOUrMH8+fOELiYtjTCpZomkywh4T3ELFr292CdnBMyb0X4t11nl1ONFM8RJHfX3KtvZuevq0vyXzza1tY75PsyWa2eiPwLEAIrI3AiFoFgAAIABJREFUUAg0iUidiOR752cBs4H1KStlDli8GFpb4YEHEhfj61+HpiY3ASjP/hqbLGVvbZMRaspdq+Xb3njGQ6YPJJqxrqNZleIWzd4+5Uu/e5n7Xt5Ib5+bZfB2s6tPvIlmIN/+K2eJXwKzRGQtcBewSFUVOBpYLSKrgHuB81S1OYXlzHof/jDMmpW47vPnn4dbbnFLGs2dm5gYxqQDawYxGaG23G2TucHrOg+3aBYW5FESyI/pHjXlRYjA+saxJ3V+eGTtFh5es5mH12zuPxdOnONNNOvSfBtRExtV7QbOinL+PuC+5Jcod+XluaWOrr7aLTs0Y4Z/9+7thfPPh912g2uu8e++xqQjawYxGSHcavnejk5KC/OZMbGUkkA+E0sLcXMlRldVEuDwmRP50+rNaLLWLYlw69O79nS+67VoNjS2j7lMze3dLDpiBk9+9RgO3XOiL2U0xgw45xw3K/w3v/H3vj/7Gbz8Mtx4I1RGndJlTPawRNNkhNLCfIoDefQplBcVkJcnzJ5cHvOM87BPzJ3K+qZ21mxqSVBJo2vpDPGPd3fscr7H60Jv6QzR1Na9y/XhhHr72NnVw8SyImbUlPlWTmPMgJkz4Zhj4I47/FtTc/Nm+MY34MQT3YQjY7KdJZomI4gINWWuezg88eUrJ+7DZSeMbeLtwgN2oyBP+Mur8S2WPlbb2tze7HOm7tp8MaWyGBhb9/n2DpeUToxxIpQxZnwWL4Y334RnnvHnfpddBsEg/PSnbstLY7KdJZomY4QnBIXXjDx67zoW7Dd5TPeoKgkwqaKIzS1doz/ZR+HEcN4M18WdF/EH5uAZ1QC81RT7Oirb292EprG26BpjxubTn3bLHfkxKWjFCvj97+GKK2CvveK/nzGZwBJNkzFqysKJZnxz2CaWF/YnfsmyzesWP2ymSzR3qyqhwMs2951SSWFBXv9Ep1iEJwLFOuPeGDM+5eVw6qlw993QMb5NvADXivmlL7kE84or/CufMenOEk2TMSYO6TqP5z7b2pObaIYTwwOmVlFYkEdtRVH/cksTywqZMbF0TC2almgakzx+rKn5gx/Av/7lusyLi30rmjFpL+5EU0ROE5FXRaRPROYNuXaliLwpIutE5CMR50/yzr0pIvbZzsSkNtx1XhTfuMSJpQGa24N+FClmzV4Lam15ER/YrZIZE0up8rbErC4NsGdtGRua2rnlyQaeaWiK+X6x7opkjBm/o4+GPfd0k4LGo6EBrr0WTj/dTQIyJpf40aK5FvgU8FTkSRHZD/gMsD9wEvAzEcn3drf4KbAQ2A8403uuMSMKt96V+9Ci2TyGGd5+aG7rpiSQT0lhPrcvmse3T5nT36JZXVLIzNoyNmxr57pHXueeF94d9X7h7SdtjKYxiZeXB4sWuTGW747+33MQVbcoe2GhW87ImFwTd6Kpqq+p6rool04B7vL26H0LeBM4zDveVNX13uLEd3nPNWZENeX+dJ3XlBfS3t1LV6jXj2LFpLm9uz9Rri0voqo00L8lZnVpgD1rygj1Kqrw/s7RW1ub27upKC6wHYGMSZLxrql5//2wfLlbmH333RNTNmPSWSL/Sk0FIj/7bfTODXfemBENTAaKr+t8gtfdnMwJQdvau/tnzYdVRSaataX9599v7eInj73BGT9/dtj7be/o7q+HMSbxZs1yXei//nXsa2q2tsIll7gtJi+8MKHFMyZtxZRoisgKEVkb5UhoS6SILBGRF0XkxcbGxkSGMhmgf3mjoni7zt19tiWx+zxaYjiQaBayZ8Si6++3dPH8hmbWjrCofEtniOpSW0PTmGRavBjeeANWrozt+VdfDe+9BzffDAW24bPJUTElmqq6QFXnRDkeHOHbNgF7RHw9zTs33PlocW9V1XmqOq+uri6WoposNntSBR/ZfzKHz4pvu8VwwtqcxJnn29q6+1tkw/aeUsHU6hLKCvPZraqYi4/bi08fPI327l5e39JKe3cvod6+qPfb0RHqT1SNMclx6qlQWhrbmpqrV8OPfwznngvz5ye8aMakrUR2nT8EfEZEikRkJjAbeB54AZgtIjNFpBA3YeihBJbDZImSwnx+fva8uLdcTEXXeeQYzbDPHjadpy8/FhFBRPjyifvw4dk1ADS2unGaOztDUe/X0mmJpjHJVlHhks277oLOzuGf19cH558PEyfC976XvPIZk478WN7okyKyETgCeFhEHgVQ1VeBe4B/Ao8AX1LVXlXtAS4EHgVeA+7xnmtMUtQkueu8s7uXzlDvLjPERYS8vMF70E2uHLzAXssIiaZ1nRuTfIsWwc6d8Mc/Dv+cX/3KbVl5/fUu2TQml8U9akRVHwCiLmOrqtcC10Y5/2fgz/HGNmY8qkoC5Enyus7Da14O7TqPJpZEU1Vdollik4GMSbZjjoHp0133+Zln7nq9qQkuvxyOOsrNVDcm19naKCbn5OUJE0oLWd/UxpYk7Hk+ljUvY0k024I99PapdZ0bkwKRa2puijK74IorXIvnzTeDyK7Xjck1lmianCQi/HnNFk4fYQkhv+zscsliLIlheVEBZYX5/V9HSzR3dHj3s65zY1Ji0SI3DnPompp//zvcfjt8+cuw//6pKZsx6cYSTZOTzj1qJgDvNHfQ1xd9UbzePh121vdYtHb1AC6JjMXkqmIKC9x/zWiTgcLJZ7W1aBqTEvX1rms8ck3NUMhNAJo+Hb75zZQWz5i0YommyUlf/Ld6rvq42/l0uNnnP/rLOk69+Zm4Y7V5iWasOxrV15Uzb8YEIHqLZvicdZ0bkzoLF8Jb64LUlXeSn9fHpIpO1q0JcuWVUBbfwhjGZBVLNE3Oqqtw4yEb26Jv+fjKOzt4Y2tb3HHaguFEM7bE8IbTD+KWsw+hOJA3Ytd5te0MZExKLF8ON3ynnYtYynMdcwhqIS8G53CxLOW/L2tn+fJUl9CY9GGJpslZdRVu7/Stw+wt/k5zBx3dvQR74tsTvdUbo1lWlD/KM52K4gCVxQGqSgIjtmja8kbGJF9DA5xzajsPdSzgh1xOPespoJd61nO9Xs5DHQs459R2GhpSXVJj0oMlmiZnhRPN8OLokbp7+tjc4lZkDrcgjldrsIfCgjyKCmJLNMOGSzR3dHb3XzfZR0TuFpFV3rFBRFZFXLtSRN4UkXUi8pFUljNX3fSjIOeGfsYRRN+H8ghW8oXQzfz0xugfYI3JNZZompw1KZxoRuk637i9g/AcoXjX22zr6hnX/uzDtmh2hCgqyKM4MLbE1WQGVT1DVeeq6lzgPuB+ABHZD7eT2v7AScDPRMTeBEl252/7+HzolhGf84XQzdz5m/h6QozJFpZompxVVlRAaWF+1BbNd5o7+h/Hu1Vla1dPzBOBIrlEs2eX87YrUG4QEQFOB37vnToFuEtVg6r6FvAmcFiqypermtqKmMHbIz5nOu/Q1FY84nOMyRWWaJqcVldRNGqiGW/XeVuwh/JxJJqVJYGoyxvt6LB9znPEUcD7qvqG9/VU4N2I6xu9c4OIyBIReVFEXmxsbExCMXNLbXmQt5kx4nPeYTq15YnfDMKYTGCJpslpk4ZJNN/e5l+LZltXT8xraEYaaYymJZqZTURWiMjaKMcpEU87k4HWzJip6q2qOk9V59XV1flXaAPAZ8/K4/bAeSM+57bA+Xz2bBvVYAxYomlyXF1FERu2tfPnNZsHnX+nuYM9a0qBgS0kx2tnVyjmpY0iVZUEaAv20DNk0fjm9m5qyoriKpNJLVVdoKpzohwPAohIAfAp4O6Ib9sE7BHx9TTvnEmiCy8r4heBC3iW+VGvP8t8bgucz5cutf+jxoAlmibH7VZVwuaWLi743cu8G9Fd/lZTO3tPrqC0MJ/tPnSdj2cyUI23N/q2IYluc3s3E8ttDc0stwB4XVU3Rpx7CPiMiBSJyExgNvB8SkqXw+rrYdm9ZZxcuoIrA9fTwCxCFNDALK4MXM/JpStYdm8Z9fWpLqkx6cESTZPTzj+mnkuOnw3A5hY3pirY09ufaE4oLYy/63ycYzSn17jtRTY0tfef6+tTr0XTEs0s9xmGdJur6qvAPcA/gUeAL6mqTW1OgYULYeXqMoJLLuLIyjWU5AU5snINwSUXsXJ1GQsXprqExqQPSzRNTqstL+LfD9wNgPd3ukTzraZ2evuUvadUUF0aiKvrXFVpHecYzZleovlWRKK5ozNEn8JESzSzmqouVtVd1tBR1WtVtV5V91FV238mherr4YabitjSUkpPbx5bWkq54aYia8k0ZghLNE3Om+xtRRlONNdtaQVgn/4WzfF3nXeF+ujt03GN0dy9uphAvvDWtoFEs7ndTVyyRNMYY0wmGHszizFZprKkgKKCPLZ6s8/feL+NgjxhZm0ZE8oK2bi9Y5Q7RLe+sY3//YebZDServOC/Dz2mFg6qOt8W5trXa0tt4kGxhhj0p8lmibniQiTK4vZ3NLFz59s4I+rNjGztozCgjwmlAbG3aJ58V2vsHbTToBxTQYC132+oWkg0Q3vUmQtmsYYYzKBdZ0bA0yuLOJvbzTyveWvs3F7J7tVlwAwobSQls4QoSFLDMUiP2/gv9d4dgYC2LO2jA3b2unz9sNs8hJNmwxkjDEmE1iiaQwwqaK4v+XyiFk1fPaw6QBMqRo8fnM4Hd09LP7V86zeuKP/3NTqgS3oxjMZCFyiGezpY4sXv9nrOp9giaYxxpgMYImmMcCkSjfmsaaskDvPPZyT5kwBYKrXsrlpe+eI39+wtZ0n1jVy8k1/719gvbVrYJ/y8YzRBNirrhyAde+7CUrN7UGqSgIE8u2/rjHGmPQX118rETlNRF4VkT4RmRdx/gQReUlE1nj/Hhdx7QkRWSciq7xjUjxlMMYPkytd6+MHp1cjIv3np07wEs0dIyea29oHtrG86wW3HfXOiESzchyzzgEO2qOKgjzh+beaAdd1bt3mxhhjMkW8zSJrcdukPTXkfBPwcVU9AFgE/GbI9c+p6lzv2BpnGYyJ22SvRfOD0ycMOh9u0XxvlEQzvKh7YUEef1r9HgCtnSE+esAUHvzSkewxsXRc5SotLOCAaVX9iWZzW7dNBDLGGJMx4ko0VfU1VV0X5fwrqvqe9+WrQImI2HosJm3NqnVd1PNnTRx0vjiQT01Z4egtmt7YydMOmcYLG7bT0hmipTPEhNJCDtqjOq6yHT6zhtUbd9DZ3eu2n7RE0xhjTIZIxkCvTwMvq2ow4tyvvG7z/5bIfsohRGSJiLwoIi82NjYmvqQmZx20RzVPX34sh8yYuMu1qRNK2DjKGM3tHd3k5wmf+OBUevuUp99oZGdXiMqS8XWZRzps5gRCvco/Nu5gW3uQGtvn3BhjTIYYNdEUkRUisjbKcUoM37s/cB3wxYjTn/O61I/yjrOH+35VvVVV56nqvLq6utFrY0wchuvenlpdMmqLZnN7NxNKCzl4+gQmlAZYvnYLoV4d99jMSLMnVQDwr/dbaWrrZreqkrjvaYwxxiTDqFNhVXXBeG4sItOAB4BzVLUh4n6bvH9bReRO4DBg2XhiGJMMU6tLeHzdVlSV4Rrgt7W5STr5ecIHdqtk1TtumaMqH1o0p1QVkyfwwobtAOxebYmmMcaYzJCQrnMRqQYeBq5Q1b9HnC8QkVrvcQD4GG5CkTFpa+qEErpCff278kSzvaObCWUuqdw9ogW0siT+zbcC+XlMqSzmBW9C0O4R63MaY4wx6Sze5Y0+KSIbgSOAh0XkUe/ShcBewDeHLGNUBDwqIquBVcAm4BfxlMGYRAt3VW9uGX7R9m3t3dSUuflukS2OfnSdg0t2w4u2T7UWTWOMMRkiruYWVX0A1z0+9Px3gO8M822HxBPTmGQLt0pGLsA+VORs8N2rBloc/ZgMBDBtQml/1/mUKmvRNMYYkxlsexFjRhFulWwLRk80e3r73FJG4UQzosXRjzGaMNCKWVteRFFBvi/3NMYYYxLNEk1jRhHep7y1KxT1+o7OEKr079gTOYaycpxbTw4V3qFoqo3PNMYYk0Es0TRmFOF9yodr0dzuTRIKd51HLj9U4dcYTa9F02acG2OMyST+NLcYk8Uqikceo7ltSKJZVlRAdWmAYKiPwgJ/PsuFWzQt0TTGGJNJrEXTmFEUFeRTmJ83bKK5o8N1qUeOx9ytqsS38ZkA0yaUMG1CCQcP2YvdGGOMSWfWomlMDCqKC2gLRh+jubNr10RzZm0pBXnD7q46ZkUF+fzta8f5dj9jjDEmGSzRNCYG5cUFw7Zo7ux0iWbkUkZXfXx/OkO9SSmbyR4icjewj/dlNbBDVeeKyJ7Aa8A679pKVT0v+SU0xpixsUTTmBhUFBfQNlyi2dWDCFQUDfx3mlRps8PN2KnqGeHHIvIjoCXicoOqzk1+qYwxZvws0TQmBuVFu7ZoNrd3s7W1i52dISqKCsjzsavc5DYREeB0wMZLGGMymk0GMiYG5UUBWocsb7T0sTc467bn2dkZ8m0HIGM8RwHvq+obEedmisgrIvKkiByVqoIZY8xYWIumMTGoLC7g9SELtm/a0UlTW5Cm9m7f9jQ32U9EVgBTolz6uqo+6D0+E/h9xLXNwHRV3SYihwB/FJH9VXVnlPsvAZYATJ8+3d/CG2PMGFmiaUwMyosLdlmwvbE1CMCGpvZBuwEZMxJVXTDSdREpAD4FHBLxPUEg6D1+SUQagL2BF6Pc/1bgVoB58+apfyU3xpixs65zY2JQ4c06Vx34u93U5hLNjds7fF0z0+S8BcDrqroxfEJE6kQk33s8C5gNrE9R+YwxJmbWomlMDMqLAvT2KV2hPkoK81HV/kSzT7Guc+OnzzC42xzgaOAaEQkBfcB5qtqc9JIZY8wYWaJpTAwGtqEMUVKYT3t3L12hvv7rNhnI+EVVF0c5dx9wX/JLY4wx8bGuc2Ni0J9oeuM0m7zxmWHWommMMcbsyhJNY2JQXhRu0exhW1uQDdvaB12vLLHOAWOMMWYo++toTAwqvBbLtq4ezr79ed5r6Rx03Vo0jTHGmF1Zi6YxMQjPKt/a2sW691vZ0eHW1JxcWTToujHGGGMGWKJpTAxm1JSSJ/D4ukZ6+waWONp7cgVgk4GMMcaYaOJKNEXkNBF5VUT6RGRexPk9RaRTRFZ5xy0R1w4RkTUi8qaILPX29DUmrRUH8plRU8bjr2/tPzehNMDkSrdQu43RNMYYk0kaGuDSC4JMruwkP6+PyZWdXHpBkIYGf+PE26K5FreDxVNRrjWo6lzvOC/i/M3AubgFh2cDJ8VZBmOSYvak8v7dgQoL8qgtL6KmvBCwMZrGGGMyx/LlMP/AdkpuW8ozrXMIaiHPtM6h5LalzD+wneXL/YsVV6Kpqq+p6rpYny8iuwGVqrpS3RYry4BPxFMGY5Il3E1eXRrgI/tPYc7UKupryykvKmBCaWGKS2eMMcaMrqEBzjm1nYc6FvDd0OXUs54CeqlnPd8NXc5DHQs459R231o2E9nfN1NEXgF2At9Q1aeBqcDGiOds9M4Zk/ZmTy4HYGZtGUs/MxcRobdPWbDfZEoK81NcOmOMMWZ0N/0oyLmhn3EEK6NeP4KVfCF0Mz+98SJuuKko7nijtmiKyAoRWRvlOGWEb9sMTFfVDwJfBu4UkcqxFk5ElojIiyLyYmNj41i/3RhfhVs0Z9aWER5anJ8nTCyz1kxjjDGZ4c7f9vH50C0jPucLoZu58ze9vsQbtUVTVReM9aaqGgSC3uOXRKQB2BvYBEyLeOo079xw97kVuBVg3rx5OtzzjEmGWXVlVJcGOGhadaqLYowxxoxLU1sRM3h7xOdM5x2a2op9iZeQ5Y1EpE5E8r3Hs3CTftar6mZgp4jM92abnwM8mIgyGOO3ooJ8nr78WM6aPyPVRTHGGGPGpbY8yNuM/HfsHaZTW97lS7x4lzf6pIhsBI4AHhaRR71LRwOrRWQVcC9wnqo2e9cuAG4D3gQaAB/nNhmTWBXFAfLzbEUuY4wxmemzZ+Vxe+C8EZ9zW+B8Pnu2P3MP4poMpKoPAA9EOX8fcN8w3/MiMCeeuMYYY4wxZuwuvKyI+XdcwMdD90edEPQs87ktcD4rL41/IhDYzkDGGGOMMTmjvh6W3VvGyaUruDJwPQ3MIkQBDcziysD1nFy6gmX3llFf7088SzSNMcYYY3LIwoWwcnUZwSUXcWTlGkryghxZuYbgkotYubqMhQv9i2X75hljjDHG5Jj6erjhpiJuuCl8pjQhcaxF0xhjjDHGJIQlmsYYY4wxJiEs0TTGGGOMMQkhqpmx4Y6INMIoS9kPqAWaElicVMazulm8dIuV7Hh+x5qhqnU+3i9tjPH3ZqRk/TyTEcfqkptxsqEOyYwx1jhNAKp60mhPzJhEcyxE5EVVnZeN8axuFi/dYiU7XrLrlouS9RonI47VJTfjZEMdkhkjkXGs69wYY4wxxiSEJZrGGGOMMSYhsjXRvDWL41ndLF66xUp2vGTXLRcl6zVORhyrS27GyYY6JDNGwuJk5RhNY4wxxhiTetnaommMMcYYY1LMEk1jjDHGGJMQGZFoisgeIvK4iPxTRF4VkUu88xNF5K8i8ob37wTvvIjIUhF5U0RWi8jBEfd6RER2iMifEh1PRGaIyMsissq7z3kJrluvF2uViDyU4LodGxFrlYh0icgnEly/60RkrXec4UOsfUXkWREJishXhtzrlyKyVUTW+vg+iRpPRIpF5HkR+Yd3n6sTXLcNIrLG+7m9mOC67TPkfbJTRP4rgXW7xHt/vDo0jjHGmBRQ1bQ/gN2Ag73HFcC/gP2AHwBXeOevAK7zHn8UWA4IMB94LuJexwMfB/6U6HhAIVDkPS4HNgC7J7Bubcl8LSPuORFoBkoT+Fr+O/BXoAAoA14AKuOMNQk4FLgW+MqQex0NHAys9fG1jBrPq2u59zgAPAfMT2DdNgC1Pr9Pho0Xcc98YAtugfREvI5zgLVAqfc+WQHslczfVXbYYYcddgw+MqJFU1U3q+rL3uNW4DVgKnAKcIf3tDuAcIvaKcAydVYC1SKym/f9jwGtyYinqt2qGvSeU0SUFmQ/6xaLBMU7FViuqh0JjLcf8JSq9qhqO7AaGLQjwVhjqepWVX0BCEUp91O45HlYfsXz6trmfRnwDh3yHN/qFosExTseaFDVQTvV+BjrA7gPJh2q2gM8CXxqrHXPNSJSKCLniMgC7+vPishNIvIlEQn4HGuWiHxFRP5HRG4QkfNEpNLnGB8RkZtF5CHvuFlERt29xMf43/TxXh8Rkc+LyJ5Dzv+nXzFGiO1bPbz7Jawu4pwuIqd5j48X1zN2gYgkNM8Rkf/z+X61Q74+y6vLEhERn2J8UkQmeo/rRGSZ18t1t4hM8yNGpIxINCN5b9IP4lp9JqvqZu/SFmCy93gq8G7Et230ziU9ntctuNq7fp2qvpeoWECxiLwoIislSjd2AuKFfQb4fYLj/QM4SURKvf+IxwJ7xBnLN/HGE5F8EVkFbAX+qqrPJSoWLon9i4i8JCJLYihbvPHCRn2fxBlrLXCUiNSISCmudXzY94jp9ytcj8ElIvIb4DTc638ocJtfQUTkYuAWoNi7dxHu57NSRI7xKcaPgUtwHzJ+4B1PAheLyP/4ESMGX/DjJiLyXeDrwAHAYyJyUcTlC/2IMQpf6gFJqctPgdOBs4HfAOfher2OBm704f4AiBvOFXmsAY4Mf+1TmL9ExPsGrk4vAScAN/gU41pVDTek3AS8AizE9Sb+yqcY/Qr8vmEiiUg5cB/wX6q6MzK5V1UVEV/XavIjnqq+CxwoIrsDfxSRe1X1/UTEwnVJbhKRWcD/icgaVW1IVN28++yG++Xx6CjPiyueqv5FRA4FngEagWeB3kTWLVY+vU96gbkiUg08ICJzVHWX8aE+1e3D3vtkEvBXEXnda8FNSN28+xQCJwNXjvCceN8jr4nIdbhf1O3AKoZ5j5hBDlDVA0WkANiEG97TKyK/xX3A88u5wFzv3jcAf1bVY0Tk58CDuA8Y8fqoqu499KSI3I0bknGJDzEQkZ3DXQJK/IiBG+L1QVXtEZGrgDtFZJaqXurFiVuS6gGJr8tRqnqAuBb4LcBuqtotIr8HXvbh/mEbgJ3Ad4BOXNmfxtXPL5Gvx6dwdWsXkTvxry75EY/3UtXwnIdfSwLGtmdMi6b3BroP+J2q3u+dft9LdMIJz1bv/CYGt2RM886lLJ7XkrkWOCpRsVQ1/O964AmG+cXtc91OBx5Q1WG7TH2s37WqOldVT8D9Z/xXnLHi5nc8Vd0BPM6QYQF+xop4n2wFHgAOS0LdFgIvR/uQ5XPdblfVQ1T1aGA7Ud4jZhd53geBCtz41irvfBFuGIefwo0bRbhx66jqOz7G6fI+kA51KNDlUwyAHcBsVa0cclQAm0f75hgVeENAwr8XPg5UisgfcOP//ZCMekDi6xK+dwh4QVW7va97gD4f7o93v5Nxv6duBQ5S1Q1ASFXfHjokKA4lIvJBETkEyFc3VCxcN78+OD8hIteISIn3+JPgJvkCLT7F6JcRiaa4po3bgddUNbLp+CFgkfd4Ee5Tcfj8OeLMB1oiuuCSFk9Epnk/SMTNmP0wsC5BsSaISJF3z1rgSOCfiapbxPedyQjdoT7WL19Earx7HggcSEQXwzhjxcWveOLGyFR7j0twXSSvJyhWmYhUhB8DJ+I+ACWkbhGGfZ/4GctrpUVEpuNaA+6MsXy57Hbc+20VrnvzDyLyC1zX410+xrkNeMG797O47k5EpI5RxkOPwWLgJnErGPzFO14DlnrX/LIMmDHMNb/ecw0i8m/hL1S1V1U/j/sb8gGfYiSjHpD4umwR1yOCqvZ/SBeRKUC3D/fvp6oP4D44HyMiD+Jf0h+2GddF/kOgOeLDdg1eQu2DC3EJ+DrcUJn7RKQV1+twtk8xBmgazEga7cAlaIqbALLKOz4K1ACPAW/gZpiAviZIAAAIGklEQVRO9J4vuF9iDcAaYF7EvZ7Gdb124sb/fSRR8XAJw2pc99NqYEkCY33I+/of3r+fT8JruSeutTEv0T873Liuf3rHSlwXXLyxpnjvgZ24T/Yb8Way45KizbgJJxujvZ5+xcMlza9491kLfDOBsWZ575F/AK8CX/fp5zbSa1kGbAOqkhDrae898g/g+FT/7sqUA9gdb0UMoBo3we+wBMTZ37v3vgmuzxTgEO+YkurXd5x1KAFKhrk2NdXly4S6eL97JiXw/gcB5yXpNcwnysouPty3CqhJZNltC0pjjMlxXqvyYQxM9NsEPK8+/4FIVpxhYu+rqq+P/sz0j+NnDBGpwg3VifyZPKqui9s3iY6TLfXIphhhGdF1bowxJjFE5ERcq/FVuJbkjwJXA2941zIqzgj+MvpTMiaOLzFE5BzcBJNjcONzS3ErerzkXfNFouNkSz2yKcageNaiaYwxucsbw7hQ3cSGyPMzcTPDfRkPmIw4IrJ0uEvAIlX1Zc3OZMRJUox1wOFDW7G8OQXPaZQZ/OkYJ1vqkU0xImXU8kbGGGN8V4Ab5zrUJvyddZ6MOP8BXAYEo1w706cYyYqTjBjCkM0hPH34tIRSkuJkSz2yKUY/SzSNMSa3/RI3G/wuBjZLmA6cgZuRnklxXsBtG/vM0Avi1m/0SzLiJCPGtcDLIvIXBv9MTgC+7VOMZMTJlnpkU4x+1nVujDE5TkQ+gNvyM3JiwEOqussSaekcR9y2el0aZTtcPyUjThLrMgH4CLtOCtmeSXGypR7ZFKM/liWaxhhjIonIwertP5/pcawuY47xMVX9UyJjJCNOttQjG2JYommMMWYQEXlZVQ/OhjhWl/SLkYw42VKPbIhhyxsZY4wZyvcJASmMY3VJvxjJiJMt9cj4GJZoGmOMGerqLIpjdRmbLyYhRjLiZEs9Mj6GzTo3xpgcJyJHA++r6joRORLYS0T+XVUfzrQ4VpcxxyjH7RCzB9AL/EtE8lS1z68YyYiTLfXIphj9sWyMpjHG5C4R+TFuW8gC4FHgeGA58G/AK6r61UyJY3UZc4zTga8Aq3E7wzyD6+k8APicqq6JN0Yy4mRLPbIpxiCJ3EjdDjvssMOO9D6AV3Hjs0qB7UCpdz6AW8cxY+JYXcYcY3XEfWtxy9sAHAg84+PrldA42VKPbIoRedgYTWOMyW2q7q9MuMss3M3Vh7/j+JMRx+oyNgJ0eo/bgUle4NWAL9t1JilOttQjm2L0szGaxhiT2x4WkaeBYuA24B4RWYnron0qw+JYXcbmz8AjIvIUbrzeH6B/sXg/ZyEnOk621CObYvSzMZrGGJPjROQIXAvaShGpBz4JvAPcq/5OQEh4HKvLmGN8FNgP+Ieq/tU7lwcEVDXaPutpGSdb6pFNMfpjWaJpjDFGRCYTsR2dqr6fqXGsLukXIxlxsqUe2RQDLNE0xpicJiJzgVuAKtx+xwDTgB3ABerTlofJiGN1Sb8YyYiTLfXIphiD+D27yA477LDDjsw5gFXA4VHOz8d1q2VMHKtL+sVIRpxsqUc2xYg8bNa5McbktjJVfW7oSVVdCZRlWByrS/rFSEacbKlHNsXoZ7POjTEmty0XkYeBZcC73rk9gHOARzIsjtUl/WIkI0621CObYvSzMZrGGJPjRGQhcAoREwOAh1T1z5kWx+qSfjGSESdb6pFNMfpjWaJpjDHGGGMSwcZoGmNMDhORKhH5voi8JiLNIrLNe/x9EanOpDhWl/SLkYw42VKPbIoRyRJNY4zJbffg9tI+VlUnqmoNcCxuqZN7MiyO1SX9YiQjTrbUI5ti9LOuc2OMyWEisk5V9xnrtXSMY3VJvxjJiJMt9cimGJGsRdMYY3Lb2yJyubhdQgC3Y4iIfI2BGamZEsfqkn4xkhEnW+qRTTH6WaJpjDG57QygBnhSRLaLSDPwBDAROD3D4lhd0i9GMuJkSz2yKUY/6zo3xpgcJyL74ragW6mqbRHnT1JV39bVS0Ycq0v6xUhGnGypRzbFCLMWTWOMyWEicjHwIHAhsFZETom4/N1MimN1Sb8YyYiTLfXIphiRbGcgY4zJbecCh6hqm4jsCdwrInuq6v8AkmFxrC7pFyMZcbKlHtkUo58lmsYYk9vywl1nqrpBRI7B/eGZgb9/dJIRx+qSfjGSESdb6pFNMQaC+X1DY4wxGeV9EZkb/sL7A/QxoBY4IMPiWF3SL0Yy4mRLPbIpRj+bDGSMMTlMRKYBPaq6Jcq1I1X175kSx+qSfjGSESdb6pFNMQbd0xJNY4wxxhiTCNZ1bowxxhhjEsISTWOMMcYYkxA269wYY4zJQiJSAzzmfTkF6AUava87VPVDKSmYySk2RtMYY4zJciJyFdCmqj9MdVlMbrGuc2OMMSbHiEib9+8xIvKkiDwoIutF5Psi8jkReV5E1ohIvfe8OhG5T0Re8I4jU1sDkyks0TTGGGNy20HAecAHgLOBvVX1MOA24CLvOf8D3KiqhwKf9q4ZMyobo2mMMcbkthdUdTOAiDQAf/HOrwGO9R4vAPYT6d84plJEysM7zBgzHEs0jTHGmNwWjHjcF/F1HwN5Qh4wX1W7klkwk/ms69wYY4wxo/kLA93oRG5haMxILNE0xhhjzGguBuaJyGoR+SduTKcxo7LljYwxxhhjTEJYi6YxxhhjjEkISzSNMcYYY0xCWKJpjDHGGGMSwhJNY4wxxhiTEJZoGmOMMcaYhLBE0xhjjDHGJIQlmsYYY4wxJiEs0TTGGGOMMQnx/wFO5cmgQ/GKuQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 792x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "ts = create_ts(start = '2001', n = 222)\n",
    "\n",
    "# Left\n",
    "plt.figure(figsize=(11,4))\n",
    "plt.subplot(121)\n",
    "plt.plot(ts.index, ts)\n",
    "plt.plot(ts.index[90:100], ts[90:100], \"b-\", linewidth=3, label=\"A training instance\")\n",
    "plt.title(\"A time series (generated)\", fontsize=14)\n",
    "\n",
    "# Right\n",
    "plt.subplot(122)\n",
    "plt.title(\"A training instance\", fontsize=14)\n",
    "plt.plot(ts.index[90:100], ts[90:100], \"b-\", markersize=8, label=\"instance\")\n",
    "plt.plot(ts.index[91:101], ts[91:101], \"bo\", markersize=10, label=\"target\", markerfacecolor='red')\n",
    "plt.legend(loc=\"upper left\")\n",
    "plt.xlabel(\"Time\")\n",
    "plt.xticks(rotation=90)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://github.com/thomaspernet/Tensorflow/blob/master/tensorflow/21_RNN_V3_files/image010.png)\n",
    "\n",
    "The right part of the graph shows all series. It starts from 2001 and\n",
    "finishes in 2019 It makes no sense to feed all the data in the network,\n",
    "instead, you need to create a batch of data with a length equal to the\n",
    "time step. This batch will be the X variable. The Y variable is the same\n",
    "as X but shifted by one period (i.e., you want to forecast t+1).\n",
    "\n",
    "Both vectors have the same length. You can see it in the right part of\n",
    "the above graph. The line represents the ten values of the X input,\n",
    "while the red dots are the ten values of the label, Y. Note that, the\n",
    "label starts one period ahead of X and finishes one period after.\n",
    "\n",
    "# Build an RNN to predict Time Series\n",
    "\n",
    "Now, it is time to build your first RNN to predict the series above. You\n",
    "need to specify some hyperparameters (the parameters of the model, i.e.,\n",
    "number of neurons, etc.) for the model:\n",
    "\n",
    "-   Number of input: 1\n",
    "\n",
    "-   Time step (windows in time series): 10\n",
    "\n",
    "-   Number of neurons: 120\n",
    "\n",
    "-   Number of output: 1\n",
    "\n",
    "Your network will learn from a sequence of 10 days and contain 120\n",
    "recurrent neurons. You feed the model with one input, i.e., one day.\n",
    "Feel free to change the values to see if the model improved.\n",
    "\n",
    "Before to construct the model, you need to split the dataset into a\n",
    "train set and test set. The full dataset has 222 data points; you will\n",
    "use the first 201 point to train the model and the last 21 points to\n",
    "test your model.\n",
    "\n",
    "After you define a train and test set, you need to create an object\n",
    "containing the batches. In this batches, you have X values and Y values.\n",
    "Remember that the X values are one period lagged. Therefore, you use the\n",
    "first 200 observations and the time step is equal to 10. The `X_batches`\n",
    "object should contain 20 batches of size 10\\*1. The `y_batches` has the\n",
    "same shape as the `X_batches` object but with one period ahead.\n",
    "\n",
    "**Step 1)** Create the train and test\n",
    "\n",
    "First of all, you convert the series into a numpy array; then you define\n",
    "the windows (i.e., the number of time the network will learn from), the\n",
    "number of input, output and the size of the train set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "series = np.array(ts)\n",
    "n_windows = 20   \n",
    "n_input =  1\n",
    "n_output = 1\n",
    "size_train = 201"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After that, you simply split the array into two datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(201,) (21,)\n"
     ]
    }
   ],
   "source": [
    "## Split data\n",
    "train = series[:size_train]\n",
    "test = series[size_train:]\n",
    "\n",
    "print(train.shape, test.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Step 2)** Create the function to return `X_batches` and `y_batches`\n",
    "\n",
    "To make it easier, you can create a function that returns two different\n",
    "arrays, one for `X_batches` and one for `y_batches`.\n",
    "\n",
    "Let's write a function to construct the batches.\n",
    "\n",
    "Note that, the X batches are lagged by one period (we take value t-1).\n",
    "The output of the function should have three dimensions. The first\n",
    "dimensions equal the number of batches, the second the size of the\n",
    "windows and last one the number of input.\n",
    "\n",
    "The tricky part is to select the data points correctly. For the X data\n",
    "points, you choose the observations from t = 1 to t =200, while for the\n",
    "Y data point, you return the observations from t = 2 to 201. Once you\n",
    "have the correct data points, it is straightforward to reshape the\n",
    "series.\n",
    "\n",
    "To construct the object with the batches, you need to split the dataset\n",
    "into ten batches of equal length (i.e., 20). You can use the `reshape`\n",
    "method and pass -1 so that the series is similar to the batch size. The\n",
    "value 20 is the number of observations per batch and 1 is the number of\n",
    "input.\n",
    "\n",
    "You need to do the same step but for the label.\n",
    "\n",
    "Note that, you need to shift the data to the number of time you want to\n",
    "forecast. For instance, if you want to predict one timeahead, then you\n",
    "shift the series by 1. If you want to forecast two days, then shift the\n",
    "data by 2.\n",
    "\n",
    "x\\_data = train\\[:size\\_train-1\\]: Select all the training instance\n",
    "minus one day\n",
    "\n",
    "X\\_batches = x\\_data.reshape(-1, windows, input): create the right shape\n",
    "for the batch e.g (10, 20, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_batches(df, windows, input, output):\n",
    "    ## Create X         \n",
    "        x_data = train[:size_train-1] # Select the data\n",
    "        X_batches = x_data.reshape(-1, windows, input)  # Reshape the data \n",
    "    ## Create y\n",
    "        y_data = train[n_output:size_train]\n",
    "        y_batches = y_data.reshape(-1, windows, output)\n",
    "        return X_batches, y_batches"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that the function is defined, you can call it to create the batches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "X_batches, y_batches = create_batches(df = train,\n",
    "                                      windows = n_windows,\n",
    "                                      input = n_input,\n",
    "                                      output = n_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can print the shape to make sure the dimensions are correct."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10, 20, 1) (10, 20, 1)\n"
     ]
    }
   ],
   "source": [
    "print(X_batches.shape, y_batches.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You need to create the test set with only one batch of data and 20\n",
    "observations.\n",
    "\n",
    "Note that, you forecast days after days, it means the second predicted\n",
    "value will be based on the true value of the first day (t+1) of the test\n",
    "dataset. In fact, the true value will be known.\n",
    "\n",
    "If you want to forecast t+2 (i.e., two days ahead), you need to use the\n",
    "predicted value t+1; if you're going to predict t+3 (three days ahead),\n",
    "you need to use the predicted value t+1 and t+2. It makes sense that, it\n",
    "is difficult to predict accurately t+n days ahead."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10, 20, 1) (10, 20, 1)\n"
     ]
    }
   ],
   "source": [
    "X_test, y_test = create_batches(df = test, windows = 20,input = 1, output = 1)\n",
    "print(X_test.shape, y_test.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alright, your batch size is ready, you can build\n",
    "the RNN architecture. Remember, you have 120 recurrent neurons.\n",
    "\n",
    "**Step 3)** Build the model\n",
    "\n",
    "To create the model, you need to define three parts:\n",
    "\n",
    "1.  The variable with the tensors\n",
    "\n",
    "2.  The RNN\n",
    "\n",
    "3.  The loss and optimization\n",
    "\n",
    "**Step 3.1)** Variables\n",
    "\n",
    "You need to specify the `X` and `y` variables with the appropriate\n",
    "shape. This step is trivial. The tensor has the same dimension as the\n",
    "objects `X_batches` and `y_batches`.\n",
    "\n",
    "For instance, the tensor `X` is a `placeholder` (Check the tutorial on\n",
    "[*Introduction to Tensorflow*]to refresh\n",
    "your mind about variable declaration) has three dimensions:\n",
    "\n",
    "-   `Note`: size of the batch\n",
    "\n",
    "-   `n_windows`: Lenght of the windows. i.e., the number of time the\n",
    "    model looks backward\n",
    "\n",
    "-   `n_input`: Number of input\n",
    "\n",
    "The result is: tf.placeholder(tf.float32, \\[None, n\\_windows,\n",
    "n\\_input\\])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf.reset_default_graph()\n",
    "r_neuron = 120    \n",
    "\n",
    "## 1. Construct the tensors\n",
    "X = tf.placeholder(tf.float32, [None, n_windows, n_input])   \n",
    "y = tf.placeholder(tf.float32, [None, n_windows, n_output])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Step 3.2)** Create the RNN\n",
    "\n",
    "In the second part, you need to define the architecture of the network.\n",
    "As before, you use the object `BasicRNNCell` and `dynamic_rnn` from\n",
    "TensorFlow estimator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "## 2. create the model\n",
    "basic_cell = tf.contrib.rnn.BasicRNNCell(num_units=r_neuron, activation=tf.nn.relu)   \n",
    "rnn_output, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32)   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next part is a bit trickier but allows faster computation. You need to transform the run output to a dense layer and then convert it again to have the same dimension as the input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "stacked_rnn_output = tf.reshape(rnn_output, [-1, r_neuron])          \n",
    "stacked_outputs = tf.layers.dense(stacked_rnn_output, n_output)       \n",
    "outputs = tf.reshape(stacked_outputs, [-1, n_windows, n_output])  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Step 3.3)** Create the loss and optimization\n",
    "\n",
    "The model optimization depends of the task you are performing. In the\n",
    "previous tutorial on CNN, your objective was to classify images, in this\n",
    "tutorial, the objective is slightly different. You are asked to make a\n",
    "prediction on a continuous variable compare to a class.\n",
    "\n",
    "This difference is important because it will change the optimization\n",
    "problem. The optimization problem for a continuous variable is to\n",
    "minimize the mean square error. To construct these metrics in TF, you\n",
    "can use:\n",
    "\n",
    "-   `tf.reduce_sum(tf.square(outputs - y))`\n",
    "\n",
    "The remaining of the code is the same as before; you use an Adam\n",
    "optimizer to reduce the loss (i.e., MSE):\n",
    "\n",
    "-   `tf.train.AdamOptimizer(learning_rate=learning_rate)`\n",
    "\n",
    "-   `optimizer.minimize(loss)`\n",
    "\n",
    "That's it, you can pack everything together, and your model is ready to\n",
    "train."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "tf.reset_default_graph()\n",
    "r_neuron = 120    \n",
    "\n",
    "## 1. Construct the tensors\n",
    "X = tf.placeholder(tf.float32, [None, n_windows, n_input])   \n",
    "y = tf.placeholder(tf.float32, [None, n_windows, n_output])\n",
    "\n",
    "## 2. create the model\n",
    "basic_cell = tf.contrib.rnn.BasicRNNCell(num_units=r_neuron, activation=tf.nn.relu)   \n",
    "rnn_output, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32)              \n",
    "\n",
    "stacked_rnn_output = tf.reshape(rnn_output, [-1, r_neuron])          \n",
    "stacked_outputs = tf.layers.dense(stacked_rnn_output, n_output)       \n",
    "outputs = tf.reshape(stacked_outputs, [-1, n_windows, n_output])   \n",
    "\n",
    "## 3. Loss + optimization\n",
    "learning_rate = 0.001  \n",
    " \n",
    "loss = tf.reduce_sum(tf.square(outputs - y))    \n",
    "optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)         \n",
    "training_op = optimizer.minimize(loss)                                          \n",
    "\n",
    "init = tf.global_variables_initializer() "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You will train the model using 1500 epochs and print the loss every 150\n",
    "iterations. Once the model is trained, you evaluate the model on the\n",
    "test set and create an object containing the predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 \tMSE: 564765.3\n",
      "150 \tMSE: 16485.17\n",
      "300 \tMSE: 4646.2925\n",
      "450 \tMSE: 3043.4048\n",
      "600 \tMSE: 2535.0784\n",
      "750 \tMSE: 2374.1853\n",
      "900 \tMSE: 2270.443\n",
      "1050 \tMSE: 2448.813\n",
      "1200 \tMSE: 2083.7778\n",
      "1350 \tMSE: 2017.2399\n"
     ]
    }
   ],
   "source": [
    "iteration = 1500 \n",
    "\n",
    "with tf.Session() as sess:\n",
    "    init.run()\n",
    "    for iters in range(iteration):\n",
    "        sess.run(training_op, feed_dict={X: X_batches, y: y_batches})\n",
    "        if iters % 150 == 0:\n",
    "            mse = loss.eval(feed_dict={X: X_batches, y: y_batches})\n",
    "            print(iters, \"\\tMSE:\", mse)\n",
    "    \n",
    "    y_pred = sess.run(outputs, feed_dict={X: X_test})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At last, you can plot the actual value of the series with the predicted\n",
    "value. If your model is corrected, the predicted values should be put on\n",
    "top of the actual values.\n",
    "\n",
    "As you can see, the model has room of improvement. It is up to you to\n",
    "change the hyperparameters like the windows, the batch size of the\n",
    "number of recurrent neurons."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEXCAYAAABPkyhHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJztnXmcHFW5sJ93epZkSBAy00RkMZPckJCwRQMoEa5RriAQV1AQUHG7EpDw8TGRmA9C8HJZhquAJnFBRRGiCG5Br3JRIIKXJShK9pAFDAQyM0HIZGYymZ73+6Oqeqp7qnrfpud98qvfpKuqq0+drj7vOe8qqophGIYxsqkpdwMMwzCM8mPCwDAMwzBhYBiGYZgwMAzDMDBhYBiGYWDCwDAMw8CEgWEYOSIid4rIA+Vuh1EYTBgYBcEdGDRgO67cbcsHEXm3ex/NZWzD7SISE5HP5/DebSJyZTHaZVQXJgyMQvIQcHDStjrXi4lIXYHaNWwRkQbgfOBG4HNlbo5RxZgwMArJXlV9JWnrB2dQE5FbReRVEekVkSdE5F3eG30z8DNE5CkR6QNOc4/NEZFn3PdtFZHrRaTe9956EflPEXlBRPaKyBYRucw9FhGR77nv6xGRTSIyX0RqfO8/WkT+ICJviEiXiPxNRGaLyATgYfe0drd9dybftIjUiMg/RORLSfuPcN/zNvf1v4vIRvc+OkTk9yJSm6ZPPwJsA64HponIUQGff4aIPOneX6eIrBCRUSLyCPBWoM1bqbnnf1pEupKukbACEpEmEVkuItvd664RkYvStNUYxqR7EA2jUNwMfAz4DLAFuAL4nYhMVtUdvvNuAv4v8DywW0ROA+4G5gErgcOBbwENgKf++CFwsnvOX3EGwMPcYzXAS+5ntwMnAN8BOoHvuefcA/zNPdYPHA30Av8APgrcD0wHdgE9yTemqgMishxnBv8N36HzgXWq+hcRmQksAT4FPAYcALwng377HPBjVe0Wkfvd15d7B0XkdODXOCuHi3B+0+9z7/sj7n19H1iWwWf5GQX8Bef7eAM4Ffi2iLyoqn/I8lrGcEBVbbMt7w24E2cg7fJt/+0e2w/oAz7pOz8CbAb+w339bkCBjyZddyVwddK+D7nXF2Cy+77Ts2jrjcBDvtdvAJ8KOddrV3Oaax7jnjfJt28T8BX3/x8BXgfGZtHOFrff3uy+fg/QATT4znkc+EmKa2wDrkza92mgK9v7BH4C3JH0nT9Q7mfPtsJspiYyCslK4Djf5um4JwF1OAMXAKoaA/4XmJZ0jVVJr98OLHTVN12ueuMeHAHzZmAGMMCgOmcIIvJFEVklIu3u+/8PzgrD42vAHSLyRxFZKCJTs7lp937+DjyHsxpARE507/tu95T/AV4AtorI3SLyKREZm+aynwX+oKqvuK8fAbpxhKHHDKDgM3VXvbZQRP7uqp66cATa4eneawxPTBgYhaRbVZ/3bS9l8J7ktLl7kl7XAItJFDLH4KwI2tNdXEQ+DtyKM4s9zX3/UiBuc1DVa3GE0i+Bk4C/i8hnMmh7Mj/GFQbu38dU9QX3M3YDb8NRV70ILADWi8hbQtodwZnBnyYi/SLSj7NKOJT8DckDOKsqP8nG+itx1HVtwHtx+u2X+PrNqC5MGBilYDPOQDbL2+EOdu8E1qZ571+AqUlCxtv6gWdxnuPZIe9/F/Ckqn5TVf+iqs/jzNgTUNVNqnq7qp6JY0vwBtw+928kg/u8B/gXEXkH8HEc4eD/jH5V/aOqLsARaPsBZ4Vc63SgCZhJoiA8C3iva9wGx0by3hRt6gtoezvQKCL7+/YluwC/C1ihqnep6rM43+ERKT7HGOaYAdkoOqq6R0SWATeJSAewFUdVMx5nlp6K64AHROQF4F4cu8RRwAmqOl9VN4rIvThqnnk4wuNQYIKq3gVsBD4tIu/HMUqfC/wr8BqAiIwGbgF+hqNfH48rQNzPfwFn9XKmiKwAelQ1wRPHd5/bReRRHAP3m9xr4n7OWThCaCWOIXo2MBZYF3Lfn8Oxufwlaf9qEdmAY4i/BsfLaIWIPI8jjATHgPxtVe127+lkEfkxjrdXh3tve4AbROTrwLHA3KTP2Qh83PX46gC+hGPD+GtIe43hTrmNFrZVx0YaYyKO98+twKvAXuAJ4F2+4+8mxICJM7j9CUdf/gaOXeHSpGvfjOM1tBdnFnupe6weZ6b/GvBP9//XANt8x+/BGTT3Ai/jeBvt77v+1cAOHPXKnWn64TPuffw8af+7cOwanTgeSauBi0KuMR7YB3wi5Ph1OJ5ONe7rDwDPuO3vwPEuGuUeeweOR1Gv83OPX+ODOAN+D/B74AJ//wMHAj8HdgM73f5dCjyS6Xdu2/DaxP1SDcMwjBGM2QwMwzAMEwaGYRhGkYWBiBwmIg+LyFo3nH2eu/9aEXlJRJ51tzOK2Q7DMAwjNUW1GYjIwcDB6oTjj8Uxcn0Ix9e6S1VvKdqHG4ZhGBlTVNdSdXLO7HD/v1tE1gGH5HKt5uZmnTBhQgFbZxiGUf0888wzHaoaTXdeyeIM3CCZGTg+zrOAS0Xkkzhugv9XVV8LeM8XgC8AHH744axalZypwDAMw0iFG6OTlpIYkEVkDE7mx8tV9Q2cDIqTcKIedwD/FfQ+Vf2Oqs5U1ZnRaFrBZhiGYeRI0YWBOAVK7gfuVtWfA6jqq6oaU9UB4Ls4qYMNwzCMMlFsbyLBifhcp6pf8+0/2Hfah8mjGpZhGIaRP8W2GcwCLgSeE5Fn3X1fAc4Tpzau4qQB+Pcit8MwDMNIQbG9iR5jaKpcgN8W83MNwzCGE119XbQ93sbSVUvp7O6kqbGJuTPn0jqrlTH1Y0rShmGTm2jmzJlq3kSGYVQbXX1dvOOOd7D5tc309vfG94+qHcWkAyfxxOeeyEsgiMgzqjoz3XmWjsIwDKNMdPV1cebdZ7KmfQ0H7+xl9RLYtxhWL4GDd/ayvmM916+8viRtMWFgGIZRBrwVwcoXVwKwYjlMbYdahWntsHYJHN4Zo+3PbXT1BZbQKCgmDAzDMMpA2+NtbH5tMy27nJXAtPbBknQCNMQcARHTGG2PtxW9PSYMDMMwysDSVUvp7e91VgQdjgDwW3AFmNLh/H/ZqmVFb48JA8MwjDLQ2d0JOAN+xJUCfoEQE9jQ7J7b01n09pgwMAzDKANNjU2AM+DHXAf8GLA3Av0C65thznnuuaObit4eEwaGYRhlYO7MuYyqHcWc85yBv19gfRSmXQJ1i+CoS2DrOMfF9OKZFxe9PRZnYBiGUQbC4gv8FCLWwOIMDMMwKpgx9WN44nNPMP+k+UQbowhCY20jjXWNCEK0Mcr8k+bnHXSWKSWrZ2AYhjHSCUs7sWXelpKlnQjD1ESGYRgloNhpJ8IwNZFhGEYF4QWZBaWd2Pza5pIElqXChIFhGEYJSA4yq1Xn74rl0NvfW5LAslSYMDAMwygyXX1ddHQ74cT+ILOIDkYZe4FlXX1dLHp4EdG2KDWLa4i2RVn08KKi5ycyYWAYhlFEPFuBR0KQmS/KuGl0U/zcm/98Mx3dHShKR3cHN//5Zt5xxzuKKhBMGBiGYRSR5IR0U9phX01ilLEXWFZOu4IJA8MwjCIyxFYA1A04KwIvynjSgZNondVaVruCCQPDMIwiEpSQzm8rAOJupenOLWbCOhMGhmEYRSQwIZ3PVhBtjMbjC9KdW8yEdWUTBiJyuohsEJHnReSqcrXDMAyjmAQmpHNtBRGJsKdvT9xraFrztNBzi52wriwRyCISATYC/wZsB54GzlPVtWHvsQhkwzDKTVg6idZZraHRw2GRx4Iz9VdfSZuGSINzTKRgUcqVHoF8AvC8qm5R1T7gJ8AHy9QWwzCMtP79ubp9Jiekq5EaGmsbidREmLBLE7yG3tK+F4AT3nJC/NxSJawr18rgbOB0Vf2c+/pC4ERVvTTpvC8AXwA4/PDD3/7CCy+UvK2GYVQ/meQNanu8jZv/fHNguulRtaOYf9J8Fs9enNHnRduidHR3sHqJ4y0UUcc2sN71MIo2RtnZurMg91bpK4OMUNXvqOpMVZ0ZjUbL3RzDMKoUz78/eaDv7R/07/fcPoPI1u0znddQR3dHyaOQyyUMXgIO870+1N1XMMoV0m0YxvDDG+i9wDBPbdOya3Cg9wbwMMLcPoPGotG1o4Fwr6EaqeHeB27kkbYO+hYrj7R1cO8DNxY1CrlcwuBpYLKItIhIPXAu8OtCXbycId2GYQw/vIE+KNgLnIHec/sMI8jtM2ws2hvbiyChHkYiwn139SW05b67+ooahVwWYaCq/cClwO+BdcC9qrqmUNev9FSxhmFUFt5AH6a2aRrdFHcRDSLM7TNsLDq8MwbAP5prOcpX83jHQc71+wf6A9tSzCjkstkMVPW3qnqEqk5S1esLee1KTxVrGEZl4Q30QWobb6BvndXKpAMnDREInpG5dVbrkOumGosUpX+gH3H/NTc2M/+k+QzoABCuQipWFHJFG5BzpZwh3ekwW4ZhVB7eQH/2hfUJapuzL6yPD/RBLqLp3D4zSUWhKA21DYzfbzyts1rjq5QgFRIULwq5KsteltJtKxvKVfbOMIz0eAFly1Ytc2wEo5viK4Jcf5dhY9HmA2FfxBEKG9yBfsdBjnsqUDAXVqgS19JcSRX+XeyQ7iC81cBBbQexpn1NSvc1wzDKw5j6MSyevZidrTuJXRNjZ+tOFs9enNcELWwsgnAVdi7qqEJQlSuDSpqBB7WlZZfz5ftnBVvHlW/FYhhGcQgbi/YtdgSBR784RuQaqSF2Taygq5QRvTLIRbdXLK5feT0bOjckPAip3NcMw6gekscij3RZSYuxSklHbdGuXGa8zsxGt5YvyUmsxo0ex66eXQmJqCC1+5phGMOLsOR1Fx9/McueXpaw/5TDT+Gpl59iznm9Q7QD5VBh+6lKNVE58JaDsc2buO+uviEqID9Bhu2Z87I3DBmGUR48AbDk6SWBK/qw7KPFyEqajkzVRFW7Mig1XnDJKjdqMKIwrR023w5rozD3DFj6W2cmsOVAx5tg4muOwPC7rxmGUV7SzfSDBMBQO+Beto4L3v9ytIET3nIC6zrWFcxrqRDYyqAAdPV1Mb5tPN393UMMQ+DM/vfVOHVPk91cG+saufKdV5b9QTAMI3yFf/YF9WwdJwkzev9A3x/y+64E93ZbGZQI7+Hp7u8GnAdnqs8mAM7/a2K4pSwG7QQRifDqla+aEDCMCiFohT+1A+77cR9HXRIuACIBv2+ozMDXMEwY5In38MQfknYnmKTGST2CELwy2NAMrSfZasAwKgFPNfTVlV9F0SGD+LT2QUHgCQm/ABBA3b8xHEGxb7HzV5J+91CZziJV6VpaSobkHsEZ9DeNc2wFXpDJaRcMBp1siAr/99LJLDxlYcprW+oKwyg+/syinuffhmZIVqB7KwJPSHgCABwBsDfi/L73RaAu5qiLG2IgWv7A10ywlUGehOUemfiaE0QC0NzoTAeOmbgrbiy6L42NIChYxUvDff+6+0sSL5FLvVfDGG74M4t6A/6WAxPPEQaNwNPaB1cE4AiEGuDFN8HpF8DGb0DE9z5w3nfUJVBbU8uUCnUWqWphUIrBrKmxiY7ujgRbgX852FjbyNZ5W7P+vKAH1PFG6GUzTuqKYrqhVoIwMoxSkLy6jyhMes2Z6Serduec53gIevjVQ5Nec1YPyQLDEyQA9TX1FfvbqVo1UakK3KTKgxQRZ35wbOtY1o+vZSBSQ2zakbBlS9rrljsNd7lqQphqzCg1Yav72tjQ3/RWV/3rRQ97gsB7z5QO57y9EZ8KyTc57I31VqQggCoWBqUazLykUjsOGpVQpGLrOIhpjO7+bn69HCa3x6gZUFi/nthZZ6Yd9Mqdhrscwsgq1BnlwEsZPSRFRJQhv2lITC29NzI0rcTWcTDtkkSbYbHTTxeCqhUGpRzM5hwxB0nQIiaSPKDLhg1pB73QB7RE3gjphFF7d3vBZ+z5CHBbURi5kmp1H8TWcYNCYtolwe/xn+MJkko1HHtUbdBZzeIaFE2bHTAfwjISeniuaNPandeem+mGqPD2yxoC7AFDc5qnOqeYNoN0NSE8IhKh9aRWFp6yMO/lb651KCopS60x/Ah7fjw1b0xjNNY2gkDPvh6aG5s5svlInnr5qcDfvSBEaiL0D/TH95XzWRzRWUshxdKvgDPrsJlsyy7nuLcq8dYMijOwnXWupl21hKmfdhxU3JzmHkGzpc0HOi5z/vuMaYxb/veWgqhxclWNed+D1YkwciEsy/HCkxey/YrtXHPKNTTWN9Kzr4emRscb8Kfn/DSw5gA4lcvqauporGtEkLJlTM6WqhUGpShwk0oVBUN9kmOSqHtMNehlm4a70GqSIGG0L+J4TNT68i6tXgKHdfQXZNDNVYB730MQVvPayISglNGts1o59Uencu8DN/JIWwd9i5VH2jq494EbOfVHp/LQJx9i/knzaR7dPOR6Pf09DOgA06LT2DJvS9HTTxeCogkDEWkTkfUi8ncR+YWIHODunyAiPSLyrLt9qxifX4qZdbqZbNig5hF2fNwoR1pkmtPcW+be+8CNPH5jB7FrlZ3zO7jw7Os4+8a35SQQPGF0+YmXA84qYFp7onATCmuHyVWAe99DGJUY+m9UPt6K8767+pjaPjgJ+uttfcQ2b2LZ08tYPHsxc493ntuWXZTU867QFHNl8D/AUap6DLARWOA7tllVj3O3Lxbjw0tR4CbdTDZoUPM/MHUxR/WSbHwSkfgAnsmM3//QTt41OFBP2gVfW/J8zg/jmPox1EfqgcHVTjKF9HDKVYB730Pyj9FT11WyB4dRuXgrzikdiUFkDTG4766++OSn3G7ghaJowkBVH1RVz4LyBHBosT4rjGJWC+rq62Ja8zQgeNCHYI8C/wMz6TXnvA3NzoC6YrkzgO3u203b420Zu1r6H1q/T5MAR7RrXg/j0lVLAYZcO8iHOt9BNxcB7v8eViwnYQa3dglMfaO+oj04jMrFW3Emp6bwgsi8yU+53cALRakikD8D/NT3ukVE/gq8Afw/Vf1TidpRELxB+vldzwODg75HQ6SB5oaxAOzqcVJQ7OnbQ3d/95AHZvIuGJDB7IgrlsNRlwzOJjKJQvY/tP7IR3X35fMw+q8d9/DBsR/UDhS+SlM2FeqSv4egGdyKe4Q331B5of9G5eNlF5hznjOxaIj5PAKbByc/6bIQDJeVaV4rAxF5SERWB2wf9J2zEOgH7nZ37QAOV9UZwBXAPSKyf8j1vyAiq0RkVXt7ez5NLSieWuYt7XsD1RInHnIiW+dtpb21Pb4i6envAYaqlCB8NpHp8tNTk8w5z0mQpzjbpnHOPs8GkQv+a8dXP1HHv7rUHk7JeN/D3theIHgGN6m9v+INd0Zl4tmwgoLIzr5wcMVZCmeVUlDUOAMR+TTw78B7VbU75JxHgCtVNWUQQSUVt8nFH957T3Llo7qYoy4KuoanGkoXK7Ho4UXc9PhN8UFxSHsbo2yZtyWnQXHRw4u4+c83h3rrlLM4j9enHi27hs7gIkdOgzVrStouozrINH6l0uNcyh5nICKnA/OBD/gFgYhERZxoDhGZCEwG0ifrqSBy0RH6Zxl+O8LpFwTbG0bXjo6/N52rZeusVsa6aqkgPBtELnhG3WR/6lG1o5genc6rV75aNLe5TFN2eATN4FixouDtMkYGmdqwSuGsUgqKtjIQkeeBBsD7xT6hql8UkY8C1wH7gAFgkaqm/cUO95VBptHK/khjLx4h6FhyFHLzzc0JQij5PZ/+zDievjE324GX/XXZqmUlq9kaWn7wwnoikybzxOeeoOW2loSVQTKlLC1oGJVK2VcGqvovqnpYsgupqt6vqtPdfW/LRBDkS6EDsnLREQbNHppHN3P4/ocDDLEL/O7Hg26SK5Y7106lo9/Vsyvh85Kvd+f3E49n018tt7UAsGXelgSvLKBo+YASfLx993HfXX1x323vewhiOOlqDaMSqNrcRB7F0OcV8preKiPZLqAMehn5Vx2CcPUpVw+ZlSfrz4PsDLUD6b/rcuhJg+pOdPd1093fHWov8ewglayrNYxKoOwrg0qhGKmsC6kj9LtuZuJlJCKBOvrkWXLy9XYdPjRkPohM8/wUql/DYim6+7sD78Ozl+SSssMYuYRpB17pesWy3bpU/cog10yYpSIXL6NMMnf6r/d8NML5nx7LXxtfT1vtLaw9ng3D+/xC9avnrRSUnXXruHBbSrm/N2P4EGp/uqCereMEEanqlaWtDFwqPTowGy+jZD24f7az/w3780rXK5zwlhNobmzmhaYa3n1lM2+5KcqMy+r4S+M/MyoW4/VXWPK9QkddBsVSeEnwNtzu2E6SBYHZA4xsCLU//biPvbG9Ja/mV6lUvTAoZCrrYhRQCXPdTBYOyUbjIPVKZ08nT738FOP3G8/rV73O3OPnsrtvd1YPu9dfYYO8P+oS8u/XIKHi5VaavMvZkgXS3v699MX6RuRS3siehBxDAc/0cM8pVCiqXhgUKjqwWCUZw7yMTjn8FJobm0P14Kl09us71nP9yutzSqDl9VfQIO/vr0L1a5BQ8fCEAiT+eBXl1idvtVKYRkaE2eW8iUulag1KTdXbDArl9ZJKt12KymPJJOjs252cPIpTk3XaJfBiU4QBHci62lsm/v2F9CZK7ld/VTivyUJwPqSXow2ceMiJrO1YS2d3J+NGj2N6dDpr2tc4OaHS2EeM6qerr4vxbePp7u8OtT9Vqj2xUJjNwKVQHieVlqY2Qb3i7osnZ1vuVCDzopizUeV4/fWxs65idmuUhmtrmN0a5WNnXVWUqEtPTfZiU4SjLoFJlw1GEG8a52z94giCuoHEft8b28vKF1cydnsHzy1RXrmqk6VfXsn+L3UWbOVmDF9e6XqFibdNZPzOblYvgY3fcPYf8aXwAvfDMadQoaj6lUEYQb7tqWaRpaipnE3bvdnO6iWJmUr9bWqsbWSAgYpZzYTR1dfF9Suvp+3PbcQ0uA/D+j0oH1FyneZRtZVzr0Zp6Orrikeoh838IxKhtqbWvIlcqn5lEIS/MlhyObuwWWQpaiqna/OihxfRfHMzY28YG/fDn3OeoxoKqi/Q099TlGpvfkO6LBYar29kv+v3y9moPqZ+DDecegP/vOqfXHPKNYFlBMP6fcXyQUEAiTpfj5FoDBzptD3eFneR9lfo8z8fDZEGtl2+zeJUXEakMMgk1YGHN/Dt6dsDlGdJ6Tdee0Ytr6rXxm/Ai28aVKf4k901NzbHS1c21jbGr1dDDXOOmJN3W7yI557+Hrr7u5mwyxGqV7/3Ol55azN71j+X1bW9Wgbt89vZvWC3U4jcbXdYAaGgojvJ5UVh5BkDRzpeUabkCn3+56M31subx7y5aAWwhhsjUk0UlgLCn+pgZ+vOtMnloDRLyiDjdX+No0NPXvr62zX/pPm0zmotaMqGoLZsOdA5NtlNf+SpazoPb+agbfnVoUj3HYQZ0LcmlXCoFmOgkZ6uvi7G3uBk8Q1K8zLpMuf5EHcaUe2OBqYmSkE6VzNvFhnmvukVsWmsayzJkjLIeN0QC176wuBA3zqrteDpOILa4sUDJLuCjnsxPKNopoQZqk85/JRB11bX4Lw2GiwIRqIxcKTiTR48kn/ja6ODz4e6/8zRwGFECoPA6l0+1YOn/0/lQQTQs6+Hr678Ki23tRQ1n0lYYFaQnUCQBAFVaC+oVEFifvxtCiPTIL6gWta/Of83gfaQIEFQjipsRnnwJj+eGnVKO+yrGfob946P9KhjPyNKGCTr/4MK1vtnkelSLpRqZhFovMZRiSTbLq4+5eoEnWeh03EEtcUrs+n//3q3hkIY+Qbx5Rqs5/98S1BWfQyZ/OCoUzc0J04WKsVFvJIYMcLAP/h49YiTiUiEfbF9fHXlV4m2RVP66Rd6ZpFqcAqM9k1Rh7irr4sFDy2g8fpG1B2mM/WCSjdIBrXFHw+wNuroZGfOG8UZp10aer+FUF8lrxja57fz6EWPJtSeDjIGFiua3Cg/6SY/HhZ1PJQRY0BOlx1TEP7lnzX84u5Y/NiHPlHD5gOVCbt0yHu8mUUhohZTGUkjEmHeifP4zabfsLFzY3xwTz6n9aRWFp6yEIDjv3s8Gzo2JJybSbW0TKKPgYIY1fPJepptjEgylRRNnu+9GImke66834E/0r0ao479mAE5iXT6f0X5xd2xhGO/vGcAgH801w5RJxVyZpHKUB3TGLc/dTsd3R1EJDJkRdKyC+oiddRH6hlTP4a2x9vY1LkJRRPOTa6WtnUc7IvtS0j4lonLbbJ6RhAaaxtprGtEkIz9tHNVX3lRpdetvC7nWX2mdpRiq5JshVJ4UuXMgsFJnGfj8lSaIzXq2M+IWRmkiyCG8CjXxtpG9qvfj86eTlQdO0Eh85kEzWa8ZqyNJtZDTve5/opnYef6GVU7igkHTOBDUz7ETY/flLKP0t1bNrPcXOtIt9zWwtjtHfzux4OurJvGOSm/M53VZxJN/vpVr2eUoykfKmmFUi2kc0VO9fufHp1elcFmtjJIIl0EcapjvbHeuF766lOuLki2Tj+pPHT8q5fkc4Jm0t61ws4NsnVs6NjALf97S1r7QqpVT7az3FyynnpRpSuWJ7qyTt6VnQEwk2jybAITc6XS8l1VA2GuyF7wYth33ljbWJWCIBtGjDBIt3yE1K6mnspgydNL6O3vTVtvIBtSpXFONn6lMwR71wo7N2jgUZTDOvrjQqIuBpsPDHe5DSLTcpkeXoK6bFJleFGlyVHHQnZqukwEUaoc+IUaqCu98NJwJcgV+cqTrkz5nV950pUjWhBAEYWBiFwrIi+JyLPudobv2AIReV5ENojIacVqgx9v8Hk52jBE/z+6djQNkYZQV9PPzvhsPJfRo7d0Dgk+a25sziv4LHlw8rtqJq9e0s2k586cS0QioedmUuBj0mtultAQl9sgvMEziKDBM5esp/5gQb9y059iwBPct959GevH19JfI6wfX8utd18WX51kIogyDUzMh3LnuxpJ5DL5GGkUe2XwdVU9zt1+CyAi04BzgenA6cBSEYmkukghGFM/hoc++RDgBDkRAAAgAElEQVT7N+w/5NiAOobi5GpjnlcMEKgyWLHcOWfuzLl55TNJlcbZG8QbIg1EG6NpH+bWWa1MbpqMIIHCLdsCH/5+SPWD8aunAo8HDJ5BM7iwfuzq64q7+s45z7ETeEJz07jB1YsnuN93xTeZ3B5zIqTbY7zvim/G1VWZCKJMAxNzoRLyXY00wuJSTnjLCbzS9Qr737D/iI81KZoBWUSuBbpU9Zak/QsAVPUG9/XvgWtV9X9TXa8QuYlSGezihVLa19LR04EgKErT6CZ69jmJ2HI1rGZCqjTO3mD80CcfYtnTy1i2ahmdPZ00jW7i4pkXDzHQete6/cnb49lNa2tqQeGwzv4hOYUmvhae60gQrj7l6rSujn7DdeDxPPrIs0fs3biOX94zMMQt2P8Zn53xWW598lZ2X9075Lsa+9XMjbLesxK02sknJXaY+67/XqothXIlUqjiTMOBSjEgXyoifxeR74uIO/RwCPAP3znb3X1DEJEviMgqEVnV3p5fwjNIbbDbG9vL2va1jB8znlG1o+LG1M6ezviAWkyVQXIa56AZa6YZFr1r7Vm4B12k6CLltS+/xpTmKQlqsn0RRyVUq46dIDlsPyiiOQxP1RVE0Cw3G7dNzx7xy3sGAt2CwVHV/f3iv3PHX++gt7838LvKRtcfVpsanOsseXpJ1rPIrr4uzrz7TNa0rwlcZULp8l2NdAqds6sayGtlICIPAW8OOLQQeALowFnJfxU4WFU/IyLfBJ5Q1R+71/ge8N+qel+qzyrEyiAT99JRtaNCA9PCyuYNl0CVV7pe4ZhlxzBme3s88MZviE3uh2xmSN6127sThXbQdbKdlaXLMttY28irra8ypn5M/DsO+64yLULkra5ue/K20Ij1bPrIu+c17WuAcBfH4fIsDXfyCXocbmS6MqjN50NU9dQMG/Nd4AH35UvAYb7Dh7r7ik5TYxMd3R1saE58APxJ3vwrh4hv1ubp3IP89IeLbnfZ08vY3bebh32BN8pgFGZysrtMo2C7+ro49Uen8sbeN4Yc82w1/uv4Z2WJA3Yvm3FmZX4VjN+YG/S99cZ649f3vuOg7woy0/X7VTlP+1Q5c8+Apb9N394g/AnUViwfjCVJ7vv27naibVGLRi4y5sk1lGJ6Ex3se/lhYLX7/18D54pIg4i0AJOBp4rVDj/pXAo91VDyAzKtfWj6ahh+GTGD3CU9gZCLasjDG+j2xvbG93nxDDu+3MGuSW9J8ObJ1r8+G2NutuqqVPeTrMr5/Y9zjwdIvuegCFgPi0YuPubJNZRi2gxuFpHnROTvwGzg/wCo6hrgXmAt8DvgEtWQwrcFJsy97OVoA2Pqx8SLXSRn5IShut3kVNHDgTB3ybXRwZVPLsLN71bqCYHNtztCtFZhSrsmePNkOyvzBvh0WWahMLr+sBiDsBoSmcwiwwILY5KYTdNSK+dHpraoXIIeq52iCQNVvVBVj1bVY1T1A6q6w3fselWdpKpTVPW/i9WGZILy6oyOjKYv1hefjUHiDBSC6+uKyLArj5duhp1rFKbfrdQ/8/X32xHtGh/Usp2VhQ3wQSsz/3ccVEu5s6eT61Zex9gbxtJ8c3PgQBEmNPdGcp9FZhIBD5ZaOVe8TL0H3HhARnmrLO5gKCMmAtnD823fMm8L06LT6BvoS5nUbW20epaR6WbYuUZh+qOe/TNfj2RvnmxnZdkGqHnf8dzjnc8JSu4HjmC4+c83c/x3j2fBQwvis0mP5PaddkHus8hMIuCT+2+k67AzxbPxBLllQ3AUfC5Bj9XOiElUl0xyzEFYSttMUj8PF9Il8Woa3cQlx1+StcHS75OfLtmePwlcsX28Az1GcFxqaweCYxWyIRdvorB7fqXrFTp7OkeEd0uhGRI/1A79Ad/xSO3DSokzqFiCDHpB6qBC5iAqN5moUHIxWPrVOP6Zr1foxq8TbxrdVLJZWaBtAkf371fDtOyCDbfDwLXOtuH2REcBj4hEiEgkq1TdHmGpv2uoYU37Gl7vfR2waORcCKpulvwdg62u0jFiVwZhMQeQOCPzp68Oi/gdjhQ6fbKXvvqWP98SD9JLJp/I3VwISw3uj61QHFtAQyzRw8czqntOBV4qjJ7+nrxdPtOt0PxUY0RsoUn1WwaL4bCVQRpS1fJNzmaYSe6c4Uah0yd7evpXW19lenR6RsbeYjN35lwaIg0Js22/EdgTDH5BAIlZUAGmRacxwADd/d1xo2Q6I3QqgqJfN9zubMl2jf6BfuYcMSfPnqhuwn7LMGivstVVekasMAgy6PnVGv9orh226qBMSOfe2d7dnlPSrkoyzLXOamVsw1i2uonsNjRDrZt2w79C8GItPPxZUBVlQ+eGwCp0kJtqLUgQT97lbMmqjf6Bfm598laLNUjBkN8yjtD3VG0fPj9S1b/lQjFi1UTp6g57NYWrYRUQRLpwfKgOFUXzzc2Bhtl9/sR8OIn66p3ktfHKaX7DcrqqcdmowNKpNcARSN6sdjg7LZSCkf5bToepidIQNoO95pRr+OdV/+SGU2+o6ocnaGW0+UA3YV0VBTzt6nGm8MkroNqYz1AbhSMvhZprne30C5yZuX8VkC4qPRvVWioVJQQHOlqsQTgj/bdcKEbsymCkEzSbqka3xkxWQMkEnQtDXWaT3ZAzTYIXZLz3pxOPaHACwUyvP9LIpvb2SMRWBkZK/LMpj2oMeMok2KuxtpHGusb466B+yCQqPdNAxKDo1ymXOVu1BToWm2xrbxvhmDAYwXgeQM2NzmhTTUm7MqlZPT06nd0LdrNn4R72fGVPyn7wvz9osE7nreLPmbP/DfvzStcrnPCWE2hubI7HHNTWOEmELdYgc6wuQeEwYWBUXdIu/2wxaFUTVrM6VT9EJJJQdtN//OwL61N6qwTNXjt7Onnq5acYv9943ljwBnsW7uG1L7/G9Oh0y5eTBYV2kR7JmDAIIJsqXNVAKZN2laJvw2aLLbtS16xO1Q9Tm6eys3Unuxfs5sIPXcPs1igN19YwuzXKx866KqXHVaaz10pyyx0uWF2CwmEG5CTCatSefWE9kUmTq/ZH6Rnh0tVXzodsqqHlQ65VrIKqmzXWNXLZCZfl5Zo4kqpqlRrr2/SYATlHwgqb3HdXX1XrID37QaGirZNXAM03NzPptkm0d7cXPWd/LrNFbxJw65O3JpS5HNABVmxcUfL2GJlRbSrOcmLCIImwwiZTOkwHmSnewHrvAzfySFsHfYuVR2/pZPzOblp2wdolg4VviqHfzaWKVTENkVZVq3hYXYLCYcIgibDCJt4P12Zx6QlbXa1Y7mz+XEDFmCHnMlsspiHSZq/Fw+wshcNsBkl4OsigOgYjOSd6Nnh9mJxuwR+s5d/nZQgtVN+mqx0QNEiEpYgoRMBXLu0xjEJhNoMcyaberhFM0OrKLwj8aRf2RgYDwArVt7nMFoupyrHZqzEcsJVBEjaLy5+g1VVyigVvReBfcW2Zt6VsfVvo+g5G8bE0FJmR6cqgaMJARH4KTHFfHgD8U1WPE5EJwDpgg3vsCVX9YrrrlTI3USncLKsZfxlMj1S5gZobm3nu4ud485g3l6nFNgkYbtj3lTllVxOp6sdV9ThVPQ64H/i57/Bm71gmgqDUFNrNcqThL4PpEZYb6JTDT2HrvK1lFQSQhypnyxaYPh1qa52/W7YUpD0jLfAxWzwnheSU1b39loYiV4quJhIRAV4E3qOqm9yVwQOqelQ217GspcOL+BL+6aV09HQMOT6cZ3B+9cQjbR2D2UxrapCpU2HNmryvPxIDH7PBU0WGHjdHjzhlXxn4OBl4VVU3+fa1iMhfReRRETk57I0i8gURWSUiq9rb28NOMyoQb3XVPr+d3Qt2c80p11SF8TQ5z5A/HkUGBtANG1JfIANGauBjNnhOCqHHzQU8a/JaGYjIQ0DQ+n6hqv7KPWcZ8Lyq/pf7ugEYo6qdIvJ24JfAdFV9I9Vn2crAqASSDc3T3DmKV9ug8/BmDtqW38QlzDV3pBd292Mrg8wpycpAVU9V1aMCNk8Q1AIfAX7qe89eVe10//8MsBk4Ip92GEapSA5O8zykFMcWcua5A3l/hgU+psdzAQ/CXMBzo9hqolOB9aq63dshIlERibj/nwhMBgpjdTOMIhOUZ8hbFRx1Cfyl8Z8J5+diCPZiHsKM7pa+IthJAQZtUZaGInuKLQzOBZYn7TsF+LuIPAvcB3xRVXcVuR1Gmag2r5hsgtNyrcJlgY/p8by/bpjwedYtjbBvMaxbGuGGCZ8flraoSqCowkBVP62q30rad7+qTnfdSt+mqvmlhDQqlmosSZiujOaRzUfG7yvX5Hc2682MMfVjuPw//8DUDnWN7Mrl//kHEwQ5YukojKJRjSUJvYH65WjDkFk7wFMvPxUXdLkmv7P0FVmwYQMMuHaagQHntZETlo7CKBrVWnikq6+LM+8+k5Uvrszo/GIkvzNcpk+H9esdQVBTAwWI86g2KinOwBihVGtRlzH1Y1jbsTb+OrlYT0uSBczqGBSRFSscARCJOH9XmNY5V0wYGEWjmou6+IOegtRAfqyOQRGZONFZCfT3O38nTix3i4YtJgyMolHNRV08QQdDVz7T2hNXCcleQdVehauYHmTV5p1WSZjNwCga1ZxZ0p+Z1W8T8ddtSM7OKgjNjc1VnQE37Dv3aBrdxCXHX5LT/Sdf258ifdv4Bg5++Gn2m3p0oW6lajCbgVF2qtkrxu/+6V/5QGJJz2ntgyuEadFpbJm3paoz4IZ5kHl2lM6ezpxdi5MzlfrVcy2v7mXP6e8p9O2MKGxlYBg50tXXxfUrr+f2J2+nu78bYMgqwb9CmDmv+ovkBHqQAfsiUDuQX8Gg5HxEQV5atQPDYzwrJbYyMIwSsGLjCgYYzEfkrRI8QQCDHlSpYguGO3vWP8fOCVF2fNkRBAl2FKAhlnmcRRjJmUrDHBOM3DBhYBg5ElRgxTMWr43mn2RuuBhLu/q62DH7eJpe7IgP+P01ifWvkwUjZO9a7Dfaw1AvrY9dOKri+mY4YcLAMHLEizAOIt8kc8MplUfb421MeHVvgkdVQww2H+jc/95IYVyLkzOVJntpbTygv+L6ZjhhwsAwciRVgZWgJHOQmLsoFcMplceSp5ewoXnQk8pjX8S5/2mXFMa12DPaR5ykx0PoH+ivuL4ZTpgwMIwcSVZbZII/d1Eq0uU1uuXPt1TEDLirr4vOns74ysdDGFQHeYLxiC85rzd+A1bd1st+23dmdQ+ed1pDpCH0nGq2yxQbEwaGkSOpCqyIqyVPTlWR6cw+XSqP7v7uilCJePexdVy4nWR0ZDQRiQwRbGdd+e2s72FM/Rh6+ntSnjNc05yUGxMGhpEjqVJN14jz08o2Y6lHUCoPxREInt9+JahElq5aGv9/mJ3k+EOOpy5SN0SwHdGuOd1DuhXZcE5zUk5MGBhGjqQKqhtQx9001yR9yak8/JHN+bhnFhq/3STMTrKmfQ29/b2BrqC53IOVvCwOJgwMIw/G1I9h8ezF7GzdSeyaGDtbd7J49uK8k/R5qw5vgI1JYdwzC026Wbog8TaGrRyyvQcr/lMcTBgYRqHYssXJr19by5olMPWN+pyT9HmrjsbaRqByM78GzdL9dpLnlmg8FUXYykFVs4qfGLN9J3/7Zow91+xl3dIIk16TqklzUk4sHYVhFApfoRWtqWHzQXUcfanklaTPS4h38M7eeFK2fFI6FJqgxHRhxYxSkVW/WEGbrLB0FIZRanwlGGVggEnt/Xkn6fNUIjsOGlWRabD9dhPPgyrMTuInyMtqTfsaxreND10leBHZ/evWJpS6VCt1WRDyXhmIyDnAtcCRwAmqusp3bAHwWZxcVZep6u/d/acDt+GkLblDVW9M9zm2MjAqniLNWLv6umh7vI1lq5bR2dNJ0+imikyDXbO4BkUzWhmkSmR39oX1RCZNThCa/hXIqtt6E66/dXwDb36ho6L6opIo5cpgNfARIKEgrIhMA84FpgOnA0tFJCIiEWAJ8H5gGnCee65hDFu6+rq49SvvZX2zuPYB4davvLcgcQBhRupKG/w8Y3KYodhPqkR2993VN8Tl1J8Hasj1P6Fld7GtBgpmMxCRR4ArvZWBuypAVW9wX/8eZwUBcK2qnhZ0Xhi2MjAqlWou4pMN/oI/Hv4CNBuaYe4ZsPS3Tp0HcDyk/InswBnk6xZBtDHKztadwND01cn4zzUSqQSbwSHAP3yvt7v7wvYPQUS+ICKrRGRVe3t70RpqGPkQlL0UHB/6ogWG+TyXmD7deV1mglw+k4Pufv9j5683+Cvhiez8Lqep8kAln2vkRkbCQEQeEpHVAdsHi9k4Vf2Oqs5U1ZnRaLSYH2UYOZMqe2nRAsPmzHHsE7GY83fOnMJ/RpZkYkxuiA2+9gr/BCWyg0S3WYs6Lj61mZykqqfmcO2XgMN8rw9195Fiv2EMO8oya/V5LjEw4LwuMZ5he+mqpXR2d9LU2MTcmXNpndXKV1d+1Wlmc6IxeV8N1A0Mvt7QPBh/4Cc5FmPuzLlDVFBh5xq5UUw10a+Bc0WkQURagMnAU8DTwGQRaRGRehwj86+L2A7DKCqlnLV67pXrmwZVKwMixI6YXLDPyLQdqeotjBvtRJQlG3tPuyC9cTkoktiijotP3sJARD4sItuBdwK/cQ3FqOoa4F5gLfA74BJVjalqP3Ap8HtgHXCve65hDEtKkSunq6+LBQ8t4IAbD+CuX15HTX+MGrfO8pYDlTPPjZU0g2k6O8n06HRG1Y4aEnW8siXx9cvRBpobm2lubE4Zi5EqD9RIMdAXG4tANow8KbY3kXf99R3riWks0I9/5ryh0cip1Dj5Dp7pvHuaRzczfsz4IX3iFaaJaSyeaqOnv6egbTMSqQRvIsMYERR71urNwg/vdATBtPahEb7Jhupil81MZyfZ1bsrsE8WnryQ7VdsZ3p0OgMM0N3fXdElPUcStjIwjArHm4X7VwSeb74/wrdGaohdEwOKn9MoH7//Ss+3VG3YysAwqoSgqmdesFaYK2a6spn5urvmYycpdtuM3DBhYBgVTlhthLXRwVTQyQNwurKZ+bq75uPdk1fbKjDYrlowYWAYFU5y1bNkt8zamtohA3C+xXXSkY+dJK+2VWCwXbVgwsAwKpywNNZbxzneOVe+88ohA3AqAVIod9cx9WNY/NZPsfPOKLHrhJ13Rln81k+lNZjn1bYKCLarVsyAbBjDgGzTWJcseV4OabuzbZvfRfaRto5BI3pNDWKFbdKSqQHZhIFhVCklqYNQW+uobDwiEejvL1jbkgWHPwvqtvENHPzw0+w39ejC3EuVYsLAMKoUbyBd8vQSOns6EQRFaRrdxCXHX8LFx1/MT371n5x+xVL+pT3G89EIv/vaXD53zn8WPqCrAAV9UgXHtT3eZm6oeWLCwDCqEG+m/Pyu59kb2zvkeEOkAYC/fKOPKe06mBAuKnxs0bTCp27YssUx4m7YAFOmwIoVMHFixm/37ie2eRP33dUXH+y9amevdL1CZ09naPU0q2OQHoszMIwqxItGDhIEAHtje9kb28sRriAAZwA9ol2LU1th4kRnJdDf7/zNQhDA4P3cd1dfQsyBV+3MczMtlousMYgJA8MYRqSqneAnyG2z4AFdBfD59+4naLDv7e+N10UolousMYgJA8MYRiTnBGrZ5RSX37fY+duyy9kfFpNQ0Jl0AXz+vfsJG+wVLbqLrOGQUXEbwzAqg6bGpoScQF5Kh4gvpYMXg5BcMAbyn0n7jb071nVQ65kcc/T59+5nznkMMRCDL/spmznqkqFuqFbHoHDYysAwhhHJOYHCdOlh7Onbw6KHF+WUGTQ5E2rybD6XAjve/STXPfBSbMw9fq7VMSgR5k1kGMOIZG+iBC8bYF8EagcGZ9dbxw29RiaBZ0HuntOap/HUy0/FbRZ+n/+NUeHBr13K5effntP9FD04bgRjrqWGUaXEB+qnlzL2pY74gNzvqy/s/arXRmHuGbD0t5n76Ie5e4YJF49c3Tz999PR0zEkbsIK3uSHCQPDGCF4tQX2LWZQh+8SVIQ+nY++V29g1W29gb79qdi9YHdOA7etEIqHxRkYxgghyCPHI6LQEMvORz+Vu2c6cq1Ulq6mcsHjI4whmDAwjGHOuNGO7sZzv1QG1UQxgb2R7Hz0E4SLu09x1FCe62oYuQ7cngAKcpW1gjelwYSBYQxjuvq6EHFGes8jZ9Jljq3A88k/7YLsfPS9egNzznMM0l6JzboBx2CcilwHbk8ABVU/A2jvbqdmcQ3RtmjO3lBGavISBiJyjoisEZEBEZnp2/9vIvKMiDzn/n2P79gjIrJBRJ51t4PyaYNhjGTaHm9j997dCfuS3TRXtiS+3nFQah/9uTPnxq9TOwCe5ilTVVEugW2eAEqlmlKUju4Obv7zzTmro4xw8l0ZrAY+AqxM2t8BzFHVo4FPAXclHT9fVY9zN8syZRg5snTV0tA8RR4RiRCRCIJk5KPfOquViESARDuE4gzQ/kjnIHIJbPPiDYIikZNVRwfvNDtCMchLGKjqOlUdEnaoqn9V1Zfdl2uA0SLSkM9nGYYxFE+9EpaWAmDhyQv551X/ZGDRADtbd7J49uKUnjlj6sfQelIrtTW1CXYIcFYJfvVNMrmmiPCqucXTTuB4QU3pgLVLhqqOzI5QeEphM/go8BdV9U9ffuCqiK4WT+EZgIh8QURWiciq9vb24rfUMIYZnnolTNcebYzGYwkWPbyIaFs0I937wlMWMqVpSlzlFJP06qJ8UkR4NZW9z9sQdWwUtTl4Qxm5kVYYiMhDIrI6YPtgBu+dDtwE/Ltv9/mu+uhkd7sw7P2q+h1VnamqM6PRaPq7MYwRhqdeCdK1e7N0z4f/3gdu5JG2DvoWK4+0dXDvAzeG6t69wdkjLJEcULAUEWPqx9Dc6FzYfz9ConeUZSwtDmmFgaqeqqpHBWy/SvU+ETkU+AXwSVXd7LveS+7f3cA9wAn53YJhjFw89crGqCQM1hujEp+lp6sZEKZ79w/O/qyhmw+Eupijklq3NELsgk0ZqZ8yIdB2gOMeaxlLi0tBIpBF5BHgSlVd5b4+AHgUWKyqP/edVwscoKodIlIHLAceUtVvpfuMoAjkffv2sX37dnp70+d3NxxGjRrFoYceSl1dXbmbYhSIrr4u7vjZV0LLXIZFKPeL42GUKo2EF43sDwbz50MaEKHmyCMLVpQ+k1QYFpWcHSVJRyEiHwa+AUSBfwLPquppIvL/gAXAJt/p7wP24Hge1QER4CHgClWNkYYgYbB161bGjh1LU1MTKUwPhouq0tnZye7du2lpaSl3c4wSUbO4BkVDS0fWSA2xa4J/gkFpIoakvYhEnEpnBcLLVbRs1TI6ujsYXTsaBHr29dDc2MzFMy+2fEVZMCJyE61bt46pU6eaIMgCVWX9+vUceeSR5W6KUSK8lYE/y6h/tu1fGQRlK/3cjM8B8L2/fo/Onk7WLBGOaB+gRhVqamDq1IKtDIzCk6kwGPbFbUwQZIf118hj7sy53Pznm9k6rndIojm/7j1oFdDR3cGtT97KpAMnsWXeFmc2fsEWp6rZhg0wZQqsWFHwNgcJpbkz59qKoIiMmHQUXX1dWbnWGUa14BmZ/UVxYKgraLpkcdevvN75Dd1/IjUfW0f0hgNZ9M2z6Tq0sEkEkovoWORxaRgRwqDYD9cvf/lLRIT169enPO/OO+/k5ZdfTnlOKh555BHOOuusnN9vjEw8N9F01cLSJYtr+3NbSQZoTygdvLPXIo9LyIgQBsVOj7t8+XLe9a53sXx56ixe+QoDw8iVMfVjWDx7MTtbdxK7JhboCpouWVxMYyVJMe0JpaB2WORx8RgRwsB7uILI9+Hq6uriscce43vf+x4/+clP4vtvuukmjj76aI499liuuuoq7rvvPlatWsX555/PcccdR09PDxMmTKCjwwmnXLVqFe9+97sBeOqpp3jnO9/JjBkzOOmkk9iQQ6Fxw8iWTJLFlSLFtCeUwtphkcfFYdgbkDPBe7hCj+fxcP3qV7/i9NNP54gjjqCpqYlnnnmGnTt38qtf/Yonn3ySxsZGdu3axbhx4/jmN7/JLbfcwsyZqQ37U6dO5U9/+hO1tbU89NBDfOUrX+H+++/PuY2GkQrPWLunbw/geBr5XVD90cbebD3im60fdUlhB+imxiY6ujtC22GRx8VhRAgD7+EKPZ7Hw7V8+XLmzZsHwLnnnsvy5ctRVS666CIaGxsBGDcuReHYAF5//XU+9alPsWnTJkSEffv25dw+w0iFP8jraTfIa8uBTpTxxNecAfjsC+uJSIyYxkJn64UcoD3vpznn9Q5xhbXI4+IxIoSB93AFqYryebh27drFH//4R5577jlEhFgshohwzjnnZPT+2tpaBgYGABKiqK+++mpmz57NL37xC7Zt2xZXHxlGofHsaavcVBURhUmvOQFpdYugsa6RK995JX2xPm598lY2NPcOma0XeoBundXK/evuZzObOeqSofWQc0mEZ6RnRNgMMnWty5b77ruPCy+8kBdeeIFt27bxj3/8g5aWFt70pjfxgx/8gO7ubsARGgBjx45l9+7BQiQTJkzgmWeeAUhQA73++usccsghgGN0Noxika7e8X51+7F49mIWnrKQSQdO4uwL6xOqpp19YX3BB+hMvZ+MwjIihEGxHq7ly5fz4Q9/OGHfRz/6UXbs2MEHPvABZs6cyXHHHcctt9wCwKc//Wm++MUvxg3IixYtYt68ecycOZNIJBK/xvz581mwYAEzZsygv4Bh/oaRTEK944CspJ4twPsNfeysq5jdGqXh2hpmt0b52FlXFWWAzsT7ySgswz4dhaVVyB7rN8Mjm1QVxvBkxKSjMAwjdzJNVWFUPyNCTWQYRjDFsqcZww8TBoYxAvFydbXc1sKa9jUIQqF2uHwAAAwESURBVGNdI4KYsXaEYmoiwxgheMFlS55eMiRIrKe/h1G1o5gWnWZCYIRiwsAwRgD+4LJHfcFlMBhcNue8Xjbj5BlaPHtxeRtslBxTExnGCCCoDvLkXc5mieAMMGGQN5FIhOOOOy6+bdu2rdxNAmDbtm3cc8895W6GUSEEBZeJu4ElgjNGmjDYsgWmT4faWufvli15X3L06NE8++yz8W3ChAkZva/YwWQmDAw/QcFl6m6QGGg2oANWAGoEMrKEwZw5sH49xGLO3zlzivIxvb29XHTRRRx99NHMmDGDhx9+GHBSS3zgAx/gPe95D+9973sBaGtr4/jjj+eYY45h0aJF8Wv86Ec/4phjjuHYY4/lwgsvBGDFihWceOKJzJgxg1NPPZVXX30VgEcffTS+MpkxYwa7d+/mqquu4k9/+hPHHXccX//614tyn8bwwUtPPec84ukkNo1zNi+1xJzzBs+36mIjEFXNeQPOAdYAA8BM3/4JQA/wrLt9y3fs7cBzwPPA7bhR0Om2t7/97ZrM2rVrh+xLSSSiCoNbJJLd+wOoqanRY489Vo899lj90Ic+pKqqt9xyi1500UWqqrpu3To97LDDtKenR3/wgx/oIYccop2dnaqq+vvf/14///nP68DAgMZiMT3zzDP10Ucf1dWrV+vkyZO1vb1dVTV+/q5du3RgYEBVVb/73e/qFVdcoaqqZ511lj722GOqqrp7927dt2+fPvzww3rmmWcGtjnrfjOGPdf88Rod9R+jlGsZsrVchq6OovsE3TDO2faJs6/lMnTUf4zSa/54TblvwcgRYJVmMMbm6020GvgI8O2AY5tV9biA/cuAzwNPAr8FTgf+O892ZMaUKc6KYGAAamqc13niqYn8PPbYY3zpS18CnNoEb33rW9m4cSMA//Zv/xZPaf3ggw/y4IMPMmPGDMAplLNp0yb+9re/cc4559Dc7KzbvfO3b9/Oxz/+cXbs2EFfXx8tLS0AzJo1iyuuuILzzz+fj3zkIxx66KF535dRXcQzgQZU/PPXKJjs5FRE8NcrcIzK5mFU3eSlJlLVdaqacRkuETkY2F9Vn3Al1o+AD+XThqxYsQKmToVIxPm7YkXJPtpjv/32i/9fVVmwYEHc3vD888/z2c9+NvS9X/rSl7j00kt57rnn+Pa3vx1Pe33VVVdxxx130NPTw6xZs9LWYjZGHl6iuctPvJyIRBIqlk1rN6OyUVybQYuI/FVEHhWRk919hwDbfedsd/cFIiJfEJFVIrKqvb09/xZNnAhr1kB/v/N34sT8rxnAySefzN133w3Axo0befHFF5kSsAo57bTT+P73v09Xl6OPfemll9i5cyfvec97+NnPfkZnp/MD9FJg+1Nb//CHP4xfZ/PmzRx99NF8+ctf5vjjj2f9+vVD0mUbxpj6MdRH6qmL1CXUF4ZBQ3KYUdmqi1U/aYWBiDwkIqsDtg+meNsO4HBVnQFcAdwjIvtn2zhV/Y6qzlTVmdFoNNu3l425c+cyMDDA0Ucfzcc//nHuvPNOGhoahpz3vve9j0984hO8853v5Oijj+bss89m9+7dTJ8+nYULF/Kv//qvHHvssVxxxRUAXHvttZxzzjm8/e1vj6uQAG699VaOOuoojjnmGOrq6nj/+9/PMcccQyQS4dhjjzUDshEnzMUUwo3KlrBuZFCQFNYi8ghwpaquSnUceAl4WFWnuvvPA96tqv+e7jMshXXhsH4budQsrkFRVi9JrC+8vpkhWUthMGGdpagYvmSawrooaiIRiYpIxP3/RGAysEVVdwBviMg7RESATwK/KkYbDMMYSpCLabJbqSWsG5nk5U0kIh8GvgFEgd+IyLOqehpwCnCdiOzDcTv9oqq6fgrMBe4ERuN4EZXGk8gwjLT1C+afNN+8hkYoeQkDVf0F8IuA/fcD9w99B7iqpKPy+dyk6+EsMoxMKIRa0Bi+hLmYWv0CY1hHII8aNYrOzk4b4DJEVens7GTUqFHpTzaqEis2b4QxrGsg79u3j+3bt8f97Y30jBo1ikMPPZS6urpyN8UwjBIwImog19XVxaNwDcMwjNwZ1moiwzAMozCYMDAMwzBMGBiGYRjDyIAsIu3ACzm+vRnoKGBzCoW1K3sqtW3Wruyo1HZB5bYt13a9VVXT5vMZNsIgH0RkVSbW9FJj7cqeSm2btSs7KrVdULltK3a7TE1kGIZhmDAwDMMwRo4w+E65GxCCtSt7KrVt1q7sqNR2QeW2rajtGhE2A8MwDCM1I2VlYBiGYaTAhIFhGIZR3cJARE4XkQ0i8ryIXFXmthwmIg+LyFoRWSMi89z914rISyLyrLudUYa2bROR59zPX+XuGyci/yMim9y/B5a4TVN8ffKsiLwhIpeXq79E5PsislNEVvv2BfaRONzuPnd/F5G3lbhdbSKy3v3sX4jIAe7+CSLS4+u7b5W4XaHfnYgscPtrg4icVuJ2/dTXpm0i8qy7v5T9FTY+lO4ZU9Wq3IAIsBmYCNQDfwOmlbE9BwNvc/8/FtgITAOuxSkZWs6+2gY0J+27GbjK/f9VwE1l/i5fAd5arv7CKdj0NmB1uj4CzsAp2iTAO4AnS9yu9wG17v9v8rVrgv+8MvRX4Hfn/g7+BjQALe7vNlKqdiUd/y/gmjL0V9j4ULJnrJpXBicAz6vqFlXtA34CfLBcjVHVHar6F/f/u4F1wCHlak8GfBD4ofv/HwIfKmNb3gtsVtVcI9DzRlVXAruSdof10QeBH6nDE8ABInJwqdqlqg+qar/78gng0GJ8drbtSsEHgZ+o6l5V3Qo8j/P7LWm73FK8HwOWF+OzU5FifCjZM1bNwuAQ4B++19upkMFXRCYAM4An3V2Xuku975daHeOiwIMi8oyIfMHdN16dmtXgzMrHl6FdHueS+AMtd395hPVRJT17nyGxtGyLiPxVRB4VkZPL0J6g765S+utk4FVV3eTbV/L+ShofSvaMVbMwqEhEZAxOSdDLVfUNYBkwCTgO2IGzTC0171LVtwHvBy4RkVP8B9VZl5bFB1lE6oEPAD9zd1VCfw2hnH0UhogsBPqBu91dO4DDVXUGcAVwj4jsX8ImVeR35+M8EicdJe+vgPEhTrGfsWoWBi8Bh/leH+ruKxsiUofzRd+tqj8HUNVXVTWmqgPAdynS8jgVqvqS+3cnTk3rE4BXvWWn+3dnqdvl8n7gL6r6qtvGsveXj7A+KvuzJyKfBs4CzncHEVw1TKf7/2dwdPNHlKpNKb67SuivWuAjwE+9faXur6DxgRI+Y9UsDJ4GJotIizu7PBf4dbka4+ojvwesU9Wv+fb79XwfBlYnv7fI7dpPRMZ6/8cxPq7G6atPuad9CvhVKdvlI2G2Vu7+SiKsj34NfNL1+HgH8LpvqV90ROR0YD7wAVXt9u2PikjE/f9EYDKwpYTtCvvufg2cKyINItLituupUrXL5VRgvapu93aUsr/CxgdK+YyVwlJerg3H4r4RR6IvLHNb3oWzxPs78Ky7nQHcBTzn7v81cHCJ2zURx5Pjb8Aar5+AJuAPwCbgIWBcGfpsP6ATeJNvX1n6C0cg7QD24ehnPxvWRzgeHkvc5+45YGaJ2/U8jj7Ze86+5Z77Ufc7fhb4CzCnxO0K/e6AhW5/bQDeX8p2ufvvBL6YdG4p+ytsfCjZM2bpKAzDMIyqVhMZhmEYGWLCwDAMwzBhYBiGYZgwMAzDMDBhYBiGYQC15W6AYVQaIuK58wG8GYgB7e7rblU9qSwNM4wiYq6lhpECEbkW6FLVW8rdFsMoJqYmMowsEJEu9++73eRlvxKRLSJyo4icLyJPiVMbYpJ7XlRE7heRp91tVnnvwDCCMWFgGLlzLPBF4EjgQuAIVT0BuAP4knvObcDXVfV4nIjWO8rRUMNIh9kMDCN3nlY3H4yIbAYedPc/B8x2/38qMM1JPQPA/iIyRlW7StpSw0iDCQPDyJ29vv8P+F4PMPjbqgHeoaq9pWyYYWSLqYkMo7g8yKDKCBE5roxtMYxQTBgYRnG5DJjpVvdai2NjMIyKw1xLDcMwDFsZGIZhGCYMDMMwDEwYGIZhGJgwMAzDMDBhYBiGYWDCwDAMw8CEgWEYhgH8f5MBlNtoUSsdAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.title(\"Forecast vs Actual\", fontsize=14)\n",
    "plt.plot(pd.Series(np.ravel(y_test)), \"bo\", markersize=8, label=\"Actual\", color='green')\n",
    "plt.plot(pd.Series(np.ravel(y_pred)), \"r.\", markersize=8, label=\"Forecast\", color='red')\n",
    "plt.legend(loc=\"lower left\")\n",
    "plt.xlabel(\"Time\")\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://github.com/thomaspernet/Tensorflow/blob/master/tensorflow/21_RNN_V3_files/image012.png)\n",
    "\n",
    "## Summary\n",
    "\n",
    "A recurrent neural network is a robust architecture to deal with time\n",
    "series or text analysis. The output of the previous state is feedback to\n",
    "preserve the memory of the network over time or sequence of words.\n",
    "\n",
    "In TensorFlow, you can use the following codes to train a recurrent\n",
    "neural network for time series:\n",
    "\n",
    "## Parameters of the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_windows = 20\n",
    "n_input = 1\n",
    "n_output = 1\n",
    "size_train = 201"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "X = tf.placeholder(tf.float32, [None, n_windows, n_input])\n",
    "y = tf.placeholder(tf.float32, [None, n_windows, n_output])\n",
    "\n",
    "basic_cell = tf.contrib.rnn.BasicRNNCell(num_units=r_neuron,\n",
    "activation=tf.nn.relu)\n",
    "rnn_output, states = tf.nn.dynamic_rnn(basic_cell, X,\n",
    "dtype=tf.float32)\n",
    "\n",
    "stacked_rnn_output = tf.reshape(rnn_output, [-1, r_neuron])\n",
    "stacked_outputs = tf.layers.dense(stacked_rnn_output, n_output)\n",
    "outputs = tf.reshape(stacked_outputs, [-1, n_windows, n_output])\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Construct the optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "learning_rate = 0.001\n",
    "\n",
    "loss = tf.reduce_sum(tf.square(outputs - y))\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)\n",
    "training_op = optimizer.minimize(loss)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "init = tf.global_variables_initializer()\n",
    "iteration = 1500 \n",
    "\n",
    "with tf.Session() as sess:\n",
    "    init.run()\n",
    "    for iters in range(iteration):\n",
    "        sess.run(training_op, feed_dict={X: X_batches, y: y_batches})\n",
    "        if iters % 150 == 0:\n",
    "            mse = loss.eval(feed_dict={X: X_batches, y: y_batches})\n",
    "            print(iters, \"tMSE:\", mse)\n",
    "    \n",
    "    y_pred = sess.run(outputs, feed_dict={X: X_test})\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}