machine_translation/mini_projet/nlp_machine_translation_eng_fr.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Natural Language Processing and Application\n",
    "#### Université Mohamed V, Faculté des Sciences de Rabat \n",
    "#### Pr. Abdelhak Mahmoudi\n",
    "## Machine Translation Mini-Project\n",
    "\n",
    "\n",
    "In this notebook, we are going to carry out our Natural Language Processing project on **machine translation**.\n",
    "We are going to build a deep neural network using the **transform** \"*Attention is all you need*\" mecanism.\n",
    "We will train a \"Neural Machine Traslation\" model which will receive a text from an original language (here in English) and translate it into another language (here in French).\n",
    "\n",
    "- **Pre-processing**  Clean and convert text to sequence of integers.\n",
    "- **Model** Create models which accepts a sequence of integers as input and returns a probability distribution over possible translations.\n",
    "- **Prediction** Run the model to predict the translation corresponding to the original text."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading the libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "import re\n",
    "from unicodedata import normalize\n",
    "import string\n",
    "\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import pickle\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import LSTM\n",
    "from tensorflow.keras.layers import GRU\n",
    "from tensorflow.keras.layers import Dense\n",
    "from tensorflow.keras.layers import Embedding\n",
    "from tensorflow.keras.layers import RepeatVector\n",
    "from tensorflow.keras.layers import TimeDistributed\n",
    "from tensorflow.keras.callbacks import ModelCheckpoint\n",
    "from tensorflow.keras import layers\n",
    "\n",
    "import tensorflow_datasets as tfds\n",
    "\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
    "\n",
    "import sklearn\n",
    "from sklearn.model_selection import train_test_split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Verify the CPU\n",
    "\n",
    "This python environment has been configured with intelpython oneAPI Deep Neural Network Library for CPU optimization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[name: \"/device:CPU:0\"\n",
      "device_type: \"CPU\"\n",
      "memory_limit: 268435456\n",
      "locality {\n",
      "}\n",
      "incarnation: 13484731816169094901\n",
      ", name: \"/device:XLA_CPU:0\"\n",
      "device_type: \"XLA_CPU\"\n",
      "memory_limit: 17179869184\n",
      "locality {\n",
      "}\n",
      "incarnation: 7202560758743711536\n",
      "physical_device_desc: \"device: XLA_CPU device\"\n",
      "]\n"
     ]
    }
   ],
   "source": [
    "\n",
    "from tensorflow.python.client import device_lib\n",
    "print(device_lib.list_local_devices())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.0.0\n"
     ]
    }
   ],
   "source": [
    "print(tf.__version__)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Setting some parameters and hyperparameters for our model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Parameters for our model\n",
    "INPUT_COLUMN = 'feature'\n",
    "TARGET_COLUMN = 'target'\n",
    "NUM_SAMPLES = 20000\n",
    "\n",
    "BATCH_SIZE = 64  # Batch size for training.\n",
    "EPOCHS = 10  # Number of epochs to train for.\n",
    "\n",
    "\n",
    "# Global parameters\n",
    "DATA_PATH = './data_eng_fr'\n",
    "train_filename='fra_eng.txt'\n",
    "checkpoint_folder = 'checkpoint'\n",
    "model_folder = 'model'\n",
    "\n",
    "# Variable for data directory\n",
    "train_filenamepath = os.path.abspath(os.path.join(DATA_PATH, train_filename))\n",
    "checkpoint_path = os.path.abspath(os.path.join(DATA_PATH, checkpoint_folder))\n",
    "model_path = os.path.abspath(os.path.join(DATA_PATH, model_folder))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The dataset and text processing\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We start by studying the dataset that will be used to train and evaluate our model.\n",
    "For this exercise, we will use sentence pairs, the source in English and the target in French, from the Tatoeba project where people help add translations every day. This is the link [www.manythings.org/anki](http://www.manythings.org/anki/) to some translations in different languages. You can download the French-English file fra_eng.zip there, it contains 190207 pairs of sentences.\n",
    "\n",
    "However, it will take a long time to train the neural network with this number of sentences.\n",
    "We will therefore use this reduced data set down to 20,000 to train the model within a somewhat reasonable time frame which will allow us to have at least some very acceptable predictions, because our model which will be built with the attention mechanism will take too much time to complete a single training epoch."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the dataset\n",
    "\n",
    "The data can be found in `data_eng_fr/fra_eng.txt`.\n",
    "The data will be loaded with a desired number of lines per pandas DataFrame and will be shuffled again.\n",
    "Then we get the **input** and **target** columns in two variables to apply the preprocessing functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                          feature  \\\n",
      "0                                             Go.   \n",
      "1                                             Go.   \n",
      "2                                             Go.   \n",
      "3                                             Hi.   \n",
      "4                                             Hi.   \n",
      "...                                           ...   \n",
      "19995                  I thought it might be you.   \n",
      "19996               We're going to be aggressive.   \n",
      "19997      I don't yet know what I'm going to do.   \n",
      "19998                                Tom's alive.   \n",
      "19999  Tom hates getting up early in the morning.   \n",
      "\n",
      "                                      target  \n",
      "0                                       Va !  \n",
      "1                                    Marche.  \n",
      "2                                    Bouge !  \n",
      "3                                    Salut !  \n",
      "4                                     Salut.  \n",
      "...                                      ...  \n",
      "19995   Je pensais que ça pouvait être vous.  \n",
      "19996            Nous allons être agressifs.  \n",
      "19997  J'ignore encore ce que je vais faire.  \n",
      "19998                        Tom est vivant.  \n",
      "19999     Tom déteste se lever tôt le matin.  \n",
      "\n",
      "[20000 rows x 2 columns]\n",
      "\n",
      "Number of sentences:  20000\n",
      "[\"Our son's name is Tom.\" \"I don't really like any of that.\"\n",
      " \"Don't be so impatient.\" 'What is the capital of Haiti?'\n",
      " 'Can you please look the other way?']\n",
      "['Le nom de notre fils est Tom.' \"Je n'apprécie vraiment rien de tout ça.\"\n",
      " 'Ne soyez pas si impatiente !' 'Quelle est la capitale de Haïti\\u202f?'\n",
      " \"Pouvez-vous regarder de l'autre côté, je vous prie ?\"]\n"
     ]
    }
   ],
   "source": [
    "# Load the dataset: sentence input and sentence output \n",
    "data_file = pd.read_csv(train_filenamepath,\n",
    "                        sep=\"\\t\",\n",
    "                        header=None,\n",
    "                        names=[INPUT_COLUMN, TARGET_COLUMN],\n",
    "                        usecols=[0,1],\n",
    "                        nrows=NUM_SAMPLES\n",
    "                       )\n",
    "\n",
    "print(data_file)\n",
    "data_file = sklearn.utils.shuffle(data_file)  # shuffle the dataset\n",
    "\n",
    "# Save the dataset selected as csv and txt file\n",
    "data_file.to_csv(DATA_PATH+'/data_file.csv', header=None, index=False)\n",
    "data_file.to_csv(DATA_PATH+'/data_file.txt', sep=\"\\t\", header=None, index=False)\n",
    "\n",
    "# load the input and target data\n",
    "input_data = data_file[INPUT_COLUMN].values\n",
    "target_data = data_file[TARGET_COLUMN].values\n",
    "\n",
    "print('\\nNumber of sentences: ', len(input_data))\n",
    "print(input_data[:5])\n",
    "print(target_data[:5])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### The following function will be used to apply data cleansing.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "# clean a list of lines\n",
    "def clean_preprocess_text(lines):\n",
    "    clean_pair = list()\n",
    "    for line in lines:\n",
    "        # normalize unicode characters\n",
    "        line = normalize('NFD', line).encode('ascii', 'ignore')\n",
    "        line = line.decode('UTF-8')\n",
    "        # convert to lowercase\n",
    "        line = line.lower()\n",
    "        # put space before and after punctuation\n",
    "        line = re.sub(r\"([?.!])\", r\" \\1 \", line)\n",
    "        # replace ' and - with a space\n",
    "        line = re.sub(r\"['-]\", r\" \", line)\n",
    "        # remove digits\n",
    "        remove_digits = str.maketrans('', '', string.digits)\n",
    "        line = line.translate(remove_digits)\n",
    "        # reduces spaces into one\n",
    "        line = re.sub(r\" +\", r\" \", line)\n",
    "        # remove space at start and end of line\n",
    "        line = line.strip()\n",
    "        # add the cleaned line in a new table which will contain all the new sentences\n",
    "        clean_pair.append(line)\n",
    "        \n",
    "    return np.array(clean_pair)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Apply cleansing on input and target data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['our son s name is tom .' 'i don t really like any of that .'\n",
      " 'don t be so impatient .' 'what is the capital of haiti ?'\n",
      " 'can you please look the other way ?']\n",
      "['le nom de notre fils est tom .'\n",
      " 'je n apprecie vraiment rien de tout ca .' 'ne soyez pas si impatiente !'\n",
      " 'quelle est la capitale de haiti ?'\n",
      " 'pouvez vous regarder de l autre cote, je vous prie ?']\n",
      "['the policeman shot into the air .' 'are you sure you don t want one ?'\n",
      " 'friends are always willing to help each other .'\n",
      " 'i like to put an ice cube into my coffee, because it s usually too hot .'\n",
      " 'you re supposed to say yes .']\n",
      "['le policier a tire en l air .'\n",
      " 'etes vous surs de ne pas en vouloir une ?'\n",
      " 'les amis sont toujours desireux de s entraider .'\n",
      " 'j aime mettre un glacon dans mon cafe, parce que c est generalement trop chaud .'\n",
      " 'tu es cense dire oui .']\n"
     ]
    }
   ],
   "source": [
    "# clean input and target data\n",
    "input_data = clean_preprocess_text(input_data)\n",
    "target_data = clean_preprocess_text(target_data)\n",
    "\n",
    "# save the data cleaned with pandas DataFrame\n",
    "data_cleaned = pd.DataFrame(columns=[INPUT_COLUMN, TARGET_COLUMN])\n",
    "data_cleaned[INPUT_COLUMN] = input_data\n",
    "data_cleaned[TARGET_COLUMN] = target_data\n",
    "\n",
    "data_cleaned.to_csv(DATA_PATH + '/data_cleaned.csv', header=None, index=False)\n",
    "data_cleaned.to_csv(DATA_PATH + '/data_cleaned.txt', sep=\"\\t\", header=None, index=False)\n",
    "\n",
    "# Show some sentences cleaned\n",
    "print(input_data[:5])\n",
    "print(target_data[:5])\n",
    "\n",
    "print(input_data[-5:])\n",
    "print(target_data[-5:])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- By looking at the sentences, you can see that they have already been preprocessed. The punctuations were delimited by spaces. All text has been converted to lowercase."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tokenize the text data\n",
    "\n",
    "For a neural network to predict on text data, it first has to be turned into data it can understand. Text data like \"cat\" is a sequence of ASCII character encodings.  Since a neural network is a series of multiplication and addition operations, the input data needs to be number(s).\n",
    "\n",
    "Next, let's see how to prepare the data for our model. It's very simple and the steps are as follows:\n",
    "\n",
    "- Create vocabulary from the corpus using the tokenization of words. The `create_tokenizer()` function creates the **tokenizer** using [**Keras**](https://keras.io/preprocessing/text/#tokenizer) to get the *token for each word* in the corpus, accepting some punctuation ('.!?').\n",
    "- Calculate the maximum length of the input and output sequences. `max_length()` function\n",
    "- Convert plain text to a sequence of integers. `encode_sequences()`\n",
    "- Filling of sentences: you must fill in zeros at the end of the sequences so that all the sequences have the same length. Otherwise, we will not be able to train the model on batches"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# fit a tokenizer\n",
    "def create_tokenizer(lines):\n",
    "    to_exclude = '\"#$%&()*+,-/:;<=>@[\\\\]^_`{|}~\\t\\n'\n",
    "    to_tokenize = '.!?'\n",
    "    tokenizer = tf.keras.preprocessing.text.Tokenizer(filters=to_exclude)\n",
    "    tokenizer.fit_on_texts(lines)\n",
    "    return tokenizer\n",
    "\n",
    "\n",
    "# max sentence length\n",
    "def max_length(lines):\n",
    "    # add 2 to max length because sos_token(start sent) and eos_token(end sent) will be added in the sentence\n",
    "    return max(len(line.split()) for line in lines) + 2\n",
    "\n",
    "\n",
    "# encode the corpus and add paddind to max length\n",
    "def encode_sequences(corpus, tokenizer, max_length, sos_token, eos_token):\n",
    "    # encode the corpus\n",
    "    sentences = tokenizer.texts_to_sequences(corpus)\n",
    "\n",
    "    # add START and END token to the corpus\n",
    "    sentences = [sos_token + sentence + eos_token for sentence in sentences]\n",
    "\n",
    "    # Pad the sentences\n",
    "    sentences = tf.keras.preprocessing.sequence.pad_sequences(sentences,\n",
    "                                                              value=0,\n",
    "                                                              padding='post',\n",
    "                                                              maxlen=max_length\n",
    "                                                              )\n",
    "    return sentences\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'to': 1, 'this': 2, 'is': 3, 'a': 4, 'sentence': 5, 'the': 6, 'laws': 7, 'were': 8, 'very': 9, 'difficult': 10, 'enforce': 11, '.': 12, 'would': 13, 'you': 14, 'care': 15, 'come': 16, '?': 17}\n",
      "\n",
      "Sequence 1\n",
      "  Input:  this is a sentence\n",
      "  Output: [2, 3, 4, 5]\n",
      "Sequence 2\n",
      "  Input:  the laws were very difficult to enforce .\n",
      "  Output: [6, 7, 8, 9, 10, 1, 11, 12]\n",
      "Sequence 3\n",
      "  Input:  would you care to come ?\n",
      "  Output: [13, 14, 15, 1, 16, 17]\n",
      "\n",
      "After padding: \n",
      "Max length:  10\n",
      "Sequence 1\n",
      "  Input:  [8 2 3 4 5 9 0 0 0 0]\n",
      "Sequence 2\n",
      "  Input:  [ 8  6  7  8  9 10  1 11 12  9]\n",
      "Sequence 3\n",
      "  Input:  [ 8 13 14 15  1 16 17  9  0  0]\n"
     ]
    }
   ],
   "source": [
    "# Tokenizer example\n",
    "expl_sentences = [\n",
    "    'this is a sentence',\n",
    "    'the laws were very difficult to enforce .',\n",
    "    'would you care to come ?'\n",
    "]\n",
    "tokenizer_expl = create_tokenizer(expl_sentences)\n",
    "print(tokenizer_expl.word_index)\n",
    "\n",
    "print()\n",
    "for sample_i, (sent, token_sent) in enumerate(zip(expl_sentences, tokenizer_expl.texts_to_sequences(expl_sentences))):\n",
    "    print('Sequence {}'.format(sample_i + 1))\n",
    "    print('  Input:  {}'.format(sent))\n",
    "    print('  Output: {}'.format(token_sent))\n",
    "\n",
    "print()\n",
    "expl_max_length = max_length(expl_sentences)\n",
    "sos_token_expl = [expl_max_length - 2]\n",
    "eos_token_expl = [expl_max_length - 1]\n",
    "expl_sentences_encoded = encode_sequences(expl_sentences, \n",
    "                                          tokenizer_expl, \n",
    "                                          expl_max_length, \n",
    "                                          sos_token_expl, \n",
    "                                          eos_token_expl\n",
    "                                         )\n",
    "print(\"After padding: \")\n",
    "print(\"Max length: \", expl_max_length)\n",
    "for sample_i, seq_pad in enumerate(expl_sentences_encoded):\n",
    "    print('Sequence {}'.format(sample_i + 1))\n",
    "    print('  Input:  {}'.format(seq_pad))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tokenize and pad the input sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input Vocabulary Size (unique words): 6217\n",
      "Input Max Length: 51\n",
      "[[6215  125  366   13  257   10    9    1 6216    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0]\n",
      " [6215    2   22    8   91   38  120   16   12    1 6216    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0]]\n",
      "[[6215    2   38    4  213   80  707 6214  197   26  396  287   11   13\n",
      "   599  101  380    1 6216    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0]\n",
      " [6215    3   32  448    4  133 1126    1 6216    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0]]\n"
     ]
    }
   ],
   "source": [
    "# prepare input tokenizer\n",
    "tokenizer_inputs = create_tokenizer(input_data)\n",
    "input_vocab_size = len(tokenizer_inputs.word_index) + 3  # (+3) to add start and end token, and the 0 index\n",
    "input_max_length = max_length(input_data)\n",
    "sos_token_input = [input_vocab_size - 2]\n",
    "eos_token_input = [input_vocab_size - 1]\n",
    "\n",
    "encoder_inputs = encode_sequences(input_data, tokenizer_inputs, input_max_length, sos_token_input, eos_token_input)\n",
    "\n",
    "print('Input Vocabulary Size (unique words): %d' % input_vocab_size)\n",
    "print('Input Max Length: %d' % (input_max_length))\n",
    "\n",
    "print(encoder_inputs[:2])\n",
    "print(encoder_inputs[-2:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Tokenize and pad the outputs sequences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Output Vocabulary Size (unique words): 9181\n",
      "Output Max Length: 63\n",
      "[[9179   12  289    4  159  353    6   15    1 9180    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0]\n",
      " [9179    2   19  354   88  110    4   46   35    1 9180    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0]]\n",
      "[[9179   16   85  482   20 9178   39   51  374  505    9   29    6 2679\n",
      "   118  436    1 9180    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0]\n",
      " [9179   14   65  890   93 1427    1 9180    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0    0    0    0    0    0    0    0\n",
      "     0    0    0    0    0    0    0]]\n"
     ]
    }
   ],
   "source": [
    "# prepare output tokenizer\n",
    "tokenizer_outputs = create_tokenizer(target_data)\n",
    "output_vocab_size = len(tokenizer_outputs.word_index) + 3  # (+3) to add start and end token, and the 0 index\n",
    "output_max_length = max_length(target_data)\n",
    "sos_token_output = [output_vocab_size - 2]\n",
    "eos_token_output = [output_vocab_size - 1]\n",
    "    \n",
    "decoder_outputs = encode_sequences(target_data, tokenizer_outputs, output_max_length, sos_token_output, eos_token_output)\n",
    "\n",
    "print('Output Vocabulary Size (unique words): %d' % output_vocab_size)\n",
    "print('Output Max Length: %d' % (output_max_length))\n",
    "\n",
    "print(decoder_outputs[:2])\n",
    "print(decoder_outputs[-2:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating training and validation dataset using *sklearn*\n",
    "split 80% training and 20% validation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "encoder_inputs, encoder_inputs_valid, decoder_outputs, decoder_outputs_valid = train_test_split(encoder_inputs, decoder_outputs, test_size=0.2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Models\n",
    "\n",
    "In this section, we will experiment with different neural network architectures.\n",
    "We will begin by training one relatively simple architecture.\n",
    "\n",
    "- **Model 1** : an Encoder-Decoder RNN\n",
    "\n",
    "After experimenting with this simple architecture, we will build a deeper architecture designed to surpass this model. Because this RNN architecture has a great limitation when working with long sequences, their ability to retain information from the first elements will be lost when new elements are incorporated into the sequence.\n",
    "\n",
    "- **Model 2** : a Transformer, encoder-decoder with Attention\n",
    "\n",
    "The **Transformer**, a model architecture eschewing recurrence and instead relying entirely on an attention mechanism to draw global dependencies between input and output. The Transformer allows for significantly more parallelization … the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### We define the functions that will be used for the prediction of model 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "i want this to be yours .\n",
      "je veux que ce soit le votre .\n",
      "(10000, 12)\n",
      "(10000, 12, 1)\n",
      "(5, 12)\n",
      "[[ 31 131   1   0   0   0   0   0   0   0   0   0]\n",
      " [  2  35  20   4  31 414   1   0   0   0   0   0]\n",
      " [  2  24 694   1   0   0   0   0   0   0   0   0]]\n",
      "[[[ 199]\n",
      "  [ 368]\n",
      "  [  24]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]]\n",
      "\n",
      " [[   2]\n",
      "  [  54]\n",
      "  [   9]\n",
      "  [  17]\n",
      "  [ 156]\n",
      "  [  12]\n",
      "  [  71]\n",
      "  [   1]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]]\n",
      "\n",
      " [[  16]\n",
      "  [  21]\n",
      "  [4168]\n",
      "  [   1]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]\n",
      "  [   0]]]\n"
     ]
    }
   ],
   "source": [
    "# map an integer to a word\n",
    "def word_for_id_RNN(integer, tokenizer):\n",
    "    for word, index in tokenizer.word_index.items():\n",
    "        if index == integer:\n",
    "            return word\n",
    "    return None\n",
    "\n",
    "\n",
    "# predict text target\n",
    "def predict_sequence_RNN(sequences, tokenizer):\n",
    "    target = list()\n",
    "    for i in sequences:\n",
    "        word = word_for_id_RNN(i, tokenizer)\n",
    "        if word is None:\n",
    "            break\n",
    "        target.append(word)\n",
    "    return ' '.join(target)\n",
    "\n",
    "\n",
    "# This function selects the lines whose length does not exceed the given limit.\n",
    "# This function will be used for the RNN model to choose only short sentences.\n",
    "# The transformer model will work with the maximum sentence length without limit.\n",
    "LIMITE = 12\n",
    "N_ROWS = 10000\n",
    "# eos_token (max in sequence) allows to detect the end of sentence\n",
    "def limit_seq_len(sequences, limite, n_rows, eos_token):\n",
    "    out = list()\n",
    "    tab_index = list()\n",
    "    j = 0\n",
    "    for i in range(n_rows):\n",
    "        while np.argmax(sequences[j]) >= limite and j < len(sequences):\n",
    "            j += 1\n",
    "        seq = sequences[j][:limite+1]\n",
    "        seq = np.delete(seq, [0])  # delete sos_token\n",
    "        seq[seq==eos_token] = 0    # delete eos_token\n",
    "        out.append(seq)\n",
    "        tab_index.append(j)\n",
    "        j += 1\n",
    "    return np.array(out), tab_index\n",
    "\n",
    "\n",
    "Y_train, y_index = limit_seq_len(decoder_outputs, LIMITE, N_ROWS, eos_token_output)\n",
    "\n",
    "# get the X inputs corresponding to the Y targets using the indexes of Y\n",
    "X_train = np.array([np.delete(y, [0]) for y in encoder_inputs[y_index][:, :LIMITE+1]])\n",
    "X_train[X_train==eos_token_input] = 0\n",
    "\n",
    "print(predict_sequence_RNN(X_train[1], tokenizer_inputs))\n",
    "print(predict_sequence_RNN(Y_train[1], tokenizer_outputs))\n",
    "\n",
    "Y_train = Y_train.reshape(Y_train.shape[0], Y_train.shape[1], 1)\n",
    "\n",
    "# We select a few sentences not to train to predict their translation\n",
    "X_eval, _ = limit_seq_len(encoder_inputs_valid, LIMITE, 5, eos_token_input)\n",
    "\n",
    "print(X_train.shape)\n",
    "print(Y_train.shape)\n",
    "print(X_eval.shape)\n",
    "print(X_train[:3])\n",
    "print(Y_train[:3])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model 1: Encoder-Decoder RNN\n",
    "\n",
    "Now let's look at the encoder-decoder models.  This model is composed of an encoder and decoder. The encoder creates a matrix representation of the sentence.  The decoder takes this matrix as input and predicts the translation as output.\n",
    "\n",
    "![enc_dec_rnn.png](images/enc_dec_rnn.png)\n",
    "\n",
    "#### Encoder\n",
    "- A stack of several recurrent units (LSTM or GRU cells for better performance) where each accepts a single element of the input sequence, collects information for that element and propagates it forward.\n",
    "\n",
    "#### Encoder Vector\n",
    "- This is the final hidden state produced from the encoder part of the model.\n",
    "\n",
    "#### Decoder\n",
    "- A stack of several recurrent units where each predicts an output y_t at a time step t.\n",
    "- Each recurrent unit accepts a hidden state from the previous unit and produces and output as well as its own hidden state.\n",
    "\n",
    "Let's create an encoder-decoder model in the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding (Embedding)        (None, 12, 256)           1591552   \n",
      "_________________________________________________________________\n",
      "lstm (LSTM)                  (None, 256)               525312    \n",
      "_________________________________________________________________\n",
      "repeat_vector (RepeatVector) (None, 12, 256)           0         \n",
      "_________________________________________________________________\n",
      "lstm_1 (LSTM)                (None, 12, 256)           525312    \n",
      "_________________________________________________________________\n",
      "time_distributed (TimeDistri (None, 12, 9181)          2359517   \n",
      "=================================================================\n",
      "Total params: 5,001,693\n",
      "Trainable params: 5,001,693\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "\n",
      "Train on 10000 samples\n",
      "Epoch 1/50\n",
      "10000/10000 [==============================] - 84s 8ms/sample - loss: 4.3513\n",
      "Epoch 2/50\n",
      "10000/10000 [==============================] - 75s 8ms/sample - loss: 3.4878\n",
      "Epoch 3/50\n",
      "10000/10000 [==============================] - 75s 8ms/sample - loss: 3.3470\n",
      "Epoch 4/50\n",
      "10000/10000 [==============================] - 75s 8ms/sample - loss: 3.2533\n",
      "Epoch 5/50\n",
      "10000/10000 [==============================] - 75s 8ms/sample - loss: 3.1895\n",
      "Epoch 6/50\n",
      "10000/10000 [==============================] - 76s 8ms/sample - loss: 3.1436\n",
      "Epoch 7/50\n",
      "10000/10000 [==============================] - 76s 8ms/sample - loss: 3.0941\n",
      "Epoch 8/50\n",
      "10000/10000 [==============================] - 76s 8ms/sample - loss: 3.0652\n",
      "Epoch 9/50\n",
      "10000/10000 [==============================] - 83s 8ms/sample - loss: 3.0260\n",
      "Epoch 10/50\n",
      "10000/10000 [==============================] - 83s 8ms/sample - loss: 2.9760\n",
      "Epoch 11/50\n",
      "10000/10000 [==============================] - 77s 8ms/sample - loss: 2.9383\n",
      "Epoch 12/50\n",
      "10000/10000 [==============================] - 75s 7ms/sample - loss: 2.9040\n",
      "Epoch 13/50\n",
      "10000/10000 [==============================] - 80s 8ms/sample - loss: 2.8449\n",
      "Epoch 14/50\n",
      "10000/10000 [==============================] - 81s 8ms/sample - loss: 2.7866\n",
      "Epoch 15/50\n",
      "10000/10000 [==============================] - 86s 9ms/sample - loss: 2.7290\n",
      "Epoch 16/50\n",
      "10000/10000 [==============================] - 93s 9ms/sample - loss: 2.6748\n",
      "Epoch 17/50\n",
      "10000/10000 [==============================] - 97s 10ms/sample - loss: 2.6299\n",
      "Epoch 18/50\n",
      "10000/10000 [==============================] - 82s 8ms/sample - loss: 2.5805\n",
      "Epoch 19/50\n",
      "10000/10000 [==============================] - 82s 8ms/sample - loss: 2.5352\n",
      "Epoch 20/50\n",
      "10000/10000 [==============================] - 89s 9ms/sample - loss: 2.4948\n",
      "Epoch 21/50\n",
      "10000/10000 [==============================] - 89s 9ms/sample - loss: 2.4566\n",
      "Epoch 22/50\n",
      "10000/10000 [==============================] - 86s 9ms/sample - loss: 2.4192\n",
      "Epoch 23/50\n",
      "10000/10000 [==============================] - 77s 8ms/sample - loss: 2.3751\n",
      "Epoch 24/50\n",
      "10000/10000 [==============================] - 77s 8ms/sample - loss: 2.3392\n",
      "Epoch 25/50\n",
      "10000/10000 [==============================] - ETA: 0s - loss: 2.309 - 78s 8ms/sample - loss: 2.3092\n",
      "Epoch 26/50\n",
      "10000/10000 [==============================] - 78s 8ms/sample - loss: 2.2665\n",
      "Epoch 27/50\n",
      "10000/10000 [==============================] - 77s 8ms/sample - loss: 2.2314\n",
      "Epoch 28/50\n",
      "10000/10000 [==============================] - 78s 8ms/sample - loss: 2.1979\n",
      "Epoch 29/50\n",
      "10000/10000 [==============================] - 78s 8ms/sample - loss: 2.1628\n",
      "Epoch 30/50\n",
      "10000/10000 [==============================] - 78s 8ms/sample - loss: 2.1274\n",
      "Epoch 31/50\n",
      "10000/10000 [==============================] - 78s 8ms/sample - loss: 2.0902\n",
      "Epoch 32/50\n",
      "10000/10000 [==============================] - 78s 8ms/sample - loss: 2.0543\n",
      "Epoch 33/50\n",
      "10000/10000 [==============================] - 82s 8ms/sample - loss: 2.0162\n",
      "Epoch 34/50\n",
      "10000/10000 [==============================] - 87s 9ms/sample - loss: 1.9848\n",
      "Epoch 35/50\n",
      "10000/10000 [==============================] - 76s 8ms/sample - loss: 1.9483\n",
      "Epoch 36/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.9074\n",
      "Epoch 37/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.8748\n",
      "Epoch 38/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.8386\n",
      "Epoch 39/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.8024\n",
      "Epoch 40/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.7633\n",
      "Epoch 41/50\n",
      "10000/10000 [==============================] - 87s 9ms/sample - loss: 1.7312\n",
      "Epoch 42/50\n",
      "10000/10000 [==============================] - 82s 8ms/sample - loss: 1.6934\n",
      "Epoch 43/50\n",
      "10000/10000 [==============================] - 77s 8ms/sample - loss: 1.6604\n",
      "Epoch 44/50\n",
      "10000/10000 [==============================] - 76s 8ms/sample - loss: 1.6164\n",
      "Epoch 45/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.5816\n",
      "Epoch 46/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.5531\n",
      "Epoch 47/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.5126\n",
      "Epoch 48/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.4759\n",
      "Epoch 49/50\n",
      "10000/10000 [==============================] - 75s 8ms/sample - loss: 1.4438\n",
      "Epoch 50/50\n",
      "10000/10000 [==============================] - 74s 7ms/sample - loss: 1.4067\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x1b4b054828>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# define NMT model\n",
    "def Encoder_decoder_model_RNN(src_vocab, tar_vocab, src_length, tar_length, n_units):\n",
    "    model = keras.models.Sequential()\n",
    "    model.add(keras.layers.Embedding(src_vocab, n_units, input_length=src_length, mask_zero=True))\n",
    "    model.add(keras.layers.LSTM(n_units))\n",
    "    model.add(keras.layers.RepeatVector(tar_length))\n",
    "    model.add(keras.layers.LSTM(n_units, return_sequences=True))\n",
    "    model.add(keras.layers.TimeDistributed(keras.layers.Dense(tar_vocab, activation='softmax')))\n",
    "    \n",
    "    model.compile(loss='sparse_categorical_crossentropy',\n",
    "                  optimizer='adam')\n",
    "    return model\n",
    "    \n",
    "\n",
    "# Train the neural network\n",
    "enc_dec_rnn_model = Encoder_decoder_model_RNN(input_vocab_size, \n",
    "                                              output_vocab_size, \n",
    "                                              LIMITE, \n",
    "                                              LIMITE,\n",
    "                                              256\n",
    "                                             )\n",
    "enc_dec_rnn_model.summary()\n",
    "print()\n",
    "\n",
    "enc_dec_rnn_model.fit(X_train, \n",
    "                      Y_train, \n",
    "                      batch_size=64, \n",
    "                      epochs=50\n",
    "                     )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Make prediction for RNN model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "in :  come alone .\n",
      "out :  venez seule !\n",
      "\n",
      "in :  i am looking for room with twin beds .\n",
      "out :  je vais dans avec maison .\n",
      "\n",
      "in :  do you speak french every day ?\n",
      "out :  veux tu une une hier ?\n",
      "\n",
      "in :  this job is kind of boring .\n",
      "out :  cette travail est est bon .\n",
      "\n",
      "in :  he enjoys his position .\n",
      "out :  il a son son . .\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Make prediction\n",
    "for seq_input in X_eval:\n",
    "    seq_input = seq_input.reshape(1, seq_input.shape[0])\n",
    "    #print(seq_input, \" --- \", seq_input.shape)\n",
    "    \n",
    "    # predict\n",
    "    prediction = enc_dec_rnn_model.predict(seq_input, verbose=0)[0]\n",
    "    #print(\"prediction: \", prediction)\n",
    "    seq_predicted = tf.squeeze([tf.cast(tf.argmax(vector), tf.int32) for vector in prediction])\n",
    "    #print(\"seq_predicted: \", seq_predicted)\n",
    "    \n",
    "    seq_input_text = predict_sequence_RNN(tf.squeeze((seq_input)), tokenizer_inputs)\n",
    "    print(\"in : \", seq_input_text)\n",
    "    prediction_text = predict_sequence_RNN(seq_predicted, tokenizer_outputs)\n",
    "    print(\"out : \", prediction_text)\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model 2: Transformer\n",
    "\n",
    "## Attention is all you need: Discovering the Transformer model\n",
    "\n",
    "### Neural machine traslation using a Transformer model\n",
    "\n",
    "This paper was a more advanced step in the use of the attention mechanism being the main basis for a model called Transformer.\n",
    "\n",
    "Attention is all you need: https://arxiv.org/pdf/1706.03762.pdf\n",
    "\n",
    "We will describe the components of this model, analyze their operation and build a simple model that we will apply to our Machine Translation problem."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![transformer.png](images/transformer.png)\n",
    "\n",
    "## Why we need Transformer?\n",
    "\n",
    "In sequence-to-sequence treatment problems such as the neural machine translation, the initial proposals were based on the use of RNNs in an encoder-decoder architecture. These architectures presented a great limitation when working with long sequences, their ability to retain information from the first elements was lost when new elements were incorporated into the sequence. In the encoder, the hidden state in every step is associated with a certain word in the input sentence, the more recent. Therefore, if the decoder only accesses the last hidden state of the decoder, it will lose relevant information about the first elements of the sequence. Then to deal with this limitation, a new concept were introduced **the attention mechanism**.\n",
    "\n",
    "Instead of paying attention to the last state of the encoder as is usually done with the RNNs, in each step of the decoder we look at all the states of the encoder, being able to access information about all the elements of the input sequence. This is what attention does, it extracts information from the whole sequence, a weighted sum of all the past encoder states. This allows the decoder to assign greater weight or importance to a certain element of the input for each element of the output. Learning in every step to focus in the right element of the input to predict the next output element.\n",
    "\n",
    "But this approach continues to have an important limitation, each sequence must be treated one at a time. Both the encoder and the decoder have to wait till the completion of t-1 steps to process at 't'th step. So when dealing with huge corpus **it is very time consuming and computationally inefficient**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create the batch data generator"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Create a batch data generator: we want to train the model on batches, group of sentences, so we need to create a Dataset using the tf.data library and the function batch_on_slices on the input and output sequences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a dataset for training\n",
    "train_dataset = tf.data.Dataset.from_tensor_slices(\n",
    "    (encoder_inputs, decoder_outputs))\n",
    "train_dataset = train_dataset.shuffle(len(encoder_inputs), reshuffle_each_iteration=True).batch(\n",
    "    BATCH_SIZE, drop_remainder=True)\n",
    "\n",
    "train_dataset = train_dataset.prefetch(tf.data.experimental.AUTOTUNE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a dataset for validation\n",
    "valid_dataset = tf.data.Dataset.from_tensor_slices(\n",
    "    (encoder_inputs_valid, decoder_outputs_valid))\n",
    "valid_dataset = valid_dataset.shuffle(len(encoder_inputs_valid), reshuffle_each_iteration=True).batch(\n",
    "    BATCH_SIZE, drop_remainder=True)\n",
    "\n",
    "valid_dataset = valid_dataset.prefetch(tf.data.experimental.AUTOTUNE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Building a Transformer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is Transformer?\n",
    "*In this work we propose the Transformer, a model architecture eschewing recurrence and instead relying entirely on an attention mechanism to draw global dependencies between input and output. The Transformer allows for significantly more parallelization … the Transformer is the first transduction model relying entirely on self-attention to compute representations of its input and output without using sequence-aligned RNNs or convolution.*\n",
    "\n",
    "\"Attention is all you need\" paper\n",
    "\n",
    "The Transformer model extract  features for each word using a self-attention mechanism to figure out how important all the other words in the sentence are w.r.t. to the aforementioned word. And no recurrent units are used to obtain this features, they are just weighted sum and activations, so they can be very parallelizable and efficient.\n",
    "\n",
    "But We will dive deeper to understand what all this means."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Self-attention: the fundamental operation\n",
    "\n",
    "*Self-attention is a sequence-to-sequence operation: a sequence of vectors goes in, and a sequence of vectors comes out. Let's call the input vectors 𝐱1,𝐱2,…𝐱t and the corresponding output vectors 𝐲1,𝐲2,…,𝐲t. The vectors all have dimension k. To produce output vector 𝐲i, the self attention operation simply takes a weighted average over all the input vectors, the simplest option is the dot product.*\n",
    "\n",
    "**Transformers from scratch by Peter Bloem**\n",
    "\n",
    "In the self-attention mechanism of our model we need to introduce three elements: **Queries**, **Values** and **Keys**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Queries, Keys and Values\n",
    "Every input vector is used in three different ways in the self-attention mechanism: the Query, the Key and the Value. In every role, it is compared to the others vectors to get its own output yi (Query), to get the j-th output yj (Key) and to compute each output vector once the weights have been established (Value).\n",
    "\n",
    "To obtain this roles, we need three weight matrices of dimensions k x k and compute three linear transformation for each xi:\n",
    "\n",
    "![query_key_value.png](images/query_key_value.png)\n",
    "![K_Q_V.png](images/K_Q_V.png)\n",
    "\n",
    "These three matrices are usually known as K, Q and V, three learnable weight layers that are applied to the same encoded input. Consequently, as each of these three matrices come from the same input, we can apply the attention mechanism of the input vector with itself, a \"self-attention\".\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scale dot-product Attention\n",
    "\n",
    "*The input consists of queries and keys of dimension dk, and values of dimension dv. We compute the dot products of the query with all keys, divide each by the square root of dk, and apply a softmax function to obtain the weights on the values.*\n",
    "\n",
    "\"Attention is all you need\" paper\n",
    "\n",
    "Then we use the Q, K and V matrices to calculate the attention scores. **The scores measure how much focus to place on other places or words of the input sequence w.r.t a word at a certain position**. That is, the dot product of the query vector with the key vector of the respective word we're scoring. So, for position 1 we calculate the dot product (.) of q1 and k1, then q1 . k2, q1 . k3,… \n",
    "\n",
    "Next we apply the \"scaled\" factor to have more stable gradients. The softmax function can not work properly with large values, resulting in vanishing the gradient and slow down the learning. After \"softmaxing\" we multiply by the Value matrix to keep the values of the words we want to focus on and minimizing or removing the values for the irrelevant words (its value in V matrix should be very small).\n",
    "\n",
    "The formula for these operations is:\n",
    "\n",
    "$Attention(Q, K, V ) = \\text{softmax}\\left(\\dfrac{QK^T}{\\sqrt{d_k}}\\right)V $\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def scaled_dot_product_attention(queries, keys, values, mask):\n",
    "    # Calculate the dot product, QK_transpose\n",
    "    product = tf.matmul(queries, keys, transpose_b=True)\n",
    "    # Get the scale factor\n",
    "    keys_dim = tf.cast(tf.shape(keys)[-1], tf.float32)\n",
    "    # Apply the scale factor to the dot product\n",
    "    scaled_product = product / tf.math.sqrt(keys_dim)\n",
    "    # Apply masking when it is requiered\n",
    "    if mask is not None:\n",
    "        scaled_product += (mask * -1e9)\n",
    "    # dot product with Values\n",
    "    attention = tf.matmul(tf.nn.softmax(scaled_product, axis=-1), values)\n",
    "    \n",
    "    return attention"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-Head Attention\n",
    "\n",
    "In the previous description the attention scores are focused on the whole sentence at a time, this would produce the same results even if two sentences contain the same words in a different order. Instead, we would like to attend to different segments of the words. We can give the self attention greater power of discrimination, **by combining several self attention heads**, dividing the words vectors into a fixed number (h, number of heads) of chunks, and then self-attention is applied on the corresponding chunks, using Q, K and V sub-matrices. \n",
    "\n",
    "This produce h different output matrices of scores.\n",
    "\n",
    "\n",
    "![Scaled_Dot_Product_Attention.png](images/dot_product_multihead.png)\n",
    "\n",
    "\n",
    "But the next layer (the Feed-Forward layer) is expecting just one matrix, a vector for each word, so after calculating the dot product of every head, we concat the output matrices and multiply them by an additional weights matrix $W_O$. This final matrix captures information from all the attention heads.\n",
    "\n",
    "$MultihHead(Q, K, V ) = \\text{Concat}(head_1,...,head_n)W^O$\n",
    "\n",
    "where $head_i=Attention(QW_i^Q,QW_i^K,QW_i^V)$ and $i$ is the head index.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "class MultiHeadAttention(layers.Layer):\n",
    "    \n",
    "    def __init__(self, n_heads):\n",
    "        super(MultiHeadAttention, self).__init__()\n",
    "        self.n_heads = n_heads\n",
    "        \n",
    "    def build(self, input_shape):\n",
    "        self.d_model = input_shape[-1]\n",
    "        assert self.d_model % self.n_heads == 0\n",
    "        # Calculate the dimension of every head or projection\n",
    "        self.d_head = self.d_model // self.n_heads\n",
    "        # Set the weight matrices for Q, K and V\n",
    "        self.query_lin = layers.Dense(units=self.d_model)\n",
    "        self.key_lin = layers.Dense(units=self.d_model)\n",
    "        self.value_lin = layers.Dense(units=self.d_model)\n",
    "        # Set the weight matrix for the output of the multi-head attention W0\n",
    "        self.final_lin = layers.Dense(units=self.d_model)\n",
    "        \n",
    "    def split_proj(self, inputs, batch_size): # inputs: (batch_size, seq_length, d_model)\n",
    "        # Set the dimension of the projections\n",
    "        shape = (batch_size,\n",
    "                 -1,\n",
    "                 self.n_heads,\n",
    "                 self.d_head)\n",
    "        # Split the input vectors\n",
    "        splited_inputs = tf.reshape(inputs, shape=shape) # (batch_size, seq_length, nb_proj, d_proj)\n",
    "        return tf.transpose(splited_inputs, perm=[0, 2, 1, 3]) # (batch_size, nb_proj, seq_length, d_proj)\n",
    "    \n",
    "    def call(self, queries, keys, values, mask):\n",
    "        # Get the batch size\n",
    "        batch_size = tf.shape(queries)[0]\n",
    "        # Set the Query, Key and Value matrices\n",
    "        queries = self.query_lin(queries)\n",
    "        keys = self.key_lin(keys)\n",
    "        values = self.value_lin(values)\n",
    "        # Split Q, K y V between the heads or projections\n",
    "        queries = self.split_proj(queries, batch_size)\n",
    "        keys = self.split_proj(keys, batch_size)\n",
    "        values = self.split_proj(values, batch_size)\n",
    "        # Apply the scaled dot product\n",
    "        attention = scaled_dot_product_attention(queries, keys, values, mask)\n",
    "        # Get the attention scores\n",
    "        attention = tf.transpose(attention, perm=[0, 2, 1, 3])\n",
    "        # Concat the h heads or projections\n",
    "        concat_attention = tf.reshape(attention,\n",
    "                                      shape=(batch_size, -1, self.d_model))\n",
    "        # Apply W0 to get the output of the multi-head attention\n",
    "        outputs = self.final_lin(concat_attention)\n",
    "        \n",
    "        return outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Positional Encoding\n",
    "\n",
    "We mentioned briefly that the order of the words in the sentence is an issue to solve in this model, because the network and the self-attention mechanism is permutation invariant. If we shuffle up the words in the input sentence, we get the same solutions. We need to create a representation of the position of the word in the sentence and add it to the word embedding.\n",
    "\n",
    "*To this end, we add \"positional encodings\" to the input embeddings at the bottoms of the encoder and decoder stacks. The positional encodings have the same dimension as the embeddings, so that the two can be summed. There are many choices of positional encodings.*\n",
    "\n",
    "\"Attention is all you need\" paper\n",
    "\n",
    "![pos_encoding.png](images/pos_encoding.png)\n",
    "\n",
    "So, we apply a function to map the position in the sentence to real valued vector. The network will learn how to use this information. Another approach would be to use a position embedding, similar to word embedding, coding every known position with a vector. It would requiere sentences of all accepted positions during training but positional encoding allow the model to extrapolate to sequence lengths longer than the ones encountered.\n",
    "\n",
    "In the paper a sinusoidal function is applied:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$PE_{(pos,2i)} =\\sin(pos/10000^{2i/dmodel})$\n",
    "\n",
    "$PE_{(pos,2i+1)} =\\cos(pos/10000^{2i/dmodel})$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "class PositionalEncoding(layers.Layer):\n",
    "\n",
    "    def __init__(self):\n",
    "        super(PositionalEncoding, self).__init__()\n",
    "    \n",
    "    def get_angles(self, pos, i, d_model): # pos: (seq_length, 1) i: (1, d_model)\n",
    "        angles = 1 / np.power(10000., (2*(i//2)) / np.float32(d_model))\n",
    "        return pos * angles # (seq_length, d_model)\n",
    "\n",
    "    def call(self, inputs):\n",
    "        # input shape batch_size, seq_length, d_model\n",
    "        seq_length = inputs.shape.as_list()[-2]\n",
    "        d_model = inputs.shape.as_list()[-1]\n",
    "        # Calculate the angles given the input\n",
    "        angles = self.get_angles(np.arange(seq_length)[:, np.newaxis],\n",
    "                                 np.arange(d_model)[np.newaxis, :],\n",
    "                                 d_model)\n",
    "        # Calculate the positional encodings\n",
    "        angles[:, 0::2] = np.sin(angles[:, 0::2])\n",
    "        angles[:, 1::2] = np.cos(angles[:, 1::2])\n",
    "        # Expand the encodings with a new dimension\n",
    "        pos_encoding = angles[np.newaxis, ...]\n",
    "        \n",
    "        return inputs + tf.cast(pos_encoding, tf.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Encoder\n",
    "\n",
    "Now that all the main pieces of the model have been described we can introduce the encoder components. \n",
    "\n",
    "- Positional encoding: Add the position encoding to the input embedding (our input words are transformed to embedding vectors). *The same weight matrix is shared between the two embedding layers (encoder and decoder) and the pre-softmax linear transformation. In the embedding layers, we multiply those weights by square root of the model dimension* [1], ${\\sqrt{d_{model}}}$.\n",
    "\n",
    "- N = 6, identical layers, containing two sub-layers: a **multi-head self-attention mechanism**, and a **fully connected feed-forward network**. This FC layer is applied to each position separately and identically and consists of two linear transformations with a ReLU activation in between. But it is applied position-wise to the input, which means that the same neural network is applied to every single \"token\" vector belonging to the sentence sequence.\n",
    "\n",
    "$$FFN(x)= max(0,xW_1+b_1)W_2+b_2$$\n",
    "\n",
    "- There is a residual connection around each sub-layer (attention and FC network) followed by a layer normalization.\n",
    "\n",
    "*Normalization and residual connections are standard tricks used to help deep neural networks train faster and more accurately. The layer normalization is applied over the embedding dimension only.*\n",
    "\n",
    "**Peter Bloem, \"Transformers from scratch\"**\n",
    "\n",
    "The next figure will show the components detailed:\n",
    "\n",
    "![encoder.PNG](images/encoder.png)\n",
    "\n",
    "Keep in mind that **only the vector from the last layer (6-th) is sent to the decoder**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "class EncoderLayer(layers.Layer):\n",
    "    \n",
    "    def __init__(self, FFN_units, n_heads, dropout_rate):\n",
    "        super(EncoderLayer, self).__init__()\n",
    "        # Hidden units of the feed forward component\n",
    "        self.FFN_units = FFN_units\n",
    "        # Set the number of projectios or heads\n",
    "        self.n_heads = n_heads\n",
    "        # Dropout rate\n",
    "        self.dropout_rate = dropout_rate\n",
    "    \n",
    "    def build(self, input_shape):\n",
    "        self.d_model = input_shape[-1]\n",
    "        # Build the multihead layer\n",
    "        self.multi_head_attention = MultiHeadAttention(self.n_heads)\n",
    "        self.dropout_1 = layers.Dropout(rate=self.dropout_rate)\n",
    "        # Layer Normalization\n",
    "        self.norm_1 = layers.LayerNormalization(epsilon=1e-6)\n",
    "        # Fully connected feed forward layer\n",
    "        self.ffn1_relu = layers.Dense(units=self.FFN_units, activation=\"relu\")\n",
    "        self.ffn2 = layers.Dense(units=self.d_model)\n",
    "        self.dropout_2 = layers.Dropout(rate=self.dropout_rate)\n",
    "        # Layer normalization\n",
    "        self.norm_2 = layers.LayerNormalization(epsilon=1e-6)\n",
    "        \n",
    "    def call(self, inputs, mask, training):\n",
    "        # Forward pass of the multi-head attention\n",
    "        attention = self.multi_head_attention(inputs,\n",
    "                                              inputs,\n",
    "                                              inputs,\n",
    "                                              mask)\n",
    "        attention = self.dropout_1(attention, training=training)\n",
    "        # Call to the residual connection and layer normalization\n",
    "        attention = self.norm_1(attention + inputs)\n",
    "        # Call to the FC layer\n",
    "        outputs = self.ffn1_relu(attention)\n",
    "        outputs = self.ffn2(outputs)\n",
    "        outputs = self.dropout_2(outputs, training=training)\n",
    "        # Call to residual connection and the layer normalization\n",
    "        outputs = self.norm_2(outputs + attention)\n",
    "        \n",
    "        return outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Encoder(layers.Layer):\n",
    "    \n",
    "    def __init__(self,\n",
    "                 n_layers,\n",
    "                 FFN_units,\n",
    "                 n_heads,\n",
    "                 dropout_rate,\n",
    "                 vocab_size,\n",
    "                 d_model,\n",
    "                 name=\"encoder\"):\n",
    "        super(Encoder, self).__init__(name=name)\n",
    "        self.n_layers = n_layers\n",
    "        self.d_model = d_model\n",
    "        # The embedding layer\n",
    "        self.embedding = layers.Embedding(vocab_size, d_model)\n",
    "        # Positional encoding layer\n",
    "        self.pos_encoding = PositionalEncoding()\n",
    "        self.dropout = layers.Dropout(rate=dropout_rate)\n",
    "        # Stack of n layers of multi-head attention and FC\n",
    "        self.enc_layers = [EncoderLayer(FFN_units,\n",
    "                                        n_heads,\n",
    "                                        dropout_rate) \n",
    "                           for _ in range(n_layers)]\n",
    "    \n",
    "    def call(self, inputs, mask, training):\n",
    "        # Get the embedding vectors\n",
    "        outputs = self.embedding(inputs)\n",
    "        # Scale the embeddings by sqrt of d_model\n",
    "        outputs *= tf.math.sqrt(tf.cast(self.d_model, tf.float32))\n",
    "        # Positional encodding\n",
    "        outputs = self.pos_encoding(outputs)\n",
    "        outputs = self.dropout(outputs, training)\n",
    "        # Call the stacked layers\n",
    "        for i in range(self.n_layers):\n",
    "            outputs = self.enc_layers[i](outputs, mask, training)\n",
    "\n",
    "        return outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The Decoder\n",
    "\n",
    "The decoder share some components with the encoder but they are used in a different way to take into account the encoder output.\n",
    "\n",
    "- Positional encoding: Similar that the one in the encoder\n",
    "- N=6 identical layers, containing 3 three sublayers. First, the Masked Multi-head attention or **masked causal attention** to prevent positions from attending to subsequent positions, hiding those features that belong to future states of the sequence. \"This masking, combined with fact that the output embeddings are offset by one position, ensures that the predictions for position i can depend only on the known outputs at positions less than i\" [1]. It is implemented setting to −∞ the values corresponding to the forbidden states in the softmax layer of the dot-product attention modules. The second component or **\"encoder-decoder attention\"** performs multi-head attention over the output of the decoder, the Key and Value vectors come from the output of the encoder but the queries come from the previous decoder layer. *This allows every position in the decoder to attend over all positions in the input sequence* [1]. And finally the fully-connected network.\n",
    "\n",
    "- The residual connection and layer normalization around each sub-layer, similar to the encoder.\n",
    "\n",
    "![decoder.png](images/decoder.png)\n",
    "\n",
    "At the end of the N stacked decoders, the **linear layer**, a fully-connected network, transforms the stacked outputs to a much larger vector, the *logits*. The **softmax layer** then turns those scores (logits) into probabilities (all positive, all add up to 1.0). The cell with the highest probability is chosen, and the word associated with it is produced as the output for this time step."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "class DecoderLayer(layers.Layer):\n",
    "    \n",
    "    def __init__(self, FFN_units, n_heads, dropout_rate):\n",
    "        super(DecoderLayer, self).__init__()\n",
    "        self.FFN_units = FFN_units\n",
    "        self.n_heads = n_heads\n",
    "        self.dropout_rate = dropout_rate\n",
    "    \n",
    "    def build(self, input_shape):\n",
    "        self.d_model = input_shape[-1]\n",
    "        \n",
    "        # Self multi head attention, causal attention\n",
    "        self.multi_head_causal_attention = MultiHeadAttention(self.n_heads)\n",
    "        self.dropout_1 = layers.Dropout(rate=self.dropout_rate)\n",
    "        self.norm_1 = layers.LayerNormalization(epsilon=1e-6)\n",
    "        \n",
    "        # Multi head attention, encoder-decoder attention \n",
    "        self.multi_head_enc_dec_attention = MultiHeadAttention(self.n_heads)\n",
    "        self.dropout_2 = layers.Dropout(rate=self.dropout_rate)\n",
    "        self.norm_2 = layers.LayerNormalization(epsilon=1e-6)\n",
    "        \n",
    "        # Feed foward\n",
    "        self.ffn1_relu = layers.Dense(units=self.FFN_units,\n",
    "                                    activation=\"relu\")\n",
    "        self.ffn2 = layers.Dense(units=self.d_model)\n",
    "        self.dropout_3 = layers.Dropout(rate=self.dropout_rate)\n",
    "        self.norm_3 = layers.LayerNormalization(epsilon=1e-6)\n",
    "        \n",
    "    def call(self, inputs, enc_outputs, mask_1, mask_2, training):\n",
    "        # Call the masked causal attention\n",
    "        attention = self.multi_head_causal_attention(inputs,\n",
    "                                                inputs,\n",
    "                                                inputs,\n",
    "                                                mask_1)\n",
    "        attention = self.dropout_1(attention, training)\n",
    "        # Residual connection and layer normalization\n",
    "        attention = self.norm_1(attention + inputs)\n",
    "        # Call the encoder-decoder attention\n",
    "        attention_2 = self.multi_head_enc_dec_attention(attention,\n",
    "                                                  enc_outputs,\n",
    "                                                  enc_outputs,\n",
    "                                                  mask_2)\n",
    "        attention_2 = self.dropout_2(attention_2, training)\n",
    "        # Residual connection and layer normalization\n",
    "        attention_2 = self.norm_2(attention_2 + attention)\n",
    "        # Call the Feed forward\n",
    "        outputs = self.ffn1_relu(attention_2)\n",
    "        outputs = self.ffn2(outputs)\n",
    "        outputs = self.dropout_3(outputs, training)\n",
    "        # Residual connection and layer normalization\n",
    "        outputs = self.norm_3(outputs + attention_2)\n",
    "        \n",
    "        return outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Decoder(layers.Layer):\n",
    "    \n",
    "    def __init__(self,\n",
    "                 n_layers,\n",
    "                 FFN_units,\n",
    "                 n_heads,\n",
    "                 dropout_rate,\n",
    "                 vocab_size,\n",
    "                 d_model,\n",
    "                 name=\"decoder\"):\n",
    "        super(Decoder, self).__init__(name=name)\n",
    "        self.d_model = d_model\n",
    "        self.n_layers = n_layers\n",
    "        # Embedding layer\n",
    "        self.embedding = layers.Embedding(vocab_size, d_model)\n",
    "        # Positional encoding layer\n",
    "        self.pos_encoding = PositionalEncoding()\n",
    "        self.dropout = layers.Dropout(rate=dropout_rate)\n",
    "        # Stacked layers of multi-head attention and feed forward\n",
    "        self.dec_layers = [DecoderLayer(FFN_units,\n",
    "                                        n_heads,\n",
    "                                        dropout_rate) \n",
    "                           for _ in range(n_layers)]\n",
    "    \n",
    "    def call(self, inputs, enc_outputs, mask_1, mask_2, training):\n",
    "        # Get the embedding vectors\n",
    "        outputs = self.embedding(inputs)\n",
    "        # Scale by sqrt of d_model\n",
    "        outputs *= tf.math.sqrt(tf.cast(self.d_model, tf.float32))\n",
    "        # Positional encodding\n",
    "        outputs = self.pos_encoding(outputs)\n",
    "        outputs = self.dropout(outputs, training)\n",
    "        # Call the stacked layers\n",
    "        for i in range(self.n_layers):\n",
    "            outputs = self.dec_layers[i](outputs,\n",
    "                                         enc_outputs,\n",
    "                                         mask_1,\n",
    "                                         mask_2,\n",
    "                                         training)\n",
    "\n",
    "        return outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Transformer model\n",
    "\n",
    "Once we have defined our components and created the encoder, the decoder and the linear-softmax final layer, we join the pieces to form our model, the Transformer.\n",
    "\n",
    "![transformer_architecture.PNG](images/transformer_architecture.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Transformer(tf.keras.Model):\n",
    "    \n",
    "    def __init__(self,\n",
    "                 vocab_size_enc,\n",
    "                 vocab_size_dec,\n",
    "                 d_model,\n",
    "                 n_layers,\n",
    "                 FFN_units,\n",
    "                 n_heads,\n",
    "                 dropout_rate,\n",
    "                 name=\"transformer\"):\n",
    "        super(Transformer, self).__init__(name=name)\n",
    "\n",
    "        # Build the encoder\n",
    "        self.encoder = Encoder(n_layers,\n",
    "                               FFN_units,\n",
    "                               n_heads,\n",
    "                               dropout_rate,\n",
    "                               vocab_size_enc,\n",
    "                               d_model)\n",
    "        # Build the decoder\n",
    "        self.decoder = Decoder(n_layers,\n",
    "                               FFN_units,\n",
    "                               n_heads,\n",
    "                               dropout_rate,\n",
    "                               vocab_size_dec,\n",
    "                               d_model)\n",
    "        # build the linear transformation and softmax function\n",
    "        self.last_linear = layers.Dense(units=vocab_size_dec, name=\"lin_ouput\")\n",
    "    \n",
    "    def create_padding_mask(self, seq): #seq: (batch_size, seq_length)\n",
    "        # Create the mask for padding\n",
    "        mask = tf.cast(tf.math.equal(seq, 0), tf.float32)\n",
    "        return mask[:, tf.newaxis, tf.newaxis, :]\n",
    "\n",
    "    def create_look_ahead_mask(self, seq):\n",
    "        # Create the mask for the causal attention\n",
    "        seq_len = tf.shape(seq)[1]\n",
    "        look_ahead_mask = 1 - tf.linalg.band_part(tf.ones((seq_len, seq_len)), -1, 0)\n",
    "        return look_ahead_mask\n",
    "    \n",
    "    def call(self, enc_inputs, dec_inputs, training):\n",
    "        # Create the padding mask for the encoder\n",
    "        enc_mask = self.create_padding_mask(enc_inputs)\n",
    "        # Create the mask for the causal attention\n",
    "        dec_mask_1 = tf.maximum(\n",
    "            self.create_padding_mask(dec_inputs),\n",
    "            self.create_look_ahead_mask(dec_inputs)\n",
    "        )\n",
    "        # Create the mask for the encoder-decoder attention\n",
    "        dec_mask_2 = self.create_padding_mask(enc_inputs)\n",
    "        # Call the encoder\n",
    "        enc_outputs = self.encoder(enc_inputs, enc_mask, training)\n",
    "        # Call the decoder\n",
    "        dec_outputs = self.decoder(dec_inputs,\n",
    "                                   enc_outputs,\n",
    "                                   dec_mask_1,\n",
    "                                   dec_mask_2,\n",
    "                                   training)\n",
    "        # Call the Linear and Softmax functions\n",
    "        outputs = self.last_linear(dec_outputs)\n",
    "\n",
    "        return outputs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is worth mentioning that we create 3 masks, each of which will allow us:\n",
    "- *Encoder mask*: It is a padding mask to discard the pad tokens from the attention calculation.\n",
    "- *Decoder mask 1*: this mask is a union of the padding mask and the look ahead mask which will help the causal attention to discard the tokens \"in the future\". We take the maximum value between the padding mask and the look ahead one.\n",
    "- *Decoder mask 2*: it is the padding mask and is applied in the encoder-decoder attention layer.\n",
    "\n",
    "As you can see then we call the encoder, the decoder and the final linear-softmax layer to get the predicted output from our Transformer model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training the Transformer model\n",
    "\n",
    "Now that we have described in detail the components in the paper we are ready to implement them and train a transformer model on a NMT problem. It is a toy problem for educational purposes.\n",
    "\n",
    "We need to create a custom loss function to mask the padding tokens and we define the Adam optimizer described in the paper, with beta1 = 0.9, beta2 = 0.98 and epsilon= 10e-9. And then we create a scheduler to vary the learning rate over the training process according to:\n",
    "\n",
    "$lrate = d_{model}^{-0.5}*min(step\\_num^{-0.5}, step\\_num*warmup\\_steps^{-1.5})$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CustomSchedule(tf.keras.optimizers.schedules.LearningRateSchedule):\n",
    "    \n",
    "    def __init__(self, d_model, warmup_steps=4000):\n",
    "        super(CustomSchedule, self).__init__()\n",
    "        \n",
    "        self.d_model = tf.cast(d_model, tf.float32)\n",
    "        self.warmup_steps = warmup_steps\n",
    "    \n",
    "    def __call__(self, step):\n",
    "        arg1 = tf.math.rsqrt(step)\n",
    "        arg2 = step * (self.warmup_steps**-1.5)\n",
    "        \n",
    "        return tf.math.rsqrt(self.d_model) * tf.math.minimum(arg1, arg2)\n",
    "\n",
    "\n",
    "def loss_function(target, pred):\n",
    "    mask = tf.math.logical_not(tf.math.equal(target, 0))\n",
    "    loss_ = loss_object(target, pred)\n",
    "    \n",
    "    mask = tf.cast(mask, dtype=loss_.dtype)\n",
    "    loss_ *= mask\n",
    "    \n",
    "    return tf.reduce_mean(loss_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And that's all, we have all the necessary elements to train our model using an usual loop for sequence-to-sequence tasks:\n",
    "- For every iteration on the batch generator that produce batch size inputs and outputs\n",
    "- Get the input sequence from 0 to length-1 and the actual outputs from 1 to length, the next word expected at every sequence step.\n",
    "- Call the transformer to get the predictions\n",
    "- Calculate the loss function between the real outputs and the predictions\n",
    "- Apply the gradients to update the weights in the model\n",
    "- Calculate the mean loss and the accuracy for the batch data\n",
    "- Show some results and save the model in every epoch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Definition of training step `train_step` and validation step `valid_step` functions which will be called in the training of the model in the `main_train()` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_step(enc_inputs, dec_inputs, dec_outputs_real):\n",
    "    with tf.GradientTape() as tape:\n",
    "        # Call the transformer and get the predicted output\n",
    "        predictions = transformer(enc_inputs, dec_inputs, True)\n",
    "        # Calculate the loss\n",
    "        loss = loss_function(dec_outputs_real, predictions)\n",
    "    # Update the weights and optimizer\n",
    "    gradients = tape.gradient(loss, transformer.trainable_variables)\n",
    "    optimizer.apply_gradients(zip(gradients, transformer.trainable_variables))\n",
    "    # Save and store the metrics\n",
    "    train_loss(loss)\n",
    "    train_accuracy(dec_outputs_real, predictions)\n",
    "\n",
    "\n",
    "def valid_step(enc_inputs, dec_inputs, dec_outputs_real):\n",
    "    # Call the transformer and get the predicted output\n",
    "    predictions = transformer(enc_inputs, dec_inputs, True)\n",
    "    t_loss = loss_function(dec_outputs_real, predictions)\n",
    "    # Set the metrics for the test\n",
    "    valid_loss(t_loss)\n",
    "    valid_accuracy(dec_outputs_real, predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def main_train(train_dataset, valid_dataset, transformer, n_epochs, print_every=50):\n",
    "  ''' Train the transformer model for n_epochs using the data generator dataset '''\n",
    "  losses = []\n",
    "  accuracies = []\n",
    "  losses_valid = []\n",
    "  accuracies_valid = []\n",
    "  start_train = time.time()\n",
    "  # In every epoch\n",
    "  for epoch in range(n_epochs):\n",
    "    print(\"Starting epoch {}\".format(epoch+1))\n",
    "    start = time.time()\n",
    "    # Reset the losss and accuracy calculations\n",
    "    train_loss.reset_states()\n",
    "    train_accuracy.reset_states()\n",
    "    valid_loss.reset_states()\n",
    "    valid_accuracy.reset_states()\n",
    "    # Get a batch of inputs and targets\n",
    "    for (batch, (enc_inputs, targets)) in enumerate(train_dataset):\n",
    "        # Set the decoder inputs\n",
    "        dec_inputs = targets[:, :-1]\n",
    "        # Set the target outputs, right shifted\n",
    "        dec_outputs_real = targets[:, 1:]\n",
    "        # call of the training function\n",
    "        train_step(enc_inputs, dec_inputs, dec_outputs_real)\n",
    "                \n",
    "        if batch % print_every == 0:\n",
    "            losses.append(train_loss.result())\n",
    "            accuracies.append(train_accuracy.result())\n",
    "            print(\"Epoch {} Batch {} Loss {:.4f} Accuracy {:.4f}\".format(\n",
    "                epoch+1, batch, train_loss.result(), train_accuracy.result()))\n",
    "    \n",
    "    # Checkpoint the model on every epoch        \n",
    "    ckpt_save_path = ckpt_manager.save()\n",
    "    print(\"Saving checkpoint for epoch {} in {}\".format(epoch+1, ckpt_save_path))\n",
    "    \n",
    "    for (batch, (enc_inputs, targets)) in enumerate(valid_dataset):\n",
    "        # Set the decoder inputs\n",
    "        dec_inputs = targets[:, :-1]\n",
    "        # Set the target outputs, right shifted\n",
    "        dec_outputs_real = targets[:, 1:]\n",
    "        # call of the validation function\n",
    "        valid_step(enc_inputs, dec_inputs, dec_outputs_real)\n",
    "\n",
    "        losses_valid.append(valid_loss.result())\n",
    "        accuracies_valid.append(valid_accuracy.result())\n",
    "        \n",
    "    print(\"Epoch {}, Validation Loss {:.4f}, Validation Accuracy {:.4f}\".format(\n",
    "        epoch+1, valid_loss.result(), valid_accuracy.result()))\n",
    "            \n",
    "    print(\"Time for this epoch: {} secs\\n\".format(time.time() - start))\n",
    "  \n",
    "  train_time = time.time() - start_train\n",
    "  s = int(train_time % 60)\n",
    "  m = int(train_time // 60) % 60\n",
    "  h = int(train_time // 60) // 60\n",
    "  print(\"Total training time :  {}:{}:{}\".format(h, m, s))\n",
    "\n",
    "  return losses, accuracies, losses_valid, accuracies_valid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Setting the hyperparameters and parameters of the model and training process:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set hyperparamters for the model\n",
    "D_MODEL = 512\n",
    "N_LAYERS = 4\n",
    "FFN_UNITS = 512\n",
    "N_HEADS = 8\n",
    "DROPOUT_RATE = 0.1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we define and create all the elements to train the model and evaluate it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Clean the session\n",
    "tf.keras.backend.clear_session()\n",
    "# Create the Transformer model\n",
    "transformer = Transformer(vocab_size_enc=input_vocab_size,\n",
    "                          vocab_size_dec=output_vocab_size,\n",
    "                          d_model=D_MODEL,\n",
    "                          n_layers=N_LAYERS,\n",
    "                          FFN_units=FFN_UNITS,\n",
    "                          n_heads=N_HEADS,\n",
    "                          dropout_rate=DROPOUT_RATE\n",
    "                         )\n",
    "\n",
    "# Define a categorical cross entropy loss\n",
    "loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True, reduction=\"none\")\n",
    "\n",
    "# Define a metric to store the mean loss of every epoch\n",
    "train_loss = tf.keras.metrics.Mean(name=\"train_loss\")\n",
    "valid_loss = tf.keras.metrics.Mean(name='valid_loss')\n",
    "# Define a matric to save the accuracy in every epoch\n",
    "train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name=\"train_accuracy\")\n",
    "valid_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='valid_accuracy')\n",
    "\n",
    "# Create the scheduler for learning rate decay\n",
    "leaning_rate = CustomSchedule(D_MODEL)\n",
    "# Create the Adam optimizer\n",
    "optimizer = tf.keras.optimizers.Adam(leaning_rate,\n",
    "                                     beta_1=0.9,\n",
    "                                     beta_2=0.98,\n",
    "                                     epsilon=1e-9\n",
    "                                    )\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is very useful to checkpoint and save our model during training. Training can take a lot of time and we can restore the model for future training or use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Last checkpoint restored.\n"
     ]
    }
   ],
   "source": [
    "# Create the Checkpoint \n",
    "ckpt = tf.train.Checkpoint(transformer=transformer,\n",
    "                           optimizer=optimizer)\n",
    "\n",
    "ckpt_manager = tf.train.CheckpointManager(ckpt, checkpoint_path, max_to_keep=5)\n",
    "\n",
    "if ckpt_manager.latest_checkpoint:\n",
    "    ckpt.restore(ckpt_manager.latest_checkpoint)\n",
    "    print(\"Last checkpoint restored.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Model training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Starting epoch 1\n",
      "Epoch 1 Batch 0 Loss 0.6600 Accuracy 0.0587\n",
      "Epoch 1 Batch 20 Loss 0.5620 Accuracy 0.0617\n",
      "Epoch 1 Batch 40 Loss 0.5399 Accuracy 0.0637\n",
      "Epoch 1 Batch 60 Loss 0.5242 Accuracy 0.0653\n",
      "Epoch 1 Batch 80 Loss 0.5116 Accuracy 0.0664\n",
      "Epoch 1 Batch 100 Loss 0.5018 Accuracy 0.0670\n",
      "Epoch 1 Batch 120 Loss 0.4916 Accuracy 0.0676\n",
      "Epoch 1 Batch 140 Loss 0.4830 Accuracy 0.0682\n",
      "Epoch 1 Batch 160 Loss 0.4748 Accuracy 0.0687\n",
      "Epoch 1 Batch 180 Loss 0.4679 Accuracy 0.0692\n",
      "Epoch 1 Batch 200 Loss 0.4627 Accuracy 0.0696\n",
      "Epoch 1 Batch 220 Loss 0.4573 Accuracy 0.0699\n",
      "Epoch 1 Batch 240 Loss 0.4532 Accuracy 0.0703\n",
      "Saving checkpoint for epoch 1 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-11\n",
      "Epoch 1, Validation Loss 0.3936, Validation Accuracy 0.0755\n",
      "Time for this epoch: 2553.667550086975 secs\n",
      "\n",
      "Starting epoch 2\n",
      "Epoch 2 Batch 0 Loss 0.3121 Accuracy 0.0781\n",
      "Epoch 2 Batch 20 Loss 0.3456 Accuracy 0.0788\n",
      "Epoch 2 Batch 40 Loss 0.3443 Accuracy 0.0802\n",
      "Epoch 2 Batch 60 Loss 0.3378 Accuracy 0.0808\n",
      "Epoch 2 Batch 80 Loss 0.3360 Accuracy 0.0806\n",
      "Epoch 2 Batch 100 Loss 0.3354 Accuracy 0.0811\n",
      "Epoch 2 Batch 120 Loss 0.3340 Accuracy 0.0813\n",
      "Epoch 2 Batch 140 Loss 0.3329 Accuracy 0.0813\n",
      "Epoch 2 Batch 160 Loss 0.3318 Accuracy 0.0815\n",
      "Epoch 2 Batch 180 Loss 0.3309 Accuracy 0.0814\n",
      "Epoch 2 Batch 200 Loss 0.3313 Accuracy 0.0813\n",
      "Epoch 2 Batch 220 Loss 0.3308 Accuracy 0.0815\n",
      "Epoch 2 Batch 240 Loss 0.3307 Accuracy 0.0816\n",
      "Saving checkpoint for epoch 2 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-12\n",
      "Epoch 2, Validation Loss 0.3604, Validation Accuracy 0.0790\n",
      "Time for this epoch: 2502.8262462615967 secs\n",
      "\n",
      "Starting epoch 3\n",
      "Epoch 3 Batch 0 Loss 0.2682 Accuracy 0.0809\n",
      "Epoch 3 Batch 20 Loss 0.2743 Accuracy 0.0888\n",
      "Epoch 3 Batch 40 Loss 0.2740 Accuracy 0.0889\n",
      "Epoch 3 Batch 60 Loss 0.2715 Accuracy 0.0890\n",
      "Epoch 3 Batch 80 Loss 0.2702 Accuracy 0.0884\n",
      "Epoch 3 Batch 100 Loss 0.2690 Accuracy 0.0886\n",
      "Epoch 3 Batch 120 Loss 0.2671 Accuracy 0.0884\n",
      "Epoch 3 Batch 140 Loss 0.2683 Accuracy 0.0884\n",
      "Epoch 3 Batch 160 Loss 0.2692 Accuracy 0.0882\n",
      "Epoch 3 Batch 180 Loss 0.2688 Accuracy 0.0883\n",
      "Epoch 3 Batch 200 Loss 0.2696 Accuracy 0.0883\n",
      "Epoch 3 Batch 220 Loss 0.2699 Accuracy 0.0882\n",
      "Epoch 3 Batch 240 Loss 0.2717 Accuracy 0.0881\n",
      "Saving checkpoint for epoch 3 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-13\n",
      "Epoch 3, Validation Loss 0.3403, Validation Accuracy 0.0824\n",
      "Time for this epoch: 2466.738482952118 secs\n",
      "\n",
      "Starting epoch 4\n",
      "Epoch 4 Batch 0 Loss 0.2180 Accuracy 0.0922\n",
      "Epoch 4 Batch 20 Loss 0.2178 Accuracy 0.0948\n",
      "Epoch 4 Batch 40 Loss 0.2188 Accuracy 0.0951\n",
      "Epoch 4 Batch 60 Loss 0.2210 Accuracy 0.0952\n",
      "Epoch 4 Batch 80 Loss 0.2243 Accuracy 0.0949\n",
      "Epoch 4 Batch 100 Loss 0.2262 Accuracy 0.0945\n",
      "Epoch 4 Batch 120 Loss 0.2278 Accuracy 0.0941\n",
      "Epoch 4 Batch 140 Loss 0.2272 Accuracy 0.0938\n",
      "Epoch 4 Batch 160 Loss 0.2284 Accuracy 0.0934\n",
      "Epoch 4 Batch 180 Loss 0.2305 Accuracy 0.0931\n",
      "Epoch 4 Batch 200 Loss 0.2328 Accuracy 0.0929\n",
      "Epoch 4 Batch 220 Loss 0.2354 Accuracy 0.0926\n",
      "Epoch 4 Batch 240 Loss 0.2354 Accuracy 0.0925\n",
      "Saving checkpoint for epoch 4 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-14\n",
      "Epoch 4, Validation Loss 0.3345, Validation Accuracy 0.0836\n",
      "Time for this epoch: 2467.786703824997 secs\n",
      "\n",
      "Starting epoch 5\n",
      "Epoch 5 Batch 0 Loss 0.2012 Accuracy 0.0917\n",
      "Epoch 5 Batch 20 Loss 0.1895 Accuracy 0.0952\n",
      "Epoch 5 Batch 40 Loss 0.1899 Accuracy 0.0968\n",
      "Epoch 5 Batch 60 Loss 0.1918 Accuracy 0.0971\n",
      "Epoch 5 Batch 80 Loss 0.1950 Accuracy 0.0967\n",
      "Epoch 5 Batch 100 Loss 0.1960 Accuracy 0.0962\n",
      "Epoch 5 Batch 120 Loss 0.1987 Accuracy 0.0960\n",
      "Epoch 5 Batch 140 Loss 0.2010 Accuracy 0.0960\n",
      "Epoch 5 Batch 160 Loss 0.2031 Accuracy 0.0959\n",
      "Epoch 5 Batch 180 Loss 0.2041 Accuracy 0.0957\n",
      "Epoch 5 Batch 200 Loss 0.2055 Accuracy 0.0955\n",
      "Epoch 5 Batch 220 Loss 0.2080 Accuracy 0.0954\n",
      "Epoch 5 Batch 240 Loss 0.2103 Accuracy 0.0952\n",
      "Saving checkpoint for epoch 5 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-15\n",
      "Epoch 5, Validation Loss 0.3374, Validation Accuracy 0.0820\n",
      "Time for this epoch: 2461.9750628471375 secs\n",
      "\n",
      "Starting epoch 6\n",
      "Epoch 6 Batch 0 Loss 0.1904 Accuracy 0.1023\n",
      "Epoch 6 Batch 20 Loss 0.1707 Accuracy 0.1004\n",
      "Epoch 6 Batch 40 Loss 0.1741 Accuracy 0.0991\n",
      "Epoch 6 Batch 60 Loss 0.1768 Accuracy 0.0989\n",
      "Epoch 6 Batch 80 Loss 0.1782 Accuracy 0.0987\n",
      "Epoch 6 Batch 100 Loss 0.1818 Accuracy 0.0986\n",
      "Epoch 6 Batch 120 Loss 0.1837 Accuracy 0.0988\n",
      "Epoch 6 Batch 140 Loss 0.1861 Accuracy 0.0984\n",
      "Epoch 6 Batch 160 Loss 0.1880 Accuracy 0.0980\n",
      "Epoch 6 Batch 180 Loss 0.1894 Accuracy 0.0980\n",
      "Epoch 6 Batch 200 Loss 0.1921 Accuracy 0.0977\n",
      "Epoch 6 Batch 220 Loss 0.1926 Accuracy 0.0975\n",
      "Epoch 6 Batch 240 Loss 0.1941 Accuracy 0.0973\n",
      "Saving checkpoint for epoch 6 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-16\n",
      "Epoch 6, Validation Loss 0.3327, Validation Accuracy 0.0840\n",
      "Time for this epoch: 2462.8084461688995 secs\n",
      "\n",
      "Starting epoch 7\n",
      "Epoch 7 Batch 0 Loss 0.1521 Accuracy 0.0985\n",
      "Epoch 7 Batch 20 Loss 0.1563 Accuracy 0.1017\n",
      "Epoch 7 Batch 40 Loss 0.1639 Accuracy 0.1016\n",
      "Epoch 7 Batch 60 Loss 0.1636 Accuracy 0.1015\n",
      "Epoch 7 Batch 80 Loss 0.1638 Accuracy 0.1012\n",
      "Epoch 7 Batch 100 Loss 0.1655 Accuracy 0.1005\n",
      "Epoch 7 Batch 120 Loss 0.1686 Accuracy 0.1003\n",
      "Epoch 7 Batch 140 Loss 0.1705 Accuracy 0.1002\n",
      "Epoch 7 Batch 160 Loss 0.1717 Accuracy 0.0998\n",
      "Epoch 7 Batch 180 Loss 0.1730 Accuracy 0.0998\n",
      "Epoch 7 Batch 200 Loss 0.1749 Accuracy 0.0997\n",
      "Epoch 7 Batch 220 Loss 0.1765 Accuracy 0.0995\n",
      "Epoch 7 Batch 240 Loss 0.1778 Accuracy 0.0994\n",
      "Saving checkpoint for epoch 7 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-17\n",
      "Epoch 7, Validation Loss 0.3329, Validation Accuracy 0.0850\n",
      "Time for this epoch: 2467.451497077942 secs\n",
      "\n",
      "Starting epoch 8\n",
      "Epoch 8 Batch 0 Loss 0.1351 Accuracy 0.1129\n",
      "Epoch 8 Batch 20 Loss 0.1433 Accuracy 0.1082\n",
      "Epoch 8 Batch 40 Loss 0.1438 Accuracy 0.1067\n",
      "Epoch 8 Batch 60 Loss 0.1471 Accuracy 0.1065\n",
      "Epoch 8 Batch 80 Loss 0.1500 Accuracy 0.1057\n",
      "Epoch 8 Batch 100 Loss 0.1511 Accuracy 0.1050\n",
      "Epoch 8 Batch 120 Loss 0.1514 Accuracy 0.1045\n",
      "Epoch 8 Batch 140 Loss 0.1520 Accuracy 0.1040\n",
      "Epoch 8 Batch 160 Loss 0.1528 Accuracy 0.1037\n",
      "Epoch 8 Batch 180 Loss 0.1543 Accuracy 0.1036\n",
      "Epoch 8 Batch 200 Loss 0.1564 Accuracy 0.1033\n",
      "Epoch 8 Batch 220 Loss 0.1582 Accuracy 0.1029\n",
      "Epoch 8 Batch 240 Loss 0.1591 Accuracy 0.1025\n",
      "Saving checkpoint for epoch 8 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-18\n",
      "Epoch 8, Validation Loss 0.3421, Validation Accuracy 0.0842\n",
      "Time for this epoch: 2464.2276000976562 secs\n",
      "\n",
      "Starting epoch 9\n",
      "Epoch 9 Batch 0 Loss 0.1375 Accuracy 0.1064\n",
      "Epoch 9 Batch 20 Loss 0.1267 Accuracy 0.1084\n",
      "Epoch 9 Batch 40 Loss 0.1286 Accuracy 0.1083\n",
      "Epoch 9 Batch 60 Loss 0.1294 Accuracy 0.1078\n",
      "Epoch 9 Batch 80 Loss 0.1309 Accuracy 0.1073\n",
      "Epoch 9 Batch 100 Loss 0.1324 Accuracy 0.1071\n",
      "Epoch 9 Batch 120 Loss 0.1347 Accuracy 0.1070\n",
      "Epoch 9 Batch 140 Loss 0.1369 Accuracy 0.1064\n",
      "Epoch 9 Batch 160 Loss 0.1382 Accuracy 0.1061\n",
      "Epoch 9 Batch 180 Loss 0.1398 Accuracy 0.1057\n",
      "Epoch 9 Batch 200 Loss 0.1422 Accuracy 0.1053\n",
      "Epoch 9 Batch 220 Loss 0.1437 Accuracy 0.1052\n",
      "Epoch 9 Batch 240 Loss 0.1447 Accuracy 0.1049\n",
      "Saving checkpoint for epoch 9 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-19\n",
      "Epoch 9, Validation Loss 0.3425, Validation Accuracy 0.0849\n",
      "Time for this epoch: 2465.193691968918 secs\n",
      "\n",
      "Starting epoch 10\n",
      "Epoch 10 Batch 0 Loss 0.1243 Accuracy 0.1167\n",
      "Epoch 10 Batch 20 Loss 0.1133 Accuracy 0.1106\n",
      "Epoch 10 Batch 40 Loss 0.1156 Accuracy 0.1119\n",
      "Epoch 10 Batch 60 Loss 0.1152 Accuracy 0.1109\n",
      "Epoch 10 Batch 80 Loss 0.1162 Accuracy 0.1106\n",
      "Epoch 10 Batch 100 Loss 0.1190 Accuracy 0.1105\n",
      "Epoch 10 Batch 120 Loss 0.1197 Accuracy 0.1100\n",
      "Epoch 10 Batch 140 Loss 0.1216 Accuracy 0.1095\n",
      "Epoch 10 Batch 160 Loss 0.1237 Accuracy 0.1091\n",
      "Epoch 10 Batch 180 Loss 0.1257 Accuracy 0.1085\n",
      "Epoch 10 Batch 200 Loss 0.1276 Accuracy 0.1083\n",
      "Epoch 10 Batch 220 Loss 0.1294 Accuracy 0.1080\n",
      "Epoch 10 Batch 240 Loss 0.1306 Accuracy 0.1076\n",
      "Saving checkpoint for epoch 10 in /Users/milkzo/Documents/Jupyter/mini_projet/data_eng_fr/checkpoint/ckpt-20\n",
      "Epoch 10, Validation Loss 0.3435, Validation Accuracy 0.0854\n",
      "Time for this epoch: 2475.0423259735107 secs\n",
      "\n",
      "Total training time :  6:53:7\n"
     ]
    }
   ],
   "source": [
    "# Train the model\n",
    "losses, accuracies, losses_valid, accuracies_valid = main_train(train_dataset,\n",
    "                                                                valid_dataset,\n",
    "                                                                transformer,\n",
    "                                                                EPOCHS,\n",
    "                                                                20\n",
    "                                                               )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- This is the result of a second training carried out on our model restored thanks to the backups of the checkpoints of the previous training.\n",
    "- The model received from the first training, with only 10 epochs, gave some good results and better than those of the RNN model made at the top with 50 epochs. But we rerun the training of the model with 10 more epochs, which allowed us to have better results than the previous one.\n",
    "- As we can see, the duration of the training with only 10 epochs takes about 7 hours to complete, so 14 hours of training with only 10% accuracy for our model, which is small in percentage, but at the less it gives good results because it is realized with a Transformer architecture.\n",
    "- For us the main thing was the writing of the code which allowed the realization of the Transformer model using the attention mechanism, which with a long period of training and a large database, will give us very good results prediction.\n",
    "- The following graphs were not rerun after this new training.\n",
    "- Below, a photo of the result of the first training.\n",
    "\n",
    "![1er_train.png](images/1er_train.png)\n",
    "\n",
    "- The notebook from the first training session will be included in this same folder."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Show some results from training\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABq8AAARuCAYAAABN3K00AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeXxU5aH/8e8zWyaTZSY7SwIJ+w4K\nKCrGtYpb3W1t1Wpdut2uXq+ttrW9va1t7eqtty291+vWuly1/dlqwR1UQFkEAcMikECAhED2PZl5\nfn/MgChbgCRnMvN5v17zmpkzZ+Z8z1Tr6+Q7z/MYa60AAAAAAAAAAACAeOByOgAAAAAAAAAAAACw\nF+UVAAAAAAAAAAAA4gblFQAAAAAAAAAAAOIG5RUAAAAAAAAAAADiBuUVAAAAAAAAAAAA4gblFQAA\nAAAAAAAAAOIG5RUA4ADGGLcxptkYM6w39wUAAAAAxC+uBQEA8YLyCgASQOyCYe8tYoxp2+/5Z4/2\n86y1YWtturV2a2/ue7SMMf9hjHmotz8XAAAAABJBol4L7mWMucUYY40xV/TVMQAA8cnjdAAAwPGz\n1qbvfWyMKZd0i7X25UPtb4zxWGu7+yMbAAAAAKBvJMG14Ock1cbun+3PAxtj3NbacH8eEwDwIUZe\nAUASiI1getIY87gxpknSdcaYU4wxS4wx9caYncaY+40x3tj+ntiv24pjzx+Lvf5PY0yTMWaxMabk\naPeNvX6BMWaDMabBGPOfxpi3jDE3HsM5TTTGLIjlX22MuWi/1y42xpTFjl9pjPlmbHu+MeaF2Htq\njTELj/U7BQAAAIB4N5CvBY0xIySdJukLki4wxuR97PUrjDErjTGNxpgPjDHnxbbnGGMeip1bnTHm\nmdj2W4wxr+/3/oPlf8AYM88Y0yLpdGPMJ2PHaDLGbDXGfO9jGUpj32WDMWabMeb62Pe7wxjj2m+/\nTxljlh3F/3QAkPQorwAgeVwu6S+SgpKelNQt6euSchW9IJij6EXBoXxG0vckZUvaKulHR7uvMSZf\n0lOS7ogdd4ukk472RIwxPkn/kPS8pDxJ35T0pDFmVGyX/5V0s7U2Q9IUSQti2++QtDn2nkGxjAAA\nAACQyAbqteDnJC2x1j4taZOka/e+YIw5VdKDkm6XFJJ0lqSK2Mt/keSTNEFSgaTfHuE4H8//Q0kZ\nkhZLapZ0naLf3SWSvm6MuTiWoUTRa9JfScqRdIKk1dbaxZKaJJ2z3+deJ+nRo8gBAEmP8goAkseb\n1tq/W2sj1to2a+1Sa+3b1tpua+1mSXMlnXGY9z9trV1mre2S9GdJ045h34slrbTW/r/Ya7+WtPsY\nzuU0RS9G7rPWdsWmxfinpE/HXu+SNMEYk2GtrbXWrthv+xBJw6y1ndbaBQd8MgAAAAAklgF3LWiM\nMZKuV7SIUuz+c/vtcrOkP1lrX4md1zZr7XpjTJGipdGXrLV1seu+o5lx46/W2sWxz+yw1r5qrV0T\ne75K0hP68Lu6TtI8a+1Tse9yt7V2Zey1R2KvyxiTG8v0+FHkAICkR3kFAMlj2/5PjDHjjDHPG2Oq\njDGNkv5d0V/AHUrVfo9bJaUfasfD7Dtk/xzWWiupsgfZP26IpK2x9+9VIWlo7PHlkj4paasx5nVj\nzMmx7T+N7feKMWaTMeaOYzg2AAAAAAwkA/FasFRSkaKjtaRoeXWiMWZS7HmRoqOxPq5I0m5rbcNh\nPvtwPv5dnRK7pqwxxjRIukUffleHyiBFR1ldZowJKPojy9estbuOMRMAJCXKKwBIHvZjz/8oaY2k\nUdbaTEnfl2T6OMNOSYV7n8R+TTf00Lsf0g5JRbH37zVM0nZJiv2K8JOS8hWdXvCJ2PZGa+03rbXF\nki6TdKcx5nC/MAQAAACAgW4gXgt+TtG/W75njKmS9Jai53FD7PVtkkYe5H3bJOUaYzIP8lqLpMB+\nzwcdZJ+Pf1dPSHpGUpG1Nijpv/Xhd3WoDLLWbpW0TNKlio4gY8pAADhKlFcAkLwyJDVIajHGjNfh\n5zjvLf9Q9NdylxhjPIrOs553hPe4jTH+/W4pkhYpOk/77cYYrzHmbEkXSnrKGJNqjPmMMSYzNh1F\nk6SwJMWOOzJ2odQQ2x7um1MFAAAAgLgU19eCsdFKVyk6NeC0/W7flHSdMcYt6X8k3WKMOcsY4zLG\nFBpjxlprt0l6WdIDxphQ7HqxNPbRqyRNMcZMNsakSrqnB7kzJNVaa9uNMbP04VT1kvSYpDnGmCuN\nMR5jTK4xZup+rz8i6TuSxkn6fz04FgBgP5RXAJC8blf012xNiv7y7sm+PqC1tlrSpxRd0HaPor9S\ne1dSx2Hedp2ktv1u6621HYoulnupovOk3y/pM9baDbH3fE5SRWwKjJsV/aWbJI2V9Kqii+6+Jem3\n1to3e+0EAQAAACD+xfu14BWxbI9Za6v23iT9SVKqpE9YaxdJulXRa8EGSa8pOo2fFFtrStIGSdWS\nvhrL8L6kn0h6XdJ6ST1ZC+tLku41xjRJuksfTmMoa+0WRa9L75RUK2mFpMn7vfcZSSMUXQesrQfH\nAgDsx3x0uRAAAPpP7BdzOyRdZa19w+k8AAAAAIC+lwzXgrEZP7ZIutFa+7rDcQBgwGHkFQCgXxlj\n5hhjgrHp/76n6PR/7zgcCwAAAADQh5LwWvAaRUeWLXA6CAAMRB6nAwAAks5sSX+W5JO0VtJlsWkA\nAQAAAACJK2muBY0xb0oaLemzlmmvAOCYMG0gAAAAAAAAAAAA4gbTBgIAAAAAAAAAACBuUF4BAAAA\nAAAAAAAgbji25lVubq4tLi526vAAAAAAetny5ct3W2vznM6BxMQ1JAAAAJBYDncN6Vh5VVxcrGXL\nljl1eAAAAAC9zBhT4XQGJC6uIQEAAIDEcrhrSKYNBAAAAAAAAAAAQNygvAIAAAAAAAAAAEDcoLwC\nAAAAAAAAAABA3HBszSsAAAAg0XV1damyslLt7e1OR+lVfr9fhYWF8nq9TkcBAAAAgISTaNeSx3IN\nSXkFAAAA9JHKykplZGSouLhYxhin4/QKa6327NmjyspKlZSUOB0HAAAAABJOIl1LHus1JNMGAgAA\nAH2kvb1dOTk5A/5iY3/GGOXk5CTMLwABAAAAIN4k0rXksV5DUl4BAAAAfSgRLjY+LhHPCQAAAADi\nSSJddx3LuVBeAQAAAAksPT3d6QgAAAAAABwVyisAAAAAAAAAAADEDcorAAAAIAlYa3XHHXdo0qRJ\nmjx5sp588klJ0s6dO1VaWqpp06Zp0qRJeuONNxQOh3XjjTfu2/fXv/61w+kBAAAAAP3tsssu0/Tp\n0zVx4kTNnTtXkjRv3jydeOKJmjp1qs455xxJUnNzs2666SZNnjxZU6ZM0TPPPHPcx/Yc9ycAAAAA\nOKIf/n2t3t/R2KufOWFIpu65ZGKP9n322We1cuVKrVq1Srt379bMmTNVWlqqv/zlLzr//PN19913\nKxwOq7W1VStXrtT27du1Zs0aSVJ9fX2v5gYAAAAA9IyT15IPPvigsrOz1dbWppkzZ+rSSy/Vrbfe\nqoULF6qkpES1tbWSpB/96EcKBoNavXq1JKmuru64M1JeAQAAAEngzTff1LXXXiu3262CggKdccYZ\nWrp0qWbOnKnPf/7z6urq0mWXXaZp06ZpxIgR2rx5s7761a/qoosu0nnnned0fAAAAABAP7v//vv1\n17/+VZK0bds2zZ07V6WlpSopKZEkZWdnS5JefvllPfHEE/vel5WVddzHprwCAAAA+kFPR0j1FWvt\nQbeXlpZq4cKFev7553X99dfrjjvu0A033KBVq1Zp/vz5euCBB/TUU0/pwQcf7OfEAAAAAACnriVf\nf/11vfzyy1q8eLECgYDOPPNMTZ06VevXrz9gX2utjDG9enzWvAIAAACSQGlpqZ588kmFw2HV1NRo\n4cKFOumkk1RRUaH8/Hzdeuutuvnmm7VixQrt3r1bkUhEV155pX70ox9pxYoVTscHAAAAAPSjhoYG\nZWVlKRAIaN26dVqyZIk6Ojq0YMECbdmyRZL2TRt43nnn6Xe/+92+9zJtIAAAAIAeufzyy7V48WJN\nnTpVxhj9/Oc/16BBg/Twww/rvvvuk9frVXp6uh555BFt375dN910kyKRiCTp3nvvdTg9AAAAAKA/\nzZkzR3/4wx80ZcoUjR07VrNmzVJeXp7mzp2rK664QpFIRPn5+XrppZf03e9+V1/5ylc0adIkud1u\n3XPPPbriiiuO6/jmUNOH9LUZM2bYZcuWOXJsAAAAoD+UlZVp/PjxTsfoEwc7N2PMcmvtDIciIcFx\nDQkAAIBkkYjXkkd7Dcm0gQAAAAAAAAAAAIgblFcAAAAAAAAAAACIG5RXAAAAAAAAAAAAiBtJXV51\nhSP68p+Xa/7aKqejAAAAIEE5tcZsX0rEcwIAAACAvZaW1+oHz63V5ppmxzIk0nXXsZxLUpdXbV1h\n7Wxo15f/vELPLK90Og4AAAASjN/v1549exLuomPPnj3y+/1ORwEAAACAPvEf/3hfDy0q17m/WqBv\nPbVSW3a39OvxE+la8livIT19lGdAyPR79djNJ+u2R5fp9v9bpab2Lt14WonTsQAAAJAgCgsLVVlZ\nqZqaGqej9Cq/36/CwkKnYwAAAABAr1u1rV6rKhv09XNGq7WzW48uqdDf3t2uy04Yqq+ePVoluWl9\nniHRriWP5RoyqcsrSUpL8ejBG2fqa4+/qx/8/X01tHXra+eMkjHG6WgAAAAY4Lxer0pK+HEUAAAA\nAAwUjy2pUMDn1i2nlyjD79VtpSM1d+Gmfi2xuJZM8mkD90rxuPXAZ07UlScW6tcvb9CP/lGmSGTg\nD8cDAAAAAAAAAAA9U9fSqedW7dDlJwxVht8rScrLSNHdF03QG/92tm6eXaIXVu/UOb98Xd96cqU2\nObgmVqJL+pFXe3ncLt131RRlpnr04Ftb1NjepZ9eMVkeN/0eAAAAAAAAAACJ7unllerojui6WcMP\neG1viXVb6Uj9ccEmPfZ2hf62crsumTpE/3LWKI0uyHAgceKivNqPy2X0/YsnKJjq1W9e3qjm9m79\n9tppSvG4nY4GAAAAAAAAAAD6SCRi9djbFZpZnKXxgzMPuV9eRoq+e/EEffHMkfrTG5v16OIKPbdq\nhy6cPFhfPXuUxg069HvRcwwr+hhjjL5x7hh9/+IJmre2Src8vEztXWGnYwEAAAAAAAAAgD6ycGON\nKva06vpTinu0f256ir5zwXi9eefZ+vKZI7VgfY3m/OYNfemx5Xp/R2Pfhk0CjLw6hM/PLlGG36M7\nnn5Pjywu122lI52OBAAAAAAAAAAA+sBjSyqUm+7TnImDjup92Wk+3XH+ON16+gg9+Fa5/vetLfrn\nmioNDaVqeE5Aw3MCGpadFruPPt+7nhYOjfLqMK6eUaT/W16pRxZX6ObZI+R2GacjAQAAAAAAAACA\nXrSttlWvrNulr5w5Sj7PsU1YFwr49K1PjNHNs0v05NKten9HoypqW/Xi2mrtaen8yL656SmaPSpH\n504oUOmYPGVSZh2A8uoIbjq1WF/68wq9Ulat846ycQUAAAAAAAAAAPHt8Xe2yki69uRhx/1ZwVTv\nATO5NbV3aWttq7buaVX5nlZtqG7Sgg01+tvKHfK6jWaNyNG54wt0zvh8FWYFjjtDIqC8OoJPTCjQ\nkKBfDy0qp7wCAAAAAAAAACCBdHSH9eTSbTp3fIGGhlL75BgZfq8mDglq4pDgvm3hiNW7W+v0Ulm1\nXnq/Wvc8t1b3PLdW4wdn6qyxeTpxWJamFAWVn+Hvk0zxjvLqCDxul64/pVg/m7dO66uaNHZQhtOR\nAAAAAAAAAABAL/jn6irtaenU9acM79fjul1GM4qzNaM4W9+5YLw21TTrlbJqvVy2S39cuFnhiJUk\nDQn6NaUwpKlFIU0tDGpSYTApphmkvOqBT88s0m9e3qCHFpXr3ismOx0HAAAAAAAAAAD0gkeXVKgk\nN02njcx1NMfIvHSNzEvXbaUj1dYZ1todDVq5rV7vVTZoVWW95q2t2rdvSW6aRuena0xBhkYXpGvs\noAyV5KYpxeN28Ax6F+VVD2Sl+XT5CUP113crdeecsQoFfE5HAgAAAAAAAAAAx2HtjgYtr6jTdy8a\nL5fLOB1nn1Sfe9+orL3qWzujRda2er2/s1Ebqpv0yrpd+0ZouV1GxTkBjSnIUFF2QHnpKcrL2O+W\nnqJQwCtj4uc8D4fyqoc+d2qxnli6TU8u3aYvnDHyyG8AAAAAAAAAAABx67ElFfJ7Xbp6epHTUY4o\nFPCpdEyeSsfk7dvW0R3Wlt0tWl/VpI3VzdpQ3aR1VU16dd0udXRHDvgMr9soNz1F3794gi6YPLg/\n4x81yqseGj84U7NGZOuRxRW6eXaJPG6X05EAAAAAAAAAAMAxaGjr0t/e3aFLpw5VMDAw15BK8bg1\nblCmxg3K/Mh2a62aOrpV09Tx0Vtzh3Y1dig/0+9Q4p6jvDoKN55aoi8+tlwvl+3SnEmDnI4DAAAA\nAAAAAACOwbMrKtXWFdb1pwx3OkqvM8Yo0+9Vpt+rkXnpTsc5JgwfOgrnjs/X0FCqHlq0xekoAAAA\nAAAAAADgGFhr9eiSCk0rCmnS0KDTcXAQlFdHweN26YZThmvJ5lqV7Wx0Og4AAAAAAAAAADhKizbt\n0eaaFl0/K/FGXSUKyquj9KmZRfJ7XXp4UbnTUQAAAAAAAAAAwFF6dHGFsgJeXTRlsNNRcAiUV0cp\nFPDp8hMK9dd3t6u2pdPpOAAAAAAAAAAAoId2NrTppbJqXTOzSH6v2+k4OATKq2Nw46nF6uiO6Iml\nW52OAgAAAAAAAAAAeujxt7cqYq2uO5kpA+MZ5dUxGDsoQ6eOzNGjiyvUHY44HQcAAAAAAAAAABxB\nZ3dEjy/dprPG5qsoO+B0HBwG5dUxuvHUYu1saNeL71c7HQUAAAAAAAAAABzBi+9XqaapQ9fPYtRV\nvKO8OkbnjC9QUXaqHnqr3OkoAAAAAAAAAADgCB5ZXKGi7FSVjslzOgqOgPLqGLldRjeeWqJ3yms1\nb02V03EAAAAAAAAAAMAhrK9q0jtbanXdycPldhmn4+AIKK+Ow/WzhmvS0Ezd9dfV2tXU7nQcAAAA\nAAAAAABwEI8tqZDP49LVM4qcjoIeoLw6Dj6PS7/51DS1dHTrzqffk7XW6UgAAAAAAAAAAGA/Te1d\nenZFpS6eMljZaT6n46AHKK+O06j8DH3ngnF6bX2N/vz2VqfjAAAAAAAAAACA/fzt3e1q6QzrhlOK\nnY6CHqK86gU3nFKs00fn6sfPl2lzTbPTcQAAAAAAAAAAgCRrrR5dUqHJQ4OaWhh0Og56iPKqF7hc\nRvddNVU+j0vffGqVusIRpyMBAAAAAAAAAJD03tlSqw3Vzbp+1nAZY5yOgx6ivOolg4J+/eTyyVq1\nrV4PvPaB03EAAAAAAAAAAEh6jyypUDDVq0umDnE6Co4C5VUvumjKYF1+wlD956sf6N2tdU7HAQAA\nAAAAAAAgae1qbNf8NVW6enqhUn1up+PgKHicDpBofnjpRL2zpVbfemqVnv/abAV8fMUAAAAAAAAA\ngMQRiViV72nR6u0Nau7o3rfd2o/uZ4yUm56iIcFUDQ75lZPm69ep+55Yuk3dEavPzhreb8dE76BZ\n6WWZfq9+cfVUfea/l+jHz5fpx5dPdjoSAAAAAAAAAAAHaO7o1tLyWqX5PAqmepWZGr1P9br3lUzW\nWlXWtem9yga9t71e721r0JrtDWrar7TqKZ/bpUFBvwbHbkXZAU0eGtS0opDyM/29em7d4Yj+8vZW\nlY7JU0luWq9+NvreEcsrY8yDki6WtMtaO+kgr39W0p2xp82SvmStXdWrKQeYU0bm6NbTR2juws06\nZ3y+zh5X4HQkAAAAAAAAAAA+4g+vb9LvXvvggO1et4mWWX6v6lo7VdfatW/7+MGZuvSEIZoyNKRJ\nQ4PKTfd99M37DawKR6xqmjq0s6FdO+vbtLOhXTsa2lXV0Kal5XX6+3s7FY5Eh2sNDvo1tTCkqUUh\nTS0KavLQoDL83mM+t5fLqlXV2K4fXXZArYEBoCcjrx6S9DtJjxzi9S2SzrDW1hljLpA0V9LJvRNv\n4Lr9vDFauKFG33l2tV771xymDwQAAAAAAAAAxJW3t+zR+MGZuuvCcWps61ZDW5ca2rrU2N6173G6\nz6PJhUFNLQxpzKB0pXiObu2owcFUTSk8+GvtXWGt3dGoVdvqtaqyXqu21Wve2ipJ0SkHhwRTlZPu\nUyjgU3bAq6w0n7IDPoVi936vSy5jZIxkjJHLKPpc0v+8uUVDQ6k6e1z+cX5LcMIRGxVr7UJjTPFh\nXl+039Mlkg7xj2FySfG49ePLJ+nK3y/W3IWb9Y1zxzgdCQAAAAAAAAAASdHiaNW2Bn3u1OE6fXSe\nIxn8XremD8/S9OFZ+7bVtXTqve0NWrm1XuV7WqIjv1o6tWV3s+pbuo5qusI754yT29V/a2yh9/T2\ncKCbJf2zlz9zwJo+PFsXTRmsPy7YrE/PHKZBwd6dsxMAAAAAAAAAgGOxZnuDOsMRzSjOdjrKR2Sl\n+XTGmDydMebghVpnd0T1rZ2qbe1UR1dEVlLEWllrZa0UsdHnbpfRtKJQ/4ZHr+m18soYc5ai5dXs\nw+xzm6TbJGnYsGG9dei49u054/TS2mr94sX1+sXVU52OAwAAAAAAAACAlpbXSZJm7DfqaSDweVzK\nz/QrP5PBIonM1RsfYoyZIum/JV1qrd1zqP2stXOttTOstTPy8pwZhtjfirIDuum0Yj2zolJrtjc4\nHQcAAAAAAAAAAC2vqNWIvDTlpKc4HQU4wHGXV8aYYZKelXS9tXbD8UdKPF8+a5RCqV79+PkyWWud\njgMAAAAAAAAASGKRiNWyijrNHB5fUwYCex2xvDLGPC5psaSxxphKY8zNxpgvGmO+GNvl+5JyJP2X\nMWalMWZZH+YdkIKpXn3zE2O0ePMevVy2y+k4AAAAAAAAAIAktqmmWfWtXZpePLCmDETyOOKaV9ba\na4/w+i2Sbum1RAnq2pOG6eFF5br3hTKdOTZPXnevzNgIAAAAAAAAAMBR2bve1cxiRl4hPtGg9BOv\n26W7Lhyvzbtb9OclFU7HAQAAAAAAAAAkqWUVtcpN96k4J+B0FOCgKK/60dnj8nXaqBz95pWNamjt\ncjoOAAAAAAAAACAJLSuv04zh2TLGOB0FOCjKq35kjNHdF05QQ1uXfvfaRqfjAAAAAAAAAACSTHVj\nu7bWtmoG610hjlFe9bMJQzJ19fRCPbSoXBV7WpyOAwAAAAAAAABIIsti613NYL0rxDHKKwfcft5Y\neVwu/fSf65yOAgAAAAAAAABIIssqauX3ujRxSKbTUYBDorxyQEGmX188Y6T+uaZKS8trnY4DAAAA\nAAAAAEgSy8rrdEJRlrxu6gHEL/7pdMitpSUalOnX3X9drY7usNNxAAAAAAAAAAAJrrmjW2t3NLDe\nFeIe5ZVDAj6P7r1isjZUN+vXL210Og4AAAAAAAAAIMGt3FqviGW9K8Q/yisHnTUuX5+eWaS5Czdp\neQXTBwIAAAAAAAAA+s6yilq5jHTisJDTUYDDorxy2N0XjdfgYKpuf2qVWju7nY4DAAAAAAAAAEhQ\ny8rrNG5QpjL8XqejAIdFeeWwDL9X9109ReV7WvXzeeudjgMAAAAAAAAASEDd4YhWbK1jvSsMCJRX\nceDUkbm68dRiPbSoXIs+2O10HAAAAAAAAABAginb2aTWzjDrXWFAoLyKE3fOGaeS3DTd8fR7amrv\ncjoOAAAAAAAAACCBLC2vlSTNZOQVBgDKqziR6nPrF1dP1c6GNv3HP8qcjgMAAAAAAAAASCDLK+o0\nNJSqwcFUp6MAR0R5FUemD8/SF84YqSeXbdOr66qdjgMAAAAAAAAASADWWi0tr2W9KwwYlFdx5hvn\njtbYggzd+cxq1bV0Oh0HAAAAAAAAAAa8hrYuLSuvVXtX2OkojthW26ZdTR2sd4UBw+N0AHxUiset\nX14zVZc98JbueW6t7r/2BKcjAQAAAAAAAMCAVFnXqgffLNeTS7eqpTOsVK9bp47M0Znj8nXmmDwV\nZQeO6/PDEauO7rC6uq1kJLfLyOMycpnYvctIio58au+KqLG9S41tXdH79u7Y424ZScOyAxqeE9CQ\nUKq87t4dd8J6VxhoKK/i0KShQX3tnNH61UsbdN7EAl08ZYjTkQAAAAAAAABgwFizvUFzF27W86t3\nyki6ZOoQnTM+X0u31Oq19TV6Zd0uSdLo/HSdOTZPZ43NV3FumnY1dai6sV27GttV3Rh9XN3UoV2N\n7Wru6FZnd0Qd3RF1dkfUGY4oHLFHzOKOFVg92Xfv/kNCfg3LDmhYdpqG5wSUm56iNJ9bgRRP9N7n\nUVqKW2kpHqWneOT3ug/7mcsq6pTh92hMfkaPMgBOo7yKU186c6ReWbdLdz27WicOy9KQEIvoAQAA\nAAAAAMChWGv1+oYazV2wWYs371F6ikc3zy7RjacW7/v76sVThugH1mrz7ha9vr5Gr6/fpYcXVehP\nb2w54PPcLqO89BQVZKaoMCugTL9HKV6XfG6XfJ7oLcXjls/jktftkrVWEWsVjkjhSCR6b63CkYgk\nKT3Fq8xUjzL9XmX4PcpM9SrT71Wm36Owtdq6p1Vbaz+8Vexp1Ytrq7SnB8vLXDOjUP9+6aRDlljL\nyms1Y3jWvpFgQLyjvIpTXrdLv/nUNF10/xu6/alV+vMtJ/N/LAAAAAAAAABwEHUtnfrMf7+tsp2N\nGpTp110XjtOnTxqmTL/3gH2NMRqZl66Reem6eXaJWjq6tWjTHtU0daggM0UFmX7lZ6YoJy1l36ip\n/jA4mKqTR+QcsL2pvUv1rV1q6exWS0dYrfvfd4b1QXWTHl5codXbG/WH607U8Jy0j7y/rqVTG3c1\n67IThvbXqQDHjfIqjpXkpumeSybozmdW609vbNYXzhjpdCQAAAAAAAAAiDvPrdqhsp2NuveKybry\nxEL5PD1fMyotxaNPTCjow3THJ8PvVcZBSrj9nTk2X994cqUu/s839etrpunc/c5neUWdJGnGcNa7\nwsDRu6u+odddM6NIcyYO0jdV+QcAACAASURBVC9eXK812xucjgMAAAAAAAAAcWfemiqNzEvTtScN\nO6riKlGcNS5f//jqbBXnpOmWR5bp5/PWqTscna5wWUWdvG6jqUUhh1MCPZd8/xYPMMYY3XvFZGWn\n+fT1J95VW2fY6UgAAAAAAAAAEDfqWjr1Tnmtzp84yOkojirKDuj/vniKrj1pmP7r9U264cF3tLu5\nQ8vKazV5aPCQ62EB8YjyagDISvPpl1dP06aaFv3khTKn4wAAAAAAAABA3Hi5rFrhiNWcScldXkmS\n3+vWvVdM1n1XTdHyijpdfP+beq+yQTOKs52OBhwVyqsBYvboXN0yu0SPLqnQK2XVTscBAAAAABlj\n5hhj1htjPjDGfPsgr6cYY56Mvf62MaY4tt1rjHnYGLPaGFNmjPlOf2cHAACJY/7aag0J+jV5aNDp\nKHHj6hlFevbLp8rncakzHGG9Kww4lFcDyB1zxmrcoAz929Pvqaapw+k4AAAAAJKYMcYt6QFJF0ia\nIOlaY8yEj+12s6Q6a+0oSb+W9LPY9qslpVhrJ0uaLukLe4stAACAo9Ha2a03NtbovImDZIxxOk5c\nmTgkqL9/dbZ+/ampOmd8gdNxgKNCeTWApHjcuv/aE9Tc0a1/e3qVrLVORwIAAACQvE6S9IG1drO1\ntlPSE5Iu/dg+l0p6OPb4aUnnmOhflaykNGOMR1KqpE5Jjf0TGwAAJJIF62vU0R3ReRMpZw4mmOrV\n5ScUyu2i2MPAQnk1wIwpyNBdF47Xa+tr9OiSCqfjAAAAAEheQyVt2+95ZWzbQfex1nZLapCUo2iR\n1SJpp6Stkn5hra3t68AAACDxzFtbpayAVyexphOQUCivBqAbThmuM8fm6cfPl6l8d4vTcQAAAAAk\np4P9fPfj00Mcap+TJIUlDZFUIul2Y8yIAw5gzG3GmGXGmGU1NTXHmxcAACSYzu6IXl23S+eML5DH\nzZ+6gUTCv9EDkDFGP7tyitwuo5/PX+d0HAAAAADJqVJS0X7PCyXtONQ+sSkCg5JqJX1G0jxrbZe1\ndpektyTN+PgBrLVzrbUzrLUz8vLy+uAUAADAQLZ48x41tXdrzsRBTkcB0Msorwaogky/bisdoRdW\nV2l5BbNrAAAAAOh3SyWNNsaUGGN8kj4t6bmP7fOcpM/FHl8l6VUbXbx3q6SzTVSapFmS+GUeAAA4\nKvPXVingc2v26FynowDoZZRXA9htpSOUn5Gi/3i+TNHrPwAAAADoH7E1rP5F0nxJZZKestauNcb8\nuzHmk7Hd/kdSjjHmA0nfkvTt2PYHJKVLWqNoCfa/1tr3+vUEAADAgBaJWL30frXOHJsnv9ftdBwA\nvczjdAAcu4DPo9vPG6M7n1mtF1ZX6aIpg52OBAAAACCJWGtfkPTCx7Z9f7/H7ZKuPsj7mg+2HQAA\noKfe3VanmqYOnc+UgUBCYuTVAHfV9CKNG5Shn81bp47usNNxAAAAAAAAAKDPzVtTJa/b6Kxx+U5H\nAdAHKK8GOLfL6K4Lx2trbaseXVzhdBwAAAAAAAAA6FPWWs1fW61TRuYq0+91Og6APkB5lQBKx+Sp\ndEye/vPVD1Tf2ul0HAAAAAAAAADoM+uqmrS1tlVzmDIQSFiUVwnirgvHqam9S7979QOnowAAAAAA\nAABAn5m/tkrGSJ+YUOB0FAB9hPIqQYwblKmrpxfp4cXl2rqn1ek4AAAAAAAAANAn5q+t1vRhWcrL\nSHE6CoA+QnmVQL513hh5XC79bP46p6MAAAAAAAAAQK/buqdVZTsbdT5TBgIJjfIqgRRk+nVb6Qg9\n/95OLa+oczoOAAAAAAAAAPSq+WurJInyCkhwlFcJ5rbSEcrLSNFPXiiTtdbpOAAAAAAAAADQa+av\nrdK4QRkalhNwOgqAPkR5lWDSUjy6/RNjtLyiTvPWVDkdBwAAAAAAAAB6RU1Th5ZvrdOcSYy6AhId\n5VUCunpGkcYWZOin89apszvidBwAAAAAAAAAOG4vvV8ta5kyEEgGHqcDoPe5XUZ3XTRen3vwHT2y\nuFy3nD7C6UgAAAAAAAAAklRbZ1hvb9mjNzbu1sZdzXKb6N8wP7y55DaSy2WU4nErzedWIMXz0Xuf\nR8+uqNSw7IDGDcpw+pQA9DHKqwR1xpg8lY7J0/2vbNSVJxYqK83ndCQAAAAAAAAAScBaq3VVTXpj\nY40Wbtitd8pr1dkdkc/j0tiCaPEUjtjozVpFIlbdsecd3RG1dnartTN80M/+4hkjZYzpz9MB4ADK\nqwR294XjdcFvF+r+VzfqnksmOh0HAAAAAAAAQIKw1qqhrUtVje2qamhXdWO7qhs7tGV3i978YLdq\nmjokSWMK0nX9rOEqHZOnk4qzlepz9+jzIxGrtq6wWjq71doRvW/vimjikMy+PC0AcYLyKoGNHZSh\nT580TI8urtD1s4ZrRF6605EAAAAAAAAA9KEPdjXpjws2a2ZJts4el6/c9JQev9daq/I9rdpe16b6\ntk7Vt3apoa1L9a2dqmvtUn1r9PGupg5VN7aroztywGfkZaRo1ogcnT46V6Wj8zQo6D+m83C5jNJS\nPEpL8UjMEggkHcqrBPfNc8fouZU7dO8/1+lPN8xwOg4AAAAAAACAPmKt1befWa1lFXX6v+WVMkaa\nWhjSOePydc74Ao0fnPGRKfe6wxGV7WzS0vLafbfdzZ0HfG7A51Yo1atgwKdQqlfTikIaFPSrINOv\nQZl+FWSmqCDTr/zMFKV4ejayCgAOh/IqweVlpOhLZ47UffPXa9Gm3Tp1ZK7TkQAAAAAAAAD0gedW\n7dCyijr99IrJmlwY1Ktlu/Tyul365Usb9MuXNmhw0K+zx+WrINOvZRV1WlFRp+aObklSYVaqSkfn\naWZJtkbkpikrzadQwKtgqpdCCkC/o7xKAjfPLtFf3t6qHz9fpr//y2y5XCxoCAAAAAAAACSS1s5u\n3fvCOk0eGtQ1M4rkchlNHBLUV88ZrZqmDr22fpdeLdulv727XS2dYY0tyNBlJwzRzOJsnVSSrcHB\nVKdPAQD2obxKAn6vW/82Z6y+/sRKPfvudl01vdDpSAAAAAAAAAB60X+9tklVje164LMnHPDj9byM\nFF0zo0jXzChSR3dYHd0RZfq9DiUFgCNzOR0A/eOTU4doWlFI981fp9bObqfjAAAAAAAAAOglW/e0\nau4bm3XZtCGaPjz7sPumeNwUVwDiHuVVkjDG6HsXj1d1Y4fmLtzsdBwAAAAAAAAAveTHL7wvj8vo\n2xeMdzoKAPQKyqskMn14ti6aPFh/XLBZ1Y3tTscBAAAAAAAAcJze+mC35q+t1lfOGqVBQb/TcQCg\nV1BeJZk754xTOGL1i/nrnY4CAAAAAAAA4Dh0hyP64d/Xalh2QDfPLnE6DgD0GsqrJDMsJ6CbTivW\n0ysqtWZ7g9NxAAAAAAAAAByjx5ZUaEN1s+6+aLz8XrfTcQCg11BeJaEvnzVKoVSvfr9gk9NRAAAA\nAAAAAByD2pZO/eqlDZo9KlfnTShwOg4A9CrKqyQUTPVq0tCgdtS3OR0FAAAAAAAAwDH4xYvr1dIZ\n1j2XTJAxxuk4ANCrKK+SVDDVq4bWLqdjAAAAAAAAADhKa3c06PF3tur6WcM1uiDD6TgA0Osor5JU\nKOBVfRvlFQAAAAAAADCQWGv1w+feVyjVq2+eO8bpOADQJyivklQo1af61k5FItbpKAAAAAAAAAB6\n6B/v7dQ75bX61/PHKhjwOh0HAPoE5VWSCgW8ilipubPb6SgAAAAAAAAAeqCtM6x7XyjThMGZ+vTM\nYU7HAYA+Q3mVpIKp0V9l1LcwdSAAAAAAAAAwEPx+wSbtaGjXDz45UW6XcToOAPQZyqsklRXwSZLq\n2zodTgIAAAAAAADgSLbVtuqPCzbpk1OH6KSSbKfjAECforxKUqHYfLj1rYy8AgAAAAAAAOLdT14o\nk8sYfefCcU5HAYA+R3mVpPaVV22UVwAAAAAAAEA8W/TBbv1zTZW+fOZIDQ6mOh0HAPoc5VWSCqZG\npw1saGXaQAAAAAAAACBedYcj+uHf31dRdqpuLR3hdBwA6BeUV0kqmMq0gQAAAAAAAEC8+/PbW7W+\nukl3XzhBfq/b6TgA0C8or5KUz+NSms/NtIEAAAAAAABAnKpt6dSvXtqg2aNydf7EAqfjAEC/obxK\nYqGAj5FXAAAAAAAAQJz65Yvr1dzRrXsumSBjjNNxAKDfUF4lsWCqVw1trHkFAAAAAACA+NMdjuj+\nVzZqxdY6p6M44v0djXr8na26ftZwjS7IcDoOAPQryqsklpXmVR0jrwAAAAAAABCH3imv1a9e2qCr\nfr9IP3mhTO1dYacj9RtrrX7w97UKBXz65rljnI4DAP2O8iqJhVJ9qm9l5BUAAAAAAADiz4INNfK6\nja6ZUaS5Czfrwt++oWXltU7H6hf/eG+n3tlSq389b6yCAa/TcQCg33mcDgDnBANeNbQx8goAAAAA\nAADxZ+GG3Zo+PEs/vXKKLpk6RHc+856u/uNi3XRqie44f6xSfW7HsllrVdfapYi1chkjl5FM7D76\n3ChsrVo6utXU3q3mjm41t3eruaNLje3Rx43tXapv7VJ9a6fq2z76uKGtSxOHZOpTM4scO0cAcBLl\nVRILpXpV39olay0LPgIAAAAAACBuVDe2q2xno+6cM06SdNqoXM3/Rql+Nm+dHnxri15ZV62fXzlF\nJ4/I6fMs7V1hbahuUtnORpXtbNK6qkatq2pSfS8sx5GR4lEozatQqk+hgFdF2QGFUr3KSvPpmhmF\ncrv4mx2A5HTE8soY86CkiyXtstZOOsjrRtJvJV0oqVXSjdbaFb0dFL0vFPCqO2LV0hlWego9JgAA\nAAAAAOLDwg01kqTSMbn7tqWlePTvl07SBZMG685n3tOn5i7RVdMLVZiVKmslq+iIqOjj6L3LGHnd\nLnk9Rj63Sykel7xul3yx+4i1aukIq7Wz+8P7zm61doTV1NGtTTXNKt/dooiNZgj43Bo7KEMXTBqs\n0fnp8rqNIlaKWKuIjR4/HIk+dhkp3e9ReopHGX6P0lO8+z32KN3vkdfNqi4AcDA9aSwekvQ7SY8c\n4vULJI2O3U6W9PvYPeJcKNUnSapv7aS8AgAAAAAAQNxYuHG38jJSNGFw5gGvnTIyR/O+cbrum79e\njy6uUPfeZkmSMZJRdAo/ow9LpZ7ye11K83kUSHErzefRqLx0XTJliMYPztC4QZkalh2Qi9FQANDn\njthYWGsXGmOKD7PLpZIesdZaSUuMMSFjzGBr7c5eyog+snexx/rWLhVmORwGAAAAAAAAkBSOWL2x\nsUZnj8s/5FIXAZ9H91wyUd+7aIKkWGl1iH3DEauucEQd3RF1hSPq3O/e7TJKS/Eo4HMr4PMwTR8A\nxIneGG4zVNK2/Z5XxrZRXsW5UGq0vGpoO/75eQEAAAAAAIDesHp7g+pbu3TGmLwj7tuTUVBul5Hb\n5Zbf6+6NeACAftAbk6oe7L8QBx2Ma4y5zRizzBizrKamphcOjeMRCkSnDaxr7XQ4CQAAAAAAABC1\nYH2NjJFOH33k8goAkJh6o7yqlFS03/NCSTsOtqO1dq61doa1dkZeHv/xcVrWftMGAgAAAAAAAPFg\n4cYaTRkaVHaaz+koAACH9EZ59ZykG0zULEkNrHc1MGQybSAAAAAAAADiSENrl97dWqfSHkwZCABI\nXEdc88oY87ikMyXlGmMqJd0jyStJ1to/SHpB0oWSPpDUKummvgqL3uX3upXqdaueaQMBAAAAAAAQ\nB97atFsRqx6tdwUASFxHLK+stdce4XUr6Su9lgj9KhTwMm0gAAAAAAAA4sKC9TXK8Hs0rSjkdBQA\ngIN6Y9pADGDBVK/qmTYQAAAAAAAADrPWauHGGs0elSuPmz9bAkAy478CSS4U8KqBkVcAAAAAAABw\n2MZdzdrZ0M56VwAAyqtkF0r1qb6NNa8AAAAAAADgrIUbaiSJ8goAQHmV7FjzCgAAAAAAAPFgwYYa\njcpP19BQqtNRAAAOo7xKcqGAT/WtXbLWOh0FAAAAAAAASaqtM6y3t9TqDEZdAQBEeZX0QgGvOsMR\ntXWFnY4CAAAAAACAJLVkyx51dkeYMhAAIInyKumFUr2SxNSBAAAAAAAAcMyC9TVK8bh0ckm201EA\nAHGA8irJhQKUVwAAAAAAAHDWwo01mjUiR36v2+koAIA4QHmV5IKpPklSfVunw0kAAAAAAACQjLbV\ntmpzTQtTBgIA9qG8SnJ7R141MPIKAAAAAAAADli4sUaSdAblFQAghvIqye2bNrCN8goAAAAAAAD9\nb8H6Gg0NpWpkXprTUQAAcYLyKsmF9k4byMgrAAAAAAAA9LOucESLNu1R6Zg8GWOcjgMAiBOUV0nO\n73XJ53GpvpU1rwAAAAAAANC/VlTUqbmjW2eMyXU6CgAgjlBeJTljjLICXkZeAQAAAAAAoN8t3Fgj\nt8vo1FGUVwCAD3mcDgDnhVJ9qm9j5BUAAAAAAACOXXtXWFUN7drR0Kad9e2qamzXjvo21bd1qTsc\nUVfYqiscUXfYqjsSUWfYanNNs04cFlKm3+t0fABAHKG8goKMvAIAAAAAAEAPWWu1eXeLllfUaXl5\nnd7f2agd9W3a03Lgj6Oz03wKBbzyuV3yuI08Lpd8bpe8bpdSfS6dXJKt62YNd+AsAADxjPIKCqV6\ntbW21ekYAAAAAAAAcEA4YhWOWBkjGUWXmYjeRx+3d4W1enuDlpXXaXlFnVZsrVNtrKgKpno1pTCo\nSUODGhL0a3Aodd/94KBffq/b0XMDAAxMlFdQKODVe5WMvAIAAAAAAEgE4YhVbUundjW1a1dTh2qa\nOrSnuVN1rZ2qbelUfey+rrVLtS2damjr+d+FRuSm6exx+ZoxPEszirM0IjddLpfpw7MBACQjyiso\nFGDNKwAAAAAAgIGmozusFRX1WrRpt9Zsb/iwqGrpVDhiD9jf53EpJ82nrIBP2Wk+Dc0KKDvgVSjg\nk9dtZK1kpdi93ffcbYzGD87Q9OFZyklP6ffzBAAkH8orKJjqVXtXRO1dYYZyAwAAAAAAxKlwxGrt\njga99cEeLdq0W0vLa9XeFZHbZTSmIEODMlM0aUhQeRkpys9MUV763nu/ctJ9CvjcMoZRUgCA+Ed5\nBYUCXklSQ1sX5RUAAAAAAECc2d3coXueW6s3N+7eN8XfmIJ0XXvSMJ02MlcnjchWpt/rcEoAAHoP\n5RWUFfBJkupaO1WQ6Xc4DQAAAAAAAPb3yxfX68W1Vbr8hKE6bVSuThmZo/wM/oYDAEhclFdQKDX6\ny5z61p4vzgkAAAAAAIC+t7mmWU8tq9T1s4brB5+c6HQcAAD6hcvpAHBeMEB5BQAAAAAAEI9+9dIG\n+dwufeWsUU5HAQCg31BeQaHYtIENbZ0OJwEAAAAAAMBea3c06B/v7dTnZxcrLyPF6TgAAPQbyisw\nbSAAAAAAAEAc+sX89QqmenVb6UinowAA0K8or6CAzy2v26i+jfIKAAAAAAAgHiwtr9Vr62v0xTNG\nKhj74TEAAMmC8goyxiiY6mPkFQAAAAAAQByw1urn89YpLyNFN55a7HQcAAD6HeUVJEmhgJc1rwAA\nAAAAAOLA6xtqtLS8Tl87e5RSfW6n4wAA0O8oryApuu4VI68AAAAAAACcFYlY3TdvvYqyU/WpmcOc\njgMAgCMoryBJCgV8qqO8AgAAAAAAcNQLa3bq/Z2N+tYnxsjn4U93AIDkxH8BISk2bWAr0wYCAAAA\nAAA4pTsc0a9e3KCxBRn65NShTscBAMAxlFeQFJs2sI2RVwAAAAAAAE55enmlNu9u0e3njZHbZZyO\nAwCAY/4/e/cdn3dd7///8c6VnWY0s03TdNDdUuhitYwyFGQ5GO6FoijoOXoUPT8HRznn63EcQEUU\nAeGIR0RcCChTVlktFLqgg7a06UjaZrbZyef3R0ptS4GOJJ+kedxvt+uWa7yvq8/rDzG5ntfr/ba8\nEtA1edXY2kFLe0fcUSRJkiRJkgac5rYOrnt4JdPK8zhjUknccSRJipXllQDIzUwFoM7pK0mSJEmS\npF53+zOvsamuma+8czwhOHUlSRrYLK8EdG0bCFDXaHklSZIkaf+EEM4MISwPIawKIXxtH4+nhRB+\nt/PxZ0MII3d7bGoI4ekQwtIQwuIQQnpvZpekvmRV1Xau/8cqThxbyAlHFMYdR5Kk2FleCejaNhDw\n3CtJkiRJ+yWEkACuB84CJgEfCCFM2mvZJUBNFEVjgGuA/9753GTgduCzURRNBk4B/GNE0oD0zOpt\nvPdn80gkBb51zt7/GZUkaWCyvBIAeRld2wbWOnklSZIkaf8cA6yKomh1FEWtwB3A+XutOR+4bef1\nu4DTQtdeWO8AFkVR9BJAFEXboijyAF5JA85fXtzAR29+jqLsNP70udmMLcmOO5IkSX2C5ZWAf05e\n1TS2xpxEkiRJUj8xDFi/2+2Knfftc00URe1AHVAAjAOiEML9IYQXQghf3dc/EEK4NISwIISwYMuW\nLd3+BiQpLlEUcf0/VvHFO15kWnkef7xsNsPzM+OOJUlSn5EcdwD1Da+XV555JUmSJGk/hX3cF+3n\nmmRgDjALaAQeDiE8H0XRw3ssjKIbgRsBZs6cufdrS1K/1NbRyTf/vIQ75q/n/KNL+f4FU0lLTsQd\nS5KkPsXJKwEwKC2ZRFKgtsnJK0mSJEn7pQIYvtvtMmDjm63Zec5VLlC98/7HoijaGkVRI3AfML3H\nE0tSzLa3tHPJbQu4Y/56Lp87hmsvPtriSpKkfbC8EgAhBPIyUjzzSpIkSdL+mg+MDSGMCiGkAu8H\n7t5rzd3Ax3ZevwB4JIqiCLgfmBpCyNxZap0MLOul3JLUK6IoorW9kx0t7dQ2trKqajsX/vxp5q3a\nyvfeeyT/9s7xdB0DKEmS9ua2gdolNzOF2ibLK0mSJElvL4qi9hDC5XQVUQngliiKloYQvgMsiKLo\nbuBm4NchhFV0TVy9f+dza0II/0NXARYB90VRdG8sb0SSDlJLewevbGpg8YY6FlfUsXhDHRtqm2jr\n6Nx5eeNup1mpCW75+CxOHlcUQ2JJkvoPyyvtkpeR4plXkiRJkvZbFEX30bXl3+73fWu3683AhW/y\n3NuB23s0oCQdoiiKaGhpZ3NdM5vrmllX3ciSDV1F1fLNDbR3dhVUeZkpHDkslxkjBpOWnERKchIp\niaSu64lASiKJ5EQSc8YUMqowK+Z3JUlS32d5pV3yMlOpamiOO4YkSZIkSVKvqdnRyuqtO1izdQdr\nt+5gY11TV1lV31VYNbZ27LH+9aLq0yeNZuqwXKYMy6VscIZbAEqS1I0sr7RLXkYKKyob4o4hSZIk\nSZLU7Rqa23huTTWvbG5g9ZYdrNm6nTVbd1Cz2y40iaTAkJx0SnLSmDAkm1PGFTMkN40huRkMyUln\n2OAMSnPTLaokSephllfaJTfTbQMlSZIkSdLhoaMzYlFFLU+s3MoTK7ewcF3trm3+SnLSGFWYxVlH\nDmV0YRajCrMYXTSIssEZpCSSYk4uSZIsr7TL4MxUGlraaevo9Bc1SZIkSZLUr3R2RqzZtoPn1lTz\nxMotzFu1jbqmNkKAKaW5XHrSaOaMLWRqWR6D0vxITJKkvsz/p9YueZkpANQ1tVE4KC3mNJIkSZIk\nSfvW3tHJ6q07WLKhjsUb6li6oZ6lG+vYsfN8qqG56bxzcglzxhYx+4gCCvycQ5KkfsXySrvkZnSV\nV7WNlleSJEmSJKlvWVnZwL2LN/H4ii0s21RPc1snABkpCSaV5nDBjDImD8tlenkeRxQN8lwqSZL6\nMcsr7ZKXmQpAXVNrzEkkSZIkSZJgVdV27lu8iXsXbWJ5ZQMhwLTheXzwmBEcWZbDlNJcRhcNIpFk\nUSVJ0uHE8kq75O02eSVJkiRJkhSHNVt3cO+ijdyzaBOvbO4qrGaNyOc/zpvMWVOGUJyTHndESZLU\nwyyvtMvrZ15ZXkmSJEmSpN5W1dDM9+57hT8u3ADAzBGD+fa5kzhrylCG5FpYSZI0kFheaZe8jK5t\nA2ubLK8kSZIkSVLvaO/o5LanX+PaB1fQ0t7JZaccwUePH8HQ3Iy4o0mSpJhYXmmX7PRkkgLUNXrm\nlSRJkiRJ6nnPrN7Gt/+ylOWVDZw8roirzpvMqMKsuGNJkqSYWV5pl6SkQG5GCjVuGyhJkiRJknpQ\nZX0z/3nvy9z90kaG5WXwi4/M4B2TSgghxB1NkiT1AZZX2kNeZqrbBkqSJEmSpB5z21Nr+f7fX6Gt\nM+ILp43lspOPICM1EXcsSZLUh1heaQ+5GSnUum2gJEmSJEnqAS+tr+Xbdy/lxLGFXP3uKYwocItA\nSZL0RpZX2kNeZgrVOyyvJEmSJElS9/vpP1aRm5HCDR+ewaA0P5aSJEn7lhR3APUteRkp1HrmlSRJ\nkiRJ6mavbK7nwWWVfPyEkRZXkiTpLVleaQ95maluGyhJkiRJkrrdz/7xKlmpCT4xe2TcUSRJUh9n\neaU95GakUN/cTkdnFHcUSZIkSZJ0mFi7dQf3LNrIh48bQV5matxxJElSH2d5pT3kZaYAUN/k1oGS\nJEmSJKl73PDoq6QkkrjkxFFxR5EkSf2A5ZX28Hp5VWt5JUmSJEmSusHG2ib+uLCC988aTnF2etxx\nJElSP2B5pT28Prpf47lXkiRJkiSpG9z4+GqiCC49+Yi4o0iSpH7C8kp7yMvomryqa3TySpIkSZIk\nHZotDS389rl1vHf6MIblZcQdR5Ik9ROWV9rD65NXtU1OXkmSJEmSpENz85NraOvo5LJTxsQdRZIk\n9SOWV9rD65NXtU5eSZIkSZKkQ1DX2Mbtz7zG2VNLGVWYFXccSZLUj1heaQ85lleSJEmSJKkb3PrU\nWra3tPP5uZ51JUmSDozllfaQSArkpCdT12R5JUmSJEmSDs6OlnZ+9dQaTp9YwoQhOXHHkSRJ/Yzl\nld4gLzOV2kbPvJIkqs3msgAAIABJREFUSZIkSQfnN8++Rm1jG5ef6llXkiTpwFle6Q3yMlOodfJK\nkiRJkiQdhOa2Dn75xBrmjCnk6OF5cceRJEn9kOWV3qBr8srySpIkSZIkHZjmtg5ufnINWxpa+Pxc\np64kSdLBSY47gPqevIwU1m3bEXcMSZIkSZLUhzW3dfDK5gYWV9SyeEMdiyrqWFm1nY7OiGNH5XPc\n6Py4I0qSpH7K8kpv4LaBkiRJkiRpdx2dEauqtvPi+hpeXF/LS+vrWFHZQHtnBEB+VipHDsvl9Ikl\nHFmWy5wxhYQQYk4tSZL6q/0qr0IIZwLXAQngpiiKvrfX4+XAbUDezjVfi6Lovm7Oql6Sl5FCXVMb\nnZ0RSUn+oilJkiRJ0kBTVd/MwvW1vLi+lhfXdU1WbW9pByAnPZmjhudx6fjRTC3L5ciyPEpz0y2r\nJElSt3nb8iqEkACuB84AKoD5IYS7oyhattuybwB3RlF0QwhhEnAfMLIH8qoX5GamEkXQ0NxObmZK\n3HEkSZIkSVIPiqKIV7dsZ/7aGuavrWb+2mrWVzcBkJwUmDg0h/dMG8bRw/M4ujyPUQVZftlVkiT1\nqP2ZvDoGWBVF0WqAEMIdwPnA7uVVBOTsvJ4LbOzOkOpdeRldhVVtU6vllSRJkiRJh5nOzojFG+p4\nds025q+tYcHaamoau44PKMhKZebIwXzs+JFMK89jcmku6SmJmBNLkqSBZn/Kq2HA+t1uVwDH7rXm\nKuCBEMIVQBZwerekUyzydhZWtY1tjCiIOYwkSZIkSTpkHZ0Rz62p5u9LNvH3pZuprG8BYGRBJqdP\nLGHWyHxmjhzMqMIst/+TJEmx25/yal+/sUR73f4AcGsURT8KIRwP/DqEMCWKos49XiiES4FLAcrL\nyw8mr3rBrvKqqS3mJJIkSZIk6WC1dXTy9Kvb+NuSTTywtJJtO1pJS07i5HFFnHXkEGaPKaQ4Oz3u\nmJIkSW+wP+VVBTB8t9tlvHFbwEuAMwGiKHo6hJAOFAJVuy+KouhG4EaAmTNn7l2AqY/IzUgFoLax\nNeYkkiRJkiTpQLR1dPLkqq3cu2gTDy6rpK6pjazUBHMnFHPWlKGcMr6IrLT9+ThIkiQpPvvz28p8\nYGwIYRSwAXg/8MG91qwDTgNuDSFMBNKBLd0ZVL1n8G7bBkqSJEmSpL6tvaOTZ1ZXc8+ijfx96WZq\nG9vITk/mjEklnDVlKCeOLfTcKkmS1K+8bXkVRVF7COFy4H4gAdwSRdHSEMJ3gAVRFN0NfBn4ZQjh\nX+naUvDjURQ5WdVP5WakkJGS4KGXK/nIcSNISnKva0mSJEmS+pLOzojn1u4srJZsZuv2VrJSE5w+\nqYRzppZy0rhC0pItrCRJUv+0X3PiURTdB9y3133f2u36MmB290ZTXJITSfz72RP55p+X8MsnVvOZ\nk4+IO5IkSZIkSQI21zVz54L1/G7+ejbUNpGeksRpE0o4Z+pQ5k4odsJKkiQdFtzkWPv04WPLeWrV\nVn5w/3JmjcpnevnguCNJkiRJkjQgtXd08ujyLdwxfx2PvFJFZwRzxhTy1TPHc/rEEs+wkiRJhx1/\nu9E+hRD43vumsnjDE1zxfwu57wsnkrvzLCxJkiRJktTzKmoauXP+eu5cUMHm+mYKB6Xx2ZOP4OJZ\nwxlRkBV3PEmSpB5jeaU3lZuRwk8/OJ0LbniKK/+wiBs+PJ0QPP9KkiRJkqSeULOjlefWVvPs6mqe\nXbONZZvqATh5XBFXnTeZ0yYWk5JIijmlJElSz7O80ls6engeV545gf+872V+/cxrfPT4kXFHkiRJ\nkiSp3+vsjKhqaGHBa11l1XNrqlle2QBAWnIS08sH86XTx/Ge6cMoG5wZc1pJkqTeZXmlt3XJnFE8\n9epWrr7nZWaMGMzk0ty4I0mSJEmS1Oc9/1oNSzbUsaWhpeuyvWXX9a3bW2jvjADITE0wY8Rgzju6\nlGNH5XNkWS5pyYmY00uSJMXH8kpvKykp8KOLjuas6x7n8v9byF+vmMMgD4OVJEmSJOlNNbd18IFf\nPkNreydJAQoHpVGU3XWZMCSbouw0SnLSmVqWy5RhuW4HKEmStBsbCO2X/KxUfvz+aXzgl8/wjT8t\n5pqLj/b8K0mSJEmS3sTiDXW0tndy3fuP5pyppSSS/BtakiRpf/m1Hu23Y0cX8C+nj+PPL27k989X\nxB1HkiRJkqQ+a+G6GgBmjym0uJIkSTpAllc6IJ+fO4YTjijgW39ZwqKK2rjjSJIkSZLUJy1cV0t5\nfiaFg9LijiJJktTvWF7pgCSSAtdefDQFWWl8+KZnWVxRF3ckSZIkSZL6lCiKeGFdDdPK8+KOIkmS\n1C9ZXumAFeekc8elx5GdnsKHb36WJRsssCRJkiRJet2mumYq61uYNtzySpIk6WBYXumgDM/P5I5L\nj2NQWjIfuskCS5IkSZKk1y1c17XN/vQRg2NOIkmS1D9ZXumg7V5gffjmZ1m60QJLkiRJkqSF62pI\nS05iwpCcuKNIkiT1S5ZXOiTD8zP57aePIzMlwYdussCSJEmSJOmFdTUcOSyX1GQ/dpEkSToY/hal\nQ1ZekMkdlx6/q8BatrE+7kiSJEmSJMWipb2DJRvr3TJQkiTpEFheqVuUF2Ty20uPIyMlwYduesYC\nS5IkSZI0IL28qYHW9k6mDc+LO4okSVK/ZXmlbjOiIIs7Lj2O9JQEF//iaX54/3K2bW+JO5YkSZIk\nSb3mhddqAJhW7uSVJEnSwbK8UrcaUZDFnZ85njljC7n+0VXM/u9H+PZflrC+ujHuaJIkSZIk9biF\n62spzU1nSG563FEkSZL6reS4A+jwMzw/kxs+PINVVdu58fFX+b/n1nH7s+s476hSPnvyEYwfkh13\nREmSJEmSesTCdTVOXUmSJB0iJ6/UY8YUD+L7FxzF41+dy8dPGMn9Szfzzmsf51O3zeeFdTVxx5Mk\nSZIkqVtVNTRTUdPEtHLPu5IkSToUllfqcUNzM/jmOZOYd+Wp/MvpY1nwWg3v/dlTfPLW+SzdWBd3\nPEmSJEmSusXCdbWA511JkiQdKssr9ZrBWan8y+njmHflqXzlneNZsLaas3/8JJ//zQusqtoedzxJ\nkiRJkg7JwnW1pCQCk0tz4o4iSZLUr1leqddlpSXz+bljeOLKU7ni1DH8Y3kV77jmMb5850usr26M\nO54kSZIkSQflhXU1TCrNJT0lEXcUSZKkfs3ySrHJzUjhy+8YzxNfncsnZ4/ir4s2cuqPHuUbf15M\nVX1z3PEkSZIkSdpv7R2dLKqoZbrnXUmSJB0yyyvFrmBQGt84ZxKPfeUULpw5nDueW88pP3yUnzy8\nkqbWjrjjSZIkSZL0tl7Z3EBzW6fnXUmSJHUDyyv1GUNzM/iv9xzJQ186mZPHFfGjB1dw6o8e5U8L\nK+jsjOKOJ0mSJEnSm1q4rgaAacOdvJIkSTpUllfqc0YWZnHDh2dw52eOpyg7jX/93Uu8+2fzeG5N\nddzRJEmSJEnap4XrainKTqNscEbcUSRJkvo9yyv1WceMyufPn5vNNRcfxZaGFi76xdNcdvvzvLZt\nR9zRJEmSJEnaw8L1tUwbnkcIIe4okiRJ/Z7llfq0pKTAe6aV8ciXT+HLZ4zjsRVbOP1/HuPf/7SY\n9dWNcceTJEmSJInqHa2s2brD864kSZK6ieWV+oWM1ARXnDaWR//tFC6aOZy7FlRwyg8f5Uu/e5FV\nVQ1xx5MkSZIkDWAvru8672p6ueddSZIkdQfLK/UrxTnp/Od7juTxr87l4yeM5G9LNnPGNY9z2e3P\ns2RDXdzxJEmSJEkD0MJ1tSSSAkeW5cYdRZIk6bCQHHcA6WAMyU3nm+dM4vNzx3DLk2u47em1/G3J\nZk4ZX8Tlc8cwc2R+3BElSZIkSQPEC+tqmDAkm8xUP2aRJEnqDk5eqV/Lz0rl3945nnlfO5WvvHM8\niyrquODnT3PBDU/x0LJKOjujuCNKkiRJkg5jHZ0RL62vY7rnXUmSJHUbyysdFnLSU/j83DHMu/JU\nvn3uJDbVNfOp/13Amdc9zl3PV9Da3hl3REmSJEnSYWhV1Xa2t7QzzfOuJEmSuo3llQ4rGakJPjF7\nFI9+5RSuvfhokkLg337/Eif/4B/c9MRqdrS0xx1RkiRJknQYeWFdDQDTnLySJEnqNpZXOiylJJJ4\n97Rh/O2LJ/KrT8yiPD+Tq+99mRO+9wg/vH85VQ3NcUeUJEmSJB0GFq6rYXBmCiMLMuOOIkmSdNjw\nJFEd1kIIzB1fzNzxxSxcV8PPH3uV6x9dxY2Pr+bd00r51ImjGVeSHXdMSZIkSVI/tXBdLdPKBxNC\niDuKJEnSYcPySgPGtPLB/OIjM1mzdQe3PLmG3z+/njsXVHDyuCIuPWk0JxxR4B8bkiRJkqT9VtfU\nxsqq7Zx3VGncUSRJkg4rbhuoAWdUYRbfffcUnv7aaXz5jHEs3VjPh256lnf9+En++EIFre2dcUeU\nJEmSJPUDL62vBWD6CM+7kiRJ6k6WVxqwBmelcsVpY3nyyrl8/31Tae/o5Et3vsQlt82nozOKO54k\nSZIkqY9buK6WEGBqWW7cUSRJkg4rllca8NJTElw0azgP/OtJXHXuJJ5YuZXrHloRdyxJkiRJUh/V\n1tHJnQvW89vn1jGuOJvs9JS4I0mSJB1WPPNK2imEwMdnj2Lpxnp+/Mgqpo8YzCnji+OOJUmSJEnq\nI5rbOrhzwXp+8dhqNtQ2Mbk0h++cPznuWJIkSYcdyytpL985fwqLN9Txr797kXu+cCLD8jLijiRJ\nkiRJitGOlnZ+8+xr3Pj4GrZub2HGiMFc/e4pnDK+iBBC3PEkSZIOO5ZX0l4yUhP87EPTOe+n8/j8\nb17gzs8cT2qyO2xKkiRJ0kBT19TGrfPW8qun1lDb2MbsMQX8ZO40jhudb2klSZLUgyyvpH0YXTSI\n718wlc/95gX+399e5tvnug2EJEmSJA0UdU1t3PLkGm6Zt4aG5nZOn1jM5+aOYXr54LijSZIkDQiW\nV9KbeNeRQ/nE7JH8at5aZo7I5+ypQ+OOJEmSJPUpIYQzgeuABHBTFEXf2+vxNOB/gRnANuDiKIrW\n7vZ4ObAMuCqKoh/2Vm7pzdQ1tfGreWu4+cmu0uqdk0v4wmljmVyaG3c0SZKkAcXySnoLXz9rIi+u\nr+XKPyxi4tBsRhcNijuSJEmS1CeEEBLA9cAZQAUwP4RwdxRFy3ZbdglQE0XRmBDC+4H/Bi7e7fFr\ngL/1VmbpzdQ3d01aWVpJkiT1DR7kI72F1OQkrv/gdFISgc/95gWaWjvijiRJkiT1FccAq6IoWh1F\nUStwB3D+XmvOB27bef0u4LSw86CgEMK7gdXA0l7KK71BY2s71z20kjnfe4RrH1rJ8aMLuOeKOfzi\nIzMtriRJkmJkeSW9jdK8DK65+GiWVzbwjT8vIYqiuCNJkiRJfcEwYP1utyt23rfPNVEUtQN1QEEI\nIQu4EviPXsgp7dPDL1dyxv88zjUPreC4naXVjR+dyZRhllaSJElxc9tAaT+cMr6YK+aO4cePrGLu\nhCLOmVoadyRJkiQpbmEf9+39Ta83W/MfwDVRFG3fOYi1738ghEuBSwHKy8sPMqa0p811zVx191L+\nvnQz40oG8fvPHs+skflxx5IkSdJuLK+k/fTF08fx96Wb+ekjqzj7yKG81R/ZkiRJ0gBQAQzf7XYZ\nsPFN1lSEEJKBXKAaOBa4IITwfSAP6AwhNEdR9NPdnxxF0Y3AjQAzZ850CwQdko7OiF8/vZYfPrCC\nto5OvvLO8Xz6xNGkJrspjSRJUl9jeSXtp0RS4FMnjuardy3iyVVbOXFsUdyRJEmSpDjNB8aGEEYB\nG4D3Ax/ca83dwMeAp4ELgEeirn24T3x9QQjhKmD73sWV1J2WbKjj3/+0mEUVdZw0roirz59CeUFm\n3LEkSZL0Jvx6kXQAzj+6lKLsNH75xJq4o0iSJEmx2nmG1eXA/cDLwJ1RFC0NIXwnhHDezmU303XG\n1SrgS8DX4kmrgaq5rYPv3rOM8376JJvqmvnJB6Zx2ydmWVxJkiT1cU5eSQcgLTnBx08YyQ/uX87L\nm+qZODQn7kiSJElSbKIoug+4b6/7vrXb9Wbgwrd5jat6JJwGvBWVDVzxfwtZXtnAh44t56tnTiA3\nIyXuWJIkSdoPTl5JB+hDx5aTkZLgJqevJEmSJKnPiaKI2595jXN/8iTbdrRw2yeP4T/fc6TFlSRJ\nUj9ieSUdoLzMVC6eNZy7X9pAZX1z3HEkSZIkSTvV7GjlM79+nm/8eQnHji7gb188iZPHeV6xJElS\nf2N5JR2ET84eRUdnxK1PrY07iiRJkiQJeGb1Ns667gn+sbyKb5w9kVs/Poui7LS4Y0mSJOkgWF5J\nB6G8IJMzpwzhN8+8xvaW9rjjSJIkSdKA1d7RyY8eWM4HfvkMGakJ/njZbD514miSkkLc0SRJknSQ\nLK+kg/TpE0dT39zOnfPXxx1FkiRJkgak2sZWPnjTs/zkkVVcML2Me66Yw5FluXHHkiRJ0iGyvJIO\n0rTywcwaOZhb5q2hvaMz7jiSJEmSNKCsr27kfTc8xYvrarnm4qP4wYVHkZWWHHcsSZIkdQPLK+kQ\nfOrE0VTUNPH3pZvjjiJJkiRJA8aSDXW894an2NLQwq8vOYb3TCuLO5IkSZK6keWVdAhOn1jCqMIs\nfvn4aqIoijuOJEmSJB32Hl1exUW/eJrURBJ/uOwEjh1dEHckSZIkdTPLK+kQJJICl8wZxUsVdTy3\npjruOJIkSZJ0WLtz/nouuW0BIwuy+OPnTmBsSXbckSRJktQDLK+kQ/S+6WXkZ6XyyyfWxB1FkiRJ\nkg5LURRx7UMr+OofFnHCEQXc+dnjKclJjzuWJEmSeojllXSIMlITfPi4ETz0ciWvbtkedxxJkiRJ\nOqy0dXRy5R8Wce1DK7lgRhm3fHwWg9KS444lSZKkHmR5JXWDjx4/gtTkJG5y+kqSJEmSuk1lfTMf\nuflZ7lxQwRdOG8sPLphKSsKPMiRJkg53/sYndYPCQWm8b3oZf3ihgmdXb4s7jiRJkiT1e48ur+Jd\n1z3BS+vr+NGFR/GlM8YRQog7liRJknqB5ZXUTT53yhEUDUrj4huf4Ut3vsiWhpa4I0mSJElSv9PW\n0cn/+9vLfPxX8ykclMZfr5jN+2aUxR1LkiRJvcjySuomw/MzefBLJ/G5U47gry9t5NQfPcqvn15L\nR2cUdzRJkiRJ6hfWVzdy0S+e5hePreaDx5bzl8tnM6Y4O+5YkiRJ6mWWV1I3ykxN5qtnTuBvXzyJ\nqWW5fPMvS3n39fN4cX1t3NEkSZIkqU/7+5LNnP3jJ1hVuZ2ffnAa//WeI0lPScQdS5IkSTGwvJJ6\nwJjiQdx+ybH85APTqKxv5j0/m8fX/7iY2sbWuKNJkiRJUp9SvaOVb/9lCZ+9/XlGFmZx7xdO5Jyp\npXHHkiRJUoyS4w4gHa5CCJx7VCmnjC/i2odWcutTa7lv8SY+NWcUH589kuz0lLgjSpIkSVKvi6KI\n5ZUNPPxyFY+8UsUL62qIIrhkziiuPHMCqcl+z1aSJGmgs7ySelh2egrfPGcSF84s44f3L+dHD67g\nl0+s5pI5o/nEnJHkWGJJkiRJ6uf+vHADr27ZTl5mKoMzUxicmUrezp+DM1NJS0nimdXbdhVWG2qb\nADhyWC5fPG0s75g0hEmlOTG/C0mSJPUVlldSL5kwJIebPjaLJRvquO7hlVzz0ApufnI1n5wzik/M\nHkVuhiWWJEmSpP4niiK+/sfFNLV1vO3ajJQEc8YWcsWpY5g7oZiSnPReSChJkqT+xvJK6mVThuXy\ny4/OZMmGOn788EqufWglNz+5hk/MHsUnZ48kLzM17oiSJEmStN8aWtppauvg62dN4OJZw6lpbKOm\nsZW6nT9rGtvY3tzOUcNzOW50AekpibgjS5IkqY+zvJJiMmVYLjd+dCZLN9bxk4dX8eOHV/LzR1/l\npHFFnHd0KadPLCYz1f+JSpIkSerbquqbARiSm05eZip5mamMIivmVJIkSerP9uuT8RDCmcB1QAK4\nKYqi7+1jzUXAVUAEvBRF0Qe7Mad02JpcmsvPPzKD5Zsb+P2C9dyzaBMPvVxJZmqC0yeWcN5RpZw0\nrshDiyVJkiT1SZX1LQBuAShJkqRu87blVQghAVwPnAFUAPNDCHdHUbRstzVjga8Ds6MoqgkhFPdU\nYOlwNX5INt84ZxL//q6JPLe2mrtf2sh9izdx90sbyc1I4awpQzjvqFKOHV1AIinEHVeSJEmSAKhq\n6Jq8Ks5OizmJJEmSDhf7M3l1DLAqiqLVACGEO4DzgWW7rfk0cH0URTUAURRVdXdQaaBISgocN7qA\n40YX8B/nTebJlVu5+6WN/PWljdwxfz3F2WmcPXUo5x1VytHD8wjBIkuSJElSfF6fvCp28kqSJEnd\nZH/Kq2HA+t1uVwDH7rVmHEAIYR5dWwteFUXR3/d+oRDCpcClAOXl5QeTVxpQUhJJzJ1QzNwJxTS1\ndvDIK1Xc/dIGfvPMOn41by3D8zM4d2op5x1dyoQhOXHHlSRJkjQAVdY3MygtmUFpntkrSZKk7rE/\nv1nua6wj2sfrjAVOAcqAJ0IIU6Ioqt3jSVF0I3AjwMyZM/d+DUlvISM1wdlTh3L21KHUN7dx/5LN\n3P3SRn7x+Gp+9uirjCsZxNzxxcweU8iskflkpCbijixJkiRpAKhqaHHLQEmSJHWr/SmvKoDhu90u\nAzbuY80zURS1AWtCCMvpKrPmd0tKSXvISU/hwpnDuXDmcLZub+G+xZu4d9Embpm3hl88vprURBLT\nR+QxZ0whJ4wpZOqwXJITSXHHliRJknQYqqpvpjjH8kqSJEndZ3/Kq/nA2BDCKGAD8H7gg3ut+TPw\nAeDWEEIhXdsIru7OoJL2rXBQGh89fiQfPX4kja3tzF9bw7xVW3ly5VZ++MAKeGAF2WnJzBw5mNFF\ngxhRkEl5fiYjC7IYNjiDFEstSZIkSYegsr6FaeV5cceQJEnSYeRty6soitpDCJcD99N1ntUtURQt\nDSF8B1gQRdHdOx97RwhhGdABfCWKom09GVzSG2WmJnPyuCJOHlcEwLbtLTy9ehvzVm1j4boanlld\nTVNbx671iaRAaV46IwuymDg0hzMmlTC9fDCJpH3tFipJkiRJe4qiiMr6Zkpy0uOOIkmSpMPIfp2m\nGkXRfcB9e933rd2uR8CXdl4k9REFg9I4Z2op50wtBbr+sNzS0MJr1Y28tq2R17bt2PXz1nlrufHx\n1RRkpXLaxGLOmDSEE8cWkp7i2VmSJEmS9q2+uZ2W9k7PvJIkSVK32q/yStLhIYRAcU46xTnpzBqZ\nv8djDc1tPLZiCw8sreRvSzZz54IKMlISnDSukDMmDeHsI4eSkWqRJUmSJOmfquqbASh28kqSJEnd\nyPJKEgDZ6Sm7prRa2zt5ds02HlhayYPLKrl/aSVPrdrK/1x8dNwxJUmSJPUhlfUtAJQ4eSVJkqRu\nlBR3AEl9T2pyEieOLeK7757C018/lQ8cU849izdR19gWdzRJkiRJfUhVg5NXkiRJ6n6WV5LeUgiB\nDx5TTmt7J39dtDHuOJIkSZL6kNcnrzzzSpIkSd3J8krS25oyLIfxJdnc9XxF3FEkSZIk9SGV9c1k\npyWTleapBJIkSeo+lleS3lYIgQtmlPHi+lpWVTXEHUeSJElSH7GloYWiHKeuJEmS1L0sryTtl/On\nlZJICtz1/Ia4o0iSJEnqIyrrmynJ9rwrSZIkdS/LK0n7pTg7nVPGFfGnhRV0dEZxx5EkSZLUB1Q2\nNFPi5JUkSZK6meWVpP124cwyKutbeGLllrijSJIkSYpZFEVU1rdQkuPklSRJkrqX5ZWk/XbqhBIG\nZ6Zw1/MVcUeRJEmSFLP6pnZa2zspynbySpIkSd3L8krSfktNTuL8o4fxwLJK6hrb4o4jSZIkKUaV\nDc0ATl5JkiSp21leSTogF8woo7W9k78u2hh3FEmSJEkxqqy3vJIkSVLPsLySdEAml+YwYUi2WwdK\nkiRJA1xVfQsAxW4bKEmSpG5meSXpgIQQuGBGGS+ur2VVVUPccSRJkiTF5PVtA4tzLK8kSZLUvSyv\nJB2w848eRiIpcNfzG+KOIkmSJCkmVfUtZKcnk5maHHcUSZIkHWYsryQdsKLsNOaOL+JPCyvo6Izi\njiNJkiQpBlUNzW4ZKEmSpB5heSXpoFwwo4zK+haeWLkl7iiSJEmSYlBZ30JJTnrcMSRJknQYsryS\ndFBOnVDC4MwU7nq+Iu4okiRJkmJQWd9seSVJkqQeYXkl6aCkJidx/tHDeGBZJXWNbXHHkSRJktSL\noiiiqr6F4hy3DZQkSVL3s7ySdNAumFFGa3snf120Me4okiRJknpRXVMbrR2dFGc7eSVJkqTuZ3kl\n6aBNLs1hwpBstw6UJEmSBpjK+hYASpy8kiRJUg9IjjuApP4rhMAFM8q4+t6X+cnDKxmSm05eZip5\nmSnkZaSQm5lCXkYqqcn25JIkSdLhpLK+GcAzryRJktQjLK8kHZJ3TxvGDY++yo8eXPGmawalJTM0\nN53SvAxK8zIYlvfP66W5GQzNSyclYcElSZIk9RdVDV2TV8XZTl5JkiSp+1leSTokhYPSWPCN09ne\n0k5tY1vXpal158826hpb2bq9lc11zWysa2Lpxjq2bm/d4zUyUhIcMyqfE8cWMmdsIeNLsgkhxPSO\nJEmSJL2d1yevPPNKkiRJPcHyStIhCyGQnZ5CdnoKw/Pffn1zWweb6prZWNvEhtomlm2s54mVW7j6\n3pcBKMpOY86YQuaMKeTEsYUUuxWJJEmS1KdU1TeTk55MRmoi7iiSJEk6DFleSep16SkJRhVmMaow\na4/7N9Y28eSqrTy5ciuPrdjCnxZuAGBYXgYjCjIZUZDFyNd/FmZSnp9JZqr/GZMkSZJ6W2V9i18y\nkyRJUo/xU18Ho0yrAAAgAElEQVRJfUZpXgYXzRzORTOH09kZsWxTPU+u2sorm+pZu62Rvy/ZRE1j\n2x7PKclJY+aIfE4eV8RJ44oYkusf0JIkSVJPq2popiTH864kSZLUMyyvJPVJSUmBKcNymTIsd4/7\n65raWLetkbXbdvDath28umUHT7+6jXsXbwJgwpBsThpXxMnjipg5cjBpyW5jIkmSJHW3yvoWjh21\nH3uGS5IkSQfB8kpSv5KbkcKRZbkcWfbPUiuKIlZUbuexFVU8tmILv5q3hhsfX01GSoJjR+czsiCL\nIbnpDMlJ3+NneorFliRJknSgoihiS4PbBkqSJKnnWF5J6vdCCIwfks34IdlcetIR7Ghp55nV23hs\nxRaeXV3N82traGhpf8Pz8jJTGD44kxkjBjNrZD6zRg72D3BJkiTpbdQ2ttHa0UlxttsGSpIkqWdY\nXkk67GSlJXPaxBJOm1iy677tLe1srmvuutQ3s7muic31zbxatYPfzV/PrU+tBWBEQSYzR+RzzKjB\nzByZz+jCLEIIMb0TSZIkqe+pbGgGoMQvfkmSJKmHWF5JGhAGpSUzpngQY4oHveGxto5Olm6sZ/6a\nauavreYfy6v4wwsVABRkpTJz5OuTWflMLs0hOZHU2/ElSZKkPqOyvgWAkhwnryRJktQzLK8kDXgp\niSSOHp7H0cPz+PRJo4miiFe37GD+2updl/uXVgKQmZpgevk/txk8dnQBiSQnsyRJkjRwVNV3TV4V\nZzt5JUmSpJ5heSVJewkh7JrS+sAx5QBsrmvercyq4dqHVxBFcMGMMn544VExJ5YkSZJ6T1VD1+RV\nsZNXkiRJ6iGWV5K0H4bkpnPuUaWce1QpAHVNbVzz4ApufWotF84o49jRBTEnlCRJknpHZX0zuRkp\npKck4o4iSZKkw5QHt0jSQcjNSOHKMycwLC+Db/1lKW0dnXFHkiRJknpFZX0zxdlOXUmSJKnnWF5J\n0kHKSE3wzXMmsbyygf99+rW440iSJEm9oqqhhZIcz7uSJElSz7G8kqRD8M7JJZw8rohrHlyx6+Bq\nSZIk6XBWVd/ieVeSJEnqUZZXknQIQghcdd5kWts7+X9/eyXuOJIkSVKPiqKIqoZmJ68kSZLUoyyv\nJOkQjSrM4tKTRvOnhRt4dvW2uONIkiRJPaamsY22jsgzryRJktSjLK8kqRt8fu4YhuVl8K2/LKWt\nozPuOJIkSVKPqNy5VbaTV5IkSepJlleS1A0yUhN869xJLK9s4Lan1sYdR5IkSeoR/yyvnLySJElS\nz7G8kqRu8o5JJZwyvohrH1pJ1c4/6iVJkqTDSVVDCwDF2U5eSZIkqedYXklSNwkhcNW5k2lt7+S/\n7ns57jiSJElSt3v9S1pFnnklSZKkHmR5JUndaGRhFp85eTR/fnEjz6zeFnccSZIkqVtV1reQl5lC\nekoi7iiSJEk6jFleSVI3+9wpYxiWl8G3/rKEto7OuONIkiRJ3aayvplip64kSZLUwyyvJKmbZaQm\n+Pa5k1hRuZ1P/+8Cana0xh1JkiRJ6hZVDS2U5HjelSRJknqW5ZUk9YB3TB7C1e+ewlOrtnH2j5/g\nhXU1cUeSJEmSDllVfTPF2ZZXkiRJ6lmWV5LUQz583Aj+cNkJJBKBi37+NDc/uYYoiuKOJUmSJB2U\nzs5o5+SV2wZKkiSpZ1leSVIPOrIsl3suP5G5E4r57j3LuOz2F6hvbos7liRJknTAahpbae+MPPNK\nkiRJPc7ySpJ6WG5mCjd+ZAb/37sm8uDLlZz7kydZsqEu7liSJEnSAamsbwHwzCtJkiT1OMsrSeoF\nIQQ+fdJofnfpcbS0dfLeG57ipidW8/xr1azesp2aHa10drqloCRJkvquyoZmAIotryRJktTDkuMO\nIEkDycyR+dz7hTn8y+9e5Op7X97jsaQAeZmp5GWmkJ+ZyrDBGcwamc+xo/IZUzyIEEJMqSVJkiSo\nqt9ZXrltoCRJknqY5ZUk9bKCQWnc9oljeHlzPVu3t1Kzo5XqHa3UNrZS3dhKzY42ahpbefrVbfzl\nxY0ADM5MYdbIfI4Zlc+xowqYODSb5ITDs5IkSeo9VTu3DSzOsbySJElSz7K8kqQYJCUFJpfmvuWa\nKIpYV93Is2uqeW5NNfPXVvPAskoAslITjCjIIjs9mez0FHLSk8nJSNl5+/X7UsjJ2PPxnPQUUpMt\nvSRJknTgKhuaGZyZQlpyIu4okiRJOsxZXklSHxVCYERBFiMKsrho5nAANtc189zaauavqWZjbRMN\nze1U1DTS0NxOQ3Mb21vaebujs9JTkshJTyE/K5X8rFQGZ6VSsPP665eSnHTGFg8iLzO1F96pJEnq\nr0IIZwLXAQngpiiKvrfX42nA/wIzgG3AxVEUrQ0hnAF8D0gFWoGvRFH0SK+G1wGrrG+hONvzriRJ\nktTzLK8kqR8ZkpvOeUeVct5Rpft8PIoidrR20NDcRkNzO/VNbdQ3t1Hf1L7zZ9f9tY1tO7cobOXl\njfVs29FKXVPbG16vODuNcSXZOy+DGLvzZ3Z6Sk+/VUmS1MeFEBLA9cAZQAUwP4RwdxRFy3ZbdglQ\nE0XRmBDC+4H/Bi4GtgLnRlG0MYQwBbgfGNa770AHqqqhxS0DJUmS1CssryTpMBJCYFBaMoPSkhn6\n1rsSvkF7Ryc1jW1U72hlY10TKysbWL55OyurGvjtc+toauvYtTY/K5WiQWkUZqdSOChtt0sqhdlp\npCaSiCLojCIiukq1KIKIiNREghOOKCApKXTvm5ckSb3tGGBVFEWrAUIIdwDnA7uXV+cDV+28fhfw\n0xBCiKJo4W5rlgLpIYS0KIpaej62DlZVfTNjiwvjjiFJkqQBwPJKkgRAciKJouw0irLTGD8km7nj\ni3c91tkZUVHTxIrKBpZXNrChtomtDS1s3d7CwnW1bN3eQmNrx1u8+p6uufgo3jOtrCfehiRJ6j3D\ngPW73a4Ajn2zNVEUtYcQ6oACuiavXvc+YOG+iqsQwqXApQDl5eXdl1wHrLMzYktDCyVOXkmSJKkX\nWF5Jkt5WUlKgvCCT8oJMTp9Uss81ja3tbG1oZcv2Fto7OklKCgQghK6JsK7rgS/esZDfzV9veSVJ\nUv+3rzHqvU/ffMs1IYTJdG0l+I59/QNRFN0I3Agwc+bMtznZUz2purGV9s7IM68kSZLUKyyvJEnd\nIjM1mfKCZMoLMt9y3UUzh/OD+5fz2rYdjCjI6qV0kiSpB1QAw3e7XQZsfJM1FSGEZCAXqAYIIZQB\nfwI+GkXRqz0fV4eisr4ZwMkrSZIk9YqkuANIkgaW904fRlKA3y+oiDuKJEk6NPOBsSGEUSGEVOD9\nwN17rbkb+NjO6xcAj0RRFIUQ8oB7ga9HUTSv1xLroFXVd+3qWJzj5JUkSZJ6nuWVJKlXDc3N4KRx\nRdz1fAUdne7+I0lSfxVFUTtwOXA/8DJwZxRFS0MI3wkhnLdz2c1AQQhhFfAl4Gs7778cGAN8M4Tw\n4s5LMeqzXp+8Ks528kqSJEk9z20DJUm97uKZw7nsNy/w+MotzB3v51SSJPVXURTdB9y3133f2u16\nM3DhPp53NXB1jwdUt6lq6Jq8KrK8kiRJUi9w8kqS1OtOm1hCflYqv1+wPu4okiRJ2g+V9c3kZ6WS\nlpyIO4okSZIGAMsrSVKvS01O4j3ThvHgskq2bW+JO44kSZLeQmNrO8+tqWZoruddSZIkqXdYXkmS\nYnHRzOG0dUT8+cWNcUeRJEnSm+jojPjCbxfy6pbtfOmMcXHHkSRJ0gBheSVJisX4IdkcVZbLnfPX\nE0VR3HEkSZK0D9+9ZxkPvVzFt8+dzGkTS+KOI0mSpAHC8kqSFJuLZg1neWUDiyrq4o4iSZKkvdzy\n5BpufWotl8wZxcdOGBl3HEmSJA0glleSpNice1Qp6SlJ3LlgfdxRJEmStJv7l27mu/cu452TS/j3\nd02MO44kSZIGGMsrSVJsctJTeNeUodz94kaaWjvijiNJkiTgpfW1fPGOhUwty+Pai6eRSApxR5Ik\nSdIAY3klSYrVhTOH09DSzt+Xboo7iiRJ0oC3vrqRS25bQOGgNG766EwyUhNxR5IkSdIAZHklSYrV\nsaPyKc/P5Hfz3TpQkiQpTnVNbXzi1vm0tndw6ydmUZSdFnckSZIkDVCWV5KkWCUlBS6aWcYzq6t5\nbduOuONIkiQNSK3tnVx2+/O8tm0HP//IDMYUZ8cdSZIkSQOY5ZUkKXbvm1FGUoC7nq/4/9m78zC5\nyjL//++7q/d0urNvnYQECGsWCGFzFBxRQVFBEAEVZBnUGUfFr7vjbxx1HBV11FF0RAYFN0BAZRRh\nwAVEQUNCIKyCgSSdfe0svXc/vz+qgk1IoBM6daq736/r6quqznpXcaj03Z9znpN1KZIkSUNOSolP\n/Gwxf/zrBj5/xmxecsCYrEuSJEnSENen8CoiTomIxyPiyYj46PMs96aISBExr/9KlCQNdhMbajjh\noLHcsKCJ7p6UdTmSJElDyv/c/RTX39fEe15xIGceNTnrciRJkiTKX2iBiMgBlwOvApqA+RFxc0rp\nkZ2WGw68F/jTvihUkjS4vXneFP7phwv5/RPrePnB4541b+2WNh5euYWHVjSzsaWDYZXlDKsqZ1hV\njtrKcoZV5p553dWdaOnoZlt7Fy0dXWxv72Z7exfbO7pp7eiitqqckbUVjKipZERtBSOHVTKipoIR\ntZU01FRQWe5FyZIkaej4zWNr+Owtj/KamRN4/ysPyrocSZIkCehDeAUcAzyZUloCEBHXAqcBj+y0\n3GeAy4AP9muFkqQh4aRDxzGytoKr//g0rR3d+bBqZTMPr9zCuq3tzyxXV1VOS0cXe3qBVmWujOqK\nMlo6uul6npVrKnLU15TTUFNBfXUF9TUV1FeXFx4rnjOvodf04dUV5Mpibz8CSZKkonp89Vbe++NF\nHD6pni+/eQ5l/h4jSZKkEtGX8KoRWN7rdRNwbO8FIuJIYEpK6RcRsdvwKiLeAbwDYOrUqXterSRp\n0Koqz/HGIydz1R+e4rePryNXFswYV8cJM8Zy+KR6Dp9Uz2GT6hleXUFKibbOHrZ3dNHS3p1/7Ohi\nW3s35WVBbWWOuqpyaqvKqassp6Yy98wVVSkltrV3sbmlM//T2sGmlk6aW/KPW9s62dLaxZa2Tra0\ndbJuazt/XbeNLa2dNLd2vmBoNryqnOE7hV311RU01FZw0d9NZ8qo2iJ8mpIkSc9vw7Z2Lr56PrWV\nOb5z/jxqK/vy5wFJkiSpOPry2+muTr165k93EVEGfAW44IU2lFK6ArgCYN68ed7URJL0LO87aQYz\nG+s5cFwdB40fTnVFbpfLRQQ1lTlqKnNQt2f7iAiGV1cwvLqCKaP2bN2UEts7up8Jsra0drKlrYvm\nwutnBV+t+fBr5eY2HmvbyorNrVTkyvj4aw/ds51KkiT1s/aubt71gwWs29rOde88nokNNVmXJEmS\nJD1LX8KrJmBKr9eTgZW9Xg8HZgK/iwiACcDNEfGGlNJ9/VWoJGnwa6it4Iy5pXuT8Iigrqqcuqpy\nJo3Ysz/ynPaNu1nc1LyPKpMkSeqblBKf+OlDzH96E18/90iOmDIi65IkSZKk5+jLXennAzMiYnpE\nVALnADfvmJlSak4pjUkpTUspTQPuBQyuJEnqZWZjAw+tbCYlLzyWJEnZ+c7vl/CTBU2896QZvH7O\npKzLkSRJknbpBcOrlFIX8M/AbcCjwPUppYcj4tMR8YZ9XaAkSYPBrMYGtrZ1sWxjS9alSJKkIerX\nj67hc796jFNnTeTSk2ZkXY4kSZK0W326I2tK6Rbglp2m/etuln35iy9LkqTBZWZjAwCLVzSz3+hh\nGVcjSZKGmvlPb+S9P76fmZMa+NJZcygr29XtrSVJkqTS0JdhAyVJ0ot00PjhVObKWLzC+15JkqTi\n+vmiFbz1O39ifH013zl/HjWVuaxLkiRJkp5Xn668kiRJL05leRkHTxjOQ4ZXkiSpSFJKfP03T/Kf\nt/+FY6eP4tvnHcWI2sqsy5IkSZJekOGVJElFMrOxgVsWryKlRIRD9UiSpH2no6uHj920mBsXNnHG\n3EY+f8ZsKssdfEWSJEkDg7+5SpJUJLMaG2hu7WT5xtasS5EkSYNYc0sn51/1J25c2MT7X3kQXz5r\njsGVJEmSBhSvvJIkqUhmNTYA8NDKZqaOrs24GkmSNBgt29DCBd/7M00bW/nK2XN445GTsy5JkiRJ\n2mOeeiVJUpEcNKGOilyw2PteSZKkfWDB0k288Zt/YOP2Dr5/8TEGV5IkSRqwvPJKkqQiqSrPcdD4\n4TxkeCVJkvrRwyub+e87l/DLB1cydVQtV11wNPuPrcu6LEmSJGmvGV5JklREsxobuPXh1aSUiIis\ny5EkSQNUSol7l2zkW3f+lbv+so66qnIuedn+vOvEAxg5rDLr8iRJkqQXxfBKkqQimtnYwLXzl9O0\nqZUpo7zvlSRJ2jM9PYn/e2QN37rzrzywfDNj6ir50MkH87bj9qOhpiLr8iRJkqR+YXglSVIRzWps\nAOChFc2GV5IkaY/88sFVfPn2x1mybjtTR9Xy76fP5E1HTaa6Ipd1aZIkSVK/MrySJKmIDp4wnPKy\nYPGKZl4za2LW5UiSpAGgrbObT/3vI/z4z8s4ZMJwvn7ukbxm5gTKc2VZlyZJkiTtE4ZXkiQVUXVF\njhnjh/PQyi1ZlyJJkgaAZRta+McfLuDhlVt414kH8MFXH2RoJUmSpEHP8EqSpCKb1VjPHY+uJaVE\nRGRdjiRJKlH/9/BqPvCTBwjgyvPn8crDxmddkiRJklQUhleSJBXZrMYGrr+viZXNbTSOqHnB5Ztb\nO7n9kTVsae1kW3sX29q72NqWf9zW1snWti4qcmVMGlFD48gaJhceG0fUMHFENVXl3gdDkqSBpLO7\nhy/d9jjfvmsJsxob+OZb53qvTEmSJA0phleSJBXZzMYGABY3NfcpvPrSbY/z/XuXPvO6uqKMuqoK\nhleXU1eV/2nv6ubuJ9exdms7KT17/bHDqxhW+fwBVkQQAAHR6/WOC8O6ehLdPYmu7kRXT0/hMdHV\n3UNZWVBfXUF9TQX11eXU11TQUFNRmFZOVXmOXBmURVAWQa4sKCsLchFUV5Rx8uETGFblrySSJAGs\n2dLGP/9oIfOf3sRbj53K//e6w6iu8EQUSZIkDS3+pUiSpCI7dGI9ubLgoRXNnDJzwvMu29rRzc/u\nX8Gpsyfy76fNpK66nIrnuc9FR1cPq5vbaNrcwopNrazc3MbKza20dXXvdp2UIAEpJRJAgkTKTy8E\nYblcUFEW5MrKqMjlA6iKXBm5sqC7J7GlrZMtrV1saetk+cYWHmnrorlwpdgLecUhq7jy/HmUlTmE\noiRpaLt/2SYuueY+trd389Wzj+D0IxuzLkmSJEnKhOGVJElFVl2RY8a4OhavaH7BZX/10Cq2tndx\n3nH7MXJY5QsuX1lextTRtUwdXRpDC3V199DZnehOiZ6U6ClcwdWdEj098IsHV/Lvv3yUK+9ewjtO\nOCDrciVJysz29i7++Uf3U12R48eXHMeM8cOzLkmSJEnKjOGVJEkZmNnYwG8fW0tKiYjdX3F07fzl\nTBtdy7HTRxWxuv5Tnivj+W65dfFLp7Nw2Sa+cOvjHLXfSI7ab2C+T0mSXqzLbn2Mlc2t/OSdxxtc\nSZIkacjb/bhDkiRpn5nV2MCG7R2s3tK222WWrNvGn5/ayJuPnvK8AddAFhF8/szZNI6o4T0/up9N\n2zuyLkmSpKL781Mbufqepbz9+GnMm+aJHJIkSZLhlSRJGZjZ2ADA4qbdDx143X3LyZUFb5o7uVhl\nZaK+uoLL3zKX9ds6+MBPHqCnJ2VdkiRJRdPW2c1HbnyQySNr+NDJB2ddjiRJklQSDK8kScrAYRPr\nKQt4aDf3vers7uHGBU284pBxjKuvLnJ1xTdrcgP/cuqh/OaxtVx595Ksy5EkqWi+csdfeGr9dj5/\nxmyGVTmyvyRJkgSGV5IkZaKmMseB4+pYvJvw6jePrWX9tg7OOXpKkSvLzvnH78drZk7gslsfZ8HS\nTVmXI0nSPvfA8s18564lnD1vCi+dMSbrciRJkqSSYXglSVJGZjY2sHjFFlJ67jB5181fzrjhVZx4\n0NgMKstGRPCFN81m0oga3vOjhWxu8f5XkqTBq6Orh4/c+CBjh1fx8VMPzbocSZIkqaQYXkmSlJFZ\njQ2s39bOmi3tz5q+qrmV3z2+lrPmTaY8N7T+qX7W/a+uf2CXwZ4kSYPBN3/3JI+t3spnT59FQ01F\n1uVIkiRJJWVo/UVMkqQSMquxAeA5QwfecF8TPQnePG/oDBnY2477X/36sbVc+funsi5HkqR+99jq\nLXzjN09y2hGTeOVh47MuR5IkSSo5hleSJGXksEn1lAU81Cu86ulJXL9gOS85YDT7jR6WYXXZ2nH/\nqy/c+hj3LtmQdTmSJPWbru4ePnzDgzTUVPDJ1x+edTmSJElSSTK8kiQpI7WV5Rwwtu5Z4dU9Szaw\nfGMrZx89NK+62mHH/a+mjRnGO7+/gCXrtmVdkiRJ/eLKu5/iwaZmPnXa4YwaVpl1OZIkSVJJMryS\nJClDMxsbnjVs4LXzl9NQU8HJh0/IsKrSUF9dwVVvP5pcWXDx1fexaXtH1iVJkvSiLNvQwldu/wuv\nPmw8p86amHU5kiRJUskyvJIkKUMzGxtYu7WdtVva2LS9g9seWs0bj2ykuiKXdWklYeroWr5z/lGs\n2NzKu36wgI6unqxLkiRpr9328Grau3r419cfRkRkXY4kSZJUsgyvJEnK0KzGBgAWr2jmp/evoKO7\nZ8gPGbizo/YbxRffNJs/PbWRj920mJRS1iVJkrRXFi7bxJRRNUweWZt1KZIkSVJJK8+6AEmShrLD\nJ9UTkQ+vfrV4NXMmN3DoxPqsyyo5px3RyFPrt/PVO55g/7HDePffH5h1SZIk7ZGUEguXbeK4/Udn\nXYokSZJU8gyvJEnK0LCqcvYfM4wbFjTRtKmV/3jjrKxLKlnvO2kGT63fzhdve5zpY4bx2n6+V0hb\nZzdrt7SzZmsba7a0sXF7B1XlZdRVVVBXXU5dVTnDq8sZVpV/XldVTq7MIZ8kSX2zsrmNNVvamTt1\nZNalSJIkSSXP8EqSpIzNbGzg54tWUlOR4/VzvHn77kQEXzhzNk2bWnn/dYuYNKKGI6aM2OWyW9s6\neWz1VtZsaaOlo5vWju7CYxctHd1sLzzfsL2DNVvyf0xsbu3co3pyZUHjiBr2G13L1FG1zzxOHTWM\n/UbXUluZo62zh63tnWxr62J7e/czz7e1d9HVnUgkdoyCuGMwxJSgsryMkw8fz/DqihfxiUmSSsnC\npZsADK8kSZKkPjC8kiQpY7MK4dWpsycaVryA6oocV5x3FKd/8w/8w9X38bN3vwSAR1Zu4dFVW3lk\nVTOPrtrKso0tu1w/AmoqctRW5qipzDF6WBXTxwzjuP1HM76+mnHDqxhXX834+ipGDauko6uH7e3d\nbGvvZGshdNoRPm3c3sGyjS0s29jCLxevYnPLs8OvXFnQ3bP39+e66u56vnfR0YwbXr3X25AklY4F\nSzdRXVHGIROHZ12KJEmSVPIMryRJythLDhhDVXkZ5x23X9alDAij66r47gVH88Zv/pETLvstO/Kh\nCJg+ehizGhs4++gpHDpxOJNH1j4TVtVWllNdUUbEvhnqr7m1k2UbWli6cTtLN7TQ0tH1zJCDw3cM\nNVgYfnBYVTmV5WX5ugu1558HEfBgUzPvu/Z+zvzWH/n+RccybcywfVKzJKl47l+2iTmTR1CRK8u6\nFEmSJKnkRUp7f0bwizFv3rx03333ZbJvSZJKTXdP8v5Je2jR8s38fNEKDhhbx6ET6zlkwnCGVQ2e\n83IWLd/MRd+bTwDfu/AYZk1uyLok6QVFxIKU0rys69DgNJB7yLbObmZ+8jYuOWF/PnLKIVmXI0mS\nJJWE5+shPeVLkqQSYHC1546YMoJPvv5w3nbcfhy138hBFVxB/v3d8K7jqa7Icc4V9/D7J9ZlXZIk\naS8tXtFMV0/yfleSJElSHxleSZIklaj9x9Zx0z+9hCmjarnoe/O5+YGVWZckSdoLC5ZuAuDIqSMy\nrkSSJEkaGAyvJEmSStj4+mque+fxHDl1JO/98f189w9PZV2SJGkPLVy6iWmjaxlTV5V1KZIkSdKA\nYHglSZJU4hpqKrjmomM4+fDxfOp/H+GyWx8jq/uWSpL2TEqJhcs2O2SgJEmStAcMryRJkgaA6ooc\n33zrUZx7zFS++bu/8pMFTVmXJEnqg6ZNrazf1s6R+xleSZIkSX1leCVJkjRA5MqCz54+k+P3H82n\nbn6Y5Rtbsi5JkvQCdtzvaq73u5IkSZL6zPBKkiRpACkrC7705jmURfD/rl9Ed4/DB0pSKVu4bBO1\nlTkOHj8861IkSZKkAcPwSpIkaYBpHFHDp08/nPlPb+KKu5ZkXY4k6XksXLaJI6aMoDxn+y1JkiT1\nlb89S5IkDUCnH9HIqbMn8p+3P85DK5qzLkeStAstHV08umorc6d6vytJkiRpTxheSZIkDUAR+ftf\njayt5P3XLaKtszvrkiRJO3mwqZnunsTc/bzflSRJkrQnDK8kSZIGqBG1lXzxrDk8sXYbX7zt8azL\nkSTtZMHSTQAcOcUrryRJkqQ9YXglSZI0gJ140Fjefvx+/M/dT/GHJ9dnXY4kqZf7l21i/zHDGDms\nMutSJEmSpAGlPOsCJEmS9OJ89DWHcveT6/ngTx7g1ktPoKGmIuuS+qyzu4e2zm5aO7tp7+yhtbOb\n1o7uZ6b97bGHXFnwhjmTqK7IZV22JL2glBILl23mFYeMy7oUSZIkacAxvJIkSRrgaipzfOXsIzjj\nm3/kX3/+EF8758ii7Le9q5u1W9pZu7WNNVvaWbOljQ3bOtje0UVrRzctHd20dHTR0tHN9o5uWgvP\n2zrzgaBVR/wAACAASURBVFVbZzddPWmP9rng6U184U2z99E7kqT+s3RDCxu3dzB3qkMGSpIkSXvK\n8EqSJGkQmD15BO87aQZfvv0vnDBjLGfMbSQiXvR2t7Z18sTabTyxZiuPr97GX9dtY82WNtZsaWNT\nS+dzli8LGFZZTk1ljmFV5dRU5BhWlaOhpoJJDdXUVOSorsxRXZ6jprKM6vIc1c9MK6OmMkdNRf6n\nqvC4Y9p3//gU375zCScdOo5XHz7hRb83SdqXdtzvau5+IzKuRJIkSRp4DK8kSZIGiX98+QH85vG1\nfOAnD/CZXz7CnMkjmDNlBEdMaWD25BGMqava5Xrb2rtY3dzKquY2Vm1u48l12/jLmq08sWYbKza3\nPrNcdUUZB4ytY/LIWuZNG8n44dWMr69mXH0V4+vzz0fWVvRLaLYrH3jVwdz9xHo+dtNijpw6krHD\nd/1+JKkULFy2ibqqcmaMG551KZIkSdKAY3glSZI0SJTnyvjehcfwywdX8WDTZhYt38w3fvMEO0bm\nmzyyhjlTRlBbkWP1ljZWNbexurmNbe1dz9pOZa6M/ccOY960kbxl/FQOGj+cg8bnQ6tc2b4Jpvqi\nsryMr559BKd+/W4+dtODfOf8efssKJOkF2vhss0cOXVEpt+bkiRJ0kBleCVJkjSINNRU8JZjp/KW\nY6cC0NLRxUMrtvDA8s0satrMA8s309ndw4SGGg4cW8dLDxzDxIZqJjRUM6E+/9g4oobyXFnG72TX\nZowfzkdOOYTP/OIRrpu/nHOOmZp1SZL0HNvau3h89RZe9YoZWZciSZIkDUiGV5IkSYNYbWU5x0wf\nxTHTR2VdSr+58CXT+PWja/j0Lx7h+ANGs9/oYVmXJEnP8sDyzfQkmDvV+11JkiRJe6M0T6mVJEmS\ndqOsLPjSWXPIlQXvv24RXd09WZckSc+ycOkmAI6cMjLjSiRJkqSByfBKkiRJA86kETX8++kzWbhs\nM9++a0nW5UjSsyxctokDx9XRUFuRdSmSJEnSgGR4JUmSpAHpDXMm8brZE/nK7X/hoRXNWZcjSQCk\nlLh/+WaOmupVV5IkSdLeMrySJEnSgBQR/PvpMxldV8ml1y2irbM765IkiSXrt7O5pZO5+3m/K0mS\nJGlvGV5JkiRpwBpRW8mXzprDk2u38YVbH8u6HEliQeF+V3O98kqSJEnaa4ZXkiRJGtBeNmMsF7xk\nGt/9w9N8/56nsy5H0hB3/7JN1FeXc8DYuqxLkSRJkgas8qwLkCRJkl6sj7/2UJo2tfD//fxhIoK3\nHbdf1iVJGqIWLt3MkVNHUlYWWZciSZIkDViGV5IkSRrwKsvLuPytc/mnHyzkEz97iLII3nLs1KzL\n6ncpJVo6ulm/rb3w08H6be288tDxjK+vzro8acj73eNr+cvarbx21sSsS5EkSZIGNMMrSZIkDQpV\n5Tm++ba5/OMPFvLxny4mAs49ZuAEWCkl1m/rYMXmVlZsamXl5tb8882trN3azoZCYNXW2fOcdSc1\n1BheSRlq6+zmc7c8ytX3LOXg8cM5++gpWZckSZIkDWiGV5IkSRo0qspzfOttc3nX9xfwsZsWUxZw\n9tHZBVid3T2s29rOuq3tbGzpYHNLBxu3d7JpewcbWzryj9s7WLu1nRWbW+noenYwVVdVzqQR1Yyv\nr2b/McMYU1fJ6LoqxtRVMbqukrGFxzF1VRm9Q0kPr2zm0msX8cTabVz4d9P4yCmHUF2Ry7osSZIk\naUAzvJIkSdKgkg+wjuKd31/AR29aTBC8eR9cBdHS0cXKzW2sam5l1eY2Vm9pY03hJ/88f6VUSs9d\ntyxgZG0lI4dVMrK2gsMm1vOqw8YzqaGaxpG1NI6ooXFkDfXV5UR43xyVrog4BfgakAOuTCl9fqf5\nVcA1wFHABuDslNLThXkfAy4GuoH3ppRuK2LpL1pPT+LKu5fwxdseZ2RtJddcdAwnHDQ267IkSZKk\nQcHwSpIkSYNOdUWOb593FO/4/gI+ctODRMBZ8/oWYKWU2NbelQ+hmtufCaVWbm5lVfPfHptbO5+z\n7sjaCsbXVzOhoZrDJzYwvqGaCfXVjB1exahCUDVqWCX11RWUlRlKaWCLiBxwOfAqoAmYHxE3p5Qe\n6bXYxcCmlNKBEXEO8AXg7Ig4DDgHOByYBNwREQellLqL+y72zsrNrXzg+ge4Z8kGTj58PJ87Yzaj\nhlVmXZYkSZI0aBheSZIkaVCqrshxxXlHcck19/HhGx/kyXXbqK+uoL2zm/aunl4/+dcbt3U8c9VU\nS8dz/34+oraCiQ01NI6oYd60kUxsqGHSiGomNdQwsaGGcfVVDhWmoeYY4MmU0hKAiLgWOA3oHV6d\nBvxb4fkNwDcifznhacC1KaV24KmIeLKwvXuKVPte+8WDK/n4TYvp6klcduZszpo32SskJUmSpH5m\neCVJkqRBq7oix3fOn8e7frCAb9+55JnpVeVlVJWXUVmee+b5iNoKDpk4nBMPHsuEwtVT4+vzV06N\nr6+mptJgStpJI7C81+sm4NjdLZNS6oqIZmB0Yfq9O63buPMOIuIdwDsApk7N7v51O7R1dnPZrY+z\n/9g6vnr2EUwbMyzrkiRJkqRByfBKkiRJg1p1RY7vXnA029q7qCwvozJX5lUSUv/Y1f9IO9/lbXfL\n9GVdUkpXAFcAzJs3bxd3kCuu6oocP/yHY5nQUE1FrizrciRJkqRBy9+2JUmSNOhFBMOrK6gqzxlc\nSf2nCeh9M7nJwMrdLRMR5UADsLGP65akKaNqDa4kSZKkfczfuCVJkiRJe2M+MCMipkdEJXAOcPNO\ny9wMvL3w/E3Ab1JKqTD9nIioiojpwAzgz0WqW5IkSVKJc9hASZIkSdIeK9zD6p+B24AccFVK6eGI\n+DRwX0rpZuB/gO9HxJPkr7g6p7DuwxFxPfAI0AW8O6XUnckbkSRJklRyDK8kSZIkSXslpXQLcMtO\n0/611/M24KzdrPtZ4LP7tEBJkiRJA5LDBkqSJEmSJEmSJKlk9Cm8iohTIuLxiHgyIj66i/n/LyIe\niYgHI+LXEbFf/5cqSZIkSZIkSZKkwe4Fw6uIyAGXA68BDgPOjYjDdlrsfmBeSmk2cANwWX8XKkmS\nJEmSJEmSpMGvL1deHQM8mVJaklLqAK4FTuu9QErptymllsLLe4HJ/VumJEmSJEmSJEmShoK+hFeN\nwPJer5sK03bnYuBXu5oREe+IiPsi4r5169b1vUpJkiRJkiRJkiQNCX0Jr2IX09IuF4x4GzAP+OKu\n5qeUrkgpzUspzRs7dmzfq5QkSZIkSZIkSdKQUN6HZZqAKb1eTwZW7rxQRLwS+BfgxJRSe/+UJ0mS\nJEmSJEmSpKGkL1dezQdmRMT0iKgEzgFu7r1ARBwJfBt4Q0ppbf+XKUmSJEmSJEmSpKHgBcOrlFIX\n8M/AbcCjwPUppYcj4tMR8YbCYl8E6oCfRMSiiLh5N5uTJEmSJEmSJEmSdqsvwwaSUroFuGWnaf/a\n6/kr+7kuSZIkSZIkSZIkDUF9GTZQkiRJkiRJkiRJKgrDK0mSJEmSJEmSJJUMwytJkiRJkiRJkiSV\nDMMrSZIkSZIkSZIklQzDK0mSJEmSJEmSJJUMwytJkiRJkiRJkiSVDMMrSZIkSZIkSZIklYxIKWWz\n44h1wNJMdv5cY4D1WRehIcFjTcXk8aZi8VhTMXm8lbb9Ukpjsy5Cg5M9pIYojzUVk8ebisVjTcXk\n8VbadttDZhZelZKIuC+lNC/rOjT4eaypmDzeVCweayomjzdJpcDvIhWLx5qKyeNNxeKxpmLyeBu4\nHDZQkiRJkiRJkiRJJcPwSpIkSZIkSZIkSSXD8CrviqwL0JDhsaZi8nhTsXisqZg83iSVAr+LVCwe\nayomjzcVi8eaisnjbYDynleSJEmSJEmSJEkqGV55JUmSJEmSJEmSpJIxpMOriDglIh6PiCcj4qNZ\n16PBJSKmRMRvI+LRiHg4It5XmD4qIm6PiCcKjyOzrlWDQ0TkIuL+iPhF4fX0iPhT4Vi7LiIqs65R\ng0NEjIiIGyLiscJ33PF+t2lfiIj3F/4NfSgifhwR1X63ScqSPaT2JXtIFZs9pIrFHlLFYg85uAzZ\n8CoicsDlwGuAw4BzI+KwbKvSINMFfCCldChwHPDuwjH2UeDXKaUZwK8Lr6X+8D7g0V6vvwB8pXCs\nbQIuzqQqDUZfA25NKR0CzCF/3Pndpn4VEY3Ae4F5KaWZQA44B7/bJGXEHlJFYA+pYrOHVLHYQ2qf\ns4ccfIZseAUcAzyZUlqSUuoArgVOy7gmDSIppVUppYWF51vJ/8PcSP44u7qw2NXA6dlUqMEkIiYD\npwJXFl4H8ArghsIiHmvqFxFRD5wA/A9ASqkjpbQZv9u0b5QDNRFRDtQCq/C7TVJ27CG1T9lDqpjs\nIVUs9pAqMnvIQWQoh1eNwPJer5sK06R+FxHTgCOBPwHjU0qrIN+cAOOyq0yDyFeBDwM9hdejgc0p\npa7Ca7/j1F/2B9YB3y0MMXJlRAzD7zb1s5TSCuBLwDLyDUczsAC/2yRlxx5SRWMPqSKwh1Sx2EOq\nKOwhB5+hHF7FLqalolehQS8i6oAbgUtTSluyrkeDT0S8DlibUlrQe/IuFvU7Tv2hHJgLfCuldCSw\nHYd30D5QGPP+NGA6MAkYRn6orp353SapWPz9SkVhD6l9zR5SRWYPqaKwhxx8hnJ41QRM6fV6MrAy\no1o0SEVEBfmm44cppZsKk9dExMTC/InA2qzq06Dxd8AbIuJp8sPXvIL8WXQjCpdJg99x6j9NQFNK\n6U+F1zeQb0T8blN/eyXwVEppXUqpE7gJeAl+t0nKjj2k9jl7SBWJPaSKyR5SxWIPOcgM5fBqPjAj\nIqZHRCX5m7fdnHFNGkQK40X/D/BoSuk/e826GXh74fnbgZ8XuzYNLimlj6WUJqeUppH/LvtNSumt\nwG+BNxUW81hTv0gprQaWR8TBhUknAY/gd5v63zLguIioLfybuuNY87tNUlbsIbVP2UOqWOwhVUz2\nkCoie8hBJlIaulfJRcRryZ9ZkgOuSil9NuOSNIhExEuB3wOL+dsY0h8nP2b59cBU8l+qZ6WUNmZS\npAadiHg58MGU0usiYn/yZ9GNAu4H3pZSas+yPg0OEXEE+Rs7VwJLgAvJnxDjd5v6VUR8Cjgb6CL/\nPfYP5Mcn97tNUibsIbUv2UMqC/aQKgZ7SBWLPeTgMqTDK0mSJEmSJEmSJJWWoTxsoCRJkiRJkiRJ\nkkqM4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGV\nJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJ\nkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJK\nhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJ\nkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIk\nSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobh\nlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIk\nSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmS\nSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUk\nSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmS\nJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG\n4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmS\nJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJ\nkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGV\nJEmSJEmSJEmSSobhlSRJkiRJkiRJkkqG4ZUkSZIkSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJ\nkiRJkkqG4ZUkDSERMS0iUkSUF17/KiLe3pdl92JfH4+IK19MvZIkSZKkF89eUJI00BheSdIAEhG3\nRcSndzH9tIhYvafNRUrpNSmlq/uhrpdHRNNO2/6PlNI/vNht72JfF0TE3f29XUmSJEkqVfaCz9ln\niogP76t9SJKyZ3glSQPL94DzIiJ2mn4e8MOUUlfxS5IkSZIk7WPfw15wh7cDGwuPRbW3V6NJkvac\n4ZUkDSw/A0YBL9sxISJGAq8Drim8PjUi7o+ILRGxPCL+bXcbi4jfRcQ/FJ7nIuJLEbE+IpYAp+60\n7IUR8WhEbI2IJRHxzsL0YcCvgEkRsa3wMyki/i0iftBr/TdExMMRsbmw30N7zXs6Ij4YEQ9GRHNE\nXBcR1Xv64RT2e3NEbIyIJyPikl7zjomI+wqfy5qI+M/C9OqI+EFEbCjUNj8ixu/pviVJkiRpH7IX\nzC9fC7wJeDcwIyLm7TT/pRHxx8K+lkfEBYXpNRHx5YhYWtjP3YVpz7lyrFDTKwvP/y0ibij0jFuA\nCwq95T2FfayKiG9ERGWv9Q+PiNsLfemayA+jOCEiWiJidK/ljoqIdRFRsbv3K0lDmeGVJA0gKaVW\n4Hrg/F6T3ww8llJ6oPB6e2H+CPJNxz9GxOl92Pwl5BufI4F55BuC3tYW5tcDFwJfiYi5KaXtwGuA\nlSmlusLPyt4rRsRBwI+BS4GxwC3A//b+Bb/wPk4BpgOzgQv6UPPOfgw0AZMK9f9HRJxUmPc14Gsp\npXrgAPKfI+TP1msApgCjgXcBrXuxb0mSJEnaJ+wFn3EmsA34CXAbvT6PiJhKPkz7emFfRwCLCrO/\nBBwFvIR8CPhhoOf5PpReTgNuIP+5/hDoBt4PjAGOB04C/qlQw3DgDuBW8n3pgcCvU0qrgd8V3usO\nbwOuTSl19rEOSRpSDK8kaeC5GjgrImoKr88vTAMgpfS7lNLilFJPSulB8o3CiX3Y7puBr6aUlqeU\nNgKf6z0zpfTLlNJfU96dwP/R66y/F3A28MuU0u2FX8y/BNSQbxx2+K+U0srCvv+XfKPRZxExBXgp\n8JGUUltKaRFwJflhNAA6gQMjYkxKaVtK6d5e00cDB6aUulNKC1JKW/Zk35IkSZJUBPaC+ZMPr0sp\ndQM/As7tdeXSW4E7Uko/Til1ppQ2pJQWRUQZcBHwvpTSikLf98eUUnsf38M9KaWfFT7X1kLPeG9K\nqSul9DTwbf72Ob8OWJ1S+nKhL92aUvpTYd7V5AMrIiIHnAt8v481SNKQY3glSQNMSuluYB1wWkTs\nDxxN/pd2ACLi2Ij4bWH4gWbyVxKN6cOmJwHLe71e2ntmRLwmIu4tDH2wGXhtH7e7Y9vPbC+l1FPY\nV2OvZVb3et4C1PVx2733sTGltLXXtKW99nExcBDwWGFowNcVpn+f/Bl710bEyoi4zGEbJEmSJJWa\nod4LFk5Y/HvyVz8B/Byo5m/DHE4B/rqLVccUltvVvL7o/dkQEQdFxC8iYnVhKMH/4G+fx+5q2FHv\nYYX/dq8CmlNKf97LmiRp0DO8kqSB6RryZ9mdB/xfSmlNr3k/Am4GpqSUGoD/Bna+qe+urCL/i/YO\nU3c8iYgq4EbyZ8mNTymNID/cw47tphfY9kpgv17bi8K+VvShrr5aCYwqDNOww9Qd+0gpPZFSOhcY\nB3wBuCEihhXOyPtUSukw8mf/vY5nD8UhSZIkSaViKPeC55H/W+b/RsRqYAn5UGpH/7ac/BDxO1sP\ntO1m3nagtld9OfJDDva283v8FvAYMKMwLP3H+dvnsbsaSCm1kR/68a2F9+JVV5L0PAyvJGlgugZ4\nJfmxya/ead5w8lcgtUXEMcBb+rjN64H3RsTkyN/496O95lUCVeTP8uuKiNcAr+41fw0wOiIanmfb\np0bESYWrmj4AtAN/7GNtO4uIqO79k1JaXtje5wrTZpO/2uqHhRXeFhFjC2f6bS5spzsi/j4iZhWa\nlC3khxHs3su6JEmSJGlfGsq94PnAp8gPK7jj58zC9keT7/1eGRFvjojyiBgdEUcUesCrgP+MiEkR\nkYuI4wvB3F+A6og4tVDfJwrv9/kMJ987bouIQ4B/7DXvF8CEiLg0IqoiYnhEHNtr/jXk7+n1BuAH\ne/EZSNKQYXglSQNQYVztPwLDyJ9Z19s/AZ+OiK3Av5JvFvriO+SHz3sAWAjc1Gt/W4H3Fra1iXwT\ndHOv+Y+RH099SURsjohJO9X7OPmxvb9O/qy31wOvTyl19LG2nb0EaO39ExHl5McMn0b+7L6fAp9M\nKd1eWOcU4OGI2AZ8DTincObbBPI3390CPArciU2EJEmSpBI0VHvBiDiOfK93eUppda+fm4EngXNT\nSsvID2n4AWAjsAiYU9jEB4HFwPzCvC8AZSmlZvKf25XkrwbbDjS9QDkfLHwOW8l/dtf1er9byQ8J\n+HrywyE+QX6owx3z/wD0AAsL/y0lSbsRKb3Q1b2SJEmSJEmSpBcrIn4D/CildGXWtUhSKTO8kiRJ\nkiRJkqR9LCKOBm4nf1+yrVnXI0mlzGEDJUmSJEmSJGkfioirgTuASw2uJOmFeeWVJEmSJEmSJEmS\nSoZXXkmSJEmSJEmSJKlkGF5JkiRJkiRJkiSpZJRnteMxY8akadOmZbV7SZIkSf1swYIF61NKY7Ou\nQ4OTPaQkSZI0uDxfD5lZeDVt2jTuu+++rHYvSZIkqZ9FxNKsa9DgZQ8pSZIkDS7P10M6bKAkSZIk\nSZIkSZJKhuGVJEmSJEmSJEmSSobhlSRJkiRJkiRJkkpGZve8kiRJkga7zs5OmpqaaGtry7qUflVd\nXc3kyZOpqKjIuhRJkiRJGnQGWy+5Nz2k4ZUkSZK0jzQ1NTF8+HCmTZtGRGRdTr9IKbFhwwaampqY\nPn161uVIkiRJ0qAzmHrJve0hHTZQkiRJ2kfa2toYPXr0gG82eosIRo8ePWjOAJQkSZKkUjOYesm9\n7SENryRJkqR9aDA0GzsbjO9JkiRJkkrJYOq79ua9GF5JkiRJg1hdXV3WJUiSJEmStEcMryRJkiRJ\nkiRJklQyDK8kSZKkISClxIc+9CFmzpzJrFmzuO666wBYtWoVJ5xwAkcccQQzZ87k97//Pd3d3Vxw\nwQXPLPuVr3wl4+olSZIkScV2+umnc9RRR3H44YdzxRVXAHDrrbcyd+5c5syZw0knnQTAtm3buPDC\nC5k1axazZ8/mxhtvfNH7Ln/RW5AkSZL0wi69FBYt6t9tHnEEfPWrfVr0pptuYtGiRTzwwAOsX7+e\no48+mhNOOIEf/ehHnHzyyfzLv/wL3d3dtLS0sGjRIlasWMFDDz0EwObNm/u3bkmSJElS32TYS151\n1VWMGjWK1tZWjj76aE477TQuueQS7rrrLqZPn87GjRsB+MxnPkNDQwOLFy8GYNOmTS+6RMMrSZIk\naQi4++67Offcc8nlcowfP54TTzyR+fPnc/TRR3PRRRfR2dnJ6aefzhFHHMH+++/PkiVLeM973sOp\np57Kq1/96qzLlyRJkiQV2X/913/x05/+FIDly5dzxRVXcMIJJzB9+nQARo0aBcAdd9zBtdde+8x6\nI0eOfNH7NrySJEmSiqGPV0jtKymlXU4/4YQTuOuuu/jlL3/Jeeedx4c+9CHOP/98HnjgAW677TYu\nv/xyrr/+eq666qoiVyxJkiRJyqqX/N3vfscdd9zBPffcQ21tLS9/+cuZM2cOjz/++HOWTSkREf26\nf+95JUmSJA0BJ5xwAtdddx3d3d2sW7eOu+66i2OOOYalS5cybtw4LrnkEi6++GIWLlzI+vXr6enp\n4cwzz+Qzn/kMCxcuzLp8SZIkSVIRNTc3M3LkSGpra3nssce49957aW9v58477+Spp54CeGbYwFe/\n+tV84xvfeGZdhw2UJEmS1CdvfOMbueeee5gzZw4RwWWXXcaECRO4+uqr+eIXv0hFRQV1dXVcc801\nrFixggsvvJCenh4APve5z2VcvSRJkiSpmE455RT++7//m9mzZ3PwwQdz3HHHMXbsWK644grOOOMM\nenp6GDduHLfffjuf+MQnePe7383MmTPJ5XJ88pOf5IwzznhR+4/dDR+yr82bNy/dd999mexbkiRJ\nKoZHH32UQw89NOsy9oldvbeIWJBSmpdRSRrk7CElSZI0VAzGXnJPe0iHDZQkSZIkSZIkSVLJMLyS\nJEmSJEmSJElSyTC8kiRJkiRJkiRJUskwvHrLW+AHP8i6CkmSJA1SWd1jdl8ajO9JkiRJkvaJlOAb\n34Af/nAPVxs8fdfevJfyfVDHwPLzn8OkSfC2t2VdiSRJkgaZ6upqNmzYwOjRo4mIrMvpFyklNmzY\nQHV1ddalSJIkSVI2UoLLL4e//hXGjoVp0+AVr4AJE/62zPr1UF0NH/kIfPObUFYG7e1w4olwwAHP\nu/k+95I9PdDSAp2dMHw4lJde5LO3PWTpvZNiq6rKHzCSJElSP5s8eTJNTU2sW7cu61L6VXV1NZMn\nT866DEmSJEnKxne+A+95Tz6camvLTxs1ChYtgpoauPBC+MUvICIfdL3vffDrX8PFF+dDrDPPzM/7\nJyJ6BwAAIABJREFU8IfhqKPy62/aBK2t8LGPMfnhh2n66ldZN3r0s/fb3p5fr7IStm+HjRvzAdYO\n48bl979DSvnlM7Y3PaThleGVJEmS9pGKigqmT5+edRnSPhMRpwBfA3LAlSmlz+80vwq4BjgK2ACc\nnVJ6OiIqgCuBueT70mtSSp8ravGSJEnS3lizBi69FF71Krj1VujogAUL4OST4eyzobYW7r4bPv7x\n/LxTToGTToJt22D+fLj+evjJT/Kh0403wrXXwsteBnPnwsqVAFTMmMH0l70Mbrklv/6dd8KYMXDs\nsfltHn10Pig78kj44AfzV3y9972wdCn87Gfw0EPw5S/D8uX5Zd/znnzY9etfw/TpcM45MGtWxh/k\n8zO8qqzM/8eWJEmSJPVZROSAy4FXAU3A/Ii4OaX0SK/FLgY2pZQOjIhzgC8AZwNnAVUppVkRUQs8\nEhE/Tik9Xdx3IUmSJO2hK6/MXyH19a/nr6Kqroa/+7v89Isuys/71rfgXe969np1dfD3f5//+da3\nYPNm+P/Zu+8wqerz///PA0sVBQFBBQsCCtgVxfaxVxLBRpTYEo2osSTWGGOsXzWJURMjdo3B3o2J\nKLEnKvaGBZFgARWDSC/CLuf3x73zm50tsMDsni3Px3XNdc7MOXPmHmRXrnnNfb8HD4ajjoqxg999\nB5dcEiHW7rvDwIFxvSFD4KaboFWruF10Efztb7DJJjBmDHTqFNd/8EHYfvsYSwiw3XbR4XXffTB8\neDzWrRtMnw5bbGF41eDZeSVJkiRJK2JbYGKappMAkiS5FxgKVAyvhgIXlu8/CFybxND+FFglSZIS\noB2wCJhdT3VLkiRJK6asDG68EfbcEzbaqPDYYYfF4++8E51Wy9KpEzzyCBxxRARXd9wBhxySP37P\nPbD//hFc7bsvvPEG/PrXcPrpcM45VUcC9u4da3A9+GDs/9//xfFLLoH33ouQbZNNGk0eYnhleCVJ\nkiRJK6IHMLnC/SnAoJrOSdO0NEmSWUAXIsgaCnwNtAdOS9P0uzqvWJIkSVoZI0fGKL5rrqn+eNeu\nEWDVVvfu8NRT1R/bdFP48EN49tkYSdiyZXR65VS3ltWqq8Z6WxW1aROjA3Patq19fRlqsexTmjjD\nK0mSJElaEdWt/JzW8pxtgTJgbaAXcEaSJBtUeYEkGZEkyRtJkrwxbdq0la1XkiRJWnGffRYdT4MH\nw9Ch9fOa7dvDD38Y4wJbNK84p3m92+q0aeOaV5IkSZK0/KYA61S43xP4qqZzykcEdgS+A34MPJmm\n6eI0Tf8HvAQMrPwCaZrelKbpwDRNB66xxhp18BYkSZKkWnrggVjPauTI6rueVFSGV61b23klSZIk\nScvvdaBvkiS9kiRpDRwGPFbpnMeAo8v3DwGeTdM0Bb4Adk/CKsB2wPh6qluSJElafs88A/37w/rr\nZ11Js2B45dhASZIkSVpuaZqWAicDY4CPgPvTNP0gSZKLkyQZUn7arUCXJEkmAqcD55Q/PhLoALxP\nhGB/TdP0vXp9A5IkSVJtLVoE//kP7LFH1pU0GyVZF5A5wytJkiRJWiFpmo4GRld67PwK+wuBYdU8\nb251j0uSJEmZGzsWysqga1dYe21YbTV45RWYP9/wqh4ZXhleSZIkSZIkSZKkefNg552htDTul5TA\n4YdHmNW2Ley6a6blNSeGV23aRMufJEmSJEmSJElqXKZNi9Bp3XWhxUqulDRhQgRX55wDm20GL7wA\nN94Yx846Czp1Wvl6VSuuedW6tZ1XkiRJkiRJkiQVw+TJEQD9/e/w2mv5x6dNg/vvh8WLV/413nsP\nTjsNjjkGuneHXr1g4EAYNQq++GLFrzthQmx//GMYPhyuvz46r7p3j0BL9cbOK8cGSpIkSZIkSZK0\ncsrK4PTT4ZprYIMNYNIkWGMNGD8+1ow64giYMQO23RYOPTTO79cPdtsNOnQovNZXX8Epp0TQ1b17\n3Lp1g003hQ8+gDPPjOenKZxwQlznyivh6KPj+e3bQ+/ecN990L9/7d/Dxx/Htk+f2CYJ3HFHrHe1\nyior/2ekWjO8MrySJEmSJEmSJGnl3HlnBFfDh8Pzz8P++8Po0bD33vD++zBgAFx8MVx+OZxxRv55\nrVrBVlvB9tvDkCExLe3882HsWNhwQ3j99ejaKivLP2fwYLj9dlh99ViXCuDkk2HcuHjtzz+Hu+6C\nXXaJTqy2bWv3HiZMiPGD7drlH0sSg6sMGF4ZXkmSJEmSJEmStOLKyuCyy2CLLSI0ggh9brwRrr4a\nBg2Chx+GLl3gpJNg+vQInd58E558MrbXXQd/+lP+mrfcAsceG/tLlsRznn8+9n/0o7h+RS1awOab\nxw2ie+qkk2DmTFhzzdq9j48/ho02Wqk/ChWH4VWbNrBoUbQXVv7LLkmSJEmSJElSc/TEE3DRRRFG\n9e5d83lz50bINGECPPhg4efsxx8ft4qSBLp2jf099ogbwLffwttvx36XLtGNldOiRYwgHDas9vW3\nbh3b2q6xlabxHo48svavoTpjeFXxL3BuX5IkSZIkSZKk5uzuu+HVV2H33WPs36qr5o+laWyTBK64\nAh54IMYBHnTQir9e166w114rV3NFrVrFtrS0dudPngyzZ8eoQmWuRdYFZK5Nm9g6OlCSJEmSJEmS\npPDaa9CjR6wZ9a9/FR7bbz9Ye+0Y83fnndE9dc45DWu6WS68qm3n1Y03Rv377Vd3NanWDK8MryRJ\nkiRJkiRJypsxI0bojRgBq60GY8ZEB9PNN8NNN8X99u3htNNg0iQ44oisK65qecKrOXNiza2DDoK+\nfeu2LtWKYwNz4dWiRdnWIUmSJEmSJElSQ/DGG7HdcUd49114/HHYe2947rl4fI01Yn2qAw+EN9+M\nbUNTUh5/1Ca8+sc/YOZM+OUv67Ym1ZrhlZ1XkiRJkiRJkiTFGlczZ8KDD8b9gQNhn33g4Yfhu+9g\n5MgIq/bcMzqynnwSvv029hua5em8GjMGunSB7bev25pUa4ZXrVvH1vBKkiRJkiRJktSUffttjMjr\n1SvuL16cD3leeCG6q3JTyn70I+jYMbZvvw0nngibbVZ4vVatYK216q/+5VFdePXPf8Kmm8J66+Uf\nW7Ikwqu99oKWLeu3RtXI8MrOK0mSJEmSJElSc3DSSRHUjBsHX3wBe+wRnVQ9e8LBB0Pv3vCHP0Cn\nTjEyEGL/+uuzrXtFVBde7b9/bNM0/9h778E330SHmRoMwyvDK0mSJEmSJElSc/DiizBrFhx7LGy1\nVXwufs890KIFTJ8OTz0FW26ZdZXFUTm8WrIkfyxNIUli//HHY2t41aAYXuXCq1wrpCRJkiRJkiRJ\nTc2UKfDVV9C/f4RU48bF46NGxXpWP/pR0wmuoGp4NX9+/tiHH8LGG8f+ww/Ddts13PGHzVSLrAvI\nnJ1XkiRJkiRJkqSm7tVXY3vFFVBSAlOnQo8eMHkytGsX4wKbklx4VVoa24rh1dNPx/bzz+Gtt+Cg\ng+q3Ni2T4VXr1rE1vJIkSZIkSZIkNVWvvRafh++5J+y3Xzx25ZWxDtSjj8J662VbX7EtrfPq3/+O\n7T/+EdsDD6y/ulQrjg2080qSJEmSJEmS1NSNHQtbbBGfiZ95JsybB0OGwKGHZl1Z3Sgpjz+qC68+\n+CC2r74a3Wd9+tRvbVomO68MryRJkiRJkiRJTdns2RFe7b573N95Z3jmmRgX2FTV1Hm12WYwcWJk\nAm+/HYGeGhzDq1x4tWhRtnVIkiRJkiRJklQXnn021n7ad9+sK6k/NYVXAwdCWRmMGwfjx8OWW2ZT\nn5bK8MrOK0mSJEmSJElSU/bkk7DqqrDDDllXUn+WFl4B3H9/hFh2XjVIhletW8fW8EqSJEmSJEmS\n1BQ99xzstls+0GkOagqvttwSkgTuvjt/Xw2O4ZWdV5IkSZIkSZKkpuybb2D99bOuon7VFF517Qqd\nO8OXX0KnTs3vz6WRMLxyzStJkiRJkiRJUlOVpjB7Nqy2WtaV1K+awqv27WHYMOjZE+65B1oYkzRE\n/ldxbKAkSZIkSZIkqamaOzcCrI4ds66kfuXCq9LS2M6bF9v27eH662HyZNh332xq0zIZXiVJBFiG\nV5IkSZIkSZKkpmb27Ng2t86rkpLYVtd5pQbP8AoMryRJkiRJkiRJTdOsWbFtbp1XLVtG80rF8Kpl\ny3xHlho0wyuIda8MryRJkiRJkiRJTU1z7byCCKoqhlft20egpQbP8AoivFq0KOsqJEmSJEmSJEkq\nrubaeQXVh1dqFAyvwM4rSZIkSZIkSWqqxo2Dl1/Ouors2HkV+4ZXjUpJ1gU0CO3a5RdrkyRJkiRJ\nkiQ1HdtuCwsXwowZ0KlT1tXUPzuvYt/wqlGx8wriF1buB1iSJEmSJEmS1HQsXBjbv/wl2zqyYudV\n7BteNSqGVxDh1cyZWVchSZIkSZIkSSqmJUsiwIAIr9K08NjRR8Pdd2dTW33JNW6sumq2dWShVSso\nLY19w6tGxfAKol3S8EqSJEmSJEmSmpavv47Om4EDYdo0mDAhf+zf/4ZRo+DII+HHP4brrotAq7EZ\nNQquvRa+/bb647NnR3DVohnGAZU7r1ZZJdt6VGuueQV2XkmSJEmSJElSU/T557E9/HB44w146SV4\n7z247TZYtAg6dIBdd4Xnn4d77oFnnoEHH4QkybLqQvPmwS23wJQpcNllsGBB7I8aBZ99BvfdF+c9\n9BA891zV58+a1TzXuwIoKcmHV/Pm2XnViBheQX7NqzSt+ZfS009Dz57Qr1/91iZJkiRJkiQ1J88+\nC3vsAV9+CWuvnXU1auw++yy2e+8NnTvDuefCN99Ay5ZQVhZdV6NGxWfDv/tdHL/2Wjj55PxnxbNm\nwZ13wk47weab1/610zRuK9Lx9P33MGkSfPgh/OpX8N//xuOjR8djEMFMmzbw85/D1KkwblzV1x81\nKoKu5rjeFbjmVSPWDPsEq9GxY/wFXrCg+uMXXwx77RW/mEaNqt/aJEmSJEmSpObkj3+M7VtvZVuH\nmoZc59V668EOO0RwtddeMT7wwAPhzDPjeJLAOedEyHXqqRF0rbkmDBsG664bYdYuu8Arr9Tudf/4\nx/jceb314H//q/6cJUvilqbw6qtw2mkwaBCcdRastRYMGACHHBKfXT/3HFx0EXz6KZxxBtxwA3zx\nBcydCyNHQrduVaeLTZwIP/kJPPVU8+28MrxqtOy8gui8gvjhrvyXd8YMuOACOPjg+LbHr34V809L\n/KOTJEmSJEmSii73AXxz7RRRcX32GXTtGmsd7bVXjAUcORI22AAefrjw3CSJkYF33hldTNOnw7/+\nBfvuC0cfDSedFAHYeedFw0NF8+dHkLRkSTzv3HNhu+3gtdfgRz+K/Vdegcsvh802i66vIUMiRFtr\nrQhrW7eGDTeM4GvbbeGUU6BPnzi/ffsYb3juudV/Nt2xY/zsVJwu9uWX+ePN9efJ8KrRMoGBwvCq\ncivyJ5/E9sgj4xfPQQfFCMF9963fGiVJkiRJkqTmYNas2JaVZVuHmobPP4f114/9k06Kta+6dKn5\n/FVXhRNPrP7YW2/FiL5LLoEDDohJXS+/DP/5T4RZ33+fP7dDB7j/fnj00eja+s9/IkDad99Ya2vx\n4vi8ebvtYPbs6KQ69NAIocaPh759qw+pamqq6NQpP10sF9B8/XX+eHPvvEpTw6tGxvAKCsOrynLh\nVd++0Lt3tIuOGhW/ZD77DH7zm0jqc9eQJEmSJEmStOJyn9EtXJhtHWoaPv44upgg1rlaWnC1LKuv\nHiHT00/DccfFOMFHH41jQ4dGZ1eLFtHxNGhQjB084YQY3deiBUyeDPvvDwMHRkg2aBAcdVTV1+nf\nf/lry30+PWtWPqCZOjV/fM6c5b9mU5ALr77/PgIsw6tGw/AK8qlz7lsdFX3ySbRZbrBBLH43dCj8\n/e/xF/2qq+DuuyMdP+WU+q1ZkiRJkiRJaooMr1QsX38dDQjF/Oy2Y8cIsH760+jEuvTSGP+38cb5\ncX2VtW0b29694cMPi1dLRRUbNNZaK/Yrdl6NH183r9vQtWoVHVfz5sV9w6tGo0XWBTQIy+q8Wnfd\n/C+YHXaA776Lmad33BGP3XprhFmSJEmSJEmSVs78+bE1vNLKGjs2tttvX9zrHnwwfPpprGF17rmw\nySY1B1f1JfcZ9+23x7pZM2dGeJULsn7848xKy1SrVlBamv/s3wlqjYbhFSw7vOrbN39/u+1ie/bZ\ncf4hh8C778Kbb9Z9nZIkSZIkSVJTVvEL4oZXWh4ffQQ/+1mMxzvqqBjRN3YstG4NW21V/Nfr0iXG\n/jUUueliV14Zn2lffnmEV+utF+tgXXJJtvVlpaQkxgbOmBH3V18923pUa4ZXkP/BrhxepWnV8Kp/\n/5hHOmYM9OsHN90Ui+9dc0391StJkiRJkiQ1RXPn5vcNr1QbpaURztx1V0zIuv/+mJj1z3/Cyy/H\n+lJt2mRdZd3LNWjkOsD+/Gf44IPovGrbNvvOsKzk1rwyvGp0DK8gfnhbty4Mr2bPjlmoM2cWhlct\nW+YX+PvVr+Iv+09/CvfeWzhDVJIkSZIkSdLymTo1v294pdo44IBYx+idd+J+blTghx/GtKzcJK2m\nLhdelZbCaqvB99/Hz9Oaa2ZbV9YMrxotwyuI1LlTJ5g1K//YlVfCyJGxxtV++xWef9BB0WqamxN6\n6qlQVhbPkSRJkiRJkrRiDK+0vB5/PLb//ndsc+HV449HgLPFFtnUVd8qruU0eDB06xb7uTWvmivD\nq0bL8CqnU6d859X8+RFcDRkCL70U4wEr+vnPI7Vv3Tru9+kDhx8ez3n66fwPgiRJkiRJkqTa++ab\n/L7hlWqjc+fYzpkT2w8/jO2nn8Z2s83qv6Ys5KaLAfTsCXvvHfuGV4ZXjVStwqskSfZNkuTjJEkm\nJklyTjXH102S5LkkSd5OkuS9JEkGF7/UOtaxY4RXn34KO+8M06fDWWfV/vkXXBAtmXvtBYccUnd1\nSpIkSZIkSU1Vxc6r77/Prg41Hr161XyspKRqY0JTlSTxGTfA2mvDPvvEvuFVPrxq3Rratcu6ItXS\nMsOrJElaAiOB/YABwPAkSQZUOu084P40TbcEDgOuK3ahda5LF5g2Da67DsaNiwX+dtqp9s/v3Rv+\n8x/4xS/g2Wdjf/r0uM7LLxeeW1YWN0mSJEmSJEl5CxbEtmVLO69UO/Pm5ffXWafwWL9+0KZN/daT\npdzowLXXjgaLP/4R9tgj25qylguvZs6MP58kyboi1VJtOq+2BSamaTopTdNFwL3A0ErnpMBq5fsd\nga+KV2I96d0bJk6ECRNgww3z61ktj+22g8sug+7dYcQI2HFHOOKI2D77bJxz993QtSustx7cdFNx\n34MkSZIkSZLUmOW+8L3KKoZXqp05c6L76tRTYaut4rFNNont5ptnV1cWcuFVjx4xRvCMM2LbnFXs\nvHJkYKNSm/CqBzC5wv0p5Y9VdCFwRJIkU4DRwClFqa4+bbghzJ4dC/r16bPi12nfHu68M34YpkyB\nxx6LXxbnnQePPhprYw0YEGHZ8cfDUUdFaLYsU6fCP/8JabritUmSJEmSJEkNmeGVltfcubD//vDn\nP+c7r/bcMzqutt0229rqW8XOK4VWrWK5H8OrRqc24VV1fXSVE5ThwO1pmvYEBgN3JElS5dpJkoxI\nkuSNJEnemDZt2vJXW5f69o3ttGkrF15B/HKcMAHGj49fnOefH6HYgQfC1ltHF9azz8K558J998HG\nG8e4wuqUlUW4tcceca0RI2DJkpWrT5IkSZIkSWqISktja3il2kjT6Lzq0CHu9+wZ2002gfffhxNO\nyK62LOTCq+a+zlVFJSV2XjVStQmvpgAVh4X2pOpYwGOB+wHSNB0LtAW6Vr5QmqY3pWk6ME3TgWus\nscaKVVxXcuFV5f0Vtdpq+V+Wxx0XIdVpp8HDD0fq37IlXHopfPZZhF0nnQSjRhVeY/Fi2G23qOeT\nT2KU4S23wCOPrHx9kiRJkiRJUkOT67xq397wSsu2YEF80X/VVeN+rvOqZ89oUGjdOrvastC9e9za\ntcu6koajVav4OzJ9uuFVI1Ob8Op1oG+SJL2SJGkNHAY8VumcL4A9AJIk6U+EVw2stWoZ1l8/UlhY\n+c6rypIEfvQjuOoqWHfdwmNrrQV//3uEVD/5SayHddxxMSbw3HPhP//Jd26NGgUbbQS//S1ccUV0\nd9mFJUmSJEmSpKairCy+9N2uneGVlm3OnNjmwqvdd4dhw2C77bKrKUvnnQf/+lfWVTQsrVrFdto0\nw6tGZpnhVZqmpcDJwBjgI+D+NE0/SJLk4iRJhpSfdgZwXJIk7wL3AD9J00a2OFNJCWywQewXO7yq\nzWs/8ABccAEMHhxrZm22Gfzxj3DiiXDRRTFusGXLOOejj+Dss2PtrDZt4pzl+eOeOxdefBFmzoQ3\n38x/o+XJJyNImz+/bt6nJEmSJEmStDS58KpNG8MrLdvcubHNjQ1cc024/37o2DG7mrLUvXt8rqy8\nXHg1b57hVSNTUpuT0jQdDYyu9Nj5FfY/BHYsbmkZ2HBD+Pzz/Li/+tSlSwRTEGHU4MFwyCFwzTWF\n5x12WIRWq64aIwQnTIAbbojRgqefXrvXOvbY+CWe06sXHH88/PrXEYIdcQTccUdx3pckSZIkSZJU\nW7nwqm1bmDUr62rU0FXuvJIqy4VXYHjVyNQqvGo2fvrTSKZb1GaaYh3afnv46qv4n3SSFB5LEth8\n89i/7LIYGzhsGJxxBowfH2MI114bjjqq+tmmTz0VwdXRR0enWY8ecOWVcM450Lt3/DB/913dv0dJ\nkiRJkiSpsorh1f/+l3U1asgmT44bGF6pZoZXjZbhVUUHHRS3hqC2i+q1aAH33BPB29/+BosWxeOX\nXw733guDBsXxGTPgs8+ik6t3b7j++vxrHHoo/OEPEYIdd5zraEmSJEmSJCkbFcMrxwZqafbcExYs\niH3DK9WkYni1xhrZ1aHlZnjVFLRuDXfdFbc0hRdegGOOgZ13hh/8AB59NM5r2RIOPBBGjiwMxzp0\ngIsvjv0WLQyvJEmSJEmSlA3DK9VGaSn897/x9wXya15JlVUMr7bZJrs6tNwyno+noksS2HVXeOMN\nGDo0gqthw+CLL2Ic4AMPQLduNT/f8EqSJEmSJElZKS01vNKyffNNPrgCO69Us5IK/Tvdu2dXh5ab\nnVdNVefOEVS99x7071+YMC+N4ZUkSZIkSZKyUlYWHzYbXmlppkwpvG94pZrkwqttt822Di03w6um\nbrPNlu98wytJkiRJkiRlxbGBqo3K4ZVjA1WTzz+P7aBB2dah5ebYQBUyvJIkSZIkSVJWKodXaZp1\nRWqIKoZXrVvHTarO7rvHdsSIbOvQcrPzSoVatIDFi7OuQpIkSZIkSc1RxfAKYNEiaNMm25rU8FQM\nrxwZqKXZdltD8EbKzisVsvNKkiRJkiRJWakcXjk6MDsLFsBTT8HUqVlXUtXkydClS+wbXklNkuGV\nChleSZIkSZIkKSu58CrXbWV4VTxvvw3Dh8OOO8K//730c+++G3r0gL33hn794KGH6qfG2poyBTbZ\nBFZZxfWupCbK8EqFDK8kSZIkSZKUlcqdV99/n209TcVTT8HAgbH9/HM44AD46KPqz733Xjj8cOjf\nP0Kr/v3h0EPh5JPhjjvy5y1YAH/4A3z55YrXtWQJ/Pe/yz/WbcoUWGcdWHddO6+kJso1r1TI8EqS\nJEmSJElZKS11bGCxLFwISRJdbH/9a4zZ+/hjmDEDtt8+OrBuvz1G8L37LuyzD+y0U4RUgwbBCy9A\nSUl0Xx1wANxwA4wcCW++CUOHRpD117/GNZ5/Hrp1q31dCxdCx45w8cVw0UWw5Zbwk5/E6/3whxFK\nVTZ/ftQ0aVLUvP760X1V4kfcUlPkT7YKGV5JkiRJkiQpK2VlEUYYXq28ww+Hhx+GRx6B0aPhoINg\n9dXjNnYsDBkSIRTAaqvBzTdHd9bcuRFK5UKhDh3g6afjv82IEfDnP8cN4OCD4fHHI3y6+27YZRd4\n6aW4/imnRHA2Zw6ceipssQU89xw88QQsWgR9+8IXX0RgNns2/OIXcc3f/AY22ijqWGutCKiWLImw\nbMaMOGenneL6tQ3MJDU6hlcqZHglSZIkSZKkrFQeG2h4teIefTS2Bx4Y2/33zx/bYAN46y249Vbo\n2hUGD45w6Y034He/i1GBlbVsGedfdBGMHw/TpsGwYTBuXIwV3H13OPJIePBBmDcP7roL/vUvOPNM\nGDUqrtGxY3R2rbkm3HYbtGsH990XIdV//xvdVeecEyMJ1147OqxuvBEWL47w7ZhjooNsiy3suJKa\nOH/CVcjwSpIkSZIkSVkxvCqOhQtjHamzz44uqHHjYK+9Cs9p3RpOPDF//6GH4nbGGUu/ds+eccvZ\ncssYJXjWWdEdtdZa8NvfwgknQJ8+0VV1/vkxenDddSN8AjjttOiu6tQp7vfpE9vRowtfr6ws1j5r\n3375/xwkNVqGVypkeCVJkiRJkqSsVA6vFizItp7GorQ01qQ65JDoavr00wivNtsMLr0Uvvsuxv8t\nzWabxW1FrLpqvP6f/hSv265djPQ7/3w4+ugIyVq2LHxOSUk+uFqali0NrqRmqEXWBaiBMbySJEmS\nJElSVnLh1brrxv1PPsm2nsbi+edjDai11oJJk2IEH0Dv3hES1dfaUG3bRnAFsN9+8PrrMSawcnAl\nSctg55UKGV5JkiRJkiQpK2Vl0KoV9OgB3btH+KFlmzQpv/+Xv8D668d+796ZlCNJK8vOKxUyvJIk\nSZIkSVJWcp1XSQLbbGN4VVuffRYdVoMHw4MPwsSJMcqva9esK5OkFWJ4pUKGV5IkSZIkScpKaWl+\nxNw228D48TBnTrY1NQaffhqjFg89FKZMgQceiK6rJMm6MklaIYZXKmR4JUmSJEmSpKyUlUWAHY3d\nAAAgAElEQVQHEcDAgZCm8NZb2dbUGHz2GfTqBUOGwCqrwDffQJ8+WVclSSvM8EqFDK8kSZIkSZKU\nldzYQICtt47t229nV09j8emnsc5Vp07wxhtwzTVw0UVZVyVJK6wk6wLUwBheSZIkSZIkKSsVw6tu\n3aBz5xgdqJrNnx+dVr16xf1+/eImSY2YnVcqlCSGV5IkSZIkScpGxfAqSSKE+eijbGtq6D7/PLbr\nr59pGZJUTIZXKmTnlSRJkiRJkrJSMbwC6N/fzqtl+fTT2OY6rySpCTC8UiHDK0mSJEmSJGWlcnjV\nrx/873/w3XfZ1dTQvfACtGoFm2ySdSWSVDSGVypkeCVJkiRJkqSsVBdeAXz8cTb1NAb/+Afssgus\ntlrWlUhS0RheqZDhlSRJkiRJkrJSWlp9eOXowELTp8NXX8HEibEm2P77Z12RJBWV4ZUKGV5JkiRJ\nkiQpK2VlUFKSv7/++tC6teFVZdttBz16wCOPxH3DK0lNjOGVChleSZIkSZIkKSuVxwaWlMAGG0SH\nkfJyfx6XXRZBVq9e2dYjSUVmeKVChleSJEmSailJkn2TJPk4SZKJSZKcU83xNkmS3Fd+/NUkSdYv\nf/zwJEneqXBbkiTJFvVdvySpAaocXgH06bPs8OqJJ+C//627uhqaHj1iO3Mm/PSn2dYiSXXA8EqF\nDK8kSZIk1UKSJC2BkcB+wABgeJIkAyqddiwwI03TPsDVwO8B0jS9K03TLdI03QI4EvgsTdN36q96\nSVKDtbTwKk2rf86rr8LgwbDRRjBiBHz9dd3XmbXFi2Pbvj0cemi2tUhSHTC8UiHDK0mSJEm1sy0w\nMU3TSWmaLgLuBYZWOmco8Lfy/QeBPZIkSSqdMxy4p04rlSQ1HjWFV/Pnw9Sp1T/nkkugc2c44QS4\n/XbYe+8Id/75Txg+HBYsyJ87c2a8RmMyYwZsuim89VbcT1P47rsI6l55BTp2zLY+SaoDhlcqZHgl\nSZIkqXZ6AJMr3J9S/li156RpWgrMArpUOudQagivkiQZkSTJG0mSvDFt2rSiFC1JauBqCq+g+tGB\nH3wAjz8Op50G114L998P778PP/4xDBsG994Lv/hFrAv1s59B9+4RdjUmH34Y7+mhh+L+3LlQWgp9\n+0aoJUlNkOGVChleSZIkSaqdyh1UAJXnOS31nCRJBgHz0zR9v7oXSNP0pjRNB6ZpOnCNNdZY8Uol\nSY3H8oZXt90GrVrB8cfH/aFD4aCD4MEHoX9/2HFHuPnmCIBuvRWSBO64o+YRhA3RN9/E9sUXY/vd\nd7Ht3DmbeiSpHhheqZDhlSRJkqTamQKsU+F+T+Crms5JkqQE6Ah8V+H4YTgyUJJUUXXh1XrrQUlJ\n1fBqypQIovbfH3JfckiSCK4WLowxe9ddB/vtB2+8AXPmwF/+ApMmxbi977+vn/e0snLh1WuvwaJF\nhleSmgXDKxUyvJIkSZJUO68DfZMk6ZUkSWsiiHqs0jmPAUeX7x8CPJum8VX3JElaAMOItbIkSQql\npRFUVVRSAr17w3vv5R979FFYZx2YNg2OPbbw/CSBNm1if7PNYPRo2HBD6NABDjwwwrEddoAf/rBu\n30ux5Nb6WrgQxoyBb7+N+4ZXkpowwysVMrySJEmSVAvla1idDIwBPgLuT9P0gyRJLk6SZEj5abcC\nXZIkmQicDpxT4RI7A1PSNJ1Un3VLkhqw3GdSlTuvAHbfHZ57Lt8t9corMS7w2Wejs6q2unaFCy+M\n8Orpp2HkSDjxxAiGPv54pd9CnfjmG2jbNkK5IUPg5JPjccMrSU1YybJPUbPSojzPTNP4H6IkSZIk\n1SBN09HA6EqPnV9hfyHRXVXdc58HtqvL+iRJjUxZWWyrC6/23Reuvz7Ws1p11Qib+vaF3XZb/tc5\n77zo1lpnnXwQ9PjjMHlyjBw8+OAVfw91YerUWPfrT3+KoG3ChHjc8EpSE2bnlQpVDK8kSZIkSZKk\n+rK08Gr33aPTavRouP9+eP116NdvxV9rrbVibGCHDnDMMfDll9CtG/zmNzG6sCH55htYc03YYw/Y\nccf846uvnl1NklTHDK9UKBdeOTpQkiRJkiRJ9Wlp4VWHDtF91bVrfOn6q6+gf/+Ve7077oCPPoJb\nb4Xp0+GGG2J04AknwOLFK3ftYvrmG+jePfY33ji27drFTZKaKMMrFTK8kiRJkiRJUhaWFl5BjPT7\n6KP8Uhcr03kFMX6wZ8/Y79QJDjgAzj03wqxf/KLq+TNnxrHcultz5sAVV8TaWXUlTWNsYOXwqkuX\nuntNSWoAXPNKhQyvJEmSJEmSlIVlhVetW0fn1aabwnvvrXx4VVmSwKWXxnpaV10Fe+8dgRZEiHTc\ncRGgPfBAjOx74gmYNSs6oF58Ebbaqrj1AMydCwsWxNhAyIdXrnclqYmz80qFDK8kSZIkSZKUhWWF\nVzk77BDbjTaqmzouuwy23hoOOwyuvx6+/RbuvjuCq113hTFj4Nln4aCD4PHHI1AbPBjGjy9+Ld98\nE9tc59U660THmOGVpCbOzisVMrySJEmSJElSFkpLY1uyjI8szzkHdtklQpy60KYN/OtfsN9+8POf\nxyjBxYthp53gqaciUFprrfznaGPGwG67wXbbwR/+ACNGLN/rpWl0faUpfP013HxzBGGtW8NLL8U5\nuc6rJIGhQyPEkqQmzPBKhQyvJEmSJEmSlIXadl6tt17c6lLnzjB2LLz1Fpx6Knz+Odx7bwRrPXoU\nntu/f4RMI0bA8cfHCMHx46FtW/jzn2Hy5BhxuOGG+ed88QX87ndxzVmzIvz6+GOYMiWO9+gB8+ZF\nINarV3SC5dxxR92+d0lqAAyvVMjwSpIkSZIkSVmobXhVX1q0gIEDI5hatCg6smrSu3d0aw0fHuHS\nwIERRu29d4wd7NYtrvO3v8ENN8Ds2dFFNWwYrLEGPPJIBFxnnw077wybb15/71OSGiDDKxUyvJIk\nSZIkSVIWGlp4lZMkSw+uclq2jLWxLrwwurH+/W/YffcYcThuHGyyCXz/fQRaAwbAaafBuuvGc6+6\nqk7fgiQ1NoZXKmR4JUmSJEmSpCw01PBqeZSURDAFEVpNmBDrU02eDBdfHKMEr7uucb9HSaoHhlcq\nZHglSZIkSZKkLDSF8Kqy3r3z27/9LdtaJKkRaZF1AWpgDK8kSZIkSZKUhaYYXkmSVojhlQoZXkmS\nJEmSJCkLhleSpHKGVypkeCVJkiRJkqQslJbGtsSVTiSpuTO8UiHDK0mSJEmSJGXBzitJUjnDKxUy\nvJIkSZIkSVIWDK8kSeUMr1TI8EqSJEmSJElZMLySJJUzvFIhwytJkiRJkiRlwfBKklTO8EqFDK8k\nSZIkSZKUBcMrSVI5wysVSpLYGl5JkiRJkiSpPhleSZLKGV6pkJ1XkiRJkiRJyoLhlSSpnOGVChle\nSZIkSZIkKQulpbEtKcm2DklS5gyvVMjwSpIkSZIkSVmw80qSVM7wSoUMryRJkiRJkpQFwytJUjnD\nKxUyvJIkSZIkSVIWDK8kSeUMr1TI8EqSJEmSJElZMLySJJUzvFIhwytJkiRJkiRlwfBKklTO8EqF\nDK8kSZIkSZKUBcMrSVI5wysVMrySJEmSJElSFgyvJEnlDK9UyPBKkiRJkiRJWSgtja3hlSQ1e4ZX\nKmR4JUmSJEmSpCzkOq9KSrKtQ5KUOcMrFTK8kiRJkiRJUhYcGyhJKmd4pUKGV5IkSZIkSfVn8WKY\nNy/rKhoGwytJUjnDKxUyvJIkSZIkSao/558PO+2UdRUNg+GVJKmc4ZUKGV5JkiRJkiTVn08/hXHj\n8sFNc2Z4JUkqZ3ilQoZXkiRJkiRJ9Wf+/Ahtpk7NupLsGV5JksoZXqmQ4ZUkSZIkSaoLd98NU6Zk\nXUXDk1vvyj8bwytJ0v/P8EqFDK8kSZIkSVKxLVoEhx8Ou+ySdSUNz/z5sZ08uW6uP2dOBIfffls3\n1y+m0tLYGl5JUrNneKVChleSJEmSJKnYch01kyZlW0dDVIzOq3vvha22gkMOyQdAAPfdB+usE8Hh\n2WevXJ31Iff3pKQk2zokSZmrVXiVJMm+SZJ8nCTJxCRJzqnm+NVJkrxTfpuQJMnM4peqemF4JUmS\nJEmSii0XSqiqle28+vZb+MlPYOZMeOghOO20ePzpp+Gww2DAABg6FO65B6ZPL0rJdSb396SF37eX\npOZumf8nSJKkJTAS2A8YAAxPkmRAxXPSND0tTdMt0jTdAvgL8HBdFKt6YHglSZIkSZKKzfCqZrnw\nqmLn1fTpcMwx8Oqry37+bbfB99/DP/4BJ58MI0fCxx/D1VfDmmvCc8/BJZfAwoURbM1swN85Lytz\nZKAkCahd59W2wMQ0TSelaboIuBcYupTzhwP3FKM4ZcDwSpIkSZIkFVvFUXYqVHFs4MSJcPDBsPXW\n8Ne/wnnn1fycSy+FG2+MkGrXXWHjjeP8Nm3gxBPhiSfg+OPj/qabRnB1553RjdVQGV5JksrVZoBs\nD6Bi3/IUYFB1JyZJsh7QC3h25UtTJgyvJEmSJElSsdl5VbNc59XLL8Oee0Zn1JZbxhpWjzwCn34K\nvXoVPufkk+H222N//fXhiitiv3t3+NnP4Npro+vq+OPzz7nqKujUCS64INYe22CDun5ny8/wSpJU\nrjadV0k1j6U1nHsY8GCaptX+iyRJkhFJkryRJMkb06ZNq22Nqk+GV5IkSZIkqdgMr6q3aFF0pbVr\nF/fnzoVnnolRf9dcE5/THHQQPPlkHF+yBH772wiuzj0X3nkHPvoIBg7MX/OKK+Ctt+Czz2CttQpf\n75hj4pq33VZ37+mtt+Drr2P/gw9geT4DNLySJJWrTefVFGCdCvd7Al/VcO5hwEk1XShN05uAmwAG\nDhxYUwCmLBleSZIkSZKkYjO8ql6u6+rss2H77WHnnfNBVs+eMRbwssti1N/EiXDPPfD//l+EUBdd\nBCXVfLTXtm10blWnZ0/Yb7+47llnxetffjmccgr07RvnTJoEd90V4waPOw5WXz3//DSFpLrvuZe7\n80448sjY79Qpusg6d47ALU0j1ErTCNxmzYrX79ULttkGVlkF7r47OsYkSc1ebcKr14G+SZL0Ar4k\nAqofVz4pSZKNgNWBsUWtUPXL8EqSJEmSJBWb4VX1cutdrb027LNP1eM/+xnssANsthn86lfw/POw\n445wyy1LD5GW5uKLo1Pr0kuj6+svf4nrPfVUdIINGRJ1pSn84Q/wk5/ASy9FV9gnn0TQdfnlEZyN\nGwcffhiB09Sp8MtfwqBBMHx4HOvXD0aNivW2AFq3js+cBgyArl1h1VXh7bfhoYfi+IYbxqhESVKz\nt8zwKk3T0iRJTgbGAC2B29I0/SBJkouBN9I0faz81OHAvWma2lHVmBleSZIkSZKkYjO8ql6u82qV\nVWo+Z8AAOP30/LpWl1224sEVxFpaRx8NV18dXV777BOh1MEHw4wZ0KcPPP54dE399Kdw5ZXRFbbu\nurDRRvDHP8ZowJ494Y47IuTK6dIlRhIOGJB/7PTT4ZtvIuxaY43qu7e++w4WLIgQzLGBkiRq13lF\nmqajgdGVHju/0v0Li1eWMpMLr8wgJUmSJElSsZSWZl1Bw5TrvGrffunn/e53ESy99hoceODKv+7V\nV8OLL8Yowl/9KkYE7rIL7LYb3HtvjPqDeL0ZM6Bbt/xzb78dTjghQqZf/hKOOCLO6dIFNt4YWrUq\nfK0WLQrX3qoueMu9niRJ5WoVXqkZsfNKkiRJkiQVm51X1atN5xXE5zU337zsNadqq1MneOIJGDMG\ndt01rjllSnRG5T4bggiiKgZXEGMEhwyJ8Kpjx5WvRZKkahheqZDhlSRJkiRJKjbDq+rVtvMqpxjB\nVU6fPnHL6d699s+1U0qSVMdaLPsUNSuGV5IkSZIkqdgMr6pX284rSZKaGcMrFTK8kiRJkiRJxVYx\nvHL9q7zl7bySJKmZMLxSIcMrSZIkSZJUbBXDqwULsqujobHzSpKkahleqZDhlSRJkiRJKraK3Va5\nwEZ2XkmSVAPDKxXKLfxpeCVJkiRJkorFzqvq2XklSVK1DK9UyM4rSZIkSZJUbBXDKzuv8ubPj89i\nWrfOuhJJkhoUwysVMrySJEmSJEnFZudV9ebNi66r3CQcSZIEGF6pMsMrSZIkSZJUbHZeVW/+fNe7\nkiSpGoZXKmR4JUmSJEmSiq20NL9v51XevHmGV5IkVcPwSoUMryRJkiRJUrHZeVW9+fNjbKAkSSpg\neKVChleSJEmSJKnYXPOqenZeSZJULcMrFTK8kiRJkiRJxWbnVfXsvJIkqVqGVypkeCVJkiRJkorN\nzqvq2XklSVK1DK9UyPBKkiRJkiQVm51X1bPzSpKkahleqVCSxNbwSpIkSZIkFUtpaX7fzqs8O68k\nSaqW4ZWqatHC8EqSJEmSJBWPnVfVs/NKkqRqGV6pKsMrSZIkSZJUTK55VT07ryRJqpbhlaoyvJIk\nSZIkScVUMbyaOze7OhqSJUtg4UI7ryRJqobhlaoyvJIkSZIkScWUC69WXx1mz862loYiNz7RzitJ\nkqowvFJVhleSJEmSJKmYcuFV584wa1a2tTQUhleSJNXI8EpVGV5JkiRJkqRiKi2NreFV3rx5sXVs\noCRJVRheqSrDK0mSJEmSVEy5zqsuXQyvcuy8kiSpRoZXqsrwSpIkSZIkFVPF8GrmzGxraShy4ZWd\nV5IkVWF4paoMryRJkiRJUjG55lVVubGBdl5JklSF4ZWqMrySJEmSJEnFVDG8+v77uDV3dl5JklQj\nwytVZXglSZIkSZKKqWJ4BXZfgZ1XkiQtheGVqjK8kiRJkiRJxVRaGtvVV4+t4ZWdV5IkLYXhlaoy\nvJIkSZIkScVUVgYtW0LHjnHf8MrOK0mSlsLwSlUZXkmSJEmSpGIyvKrKzitJkmpkeKWqDK8kSZIk\nSVIx5cKrTp3i/syZ2dbTEOQ6r9q1y7YOSZIaIMMrVWV4JUmSJEmSisnOq6rmz4e2bePPRZIkFTC8\nUlWGV5IkSZIkqZgMr6qaN8/1riRJqoHhlaoyvJIkSZIkScVUWhrh1Wqrxf1Zs2DBAvjyy2zrytL8\n+YZXkiTVwPBKVRleSZIkSZKkYiorg5KSCLA6dIjw6rLLYPPNm+9nEPPmwSqrZF2FJEkNkuGVqjK8\nkiRJkiRJxZQbGwgxOnDWLBg3DqZPh0mTsq2tLrzzDrz+OqRpzefYeSVJUo1Ksi5ADZDhlSRJkiRJ\nKqbqwqtPP43748ZBnz7Z1bay0hTefjtCuC++gGeegdGj49gOO8CoUdC7d9Xn2XklSVKN7LxSVYZX\nkiRJkiSpmCqGV2uuCZMn5zuuxo3Lrq6VVVYGJ50EW28Nw4bBGWfA++/DBRfAddfBhx9C376w667w\n+eeFz7XzSpKkGtl5paoMryRJkiTVQpIk+wJ/BloCt6Rp+rtKx9sAo4CtgenAoWmaflZ+bDPgRmA1\nYAmwTZqmC+uvekn1qmJ4tfnmcM018RjAe+9VPf8vf4F114WhQ+uvxtpasgRuvBHGjoVPPoFXXoHT\nT4ejjoK114auXSFJ4twf/hBuuw2uvhq22SYCLohurYkTYeeds3sfkiQ1YIZXqsrwSpIkSdIyJEnS\nEhgJ7AVMAV5PkuSxNE0/rHDascCMNE37JElyGPB74NAkSUqAO4Ej0zR9N0mSLsDien4LkupTaWk+\nvNpyy3xwtcoqVTuvvv02wqANNoAhQ/JB0PKqGJgBTJgAU6cuX2D05ZfxGUmHDrD66hE6HX003Hln\n3C8pif3DD6/++eusE11YhxwCZ50Ft98ObdtGXUuWwI47rth7kySpiTO8UlWGV5IkSZKWbVtgYpqm\nkwCSJLkXGApUDK+GAheW7z8IXJskSQLsDbyXpum7AGmaTq+voiVlpKwsgh6I8Cpnn33g0UdhzhxY\nddV47IEHIuyaMAFeeAF69IjRe7X14osRfr35ZozrGzIEOneGX/4y1pn66qu4n6vrkUci0OrWDd59\nN9avGjgQxoyBM8/MX3fHHaF//wirzj8fLrwwHq9NuLbxxvl1sCRJ0jIZXqmqJDG8kiRJkrQsPYDJ\nFe5PAQbVdE6apqVJkswCugAbAmmSJGOANYB70zT9Q+UXSJJkBDACYN111y36G5BUjyp2QfXrF91H\nCxfCCSfAww/D/ffDscfG8TvvhPXXjzWidt8d2rWL7qwNNlj26/ztb3DMMbDeenDyyXHdZ5+NY6uv\nDt9/H+tSzZ0bodgLL8A778RIv4MOgt/+NoKznP33j/Br6tS49ksvxSjDCy5Y8Y4wSZK0TIZXqsrO\nK0mSJEnLVt2ntmktzykBdgK2AeYDzyRJ8maaps8UnJimNwE3AQwcOLDytSU1JhXDq5IS2HRT+Owz\n2HPP6Eq66aYIryZNgpdfht//Hp57LoKlefNgxAh46qmaA6ORI2OdrE8+icDr4Yejk+tPf4Lp02Hy\n5AjE9t47Rvd17AgLFkQn1RlnwJVXwuuvw4EHwkUXwTPPwAcfxPNXWSVe47zzopb27Q2uJEmqY4ZX\nqsrwKixeHP9I/fvf4xtbd92VdUWSJElSQzIFWKfC/Z7AVzWcM6V8nauOwHflj7+Qpum3AEmSjAa2\nAp5BUtNUef2pk06CL76IEOi442Kk3/jxMTIwSWD48DgnSeCvf40uqscfhx/+sOq177orjg8aBKec\nApdemg+ckgS6do0bwCWXwDXXwK23wlpr5a8xcCB06QJ77RX3N920+veRu64kSapThleqqkWLWIC0\nvs2aBR9+CK1awRNPwJprxjewevVaueuWlcW3rLp1q935Y8fGP3wffTQWZu3SJcYCGF5JkiRJFb0O\n9E2SpBfwJXAY8ONK5zwGHA2MBQ4Bnk3TNDcu8OwkSdoDi4BdgKvrrXJJ9a+0tDC8Ovro/P4++8R2\n7Fi4445Yp2qdCtn4iBHx5dJzz43xft2754/NnQunnQY77BCdWq1bL72OffeNW2WHHbbcb0mSJNUd\nwytVVd+dV+PHR8v+rbfCt99WPb7HHvGP1E02Wf5r33svHH88zJ4di7UOHhyjAjp1ioVW11ij8Py/\n/jXGFLRtG/94vv76WKj1ggviz6RFixV6i5IkSVJTU76G1cnAGKAlcFuaph8kSXIx8Eaapo8BtwJ3\nJEkykei4Oqz8uTOSJLmKCMBSYHSapo9n8kYk1Y/KnVcVbbhhjPi77bYY+3fWWYXHW7WCK66INal6\n9YIxY2LU4Pffw1VXwbRp8Nhjyw6uJElSo2F4parqM7xauDBa8qdOjZnUxxwD8+fHYqjTp8NDD8U/\nRIcNg3ffrf0/RB95JJ734osRUvXqFfevuip/Trt28POfR1jVrx9cdlnMr95773jdDh3ivHHjYlta\n6j+EJUmSpArSNB0NjK702PkV9hcCw2p47p3AnXVaoKSGo6ws1rqqTosWsPXW8Pzzcb+60YAHHBBf\nfh08OEKsWbNi3D/AwQfDdtvVSdmSJCkbhleqqj7DqxtugClT4Omno8Oqoi5d4Ne/hs03hx/8IOZd\n779/jA5o0yZCqcoLpM6cCaeeGmMG+vWLjqlzzonzd9stRgsMGBBh2bXXRkfX1VfHeIEXX4TDD4db\nbonOq5zcN8MMryRJkiRJWjFL67yCGAf4/POw1VaFa1FVtOGGMdJ/t93gwAPh//4PNtoovgwrSZKa\nFMMrVdWiRQQ19eGaa2KWdeXgqqLBg2Ph1ptvhocfzj++1VYxp/q002Lh1Ztugt/8BmbMiNDqN7+J\n0QI5xxyT3+/VC7bfPoKrU0+F++6LRVt/85uqgVjum2FlZSv9diVJkiRJapbKyuKLpTXZZpvY/uAH\nS7/OoEHxxVW/XCpJUpPmAj6qqr46r6ZOhU8/jRGBy3L11TES4KOPYgHWm2+OOn//exg4EA49NNa2\n2mQTeO01uPDCwuCqJt26wT33wP/+FyMDKwdXkA+v6ivQkyRJkiSpqVlW59Uee8Ta00cfvexrGVxJ\nktTkGV6pqvoKr159NbaDBtXu/JYtYxTgrrvCz34Gr78OY8dCp07w+ONwyikxfnCrrZavjiSBNdZY\n+uuC4ZUkSZIkSSuqtHTp4VXnzvDkk9C7d/3VJEmSGizHBqqq+gqvXnklupq23HLFr7HNNvDOO8Wr\nqTqODZQkSZIkaeUsq/NKkiSpAjuvVFV9dl5tvjm0a1f3r7Uy7LySJEmSJGnllJXlvxwqSZK0DIZX\nqmpFwqsvvoBhw+CQQ2Idq8sug2eeyR+fMwdGj44A6OGH4fLL4aWXYPvti1t7XbDzSpIkSZKklWPn\nlSRJWg5+5UVVLW94laZw7LHw8stxf/x4+OAD2HhjGDcu1pS66CK48kro2xc++STO22MP+PWvi19/\nseXCKzuvJEmSJElaMYZXkiRpOdh5paqWN7y67TZ4+ukIp84+O4KrFi1i+8ILEW49+CCstloEV1dc\nAZMnw1NPwdpr1937KBbHBkqSJEmStHIMryRJ0nIwvFJVq64K06fX7twvv4TTT4ddd4URI+C006Lj\n6vrroXNnOPNM+Mc/4PPP4U9/iuueeSb07BkdWY2BYwMlSZIkSVo5paWGV5IkqdYcG6iqNt0U7rwT\nZsyA1VevenzCBOjYEbp3h3PPhUWL4JZbottqtdXg/ffjvO7dYfhwGDo0/oE6ZEgEWo2NYwMlSZL+\nP/buPFzO+f7/+POTc5ITslqitkR2BLVkQWuripJaWqK1VJWitRRVWmqpoooqVUvV1lbtVNGiVNXW\n1hIRITRIfEskskgkIiLLuX9/vDO/OUlOZHLOzLnP8nxc11z3zD33zLwnVy+957zu9/sjSVLj2Hkl\nSZJWgZ1XWt5nPxvbl19e/rksgy98AU46KUYA3nwzHHss9Ou3/LH77gujR0fH1V13wVprVbbuSimc\nXNt5JUmSJElSwyxeXLw4VJIkaSU8a9DyCuHV2LGw005LPzduHEyeDC++CJdfDh06wFPxgO8AACAA\nSURBVKmnrvi9Ntkkbi2ZnVeSJEmSJDWOnVeSJGkVGF5peeutF11SY8cuvf9//4O//z3uv/FGdGHt\nvDOsu27T19iUDK8kSZIkSWocwytJkrQKDK+0vJSi+6pueDVjBvTuXXxcWxtrXx14YJOX1+QcGyhJ\nkiRJUuMYXkmSpFXgmleq32abwX//W3xc9/7OOxfvb7tt09WUFzuvJEmSJElqnEWLDK8kSVLJDK9U\nv7XWgtmzi91GEybE9oYb4K67Yq0rgGHD8qmvKRVOrg2vJEmSJElqGDuvJEnSKnBsoOrXrVtsP/wQ\nuneP8KpdO/jGNyK42mQT+OgjWHvtfOtsCoXOK8cGSpIkSZLUMIsXF39fS5IkrYRnDapfIbyaPbsY\nXvXqVey4+vnP204nkmMDJUmSJElqHDuvJEnSKjC8Uv26d4/t7NmxnTAB+vUrPj9iRNPXlJfCybWd\nV5IkSZIkNYzhlSRJWgWueaX6FTqvPvggthMmQN+++dWTJzuvJEmSJElquCyD2lrDK0mSVDLDK9Wv\n7tjAOXNgxoylO6/aEsMrSZIkSZIarjDJxPBKkiSVyPBK9asbXk2YEPfbanjl2EBJkiRJkhrO8EqS\nJK0iwyvVr+6aV209vLLzSpIkSZKkhjO8kiRJq6g67wLUTNVd82ru3LjfVsOrwsm14ZUkSZIkSauu\nEF5V+2coSZJUGjuvVL8OHaBjx2Ln1dprQ9eueVeVj8LJtWMDJUmSJEl5mD0bDjoIHnoIXnkF5s37\n9ONnzYLvfAd+97umqW9l7LySJEmryPBKK9atWzG8aqtdV+DYQEmSJElSvu69F26/HUaMgC22gN13\nj9/rixfDzJnw3HNxXJbBf/4DW28N114LRxwBN90EH31UXBIgD4Xf04ZXkiSpRPZra8Xqhlef/3ze\n1eSncHJt55UkSZIkKQ8PPQTrrgunnQbvvw/nnRdrVa+1VgRWM2fCvvvC00/H8xttBE88AeeeGwHW\n2WfDpElwzjmw4Ybx+3b33WPiSm0t9OgB7ZZc37xwYfwt4I474I034jfxI4/A5pvDr38dx64qO68k\nSdIqMrzSinXvDtOmwTvv2HkFdl5JkiRJkpreokURHu27L5x4YuwbOhTGjIFx4+Djj+P3+x//CPvt\nB7vsAoccAmusER1bu+8OEyfG/rPOqv8zBg6E44+Hf/wD7rsvQqbFi2H11WHBAthmG7jnHnjssQjA\ndtghOsBKZXglSZJWkeGVVqxbNxg9Oq7CMrwyvJIkSZIkNb0nn4w1rPbcs7hv773jVtcVVyy/VnXn\nzvH6RYtibes334T27WH+fLj//ui8Arj+ejjhBKipie3qq8M3vgGDBsXzKcHYsfDNb8Kxx8a+oUPh\nwguhb1+44ALYdtvoEHv3XfjCF+Dgg4uf26VLvMbwSpIklcjwSivWrVuMG4C2HV45NlCSJEmS1FQW\nLYIrr4QpU+CHP4Qf/ADWXz/Wu/o0ywZXBdXVxYsyBw4s7t900+L9E06IsYLt28d4wvp89rPwwgvR\nxfXII3DxxfDFLxafv+66CL222SZCrZ//vPjc5psXa5EkSSpBSWcNKaU9gMuBKuD6LMsurOeYrwHn\nABnwUpZlB5exTuWhW7fYduwIW22Vby15svNKkiRJktRUDj0Ubr897l98cWzvvju6qColJejZc+XH\nVVXBgAFxO/zwqGvChKh53LgYJdi3L7z+OowaBautFksRXHZZvL7wdwZJkqSVWGl4lVKqAq4ChgOT\ngOdTSvdnWfZqnWMGAKcDn8+ybFZKaZ1KFawmVGjr//KXK3uS3NzZeSVJkiRJagpTpsCdd8baVocc\nAn/7W3RI7bdf3pUtb/XVY4xgQf/+xfsDBy7d5XXssfD88zBkSNPVJ0mSWrRSOq+GAW9mWTYRIKV0\nO7Av8GqdY44CrsqybBZAlmXTyl2ocvDKK7FtjifJTaldu9jaeSVJkiRJqqRbb411p485BjbeONaV\nag2qq2H77fOuQpIktSDtSjhmA+CdOo8nLdlX10BgYErpXymlZ5aMGVxOSunolNKolNKo6dOnN6xi\nNZ0f/jDWulp2Edi2JqU40Ta8kiRJkiSV4uOPYw2oVf0decstEVhtvHFl6pIkSWohSgmvUj37smUe\nVwMDgF2Ag4DrU0rdl3tRll2bZdmQLMuG9OjRY1VrVVMbPhzefLM4PrAtq6pybKAkSZIkqTQ33QRH\nH11c66kU774LL74II0dWri5JkqQWopTwahJQd9XODYHJ9RxzX5ZlC7MsewsYT4RZUutg55UkSZIk\nqVTdusX29ttLf83DD8d2j3qH2UiSJLUppYRXzwMDUkp9UkodgAOB+5c55l7gCwAppbWJMYITy1mo\nlCvDK0mSJElSqdKSITajR9f/W/Lxx+Ggg+CTT+CZZ+Dkk+Haa2G99WCLLZq0VEmSpOaoemUHZFm2\nKKV0PPAwUAXcmGXZuJTSucCoLMvuX/Lc7imlV4HFwKlZlr1fycKlJuXYQEmSJElSqeoGVv/5D+y4\n49LPX399dGW9+y489VRx/+GHF4MvSZKkNmyl4RVAlmUPAg8us+/sOvcz4OQlN6n1sfNKkiRJklSq\nur8fn3tu6fAqy6LzCiK4+upX4eqr4Q9/gP33b9IyJUmSmquSwiupzauutvNKkiRJklSauuHVSy8t\n/dzEidFxde65sOaacOSRUFMDP/pR09YoSZLUjJWy5pWkqio7ryRJkiRJpVm4MLZbbQUvvgj77Qd3\n3BH7nngitvvvD8cdF8GVJEmSlmLnlVQKxwZKkiRJkkpV+P04ZEisb/XKK/G7crfd4JJLYL31YNNN\n861RkiSpGbPzSipFVZVjAyVJkiRJpakbXhW8+iocf3yMDbz1Vkgpn9okSZJaADuvpFLYeSVJkiRJ\nKtWy4VW7dvD66zB1Khx0EOyyS26lSZIktQR2XkmlqK6OzqvFi+H882HWrLwrkiRJkiQ1V4XwatCg\n+A155pmxDtaMGbD99vnWJkmS1AIYXkmlqKqKHx9jxsBZZ8Ff/pJ3RZIkSZKk5qoQXrVvD2ecAXvt\nVXxuu+3yqUmSJKkFMbySSlEYGzh5cjz+4IN865EkSZIkNV+F8KqqKrabbBLbzp1hs83yqUmSJKkF\ncc0rqRSFsYGGV5IkSZKklVm4MIKrlOJxly6w0UbQr18x0JIkSdIKGV5JpSiMDSyEV655JUmSJEla\nkUWL4iLIum6+GdZYI596JEmSWhjDK6kUjg2UJEmSJJVq0aJY76quHXbIpxZJkqQWyDWvpFJUVS09\nNtDOK0mSJEnSitTXeSVJkqSSGV5JpbDzSpIkSZJUKsMrSZKkRjG8kkpRXW3nlSRJkiSpNIZXkiRJ\njWJ4JZWiqgo+/himTYvHH3wAH30UP0gkSZIkSarL8EqSJKlRDK+kUlRXw6RJcb9Tpwivtt4azj8/\n37okSZIkSc3PwoWGV5IkSY1geCWVorq6OCpw001hzhx44w0YO7Zh7zdnTvlqkyRJkiQ1L3ZeSZIk\nNYrhlVSKqqri/Y03Lt4vdGMVZNnK3+s3v4E11oDTT491tCRJkiRJrcuiRdC+fd5VSJIktViGV1Ip\n6l4x17t38f6778Z2yhTYay9Yf3348MMVv8+8efDTn0Z4deGFcNNNxecuuACuvbb4+O234de/hgUL\nyvIVJEmSJElNxM4rSZKkRvFMSirFisKrKVNilvmZZ8KDD0bn1d//DvvtV//73HADTJ0KTzwB3/8+\n/OxncOih8MorcMYZ0eE1dGh83m67wbRpcf/YYyv69SRJkiRJZWR4JUmS1Ch2XkmlqDs2cKONivez\nDN57D558EkaMgO7d4a9/XfH73HorbL017LQTnH02TJgAv/0t/PjH8dq114bDD4djjonjt9kGzjsP\npk+Px+++C6+/Xtp4QkmSJElSPgyvJEmSGsUzKakUhR8dHTvCZz4T91OKEGn0aHjzTTj6aOjSJTqw\namuh3ZJs+IMP4F//gpoaeOYZOP/82L/PPvDFL8L3vhfvc+ml0L9/7Ae4+mrYcssIuvr0iVDr+uth\n/nz4whfgnnsi8Goqb70V36tfv6b7TEmSJElqiQyvJEmSGsXOK6kUhc6rbt2KgdFWW8X2jjtiu8MO\nse7V1Knwwguxb8YM2Hzz2D98eOwbOTK2KcF118Faa8GJJ8JJJ8Hee8O550Y4deSR8LnPwdixsMce\ncOWVMHAgXHQRPP00bLAB7LILzJlT+e9/7LHQty8MGBB13XQTfPxx5T9XkiRJklqihQsNryRJkhrB\nMympFIUfHd26RdhUXR1h1Isvwu23R0fWNttEuNSuXYwOHDIkgp7p0+HPf4bf/S66pjbeuPi+ffrE\null1f9ScdVbcCgYNgrvvjg6v/v2ha9cItW67Da65Br77XbjllgjDVmTatDi+UyeYOzfW7frnP2HB\nAthwQ1hjDfjkE3jjjRhhuOGGxde+9lp8zje+EZ1lN9wQt7PPhuefhx49yvJPLEmSJEmtxqJF0L59\n3lVIkiS1WIZXUinqhledOsG//x2h0sUXx8i/3XaLsYA1NbD99hFeff7zcN99ccxXvhK3T3vvldlm\nm+L9HXaI2/rrw5lnxr7f/x46dIj78+fHKMMrrogusKlTYdKkpd+vpiZCt9mzl67ltttg8OBY26tn\nT3joIVh9dbjssliT64orYu2ub34Tnn02usokSZIkSUWLFsXvKEmSJDWI4ZVUirpjAwGGDl36+Usv\nLd7fay84/XQ47rjocDrhhMrV9eMfR22nnx7b3/wmuqLOPz9GFlZXR63rrAN33hlh1+qrw3//Gx1g\nPXrEj6qpU2MM4OLF8doJEyK0mjIl1vi69NIIrgr/FoMHx/25cyv33SRJkiSppXLNK0mSpEbxTEoq\nReFHR2G9q4IbbohOrAEDivtGjox1qd5+G26+OTqcKiUlOO20+GF01lkxPjDL4ItfhMMPh223jVGD\ny6o76q99+6XHBN58c/H+ggXx/LIjCTt3ju1HH5Xvu0iSJElSa2F4JUmS1CieSUmlqDs2sK4jjlj+\n2P79YdasytdU1xlnwJZbxjjDXXeN9bjKoTCGcFmdOsXW8EqSJEmSlmd4JUmS1CieSUmlWHZsYHOT\nEuy9d9yaguGVJEmSJK3YwoWGV5IkSY3QLu8CpBYhy2JbGJfX1tXURKDnmleSJEmStDw7ryRJkhrF\n8Eoqxbx5sS10HLV1KcW/hZ1XkiRJkrQ8wytJkqRGMbySSlHoMLLzqsjwSpIkSZLqZ3glSZLUKIZX\nUikKIY3hVZHhlSRJkiTVb9EiaN8+7yokSZJaLMMrqRSGV8vr3Nk1ryRJkiSpPnZeSZIkNYrhlVSK\nQkjjmldFdl5JkiRJUv0MryRJkhrF8EoqxaJFsbXzqsjwSpIkSZLqt3Ch4ZUkSVIjGF5JpbjhBvj+\n92G77fKupPno1MmxgZIkSZJUHzuvJEmSGsUzKakUvXvDpZfmXUXz0rmznVeSJEmSVB/DK0mSpEax\n80pSwzg2UJIkSZKWV1sbN8MrSZKkBjO8ktQwhleSJEmStLzFi2Pbvn2+dUiSJLVghleSGqZzZ5g3\nL64olCRJkiSFRYtia+eVJElSgxleSWqYTp1iO29evnVIkiRJUnNieCVJktRohleSGqYQXjk6UJIk\nSZKKFi6MreGVJElSgxleSWoYwytJkiRJWp6dV5IkSY1meCWpYTp3ju3cufnWIUmSJEnNieGVJElS\noxleSWoYO68kSZIkAVx7LYwYkXcVzYfhlSRJUqMZXklqGMMrSZKkNi+ltEdKaXxK6c2U0mn1PF+T\nUrpjyfPPppR6L9nfO6X0cUppzJLbNU1du8roO9+Bhx7Ku4rmoxBetW+fbx2SJEktmJcBSWoYwytJ\nkqQ2LaVUBVwFDAcmAc+nlO7PsuzVOod9G5iVZVn/lNKBwEXA15c8NyHLsq2atGhV1vz50LFj3lXk\nz84rSZKkRrPzSlLDuOaVJElSWzcMeDPLsolZli0Abgf2XeaYfYE/LLl/N/DFlFJqwhrVlGbOzLuC\n5sHwSpIkqdEMryQ1TNeusZ09O986JEmSlJcNgHfqPJ60ZF+9x2RZtgiYDay15Lk+KaUXU0pPpJR2\nrHSxqqAuXWJreBUWLoyt4ZUkSVKDeSYlqWF69ICqKpgyJe9KJEmSlI/6OqiyEo+ZAvTKsuz9lNJg\n4N6U0mZZls1Z6sUpHQ0cDdCrV68ylKyK6N4dPvzQ8KrAzitJkqRGs/NKUsNUVcG668K77+ZdiSRJ\nkvIxCehZ5/GGwOQVHZNSqga6ATOzLPsky7L3AbIsewGYAAxc9gOyLLs2y7IhWZYN6dGjRwW+gsqi\ne/fYvv9+vnU0F4ZXkiRJjWZ4Janh1l/f8EqSJKnteh4YkFLqk1LqABwI3L/MMfcDhy25PxJ4LMuy\nLKXUI6VUBZBS6gsMACY2Ud0qtzXWiK2dV8HwSpIkqdE8k5LUcBtsAG+8UZ73qq2FdubpkiRJLUWW\nZYtSSscDDwNVwI1Zlo1LKZ0LjMqy7H7gBuCPKaU3gZlEwAWwE3BuSmkRsBj4bpZlJh8tleHV0grh\nVfv2+dYhSZLUghleSWq4DTaAJ54o/fgnn4wftltssfT+yZNh0CAYMgQmTIChQ+Gmm6Bjx/LWK0mS\npLLKsuxB4MFl9p1d5/584IB6Xvcn4E8VL1BNY7XVYmt4Fey8kiRJajTbHCQ13Prrw6xZ8PHHxX0L\nFsDUqcsfO2UK7LknfP/7yz93220wezaMHRvz8u++Gw4/vHJ1S5IkSSqfLIuta14FwytJkqRGM7yS\n1HAbbBDbuuteXXghbLIJzJ+/9LFnnw3z5sErryz/PrfeGl1X06bBiy/CqafCnXfC229XrnZJkiRJ\n5VFbG1s7r8LChbE1vJIkSWowwytJDVcIryZPLu57/HH44AN45pnivsWL4eabYfXVoytrxozic08/\nDaNHwyGHFPcde2xcvfmLX0RnlyRJkqTmq77w6umnYypDW2TnlSRJUqMZXklquPXXj22h82rxYnj+\n+bj/z38Wj3vrrejE+vKX4/G4cbF94w346lehXz847LDi8RttBHvvDVdeCRtv7PgRSZIkqTkrjA0s\nhFf/+hfsuCOcd15+NeXJ8EqSJKnRDK8kNdyyYwP/+1+YOzfu1w2vXnsttgcsWav7lVdiROCee8bj\nv/0N1lhj6fe+6aYYJzhjBhxzDBxxBEyYUN76Fy+O0YR11+ySJEmStGoKnVeFi84uvTS2V1wBc+bk\nU1OeDK8kSZIazfBKUsN17Qo1NRFEATz3XGz33jvGBhZCoUJ4tdtu0L07PPAAfP7zMW7wL3+B/v2X\nf+9u3eCgg+DAA+Guu+B3v4Nhw2CffZYeSfhpFi5cflTJ9Onwwx/C1ltDp07R5dWjBwwfDl//enSH\n7bQT7Lor3H13dIwtXhwjDPfcE845p/jjXJIkSdLSYwPffhvuvRdGjIDZs+H444trQLUVhleSJEmN\n5pmUpIZLKcKo2bPj8fPPR6B12GERSo0dC9tuG+HVuutGd9Xmm8NDD8Faa8E//gHbbffpn3HZZXHM\njjvCT34SAdmIEfDd78JHH8Xowq99Dfr0ieMXLYr1tebMgfPPj/W3OnaEgQPhxBPhRz+Kbq6ddoLv\nfS9GFr70EowaFT+0u3SJ2zvvRKdYhw7Qqxe8+SYMGBBdYqNHRw0vvww77xxh3WqrVfbfWpIkSWqu\nCuHVvHnwpz/F40sugaFD4ac/jd8IV16ZT21z50ao1qtX1PXMMzB4cFyEVymF8Kp9+8p9hiRJUitn\neCWpcbp1K4ZXEybEGlVbbx2PX3qpGF5tumnsu/hiGDMmupzWXHPl7/+Zz8AJJ8T9+++P9bN23hku\nuihCptmz4/6ECdFJ9c1vwh13xPGbbw6HHx4h1x//GM8NGhTh2ZZbfvrnLloUQdUTT0SwdfLJEZhd\ndhmcfXaEczU1cPXVEeAdeijssEN0Z3Xpsur/jpIkSVJLVXcywa9+FRetbbxxTC2YOhWuvTamH/Tq\n1XQ1/fnPMbbw6aej82vHHeO3w9ixcfHZDTfERW6dO8dFeeVk55UkSVKjeSYlqXG6d4/uJoBJk+JH\nau/eEeC89FIs3vzaa/CNb8Qx228ft4bq0wcmToz71dXR7TVsGJxySqy59e9/w4UXwr77Qt++0TkF\ncOyxsXD0YYcV932a6mrYa6+41XXyyfEekybBZpvBk0/CddfBb38bP45PPx0uuKDh30+SJElqabIM\nttgCXn89phnsuSe0W7JKwY9/HEHRz38Ov/lN09Ry6aXx+2DAADjppLjI7f77Y1rCd78L11wD66wT\nx2+1Fdx4Y/ECvPq88QY8+GCMEz/88OXX64X4XXDPPbHu19Spsc/wSpIkqcE8k5LUON26LR1effGL\n8UN1yy0jvDrnnBjh97nPle8z6/4IHDoURo6MH5xdukTX1de+tvxrBg2KWzmstVbcINbG2nXX6O7q\n1av4byFJkiS1FbW1sPrqcZHa448vfbFaz57w7W9HgPXjH8fjcn7uv/8Njz4Kzz4bkx0mTozRgPvv\nH+PEO3aMY3/yk+Lr9tsPxo+P3ylXXBEXww0eHGPBt9gi1snt1y/GDc6ZE+FVwU9+EhfmdekSXVuP\nPw7vvRcX7HXsGKHYu+/G76SuXcv3XSVJktqYdnkXIKmFK6x5NWdO3DbcMPZvuWV0Op17LhxxBBx8\ncOVquPLKCK/efrv+4KopdOoUV3J+8kk+ny9JkiTlpbY2LmD7whfi8bKTFk47LbY/+9mK3+PllyNQ\nqivLYpT3IYfEOO/Zs2G33WC99eL3xc47xzjA886LwOjf/4YFCyKQuuuuYnC1rOHD4fjjI0wbNy7C\nqDlz4KCDYkrD4MGxTu6aa0Zn1vnnx2+NMWNiMsPvfge//nUEWdOnQ//+8MtfwqxZ8L//xW+CqVMj\n0JMkSVKD2HklqXEKYwPffTceF66k3Hzz2PbuHWM5yj1Hvq7PfCbGd+StpiZ+LEuSJEltSSG8+ta3\nogtphx2Wfn6jjWJc3xVXxCi/H/yg+Nwzz0S30o47wscfw4gR0bW0xx7wyCMRFLVrFxMW1lwzfnvs\nsw/8859x7n3NNXEBW32j/Eqx5prxGaXo2RNuuy3uZxnMmxcXsS2rqipukiRJajDDK0mN061bXAE5\naVI8LnRe7btvLI580UXQvn1+9TWlmho7ryRJktT21NbGxWq9esHVV9d/zC9/CVOmwKmnxqjxrbaC\nBx6ITqaaGli4MEKpV1+NbqY//CFed+aZcOKJ0V01cyZ885vROZVlcWuX00CZlOoPriRJklQWhleS\nGqd797hCcsKEeFwIr9ZbL2bMtyWGV5IkSWqLSgmR2reH666Ljql99onfEPPnx8i999+HQw+Fyy+P\nYz/5BP7731hXqm/f2Fd4riClyk53kCRJUq4MryQ1TvfusX3lldhusEF+teStQwfDK0mSJLU9hbGB\nK9O9O/z85/C970WA9f77cOmlsMkmcS5dUFMTa+hKkiSpzTK8ktQ43brFdty4WHuq7o/OtsY1ryRJ\nktQW1dZCdYl/XjjqKDjiCNeEkiRJ0qfKaTi0pFaj0Hn18svFkYFtlWMDJUmS1BaV2nlVYHAlSZKk\nlTC8ktQ4hc6r99+Hfv3yrSVvjg2UJElSW1Rb6/pTkiRJKivDK0mNU+i8Athii/zqaA7svJIkSVJb\nlGWr1nklSZIkrYRnl5Iap254tfnm+dXRHLjmlSRJktqiVR0bKEmSJK2EZ5eSGqcwNhDsvHJsoCRJ\nktoiwytJkiSVWXXeBUhq4Tp3jh+qNTXQp0/e1eTLsYGSJElqiwyvJEmSVGaGV5Iap1276L7q188f\nrI4NlCRJUltUWwsp5V2FJEmSWpGS/tKcUtojpTQ+pfRmSum0ep7/VkppekppzJLbkeUvVVKzNWAA\n7Lhj3lXkz84rSZIktUVZ5oVskiRJKquVdl6llKqAq4DhwCTg+ZTS/VmWvbrMoXdkWXZ8BWqU1Nw9\n+aQ/VsE1ryRJktQ2OTZQkiRJZVbK2eUw4M0syyZmWbYAuB3Yt7JlSWpRamqgffu8q8hfTQ0sWhQ/\n3iVJkqS2wvBKkiRJZVbK2eUGwDt1Hk9asm9Z+6eUxqaU7k4p9azvjVJKR6eURqWURk2fPr0B5UpS\nM1ZTE1vXvZIkSVJbYnglSZKkMivl7LK+VVezZR7/BeidZdlngUeBP9T3RlmWXZtl2ZAsy4b06NFj\n1SqVpOauQ4fYOjpQkiRJbUltLaT6/nQgSZIkNUwp4dUkoG4n1YbA5LoHZFn2fpZlhb/WXgcMLk95\nktSCFDqvDK8kSZLUlmSZnVeSJEkqq1LOLp8HBqSU+qSUOgAHAvfXPSCltF6dh/sAr5WvRElqIQyv\nJEmS1BY5NlCSJEllVr2yA7IsW5RSOh54GKgCbsyybFxK6VxgVJZl9wMnpJT2ARYBM4FvVbBmSWqe\nXPNKkiRJbZHhlSRJkspspeEVQJZlDwIPLrPv7Dr3TwdOL29pktTCuOaVJEmS2iLDK0mSJJWZZ5eS\nVC6ODZQkSVJbVFsLKeVdhSRJkloRwytJKhfHBkqSJKktyjI7ryRJklRWnl1KUrk4NlCSJEltkWMD\nJUmSVGaeXUpSudQdG3jCCTB8eL71fJosg/feW3rfzJkwf34+9UiSJKnlMrySJElSmVXnXYAktRqF\n8Orhh+GKK+J+ljWv+f9Tp8L//R+MGwff/nYEbAsXQvv28Nhj0LMn3HQT7Ljjyt9r2jSYNQs23rji\nZUuSJKkZM7ySJElSmRleSVK5FMKrX/6yuG/2bOjePZ96lpVlMHIkPPMMrLMObLghjB8P660XXVdH\nHQV//zvsvDN85zvQty/8738RcF10UYRes2bBYYfBoEFw5pnw4Ydw4IHwm980n+8pSZKkplVb27wu\n2JIkSVKLZ3glSeVSWPOqrqlT8w91sgxuvx1efRWefjquip08Ge68Ew44YOljv4KNIwAAIABJREFU\n586FU06BG2+EBQuguhquugp69YIRI2Ks4G9/G8d+6UswZEgEWy+8AP/8J2ywQdN/P0mSJOUry+y8\nkiRJUlkZXklSuRQ6ryC6maZMidF6eY7VGz8eTj4ZHnwwHn/mM/CHP8Bf/gL77bf88Z07wzXXwC9+\nEV1WixbBrbfCMcfAWmvFMSNGwGqrwVe+ElfY7rEH7LknfPnLEXx9+cuwxhpx7IIFcevcuWm+ryRJ\nkpqeYwMlSZJUZoZXklQudcOrzTYrhld5eeAB2H//WM/qV7+C7baDLl1i5N+XvvTpr+3SJW4Q4wHr\nOvjgpR/vsAPcdVeMJDz00AiqTj8djjgCeveOzxs9umxfS5IkSc2M4ZUkSZLKzLNLSSqXumMDN9ss\ntpUKrxYujJF/N9wQjxcvjo6q006D//4X7rsPvvpV2HxzePNNOPFE2HbbCJIqYY89olPruedijawz\nzojus08+gRdfrMxnSpIkqXkwvJIkSVKZ2XklSeVSt/OqEBJVIrz6+OMY5Xf33XF77LEIiF57Lcb4\nXXRRHLfttvC3vzXdmlvt28PQoXDPPVHTM8/Eulr/+1/TfL4kSZLyUVsb56GSJElSmRheSVK5LLvm\n1VprfXp49cAD8NZbsfbUn/4U60pVr+Q/y488EmtOdegAgwfDsGFwxx2w7rrxHttvD3/+M8yeDccd\nB127lue7rapdd43bRx/BxRfHIt7+QUOSJKl1yjI7ryRJklRWhleSVC7t2xfvr702rLNO/eHVww/D\nmmvCUUfBe+/BjTdG59Sjj8K778Lvfx/j/pb18ccRcK23XoRSv/gFfOELcNVVSwdDxx5b9q/WYJ07\nw6JFsGDB0uGeJEmSWg/HBkqSJKnMDK8kqVzqBkg9etQfXs2eDfvuG2tW1dZGB9WLL8LGG8P998cx\nt90GP/tZ3J8wIdaNGjQIrrgCJk6Ef/wjuprq+9zmpnPn2M6da3glSZLUWhleSZIkqcw8u5SkSiiE\nV++8A5dfDrvtFmtS/fnPEUZ16RIj/669FnbfHUaPhqeegu22iw4siKBq2DDYaSeYMgV++cs4tm5w\n1dx16hTbuXPzrUOSJEmVY3glSZKkMrPzSpIqoWvXGB341ltw0kkxUvDAAyO06ts3uq0Kxx12WNzf\nYYcIp84/H2bMgP33h8WLYc6cCLWmTYMzzsjvOzVEofPqo4/yrUOSJEmVU1vbvKcBSJIkqcXx0ihJ\nqoSUoFu3uH/xxXDvvTB2LPzrX3DooRFade26/Ot22y1+/B9wAIwZA9dfD0ceCe+/Dz//eXRhtSR1\nxwZKkiSpdbLzSpIkSWVm55UkVcrJJ8P228Pee0eY9eabEeJsttmKX7PddhFQPf54BFn77w9f/Sr8\n+tfQsWOTlV42hleSJEmtW5bF1vBKkiRJZWR4JUmV0qMH7LNP8XG/fit/Tfv2EVw99xwMHBihV1VV\n3FoiwytJkqTWzfBKkiRJFWB4JUnldNxxsP76jXuPlGDbbctTT94MryRJklq32trYGl5JkiSpjAyv\nJKmcrrwy7wqal4aGV/Pnw+23w8iRxfeQJElS81MIr1LKtw5JkiS1Kl4aJUmqnIaEVx9+GGuFHX44\nXHVVcf8ZZ8QaYIU/kNT10EMwenTjapUkSdKqs/NKkiRJFeDZpSSpcjp1iu2qhFePPgpjxsBqq8Hf\n/x77HnkELrgA7rkHfvADuPxyeO21eO7xx2HECBg8GA45BD74oKxfQZIkSZ/CNa8kSZJUAY4NlCRV\nTlUVdOy4auHV+PGx/cY34A9/gHnz4Ic/hIEDoWdP+NWviseedFIEWv36wYEHwoUXwtNPw803w447\nlve7SJIkaXl2XkmSJKkCPLuUJFVW586rHl6ttx7stx8sWAA33ggvvQTHHAN33QX//Ce89RYcfHAE\nWYsXw623wvnnw7//De3bw5e+BK+8UrnvJEmSpGB4JUmSpArw7FKSVFkNCa8GDoSddorX/uhHsX+/\n/WCNNWCXXaB37+iueuIJeOMNGDYsjhk2LDqvunaFPfeEE0+Eq6+GOXPK/a0kSZIExfAqpXzrkCRJ\nUqtieCVJqqyGhFcbbwyrrx5rW82bB0OHQq9eSx+XUgRcq6229P5114V774W+feH66+G442CTTaIr\nS5IkSeVl55UkSZIqwDWvJEmVtSrh1fvvw8yZEV4BHH44TJ4c4dWq2G676MrKMnjuOTj0UNhjDzjl\nFPjmN6NzS5IkSY2XZbE1vJIkSVIZeXYpSaqsVQmvxo+PbSG8SgnOPDPWsGqIlGDbbWOdrE02gZ/8\nJMYOzpjRsPeTJEnS0uy8kiRJUgV4dilJqqzOneGjjz79mLlz4d13i6P9Bg0qbw0bbBAdWM89B++9\nB/36wXe+A4sXl/dzJEmS2hrDK0mSJFWAYwMlSZVVSufVKafAbbdBt26www7Qp09lahk6FB5+GK67\nDq69FubPh003hRdfhIMOgn/8A7bYIkYLduxYmRokSZJak0J4lVK+dUiSJKlVMbySJFVWfeFVlsFm\nm8UVurfdBo89BnPmxO2SSypbz847x23ddeGXv4x9XbrAnXdCVVV0Y114IRx1FFRXR5A2cmRla5Ik\nSWqp7LySJElSBRheSZIqq0sXmD07AqvCFbnTpsFrr8X9kSPhjTdiXas11oCvfrVp6rrkEjjjDJg3\nLwK23/8e9t8/6jr1VPjxj4vHXn45fO97XlEsSZK0rCyLreGVJEmSysizS0lSZfXsCZ98AlOnFveN\nHx/b3XeH11+P+2efHV1Y7ds3XW1rrBHrYXXrBieeCBtuCMOHw5gx0QU2Ywbss0889/nPw9tvN11t\nkiRJLYGdV5IkSaoAzy4lSZXVr19sJ04s7iuEVz/9aYzqq6mBwYObvrZP06ULrLUW3H03XHMNjBsH\nn/scXHYZDBkCP/lJjBiUJElqywyvJEmSVAGeXUqSKqsQXk2YUNw3fnwEVkOHwte+Fh1YNTX51Lcy\n7dvDd74DTz4JXbvCySfD//0fnHsujBgRIxDnz4f33oOnnoI99oBttoH+/WH77eHWW2HhwuIfdiRJ\nklqTwjmO45UlSZJURoZXkqTK6t07/pixbHg1YEB0Xd1yC9x3X27llWzLLeHll2HUKHjnHbjuOnj8\ncVh/fVhzTVhvPdhpJ3j11RhFOGxYrKd1yCExnnCjjaJr6+GHi2tDSJIktXR2XkmSJKkCqvMuQJLU\nytXUxFpSEybAm29GR9L48fDZz8bzLekq3aqq4njDI4+MMYK33AIffRTh1Ny5cNJJMXIQYMECOO88\nmD4dnnkmurYAjj46RhG2pO8uSZJUn8JFOYZXkiRJKiPDK0lS5fXtCzffHLfRo2P9qwMOyLuqxhs0\nCH72sxU/36FDhFcQVyVPnQoXXABXXgknnhivlyRJasnsvJIkSVIFeHYpSaq83r2L9++9FxYvho03\nzq2cXLRrF6MFDzkkHk+cmG89kiRJ5WB4JUmSpArw7FKSVHkdOhTvP/BAbNtaeFXQt29sDa8kSVJr\nUAivHIcsSZKkMjK8kiRV3llnwcUXwwYbwAsvxL62Gl716AGrrw5vvZV3JZIkSY1n55UkSZIqwLNL\nSVLl9ewJp54KAwfG43XWge7d860pLylBnz6GV5IkqXXIstgaXkmSJKmMPLuUJDWdAQNi21a7rgr6\n9nVsoCRJah3svJIkSVIFeHYpSWo6/fvHtq2HV4XOq8KVypIkSS2V4ZUkSZIqwLNLSVLTsfMq9OkD\nc+fC++/nXYkkSVLjFMKrlPKtQ5IkSa2K4ZUkqekMHgyrrQY77JB3Jfnq2ze2jg6UJEktnZ1XkiRJ\nqoDqvAuQJLUhPXvCvHl5V5G/QYNi+/LLMGxYvrVIkiQ1RmEMsuGVJEmSysizS0mSmlrfvtClC4we\nnXclkiRJjWPnlSRJkirAs0tJkppau3aw9dbw4ot5VyJJktQ4hleSJEmqAM8uJUnKw9Zbw0svweLF\neVciSZLUcIXwKqV865AkSVKrYnglSVIettkm1v96/fW8K5EkSWo4O68kSZJUAZ5dSpKUh622iu3Y\nsfnW0dwMHw4/+lHeVUiSpFJlWWwNryRJklRG1XkXIElSm7TOOrGdNSvfOpqbCROgc+e8q5AkSaWy\n80qSJEkV4NmlJEl56NIlth9+mG8dzc2CBTFOUZIktQyGV5IkSaoAzy4lScrD6qvHwuZz5+ZdSfOy\ncCF8/HHeVUiSSpRS2iOlND6l9GZK6bR6nq9JKd2x5PlnU0q9l3m+V0ppbkrplKaqWWVWCK9SyrcO\nSZIktSqGV5Ik5SGlGI9n59XSFi6080qSWoiUUhVwFbAnMAg4KKU0aJnDvg3MyrKsP3AZcNEyz18G\nPFTpWlVBdl5JkiSpAjy7lCQpL4ZXy7PzSpJakmHAm1mWTcyybAFwO7DvMsfsC/xhyf27gS+mFC06\nKaWvABOBcU1Uryohy2JreCVJkqQy8uxSkqS8dOni2MBlueaVJLUkGwDv1Hk8acm+eo/JsmwRMBtY\nK6XUCfgR8NNP+4CU0tEppVEppVHTp08vW+EqIzuvJEmSVAGeXUqSlBc7r5Zn55UktST1LXKUlXjM\nT4HLsiz71Ks4siy7NsuyIVmWDenRo0cDy1RFGV5JkiSpAqrzLkCSpDbLzqulLV4co4fsvJKklmIS\n0LPO4w2BySs4ZlJKqRroBswEtgVGppQuBroDtSml+VmWXVn5slVWhleSJEmqAMMrSZLy0rkzTJmS\ndxXNx8KFsbXzSpJaiueBASmlPsC7wIHAwcsccz9wGPAfYCTwWJZlGbBj4YCU0jnAXIOrFqoQXqX6\nmuwkSZKkhvHSKEmS8lJK59W778Kll8J77zVNTXlasCC2ixYVgyxJUrO1ZA2r44GHgdeAO7MsG5dS\nOjeltM+Sw24g1rh6EzgZOC2falUx2ZJJkXZeSZIkqYzsvJIkKS9duiy95tVdd8F//wtnnRWP334b\nBg2Cjz6CqVPhoovyqbOp1A2sPv4Y2rfPrxZJUkmyLHsQeHCZfWfXuT8fOGAl73FORYpT03BsoCRJ\nkirAs0tJkvLSufPS4dXBB8PZZ8M998TjRx+N4GqjjeBvf8unxqZUN7xy3StJkloGwytJkiRVgGeX\nkiTlpUuXCKcKf/Tp1Cm2xx4Ln3wCzzwDa6wBxxwDY8fGCMHWzPBKkqSWx/BKkiRJFeDZpSRJeenc\nOdaJmDcPpk2D2bPhy1+OEYF//jP85z+w3Xaw555x/Kp0X2VZjBncb7947/rMmgXTpzf+e5RLYc0r\niLGBkiSp+SuEVynlW4ckSZJaFde8kiQpL126xHbu3FjrCuB734NXX4Vf/hLGjYMDDoAttoCePeHe\ne+Hb34ZLLoHXXoPrr1/6D0Wvvgq/+hUMGQJPPQU33xxXQY8dC088ARtsUDw2y2CPPeCVV+AXv4hu\nr0qaPBmOOw4mTYIddohbhw4xIrGqCr7//dgW2HklSVLLkGWxtfNKkiRJZWR4JUlSXjp3ju2HH0bw\nBLD55hHynHJKPN522wiovv51uPxyuOACOOOMeO7QQ2GXXeL+U0/BbrvBokVw3XWx77zzYt/w4RFU\nPflkjCEEePppeO456NcvPm/yZDj//MZ9n5kz4YorYMIE+OIXI5h67z1YZx14/PG4MnvwYPjtbyNk\nA1hzzRgXeMstcPXVxfey80qSpJbBsYGSJEmqAM8uJUnKS6HzqhBede0K668PJ58Mf/kLXHhhhEAA\nBx4YIc8ZZ8CIEREIXXhhhE777Rf7+vSJdbEuuSS6ss48M8YO3ncfvP467L13jCScNw9++tMIjsaM\ngSOPhJ/9DP74x/iMSy+FYcPis//0p/j8yZOjK2zAAPjsZ2HUqKW/y8cfR0h2zjkx8vBb34LRo+M7\nvfEGfPWr8fjJJ+GDD2I9r/vugylT4MUXYf78CLgK7LySJKllMLySJElSBdh5JUlSXgqdV3PnRni1\n6abFMYB77RW3gm22iVtNDdx5J1x1FfzoR7D99jBjBowcGYHUuuvCD36w9Ofsumt0Nh14YIwfXG01\nmDMHrrwyavjNbyJgOuoouPjiGCU4bFiM+Bs5Eg4/HB59NIKvESMihBo+HC67LOrp2jW6pkaPjkBq\nhx3ghRdg551jNOCyOnSIjrKCddeN7YcfFvfZeSVJUstgeCVJkqQKMLySJCkvdTuvXn89RvytSEox\n6q+mJv449IMfwL//HWHR738Phx326Z81cmQEZDfcALNnx1paha6u6mq4664Iw556Cm66KUYSfvJJ\njC+8/vqo9dlnYaut4H//i26vww8vvn/HjhGo7bNPPB4+vPR/h5qa4r9DgZ1XkiS1DIXwqu46nJIk\nSVIjGV5JkpSXQng1bVqM++vf/9OPX2214v2qKrjjDnjpJRg6tLTPGzgQLrqo/ud69IAbb1x6X01N\nrGH185/D4sXQrVvs32gjeP75GAHYvTvMmhVrZ/XqVVody6quju9j55UkSS1PlsXWzitJkiSVkeGV\nJEl5KYwNHDMmtgMGrNrra2pivF+lFeqsq1072GWX8n1Gx44xPrHAzqtw0knw1lvRYSdJUnPk2EBJ\nkiRVgOGVJEl5KXRevfhibFc1vGpNamrsvKrPCy/Ae+/lXYUkSStmeCVJkqQK8OxSkqS8dO4coc2o\nUfF4ZWMDWzM7r+o3derSoZ4kSc2N4ZUkSZIqwLNLSZLyUl0Ne+0VXUbrrANdu+ZdUX6W7bwyvApT\npy4d6kmS1NwUwquU8q1DkiRJrYrhlSRJeTrooNi25ZGBEJ1XCxcWHzs2EObPhzlz4KOPin8YlCSp\nucmy2Np5JUmSpDIq6ewypbRHSml8SunNlNJpn3LcyJRSllIaUr4SJUlqxUaMgO7dYbPN8q4kXzU1\nSz+28yq6rgr895AkNVeODZQkSVIFVK/sgJRSFXAVMByYBDyfUro/y7JXlzmuC3AC8GwlCpUkqVVa\nbTX4z39g7bXzriRfHTsW73fqBL/7HUybBn/9a3415a1uePXhh7FGmiRJzY3hlSRJkiqglLPLYcCb\nWZZNzLJsAXA7sG89x50HXAzML2N9kiS1fptsYnhVt/OqcP/hh2Hx4nzqaQ7qhleueyVJaq4MryRJ\nklQBpZxdbgC8U+fxpCX7/r+U0tZAzyzL2vDl0ZIkqcHqdl7NnBnbRYtg0qR86imX2lp47bXieiCr\nwvBKktQSFMKrlPKtQ5IkSa3KSscGAvWdgf7/v8CklNoBlwHfWukbpXQ0cDRAr169SqtQkiS1fsuu\neVUwcSJstFHT1tJQs2bBjBnw97/DY49BVVWsVfXXv8Kuu8Itt8C665b+foZXkqSWoHCBhp1XkiRJ\nKqNSzi4nAT3rPN4QmFzncRdgc+DxlNL/AdsB96eUhiz7RlmWXZtl2ZAsy4b06NGj4VVLkqTWpW7n\n1fjxEQBBhFctwTPPQO/eMHAgHHccjBkTAdYDD8BRR8Xz++wT63iVatk1ryRJao4cGyhJkqQKKKXz\n6nlgQEqpD/AucCBwcOHJLMtmA/9/oY6U0uPAKVmWjSpvqZIkqdWq23k1YAD07QvV1Q0Pr/76V9hm\nG1h//fLU99JL8MYb8LnPxXtOmQJXXgkLF8IHH8DNN8f+X/0q1jDbfnuYPz+O69MH9toLvvKV6Lza\nc0+46CLYfPNP/8ypU2MEU5bZeSVJar4MryRJklQBKw2vsixblFI6HngYqAJuzLJsXErpXGBUlmX3\nV7pISZLUyhU6r6qrI7Cpro5xgaWEV//7H/ztb3D00fHaG26AI4+E4cPhkUdW/vqJE+GEE+Dpp2Gr\nrSL46tw5QqOU4I474BvfiDW4evSAJ5+Eb387uqkKodsBB8AFF8AGdZYF7dgxgiuIrquXXoK77oKr\nr4bPfx5OPRW6dIHBg+Gdd2D0aDjoIHj99eg+e+YZ6NkT3n7b8EqS1HwZXkmSJKkCSum8IsuyB4EH\nl9l39gqO3aXxZUmSpDalEAK1b1/c17fvp4dXZ58Njz8eI/XGjIFevSI8OuaYCJn+/nf4z3+iC2pZ\nWQbvvw+zZ0c31ccfRwD1+9/DvvtChw4RUn3pS3DvvbDjjvCTn8Qxm24a73HLLXDwwcu/94pssUXc\njj4adt8dzjpr+WMuuaR4f5114Igj4MILDa8kSc1XIbxK9S2XLUmSJDVMSeGVJElSRRU6rzp0KO7r\n2xf+9KcVv+b222OUH0Sn1M9+FmP8uneHUaOio+kXv4B77ln6dePGRXg0eckSnl27wrPPRijVvz/8\n+MfQrx/sthv8+c+wxx5Rx+qrw1NPxfv17w8HHtiw77rhhvDyyxGcffghvPpqhG09e0YgNmQIbLdd\n/FssWBDhlWteSZKaKzuvJEmSVAGGV5IkKX/1dV717w8zZsDMmbDmmksfP3NmBFeHHBIh04wZMYYP\n4LbbogvrkEPgmmtgzhz45z8j7Npll+huqq2FSy+N133lK8VuqtNPh1NOKdYxfnyEaIXHgwbFrbGq\nquI7rblmjEcsOPnkpY/r0CE+286rVTNtWnTlfe1reVciSa1flsXWzitJkiSVkeGVJEnKX6Hzqm54\ntdlmsR03DjbfHNZYo/jcqFGxPeII2HXX6LgaMCCO69cvnvv61+Hyy+OYe++NP6rdfnt0Zv31r7Hu\nVH3q1rDxxuX5fo3RpYvh1ar6zW/gnHNizbGhQ/OuRpJat9ra+P9YwytJkiSVkX39kiQpf/V1XhXC\nq+uvh7XXhtGji889+2z8kWzw4OLr9t23GFxBjN7baKMY+bfbbtGN89prMH36ioOr5qhzZ8cGrqrx\n42N79dX51iFJbUFtrSMDJUmSVHaeYUqSpPzVt+ZVz57RdXTzzfGHsccfj/1vvw333QebbALduq34\nPVOCm26CO+6Ahx6Kzq1NNoHqFtZ43rmznVer6vXXY3v77TBrVr61SFJrV+i8kiRJksrI8EqSJOWv\nvs6rlGJ9qcJC8M8/D6++CltuCWPHwve+t/L33WmnWPeoJf9RzfBq1WRZrIe26aYwfz68+GLeFUlS\n62bnlSRJkirAM0xJkpS/+sIriDWsCp5+Gr785ejSGjcOjjmm6erLk2terZpp02DOnBgjCRF4SpIq\nJ8sMryRJklR2nmFKkqT8FcYGrii82ndfmDQJ/u//4M47YcCAJi0vV655tWreeCO2O+0UYyVfey3f\neiSptbPzSpIkSRXQwhZ9kCRJrVKh86rumlcA3/xmPLfxxrHO1Ve+Ajvu2PT15cmxgaumsN7VwIEx\nOtDOK0mqLMMrSZIkVYDhlSRJyt+KOq/WXDPGAy5YACeeCCed1PS15a1LFzuvVsXrr8f/jjbaKNZM\ne+CBvCuSpNattrZlry0pSZKkZsnLoyRJUv5WtOZVQYcO8KtfQe/eTVZSs7HmmjBzZvxxUCv38suw\nySZQXR2dV1Onxr+fJKky7LySJElSBXiGKUmS8reizivBeuvB4sUwfXrelbQMY8bAVlvF/UGDYvvK\nK/nVI0mtXZYZXkmSJKnsPMOUJEn5W9GaV4L114/tlCmlHV9bC088EYFXWzNtGkyeXAyvtt02/qD6\nyCP51iVJrZmdV5IkSaoAzzAlSVL+7LxasUJ4NXlyacf/8Y+wyy5w9NEwfDg89ljFSmt2XnoptoXw\naq21YKed4N5786tJUuvzgx/AddflXUXzYXglSZKkCvAMU5Ik5W9la161ZeutF9tSO69+97vY3ngj\nPPooXHppPL7qqv/X3p1HR1nfexz/fE0AEUQQMBJBwIoJYBUxKot1Qb14XfEodcFqte5g8RT1VE9d\nClq1dbm4YN23qhdRXNCKWMS6Va9REUWIIgpaF7CKsjQI5Hf/+D7ThJAgapLn+WXer3PmzMwzz8z8\nhu+ZIb/5zPf3SJdd1vDjS9NTT0mrVlVfnznTz3fcsXrbsGHS7NnS/ff7vlVVvsQVAPxQ06ZJU6ak\nPYrsqKqSzNIeBQAAAJoZwisAAJA+Oq/qt+WWfr4hnVcffOBLBl5wgXTlldJRR0lPP+2B1qhRvn1D\nQ7CsKy+X9t9fOuec6m1vvil17eodVzmHHSYVFkrHHCOddpp00EHSkCHSt982/ZgBNA+lpVJFRdqj\nyA46rwAAANAI+AsTAACkj2Ne1a9VKw9jPvlE+uab9e97003+BeJJJ/myVqNGeUhz4olS375+HKxR\no6SRI6Wvvmqa8TeW5cv9fOrU6m3z5kklJWvvt/XW0vz50sknezfak09Kzz4rnX123Y87caL09tuN\nMmQAzURJifT++4TgOSEQXgEAAKDB8RcmAABIH51X61dcLN16q4dYCxbUvc/SpR5eHX64BzaSNHCg\nh1ZDh0ovvCDtuac0ebI0YYI0eLC0ZIm0erW0YkXTvZaaVq70569t1Sp/vZ99Vv99lyzx8/nzq7ct\nWCB1777uvt26SePGSa1b+7/H6NHSdddJ11/vSwrmLF0qHXusNGKEdxIAQF1KS/3HADU/f/IZnVcA\nAABoBPyFCQAA0scxr9avuNhDntWr1w5bckKQLrrIA50xY6q3b7SRL6U3darUvr2HNRMmSH/9qzRn\njnTWWdJWW0lt2kiXXNJ0r0fyY3AVF0tlZdLChb7ttde8S2zoUO+UGjpUeuMN7zibPNk7yi66yPfN\ndY6tWuX/LitXethVV3glSUVF0osv+rFq/vQnD+/OPFPafnvp3nt9n+ef98eaNUu67TaOjQWgbrkO\nz7lz0x1HVhBeAQAAoBEUpj0AAAAAFRb6F1+EV3Xr1Kn68gcfrHv7738vXXONH9Npt93Wvq2goPry\n9tv7SZIOPli66y7v5tpjD+nyyz0wKipa+/4rV0pXXCH17Cn99KfecZDrlNsQIUhm0r//LU2ZIh14\noPTSS7584c9+Js2cKfXqJe28s1828/ucfrp0yy1S//7Vj9WqlY9n9Oi1lz2cO7d6TPWFV5K0007V\nl6dOlZ57zl/bCSd4p1e7dv4cfftKp5ziYdedd274awWQHwiv1lZV5Z/gE4HqAAAQNklEQVTdAAAA\nQAPi51EAACB9Zh5qDBiQ9kiyad686su1w6uKCukPf/Cl7iZM2PDHvPhiaYstvMPo1lulykrpvPPW\n3e+mm7zb6bjjPPxp314aNkx69dX1P/6kSVLv3h4EzZ0rHXqodOSRHlQdeaR/+fvUU95tNXKkh0+H\nH+6vb9kyfy2zZvnjjB3rx6uaNMkfe/bs6mUDJam8vHo5xdySid+lbVvpgAO8o+v00/04V489Jg0a\n5B1Yp57q4V5FRf2PsXIlx7wB8lG7dlKXLuv/fMgndF4BAACgEdB5BQAAsuGxx9IeQXaNGyf95jd+\nTKYPP1z7tgsv9GM5XXXV9/vle//+0qefVn/heO650mWXebh0+ukeUi1fLl16qbTXXt7Z9e670j/+\nId1zj3dNTZ4s7b23f4G7ww7VjzVzpodppaV+TJjevX1sF14ovf661LKlB2KtW/vzXX113WPs3dtP\nObnlBWfP9s6rTTf1L5EnTpSGD/fb1td5VZeOHaXx46VDDpH228+XKtxkE+9mu/12D9HGj6/ef/ly\nX3Zx0iTpiSe80236dLoOgHxTWkrnVU4IhFcAAABocIRXAAAAWbffftJbb3mnUM3Oq6oq6W9/8+Cm\n9nJ/G6Lml41jx3qwdP750nXXSS+/7OeLFkmPPCL16+enn/9c+t3vpH328W653FJ+ffp4mDVlih9T\nqlMn6ZlnPNh66CHv3OrX78f9O3Tr5oFVrvOqY0d/3HHjvIvMTOra9Yc99j77+BfRufCrqMhf6+23\nSxdc4K+nosI7s7780p9vwAB/jTNmSEOG/LjXBiAuJSUenOeWRs1ndF4BAACgEfAXJgAAQCx69PDw\nqrLSl/qbOtWDlMGDf/xjFxZKTz4pPfusdxfttpt3RJ18sjRw4Nr7duzox4u6/nq//YYbPOTaZRfp\nnHO8C+uJJzzwGTzYH+fHBleSf0Hcp09151WHDtJJJ/n2u++Wiou9q+uH2m47D+Nyzj9fWrHCO9Iq\nK6Wjj/bnmj5d+uQTf41dukhjxvj1+nzxhQd8kn/RDSB+paX+ObR4cdojSR/hFQAAABoBf2ECAADE\nomdP/7J06609NDriCN8+aFDDPL6ZtOeeHmLtuqsvoffHP9a9b7t2fqyq666TzjhDevBBD2hGjpSm\nTfPjYzWGvn3XDq+6dfMlFSUP8hpSnz7e2XXttR7mvfGGd2INGSIVFPhxuiZM8OUUe/aU9tjDjyX2\n9dd+/zVrvDutc2cPxs44w8O1zp39uGEffVT/cxNyAdlWUuLnHPfKw6t87z4DAABAg2PZQAAAgFj0\n6OHnixdLxxwj3Xefd0Ftt13DPs+gQdKjj36/++y5p/Svf0lt2zbsWGrr29cDJMmPuyVJl18uLVvm\ntzW0a67xx548WbrjDj82Vk3Dhvkxvm6+Wfr7330Jw7/8xa+/9JIfH+vMM70z7MYbpSOP9H+jiROl\nQw/1cGuzzTyIKynxQGzsWA8Qe/SQTjxROu00XzayqKj6C3MA6Sot9fO5c6s/i/IVnVcAAABoBBZS\n+lVnWVlZKC8vT+W5AQAAojRvntSrl3c7jRjhHVhDhnz/oClm06Z5R5jkSwbeckvjP2cIHmBtuul3\n7/vCC9Kxx0oLFvj1Qw7xY4bNmuUB1IgR3qEwZYqHV2b+xW9NHTr4fnPn+jHNCgq8i0vyrq3iYumg\ng6Rf/9qPvZUhZvZaCKEs7XGgecrUHLKqSmrTxt+bvXp5QL3zzmmPKh2HH+4dqG+9lfZIAAAAEJn1\nzSHpvAIAAIjFttv6sZdyx2V65hkPM/JJze6qDh2a5jnNNiy4kqTdd5feeUe65x6vzYEH+v133NFP\nOQcfLM2f7/tUVkpLl3o9ly2TfvlLX5ZR8q6uu+7y+y5e7Pd5910/DteVV3o4tuuu0vDhUvfuDf7S\nAdRjo408WK6slObM8ff+bbd5V2y+ofMKAAAAjYDwCgAAICa54EqSdtklvXGkpbjYl9n7+mupffu0\nR1O3TTaRTj31u/fLLQPZpo0v/3jiievu06+fn2qrqPBjcT38sDRpklRWRngFNLX+/aXnn5def10a\nPdo7JrfZRhowIO2RNS3CKwAAADQCwisAAADEw8y7r156qek6r7KopES64QY/ffHFhneGAWg4kyZJ\nS5b4+/Hxx6Utt5TuvFPabTfp5JM9eD7mGGn8eGnRIj9v0cLD55491/4xQl0qK6V//tM7Lbt0kUaO\n9OeQpIULpeeekwoLpW7dfCnSigoPwxcv9ufr08eD7WXLpAMO8LDNzDs916zZ8B8AVFZK335b3REq\nSStWSK1b++XVq/1xAQAAgAZEeAUAAIC4EF6trVOntEcA5KeiIj9JHiAPGyY98IC0776+hKAkjRnj\nwY/kywuuXi29+KKHTL17Sxdf7MHS/PnexVVR4cHSwoXSQw/5/htv7AHSJZd4gLRypZ9qKiz0EO2b\nb3yf7t2l6dOl++7z2y+80D87N99ceuUVH1Pnzr5f585+38WL/Rh/BQXeSfX++75M6aJF/hhbbunh\n26pV0mef+WOtWeOdsIMGNfo/NwAAAPIL4RUAAADisv32fk54BSBLfvELD4uOP96XDzziCO9QOvVU\nacYMadw4qW1b6YorvKNq6lQ/Xt1PfiK9/bY/RmGhn9q0kU47zR/nsMOk5cu9u+vTTz3M6txZ2m8/\nv8+773p4VFy89niqqjyQatHCQ7UHHvDQauRI7+R67z3p4499LDNmeDhVUOCB1Jo1Uteufly97t2r\nw6wQ/HL37h6wtWrl4x86tGn/rQEAANDsWQghlScuKysL5eXlqTw3AAAAIjZzpjRwoPTOO770FjLD\nzF4LIZSlPQ40T5mfQ4YgXXWV9Oc/+1J/w4evf//PP/fQqUULadQoaa+9fKk/jh8FAACAPLG+OSSd\nVwAAAIhLv37ezcAxVgBkiZl09tl+2hBFRb6UYGEhgRUAAABQC+EVAAAA4kNwBaA5aNky7REAAAAA\nmcTPuwAAAAAAAAAAAJAZhFcAAAAAAAAAAADIDMIrAAAAAAAAAAAAZAbhFQAAAAAAAAAAADKD8AoA\nAAAAAAAAAACZQXgFAAAAAAAAAACAzCC8AgAAAAAAAAAAQGYQXgEAAAAAAAAAACAzCK8AAAAAAAAA\nAACQGYRXAAAAAAAAAAAAyAzCKwAAAAAAAAAAAGQG4RUAAAAAAAAAAAAyg/AKAAAAAAAAAAAAmUF4\nBQAAAAAAAAAAgMwgvAIAAAAAAAAAAEBmEF4BAAAAAAAAAAAgMwivAAAAAAAAAAAAkBmEVwAAAAAA\nAAAAAMgMwisAAAAAAAAAAABkBuEVAAAAAAAAAAAAMsNCCOk8sdliSQtSefJ1dZL0RdqDwA9G/eJF\n7eJG/eJG/eJG/eLV3GvXPYTQOe1BoHliDokGRP3iRe3iRv3iRv3iRv3i1dxrV+8cMrXwKkvMrDyE\nUJb2OPDDUL94Ubu4Ub+4Ub+4Ub94UTugeeC9HDfqFy9qFzfqFzfqFzfqF698rh3LBgIAAAAAAAAA\nACAzCK8AAAAAAAAAAACQGYRX7ua0B4AfhfrFi9rFjfrFjfrFjfrFi9oBzQPv5bhRv3hRu7hRv7hR\nv7hRv3jlbe045hUAAAAAAAAAAAAyg84rAAAAAAAAAAAAZEZeh1dmtr+ZVZjZPDP7bdrjwbrM7HYz\nW2Rmb9fYtrmZPW1m7yXnHZLtZmbXJvWcZWb90xs5JMnMupnZDDObY2azzWx0sp0aZpyZbWxm/2dm\nbya1+32yvaeZvZLUbqKZtUy2t0quz0tu75Hm+OHMrMDM3jCzx5Pr1C8SZvahmb1lZjPNrDzZxmdn\nJMysvZk9aGZzk/8DB1I/oHlgDpl9zCHjxhwyXswhmwfmkPFiDhk35pB1y9vwyswKJN0g6b8l9ZF0\ntJn1SXdUqMOdkvavte23kqaHEHpJmp5cl7yWvZLTKZJubKIxon6rJY0JIfSWNEDSyOR9Rg2zb6Wk\nISGEHSX1k7S/mQ2QdIWka5LafSXpV8n+v5L0VQhhW0nXJPshfaMlzalxnfrFZe8QQr8QQllync/O\neIyXNDWEUCppR/n7kPoBkWMOGY07xRwyZswh48UcsnlgDhk35pDxYg5Zh7wNryTtKmleCGF+COFb\nSf8r6dCUx4RaQgjPSfqy1uZDJd2VXL5L0rAa2+8O7mVJ7c2sS9OMFHUJIXwaQng9ubxU/sG7lahh\n5iU1WJZcbZGcgqQhkh5MtteuXa6mD0rax8ysiYaLOphZV0kHSro1uW6ifrHjszMCZtZO0h6SbpOk\nEMK3IYQlon5Ac8AcMgLMIePGHDJezCHjxxyyWeKzMwLMIeuXz+HVVpI+qnH942Qbsq8ohPCp5H/Y\nStoi2U5NMyxpId9J0iuihlFIlguYKWmRpKclvS9pSQhhdbJLzfr8p3bJ7V9L6ti0I0Yt/yPpXElV\nyfWOon4xCZKmmdlrZnZKso3PzjhsI2mxpDuSJVduNbM2on5Ac8D7NV58BkeIOWR8mENGjzlk3JhD\nxos5ZD3yObyq69cAoclHgYZETTPKzNpKekjSWSGEb9a3ax3bqGFKQghrQgj9JHWV/9K4d127JefU\nLkPM7CBJi0IIr9XcXMeu1C+7BocQ+suXAxhpZnusZ1/qly2FkvpLujGEsJOk5ape3qEu1A+IB+/X\n5oeaZhRzyDgxh4wXc8hmgTlkvJhD1iOfw6uPJXWrcb2rpE9SGgu+n89zrZDJ+aJkOzXNIDNrIZ90\n3BtCmJxspoYRSVqVn5WvOd/ezAqTm2rW5z+1S27fTOsu14KmM1jSIWb2oXxJoyHyX9FRv0iEED5J\nzhdJelg++eezMw4fS/o4hPBKcv1B+USE+gHx4/0aLz6DI8IcMn7MIaPEHDJyzCGjxhyyHvkcXr0q\nqZeZ9TSzlpKOkvRYymPChnlM0vHJ5eMlPVpj+3HmBkj6OtdaiXQk6x3fJmlOCOHqGjdRw4wzs85m\n1j653FrSvvL15mdIOiLZrXbtcjU9QtIzIYRm+auPGIQQzgshdA0h9JD///ZMCGGEqF8UzKyNmW2a\nuyzpvyS9LT47oxBC+EzSR2ZWkmzaR9I7on5Ac8AcMl58BkeCOWS8mEPGjTlk3JhDxo05ZP0snz9X\nzOwA+a8ICiTdHkK4NOUhoRYzu1/SXpI6Sfpc0kWSHpH0gKStJS2UNDyE8GXyR+71kvaXtELSCSGE\n8jTGDWdmu0t6XtJbql4z+Xz5muXUMMPMbAf5wSAL5D90eCCEMNbMtpH/CmtzSW9IOjaEsNLMNpZ0\nj3xN+i8lHRVCmJ/O6FGTme0l6ewQwkHULw5JnR5OrhZKui+EcKmZdRSfnVEws37yA123lDRf0glK\nPktF/YCoMYfMPuaQcWMOGS/mkM0Hc8j4MIeMH3PIuuV1eAUAAAAAAAAAAIBsyedlAwEAAAAAAAAA\nAJAxhFcAAAAAAAAAAADIDMIrAAAAAAAAAAAAZAbhFQAAAAAAAAAAADKD8AoAAAAAAAAAAACZQXgF\nAAAAAAAAAACAzCC8AgAAAAAAAAAAQGYQXgEAAAAAAAAAACAz/h8wXRVlrzX6swAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<Figure size 2160x1440 with 4 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(2, 2, figsize=(30, 20))\n",
    "# Training Loss\n",
    "ax[0][0].plot(losses, label='loss')\n",
    "ax[0][0].set_title('Training Loss')\n",
    "ax[0][0].legend()\n",
    "\n",
    "# Training accuracies\n",
    "ax[0][1].plot(accuracies, label='acc')\n",
    "ax[0][1].set_title('Training Accuracy')\n",
    "ax[0][1].legend()\n",
    "\n",
    "# Validation loss\n",
    "ax[1][0].plot(losses_valid, label='loss', c='r')\n",
    "ax[1][0].set_title('Validation Loss')\n",
    "ax[1][0].legend()\n",
    "\n",
    "# Validation accuracies\n",
    "ax[1][1].plot(accuracies_valid, label='acc', c='r')\n",
    "ax[1][1].set_title('Validation Accuracy')\n",
    "ax[1][1].legend()\n",
    "\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXzV1Z3/8dcnNytZIWHfwiqrYIwI\nrri14F5rq1Tr1C50nV+n1ulo27HrzNB2ql1Gq4xV2xkr2jq11qK4V+vGYnEBBCKLhC0hQAiQPef3\nx/mGhJCQC9zkbu/n43Ef93u/93vv9xxL3/fkfM/3HHPOISIi8S8l2gUQEZHIUKCLiCQIBbqISIJQ\noIuIJAgFuohIglCgi4gkCAW6iEiCUKBLwjOzTWZ2YbTLIdLTFOgiIglCgS5Jy8w+Z2ZlZrbbzB43\nsyHBfjOzO8yswsyqzextM5sSvHexma02sxoz22pmN0e3FiJtFOiSlMzsfOA/gI8Dg4HNwKLg7Q8B\n5wDjgQLgGqAqeO/XwOedc7nAFOD5Xiy2yFGlRrsAIlFyHXCfc+5NADO7FdhjZsVAI5ALTACWOufW\ntPtcIzDJzN5yzu0B9vRqqUWOQi10SVZD8K1yAJxz+/Gt8KHOueeB/wLuBHaa2UIzywsO/ShwMbDZ\nzP5qZrN6udwiXVKgS7LaBoxsfWFm2UAhsBXAOfcL59ypwGR818s/B/uXOeeuAAYAjwGP9HK5Rbqk\nQJdkkWZmma0PfBDfaGbTzSwD+HfgDefcJjM7zcxON7M04ABQBzSbWbqZXWdm+c65RmAf0By1Gol0\noECXZLEYqG33OBv4V+BRYDswBrg2ODYP+G98//hmfFfMfwbvfRLYZGb7gC8A1/dS+UW6ZVrgQkQk\nMaiFLiKSIBToIiIJQoEuIpIgFOgiIgkianeKFhUVueLi4midXkQkLq1YsWKXc65/Z+9FLdCLi4tZ\nvnx5tE4vIhKXzGxzV++py0VEJEEo0EVEEoQCXUQkQcTU9LmNjY2Ul5dTV1cX7aL0mszMTIYNG0Za\nWlq0iyIicS6mAr28vJzc3FyKi4sxs2gXp8c556iqqqK8vJxRo0ZFuzgiEudiqsulrq6OwsLCpAhz\nADOjsLAwqf4iEZGeE1OBDiRNmLdKtvqKSM+JuUDvTl1jM9ura2luaYl2UUREYkrcBXpDUwuVNfXU\nNUY+0Kuqqpg+fTrTp09n0KBBDB069NDrhoaGsL7jxhtvZO3atREvm4hId2Lqomg4MlL9b1B9UwvZ\nGZH97sLCQlauXAnAd7/7XXJycrj55psPO8Y5h3OOlJTOfwvvv//+yBZKRCRMcddCT0tNwTAamnpv\n5a+ysjKmTJnCF77wBUpKSti+fTvz58+ntLSUyZMn8/3vf//QsWeddRYrV66kqamJgoICbrnlFqZN\nm8asWbOoqKjotTKLSPKJ2Rb69/68itXb9nX6Xm1DMykpkJEaOqbvnDQkj+9cNvm4yrN69Wruv/9+\n7r77bgAWLFhAv379aGpq4rzzzuPqq69m0qRJh32murqac889lwULFnDTTTdx3333ccsttxzX+UVE\nuhN3LXQADHp75bwxY8Zw2mmnHXr90EMPUVJSQklJCWvWrGH16tVHfCYrK4u5c+cCcOqpp7Jp06be\nKq6IJKGYbaEfrSW9cdcBmltaGDsgt9fKk52dfWh7/fr1/PznP2fp0qUUFBRw/fXXdzqWPD09/dB2\nKBSiqampV8oqIskpLlvoIYPmKI5a3LdvH7m5ueTl5bF9+3aWLFkSvcKIiARitoV+NCkpRnNLL/e5\ntFNSUsKkSZOYMmUKo0eP5swzz4xaWUREWpnr7c7oQGlpqeu4wMWaNWuYOHFit5/dXl3Lrv0NTBmS\nlxB3WoZbbxERM1vhnCvt7L347HJJsWA8eLRLIiISO+Iz0INWebMSXUTkkPgM9JQg0KPYjy4iEmvi\nMtBTQ77YjdEc6iIiEmPiMtDTghZ6U7Na6CIirboNdDO7z8wqzOzdLt6/zszeDh6vmtm0yBfzcIda\n6JpCV0TkkHBa6A8Ac47y/kbgXOfcycAPgIURKNdRhVKMkFnEW+izZ88+4iahn/3sZ3zpS1/q8jM5\nOTkRLYOIyPHqNtCdcy8Bu4/y/qvOuT3By9eBYREq21GlhlIi3oc+b948Fi1adNi+RYsWMW/evIie\nR0SkJ0S6D/0zwJNdvWlm881suZktr6ysPKETpfbA3aJXX301TzzxBPX19QBs2rSJbdu2MX36dC64\n4AJKSkqYOnUqf/rTnyJ6XhGRSIjYrf9mdh4+0M/q6hjn3EKCLpnS0tKjp/GTt8COd7p8e2hjMy04\nSDuGKgyaCnMXdPl2YWEhM2bM4KmnnuKKK65g0aJFXHPNNWRlZfHHP/6RvLw8du3axcyZM7n88ssT\n4i5VEUkcEWmhm9nJwL3AFc65qkh8Z/fn7JkpdNt3u7R2tzjn+OY3v8nJJ5/MhRdeyNatW9m5c2fk\nTy4icgJOuIVuZiOA/wM+6Zxbd+JFChylJQ2wp7qWypoGpgyN7HwuV155JTfddBNvvvkmtbW1lJSU\n8MADD1BZWcmKFStIS0ujuLi40+lyRUSiqdtAN7OHgNlAkZmVA98B0gCcc3cDtwGFwF1BsDZ1NXFM\nJIVSUnA4mlscqaHIBXpOTg6zZ8/m05/+9KGLodXV1QwYMIC0tDReeOEFNm/eHLHziYhESreB7pw7\n6hAP59xngc9GrERhag3xphbHMa5E16158+Zx1VVXHep6ue6667jssssoLS1l+vTpTJgwIbInFBGJ\ngLicDx38KBfwgR5pH/nIR2g/rXBRURGvvfZap8fu378/4ucXETkecXnrP0BacLdok+ZzEREB4jjQ\nW1vojZrPRUQEiMFAD3cFpVCKYWY0xfl8LtFaMUpEEk9MBXpmZiZVVVVhhZyZkZYS+flcepNzjqqq\nKjIzM6NdFBFJADF1UXTYsGGUl5cT7rQAlTX1GLA/N6NnC9aDMjMzGTasV6a/EZEEF1OBnpaWxqhR\no8I+/pcPrmDtjhqe+/rsniuUiEiciKkul2M1IDeTin310S6GiEhMiOtAH5iXSU19Ewfqm6JdFBGR\nqIvrQB8Q9J1X1KiVLiIS14HeLycdgN0HGqJcEhGR6IvvQO/jA32PAl1EJM4DPTtooR9UoIuIJEag\nq4UuIhLfgd4nPUR6aoq6XEREiPNANzMKs9OpUqCLiMR3oAP07ZOuFrqICAkQ6IU5aqGLiEACBHq/\n7HRdFBURIQECXV0uIiJe3Ad6YXY6NfVN1Dc1R7soIiJRFfeBXhCMRa+ubYxySUREoivuAz0tWFs0\nnlcuEhGJhLgP9BTzgd6itTlFJMnFf6AHLfQ4XytaROSExX2gh4IaqIUuIsku7gO9tculWYEuIkku\nYQK9pUWBLiLJrdtAN7P7zKzCzN7t4n0zs1+YWZmZvW1mJZEvZtfaLor25llFRGJPOC30B4A5R3l/\nLjAueMwHfnXixQpfax96sxJdRJJct4HunHsJ2H2UQ64Afuu814ECMxscqQJ2R8MWRUS8SPShDwW2\ntHtdHuw7gpnNN7PlZra8srIyAqdWoIuItIpEoFsn+zpNV+fcQudcqXOutH///hE4NYSCcejqchGR\nZBeJQC8Hhrd7PQzYFoHvDUvQQNdFURFJepEI9MeBG4LRLjOBaufc9gh8b1haW+jqchGRZJfa3QFm\n9hAwGygys3LgO0AagHPubmAxcDFQBhwEbuypwnYmpHHoIiJAGIHunJvXzfsO+HLESnSMTHeKiogA\nCXCnaEiTc4mIAAkQ6CmHLoqqhS4iyS3+Az1FXS4iIpAAgd56UdQp0EUkycV9oB+aPld96CKS5OI/\n0DU5l4gIkACB3jrKRV0uIpLs4j7QtWKRiIiXOIGuLhcRSXIJEOj+WQ10EUl2cR/omj5XRMSL+0DX\nAhciIl78B7qmzxURARIg0EO6sUhEBEiAQNfkXCIiXvwHurpcRESABAh0rVgkIuLFX6BvfAnuvwT2\nbgHa3ykazUKJiERf/AV6fQ1s/hscrALaJudSC11Ekl38BXpGrn+urwE0Dl1EpFUcBnqefw4CPaQV\ni0REgLgM9M5b6MpzEUl2cRjorS30fUDbOHTN5SIiyS4OA/3wFrom5xIR8eIv0FMzICXtUKCbGWZa\nsUhEJP4C3cy30oNAB9+ProuiIpLs4i/QIQj0fYdehsxQj4uIJLv4DPScAVCzw283NzIv5RnGVP01\numUSEYmy1HAOMrM5wM+BEHCvc25Bh/dHAL8BCoJjbnHOLY5wWdsUjISty/320oV8L/RrWA/8eQ24\nZvjwf0BGTo+dXkQkFnXbQjezEHAnMBeYBMwzs0kdDvs28Ihz7hTgWuCuSBf0MAUjoLoctr8FS77F\nRjeYysyRsOJ+ePO38MgNsL+yR4sgIhJrwmmhzwDKnHMbAMxsEXAFsLrdMQ4IBoiTD2yLZCGP0Hck\ntDTBPedAaib3Nl3FuOET+VTzozDyDHjh3+COyTBiJuQN8Y9926H4LDhpLvTp5+9EqirzffFDSvzF\nVhGROBZOoA8FtrR7XQ6c3uGY7wJPm9k/AtnAhZ19kZnNB+YDjBgx4ljL2mZ4u9N/+Q0W/3Idl2YP\ngSs/6feNmAUvLoCKNX52Rhxk5MNbv4O0bBha4lv4ezb64wdNhaGnwtSPw8BJkNXX76/dC+8+Ci5Y\nDik1A075pMJfRGJSOIHeWXp1HFMyD3jAOfdTM5sF/I+ZTXHOHbYwnHNuIbAQoLS09PjHpfSf4Mei\njzoH+hYTSll/+LDFUWf7B8CeTdDUAIVjYftKWPZr2LoCLAUu/K6/uLrhRXhrEax4wH+maDw0N8D+\nCmg8ePi5R5wBRWOPu+giIj0lnEAvB4a3ez2MI7tUPgPMAXDOvWZmmUARUBGJQh7BDG7Z7EMdyM9K\nY8+Bhs6P7Vvctj20xD86U78f1vwZ9m2Fsmd9a3zch2DiZb7P/vVfwRt3Q1NdZOsiIhIh4QT6MmCc\nmY0CtuIven6iwzEfABcAD5jZRCAT6NmrkunZhzYH5WeyY98JBm1GDkyf57fPufnI90ed6wO9penE\nziMi0kO6HeXinGsCvgIsAdbgR7OsMrPvm9nlwWFfBz5nZm8BDwGfcr14L/7AvEx2Vvdwyzkl+O1r\nae7Z84iIHKewxqEHY8oXd9h3W7vt1cCZkS1a+AblZVJRU09Lizu0aHTEpYT8s1roIhKj4vNO0Q4G\nF2TR1OKoqKnvuZOEfH+9Al1EYlVCBPrIfn0A2FR1oOdOcqjLRYEuIrEpIQJ9VJG/QLpplwJdRJJX\nQgT6kIIs0kMpbOjRQG/tQ9dFURGJTQkR6KEU4+Rh+bxStqvnTnKohd7Yc+cQETkBCRHoABdOGsiq\nbfvYXl3bMydQl4uIxLiECfRzx/cH4OV1PdRKV6CLSIxLmECfMCiX/rkZ/K2nul10Y5GIxLiECXQz\nY9boQl7bUNUzC0brxiIRiXEJE+gAZ40torKmnjXba7o/+Fipy0VEYlxCBfrsCb4f/bk1OyP/5Qp0\nEYlxCRXoA3IzmT68gGd7NNDVhy4isSmhAh3gokkDeau8OvLDF1sDvVnj0EUkNiVcoM+ZMgiAZ1ZH\nuJWuLhcRiXEJF+hj+ufQt08aa3dE+MKoAl1EYlzCBTrA4Pwstkd6wQv1oYtIjEvQQM/sgUDXOHQR\niW0JGeiD8jPZEemLomZgIQW6iMSshAz0gXmZ7DnYSH1ThLtHUlIV6CISsxIy0Av6+OXi9tVGOHwV\n6CISwxIy0POzfKBX10Z4zLgCXURiWEIGel4Q6PvqIh3o6kMXkdiVmIGeGWMt9JZmqN0DLS1dH9Pc\nBB1niWxpgZ2rYNUfYe+WYz+viCSV1GgXoCe0drnsi3ag//UnsPQeOFgFrgUy8uGU62DAJMgbAn2L\noew5WLsYNv0NMvMgIw8Kx0DhWL9/7wf+u6bNg4/cHdn6iEhCUaAfi5TUzm8sqnofGmshIxfyhsK+\nctiyDF74IRSfDSNmQWY+bHgRXr/ryM8XjYfTPgP7K/zrijXw/gsw+lw491/gpZ9AfQ9MCSwiCSUh\nAz0vy1cr4l0uoVRobvDbTfWw4x3Y8gY8/W3fAgfoNxp2b/DbfQrh2gd9mAOc8RVorIP9O2H3+/6H\nYNQ50P+kI8/V0tx2M9Oye/35RESOIiEDPSM1xOD8TNbu3B/ZLy46CTa/Crs3wi+mt+0fP9eH9tuL\nYM8mGDQVzvgqjDq7LcxbpWVC35H+Meb8rs/VGuYAoQxoivCdryKScBIy0AFKi/uxbONunHOYWWS+\ndPKVsO5J+MvXD99/7e8gJQWuuufwlnWkpCrQRaR7YY1yMbM5ZrbWzMrM7JYujvm4ma02s1Vm9rvI\nFvPYzRzdjx376ni/MoKt9MJx/vn953zf+McegC+84sO8VaTDHCA1U4EuIt3qtoVuZiHgTuAioBxY\nZmaPO+dWtztmHHArcKZzbo+ZDeipAodr9km+CE+v3snYAbmR+dKCEW3b4z4Ekz8Sme/tTmo6NDX0\nzrlEJG6F00KfAZQ55zY45xqARcAVHY75HHCnc24PgHOuIrLFPHZDC7I4rbgvDy/bQkuL6/4D4cgu\natsed1FkvjMcaqGLSBjCCfShQPu7WsqDfe2NB8ab2Stm9rqZzensi8xsvpktN7PllZWVx1fiY3D9\nzJFsrjrIS+sjdC4zGD/HjwkfMDEy3xmOUEbb6BoRkS6EE+idXVHs2ORNBcYBs4F5wL1mVnDEh5xb\n6Jwrdc6V9u/f/1jLeszmThnMoLxMfvr0OuoaIzTz4ice7v0bfHRRVETCEE6glwPD270eBmzr5Jg/\nOecanXMbgbX4gI+q9NQUbrtsEu9sreZnz66PdnGOX2qGv9v0R8VQuzfapRGRGBVOoC8DxpnZKDNL\nB64FHu9wzGPAeQBmVoTvgtkQyYIer4unDmbulEE8sjyO50JJzfDPtXv83C4iIp3oNtCdc03AV4Al\nwBrgEefcKjP7vpldHhy2BKgys9XAC8A/O+eqeqrQx2r8wFx2H2igOVIXR3tbKKNt+4GL/R2mW99s\n29fSAn//X1i3BBoO+huf6vdDzc7eL6uIRE1YNxY55xYDizvsu63dtgNuCh4xJzfTV/NAQ9OhmRjj\nSmrG4a9/WeKf/+EJqC6Hl37cNt3AYZ/LggkXQ3MjjJgJk6+CvMFt77cGfn2NHxrpnB+aWb8P3vwf\nyOoLw0+HorE9Uy8RiaiEvVO0veyMINDr4zTQm7uYk+Y3l7ZtT7/OTw723mLIyPH7MvNh3dP+B2HN\n4/DcD2D8h2HPRti5GlrCmevG4OvvQe6gE66GiPSspAj0nCDQ99c1QX43B8eiA8Gw/rNv9jM+5g/1\nMzQ++z0/HcGUqyGrwN+lOvdHR37eOdi1Dl7+KWx82be4Z3zOh3RaH0jP9tMCNxzwF19bmv08M3s2\nweNf8XOxK9BFYl5yBHrQ5VJTH6erDe0PukYGToIpH23b/+knw/u8mZ/R8aqFx3bejNzDzy8iMS0p\nAj23fQs9Hp33bTi420830JtyghkcDkT9xl8RCUNSBHprC/1AvLbQB0yAGxd3f1ykZQc3f+1XoIvE\ng4RcU7Sj1j70uO1yiZZQGmT1U5eLSJxIikDPy0ojxeCuF8rYfUBzohyTwjF+SbzmxrabmirW+GGN\nzvm1UB/9HCz+BuzbHt2yiiS5pOhyyctMY8FHT+Zbf3yHHz6xmtuvmd79h8Qbdhosvw+WfBOWLoQb\nn4L7g7nXyp6BsuehoQYwf3PTxEth7EUw4nQ/jj0j19/stHdz8IUGFauCoZgGluJnsUzrAwd3weDp\n/nPpfaJUYZH4lRSBDvDx0uFs3HWAX734PmeNK+KqkmHRLlJ8GDzNTwy2NBghc3+7iTRX/8kvvzf3\nR75b5slvwNsP+wcABoVj/fqprWuuhit3CFz4HZh2bUSqIZIMkibQAW7+0Em8UraLHzyxmslD8jlp\nUIQWvkhkfYvbttOyofGAD/lL7vDj1wdMCI4bCfNf9FMObH/Ld89sXQHb/g4z5sOgk32ru6nBTz2c\nnu0/51pg31Y/iidngP9sw0F47Zew/hkFusgxSKpAD6UYt398Otfc8xrfePRt/vjFM0hJidB6o4mq\nYGTb9g2P+Zb4+Dn+gmlnMnKg+Ez/CFdRu4k5i8/yz+/92U9JICJhS4qLou2NHZDDty+dyFtb9vLs\nGo3e6FbOwLbtYafBxMu6DvNIyshToIsco6Rqobe6YKIPqc1VB6NckjiQkgIzPg9DTvF3nPaWjFw/\n5YCIhC0pAz03I5X0UAq7DtRHuyjx4eIf9/45M3L9rI8iErakDHQzoygnnar9GpMeszJy/VDHO6bC\nF1+BzLxj+7xzUPkevPN7eGuRv+DaWAuZBTCs1A+XrKuG9By/cMjoc2HSlZCW2TP1EekFSRnoAIU5\nGVTtVws9ZqVl+efqD2DH2/5iaWOdnzUyNQPyh7WNlNlfAW/cA/u2+R+C2t1QudZ/rlUoHQZOhprt\n8NqdgPN3wdbu9tMMv/U7eOyLkDMIQqnwxVfbJicTiRNJHOjp7FILPXYd2NW2/cAlfmKy9U+37cvI\n83PAV66BHe/6m5LAL+qR3d+H8cX/CUNKoN8o6NOv7bMtLdB40I/IaTjgb2p6/3nY/Cq8/J/+mD2b\nYNDUHq+mSCQlbaAPzM3k3a3V0S6GdKXjCJfWMC86yXefbHoZ3viV31cwAq54GMZeAJhvYR9NSkrb\nIiCtrfyxF/jHuIvgvg8H89e0C/SWFvjgVT8Gv3avnx8+lAZN9b7135sXjEW6kLSBftKgXB5evoWK\nmjoG5KrfNOZ8+N98t8o5/+xb0Acq/Jj41uDc8a5fjGPYaT5QuwvxcLVOGdxxPdZXfwHPfqftdWqm\nn2O+Yg30KYKpV8PEy/1NU60/FiK9LGkDffIQf5Ht4aVb+McLxnVztPS6vsWHr76UXnz4+4Om9Mx5\nc4KVmfbv8BdW62tg13r42x1tx2Tm+ykPdq2FMRf4H5lXf+lDP60PjJ4Nkz/i/3KwFF+X7P5qxUuP\nS9pAnz6igJIRBfzsufX0y0nnY6cOJz016e6zko7S+/j++X3b/aRkf2m37vkVd0HBcN/tktlhLcO9\nH/i/GtY95ee4Wdth/vpQBky6AobP8EE/Yqbvujm4y4+0KRipETZywsw5F5UTl5aWuuXLl0fl3K12\n7a/nhl8vZfX2fUwbls9/faKE4f00y1/Su2+u7y+3ELhmv2/Oj+D0z4fXym5pgc2v+OGQrtn/OOx4\nB9Y96fd1ynzQX3o7jL0wYlWRxGNmK5xzpZ2+l8yBDuCc4/G3tvHVRSsB+L8vnUHJiL5RLpVE1Ss/\nh2du89uzb4Uz/l9kpvNtboLdG/zsk7vW+RE5BSP8UMrqLX7o5ZBT4FNPnPi5JGEdLdCTtsullZlx\nxfShpIVS+NKDb1K+p1aBnuxmfQWW3eu7USZcErm52UOp0H+8f5w098j3mxvg9buhbt+x30glQhJO\nztWVU0f6EI/bhaQlclJCMP+vcPV9vTsWfdKV0NII7/6h987Zqup9eOHf/YVgiVsK9EDruqP76xuj\nXBKJCX36wZSP9u45h57qR9mUR6Er8pEb4K8/areylMQjBXqgT3oIM7XQJYrMIG+I71Pf+4FfLKR8\nBbz3l54/d3Nw13TDgZ4/l/SYpO9Db2Vm5GSkUlOvQJcoyhviR8T8shSa2801dMlPIW8ojPswbPyr\nv7ha+R4MLYXJV/rVoSpWw+Sr/Jqszvk7YpsbISXV/1g0HPBTGjgHDfv9j0ZKyK8itWudP0+Xo3Ak\nHoQV6GY2B/g5EALudc4t6OK4q4HfA6c556I/hOUY5WakUqMWukRT7iB4r8MoFwvBX75+5LEW8mu9\nPvkNqNvr9z3zXT+eva4aCsf5uW5yB0PReL8cYOtxXTlYFZFqhK2l2U+zkDekd8+boLoNdDMLAXcC\nFwHlwDIze9w5t7rDcbnA/wPe6ImC9oaczFR1uUh05Q5q277pPWiq9bNMrl3shzZWb4WpH/PL9hWN\nh/Kl8PYjwd2rH4aVDwHO35m64x2Y9WV/p+uBXX6umtHn+ZksM/P9kMmWZj/F8N9u9+fs7UB/8ht+\nRNGtWzVlQgSE00KfAZQ55zYAmNki4ApgdYfjfgD8GLg5oiXsReV7alm3cz8X/PRFLpk6mOtnjdQ8\nL9K7TroYnv+hH/GSN7ht/8BJnR8/5nz/aP/6WO1pN6T5ia/BtE9E5q7VhoP+esD+nX4o5oiZkFVw\n+DHL7vXPNdshQ1NwnKhwAn0o0H4tsHLg9PYHmNkpwHDn3BNm1mWgm9l8YD7AiBEjjr20PWzK0HyW\nbtxNVnqIXzxfxj0vbeDpr53DyMLsaBdNksXAyfClNw5vqfe0jA5j3je+BOM/5OeXf/N/fLi7Ft9v\nP+dHflz+ztW+D7/sWcguguwB/v3qch/aVWX+L4P2q06lZcPw0/xoHufaLsQC1Ow4fLFwOS7hBHpn\n9zofGqxqZinAHcCnuvsi59xCYCH4O0XDK2LvWfjJU2lsdvTPzeDhZR/wL4++w7a9dQp06V0DJvTu\n+drPS5OaBb/7GHx6ib9bdkuHHtQ3fwsZ+VAfTD3dfjuUDn0K4UAlDD/dz10zYqZfSAQHZc/5C7ob\nX/KTlqW2+yugZkePVrFbTQ2+3pOuhNIbo1uWExBOoJcDw9u9HgZsa/c6F5gCvGh+notBwONmdnm8\nXRgt6JN+aHvsAL9aTUNzS7SKI9I72t+VWjTW973f92H/+uRr/OIiffrB9rdh9WP+Zqui8X6SsuEz\n/apPLU1+psmsvn5kTWr6keeZcIl/dq5tTpy6algwws9uGU2b/wYbXvSPyVf6esShcAJ9GTDOzEYB\nW4FrgU+0vumcqwaKWl+b2YvAzfEW5h1lBDMvNjQp0CXBtW+hX3I7PHCpH3Vy8U/8RGGt4TvmfDjr\nn478fMfuoc7CvL32E5xl5PnZJvd+cPgx29+CVY/BiFlQOMZPQZwSCrtKYauv8V1Bqx5r27fs137+\nnu7qEYO6DXTnXJOZfQVYgh+2eJ9zbpWZfR9Y7px7vKcLGQ3pCnRJFu370IfPgH+t6L1zm8HAKX74\n5cnX+AW8t62E31ze1pXTWp4An3cAAAp+SURBVMZJl8Pg6f4O3j79/Aid2r1+u7NZMJub/Cih9/7i\nlyRMzfT9+nmDfYinpsNvLvPdLGXP+YvBu9+H53/gH2Mu8COKGg/ArjI4++uQ07/3/tsch7DGoTvn\nFgOLO+y7rYtjZ594saIvPeQDvb6pOcolEelhrS3fvqOic/78oX7YxcOfhLO+Bk/d4i+sXvsQrHzQ\nX2ytr4G//69/vHw7TLvWB/WutZCe638ICkb4PvwtS2Hbm76fvmF/9+dfHbTOJ14KI8+El3/q57Tf\n9Dd4/7m2495+GD763/7u3U0v+9WpTv+C/+uhsdaP5kkJQWbBkaN5eknST5/ble3Vtcz6j+dZcNVU\nrp0ReyNyRCJqx7v+BqTswt4/9+4N8ItT2l7nD/eTo7Uvi3OwbokP6b8u8HfGZvX1M2PW7ICyZ+Dg\nbj+qJnsAFJ/l75DNHQTFZ/ux9011/kfrQAXs2QyPfeHwcnxj4+GLiTc3+Xnt1y3xo3OeutUPrwS/\nIEnrvDd9Cv1fC603bVkK9A8ubNdV+5DPHgCpGT78i8bDqLP9aJ/joOlzj0NrC10XRSUp9NSSfuHo\nNxq++Krvu67bCzM+f+QPixmcNMdvj/+QD+TM/MNbwi3NvkulaNzR+9uLxsLIM9oC/YY/BSN0+h1+\nXCgVRp/rH+BH7qx9Ekad67+jci188LpvrTfW+vdTM/z0Crs3+jKk9Wlb5MS1+GGeKx+Es28+7kA/\nGrXQu1BT18jU7/qV5r983hjGD8xlWN8spg0rIDWkOc1E4t7bj8Drd8Fnno3cIuPhqN3j/+Lo+AMS\nJrXQj0NGatsv/J0vvH9o+/qZI/jhlb04R7aI9IyTP+4fva0Hh0SqqdmFtFDbVfMHP3s6T/3T2WSm\npbB+ZxgXWUREokCB3gVrNwxqwqBcJgzK46JJg9ixry6KpRIR6ZoCPQw5mb5nanB+Jtur64jWdQcR\nkaNRoIehtT99YF4mDU0t7D2oZepEJPYo0I/BkHw/mdDWvbVRLomIyJEU6MdgaN8sQIEuIrFJwxaP\nwbC+fQD439c380rZLs4aW8RJg3I1va6IxAQF+jHo2ycNgJfX7+Ll9bv47WubSQsZS795IX2z429m\nNhFJLAr0o7j949PITGu7wcjMuGTqYLIzQvzj+eP44V9Ws2TVTtbtrOH00VGYA0NEpB0F+lFcVTLs\niH13XldyaPtfL53EklU7Kavcr0AXkajTRdETMCQ/i6y0EBsqD0S7KCIiCvQTkZJiDM7P1N2jIhIT\nFOgnaEBeBhUKdBGJAQr0EzQgN5OKmnoAnHNs21tLk+ZQF5EoUKCfoIF5GezcV8fDyz7gjAXPc8aC\n53ng1U3RLpaIJCEF+gkaUpBFXWML//LoOwwL7iR9v1JT7IpI71Ogn6CrSoYxbkAOl548mAc/O5NJ\ng/PYua8+2sUSkSSkcegnKD8rjae/ds6h+dMH5mVQUaOLpCLS+9RCj4D2i2EMyM1UC11EokKBHmED\n8zPZtb+ehiY/0sU5x04NaxSRXqBAj7AR/frgHGzZc5CKmjpuuG8pV931KgcbmqJdNBFJcOpDj7BR\nRX6K3bteeJ+nV+2gobmFb10y8dCqRyIiPUWBHmHFwdzoj75ZzikjCvjpx6Yxun9OlEslIslAgR5h\n/bLT+eoF48hMC/GZs0aRnqpeLRHpHWEFupnNAX4OhIB7nXMLOrx/E/BZoAmoBD7tnNsc4bLGBTPj\naxeNj3YxRCQJddt8NLMQcCcwF5gEzDOzSR0O+ztQ6pw7GfgD8ONIF1RERI4unP6AGUCZc26Dc64B\nWARc0f4A59wLzrmDwcvXgSNXhhARkR4VTqAPBba0e10e7OvKZ4AnO3vDzOab2XIzW15ZWRl+KUVE\npFvhBLp1ss91eqDZ9UAp8JPO3nfOLXTOlTrnSvv37x9+KUVEpFvhXBQtB4a3ez0M2NbxIDO7EPgW\ncK5zTve+i4j0snBa6MuAcWY2yszSgWuBx9sfYGanAPcAlzvnKiJfTBER6U63ge6cawK+AiwB1gCP\nOOdWmdn3zezy4LCfADnA781spZk93sXXiYhIDwlrHLpzbjGwuMO+29ptXxjhcomIyDEy5zq9vtnz\nJzarBI735qMiYFcEixMtiVAP1SE2qA6xoTfqMNI51+mokqgF+okws+XOudJol+NEJUI9VIfYoDrE\nhmjXQRONiIgkCAW6iEiCiNdAXxjtAkRIItRDdYgNqkNsiGod4rIPXUREjhSvLXQREelAgS4ikiDi\nLtDNbI6ZrTWzMjO7Jdrl6YqZ3WdmFWb2brt9/czsGTNbHzz3Dfabmf0iqNPbZlYSvZK3MbPhZvaC\nma0xs1Vm9tVgf9zUw8wyzWypmb0V1OF7wf5RZvZGUIeHg2ktMLOM4HVZ8H5xNMvfnpmFzOzvZvZE\n8Doe67DJzN4J7ihfHuyLm39PAGZWYGZ/MLP3gv9vzIqVOsRVoIe52EaseACY02HfLcBzzrlxwHPB\na/D1GRc85gO/6qUydqcJ+LpzbiIwE/hy8N87nupRD5zvnJsGTAfmmNlM4EfAHUEd9uCnfSZ43uOc\nGwvcERwXK76Kn36jVTzWAeA859z0duO14+nfE/jV255yzk0ApuH/N4mNOjjn4uYBzAKWtHt9K3Br\ntMt1lPIWA++2e70WGBxsDwbWBtv3APM6Oy6WHsCfgIvitR5AH+BN4HT83XypHf9d4ecsmhVspwbH\nWQyUfRg+KM4HnsBPax1XdQjKswko6rAvbv49AXnAxo7/PWOlDnHVQufYF9uINQOdc9sBgucBwf6Y\nr1fwZ/spwBvEWT2CroqVQAXwDPA+sNf5iefg8HIeqkPwfjVQ2Lsl7tTPgG8ALcHrQuKvDuDXUnja\nzFaY2fxgXzz9exqNXzf5/qD7614zyyZG6hBvgR72YhtxJqbrZWY5wKPAPznn9h3t0E72Rb0ezrlm\n59x0fCt3BjCxs8OC55irg5ldClQ451a0393JoTFbh3bOdM6V4Lsivmxm5xzl2FisRypQAvzKOXcK\ncIC27pXO9God4i3Qw1psI4btNLPBAMFz69zxMVsvM0vDh/mDzrn/C3bHXT0AnHN7gRfx1wMKzKx1\nttH25TxUh+D9fGB375b0CGcCl5vZJvyavufjW+zxVAcAnHPbgucK4I/4H9h4+vdUDpQ7594IXv8B\nH/AxUYd4C/RuF9uIcY8D/xBs/wO+T7p1/w3BFfGZQHXrn2/RZGYG/BpY45y7vd1bcVMPM+tvZgXB\ndhZwIf4i1gvA1cFhHevQWrergedd0PkZLc65W51zw5xzxfh/8887564jjuoAYGbZZpbbug18CHiX\nOPr35JzbAWwxs5OCXRcAq4mVOkTzAsNxXpS4GFiH7wf9VrTLc5RyPgRsBxrxv9KfwfdjPgesD577\nBccafvTO+8A7QGm0yx+U6yz8n4dvAyuDx8XxVA/gZODvQR3eBW4L9o8GlgJlwO+BjGB/ZvC6LHh/\ndLTr0KE+s4En4rEOQXnfCh6rWv//G0//noJyTQeWB/+mHgP6xkoddOu/iEiCiLcuFxER6YICXUQk\nQSjQRUQShAJdRCRBKNBFRBKEAl1EJEEo0EVEEsT/B/w6M/I29vKiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAEICAYAAABRSj9aAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deZxcVZ338c+vq9eku9NJZ99ISMIS\nIIlNE0AWWR0QJcBESYQRFUVHmZcOOiOiwyDj8wDOCDiPjMqwCkhQFImIgLI4OiIkgUA2Ah3I0klI\nOp10J+m9qs7zx7mVrnQ66UpS3VV16/t+vepVdW/dunVOaL59+txzzzHnHCIiEl4FmS6AiIj0LwW9\niEjIKehFREJOQS8iEnIKehGRkFPQi4iEnIJeRCTkFPQSGmb2kpntMLOSTJdFJJso6CUUzGwScAbg\ngIsH8HsLB+q7RA6Vgl7C4lPAX4EHgKsSO82szMy+b2brzKzZzP5sZmXBe6eb2V/MrMnMNpjZp4P9\nL5nZ55LO8Wkz+3PStjOzL5vZO8A7wb4fBOfYaWZLzOyMpOMjZnaDma0xs13B+xPM7C4z+35yJczs\nN2b21f74B5L8paCXsPgU8Ejw+BszGxXs/w/gROCDwDDgn4G4mU0Efgf8P2AEMAtYehDfdwlwMjA9\n2F4UnGMY8DPgF2ZWGrx3HTAf+AhQCXwWaAUeBOabWQGAmQ0HzgUePZiKi/RFQS85z8xOB44Afu6c\nWwKsAT4ZBOhnga845zY652LOub845zqAK4A/OOcedc51OecanXMHE/S3OOe2O+faAJxzDwfniDrn\nvg+UAEcHx34O+LZzbrXz3giOfRVoxoc7wDzgJefclsP8JxHZi4JewuAq4Dnn3LZg+2fBvuFAKT74\ne5qwn/2p2pC8YWZfM7NVQfdQEzAk+P6+vutB4Mrg9ZXAQ4dRJpFe6UKS5LSgv/0TQMTM3g92lwBV\nwBigHZgCvNHjoxuA2fs5bQswKGl7dC/H7Jn2NeiP/wa+Zb7CORc3sx2AJX3XFGB5L+d5GFhuZjOB\nY4Ff76dMIodMLXrJdZcAMXxf+azgcSzwJ3y//X3A7WY2Nrgoemow/PIR4Dwz+4SZFZpZtZnNCs65\nFLjMzAaZ2VTg6j7KUAFEgQag0MxuxPfFJ9wD/JuZTTNvhplVAzjn6vH9+w8Bv0x0BYmkk4Ject1V\nwP3OufXOufcTD+CH+H7464Fl+DDdDtwGFDjn1uMvjn4t2L8UmBmc8w6gE9iC71p5pI8yPIu/sPs2\nsA7/V0Ry187twM+B54CdwL1AWdL7DwInoG4b6SemhUdEMsvMzsR34UxyzsUzXR4JH7XoRTLIzIqA\nrwD3KOSlvyjoRTLEzI4FmvAXje/McHEkxNR1IyIScmrRi4iEXNaNox8+fLibNGlSposhIpJTlixZ\nss05N6K397Iu6CdNmsTixYszXQwRkZxiZuv29566bkREQk5BLyIScgp6EZGQy7o++t50dXVRX19P\ne3t7posyYEpLSxk/fjxFRUWZLoqI5LicCPr6+noqKiqYNGkSZtb3B3Kcc47Gxkbq6+uZPHlyposj\nIjkuJ7pu2tvbqa6uzouQBzAzqqur8+ovGBHpPzkR9EDehHxCvtVXRPpPzgR9GOxq76K9K5bpYohI\nnlHQp6CxsZFZs2Yxa9YsRo8ezbhx4/Zsd3Z2pnSOz3zmM7z4ylLe3rKrn0srIrK3nLgYm2nV1dUs\nXerXjb7pppsoLy/n61//+l7HOOdwzlFQ0Pvvzvvvv58365sAiMcdBQXqmhGRgaEW/WGoq6vj+OOP\n54tf/CI1NTVs3ryZa665htraWo477jhuvvnmPceefvrpvLViGdFolGHDhnL99dczc+ZMTj31VLZu\n3ZrBWohI2OVci/47v1nByk0703rO6WMr+dePHXdIn125ciX3338/P/7xjwG49dZbGTZsGNFolLPP\nPpu5c+cyffp0kmeDbm5u5swzz+TWW2/luuuu47777uP6669PR1VERPahFv1hmjJlCieddNKe7Ucf\nfZSamhpqampYtWoVK1euBMDhk35wSSGlpWWcetZ5AJx44omsXbt2wMstIvkj51r0h9ry7i+DBw/e\n8/qdd97hBz/4Aa+++ipVVVVceeWV+4yFH1FeQlFxMZub2hlcXEgkEiEajQ50sUUkj6hFn0Y7d+6k\noqKCyspKNm/ezLPPPrvnvUTXTUGBUWAQd466rbuIxrRMqIj0r5xr0Wezmpoapk+fzvHHH8+RRx7J\naaedts8xkeBGqKkjy9nR2klEo29EpJ9l3ZqxtbW1rufCI6tWreLYY4/NUInSY3tLB/U72jhmdAXF\nhZGUPhOGeovIwDCzJc652t7eU9fNAEn00Gj8vIgMNAX9AIkFfzlFNIeNiAwwBf0AiccdETNNViYi\nA05BP0BimvZARDJEQT9AYnGnETYikhEaXtmP4nFHU1sXg4ojxJxT/7yIZIRa9Ck466yz9rr5CeDO\nO+/kS1/60n4/U15ezvJNzdTvaOWdrbtp6YiqRS8iGaGgT8H8+fNZsGDBXvsWLFjA/Pnz+/xsZWkR\nhUHAK+hFJBMU9CmYO3cuTz31FB0dHQCsXbuWTZs2MWvWLM4991xqamo44YQTePLJJ/f57NiqUqaO\nKGfskDJGVJQMdNFFRHKwj/5318P7y9J7ztEnwIW37vft6upqZs+ezTPPPMOcOXNYsGABl19+OWVl\nZTzxxBNUVlaybds2TjnlFC6++OK9hlCaGUWRAoYr5EUkQ9SiT1Fy902i28Y5xw033MCMGTM477zz\n2LhxI1u2bAEgMbGErr+KSKblXov+AC3v/nTJJZdw3XXX8dprr9HW1kZNTQ0PPPAADQ0NLFmyhKKi\nIiZNmrTPtMQFKOlFJLPUok9ReXk5Z511Fp/97Gf3XIRtbm5m5MiRFBUV8eKLL7Ju3bp9PqcWvYhk\nmoL+IMyfP5833niDefPmAXDFFVewePFiamtreeSRRzjmmGP2Or5AUx6ISBbIva6bDLr00ktJntZ5\n+PDhvPzyy70e+3Z9A02tnQNVNBGR/VKLvp+4uKNArXkRyQIK+n4SR/3zIpIdcqbrxjmX1f3dm5ra\naO+KUTWomK5YnGgsflgt+mxb+UtEcldOBH1paSmNjY1UV1dnZdjH445tu/1ds7s7onv2F0UO7Q8m\n5xyNjY2UlpampXwikt9yIujHjx9PfX09DQ0NmS5KrzqicRp2dVBSWEDVoCI6onGaWrv8m01lh3TO\n0tJSxo8fn8ZSiki+SinozewC4AdABLjHOXdrj/dLgJ8CJwKNwOXOubVmVgTcA9QE3/VT59wtB1vI\noqIiJk+efLAfGzAvrt7K5xcu4ldf+iAzJg6lMxrnqG//DoC1t16U4dKJyIBpaYSyoVCQXZc/+wx6\nM4sAdwHnA/XAIjNb6JxbmXTY1cAO59xUM5sH3AZcDnwcKHHOnWBmg4CVZvaoc25tuiuSSa0dMQAG\nF/t/zuJC/x958vDBGSuTiPSjja/BE1+E9iaongolFdCwGna8B+WjYMTRMPFUaGuCrSthXA0MHgFb\nVkKsA7asgMJSOPajsOt9iJRASTmMnuH3pVkqLfrZQJ1z7l0AM1sAzAGSg34OcFPw+nHgh+Y70x0w\n2MwKgTKgE9iZnqJnj9ZO3y8/qDiyZ9+bN32Yoiz7rS4iabDxNXj4Migsg6nnwzvPQrQDjjgNpp4L\nu7fC9vfgj7cBBlUTYe2fuj9fWAaTz4Ada+GF70JJJbg4dO6GEz6esaAfB2xI2q4HTt7fMc65qJk1\nA9X40J8DbAYGAf/onNve8wvM7BrgGoCJEyceZBUyr7XTt+iTg76ytChTxRGReAxiXRAp9iEaOUDU\n7d4KTet9l8vQyb13u7y/HF66xbfCVzwB5SPh009B9RT/XS4OkR7/zzeugdIqGFwNW1dB63bf+i8f\n6cdeOwetjTCo2m/H4xDvSu+/QyCVoO9tmEvPsX/7O2Y2EAPGAkOBP5nZHxJ/Hew50Lm7gbsBamtr\nc25cYUvQoh9ckhPXtkXCobPVd4GMr+2+aeX95fDEF2DLcigZ4rtUWhuhagKUDfOh2trogxeClvSu\n7nMWl8O4E2Hez3xXSsIrP4Z3nvPvA1z0fR/yAAUR/OXLHhLvA4w8dt/3zWDw8O7tggIo6J/pzFNJ\npnpgQtL2eGDTfo6pD7pphgDbgU8CzzjnuoCtZva/QC3wLiHS1hmjwKCkUF01IgOiYzf87huw9GE4\n7jJob4aNi/1zaRXMuhIa3oLSIb7rpG2HD/imddDVBsfNgaLBgIOK0TBsiv/s28/AW0/B9jUwZqb/\nrpZGv++Yj8KlP4GWrTAkt0bEpRL0i4BpZjYZ2AjMwwd4soXAVcDLwFzgBeecM7P1wDlm9jC+6+YU\n4M50FT5btHTEGFRcmJVj/EVCYf1f4blv+1Z8WZXvJ4+2+fdW/Mo/Dx7pFxGa/xgMGXdo31M51of6\nur/AE38Pn3wMnvuW/8VyxtegsDjnQh5SCPqgz/1a4Fn83yf3OedWmNnNwGLn3ELgXuAhM6vDt+Tn\nBR+/C7gfWI7v3rnfOfdmP9Qjo9q6onv1z4tImr3wXahf1L19xGnBhcuP+Vb6iGOgaNDhzztSNMg/\nr3gCtq6Au06GrhY490YYffzhnTuDUupUds49DTzdY9+NSa/b8UMpe35ud2/7w6alI6b+eZH+svbP\nftTKSZ/zfe5jPwBHX9R9gTW5n/twFQU3OHa1Bs8tUDEWPviV9H1HBiidDtM9f3qXhW9s4sgRGjMv\n0i82vuafz/kX323TnxIt+qZgoGH5KN+aP9ConRyQ26XPEOccC9/YRNw5bnvmLQAuOG50hkslMkAe\nutTf/HPJj+HdF6GwxI9UKTq06T724Rw01/v+8oII7NwIxRX9H/LQXYf2JqieBv+wuP+/cwAo6A/B\nqs27+MqCpQAcPaqCh66ezchKTUAWek0b4LfXwWV3+zHX+ah5I6x5wb9+87Hu/WXD4LhLwAr8qJbZ\nn4fRM6Fprb879K2n/N2fI4/1o1yaN0DHLmis8+83rfdDJUuH+DtFu1qg5ip/AfSVH8OQAbq/JtGi\nBz8UMyQU9IegsaVjz+uF/3AaJYW6EJsXnvu2H0v9zu9hxicyXZqBtWsLdOyEuue7942f7Vv2x1wE\nby6A5b+CeNSH/dJHUjyxQWklDJ3k+94LIjDlbFh0D7z2oA9/gOb16a5R75L/Kkln33+GKegPQeNu\nv0TgH647UyGfT5qCsElXF0WueOo6WHxv9/awKXDt4r3vIP3AFd2v23f647e/C2Nm+ZbxsMl+3PrO\njb7FPni431d1BGD73o0aj8KSB/zYePA3KA2EwqS/zCvC0x2roD8EjS0+6IeX989dbJKldr3vn9tD\nN13TgSWHPMAHrjzw7IyllXD6P/b+3vCpqX1nQVI0TTzVj7gZCMn1Gpq9M+YeLAX9IWjc3UGkwDSf\nTb5JjLxob85sOQZSV/u++078dP9/b/U0/zznLjj6I/3/fb0ZOikz39sPFPSHYHtLJ8MGF1NQoDth\n80riz/p8Cvr/O9Y//80tvqvlqAsGZjHk2Z/3I3kmnNT/37U/w8LTotfkLIegsaWT6sHFmS6GDLR4\nsExktgV9ZytsWORnR0zYtcXf3Vm/2M+ueEjnbQEXfLZqIhx94cCteF8QyWzIA4ycntnvTyO16A9B\n4+4OqssV9HknFkwhmy1B37gGnr8ZVv7abxcN9rfpt2zz0wIkfjGVVsGQCdDRDMfP9ePehx/lR7S0\nbvNDJhvr/AXSynH+7tPCUnh/Wfd3lVQMfP0y5fMv+mGeIZq7SkF/CLa3dHLC0AG4eUOyS8xfhGf3\nlsyWY+MS+PWX/XDHnRv9vo/9J6x/2U+/u3sLTP4QnH2DX9xizQt+f3EF/PkO9pplvKTS34hUNMiP\nc69f7H9BdLX5aXZrrvJdKJPPzERNM2NcTaZLkHYK+kPQuFtdN3kpGtw/8d7/+MUqykdmphzLHoeG\nYD71I06Hy37iZ1Q88aruckaKfYt0fC2cMLf7s207/AXWLcv9RGBVE/Y9v4SO+uj70BWLc/lPXuYv\nddsA6IjG2NURVdDno1gXHHm2XwVo9e8G/vvjcd/dsvppGHU8nPlPcP7N+06bW1iy/26HsqFQOQam\nna+QzyNq0fdh664OXnlvO595YBGv3HAuv1hcD0C1xtDnn1gnjJ3l+8bffsa3oJ2Dv/6XH1t/3KW+\n26N6KkTb0zc3SzwOm5f6+V9+/nd+3xWP+7AWSYGCvg8dXX7UQUc0zkd+8Cc2NbczdkgpZx4Vntuj\nJQWJ9TwjJXD0BfDaQ35ky/Pf6b7d/4+3Jn3A4IJbfPiXDfMLVhyqFb+CX17dvX3y3yvk5aAo6PvQ\n1tU9NK2osIDbPzGTi2eOpTCiXq+8kli0ubAYpl8Cr94N3z/K75sxD876BrzxmB+pUlblw/+Z6/2j\nZAhc9B9+xEtXK7Q0+OdIMQyf1vd3b1259/aFt/Z+nMh+KOj70B4E/U8/O5szjxqR4dJIxiRG3ESK\nYdJpPtzfXOD3XfYT/3z2N7uP/8i/w59u98MW1/8VfvV5+POd0LYddm3uPu7Yi4PFrSMwYbbvCtr2\ntj8mHvOjZja8AgVF3b9sRA6Sgr4PrZ0+6LVUYJ6LJgU9+HA/+kKoGNP78cWD4dx/8a/jcf9LYcmD\nUDwIZs6Dkcf5xasX/TesWrifLzU/rr2gAM75lp8crHxUWqsl+UFB34e2IOhLixT0eS3WI+jBz7+e\nioICmPVJ/+jpzK/7+dg3vOLHv5cO8dP1Vk30rfviQft+RuQgKej7kOijL1OLPr/1FvTpUFTmH6n+\n0hA5BLqi2IdEH726bvJcfwW9yABQ0Pch0Udfpq6b/JYI+sMZJimSIQr6PiS6btRHn0WWPQ4rnxzY\n71SLXnKY+uj70N4ZwwxKCvU7cS/PfstPCfCR7w38dyduHrqpGWLR7gVBwE/Vu+R+GFsDkSL/C+HY\njx3+pFx7Rt1osRnJPQr6A2jrjPGzVzdQVhTBQjRlaVq8/EP/nImgT/j3aX7K4PNvhslnwB9vg3Uv\nQ8vWvY979W447yaYdYWfrRH85ypG7f2Lwjk/A2S0w0/LO2iYf93V5qcgALXoJScp6Pfj3Ybd/ONj\nS9m2u4OPztjPWGkZeC5pit2qibDpNXjmG367pNLffDT7C1D3Bz/fzBnXwe++AX+4yT8KinxYd7X4\noYztzX4BazPYsc7f0HQgg6r7q2Yi/UZB34vfvrmZrz72Ol0xxzcuOIa/P2tKposkCYlFPz78f+CD\n1/q7R1/7qZ9N8qO3d8/kePQF3Z+Z9zN44buw6jf+Bqdhk/1c6831/samdX+B4nI45jh/I9Oo6dDa\n6LumIkV+rvbOFhh2JIw6buDrLHKYFPQ9xOOOm59awdGjK/jRFScyYZhuWDmgaIefFnegtDb658HB\ndBQFEaj9jH/sT6QIzv+Of4jkIV1h7GFnexdbdnZwyaxxCvlUtPbR1ZFuLQ3+ebC6UERSpaDvYUer\nnzhKa8KmqK8+7XRr8QvAMEjTRIukSkHfw/YWP4yuapCCPiWJFvZAaQ2CfrCCXiRVCvoemlp90A9T\n0KemuX5gv08tepGDllLQm9kFZrbazOrM7Ppe3i8xs8eC918xs0lJ780ws5fNbIWZLTOz0vQVP/0S\nXTdDFfR9CO4ryETQF5dDUVb/GIlklT5H3ZhZBLgLOB+oBxaZ2ULnXPKyN1cDO5xzU81sHnAbcLmZ\nFQIPA3/nnHvDzKqBrF49YUei62aw7oDcL+eAYDz7S7f42RdP+0p6vyMeg62rYPsaf1fqhNkw9Ajf\ndaNuG5GDksrwytlAnXPuXQAzWwDMAZKDfg5wU/D6ceCH5m8l/TDwpnPuDQDnXGOayt1vtuxsp6Sw\ngIoSjTzdr3h07+3f3wgnfQ6evQE6dvk7VRPj2cGPQX/rt771X1oJrTtgytl+HvZIkb+g27QhOHY3\nrH4amtZ3j5lPGDIRmtfDuNr+rZ9IyKSSZuOADUnb9cDJ+zvGORc1s2agGjgKcGb2LDACWOCc2+ee\neTO7BrgGYOLEiQdbh7Sqa9jNlBHlmvLgQKIdwQtjT8v+e0f6O1EB1rwAZ38LRk730w+s/PW+53jx\nu/s//9gav8Te5DP9jU0WgXdfgi3LIToLps9JY2VEwi+VoO8t8VyKxxQCpwMnAa3A82a2xDn3/F4H\nOnc3cDdAbW1tz3MPmJ3tXby0uoE5s8Zmqgi5ITGT43n/6qcdePclvxzexFPhuEth0T3w9NeDgw1O\n+jxMv9ivnLRrC6z9Hyit8tMOxLqgeqqfW6ZitP+LoGKMfy/Z2FkDWUORUEkl6OuBCUnb44FN+zmm\nPuiXHwJsD/b/0Tm3DcDMngZqgOfJMjvbu7j6gUUAzJ48LMOlyXKx4DJLSSWcdDUceZbfvuAW32Uz\n+xrfqo+2w4hjfKs8oaQChk/d/7lLKvqr1CJ5K5VRN4uAaWY22cyKgXlAz9WMFwJXBa/nAi845xzw\nLDDDzAYFvwA+xN59+1khFnd89v5FvLa+ie/NncEVJx+R6SJlt1jQdZOYybF6Clz+UHe/vBlMPReO\nuWjvkBeRjOizRR/0uV+LD+0IcJ9zboWZ3Qwsds4tBO4FHjKzOnxLfl7w2R1mdjv+l4UDnnbO/baf\n6nLIHl+ygcXrdvDvc2fw8doJfX8g3yVa9AM5x42IHLKUhpY4554Gnu6x78ak1+3Ax/fz2YfxQyyz\n0u6OKN/97SpmTxrG39aM7/sD0n0xVotwiOSEvB9D+Js3NrGrPcoNFx1LQUEIR9q0bodlv/DdKiOO\n8eHc3gzr/wqjZ8DwaX6BjWTO+VZ7PArFvUzspmX1RHJK3gf9/9ZtY8yQUmaOH5LpoqRX80bY8Aq8\n8Si889yBj40UQ9lQP+86+HHt7c1+WOPYD8C4Gv+6c7dfrCNxN2xEXTciuSDvg/61dTuonTQsN8fN\nx6Lw5mNw/GX+7tSE1u1w91ndS+qdei0ccRp07PTvlVXB+Nnw3kuwczPEu/z+zt2+W6ZiDJSPgmib\nb/m//ohfkamk0rf0o23+vJoqWCQn5HXQx+OO93e2c0Suzju/4a/w5Jf8DUmf/Lkfu77mBdj+LmBw\n2X/DyGNh9Am9f/5AwxyTxWP+uSDiu3XadvjHsCPTUg0R6V95HfS7O6PEHQwpy9GLiok7Ud95Dr5T\n1b1/0hl+aOOMT6Tnewoi3a/NfJ9+z359EclaeR30zcFMlUMG5WjQd7XvvT1yOnzuD34dVBGRQF7P\nR9/c5oO+Ktdb9HPvg+pp8ImfKuRFZB953aJvClr0ObuaVGI8+7ha+IfFmS2LiGQttejJ5T76YPRL\noRbhEJH9y+ugr9/RCkBVrvbRJ1r0mopARA4gb4P+/eZ27nqxjpqJVYysyNGg7Apa9Mlj6EVEesjb\noP+3p1bSEY1z+ydm5ebNUhC06E1TEYjIAeVl0Ndt3cVvl23mC2ceyaThOTxKJdru++dz9ReViAyI\nvAz6B/+yjuLCAj71wUnpOeHmN6F+SXrOdTCi7eqfF5E+5d3wyrbOGE+8vpGPzhjD8PI0heRPzvDP\nNzUf+Lh0S7ToRUQOIO9a9E8u3cjujihzT+yHuefj8fSf80C62qFIQS8iB5ZXQf/2ll3868IVzJxQ\nxSmT+2Hmxae+Ci/e4meVHAhq0YtICvKq62bphiY6onH+9WPT+2eRkdce9M91f4Arf+mnAwY/46OZ\nf+7c7VvisQ4/RXDrNtj2Nqx+Btqb/PHDjoRhk2H40X7oZKzLTzlcUAhDJ/mAb94IqxbCmJnpr4eI\nhEpeBX1H1HetjB+a5nHno0+A95f513Pugt98Ff7rVJh6Dmx41S/UUTbUr9i0e8t+zjEDhkzwc8a/\nv8zPSJlYyelARh2fvnqISCjlV9B3+XnVS4sifRx5kKwAhh8F8xdA9RTfIn/yy/D6w1BYBmNmwLAp\nPugrRvuRMhWjoXw0VI71ryvH7n3O9p2wa7N/HSmCQcOhq9XvKyyDwSM0VbCIpCS/gj5o0ZcUpvnS\nRDzuZ4+snuK3j/ggXLvEr9x0qMMfSyv9o+e+itGHV1YRyTt5GfTFkXQHfXTvxTkACgqgQGPcRSTz\n8mrUTUc0RklhQfqnPOgt6EVEskR+BX1XPP3dNgAu5kfEiIhkofwK+micknRfiIWgRa+gF5HslF9B\n3xXrnxZ9PAamrhsRyU75FfTRfuq6icfURy8iWSvPgj5GSaG6bkQkv+RZ0McpLeqPFr1G3YhI9sqv\noO+K90+L3sXVoheRrJVfQR+NUaIWvYjkmTwL+v66GBvVqBsRyVp5E/TxuKNhVwcVpUX9cHLdMCUi\n2SuloDezC8xstZnVmdn1vbxfYmaPBe+/YmaTerw/0cx2m9nX01Psg7d8UzONLZ2cNjXNC444pztj\nRSSr9Rn0ZhYB7gIuBKYD881seo/DrgZ2OOemAncAt/V4/w7gd4df3EP3wltbMYMPHTUyvSeO+6mP\n1UcvItkqlRb9bKDOOfeuc64TWADM6XHMHCBYXonHgXMtmDnMzC4B3gVWpKfIh+al1Q3MmlDFsMHF\n6T2xU9CLSHZLJejHARuStuuDfb0e45yLAs1AtZkNBr4BfOdAX2Bm15jZYjNb3NDQkGrZU9baGWXZ\nxmY+OKUf1omNB+vDqutGRLJUKkHf25y+LsVjvgPc4ZzbfaAvcM7d7Zyrdc7VjhgxIoUiHZw3NjQT\niztqj+iHFZkSXTcadSMiWSqVZmg9MCFpezywaT/H1JtZITAE2A6cDMw1s+8BVUDczNqdcz887JIf\nhDfr/aLbMydUpf/katGLSJZLJZ0WAdPMbDKwEZgHfLLHMQuBq4CXgbnAC845B5yROMDMbgJ2D3TI\nAyzb2My4qrL098+DLsaKSNbrM+idc1EzuxZ4FogA9znnVpjZzcBi59xC4F7gITOrw7fk5/VnoQ/W\nyk07OX5cZd8HHgpdjBWRLJdSf4Nz7mng6R77bkx63Q58vI9z3HQI5TtsHdEY67a3ctGMMf3zBeq6\nEZEsF/o7Y9duayUWd0wdWd4/X6CgF5EsF/qgf3vLLoB+DHqNuhGR7BaeoN9WB7/8HGx/b6/dyzc1\nUxQxpo2s6J/v1cVYEcly4ddgDTcAAAu4SURBVAn6gggs+wWs3utSAm9uaObo0RUU98eslaCuGxHJ\neuEJ+mGTYdTxsOqpPbu+9MgSXn63kXOOTvP8Nsk06kZEslx4gh7gmItg/V/gJx+CtX9m8/o1VNPM\n1bNHQVe7n2kyobMVGtdA63bf/dLVBu3N0NIIOzdD03robIFYV3f3TE/xOEQ7/Gu16EUkS4UrnU74\nOCy6FxpWwwMX8QRAKfCfRRDvgtIqqJoAzfXQtuPgzl1YCpFi31UT6wq6bJJ+cUT64WYsEZE0CFfQ\nD58G/7wGdm6Cp/+JRXWbqK+s4dJpRVBaCVtWQMcumHAyVI6FijHQ0uBb+5EiH9aRYv+6IAIt2yDa\nDlbQ3bqPFPrWe0FR8ByBkko44rRM115EpFfhCvqEyrEw7xGu//5LHFNdyaUX1mS6RCIiGROuPvoe\n4g4iBb1NrCkikj9CHfTReJxCBb2I5LlQB30s5ihQ0ItIngt10EfjTi16Ecl7oQ76uHPqoxeRvBfq\noFeLXkQk5EGvPnoRkbAHvVOLXkQk1EEfjTsiBaGuoohIn0KdgrG4IxLqGoqI9C20MeicC4I+tFUU\nEUlJaFMwHkwsqT56Ecl3oQ36aDwOaK4bEZHQBn0saNIr6EUk34U+6NV1IyL5LvRBrxa9iOS70AZ9\nVEEvIgKEOOjjCnoRESDEQR9VH72ICBDioE/00ReYgl5E8ltog35Piz6ioBeR/BbaoO8edRPaKoqI\npCS0Kbgn6NV1IyJ5LqWgN7MLzGy1mdWZ2fW9vF9iZo8F779iZpOC/eeb2RIzWxY8n5Pe4u+fpkAQ\nEfH6DHoziwB3ARcC04H5Zja9x2FXAzucc1OBO4Dbgv3bgI85504ArgIeSlfB+xLkvEbdiEjeS6VF\nPxuoc86965zrBBYAc3ocMwd4MHj9OHCumZlz7nXn3KZg/wqg1MxK0lHwvuxp0etirIjkuVSCfhyw\nIWm7PtjX6zHOuSjQDFT3OOZvgdedcx09v8DMrjGzxWa2uKGhIdWyH5D66EVEvFSCvrekdAdzjJkd\nh+/O+UJvX+Ccu9s5V+ucqx0xYkQKReqbJjUTEfFSCfp6YELS9nhg0/6OMbNCYAiwPdgeDzwBfMo5\nt+ZwC5wqTWomIuKlEvSLgGlmNtnMioF5wMIexyzEX2wFmAu84JxzZlYF/Bb4pnPuf9NV6FR0KehF\nRIAUgj7oc78WeBZYBfzcObfCzG42s4uDw+4Fqs2sDrgOSAzBvBaYCvyLmS0NHiPTXotebN3ZDsDw\n8gG59isikrUKUznIOfc08HSPfTcmvW4HPt7L574LfPcwy3hINja1YQZjqkoz8fUiIlkjtHfG1u9o\nY1RFKSWFkUwXRUQko0Ic9K2MG1qW6WKIiGRcaIN+Y1Mb46oU9CIioQz6WNyxualdLXoREUIa9Ft3\ntRONO7XoRUQIadBv3NEGoBa9iAghDfo1DbsBOHL44AyXREQk80IZ9O9s2U1JYQHjhw7KdFFERDIu\nlEH/1vu7mDqyXNMfiIgQwqBv6Yjy6trtnDy55yzJIiL5KXRBv2jtdjqjcc49dkCm1BERyXqhC/qV\nm3cCcML4IRkuiYhIdghf0G/ayYRhZVSWFmW6KCIiWSF0Qb9hRxuTqjWsUkQkIXRBv21XByMrNDWx\niEhCqILeOUfDrg5GVGixERGRhFAFfXNbF52xuIJeRCRJqIK+YVcHgIJeRCRJqIK+qa0LgKGDNOJG\nRCQhVEHf3hUDoLRIyweKiCSEKug7uuIAlBSGqloiIoclVInYEU0EvVr0IiIJIQt633WjFr2ISLdQ\nJWKiRa8+ehGRbqEK+sTFWLXoRUS6hSoR9/TRF4WqWiIihyVUidg96kZdNyIiCeEK+miMoohpCUER\nkSQhC/q4WvMiIj2EKujbu2K6ECsi0kOoUtG36ENVJRGRwxaqVOyIxjWGXkSkh3AFfVeMYrXoRUT2\nklIqmtkFZrbazOrM7Ppe3i8xs8eC918xs0lJ730z2L/azP4mfUXf1872LgYVq0UvIpKsz6A3swhw\nF3AhMB2Yb2bTexx2NbDDOTcVuAO4LfjsdGAecBxwAfBfwfnSzjnHyk07OWZMZX+cXkQkZ6XSop8N\n1Dnn3nXOdQILgDk9jpkDPBi8fhw418ws2L/AOdfhnHsPqAvOl3brt7eysz3KjHFD+uP0IiI5K5Wg\nHwdsSNquD/b1eoxzLgo0A9UpfhYzu8bMFpvZ4oaGhtRLn6Qr5rjw+NHUHDH0kD4vIhJWqQR9b7eZ\nuhSPSeWzOOfuds7VOudqR4wYkUKR9jV1ZDk/uvJEjhpVcUifFxEJq1SCvh6YkLQ9Hti0v2PMrBAY\nAmxP8bMiItKPUgn6RcA0M5tsZsX4i6sLexyzELgqeD0XeME554L984JROZOBacCr6Sm6iIikorCv\nA5xzUTO7FngWiAD3OedWmNnNwGLn3ELgXuAhM6vDt+TnBZ9dYWY/B1YCUeDLzrlYP9VFRER6Yb7h\nnT1qa2vd4sWLM10MEZGcYmZLnHO1vb2n20hFREJOQS8iEnIKehGRkFPQi4iEXNZdjDWzBmDdYZxi\nOLAtTcXJFNUhO6gO2SEMdYD+r8cRzrle7zjNuqA/XGa2eH9XnnOF6pAdVIfsEIY6QGbroa4bEZGQ\nU9CLiIRcGIP+7kwXIA1Uh+ygOmSHMNQBMliP0PXRi4jI3sLYohcRkSQKehGRkAtN0Pe1gHm2MLP7\nzGyrmS1P2jfMzH5vZu8Ez0OD/WZm/xnU6U0zq8lcybuZ2QQze9HMVpnZCjP7SrA/Z+phZqVm9qqZ\nvRHU4TvB/snBAvfvBAveFwf7S4LtuuD9SZksfzIzi5jZ62b2VLCdi3VYa2bLzGypmS0O9uXMzxOA\nmVWZ2eNm9lbw/8ap2VKHUAS9pbaAebZ4AL9QerLrgeedc9OA54Nt8PWZFjyuAX40QGXsSxT4mnPu\nWOAU4MvBv3cu1aMDOMc5NxOYBVxgZqfgF7a/I6jDDvzC9wTPO5xzU4E7guOyxVeAVUnbuVgHgLOd\nc7OSxprn0s8TwA+AZ5xzxwAz8f9NsqMOzrmcfwCnAs8mbX8T+Gamy3WA8k4ClidtrwbGBK/HAKuD\n1z8B5vd2XDY9gCeB83O1HsAg4DXgZPydi4U9f67w6zGcGrwuDI6zLCj7eHyAnAM8hV++M6fqEJRn\nLTC8x76c+XkCKoH3ev57ZksdQtGiJ8VFyLPYKOfcZoDgeWSwP+vrFfz5/wHgFXKsHkGXx1JgK/B7\nYA3Q5PwC97B3OffUIXi/Gage2BL36k7gn4F4sF1N7tUB/FrSz5nZEjO7JtiXSz9PRwINwP1BN9o9\nZjaYLKlDWII+pUXIc1BW18vMyoFfAl91zu080KG97Mt4PZxzMefcLHyreDZwbG+HBc9ZVwcz+yiw\n1Tm3JHl3L4dmbR2SnOacq8F3aXzZzM48wLHZWI9CoAb4kXPuA0AL3d00vRnQOoQl6HN9EfItZjYG\nIHjeGuzP2nqZWRE+5B9xzv0q2J1z9QBwzjUBL+GvN1SZX+Ae9i7nnjoE7w/BL5uZSacBF5vZWmAB\nvvvmTnKrDgA45zYFz1uBJ/C/eHPp56keqHfOvRJsP44P/qyoQ1iCPpUFzLNZ8uLqV+H7vBP7PxVc\noT8FaE78GZhJZmb4dYJXOeduT3orZ+phZiPMrCp4XQach7949iJ+gXvYtw6Jus0FXnBB52qmOOe+\n6Zwb75ybhP+Zf8E5dwU5VAcAMxtsZhWJ18CHgeXk0M+Tc+59YIOZHR3sOhe/VnZ21CGTFzDSfDHk\nI8Db+H7Wb2W6PAco56PAZqAL/1v9anw/6fPAO8HzsOBYw48mWgMsA2ozXf6gXKfj/8x8E1gaPD6S\nS/UAZgCvB3VYDtwY7D8SeBWoA34BlAT7S4PtuuD9IzNdhx71OQt4KhfrEJT3jeCxIvH/by79PAXl\nmgUsDn6mfg0MzZY6aAoEEZGQC0vXjYiI7IeCXkQk5BT0IiIhp6AXEQk5Bb2ISMgp6EVEQk5BLyIS\ncv8f3pwv5VCAQpcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(losses, label=\"Train\")\n",
    "plt.plot(losses_valid, label=\"Val\")\n",
    "plt.legend(loc='upper left')\n",
    "plt.title(\"Loss\")\n",
    "plt.show()\n",
    "\n",
    "plt.plot(accuracies, label=\"Train\")\n",
    "plt.plot(accuracies_valid, label=\"Val\")\n",
    "plt.legend(loc='upper left')\n",
    "plt.title(\"Accuracy\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Save the whole model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Saving the model ....\n",
      "Model saved.\n",
      "\n",
      "Saving the model parameters ...\n",
      "Model parameters saved.\n",
      "\n",
      "Saving the train data and valid data ...\n",
      "Train data and valid data saved.\n",
      "\n",
      "Saving the tokenizers with the vocabularies ....\n",
      "Tokenizers with the vocabularies saved.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Save the model as tensorflow format\n",
    "print('Saving the model ....')\n",
    "transformer.save_weights(model_path + '/transformer', overwrite=True, save_format='tf')\n",
    "print('Model saved.\\n')\n",
    "\n",
    "\n",
    "# Save the parameters used to construct the model\n",
    "print(\"Saving the model parameters ...\")\n",
    "model_info_path = DATA_PATH + '/model_info.pkl'\n",
    "with open(model_info_path, 'wb') as f:\n",
    "    model_info = {\n",
    "        'vocab_size_enc': input_vocab_size,\n",
    "        'vocab_size_dec': output_vocab_size,\n",
    "        'input_max_length': input_max_length,\n",
    "        'output_max_length': output_max_length,\n",
    "        'sos_token_input': sos_token_input,\n",
    "        'eos_token_input': eos_token_input,\n",
    "        'sos_token_output': sos_token_output,\n",
    "        'eos_token_output': eos_token_output,\n",
    "        'n_layers': N_LAYERS,\n",
    "        'd_model': D_MODEL,\n",
    "        'ffn_dim': FFN_UNITS,\n",
    "        'n_heads': N_HEADS,\n",
    "        'drop_rate': DROPOUT_RATE\n",
    "    }\n",
    "    pickle.dump(model_info, f)\n",
    "print('Model parameters saved.\\n')\n",
    "\n",
    "\n",
    "# Save the training and validation data\n",
    "print(\"Saving the train data and valid data ...\")\n",
    "train_valid_data_path = DATA_PATH + '/train_valid_data.pkl'\n",
    "with open(train_valid_data_path, 'wb') as f:\n",
    "    train_valid_data = {\n",
    "        'encoder_inputs_train': encoder_inputs,\n",
    "        'decoder_outputs_train': decoder_outputs,\n",
    "        'encoder_inputs_valid': encoder_inputs_valid,\n",
    "        'decoder_outputs_valid': decoder_outputs_valid,\n",
    "    }\n",
    "    pickle.dump(train_valid_data, f)\n",
    "print(\"Train data and valid data saved.\\n\")\n",
    "\n",
    "\n",
    "# Save the tokenizers with the vocabularies\n",
    "print('Saving the tokenizers with the vocabularies ....')\n",
    "vocabulary_in = DATA_PATH + '/tokenizer_inputs.pkl'\n",
    "with open(vocabulary_in, 'wb') as f:\n",
    "    pickle.dump(tokenizer_inputs, f)\n",
    "\n",
    "vocabulary_out = DATA_PATH + '/tokenizer_outputs.pkl'\n",
    "with open(vocabulary_out, 'wb') as f:\n",
    "    pickle.dump(tokenizer_outputs, f)\n",
    "print('Tokenizers with the vocabularies saved.\\n')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the saved model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading Model parameters ...\n",
      "Model parameters loaded.\n",
      "\n",
      "Create an instance of the Transforer model and load the saved model ...\n",
      "Model loaded.\n",
      "\n",
      "loading train data and valid data ...\n",
      "Train data and valid data loaded.\n",
      "\n",
      "Loading the parameters tokenizers ....\n",
      "Tokenizers loaded.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Read the parameters model\n",
    "print('Loading Model parameters ...')\n",
    "model_info_path = DATA_PATH + '/model_info.pkl'\n",
    "with open(model_info_path, 'rb') as f:\n",
    "    model_info = pickle.load(f)\n",
    "print('Model parameters loaded.\\n')\n",
    "\n",
    "sos_token_input = model_info['sos_token_input']\n",
    "eos_token_input = model_info['eos_token_input']\n",
    "sos_token_output = model_info['sos_token_output']\n",
    "eos_token_output = model_info['eos_token_output']\n",
    "input_max_length = model_info['input_max_length']\n",
    "output_max_length = model_info['output_max_length']\n",
    "\n",
    "\n",
    "# Create an instance of the Transforer model and load the saved model\n",
    "print('Create an instance of the Transforer model and load the saved model ...')\n",
    "transformer_model_loaded = Transformer(vocab_size_enc=model_info['vocab_size_enc'],\n",
    "                                      vocab_size_dec=model_info['vocab_size_dec'],\n",
    "                                      d_model=model_info['d_model'],\n",
    "                                      n_layers=model_info['n_layers'],\n",
    "                                      FFN_units=model_info['ffn_dim'],\n",
    "                                      n_heads=model_info['n_heads'],\n",
    "                                      dropout_rate=model_info['drop_rate']\n",
    "                                      )\n",
    "\n",
    "# Load the saved model\n",
    "transformer_model_loaded.load_weights(model_path + '/transformer')\n",
    "print('Model loaded.\\n')\n",
    "\n",
    "\n",
    "# Read the training and validation data\n",
    "print(\"loading train data and valid data ...\")\n",
    "train_valid_data_path = DATA_PATH + '/train_valid_data.pkl'\n",
    "with open(train_valid_data_path, 'rb') as f:\n",
    "    train_valid_data = pickle.load(f)\n",
    "\n",
    "encoder_inputs = train_valid_data['encoder_inputs_train']\n",
    "decoder_outputs = train_valid_data['decoder_outputs_train']\n",
    "encoder_inputs_valid = train_valid_data['encoder_inputs_valid']\n",
    "decoder_outputs_valid = train_valid_data['decoder_outputs_valid']\n",
    "print(\"Train data and valid data loaded.\\n\")\n",
    "\n",
    "\n",
    "# Read the parameters tokenizers with the vocabularies\n",
    "print('Loading the parameters tokenizers ....')\n",
    "input_vocab_file = DATA_PATH + '/tokenizer_inputs.pkl'\n",
    "with open(input_vocab_file, 'rb') as f:\n",
    "    tokenizer_inputs = pickle.load(f)\n",
    "\n",
    "output_vocab_file = DATA_PATH + '/tokenizer_outputs.pkl'\n",
    "with open(output_vocab_file, 'rb') as f:\n",
    "    tokenizer_outputs = pickle.load(f)\n",
    "    \n",
    "print('Tokenizers loaded.\\n')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Make predictions\n",
    "\n",
    "We have seen the scores obtained after training but what we are interested in making predictions and see how the model works with new sentences. The predict function will input a tokenize sentence to the model and return the predicted new sentence. \n",
    "- Tokenize the input sentence to a sequence of tokens\n",
    "- Set the initial output sequence to the SOS token\n",
    "- Until we reach the max length or the eos token is returned by the model\n",
    "- Get the next word predicted. The model returns the logits, remember that the softmax function is applied in the loss calculation.\n",
    "- Get the index in the vocabulary of the word with the highest probability\n",
    "- Concat the next word predicted to the output sequence\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have a function `predict_from_seq()` which receives a sequence of integers and returns the predicted sentence. It will be used to predict some sequences of the validation data and compare them to the expected targets.\n",
    "Another function `predict()` which receives a sentence, encodes it in a sequence of integers and called `predict_from_seq()` to give the predicted result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# map an integer to a word\n",
    "def word_for_id(integer, tokenizer):\n",
    "    for word, index in tokenizer.word_index.items():\n",
    "        if index == integer:\n",
    "            return word\n",
    "    return None\n",
    "\n",
    "# generate target sentence corresponding to the given sequence\n",
    "def seq2text(tokenizer, sequences):\n",
    "    target = list()\n",
    "    for i in sequences:\n",
    "        word = word_for_id(i, tokenizer)\n",
    "        if word is None:\n",
    "            continue\n",
    "        target.append(word)\n",
    "        \n",
    "    return ' '.join(target)\n",
    "\n",
    "# clean the sentence and encode it in sequence\n",
    "def encode_sentence(input_sentence, tokenizer, max_length, sos_token, eos_token):\n",
    "    inp_sentence_cleaned = clean_preprocess_text(input_sentence.split('\\n'))\n",
    "    inp_sequence = encode_sequences(inp_sentence_cleaned, tokenizer, max_length, sos_token, eos_token)\n",
    "    # inp_sequence shape: (1, max_length) => [[sequence]]\n",
    "    return inp_sequence[0]\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Transform the sequence of tokens to a sentence\n",
    "def predict_from_seq(inp_sequence, output_max_length):\n",
    "    # Reshape the input\n",
    "    enc_input = tf.expand_dims(inp_sequence, axis=0)\n",
    "    # Set the initial output sentence to sos\n",
    "    out_sentence = sos_token_output\n",
    "    # Reshape the output\n",
    "    output = tf.expand_dims(out_sentence, axis=0)\n",
    "\n",
    "    # For max target len tokens\n",
    "    for _ in range(output_max_length):\n",
    "        # Call the transformer and get the logits \n",
    "        predictions = transformer_model_loaded(enc_input, output, False) #(1, seq_length, VOCAB_SIZE_OUTPUT)\n",
    "        # Extract the logists of the next word\n",
    "        prediction = predictions[:, -1:, :]\n",
    "        # The highest probability is taken\n",
    "        predicted_id = tf.cast(tf.argmax(prediction, axis=-1), tf.int32)\n",
    "        # Check if it is the eos token\n",
    "        if predicted_id == eos_token_output:\n",
    "            break\n",
    "        # Concat the predicted word to the output sequence\n",
    "        output = tf.concat([output, predicted_id], axis=-1)\n",
    "    \n",
    "    output_final = tf.squeeze(output, axis=0).numpy()\n",
    "    \n",
    "    predicted_sentence = seq2text(tokenizer_outputs, [i for i in output_final])\n",
    "\n",
    "    return predicted_sentence\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(inp_sentence, tokenizer_in, input_max_length, output_max_length):\n",
    "    # generate sequence corresponding to the given sentence\n",
    "    inp_sequence = encode_sentence(inp_sentence, tokenizer_in, input_max_length, sos_token_input, eos_token_input)\n",
    "    # return predicted sentence\n",
    "    return predict_from_seq(inp_sequence, output_max_length)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And finally our last function receives a source sentence, calls the transformer to translate it and displays the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "def translate(sentence):\n",
    "    # Get the predicted sentence for the input sentence\n",
    "    return predict(sentence, tokenizer_inputs, input_max_length, output_max_length)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Predict a few first sequences of validation data and compare them to expected targets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Review: this is a very rare specimen .\n",
      "Original summary: c est un exemplaire tres rare .\n",
      "Predicted summary: c est un appareil pauvre .\n",
      "\n",
      "\n",
      "Review: tom pulled out his gun .\n",
      "Original summary: tom sortit son arme .\n",
      "Predicted summary: tom sortit son arme\n",
      "\n",
      "\n",
      "Review: tom isn t good looking .\n",
      "Original summary: tom n est pas beau .\n",
      "Predicted summary: tom n est pas bon .\n",
      "\n",
      "\n",
      "Review: that s not what i heard .\n",
      "Original summary: ce n est pas ce que j ai entendu .\n",
      "Predicted summary: ce n est pas ce que j ai entendu .\n",
      "\n",
      "\n",
      "Review: i almost forgot .\n",
      "Original summary: j ai presque oublie .\n",
      "Predicted summary: j ai presque oublie .\n",
      "\n",
      "\n",
      "Review: he has been studying for two hours .\n",
      "Original summary: il a etudie pendant deux heures .\n",
      "Predicted summary: il a fait deux heures deux heures .\n",
      "\n",
      "\n",
      "Review: wait !\n",
      "Original summary: attends !\n",
      "Predicted summary: attendez .\n",
      "\n",
      "\n",
      "Review: you re perfect for me .\n",
      "Original summary: tu es parfait pour moi .\n",
      "Predicted summary: tu es parfait pour moi .\n",
      "\n",
      "\n",
      "Review: there are things you need to know .\n",
      "Original summary: il y a des choses quil te faut savoir .\n",
      "Predicted summary: il faut que tu as besoin de savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir savoir\n",
      "\n",
      "\n",
      "Review: i can t believe you re considering doing that .\n",
      "Original summary: je n arrive pas a croire que tu envisages de faire ca .\n",
      "Predicted summary: je n arrive pas a croire que tu me croire .\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i in range(10):\n",
    "  print(\"Review:\", seq2text(tokenizer_inputs, encoder_inputs_valid[i]))\n",
    "  print(\"Original summary:\", seq2text(tokenizer_outputs, decoder_outputs_valid[i]))\n",
    "  print(\"Predicted summary:\", predict_from_seq(encoder_inputs_valid[i], output_max_length))\n",
    "  print(\"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, let's predict our own new phrases that we type ourselves:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: This is an example sentence.\n",
      "Output sentence: c est une phrase .\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"This is an example sentence.\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: We are testing our machine translation model.\n",
      "Output sentence: nous sommes nos machine mortes plaindre .\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"We are testing our machine translation model.\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: Why are you alone ?\n",
      "Output sentence: pourquoi etes vous seule ?\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"Why are you alone ?\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: You should pay for it.\n",
      "Output sentence: tu devrais le payer .\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"You should pay for it.\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: we have no extra money.\n",
      "Output sentence: nous n avions aucun argent dont aucun argent .\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"we have no extra money.\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: This is a problem to deal with.\n",
      "Output sentence: c est un probleme pour s occuper .\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"This is a problem to deal with.\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: This is a really powerful method!\n",
      "Output sentence: c est vraiment une excellente methode !\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"This is a really powerful method!\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: This is a mini-project for our natural language processing module.\n",
      "Output sentence: c est un noel pour notre dernier .\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"This is a mini-project for our natural language processing module.\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: I have a friend who can speak French.\n",
      "Output sentence: j ai un ami qui peut parler francais .\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"I have a friend who can speak French.\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: Since I walked very fast, I was in time for the last train.\n",
      "Output sentence: comme j ai marche tres vite j ai ete sorti pour le dernier .\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"Since I walked very fast, I was in time for the last train.\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Input sentence: I could not get anybody to go with me, and so I went by myself.\n",
      "Output sentence: je n arrive pas a ne m empecher de m aller a nouveau chez moi .\n"
     ]
    }
   ],
   "source": [
    "#Show some translations\n",
    "sentence = \"I could not get anybody to go with me, and so I went by myself.\"\n",
    "print(\"Input sentence: {}\".format(sentence))\n",
    "predicted_sentence = translate(sentence)\n",
    "print(\"Output sentence: {}\".format(predicted_sentence))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}