transformer-pre-processing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 align=\"center\">Transformer Pre-processing</h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###### By: [jihad akbar](https://github.com/jihadakbr)\n",
    "###### You can see this notebook clearly using this link [nbviewer.org](https://nbviewer.org/github/jihadakbr/transformer-pre-processing/blob/main/transformer-pre-processing.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1>Credit</h1>\n",
    "\n",
    "<div class=\"alert alert-block alert-success\" style=\"margin-top: 20px\">\n",
    "<p><i><b>This project was completed as a part of the Honors portion of the <a href=\"https://www.coursera.org/learn/nlp-sequence-models\" target=\"_blank\">Sequence Models</a> Course on <a href=\"https://www.coursera.org/\" target=\"_blank\">Coursera</a>. </b></i></p>\n",
    "<p><i><b>Credit to DeepLearning.AI and the Coursera platform for providing the course materials and guidance.</b></i></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "AbzZLqIPv6b7",
    "outputId": "19f2fc2b-6f1d-4b43-fd50-4c513e3936fd"
   },
   "source": [
    "Welcome to Week 4's first ungraded lab. In this notebook you will delve into the pre-processing methods you apply to raw text to before passing it to the encoder and decoder blocks of the transformer architecture. \n",
    "\n",
    "**After this assignment you'll be able to**:\n",
    "\n",
    "* Create visualizations to gain intuition on positional encodings\n",
    "* Visualize how positional encodings affect word embeddings\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div id=\"toc\"><h1>Table of Contents</h1></div>\n",
    "\n",
    "- [Packages](#0)\n",
    "- [1 - Positional Encoding](#1)\n",
    "    - [1.1 - Positional encoding visualizations](#1-1)\n",
    "    - [1.2 - Comparing positional encodings](#1-2)   \n",
    "- [2 - Semantic embedding](#2)\n",
    "    - [2.1 - Load pretrained embedding](#2-1)\n",
    "    - [2.2 - Visualization on a Cartesian plane](#2-2)\n",
    "- [3 - Semantic and positional embedding](#3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='0'></a>\n",
    "## Packages\n",
    "<div style=\"text-align: right;\"><a href=\"#toc\" style=\"text-decoration:none;\">↑ Table of Contents</a></div>\n",
    "\n",
    "Run the following cell to load the packages you'll need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "id": "_OpwqWL2QH5G"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "\n",
    "from tensorflow.keras.layers import Embedding\n",
    "from tensorflow.keras.preprocessing.text import Tokenizer\n",
    "from tensorflow.keras.preprocessing.sequence import pad_sequences"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='1'></a>\n",
    "## 1 - Positional Encoding\n",
    "<div style=\"text-align: right;\"><a href=\"#toc\" style=\"text-decoration:none;\">↑ Table of Contents</a></div>\n",
    "\n",
    "Here are the positional encoding equations that you implemented in the previous assignment. This encoding uses the following formulas:\n",
    "\n",
    "$$\n",
    "PE_{(pos, 2i)}= sin\\left(\\frac{pos}{{10000}^{\\frac{2i}{d}}}\\right)\n",
    "$$\n",
    "<br>\n",
    "$$\n",
    "PE_{(pos, 2i+1)}= cos\\left(\\frac{pos}{{10000}^{\\frac{2i}{d}}}\\right)\n",
    "$$\n",
    "\n",
    "<a name='ex-2'></a>\n",
    "\n",
    "\n",
    "It is a standard practice in natural language processing tasks to convert sentences into tokens before feeding texts into a language model. Each token is then converted into a numerical vector of fixed length called an embedding, which captures the meaning of the words. In the Transformer architecture, a positional encoding vector is added to the embedding to pass positional information throughout the model. \n",
    "\n",
    "The meaning of these vectors can be difficult to grasp solely by examining the numerical representations, but visualizations can help give some intuition as to the semantic and positional similarity of the words. As you've seen in previous assignments, when embeddings are reduced to two dimensions and plotted, semantically similar words appear closer together, while dissimilar words are plotted farther apart. A similar exercise can be performed with positional encoding vectors - words that are closer in a sentence should appear closer when plotted on a Cartesian plane, and when farther in a sentence, should appear farther on the plane. \n",
    "\n",
    "In this notebook, you will create a series of visualizations of word embeddings and positional encoding vectors to gain intuition into how positional encodings affect word embeddings and help transport sequential information through the Transformer architecture.  \n",
    "    \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='1-1'></a>\n",
    "### 1.1 - Positional encoding visualizations\n",
    "<div style=\"text-align: right;\"><a href=\"#toc\" style=\"text-decoration:none;\">↑ Table of Contents</a></div>\n",
    "\n",
    "The following code cell has the `positional_encoding` function which you implemented in the Transformer assignment. Nice work! You will build off that work to create some more visualizations with this function in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "id": "y78txxoHQtwG"
   },
   "outputs": [],
   "source": [
    "def positional_encoding(positions, d):\n",
    "    \"\"\"\n",
    "    Precomputes a matrix with all the positional encodings \n",
    "    \n",
    "    Arguments:\n",
    "        positions (int) -- Maximum number of positions to be encoded \n",
    "        d (int) -- Encoding size \n",
    "    \n",
    "    Returns:\n",
    "        pos_encoding -- (1, position, d_model) A matrix with the positional encodings\n",
    "    \"\"\"\n",
    "\n",
    "    # initialize a matrix angle_rads of all the angles \n",
    "    angle_rads = np.arange(positions)[:, np.newaxis] / np.power(10000, (2 * (np.arange(d)[np.newaxis, :]//2)) / np.float32(d))\n",
    "    angle_rads[:, 0::2] = np.sin(angle_rads[:, 0::2])\n",
    "    angle_rads[:, 1::2] = np.cos(angle_rads[:, 1::2])\n",
    "    \n",
    "    pos_encoding = angle_rads[np.newaxis, ...]\n",
    "    \n",
    "    return tf.cast(pos_encoding, dtype=tf.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define the embedding dimension as 100. This value must match the dimensionality of the word embedding. In the [\"Attention is All You Need\"](https://arxiv.org/abs/1706.03762) paper, embedding sizes range from 100 to 1024, depending on the task. The authors also use a maximum sequence length ranging from 40 to 512 depending on the task. Define the maximum sequence length to be 100, and the maximum number of words to be 64. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEKCAYAAADw2zkCAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOxdZ3hU1dZ+dya9kgZJ6L1IVRTFBioqKqLXhl671/Ip13LtXa/Xeu0VsVywYG+oCCiiFBVFBUGa9JKeQHqbzP5+7HP2WsOcycyQEDJhv8+TJyvrnH3OmWQye9V3CSklDAwMDAz2b0Ts6wcwMDAwMNj3MJuBgYGBgYHZDAwMDAwMzGZgYGBgYACzGRgYGBgYwGwGBgYGBgbYi5uBEOJ1IUShEGIl06UJIb4WQvxlfU9lx24XQqwXQqwVQpywt57LwMDAoC3A6TNyt+NCCPGs9bn4hxDiQHbsROuzcr0Q4raWeJ696RlMA3DibrrbAMyTUvYFMM/6GUKIQQAmATjAWvOiEMK1F5/NwMDAYF9jGnw/IznGA+hrfV0B4CUAsD4bX7CODwJwrvUZ2izstc1ASrkAQOlu6okAplvydACnMf27Uso6KeUmAOsBHLK3ns3AwMBgX8PPZyTHRABvSIWfAHQQQmRDfTaul1JulFLWA3jXOrdZiGzuBUJEJyllHgBIKfOEEB0tfWcAP7Hztls6HwghroDaJREJcVAHRAEAuow4AACwvqhan9tQ59ZyH1kCAPgLaVrXMT1ey3l56m8yrDPpdqzequXC+GQAwIjeHbXu9w2FWh7Qp4u6/zb62w6IrtKyu65Bfa9u0LrEAwZqedmaLQCA4QO6ad223//UcmSfvgCA4qJdWpdaVqTlrkOt4/W0v7vXrtNyfkIHLXfvql5Daiw5X7+v2ablLnWVAICGXn20LmLDei136pkBAJDuRq0r2r6TnmuIMlIKltPz13btSevz1WvdnJChdYMS6rS8Yqfqij+Q/a5/Y7/rA/p2BQCs3phHz9w5U8u5heUAgMTkOK2rqaH3QkSEUN9dQusa6ui1JCSo91RFeY3Wpaclapn/DXKy1ftpxw76W9i/XwDYvLUAANC7R5bWrd9Ez92/Vw4AYO2GHVo3sA+99Vev3w4AGNS3i9at+mu7lu3fxZ9/0d9vsKUDgJXr1Ht4SH96X61YS+/roZb+D65j78E/1mz1qwOAYZZ+uYNud7393l7moON6J53/9d3ZuVv86gBA1pQUSynpjbIHiEjuIuGuDepcWVPyJwB+8lQp5dQQbtcZwDb2s/256KQfFcJ1HdHam4E/CAedI0+G9cucCgCZIkaegWwAwCMLFgEATn91qT5327piLX/kfgsAcILnHK279qKRWv7Pg28DAJY8eJDW3XboP7X83IBj1fHPrte6xNOf0/KMTx8BAJx80/taN7PbYi3vWp8PAMhfRh9qRy2m4x1GXw0AWLDoBa27KYE2i9Sn1XVfnfql1p31OZ37zKwPAACvbKMPwKKxx2n58YPOoNf6lHpdZ/ShD7iko/6l5Rs3qOfKf5peS9yZp9JzPXIFAKCumDa+V2/+WMunzf0eAPB01hCtW3PTK1q+4RH1Wi8ddYnWzT1sk5a7vaf+f5Z8eq3WxZ/2rJY/m/MkAODASY9q3V0PXa7l+575BgBw+Di6/6pV9GEda33YxydGa13uRtrMDh6lPngXzPlD6y4+f7SWX57yhZZvu/0sAMDtd72qdQ8/ebWWL5usnvXF127VulMvfEDLb7x3PwDgqLPu0rp3P32InuVUte6jLx/TuqHjb9TyTOt3MeAE+vvN+vopLfc6Vv0OZ8+n92r3sfS+nvud0ncZQ7p53z+v5eyjJgMAvmW6LEsHAPMXKH3HI0n33UJ6X2YecY2Wv7fe2+mH++q4nv8PpLFzbT3XLVr8opbt/yEnHQA0LPsf7Qx7CnctIvufGvg8db9aKeXIwGf6hb/PxaA/L0NBa28GBUKIbMsryAZgfzJuB9CVndcFQG4rP5uBgYFB0xACIqLV0pn+Phej/eibhdbeDGYCuAjAI9b3z5h+hhDiSQA5UAmTnwNdLCszHjeeoTbeb3ofrL4veVsfjz/jTS2/8scyAMD0NeSlHRFN7vrtRcrrWt6RrNFGRuLnilYW9+K6TlqX2mOwll9cvBkAsGvbaq3LmXyyllP++A0A8O1csqDH5a/Ssv0G21pOYSQWxcCgbBWmqisjbyc6gp1QqIyerSXkYmdF0Zs2MjZBy/mVKiQTwUInHDWNHgBAGrOcG9m93JUqjBSVTCG1eg/9ruobpfX8bE0DhWFc0SqUJT1MF0v3sj1r6eI6gvV43jpmF9m/S7dnz4wll/VavZ6PvX6ujxC+RprLyW4LAREOhp+TzmBfQCAi0vl9uRcwE8BkIcS7UGGgMsuQLgLQVwjRE8AOqOKb85p7s722GQgh3gEwBkCGEGI7gHuhNoH3hRCXAdgK4CwAkFL+KYR4H8AqAG4A10gpGx0vbGBgYLCv0IKegZ/PyCgAkFJOATALwElQBTXVAC6xjrmFEJMBzAHgAvC6lPJPnxuEiL22GUgpz/Vz6Fg/5z8I4MFQ7lGe3hXfXvwEAKBgmrpszUi67RlPvqHlTlerOPbw757Wul8/+EHLfY6+AQBw3Yzfte6u7pR0nTnsaADAv78ky7/3yAO0vGCxsswbqsq0zjOCPIPklHQAwLaa97SufsUiLcemqLzWHwUVWpcWTW+6ARnKsm+oqdS6FGb5N2xTyeItu9K1blA0JZMj4yg/kL9LWd7cM+BvcNvKz4mL0roqF12rvkIlxuM7Z2sd96IaPL6me6ObjkdYr8vL8o6ie9l6GeH89rTvxZ+Z31O4lL6ReQYRXpa9vZ7puBcY0fJFdhHGsG8XEKD3V3PRxGekfVwCuMbPsVlQm0WLoa0kkA0MDAzaPoRAROvlDFoVYb0ZbNuah+smPwwAqP7zXQDALf3P1senD6ScQPUzSj/1OqqQKWClp889fygA4G9XPaN1Rz1E1xofqUo3P5wxX+seuv10Ld9wyxQAQGQsWeA/5FKZ6+jeqhqlhgW3i34kLySx02EAgN+2Utlitxj683RJVnHKxnoqd0yOJAu2eqvKeeQ2DNO6NGbZR8enaHn7Tuu5qqj0lcdBay3LOZWt97BAeH25Wp/Yh/IQDcyyrrbyAzx27pUzsDwaT0M96WId4rDsn457AY45A+YF2Od66+hc7Rk4xPv9weXHtG92fiCEZzBoG2jFBHKrIqw3AwMDA4NWRetWE7UqzGZgYGBgECQEBCIiowKfGIYI680gLiUVg075GwBg2BTV6Tn/CQrdvHjszVo+cJlKFufWUulpHEuKHr71KwBAdQmV65afSI06N1sJ0Kn/oQT03wZQo9NVO1VTWUa/g7Xu3d+oqzQpurt1TwoL7FiyUctpI88EAPy6iRq5jkqg0ElaBHXo2siMIQulzOpgrUymhseEThTGiUqgMFGBlUBu3EkNcBFRdC87gZwSS2/6ChYmqa9QYSJhdWUD3qWd1Q0qjsNLSxtZbMeptDQi2vetGEoCud7tkLTmYSL2LB6HklPp4Qlk39LSUNCaoR8TZmplGM/AwMDAwAAwOYM2iQEdgB9PVRZh0lWqyuqrl27Xx4vrP9XyBQ8pioLZF1OCta6crO1vL1UlqpmHUSXXzZ9TGelrvVXpKE8Qxy8nWgLb8u48qLfW/biUPIMuqapprStLyuavogaynDNVGWv+VipNTeurGb4RWbIZgPcbMTMlVsvlWxX3UlU3ek2JHakpLD4xRsvVlSpx27iTKBpcLIFsW94pLIG9iyWrG8pV4jkijjwP3nTmmECuJys7MlZd18sziKTXwh7KVwfA4+AZBEogOyHCX1LYQe/VdNbInlt7EQ5Zbb4+gAXPfxdOpwZyAAJ5McaBaCEI0WKlpW0NYb0ZGBgYGLQmBIxn0CaRu2YL7jlUWfJvLV0IALjsBiKpyn3zSi0/+tAcAEDmghlal1xdoOWXssYCAE59nMj/Pn1voZZXC0UQ1/nAK7Ru03Siu8jop2L+x40kdklOKvd1prLS70gnIrmVxVQmOnagYrp88WfKI3QcQkyX7i3KS+HNY0lMLt+umtVqUumaiTlJWrbJ2QCgxqKjaCwr0TpXNFnmtpWfxHISsSy/0lBl0Vkk0PV501mtFb/ndBkellRw6aYz1igW4+sZcDoKrwazRl+LP5BnwL0AO3/Bm86c8wh7ljPwV4Zq4Iuw+02JCC8vuj0hrDcDAwMDg1aFMJ5Bm0RilAtHZivrOOeBywAArugj9PGn0k7Tcv9xfwEATnySKCBOOpRI3fpZpGwXn9BX6157kBrQFm5UFvuEN+j4iivWavmA+3sAAM4ZnkP3z92g5dwNvQAAXQ8nz2He+5STmNhd8eL/t5C42jOP7Kfl2g3q3NhkmgGQkkrVPFsWqKazum6UB0gcRNTtcfGUMyjarvIStSWUn4iMIy/ENrzjozidBb1V7FyLYOR33Fiv0TkDZo27eTWR5RmwRjQ4WVt+/uls6gkeu+XVRBTH594Ci/kTF6AjAuUMvPQBgvGeAN6FKwDzxZ54J6E004VyroEqLTWbgYGBgYGB2QzaIuIGDsCQbxcAAP6Truik31o9TR8/8x+PaHnVZ2qgSN/jiKJ6+0o9XxpLb1I5g8gPaU1ipx5aXrnsWwDA3UfRxK5nd1J8/p9jVBXRoHiq8/e4iW5h52ZFKtj9POpDqJxBRIODMlUuobacKoziBhym5R2ffK50qWO0LrkbTdQqqlaeSz0jyktk07+SPeQZbFur+gRqS8q1LiqWpprp+7MKomiWc6ivUqa1jKL8R6MDHQU3enmfga4mYtPFRLRDzoD1GTjlDPxVE9nw+PEMbL0/WupACHSuU+0/1+1pLsKgDcD0GRgYGBgYwISJ2iZWbS7E8IvU2L21z6nxgzXv0PjAlM5k+cvHVNVRx0FHaV3hKho7mfA/dZ2FAw/VuqH/og7klG+nAQAyV36udWUNZO0ek6MsZ88ioqi2aakB6mxOGH2x1rnENLp+hYr5uxlFdURP6onYtV6NVUzsznIGbFZukWVlN1STtR+TQzmR7Eqy4pdVOeUMqDLJRgLLGURxz8DqU5BRDr0BoJyBVzWR27cDmcPZM/CXM/D1AnjOwPYCOC21EwLlAfwNt+HYGx3Aphip7UII4dWt354Q1puBgYGBQavChIkMDAwMDACTQG6TEBERiLLoIW7veCIAIPY6Iqqbve1xLU/tqeTnfqUE8fVPUujkvLfUjOQ+uTRp7IVJI7S87XlFN/HnE69rXVYs/fo8304HAGydRY1qqb3O1HLe74oOoyKbQj98UlnjKkWkx0MTVSkU5tm5Uc05SDmQKCZiulIyu9yt1rnrKKkd2YlmZndhyd76ahUeqimhkFR0KpWJ2nA10DyGaDYPuapA0VF4J5BpXWWtmhMR54eozk4g87JLEeXLBMlnGPMy0kaPbwLZ7UBKx0tLvRO4TSeQQyktdVofCoINM/m7fWtVhvq///5HyuePxiTc0fLz/QwMDAzaKYQQEBHBfQVxrROFEGuFEOuFELc5HL9ZCLHM+lophGgUQqRZxzYLIVZYx5a2xGsLa89gcI8MfDdNNZtlHqESxO9lkuUcfc9FWo61/jgnrCUK63uvpklm9qSyIYxdre/G2VrudN0xAIC37yJyusO6UtPXqtcUUV7hCqKF7vFPakArWq08mGX5NF2Mk9aV/qL+nnzi2NZyKk0tyldWfDajmIjqQqR4lVYClU9Ci8gkzyLHQ1a8naSuaaAy2JhsX8s8opaS0by0dJdl+cvoeJ81AFBjkdKlcAoIp6azxqZLS/0RzdkJ5Ag/paW25eavtFS2JPWEw/+8o86YXe0Grhb4YwohXABeADAOwHYAvwghZkopV9nnSCn/C+C/1vkTANwgpSxllxkrpSxGC8G8RQ0MDAyChUBLeQaHAFgvpdwopawH8C6AiU2cfy6Ad1roVTgirD2D8hWrMbvnQQCAkx5SsfxTJtytj//TIp8DgPsfnQAAeP3SV7Tu6l+JuuLqCrXhnjyCaBl+uZVKSw+eqUpGV1/3kdadcQOVqc56VuUKilgj1aTDe2h59SIlf7mKyPGOZRTT+UsUXUZ0AuUpVuRT/qKoVl13cGcaUuNJo9JVe7Yyt3AbE6kMtWMls6LrlUdQW0GeASeysy1bUU9eDC8tbai0ms4iqZGNo7reqbSUvJxIa96xF22zAx0Fz0NEeDWd2URzjI6i0aG0lLFKO/1z+i0t1Z6Fcx7BNI3tv1CspS2SM+gMYBv7eTuAUU4nCiHiAZwIYDJTSwBzhRASwMtSyqnNfaCw3gwMDAwMWhcilER2xm7x/KnsQ9vpIv4aYyYAWLxbiOhwKWWuEKIjgK+FEGuklAuCfTAnhPVmUNsosbZCWZwzkuYDAI567RB9/AFG1FZ4wX8AAOtup5j/jxfTWMwB49RQnFHH0OZ8/8n/0XK+NacmLZqs0ZwLLtPyyv+oaiE+5OX8vmSZP9tD0T0sXJGvdX8f3knLuT+rprT4gyjPsGTzTi0n16s4/eAcNmoyJQ1NwZ1A9++USFa+XXHEh/vEshGbdtNVRC15JjHJlHOotazwBuH89qmsU8/Ke8s8DeQZRCT5rnOisA6UMwhER8GbzngFiD0cx9/AmsgQLD+nypLmVr3s66KZ5t6+nRbbKIiQPINiKeVIP8e2A+jKfu4CINfPuZOwW4hISplrfS8UQnwCFXZq1mZgcgYGBgYGIaCFcga/AOgrhOgphIiG+sCf6XMvIVIAHA3gM6ZLEEIk2TKA4wGsbO7rCmvPIPuAXrhz5lsAgGt7/w0A8Ouwv/TxI375TstDbp4LAPjgNKKFnv7ZOi2/9s5oAMDmGNofyxru1/Lj1gjMf/fqoHVbOhygZZuojVeSdKnepOWe/ZWVvmophQmzD6Fn+W62IppLZL0ByzeTVzjUor4YkEH9AMU1TceuS2vcWk6Pp5i/Hb+vKyNrPZYdt2P9nopdWheVTPetsTyDOochMwBQU297Bs45A1esyjV45wx8q5n45bkX4HGgmWh0qBzy+Blu41RNFAjNHVjjz1vYG16A06Maq69lIATgimz+H01K6RZCTAYwB4ALwOtSyj+FEFdZx6dYp54OYK6Usoot7wTgE6vHIxLADCnlbDQTYb0ZGBgYGLQ2WqrRTko5C8Cs3XRTdvt5GoBpu+k2AhiGFkZYbwZ/FjVg+Et5AICpY7oDAH4+epI+fvB932l5yw+KYK7/j9rbQkrqYC0PWq76DyZspt/xtZ2IvO3dRT8BAEZeO07rnvudQny9rZi712D4+VR5NGH4xQCAHz6kv32Hi4iiOrdWPV96FvURlORRh7B93ZxEsqC3ldOUFtsK530KO1llUwYbTmNb5OVsuExGMlUG2d4NH4sZFU8xfbtyqd6PZ+BUTcS9ALuayAsOlUkNDqMsAWcK63o3G1Lv0IEsWG24rfeXG3DpaqTmVw3xXERz0J7D8OEEIUS77UAO683AwMDAoLXRQqWlbQ5mMzAwMDAIAWYzaIOoqyzHhoVzAADutz8GACxOJjqGxHH3aLnnEacCAMY986PWvTJpkJa/vvwlAMCykedr3ZH3nKLliqlqnnHCuUR098l/f9Xys4NVGWt1Cd1//cc/aPmYqf8CANTspKYz0Y9KU+3ZCF26UOnoz/NpRrKNDqDrL9xFSdlEayqZK4ZKQHNZ6WjvDr5hmEpGEZGWSMfjrJCKp5ISyNHJ1CBnh4lqOcUE+/+oCdB05tJNZ6STLgp/2eEf/6Wlvk1nTglk7xnIvtcJiajOz6SyiBBmH+wNmMRwK0O0HcK8lkZYbwYGBgYGrQkBgYjI9rkFh/Vm0LN7Fh6bqsj+Trr4YQDAO+vn6OOH3/KClp8/ZzgA4KBTb9G63l+/quVnpqlmtfLtVG4adRGtj33vPgDAx1soabv199+0POSSowEAJX9u1LolM/7Q8plRig6aW5AlsTTD2MahvdO1PP/9rVq2rezIks1at7GUylyTrTdoZDR5BjsY3URkLeNmsMA9g+wUShDbVrCngpreollpqZ3MrvdjudfZTWecopt7BlH22457Br5vRe/SUvoHDDQDWZeW+ms6c6CwdoKTB7CncPlZH+GQGnbS7S2004jH3oNovxTWYb0ZGBgYGLQ2WnOGQ2sirDeD2LzN6POAirtnDbkCAPDVog/18W9Oo/h74TRFN9Gh20Ctu3k+xe+HWZZxhx5UbnrP1xu03G3kkQCApz7TDLOoyKPjiRPU9eN6LdK616b8ouUzV34HAIhKIKK5lYU0PMaO+Y/IoeO1ZcU+xxs20/3XFA7R8ugYl3V9es1bSuj6EdnUs2Jb1DXM9M5MopxBjZUAqN9Fpa0xafRcuunMTeu55euuV8ddjLrDK6aucwZEdwGXE1Gdc2mpbfHzgTduhwYz6Y/C2qFpbU9j/k501e3UcDSATVS3r59i72CfvCwhxA1CiD+tgQ3vCCFihRBpQoivhRB/Wd9T98WzGRgYGPiFFSYK5ivc0OqegRCiM4BrAQySUtYIId6H4uUYBGCelPIRa+rPbQBubepahTtr8dIHquJmVd5QAMCOxKP18Rkjz9XyhioV639oIV3y1gfe0/Kqe8cDAL5JOVzrPviQCAev/Yeiu3744Rlaxxu8lrtVNdHgQ4iSvLSe6CyKv/8eAJCcTRQWCzdSU1eONQqyXzobQlNLlnmqFX+vWb9G6zbX0tjL02NVNU5MIpHXcc9AVPh6BrUeyhmkskE79iSW2l1kucd1767lBsuyrqwnugtuIdsjLiPZWFAvIrg438om52oin9MAEF21d9MZp7C2rhmAdsIV4WwLOVcTNXmpgGivFSj7HwQi2umkon31qiIBxAkhIgHEQ7H1TQQw3To+HcBp++jZDAwMDBwhjGfQcpBS7hBCPA5gK4AaKBKmuUKITlLKPOucPIun2wdCiCsAXAEAXdKS8fB/FP3Ec32VZT/898X63OVPEh21XY3z913ztO6qrVTH775EDbJ5ilmTfZ57WctXjDwHAHBnwWaty+h3sJZfXKxI6S4/jCzoOGZObvlWxfozRlLvwqK1RVq+wYrZd4qkChuOzhadROlqun9ZMss55CjqjNgUqkbaVkzeQGNJnpbtXgSeM8iIJy+n1nruOpYzEPGUi7CriKobeJ8ByxlYNBcuxmHt1GfAISN834puP3QU3Auw4VVNZFdDOegA8hgCDawJhkqitSx+41m0HbTXprNW9wysXMBEAD0B5ABIEEKc3/QqgpRyqpRypJRyZEai8wxeAwMDg70BIZQREcxXuGFfVBMdB2CTlLIIAIQQHwMYDaBACJFteQXZAAqbuoiBgYHBvkA4ftAHg32xGWwFcKg117MGwLEAlgKoAnARgEes75/5vYKF7ZGpuDVdhYlGRX0KADjnxmn6+OonTtdydZ6aDfDOOY9pXbdzHtTyuf9TyeKvei/XurhUmoccPe8VH12vAwdoedHiLQCAxBj6lQ5gFA+bVqiQUL/zafrYmpW032UNV1GxqEJqeuMJ6swMFdrZuZ5CSxU9iJoipYtiO41j96zcRU1njSU0Yc1uTONNYxlsnsEuq4y1biclkCMSiE3Vji5VM9ZTTj3htugovBLIvIErmhrY2EP5qBoancNEjUFOOuNwiuEGmoHsb730+Mls2+sDhHS85jg4nBooIhSITdVElPYeBMLT6g8G+yJnsEQI8SGA3wC4AfwOYCqARADvCyEug9owzmrtZzMwMDBoCkIA0YaOouUgpbwXwL27qeugvISgUZJXiGkPPwsAeG7rEgDAv059VB//6ogbtHx4N9U09cuj1Kj10FWHavmyyU8CABaXfK91g/5OpHS/P6HKRHscdpvWXX5cHy1fe7MiulvA5gKcOZCSufOWqQa3U4fRjOMfvqKmtKyDewEA6lb9rHUxSVQmmpqpqCd2biTyuOoOJCd3V+cmsPtXMaK6+iLyQiJjlWXOm7oSWYNYnCXXMzqLiCRq+7DX2bOOAe/SS7eVWI70mqHASj8d5h1L1nRmW/wNfI3LN4HszzOwrfjGRl5u6ktH4XX/PW0601PVmrbWA1Uj7unshGC7YdtC1+y+f4LmQ4jQZmSHE8K6A9nAwMCgNSFgcgZtEskdMzHmGkVD0XeymiX9xCNX6uP/vIUmyHUZrryA146lRq2DGolI7qI6FX//ci1RQDw2abiW5/9HlWZe+hDlCU4bQPH/y3eqmHzeGor59zmV1r+75Et1n+5ELldVRPOQM0YdCAAo/Y2eKS6V7pXaRTW1rfuFSkRryyl/kHxgNgAg0UVWd+H2MrpXHptaFqeuy4ngElkZaIzlXdSVkWeBGF+iOp4ziGKWp8ey3KN4zqCaxcmjfT0DMCvfhr+cge0Z8Dh+o9vXC5DEKehYRhoa7UTw55oy0HYM0X5zBu0z+GVgYGCwF6A8g4igvgJeS4gThRBrhRDrLdaF3Y+PEUKUCSGWWV/3BLt2TxDWnkEvVwVmJM0HAHTOVfHtM5dRI9mtrGpl3Tw1/GbEdx9o3cIxJ2t56On/UcKvNKN4dPUyLb9do8zMJw+kmH/k0k+1bBPQlTEK7I73/03LjXcoz6CLhyz0uopSWj9EtVoUTZupdUlZx2u5Q9/tAID8WqLNrmcU04k9uwEAulbTa17Drl9TROdGxRPpnF4fRW/eaGvOcl05o5iO8a0AKmc5A69qIstj8MoZVDbtGXgc6Cga/FTtUDWRM/mcprBmuYFQBtY4zUD2Z+0H8gIC5QLaqZHZrtESnoEQwgXgBQDjAGwH8IsQYqaUctVupy6UUp6yh2tDQlhvBgYGBgatiQghWqqa6BAA66WUGwFACPEuVDNuMB/ozVnrF2G9GWzfWISbz1WUET/mqVj7w10pTv/WHzTi8uonFwIATnqTiN76LycK6+mzFXXFlreoambZ7VRNZBPJ4YtntW7jHBprmTlAkeLl/krDdSp7jtZymlWh07iM6DC41ViZpiqTiv6knEX6uVTbn9BPxfmL6z/WuoYaoouI6qoI9roXU1d2XSV5BtUo13JMKlFL6PVu6lmwcwZVBURnIaOIQM8O5RMwzEUAACAASURBVFfWkmeQyOP3DkR1vNrGqZrIi3rCqhzylzOos6uJHCgmAD6K0tkzCJQzCGXoTbAIJY/gdPvWTEM43781B+60bXcphPxRhhBiKft5qpRyqiV3BrCNHdsOYBR8cZgQYjkUf9tNUso/Q1gbEsJ6MzAwMDBoTdh0FEGiWEo50t+lHHS71zz/BqC7lLJSCHESgE8B9A1ybcgI680gPSkaF45S1UG5444DAPRLpHr1wW/doeW371Cdx8eeo3MwGBtLcequ3yqLv9vD52nd4+dTNdLY4Z0AAL8+SY3RBazmf8gDKmZfso56A77dRMft5yqcT8NvXNFkxa8rUTmPvAKy9nt1o9h+dC9Ffc1HVTbWkzUvOqr7d5d0zYYq8gaqa+nc2M6+3b4RNVR5FJOsjvOeBk9Mos8a7hmkeXUgq/xKJKPF5qRvTjkDpw7iOtYnEBGgmsjjMMjGH4W1k2cQsE/Az/+/k76dMhwbWGihaqLtALqyn7tAWf8aUspyJs8SQrwohMgIZu2eIKw3AwMDA4PWRAs2nf0CoK8QoieAHVAzXc7jJwghsgAUSCmlEOIQqOrPEgC7Aq3dE5jNwMDAwCBICLRMAllK6RZCTAYwB4ALwOtSyj+FEFdZx6cAOBPA/wkh3FA8bpOkKptzXNvcZwrrzcDTvTdqp6hS0VkDDgEAPLaaJpFd358mnd3f+b8AgPh0Kg2ddPlBWv7sWrXu6HVEEZFb+7yWD/qXmmD2+IVTta6chWwmH90bALBsIc1QfuvnrfQs/VT4aMt3VHoal3qCln/Ypko/K6sp9DKqN9FZuNNV+IeTy/FEpjulMwCgM1vvZmGk6iqafRCXpMJAPMThHSZSCeT6aura4glkGxV+Skvt2QWRLGTnRc7mECZioxF0SKjBi06CUU9Ih9JStl6XlsqmS0v9wam0tL2ibadq2x5CzBk0CSnlLACzdtNNYfLzAJ7ffZ2/tc1FWG8GBgYGBq0JQ0fRRrF+cz5OuUSVfxbPuQ8AMH4uWXNXZVIy9eWbPgIA3Dl7ttZ1Puh6LS94ViVov/icSnXHs3nEOOVaAEBu7YuOzzI2S7mO3QfTpLM/lxNtdPcx/QAA3079SesSJxA1xjd/qjLX/ozi4YgsKgEtczU9yKccytrOSSZrvrGOeQbFJNtkdtyaRyU1w8V0UMlinqyubfRNxlbW0r28PIMG5Rm4Yok0z8sziPH1MpymmjUEmHQmHMpZAUos+y0ttZLZ/mK/gSisvfQOZYZOOn/ViM2tonR6rNbMX7fTz0X/aEHPoK0hrDcDAwMDg9aEmWfQRhEVl4isIarZ6rhfFQHckhmv6eNfLP9Cy4u7KHbsh2Mpz3L5pxQnH5OmrNXbPqNGssevH6Pl15cpgriurFySU0C7v5kGAJh46Gla9/DDlL/o+K8jAADrnlygdZldM7W8bbPKGXRm1nifNLKg8yqVFc4tcD78psjKFaTxRi82d5jTTXdI8vUMGncS6V1MB9XsFsgzqGClpXzes84ZOMw6BgAR7esZOJWWBhpuE+Gn6Uw41HY6eQGBZiCHgmDmJRu0D5jNwMDAwGA/R4QZbtM2cUB2HH66eygAIGG8GmHZdyyNujzylQ1afvMfiiL6s1Nu17qZI87R8rOPq8FqVzxLYy/Tpz2p5VceVB7DSwdna11VIdE1rJ6maChOe40otO9iFNWu4RcCAErrH9C6Pr2pQe2Heb60IpmCrr+4WFnhieyN6GKx9y27VE7g6O6+JHQAUMbKdbqkqvxDHLOgG8soZxCdrI7XMMu8hq23nYBKL8+ArqU9gwSqGpIeRnpnkdJxa98pP+A13MYrZ9Bo6fx4BhG+upDoKDSdhTNRXaBBOKHQXTvBaXX7/PgJQ5icgYGBgYGBgGj2Zt9WEdabQeHKv/Bc3/EAgLs+/hwAcNMRVM3TYfTVWk76bjoA4JspB2gdHy5TPEF5AUkfEBHdU0spjr75Z0UjMeIG8jxKlxPp3ZdPq3GZlzZSBRG3ILdHqJ4BXtt/zMBOWp7zlqKu5rF3V/5aLa8p6AgAyGDjKaPjqdpos+UZHJvBJrowcM8gu4Oy2LmF4y71zRnUsAqdGrdvTN/NKp9czGNpdMwZMM8g0jeX4PbqE1DXCpQz8BplyfI3duWPPwprp/xAILpqf3QUwSLCT0W/P31Lo50as/sEbZ1Ib08R1puBgYGBQWtCoPmGQVtFWG8GLgGkWENZzvpeEdEtuWWlPj7sNIrPn/y4orB+6hDqQP5+9AQtX/Cq6jwefuJRWvfKezSCssYaa+k+/m6tyx6yQssr/z0XAFC/4EOt493Oi7aqyqXMGPqVH9aVRmDWlinq6pQoRtX85xItLy87EgBwBqsWikminMOavAoAgCuL1fMza7qKWfndUpRnUMHi/LUlVFllD8qp9TjnDOwqpIY6uhcfZGP3GUQlUE5Deuj60uXrGTTKADkDly+FdYTfnIEV8/fqQPaNurdE7NdY3PsZhP+ek3BHWG8GBgYGBq0JASAqiJGW4QizGRgYGBgECRMmaqNIHzoQ5y/4DgBwXdIwn+OLvx6k5cRxao7BYbPf1rqnd9Fc39MvVjOQl37yH60bcfKNtL5TDwDAlF92aN0ZBwzUsh3m2PDpQq3L6PcPLX+6TK07g5G39Ukluga7HDOHhYFK/6AEcm7MCABAx470zDEp1LT2V4EKE3mKaFKaizV38WRwxwR1XxlJ7+raEpp9kDxEha84KV5lPZWR2tUU7noKE0Wx59alm36azhBkaWm9219pqU1HQZf0ChM1d9KZE8VDSJPK2umnhQEghAkTGRgYGOzvEGi/m31YbwYrtu5C38mqJHP+tYcCAGqKK/TxuUPHa7nTscrKP/8rKqF8szclaO0EZe81n2tdZBxN9+o7+jAAwPTPqZyUN131TlBW8Pr5W7Su331ZWl63RlnsPYeQNR+bR8lum1qiG6OgKF5JFNi7uqkGtNRelHRO7EAzkkuLFUW1u4DKZSNZUxq38rMTlWdQy5LVPIEckZTqs6asjnsG1jWZLiqBaDpsL8cV5ztrGQBklC+Fdb07eM9Al5ayf0peRupkuTk2hzmQ1/H1kiWw/SFQzbnthYRCVOevwS3Y9QZ7FyZMZGBgYLCfQwggqp3ONQ3rzcBTX4eK3PUAgK+ufBgAcExPGgjzVc5QLb90h/ISzrn8Ea2bX/i9lkdc9gQA4Ofb79K6fseQfN0ElR+46obntO6zaHpTvDBMNZDNXkqjSM8f1U3LN877DQDQ7egBWlf3B+UX7DLRjEwqFy1aRfH/yvhSAEBqH/IskpgXUVGqms7qcimnEcWa0riVnxKrrOwEVg5au4uG30SkqN8hL/esdBhk42blpry01LaoRYyvBwBQaal3zsC3jNSfZ+B2KC1t9BqEE1zT2Z4MvFHXDTAvOcBnxZ6Q4okQXIBQzt0b2NeG8958+SZMZGBgYGAAwISJ2iQG9OyED99UuYBhJ6nvGf0O1scX33y0ljv+8ToAICqWqnE++atUy2//Q617+R6ik7j/tRFaPrGz+lVdXEY5h23Lftfy4EvUvd5c/JbW/bNXqpbL8xRpXua1Y7Uu94uvtJzYSVFxd+yRp3Url5CXUV2iLP60I3toXZqHBt7kb1ZEc5U7GK1EUlctcwbqlBhlZdvjLQHv4TewYv3cm6jglUN2NVGDczWRp1LlDAT7XXM4EdXVOXgB3DOIiPDND0QwCoz6Ol/yuFAscC/yOVM5ZOAHAqLF/uZCiBMBPAM1x/hVKeUjux3/O4BbrR8rAfyflHK5dWwzgAoAjQDcUsqRzX2esN4MDAwMDFoVLcRaKoRwAXgBwDgA2wH8IoSYKaXk9MWbABwtpdwphBgPYCqAUez4WCllMVoIYb0ZNPy1DrnjjgMADD9TUViv+IqqgTY+8LiWPztQDZf5v3c+0bq0n97XcspHDwEAypi1e3I8eQkV76lzk7J7ky6PKLLjT74TABAdQX0MGUVEV9FQZVXrDB6jdXn3vkTPMvJsAED6ULr+jppFWq6vVMNv4noR0V6vMqp2+s2iy6gEeQbRjK6CIylSWdbcM6grr9OyJ0Zdl3sT5Q4jLr36DFj/hKfcqqBxGHwPAB7h8tEF6jOIYIF4m2bC33AbJ8/A5VCh4++fOthRlhyBvJB2WprebISbZ6VyBi1yqUMArJdSbgQAIcS7ACYC0JuBlPIHdv5PALq0yJ39IKw3AwMDA4PWRIh0FBlCiKXs56lSyqmW3BnANnZsO7yt/t1xGYCv2M8SwFwhhATwMrvuHiOsN4OyGjdmrVZe0qLxKuZ9/8H/p4+fc+M0LZ9bpeLYT/fdqXU7rj9Syx/d9jEAYHwW1e6vvYdI6Xb8ouL3/S54TOuWfUYx/WX1ygq3+w0AoHTWB1q2rd1NDSzOz6qFcs5U/QMxgw7RuuJ6GuHZUFMJAIjqNUTrBhTQs9ZVqPxHZQ31WcT19x0vCQAR1ep3EJtKlnvxWhpuI2OTfNaUVZNnkGlZ6TxnEM36DOya/Yg455yB7QVw8rlar/yA0tf5yRk06g7kpj0DjmB1HN6ehfM5TpVDoXUr++pa01h2vn94WeutChG4WoyhuIlYvtMv2ZcnHoAQYizUZnAEUx8upcwVQnQE8LUQYo2UcoHT+mCxTwpmhRAdhBAfCiHWCCFWCyEOE0KkCSG+FkL8ZX1PDXwlAwMDg9aDXVoazFcAbAfQlf3cBUDu7icJIYYCeBXARCmlttiklLnW90IAn0CFnZqFfdU98QyA2VLKAQCGAVgN4DYA86SUfQHMs342MDAwaENQk86C+QqAXwD0FUL0FEJEA5gEYKbXnYToBuBjABdIKdcxfYIQIsmWARwPYCWaiVYPEwkhkgEcBeBiAJBS1gOoF0JMBDDGOm06gO9AZVWO6NwvBw++dB8A4N4jbgAA3P3Bdfr4cxUUErr8bNU09tWxFEY6ai3RUfzyoDr+6Ktnat3U6yjBvNMKidw0gcjprl1B5HgvLNoEALioN9FF/PUphQvjUlVI6rtN9EzllRR6GTtQTTLzdObkcuQ12hQP7jRqZOtRR8frqxXRXGUtNY8lHEwJYh7mcFUpAyM+g+5Vt5QmkXlifMNEu1iYqJt1scZ6KkeNjKPwmKZgiKWQGEe99boieNNZo0NpqUMjmbq+r84jAySQHZrG/E0/a6915AbNR0s1nUkp3UKIyQDmQJWWvi6l/FMIcZV1fAqAewCkA3jRCt3ZJaSdAHxi6SIBzJBSzm7uM+2LnEEvAEUA/ieEGAbgVwDXAegkpcwDACllnhUL84EQ4goAVwBA147pTqcYGBgY7BUoOoqWsRaklLMAzNpNN4XJ/wDwD4d1G6EiKi2KfbEZRAI4EMA/pZRLhBDPIISQkJU1nwoASV36y+N+VWG3O7uoRqn/nkMzjJ/4fp6Wew5We8sLCWTZvzJjuZbPzbSs2El3at2Gf1ADmY0JnUl++sC+Wv7xR0Uq98DJg7Vu7ouLtZxymqLT/ux3oovoz8o1/9ZdJaB3OZC4cewSlJTt2YGs2cY6ZaVXFVRpXWIHulY0t4J3FQAAYjpQaWolS9bWNvrmscpqyHOwr2VPNAOAyAw+1azpBLJbzzBmTWdOnoHb2TOwqSf8TTrTngEjn4sMIYHsRHTnzxp00jud2lxj0l+uO9g4b3M/vkxpLKG95tf3Rc5gO4DtUko7RvMh1OZQIITIBgDre+E+eDYDAwODJhEBEdRXuKHVPQMpZb4QYpsQor+Uci2AY6EaLVYBuAjAI9b3zwJdq6q0BEtmvAkAGPizIn1b0f1AffzMZS9r+YRvVXPafQMotHTWu19oecYLFwMAbp+7XuuGpZBlbQ+HKX/rCa274tgbtDz5phcBAFmPTtS61Y/O13JOH0Vkt2ktlZN2Yxb4oExlWf9VWqt1ccwdtSmuc1meISfRlza6tIYdT6eYfRyrh2ssUQ1qsekpWlfGSOeqGnw9A54zSLRoIHjOICrBwaOJjPHVgZWWeuUMnJrOGK20Q4OZ4I1ozDOIDJAz0E1nIVj7HNzjaC7C7yNj/4ZA+/UM9lWfwT8BvG1l0TcCuATKS3lfCHEZgK0AztpHz2ZgYGDgF+01ZLZPNgMp5TIATs0Yx4ZyneyuWbj6cZVuOPCCZwAAeXNobOXdYykVsXSIomk4ZvbrWld7JtFBbDriFgDARzdRo9jCfxDp3a71Ks7+6/Nk7Z/2D7rXZRYdRMPg47WOx+GPHKIG3Ux7fY7W8TxUSrkairM8n6z5VDZ8JjpBWfFriiknMDDZNyZfWk/37J5Ox2N5zN3yDOKYZ8DHYlZbXgJ/vmo2yCfWssgb3ZQz4J6B9CjvRkaRZ+BFQa09A7Ls65gX4EhHEenrBfCxl/7oqpvScTjRSfjzLLyuuxfMxObGbtvrh1WbgGi/nkFQ7zshxN+sZrAyIUS5EKJCCFEeeKWBgYFB+4FouT6DNodgPYPHAEyQUq7emw8TKtLL83Hh/EcBAC8kKUfj/7ZRhc9oRp6W1ktVYk3+hdb3P/ZULV805UcAQOEqqgDq8tQDWs6x9O8eTxQVY37+WMtRluW+cCvtkSlRtNeeYPURPJ+/mR1noxxX/wQA+Ll4OD0/GxgTbY2iXJlL1z8zfZeWbWuaE+11YzmDGmZZ1xQoLymuEw3K4V5Mjdv2DOgNXVdDnoE9yMbDPIPIeO4ZKO9FRjpXRjnmDPjYyij1d/NLR2F5Ma5IX/I6AIiOdFnP0XTOwF9uwKlyMBRr2yl5GI4JRQNntFfPK9jNoKCtbQQGBgYG+wLtdC8IejNYKoR4D8CnADTXsZTyY/9LDAwMDNoXzNhLIBlANRQHhg0JxZuxz5CXX4EHH1SNZeuqngcAZBxN5Z6v/PKGlqPLcgAA1/yLZhjPnU5sF2PPVuGf+PQcrZu+mf7o44edBgAoa6CmtI3T39Nyx0EXAgDe+XW71o1JogTqQdmqwctdW6l1XeOoNLTkp58BAGsiu2vd2RkU5olPV91uK3eUaR2yKEzkilalqVUsETw8mcI0RSxkVVOk1iX0o3nMtSxMs9MqT+WNag18BrLFUMqbzqK9ktmK7kJGOieQbToKrqtp8E0gN3rNOKCr6wRyKPMMnJLK/hq5zKQzgybQXv/kQW0GUspL9vaDGBgYGIQD9hW7595GUJuBEKILgOcAHA7lESwCcJ2UcnuTC/cysjom4OZJhwIAZnY/CACQffq/9fHjvyLL9Ys+igLkGrZ++OoPtRwVpyz3A447RuueeIfoKtYdoyaQDWbTwf74kjWoPaSGEC39jVhorz44W8tJ+Wrqmd08BgC9OpLln7dEXaukF80j6DgoQ8vJGSqBnJ9PnkXDNk1kqJ+fk9t1Y01z9SxZXVOoyPJcqZRA5vOOyywvgHsG9cwziLI9A5ZAdllzkzmkH2qNerevZ1AfYAYy9wL0DGR/5Z4BSkv1vAWvpLLHZ43X+hAmnYVCR2F7FoEmpbVXa7S5aO3fi2ihsZdtEcFucv+DolfNgZrQ87mlMzAwMNivIERwX+GGYHMGmVJK/uE/TQhx/d54oFBQltYZX016WMmvqHTGokfH6+M9j/mnlt/eosaJTpryrtZ9c+kFWj74NkUn8eBEmjFs5xEA4JN6RWfx4Qm9tO7NT9ZqefLRynOY9Nl3Wtd3Is2bqFqsPBOek8jpk6Xl7T8qArvyePIsMgbT7IsOSSomX8xKS6s2b9FyVIKy8rmFnxpLf95EVmZbVai8D1cqEcPWB8wZsKlm1rU8DURRIRxI6fzlDOosy5xPOqth85Rdkeq53cwz4GWk9rN4UVjvUc6gac/CE8BaV+c2fTyQxe+EUCaN7eupZGH4mdcsCLTfMFGwr6tYCHG+EMJlfZ0PO0toYGBgsB9BCBHUV7ghWM/gUgDPA3gKKmfwg6Xbp9i+NR83Tn4IAFBtxf9/PXeCPt53LFUW/fb0twCAlw4ja/SaPIrPv3ORIrhL20xNZxy5y9V40YH/JXrxyg9u1PJRmcoyreRNZSfQuWvuux8AkNrjYq3L6k+Dbr79coN6HSVEcZ1+Zn8td6tROYHNf2zUuvKIPC3HdaDZyDbS4ui18nnH1cWKYM4TR3QUjaxpa5dFrR3F3tBO845lKSOSi/cdiOPx4xnU6hnG/nIG1vAcP8NtbC8g2ouigg+nCY6ozh+cKoS8ifICrW/ysEE4Q7Tfv2+w1URbAZwa8EQDAwODdgyB9jsJr8nNQAhxi5TyMSHEc1AegReklNfutScLAvEdUjH49HMAAMOe2wYAOGIOWc6/fDJOy6vWqJj+/BPJoeGVQfJlRWq37LtVWtdjNI3Q3PDdpwCA/P4naF1OLPUpuL+eBsC7wqYgjQbpbF2sCq9yrkrTupRDDtNybq1i7K5jozpjBo7V8tAiNU5zjkWIBwDltRSpi8+hcZt6fT15PnzEZfk2pff2DGhdaaV6DSkunjNwqCYqZhU0DjkDN+8TYPmBOgfPoM6hcoj3DjhRWPsdWxkkUV0o1r6/3ECwfQb+rMnWiib4v3/rfbK1l56McAwBBYNAOQObgmIp1HjK3b8MDAwM9huoDuTgvgJeS4gThRBrhRDrhRA+0x6FwrPW8T+EEAcGu3ZP0KRnIKX83BKrpZQf8GNCiH0+b6B/sgcLT1AD4FOu/QYA8MpY6uBdMvpoLY9eoKijr0smIrj7H6X8wgcPzgUAFLOqlpv/TefesVUNvH9sPnkeE3uQZb3qf+r6camjtG7uhlItlxQq8rZxw6maSPTpoWV7uAwfGNOYQ57FIIvuub6SPIfyErL8EwepnIDX4PuKAjrekSz3wj8UUZ0nlp6fo8TyDLKYOeyu1ywkiElWXgbvQHbKGdQzdyPCIWfgYj0XvAPZrhxqdLP1LD+gieoCeAaegMNtfB65Sb2BAdAyFVRCCBeAFwCMg5r++IsQYqaUchU7bTyAvtbXKAAvARgV5NqQEWw10e1B6gwMDAzaMQQiRHBfAXAIgPVSyo1SynoA7wKYuNs5EwG8IRV+AtDBGgkczNqQEShnMB7ASQA6CyGeZYeSAbidVxkYGBi0U4TWUJYhhFjKfp4qpZxqyZ0BbGPHtkNZ/xxO53QOcm3ICFRNlAuVLzgV3jmCCgA3OK5oRexYuw13Ha56395dpprKenamcMV9HaiB7N+v/QEAuDGbjosrH9Hyyls/x+64syuFGd48cgQA4Kuv/9K6O/5OTWWfPa4moKWdTTOY3/yRmsKGWI1c5/QjCojiGOcZwTZKIiiM0ydNPUt9FRHVlbPS2GRrdgFvFJOlrPS0IyWYd9arfby60fldXVqlQkKJkWyGQB2bd5ypQk48NBPhkECud5hrDAB1jaGXlnpTRzgkkNlc4shgE8h+AruBSks5QqGeCBZOjxVKo1NzwxjttXSyJSCkhAi+kbBYSuk00RFw/jPtXqTj75xg1oaMQDmD5QCWCyHellIaT8DAwGC/h5BN81gFie0AurKfu0AZ38GcEx3E2pARKEz0vpTybAC/CyH4ziMASCnl0OY+QHOQHOXC2C7Kes66+yIAwOGZV+jjn19AjVgHvaPopsd/QxTWx0//Tcs3WB5DOUtkbnuE6Krvm6yiZOPfIyK8rFsu0/Lqe2cDAHoN7qR1m1ZSArePZSUP60TkdAu2ELVEopUgtamoAWB9aa2Wh3RUem6NF9QSHUSvrCSv6wCAO2+zluOZZ2AnqysbnN/Uu6obfK7FE9tR1lQz/iwyOgTPwPICOGlfPZuB7LIS116lpVG8wUzpY/w1nTnRUYTQSKbXN4ZOJeEPxthuL5BAy2wGvwDoK4ToCWAHgEkAztvtnJkAJgsh3oUKA5VJKfOEEEVBrA0ZgcJEdqH9Kc29kYGBgUG7gGx2RAZSSrcQYjKAOQBcAF6XUv4phLjKOj4FwCyonO16qHkylzS1trnPFChMZAediwHUSCk9Qoh+AAYA+Kq5N28uYgcOwMB5iibi4YzBAIAdI7+mEz5/W4sJlysiunu3kOX+y8fTtHzc64rUruyPFVpnl5sCwFU3qXJMbiFvTCK6CJvO4dxDu2ndTTPnaDnOqleM3viT1v2wjYjqcixSudgUoq1eXkCew6FJVdgdpfVkofTtpOgqeHdUQyF5jvFZ6Vq25x1XsvW8nLLaym/EMqI7N3vd0cm2d0N02jLKl3rC2zOg53Kio6hmJb1OOYOYCHoWT4CmM5umwiun4eAFhFJC6o/UrrnzjptLembi+60M2WKeAaSUs6A+8LluCpMlvFn3m1zbXAT7XlwAIFYI0RnAPKgdalpLPoiBgYFBOEBIT1Bf4YZgieqElLJaCHEZgOcsiorf9+aDBYNVmwpw4AXPAAA2vKxCZlXxf9PHx91AdNX33aFoK+6+d5rWNdSQZbts4JkAgBFHnKN1vMIof+qTAIC0XsO0btpSmu3TL1FZxif2IQv86hKyzO0Rl+Xfz9a6BXUnaflGa0SmPd4SAJZsILqJ/0tVljnPKexk+Y3h6SpmX8KG2FRspZxF2oGU3rFHY+5iOQdehVRbZRHVJVBM32nEpfRQZZOMoudy9gx405k1XCaK5ww4XbXVNMZ0EczjkdL2DJxtmVBGXDrBsZqnnVIQGIQKCXjaZy1N0JuBEOIwAH8HYGdNg11rYGBg0D4g0WJhoraGYD/Qr4fqOP7ESnL0AjB/7z1WcIhwRSI2RdXtX5+ohtq83nurPp6yeaWWr4Cy2G+tpjj8AScSEeuVL6lY/tnj+mhdb2YZ/zb1R7Xmzou07vPvN2n5pREqF5FdTqMwuTU8yKKD2PINOVT53UZrOWu4GjSTnEV5hPVbyfKu77AGAI23BIAaFlPv2cGqNooma7lqR5GWM8dls3XKsi6udvYM6uzhNolRWudVTZToMMjGYcRlAxslySuH7EE2fvsMhE0nweksfPsM/FJYO1QDOY249GftN3fEpRNCIcUzTkhbhgQCjEgNVwRLYf09BKI+CwAAIABJREFUgO+FEElCiEQp5UYA+5Sx1MDAwGBfIBzzAcEgqM1ACDEEwBsA0tSPogjAhS1RztQcDO6RjgWvqdGVaYerpPvQ9Yv08Zs/nKnlTyaqXMKJj9D0zn+PJyK4g069BQDwaulxWjfrrAFanvqmqjL69ym05pSrXqBnuXgMAKBi7vtal9iph5a7HaCopzd+s1nrdkWQF5M9SnkkGYzwrZB5BpVxyuOISSGiuxoWk+9kWfGlHchCr8wjL8iVQevsEZf2eEsAiONVSNZYyRhG8c1HXEYk+dJlO+UMqllOw2nEpT3eEnAecclHbToNtwmJwtrB3HZ5XXPPKKyd7h+KZR8sHfK+pk1uC85Km/GY9ufNAMDLAP4lpZwPAEKIMQBeATC6qUUGBgYG7QpSAnsw1zocEOxmkGBvBAAgpfxOCOEbODYwMDBo59ivw0QANgoh7gbwpvXz+QA2NXF+q2DXitWY2f0gAMCER1X4p/yaBfr4bXXUNHZDkZp78O4p1HTW8L3u79DNXrm/UqNYv48f0HL9dMUQO0pQOWllwWYtx518DwBg+cVEUZE14Hotdx2uwiwzP1qjdVVFRDyYPkq9jsGVNAnt058pCrfLrc6NTz8YTsiMV3/KLekUrqksoEY1T3wqvRYrzFJYSTMKYh0SyDHJlPRtLKDSUhGf7HP/Rpdv01l1Q6AEMiPCY8lwm46ijpXw8WSxPU3OFYhOwk/TWVM6rg+U6FXnBjxlv0P7LsNtuaaztoZgm84uBZAJ4GPrKwNWa7SBgYHBfgXpCe4rzBCIqC4WwFUA+gBYAeBGKWVDU2taE3WNEpusBqnpMcqij3jrKn380XNoBMPfD1XNXD9N/LvW5W6gqWFjHnodADDvjQ+17tNyops+OFVZ3DumPK11rmginfu1RlnLf/1MjWbDJlGZaMLoE9U9a6fT81fQJLTIwReq++RTAnkGa1or3aUa0JJHOE8niyxTzCFJOVR6mr+sUMuNCek+a4rKyTPoFskTyMryjk0lL0PmMis7wWmqGSsNdfkmkCMcqCdcLCvb6EBhzZvOnJLFXt5CwAQyyQEnnTmYSKFYu07eQmsay873b8/WeiuiBeko2hoChYmmA2gAsBBqBNtAqJ4DAwMDg/0OAvtvzmCQlHIIAAghXgPw895/pOCRPbg3bvt8BgDg2p4qpt9pDsX8kyOnavnQme8AAP7Z8SjHa715nqKZGLNll9Y98M4yLX9wthqU8/Pr9CtIO+Y6Lb+wUM1GzmHW9nkHE2ldRbbyHOpZIxWPSVd26AkAGCYpNs89h52l6rk6HE/eCG8Ui9ipchmJ2eQ5lC2knESti6x8G4UV9KyDOPW1RdMR3ZHu5RV/T6L8g74+K3O1ZxvXup1zBrYXwecae7zWK72HsUN6NZhZzWROQ2wAPzkDp+E2AXIGHP4M671hcJtBNm0ZEmhBavO2hEDvOx0SaunhNkIIlxDidyHEF9bPaUKIr4UQf1nffT9xDAwMDPYlbDqK/S1nAGCYEMLuXBIA4qyf7eE2vmUlweM6AKuh5ikDwG0A5kkpHxFC3Gb9fGtTF/izoB5Dn1GNW/87vhcAYOz9VCG0c+7DWj7rMxV/PzeTrN1KZrnuemgyAOD+vz+kdedcTmMxe72qvIDnpl6odX1G9tPyTz8qK/xYVkEzpjv9en7OVdY2HxjDSefWlKhBNn3TqGnMrpoBgPxd6ngPNraTX6th6zoAQEJnynNwiuuKOl9rprCchuc4DbKJ6UD34s+CGN+q4jq3LyldDW86i+BNZ8qu8JcziIpxWfdkFNYO1BN+Kawdgv5O1r6/sZnNRSjGtrHMww/tNUzUpGcgpXRJKZOtryQpZSST93gjEEJ0AXAygFeZeiJUjgLW99P29PoGBgYGewdyv/UM9haeBnALAF6W0skepmONduvotFAIcQWAKwBARCdh049qmE35G2qsZcpdVA30f1t6aXnua6oP4f3PqHegesWvWn7lX2rdjZdSnJ0PgV+TQtTVNq48hkjtrvpcjdPkFnbs6m+1PDu3BwCgRzyRv8WlUs/DD1tVZdNBsc77c36tsqaHdiUqiEZ2r7ptqu0jqRtds5QNjCmrozennWuoqiJrPyHOd5BNTCpVJgFEbSEtz8CLaM5hkI1XNRHLGVQH6DOwB9lworpA1UROPQXeYy/heK4THEdk+rH3gx1kY4bYtCOE4Qd9MGjuezRkCCFOAVAopfw14MkOkFJOlVKOlFKORJRvUtTAwMBgr8GmowjmK8ywLzyDwwGcKoQ4CUAsgGQhxFsACoQQ2ZZXkA2gsMmrAOjdIwtPva6G1k+8RMX6f/rgPn384FMp5WBb4S95RmjdaWecoOUNV6qqpFV336t1XQ6mbuJ/z1Gdw6ekUkz/1P5Uu39xmaKL7pdIFnDBF0SUtzDqbHUd1geQlEOexfzV6uVeG896D2LpXHuQzZEdyZkqiCHLvHyT6jPoOOYIrati1nZxNRtOY5mZ9hAbwJuUzvaIeM5Aeqgnwyal8xpYw+5lewE8Z+A0yMYeYgM4D7LxsGtGR9K9nHIGHE79A84dyI7L23kHrUHzICHde7/VSgiRBuA9AD0AbAZwtpRy527ndIUiEM0C4AEwVUr5jHXsPgCXA7B57O+wRmX6Rat7BlLK26WUXaSUPQBMAvCtlPJ8ADMB2MMCLgLwWWs/m4GBgUGTkGgtz8AuqOkLNWr4Nodz3FCNwAMBHArgGiHEIHb8KSnlcOsr4LzkVt8MmsAjAMYJIf4CMM762cDAwKDNQEJCNjYG9dVMBCyokVLmSSl/s+QKqOrMzrufFyz26ehKKeV3AL6z5BIAx4ayPmr7JnS+Xc0z6HqQmmcQeccF+nj2CCoDvfCMwQCABx6heQMfHDFKy3f3VQRxX3y6TuuunEVzg599Vc1JeOjvlEiO+ukDLcckqfWDh1Le+6/P/9By7tCjAQDdjuyudZnZFGbaulF5gDWxv2idTZ4HUBlsnzSWJ4mlZHTZJjUvIWcSNbrxSWh5jJQuzoqj1LJJZ7Es/NVYr0pOI5OpTJXDE6PCV15hIlZSa+sra6k1xYmozqu0lBPV2U1nARLI/kpDI3TTmTOdhda1wEQzJ324TzXb14/UFn8nGhKhTDrLEEIsZT9PlVJO9Xu2N4IqqLEhhOgBYASAJUw9WQhxIYClUB7EToelGmaOsYGBgUHQCGmeQbGUcqS/g0KIb6Di/bvjzlCeSAiRCOAjANdLKe2yv5cAPAC1fT0A4AkowlG/COvNoLisDq/OVJb8yu1qKtnkzmSNLyodr+WsPGVxP1ZIpaO/f16g5bGvqUlnnxxFoblnh1MZ591r1Izknu/+S+vW3E9lqhkDVLqjTz+ytt9/YLaWyzqsAgB0vZwoqPvVEV31gjnquYvdG7QuPn2Mlu3KzWw2lzifJaPLt1cAAGQHem9x6osCL7pqiyKaTzpjnoGnVCWbIxJ9J5oBzgnkinrmBVjJ4hpW2uridBdWYpnTUdSzpji7ZNRfI1mg0lLH0lCms0nt/FnwzkR1cDw3WIRCFGdI5dowZMslkKWUx/k7JoQIqqBGCBEFtRG8LaX8mF27gJ3zCoAvAj1PW8oZGBgYGLRxSEhPY1BfzUTAghqhrIbXAKyWUj6527Fs9uPpAFYGumFYewadu6fhobvPBwA81W8CAODMQRTn3nHuBC3//oeqsBp924ta9+M7FPOf3/EYAMCBbIZw0dPcW1PW2sq4/lqz8iuy4g98tCsAIH30mVq37lYqLa3ZqTbqmEMna93R+WR5z3pDNY0VlezQug6DqYHMRnwVeTMpXViZ6Qr1+jxJtIb1gWk6CwDIivT1DLxyBtYgm4gUX9prgEjp+FxjPsjGJqqr5p6BA/UE1znRVXMKDMeYv5/a0CgHojqnclN/c40NXXUo99/PvBi7mmjv4xEA7wshLgOwFcBZACCEyAHwqpTyJKgy/QsArBBC2KyadgnpY0KI4dYTbwZwZaAbhvVmYGBgYNC6kKEkkPf8Ln4KaqSUuQBOsuRF8JPvl1Je4KRvCmG9GWyLSMV1cX8DAIyJ+QgAcMRSGnt5ffxAnzWzLj5Ay8ewaprrp6icwFdXUIXR18/StTpOuAMAcM+s1VrXh1nb1xylqC92dSISNye66tLUvlo3OpruX1tWDAAoKSrRuvRxlBOwK4AiCskbSelJ1UarFysK62oXEfFxbN9ZreX+Ucokbqgq07pYRn1tW+ROVNUAUGOR0rlYhVBlvW/lUGUd1/Gcgd105lxNpHMGrEIoxmGQjV86iiDpqv1ZtU7q5hrAfhvcglxvqKrbCGTLkhq2JYT1ZmBgYGDQugipmiisENabQWl+IWb893kAwJRtqpx35N3z9PHXjuup5QqLQnr52RTTf/MtyskMPEFVCXWeTRVCPz1Gg3COOEZVK9lVPwCQxeLcR6SrN8jHG6mUN5PRRUQlKMt7yY4KrTu2p681vnkXVf0M6UHVRilR6lr1G//UuuSelCMqqlNUTztrnePkecyLSYtW13LXVmpdbDqR0EqPFauPI51XT4H1urnOiZSuhnkLTvmBSEaOJ5kXFeNUTeREYe2nGsiRqM6hJyFQn0EwcLqCqcpox2jBaqK2hrDeDAwMDAxaF8YzMDAwMDBovWqiVkdYbwYdOmVi3HVXAQD6XqnKRAtWUtK323zqs8is3AwAuK7byVr38Of/1XJCpioNfX0jhStyYunXc+3xqqR0xDPUTd41jhrA3HPVvIS3y8Zo3UUpVK6ZnN0bADB7FZWGnhRDDXB2aKWAJV0P7kEJ3AortFOxjhLISf16a9lmNS2upvV8RnJNJZVpxnVQDKUNNRQmikunkJX0KGoLT6zz/CI7gRwRSa+/2oGhtNpP05mdLPafQHZZz+EnzOPQNMbhzFrqeKojnGYUBDu3oCVgkr1tFxLSi+akPSGsNwMDAwODVoXxDNomeohyvBalKB+6Flh8/5dcoo8fcRvRQYwZo0o/R7Omso9voqazU597GwDw1IxlWvfOCTQprX/pbwC8CddG9aSmsZX/UxPX1vQeoHV9jybSuIweKpm9ZGW+1tWCktE20R2fTva3bLLMN1jJ1p3ryJtIG3u8lm0iu+1srnEcS9pWVzA6Cut3YM86BoDoVE49oZ5RslnHXkRzVmko15WxMl1KINNriYzypaPwNwPZkY6CJ5CtBHCUi3sLbB6C07zjAKWlgUjp/OWanZLVTtjferOCRdj9XqSEbKgPfF4YIqw3AwMDA4PWRes0ne0LhPVmsH1TMW694HUAwM/5KwAAvar+0scT3vxayx+sUTHxF2YQ0dz14x/S8pTTVE4g9ZnXtG7o/+7Q8vqnnwUAdBpyjtYdQCF7fPqoKmkt9pBn0eeSw7Xcr0Y1iP08n5rWCqtXaDkpW5Wx8ka1bskUky/toryE0vXUlNa7Uw8t23TVOyrIM0hgljfPGSRajXGNpXSuK5W9GKipbp4YorvgZaRldcoL4NPLOF11ZLR67vp6Xm7qS0oX6adpzCalC6W0lCPKnpTmUG7qtd4vHYXvM4WCcCKlCzfDvE3AhIkMDAwM9nNI2RIkdG0SYb0ZZCTH4JLDlUW7ZawimlucTxUyk6w8AAB88uqHAIAPOp6kdUdlPK3lHbereccigmZILIoZrOX1HyoK6mNf7Kd1HUcRtcTqu9RUueqSXK2LOYq8kFPzFbXEN6zRLW/HZi2njjzb5/UlVObR8V4qpp/7M12/MYWGGtmkdBsLq7SuF4vT11axXEKGoqywCekAwJXqOzuj2k1eCielq7SH07CcQUWtb4OZm1UYOTWdxUW7mI7NaLa8gMYARHVRDhVGQPCkdM0lpANaL+bt//6t8wD7HSFdEzDVRAYGBgb7O6SEbDSbQZuDu1sv7HrmXQDA7EGjfI6/0nurlhOvmQQAuOXhT7Vu/bPnafnpy9S40V5XP6V1t75D8f+DLDqH247po3Ub2b0aJeOLtrA5uouWj7KmXdaWFWld/o5iLXc9W/UUJLLYOLZQTiGtXw4AYOk3m7SuzEM5BRs7GCHdSGZ5c1K6eKt/gVvjER18R1zWMM/Ai5TO6oVwRcf56AAg0qLOcDNaa1sHUE9BjEOFENcHyhk4EdIBrMKn0TlnYCOUUZahwOmxQqGoMKR0bRdSSnga3IFPDEOE9WZgYGBg0KqQMJ5BW8SGTXmYeImqCNr5rfpesfxXffy5427R8jMr3gcAvPLQcq3bdswTWs6tfRUAcOt5w7Vu8k00COcwy9zquWOx1t2dS30E/RJVV298eo7WfbmOvICr+/pau2sryDIfO1DF7BuZNV+z6jctpw5UrkVRHXVYF1Uzy9l6vvwS8gxSE8hzqK9mnkF2mvUs5VrniaMOZLtyiA+s4d3GFfX22EqeM+B9Br45g2hG2mf3FHBr3ylnwH9XURG+noHfnIHTIBynPgM/NriT3p+x3VxSOmPFhx/MZmBgYGCwn0NKCY+ZZ2BgYGBgYKqJ2iCiEpLQ9aAxAIBjflBNWZcfd60+Xu7+RMvfHH8pAGDkpY9r3UUvL9HyI8PU7OAjs4mi4bKdRB0xNlM1av31LIWOvsiksaJTR2YBADr2H6F1H/6wRctXVql0s007AXiT0p3VXSV1N8dTOKZo2Totdz1bTXTbyUIvW8voWe1JaJVsbkECm7rm5qR0mepe0kOzF5zCRFUOMwoAYGe1Cum4YlgCmTedWSWtbi86ClZG2miHiUjnlCzmCWAn6gl/5Y5RASad2fcKJVHsj7rCYD9DK1UTCSHSALwHoAfUDOOzpZQ7Hc7bDKACQCMAt5RyZCjrOcwcDgMDA4MgYVcTBfPVTNwGYJ6Usi+AedbP/jBWSjnc3gj2YD2AMPcMDsiKxZJbFTFcwgRl8a+YQyWS+e9co+UbTn8GADDn2sO0LnPsTVo+8vX7AQCbH7xT67KGTdDyQacpi/3Lp7/Xuh2jKFk9+OKxAICB9Vla99siosYoLF0EAEjuTPevaaTSzQHpysqu60IUEIUrqOmsx01qnnMlI3TbvIs8g2TLyq4qJ0K6hI7MMyijZrTITHsONBXHemLJM7C9gDLeSMa8AJuUzqadALypJyKtJHhtFSWVOfVEo1tdNy4qgGfg1Ujm0HTGG9kCTjrzWe6FQER1/uDU9OVMdGdoJ9pL35qndRLIEwGMseTpAL4DcOveXB/Wm4GBgYFBqyK00tIMIcRS9vNUKeVUv2d7o5OUMg8ApJR5QghfigD9RJgrhJAAXmbXD3a9RlhvBgUr1+Opfsp6v//zLwEAjz/5kT7+3wSinpjYfRoAYNNlZ2hdZMwgLX/sGgoAKHyTSk8nvXWvljsddAgAYPn9c7SusmCzlmPH3wcAOC+PfqXz3vhQy1vWqvh/x2OIdoLTJiSXq/xC5qAMWrOA6KrdaT0AEO0EAKzJo3nKAyxrvKaCPIDEbPIy3IXkRbjSaXayjSreYGYR0JXWMMueNZjZ1BM8D9BQ55sf8LjJS3Ginoj28hYClZY6EM35sTSDpZ7wV9bppG9Nq9b5/q05XKedmPB7A6HlDIp3C914QQjxDYAsh0N3Ouj84XApZa71Yf+1EGKNlHJBwFUOCOvNwMDAwKA1IdFy1URSyuP8HRNCFAghsi2rPhtAoZ9r5FrfC4UQnwA4BMACAEGt5wjrzSBKCGRazUxjZv9/e+cdXVd15eFv2+rFKpZ7wQ4uGIw7HYfeHAZSJhNgmZSBMIRAbCeBwUCyIKGNQ4CQBAMxJo1AAqGYMjC0xHQwJe4CGxtXhHu3JVln/jj3nXNk3Wc92dKTnry/td7y0b733XevJOucffbev30jACNv860sv3rRLW685cV7AZg4+HxnG32b30K75p63ADg12HO//VQv6zxzpV2Fhw1jQjmGd7fZ/flT+3vbri3r3XjxMjsefkU3ZysJVta1894AoMtw31DnH08vcuOqXQ1Xax9Xec9gbPR9qA4+s2ioj5/UvedX3lLa0GPcWh0WmFnPYHOQ7RQ+66Yd9lqhZxAWmGXndmxgKwg9g6g5SG4jEtYhWTG2MGawL9ITSeUo4s2xqPTEAYYx1FWnRY5iBvAt4Nbo3yf3PEFECoEOxpgt0fh04Gepvn9PMnoyUBRFSSsG6tJTZ3Ar8DcRuQhYBnwdQER6AtOMMeOAbsDj0RZiFvAXY8xze3v/3sjoyaB82BDOe+1VACYW2gyZa7Mnu+Mlvfx23bWVtg7h4EKfLz/5O/74QSddAcCgoqC94/0/ceOf1ZwNwJXdi5zt4QGj3Hjqa1ZA7vejfD5/uJr+KGouM26o3yJclx80r3nHxpnKRw1ztqpdz7rxishjyQ82ytev9dITpZEsdeiNFPX2noGp89LXdYW21iGsHajvGdj72hhITITZRBtdNpFfA9cEXkR+9D2Ma3Jv78Wu4sM4Qlx8IJkstROqS7IqDqUr3DPtZ5N7lZ1QAAzpqTMwxqwDTomxrwLGReNPgOFNef/eyOjJQFEUJa2Y+luS7YmMngzmfLqeAd+1EtYzJx4LwC9uedkdf2GFl6M+7sIpAKy6/yJn2/bAdW5c0tfm8Y8b7VfWM2/0K/OFx1k56lGXjXW2AXlezvrNN6xc9tpVrzhbcU8fc1j/jv0FOq6Pz+ef38PXAax+28YHul/g729TIBS3cK3NEgrjDFvWB3UGfWzmUO1On02U26MfntAz6AzUb2UZZg4lvID1QavMrJxcN97eSDZRTrTir63212wsm6h+tXEim6iugQ18TUGcBwA+myhZ7UBjFchxdQrJsnm0piDm81v7BloU027lKNJegSwifUTkFRFZICLzRGRCZC8XkRdE5OPo37J035uiKMpeieoMUnllGq0hR1EL/MgYMwQ4Gvi+iBzKPpRPK4qipBNjDLura1N6ZRpp3yaKquISlXFbRGQB0It9KJ+uq61h+7qVADxx6c8BOPXhc93xDteMd+OanXbL5uWhVzjb8u+e4MYXPPAIAAOO8IVqv57+JTde/4ktRqu42weVL/3MB2AvedJuKS2a7buj9fziWW7cMUrs6rZjhb/mCJ9muuw1a6/p6vsqV9f5QrC5K23vgS8E2y1bN/rU0pK+tkdy7Ry/dZTVzfdbgHfdaFudvUZWEBTesLNhgdm6cJso2BLatcP+omfn+l+fHUFvhsSWUNijIJSeSBSYNSWAHCc+F2abmhg5ipDYFNBW7muc7B60wKwt0363iVo1ZiAi/YCRwNukWD4tIpcAlwB0CPR0FEVRWhztdNb8iEgR8HdgojFmc6qroUh74z6Aw0eMMo/9cRIAQ8/6IQBb//W0O/eKbie68bl/sCv6S2/0x8/e5OWe7zrJrtIfW+bduzBYm11oJ54XNxQ42zmD/GT0zai38ewlvrvZ8ZN8D+SKaBVdM8vLWfQ4Zogbv/zMYgAO3RkUtQXfktnLNwJwYiBxvSvop9xpjJWY2P2e9wykwn9+GCzetKthp7K12wMvIM+mz67f5gvwQi8gkUaak+9tcQVmieKy0AZ+FR8Wl4VeRFbHGDmKmAKzZAHkVAvMmtK9LKl0RZJrpPpZqaKpqW0EA2Z3w37n7YFWmQxEJBs7ETxojHksMje5fFpRFCWdGEy6VEvTTtonA7EuwP3AAmPM7cGhJpdP76qsZOkJJwIwKkodHXufl3C4+XC/0zT2dFtoVXiH72E8LiggWzzpYgBu6naps009upcb/32YjS/c/NR8Zzt55FI3TjSt+WirX01/Y6R///KoEGvVS284W6JhDcCqnbYRz8KgkKwoSL3cuMamjHbq5cXnwgKzTv2sZ2DqvCz17mJfdBbnGXTMyXO2dduClXm+/b6EMYPsoIdxdeQZFJX694eeQX6O/bWqFzOIkaOoLz4XpJHGLINTbVgDqfc7bqmGNbqKb8cYMHXqGTQXxwEXAnNEJBFtvYZ9KJ9WFEVJJ8bA7motOmsWjDGvkXwLtUnl05t31vL8ItvJ7Z8n2T31osueccePfds3onl97MkADDvnemc77YjRbvw/X7cFaotGv+BsY34xwY2/un4QANPu89dfPMcrxXY99NsA7HjVy1Yf2dV/e7MOt6v0ZTOXOFvfa49040SB2fsrNzlbRY5/f8Iz6DzQl19UL9/sxtl9E01zvGdQnefPDeUkqqIVf8IDAFgdtMvMzrVezM6gEC0niBkkGujkBqv92mrvERXlNvQMcmJE6XKTtL1MxAfiGtaE52Yn0bBOrMxbumFNsnP3dk/7+v5kpPrulsoaOuCSkYzRmIGiKIoCdToZtD16De7NzfdbmeqfnHAlACOvu9cdH/OTl9z4+PdsC8lnXzre2eZvPsqNN9XYtpm7d/lsnMp+Xm78B0Pst+qX1y1wtg9XLPaf9Ys+AJRMD/JLZvnMpb4n2PacYdvMQ3L8nn6Cd5f4OMD5Bf7Hs31dFQBlh/g4xO5FQU1BDy99nWD9zrAVpfcMPo/iA9l53jNYs8V7BgkvIFFPAHtmE1kvpTjP28LvWyJzaHeSmIH3DOJzcVydQT2Jiqas1hva4pyI5HIUMbb4U+M/vwnnpvr5ShtBU0sVRVEUA9RpAFlRFOUAxxgNILdFKrdmcdKrtmfwT/tZOYYbvzfYHS88/Xo3vr6rVQjdcOWFznZRxcVu/NujbYHW08PPcLYrHvbSEs8Ptz2KEymkAO8HQdfLxtptmpUlPt1y5ZM+2Nz7q+cAsPgmH6CeXeUVRkuy7UbErOU+gNytry9q27HhMwDKDjnI2epm+GBxbWlPoH4h2cZgmyhMI/0s2hJKFNJB/TTSRDHZriCAXNjJq5Ym0kiL8nwBXFwaaVh0llevwMxeN66vMUBezPZRWGDmeyAnCSCnmEbaHMJcuqVzYGG06ExRFEXRCuQ2yvYN65n1yEMADHzDBmafH3asOz726rvd+NxjvgzA5GMnOtvCETPc+JiHbO/ky6t8odoNP/+zG8992XZU6zXycmfbGqTzyMdJAAASaklEQVSRHlNmV8FzgkK1xc9VunHvn54I1O9R8Oon69y4Z7TKXrfai891PbTCjXd9bFNoc75wMh7vGezItWmkYQrpskBuIyfwAlZEfRBy8/1qf8sWnxqaG3Vg2xp4PgVBsDiRRloUBJVDz6AoOjdc7celkYYeQF1dw2CxibGFhKvyekVnMUv+pvQd2N8eBS0hPte0vszN764ccCmkSWm/FcitIWGtKIqSmUQVyKm89odU+ruIyGAR+TB4bRaRidGx60VkZXBsXMNPqU9GewY9e3dnwu1W5XrMeFs0Nr7K9yB+4Wy/Sv7lcivXUBGkOIb7/7/faPfcv3eY31u/smqpG/9zkY0ZfONyH5PI+Z3fM695fjoAA77s01Xvm/A3N+5bYwXuwgXu65VeaO7HJXaVvm3NMmerOM6ni+6eZ2UqpJeXuA4lJtZsj2Sl83z3tJWbg0KyAu8ZfLbJegY5QQ/mndt9fKAgks5Yt9rbSgOBvEQaaXE9b8GnlhbGyFHkxhSdxYnPQbwAXVx8IFnMIM66v+JzSYvG4s0pozGHzMKQtjqDRH+XW0Xk6ujrepL+xphKYASAiHQEVgKPB6fcYYy5LdUPzOjJQFEUJa0YQ116soma2t/lFGCxMebTff3AjJ4Myjet5uvP2qY2d5YdB8CPrzrRHf/jET5z6NajbKObxXd4yaOZdWe78Q13vwbA6X18D+OKQUe48Ucf2t7KPz/GN4yZM8B7bnOn2wK3MdPudLal233M4R9L7J5/t2Cf/eVPN7px96jRzY51vldxp2HD3Ng8aAX2asv7OVvHoJBsZSQREWYILV/vRe9yCr3A3edRLCCvMPAMtnkvoLyL9S5qdoVxgMAziLyAosAzqJc5lNWwh3FcfCA3bnOf+JhBS/Qw3l+JiZaitZ0FjQ8kx5i0eQYp9XcJOA94aA/b5SLyTWAWtrvkhr1dQGMGiqIoTcDU1aX0AipEZFbwuiS8joi8KCJzY17nxn9yPCKSA5wDPBKYpwIHY7eRVgO/bOw6Ge0ZrK7awq1TbBbRoh1WhuL15V9xx+fe5vfvt0cr7vmn/8zZ/lLhV9Zd750GwItPvulsp//6e27c5Um7P1/8lp98Dxvvr//ITf8HQI/ig50tXEA8+YFtz/mfQb7+xhU+PtB9jI0PVM/wdQbZg7yQHljPoGqHv2h2IDS3dGOUIVTk4yCfrvN1DPlF/nN3RDUFeUEcYHMgnd05ihnU7vDxl9IgvpCID5QEtnAVnmhxGcYMCoJGQa45TRBAqZ8NtPcWl87WBDmKlpKYSDVzqKViA9q2Ms0Y0xTPYK0xZkzyS5lTkx0Tkab0dzkLeN8YUxVc241F5HfA03FvDFHPQFEUJVWiOoNUXvtJor8LNN7f5Xz22CKKJpAEXwHmNvaBGe0ZdO9WxH+PHwvAo71HAjDpmG+545V3fM2N53SwdQYX3Piis73e52U37jxgFABvfehtU77kM4dmH2xX3LNvf9DZRt/je/MsuPopALI/DmsH/Lf3n5VrAeg3uruzbVuz3H/+UdYLME+85Ww1XX3mUKIV5bJNvh4gp9jHLD7+3K7ic4tLnW1FsNoP4wMJz6Ck3McXdu3w1y0psJ7B7iBDqCTwIhLxgXC1n6gqBh8fCGsH4uIDeTEZRuCzhBptWNOE5jZNqR1IZ4ZPa6/r1bFoGoa0CdXF9ncRkZ7ANGPMuOjrAuA04L/2eP8UERkR3fLSmOMNyOjJQFEUJa0Yw+7qlp8MjDHriOnvYoxZBYwLvt4OdI4578I9bY2hk4GiKEqKGAN1RuUo2hwbS3vy+LnXA1A99UwAarb57l9PHn2FGz81wBaTFf3ufmf7619fc+NLHp0EQM4zv3K2wmfvcOMjfmgF7MJCspKChj0EHn7Tp/n+uKLAjX+z1PZm7nvyUGfb9ZDvXZA19JsASId3nW3FNr8CySnsBEDlWh/Uzevk+yF8XGVlLAo6+aK5rYEcRSg0t+oTm2HWNbDNC4LFnQujAHK4TZTfMLU0v942UcNgcVhIFrclFFdcZu1xBWYNz0sWQE41WNyUQrKmdCpriW0mDRS3HXbrZKAoinJgY6ifJdieyOjJYOXyz5g88VYAtlU+AUDXz3xx1aSr7nHjIVFq5uDTrnK2j4Jg8a9H2FXyvLP8av/V6/7uxid9YIvKFl/sC8nuf9sHgIcU2/c/M3e1sx18hr/W1nlLASg7wWeTmQef8MdL+wP1+xLPX+NTQ/NKrBfwYSBxXVDqg8WronNDD2DLer+y79Wrkxsv2Wa9gC7F3ouo3u6vWx6llobdy0rqidLZYHFxIO0RBosLshNFZ6FQXYwEdbBcb0xormOM3LUKzTXh89WxaDbUM1AURTnAqTNQrZ3O2h4FpeUc/rXzADj8l7Yf8RuD3nDHryr0K+c/P2P38mf87YvOtvBvPrVy/hWXATDslp862/Qh4914zSKbptkl16+Gn3h1qRtPO9IK3W345F/OdtAPT3Pj2vfeA6DuEP/5HXOe95+/NtqHL+vmbB+u9Kv1/M5WGntBYCsu90Vzm9fZ93fp7T2AqmVe7qJHqT+3Zpu9RhgzCL2Azi611MccQrnqRG/juEKy0B6u9rNiVvZZSTbXU00tTaJmkSQNteF5zbFYbqnr7g/qBbQsuk2kKIpygGMwuk3UFhncqZZ/nmRXv2VX2sygm37jC/UenOsLuFa/8QcA5Be+Oc3J03/kxjecdQMAa24a5Gw5wbLvrqfmAzBliG8485sP33Hjw759EgA7blvqbB2PvsyNO2TNAeCjrf6aoRfw+qc2w6ewixfCe2uxL2AriTKT1gcS3aVdvFz1soVWDnvMCF94OH+Tl8juUXK4G9fstNeoKPQtMsMCs065DSWoi3IaegFFOf7Xp14jmpgsoVQzhJLZmyInkepqvbUzhOx1dRmfSWgAWVEURQF0MmiTrKxcwU9OuBKAR2dbgbmq455zx4f83st/n/jIZMB7AACnftf3faiuux6AyX/5wNmmDPV5/NPftF7GmElnOdu2O7zQXM4ZVgCvw503Oltljc9sKuhsYwovLFrrbMU9B7jxKwusDlVZdx/nqFrpaybKukZyFAv9an/U4V7aovLNeQAc1NnXMYQZQj1LfOZQQoAuERuA+vGBkkhGY3cgS12S21CULpmcRE6MBHVMj/t63kKjQnVxdQZJluupegHNIUeR6qkt5QGoY5FejNFsIkVRlAMeg2YTtUk65WZxaj8r1lZ+pc38OePdP7njEwf6RjYjvmY9gpwOXsL6irt9TOGBU/oBcN8rLznbMTd92423Xmsrg3O+5iuUO/72ajd+e7PN1inq3s/ZHp3taw7K+h0GwDMf+OY1Xfp4SZGVS23so3tfn+FU+f4KNz5yuI0FLHx9trMN7Oab3zyxxVYzH1Tuq57DauyugYR1zU5bk1ARegaBF1AeVRuHMYP87KA5TWTPzYpf2cd5BjkdG3oRWUmCBnFZRnFeQEvJSseZ01knEF8nsV+XVJoJjRkoiqIogG4TKYqiHPDYmEFr30XLkNGTQe7gwfR/4R8A3NHNpk4uPM2nSE7o6lMvL77pUQDm3OADwLf8dYYbj37Cbv/s+vJdzrbqyOvcOL/MSk/8aZ4Xl0v0QACY+tonAHQf7AO4zwVyFT2jfggrgn4Ho4/o5cYzn7fbP/92Qn9ne/95X0A3rLftqvbQhs+c7Qud/fNVb7Wpqb0CobqaQHyuR7BNlOhHUJofSkw03BKq170sq+E+RV6STmU5Mds/2bGdypIVnaVoS1a0FmPf3+KwZFs/LbElpLRt1DNQFEU5wDFAWlrbtAIZPRksWFLFkRdameml91sJ6Iq7HnDHn5v9rBtvHnczAGsvnOJshS/f6ca/WmoDr92Hn+RsEx7zneL6H308AFOfWuBsA4/wncjeect6ASd80a/s//dxH6C+9OKTAbhr5tvOdvKFI9342T8uAWB0H9e3gp0bXBtTBldYL6A6CAofXOaDxbWRnEQ9DyBY7ZfmNfQCirLjU0MLsxp6BnFppHEeQDJ7XFA4zluAZD2Qmz8AnM7VvnoA7QOD0WwiRVGUAx2bTaSTQZujQ1Y2BZGA26UdbZrloFP8avr4qZVufMR/fAOAr97sZavPvMDHD26/5x8AfHv8sc527z1Pu/HPJts01Wt+6j2Pe2/zbUUveszGH86fNNbZHvqV9yJOH2Q//5bPfRzhqD6+wGznJluMdmgQ56je5ovG+pfaWEAoG9GlMBCPi+zl+fENZ0pyG67s60lMxEhQh8TFDHLjKslIEjNI0VuAZDGDuEK01Ff2cbZkq/WmrOx1xX+A0Y4DyMnkXVoNETlTRCpFZJGIXN34OxRFUdJDwjNI5ZVptCnPQEQ6Ar8FTgNWAO+KyAxjzPy48w8/qJzXp10AQKdjvw/A5jd+644nbKE9tP1rSnDuHVMBuO7EC5xtyjV+Zf+d4VZUbsIav7I/Z2CZG++Kir6O7+2b09Tu9Nk8I7raorRwZT+w1Bd9JVbxBxU3lH0A6BF5AaGta+AFJCjLjZ/fO+U0tBdnxy9rCxrJHEqQ5KOIu2ycLeZjkto70PA/V5ytKedKkv+wcfb9PfdAf386PyvZ+5uL9uoZtKnJADgSWGSM+QRARB4GzgViJwNFUZR0Ukf7laMQ04bcGRH5d+BMY8zF0dcXAkcZYy4PzrkEuCT6cigwt8GFMp8KYG2jZ2UW7fGZQJ8rkxhsjClu/LTkiMhz2O9NKqw1xpy5P5+XTtqaZxC3cVBvtjLG3AfcByAis4wxY9JxY+mkPT5Xe3wm0OfKJERk1v5eI5P+uDeVthZAXgH0Cb7uDaxKcq6iKIrSTLS1yeBdYKCI9BeRHOA8YEYj71EURVH2kza1TWSMqRWRy4HngY7AdGPMvL285b703FnaaY/P1R6fCfS5Mon2+EzNRpsKICuKoiitQ1vbJlIURVFaAZ0MFEVRlMydDNqDbIWI9BGRV0RkgYjME5EJkb1cRF4QkY+jf8sau1ZbREQ6isgHIvJ09HVGP5eIlIrIoyKyMPqZHZPpzwQgIpOi37+5IvKQiORl4nOJyHQR+VxE5ga2pM8hIpOjvx+VInJG69x12yEjJ4NAtuIs4FDgfBE5tHXvap+oBX5kjBkCHA18P3qOq4GXjDEDgZeirzORCcCC4OtMf65fAc8ZYw4BhmOfLaOfSUR6AT8AxhhjhmITN84jM5/r98CedQCxzxH9PzsPOCx6z93R35UDloycDAhkK4wx1UBCtiKjMMasNsa8H423YP+49MI+yx+i0/4AfLl17nDfEZHewJeAaYE5Y59LRDoBXwTuBzDGVBtjNpLBzxSQBeSLSBZQgK3tybjnMsbMBNbvYU72HOcCDxtjdhljlgCLsH9XDlgydTLoBSwPvl4R2TIWEekHjATeBroZY1aDnTCArq13Z/vMncBV1G8MlcnP9QVgDfBAtPU1TUQKyexnwhizErgNWAasBjYZY/6PDH+ugGTP0e7+huwvmToZNCpbkUmISBHwd2CiMWZzY+e3dUTkbOBzY8x7rX0vzUgWMAqYaowZCWwjM7ZO9kq0h34u0B/oCRSKyPjWvau00K7+hjQHmToZtBvZChHJxk4EDxpjHovMVSLSIzreA/i8te5vHzkOOEdElmK38E4WkT+T2c+1AlhhjEn0LX0UOzlk8jMBnAosMcasMcbUAI8Bx5L5z5Ug2XO0m78hzUWmTgbtQrZCRAS7B73AGHN7cGgG8K1o/C3gyXTf2/5gjJlsjOltjOmH/dm8bIwZTwY/lzHmM2C5iAyOTKdgpdUz9pkilgFHi0hB9Pt4CjZ2lenPlSDZc8wAzhORXBHpDwwE3mmF+2s7GGMy8gWMAz4CFgPXtvb97OMzHI91TWcDH0avcUBnbObDx9G/5a19r/vxjCcCT0fjjH4uYAQwK/p5PQGUZfozRc91A7AQKwf/JyA3E58LeAgb96jBrvwv2ttzANdGfz8qgbNa+/5b+6VyFIqiKErGbhMpiqIozYhOBoqiKIpOBoqiKIpOBoqiKAo6GSiKoijoZKC0Y0TkehH5cWvfh6JkAjoZKIqiKDoZKO0LEbk20qd/ERjc6BsURQGs+JaitAtEZDRW/mIk9nf7faA9ieUpSouhk4HSnhgLPG6M2Q4gIhmnV6UorYVuEyntDdVXUZR9QCcDpT0xE/iKiOSLSDHwb619Q4qSKeg2kdJuMMa8LyJ/xaq/fgq82sq3pCgZg6qWKoqiKLpNpCiKouhkoCiKoqCTgaIoioJOBoqiKAo6GSiKoijoZKAoiqKgk4GiKIoC/D81CPTumjnuAAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "EMBEDDING_DIM = 100\n",
    "MAX_SEQUENCE_LENGTH = 100\n",
    "MAX_NB_WORDS = 64\n",
    "pos_encoding = positional_encoding(MAX_SEQUENCE_LENGTH, EMBEDDING_DIM)\n",
    "\n",
    "plt.pcolormesh(pos_encoding[0], cmap='RdBu')\n",
    "plt.xlabel('d')\n",
    "plt.xlim((0, EMBEDDING_DIM))\n",
    "plt.ylabel('Position')\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You have already created this visualization in this assignment, but let us dive a little deeper. Notice some interesting properties of the matrix - the first is that the norm of each of the vectors is always a constant. No matter what the value of `pos` is, the norm will always be the same value, which in this case is 7.071068. From this property you can conclude that the dot product of two positional encoding vectors is not affected by the scale of the vector, which has important implications for correlation calculations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tf.Tensor: shape=(), dtype=float32, numpy=7.071068>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pos = 34\n",
    "tf.norm(pos_encoding[0,pos,:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another interesting property is that the norm of the difference between 2 vectors separated by `k` positions is also constant. If you keep `k` constant and change `pos`, the difference will be of approximately the same value. This property is important because it demonstrates that the difference does not depend on the positions of each encoding, but rather the relative seperation between encodings. Being able to express positional encodings as linear functions of one another can help the model to learn by focusing on the relative positions of words.\n",
    "\n",
    "This reflection of the difference in the positions of words with vector encodings is difficult to achieve, especially given that the values of the vector encodings must remain small enough so that they do not distort the word embeddings. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(3.2668781, shape=(), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "pos = 70\n",
    "k = 2\n",
    "print(tf.norm(pos_encoding[0,pos,:] -  pos_encoding[0,pos + k,:]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You have observed some interesting properties about the positional encoding vectors - next, you will create some visualizations to see how these properties affect the relationships between encodings and embeddings!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='1-2'></a>\n",
    "### 1.2 - Comparing positional encodings\n",
    "<div style=\"text-align: right;\"><a href=\"#toc\" style=\"text-decoration:none;\">↑ Table of Contents</a></div>\n",
    "\n",
    "#### 1.2.1 - Correlation\n",
    "\n",
    "The positional encoding matrix help to visualize how each vector is unique for every position. However, it is still not clear how these vectors can represent the relative position of the words in a sentence. To illustrate this, you will calculate the correlation between pairs of vectors at every single position. A successful positional encoder will produce a perfectly symmetric matrix in which maximum values are located at the main diagonal - vectors in similar positions should have the highest correlation. Following the same logic, the correlation values should get smaller as they move away from the main diagonal.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXEAAAEKCAYAAADkYmWmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2dfbBkZX3nP9/uO68MCANCRtCAAV+IUcxSCYZswgaTKLJgtmJCKhhi2CVbpRETtxRN7cbEsoqkjC95szImKr5EJcYXwkZXapQYNysGFRFEAwFEYMIw4MsMzNu9/ds/zjn3nJnue8+508/p22f6+6k6dbtPn/P083T3/fW3f+f7/B5FBMYYY7pJb7U7YIwx5vBxEDfGmA7jIG6MMR3GQdwYYzqMg7gxxnQYB3FjjOkwrQVxSe+StEPSbZV9myXdIOnO/O9xlcdeJ+kuSd+U9PNt9csYY6YFSfdK+pqkWyTdnO9bMk6Ook0l/h7gBYfsuwrYFhFnANvy+0g6E7gE+OH8nL+Q1G+xb8YYMy38p4g4KyLOzu+PjJNL0VoQj4jPAY8esvti4Jr89jXAiyv7PxQR+yLiHuAu4Mfa6psxxkwxS8XJkcy13p2DOSkitgNExHZJJ+b7Twa+UDnu/nzfEJKuAK4AWK/ef3jyuvXZA/nM0xhUZ6CqvNXPbvf61X3ld5h6ytsf3fHB/ACA+X0Li/v25PsA9g2GZ76u75WNbViTPVd/XfmSV59qsDAY6n9vruxfcV5/TXl+dbbt/J79AOx/7MDivr2VtqrPtS7v19y68sdO9bkWz6n0v/q4+v388XLfYL58XRb2Zn2Y3ztfPl4dV95uf305lrnK69Kby/vVW+rNiOHnPFDeHizeLs/vryv7umZj9pnprV1btll5qsG+7LWc37Ov0mb5Xldfi7mN67J968q2pMrrciB7LQb7y/el+h5rrvhcVM7vV36E5ofGQvlaxqDsS/EeqF/5V66eXwwsynOoztIu+lo5v/q+Lv5DLDWzOz8/ev2hfdWnP+SksiuV8xaK97XyVKNmlC9U9j22v3zfH92VvW97d+9e3Dc4UL6HseeRnRHxxNEDaUbvmFOC+b2Njo09j9wOVA/eGhFbDz0M+LSkAP4yf3ypODmSSQfxpRj1Vo/81OSD3ArwtA1HxV889Uyg/Cc+sLv8Z1ElYK/dtAaA9ceuX9y37gnrysePyh7vry0/VIOFsgt7v5O9F4/eWf64+OrDjy/evvfx8nkLnrax/Mf84S2bADjuqccu7utVgtSevP1q4Ft/XNnXzacfD8Cmk8vP4MLe/Yu3H7njgawfX3hgcd9dldei8lJwWj7WE04r+7LxhI1lv4ovvMprsfH4o8p+Hf+ErM315fj2PvK9xduPfvMhAHZ+s3yt9jxW9nXDUdl5m88oU32bTz+hfPyJ2f65DZUgW6H4wtrz8HcW9+3eXj7/rgd35+Mog8kTfvCYxdsnPvd0ADad9uTFfdXAufue+7L+3/pvi/see+ixxdvV1+qEZz8VgKNPP7Vsa235vu1/aHvWp/seWtx34PHy/3rD8Vm/jv7BJy3u6z3h+HKwg+xzvfC9RxZ3zVfOL96DuePKz0Xv6EoKNQ/Isa88J+Yr/yN5X/uV5+wddXTl/P5B/TiUWLMh+7u+PGewrvysHBTQF9ssw87Cuk2Lt7+7N3uO3fvLL5yFEUH8u5X/kZvu/+7i7Q/94z0A/OvnSz24a3v5Hh645d3fGjmIlTC/l7mnX9To0AO3vHtvJUWyFOdGxIN5oL5B0jdW2qVJu1MekrQFIP+7I99/P/DkynGnAA9OuG/GGLM8Eur1G21NiIgH8787gI+RpZGXipMjmbQSvw64DLg6//uJyv6/kfQW4EnAGcAXa1uLGFLgo9Q3lAp8lPqGUoGPUt9QKvBa9b1pWH1DqcBHqW8oFfgo9Q2lAh+lvqFU4HXqG0oFPkp9Q6nAR6lvKNXfKPUNpQIfpb6hVOCj1DeMVuCF+oZSgY9S39lYMm0ySn1DqcBHqW8oFXid+oZSgY9S31Aq8FHqG0oFPkp9Q6nAR6lvKBX4KPUNpQIfpb6hVOAj1fchfVlsM1ffUCrwWvUNiwp8lPqGUoGPUt/Zsdn/yCj1DaUCr6rvo7f80OLtR28Z3a2VIXpzo38hrrgl6SigFxG78ts/B/wBS8fJkbQWxCV9EDgPOEHS/cDv5Z26VtLlwH3ASwAi4nZJ1wJfB+aBl0fE6N9vxhizWuRKPBEnAR9Tdt1hDvibiPiUpH9hRJxcitaCeET8yhIPnb/E8W8C3rSi5xjEkAIfpb6hVOCj1DeUCnyU+oZSgY9S31Aq8FHqG0oFPkp9Q6nAR6lvKBX4KPUNpQKvU99QqspR6htKBT5KfUOpwEepbygV+Cj1nY0xU+CHo76hVOCj1DeUCnyU+oZSgY9S31Aq8Dr1DaWqHaW+oVTgo9Q3VBT4CPUNpQIfpb6hosBHqG8oFfgo9Q0VBX4Y6hsqCrxGfUOpwEepb2ie/x6lvqFU4FX1/bSfPGfx9hc+ObqLK0EccuF5DCLibuA5I/Y/whJxchTTcmHTGGOmH4leOiWehI4HcQ0p8Im6T0bkv1fbfVKnvsHuk6bukzr1DXafTJP7pFDgVfV9yU+ftni76mEeh4TplCR0PIgbY8wESZsTT4KDuDHGNESI3tya+gMnSKeDuPoaSqPMuoWwLoUCthA2tRDWpVDAFsKyzeVTKNC+hbBIo1RTKD9+Svn/kAQrcWOM6TYO4gnp9TWkwGfdQlinvsEWwqYWwjr1DbYQNp3AA+1bCAsFXlXfx65PHOKkZBbDVHQ6iBtjzCQRVuJJUb83pMBn3UJYp77BFsKmFsI69Q22EKaawAPjWwgLBZ5cfVdRj36iafep6HQQN8aYiSIr8aSopyEFPuvukzr1DXafpCofC3afNJ3Akx3brvtklALvL7VAwGEi7E4xxphO4yCeEGlYgc+6+6ROfYPdJ6nKx4LdJ0293zA590lVfW9am3jJBPvEjTGmyziIJ+dQBT7r7pM69Q12nyQrHwt2n0yR+6RQ4FX1fez6tAFXEr01dqcYY0w3cTrFGGO6jYN4Qgbzg6E0yqxbCOtSKGALYaoa4GALYXnc8ikUaN9CWKRRqimU/r7dQ+eMSzVGTAOdDuLGGDNJJCEH8XTM71sYUuCzbiGsU99gC2Gq8rFgC2HTCTzQvoWwUOAHqe9B+T+cin4/sW1xTDodxI0xZqIIK/GU7JkfLKvAZ9FCWKe+wRbCVOVjwRbCphN4oH0L4aICb0F9F2RVDB3EjTGmo4he4nos49LpIL5vEEMKfNbdJ3XqG+w+SVU+Fuw+aTqBBybgPhmlwGMwvG8cnE4xxphu4yDeEnafNPN+Z2O0+wTG936D3SdNvd8wQfdJRX339j02/PgYSNCfSxfEJfWBm4EHIuJCSW8A/hvwcH7I6yPiH5Zr44gJ4sYYMwmUNid+JXAHcExl31sj4s1NG+h0EF/fE0/beLACn3X3SZ36BrtPkpWPBbtPGnq/YQLuk1yBV9W39u4afexhIinZjE1JpwAvAt4E/M7htjNdrnVjjJly1FOjrQFvA14DHHr19RWSbpX0LknHjTjvIBzEjTFmBawgiJ8g6ebKdsViG9KFwI6I+NIhzb8D+CHgLGA78Md1/el0OmXDmt5QGmXWLYR1KRSwhTBVDXCwhbDpBB5o30JYpFGqKRQd2DN8zjiIlfjEd0bE2Us8di5wkaQLgPXAMZLeHxGXLj6V9E7g+ronsRI3xpiGCNGb6zXaliMiXhcRp0TEqcAlwGci4lJJWyqH/QJwW12fOq3E++vmhhT4rFsIx1XfYAth0wk8YAthqgk82bHjWQgLBZ5cfVdR66Vo/0jSWUAA9wK/WXdCp4O4McZMmsQWQyLiRuDG/PZLV3p+p4O4GFbgs24hrFPfYAthqvKx1b7OuoWwTn1D+xbCkQo88So8WQGspE2Ozap0R9JvS7pd0m2SPihpvaTNkm6QdGf+t9ZaY4wxEyVPpzTZJsXElbikk4FXAmdGxB5J15Il9s8EtkXE1ZKuAq4CXrtcW4OFwZACn3X3SZ36BrtPUpWPBbtPmk7ggQm6T2pe//HQQb9Qp4HV6s0csEHSHLAReBC4GLgmf/wa4MWr1DdjjBmJrMQhIh6Q9GbgPmAP8OmI+LSkkyJie37Mdkknjjo/N8xfAXDSmrVDCtzukxK7T9otHwt2n0yV+2TE9YfBY2mn3cP0VTGcuBLPc90XA6cBTwKOknTp8meVRMTWiDg7Is5+wlynr8saYzqGBP2eGm2TYjWi4POBeyLiYQBJHwV+AnhI0pZchW8BdqxC34wxZlkmGaCbsBpB/D7gHEkbydIp55PV030MuAy4Ov/7ibqGenO9oTTKrFsI61IoYAthshrgYAthTl0KBSZgIczfg2oKpfq5SIGYrMpuwmrkxG+S9BHgy8A88BVgK7AJuFbS5WSB/iWT7psxxiyHBGtrptRPmlVJKkfE7wG/d8jufWSqvDH9dXNDCnzWLYR16jsbiy2EkGACT6UvVWbRQlirvqF1C2GhwKufi9hfvscpkGBu1pW4McZ0FeGceFL6a+aGFPisWwjr1DfYQjip8rEwOxbCcdU3jG8hLD4XVfWtufJ/PAlyTtwYYzpLpsSdE09GRAwp8Fl3n9Spb7D7JFX5WLD7pPEEHmjdfVIo8Kr61rryPU6FlbgxxnSUnmR3Skrm9+wfUuCz7j6pU99g90mq8rFg90lT7ze07z4pFPhB6ntw6BrE49NPXE98XDodxI0xZpIU0+6niU4H8f2PHRhS4LPuPqlT32D3id0nR6b7ZFGBV9T3YFf5yzUVDuLGGNNRPNnHGGM6jPCFzaTsHcRQGmXWLYR1KRSwhTBZDXCwhbCgJoUCE7AQ5mmUagpl/jsPDz/nGDgnbowxHcbT7hMjhhX4rFsI69Q32EI4qfKxMDsWwnHVN4xvISwUeFV9VwviJcFK3BhjuovriSdmXU9DCnzWLYR16htsIUxWPhZsIWw6gQdatxAWCryqvuc2etq9McaYnJ4XhUjL3Lr+kAKfdfdJnfoGu08W2xlzAg/YfdJ4Ag+07j4pFHhVfVc/F0lInBOX1CdbnvKBiLhQ0mbgw8CpwL3AL0XEsjOWpusrxRhjphgh+mq2NeRK4I7K/auAbRFxBrAtv78snVbivbnekAKfdfdJnfoGu0/sPjky3SeFAj9IfVf7koheogJYkk4BXgS8CfidfPfFwHn57WuAG4HXLtdOp4O4McZMkqqtuQEnSLq5cn9rRGyt3H8b8BqgokA4KSK2A0TEdkkn1j1J54P4oQp81t0ndeob7D5JVT4W7D4pD1xefUP77pNFBV7py6Ci+pMg6DXPie+MiLNHNiNdCOyIiC9JOm+cLnU+iBtjzKQQsCbN8mznAhdJugBYDxwj6f3AQ5K25Cp8C7CjriFf2DTGmIYU6ZQm23JExOsi4pSIOBW4BPhMRFwKXAdclh92GfCJuj51Womrp6E0yqxbCOtSKGALYaoa4GALYdMJPDABC2Hel2oKZde3Hhzd78NFWkk65XC4GrhW0uXAfcBL6k7odBA3xphJItK5Uwoi4kYyFwoR8Qhw/krO73QQ7831hhT4rFsI69Q32EI4qfKxMDsWwjr1De1bCAsFXlXfex75/nD/x2QF7pSJ0Okgbowxk0SCNf3pupTY6SCufn9Igc+6hbBOfYMthGWb403gyY61hRDq1Te0byEsFHhVfa9JXACrjXTKuHQ6iBtjzKRxOiUh6vWGFPisu0/q1DfYfZKqfCzYfdJ4Ag+07j4pFHhVfR/9lJNGj+swEbISN8aYzuKVfdIymF8YUuCz7j6pU99g94ndJ5NfvBjad58UCryqvteetGV4fGOQ5cSTNjk2nQ7ixhgzSRJOu09Gp4P4wt4DQwp81t0ndeob7D5JVT4W7D5p6v2G9t0nhQKvqu9qX5MgmDKH4erUTpF0rKSPSPqGpDskPU/SZkk3SLoz/3tcfUvGGDM5Cothk21SrNZ3ytuBT0XEM4DnkK1sseIVLYwxZrIkX9lnbCaeTpF0DPBTwK8DRMR+YL+kFa9oMb93fiiNMusWwroUCthCmKoGONhC2HQCD7RvISzSKNW+7rrrXlLiyT4ZTwUeBt4t6TnAl8jWmWu0ooWkK4ArAE6sfJiMMaZtsmn3DuJzwI8CvxURN0l6OytIneTLG20FOH3NhjhUgc+6hbBWfYMthInKx4IthE0n8ED7FsKir1X1vfPWu0nNlAnxVcmJ3w/cHxE35fc/QhbUH8pXsqDpihbGGDNpeqjRNikmrsQj4t8lfVvS0yPim2S1c7+eb5eRFUVvtKJFr6chBT7rFsI69Q22EJbHjTeBB2whbDqBB9q3EBYKvKq+H9/5OCkR06fEV8sn/lvAByStBe4GXkb2q2BFK1oYY8yk8YxNICJuAUatAr2iFS366+eGFPisu0/q1DfYfZKqfCzYfdJ4Ak+lL225TwoFXlXfR51U+TWXAk2fEm+UE5f0X/JJON+T9H1JuySlXzLDGGOmGHXYJ/5HwH+OiDva7MxKmVs3N6TAZ959UqO+we6TVOVjwe6Tpt5vaN99Uijwqvo+4dnle5iKrqZTHpq2AG6MMavBlMXwxkH8ZkkfBj4O7Ct2RsRHW+mVMcZMIV2esXkM8Djwc5V9AaxqEO/N9ZdNo8yihbAuhQK2EKaqAQ62EDadwAPtWwiLNEo1hbLptKeQmimL4c2CeES8rO2OGGNMF5iySrTNgrikU4A/Bc4lU+CfB66MiPtb7Fs9PQ0p8Fm3ENapb7CFcFI1wLNjZ8RCWKO+oX0LYaHAq+o7FkZfXD9cNIXLszX9Unk3cB3wJOBk4O/zfcYYM1NIzbb6drRe0hclfVXS7ZJ+P9//BkkPSLol3y5Yrp2mOfEnRkQ1aL9H0qsanjsRbCFsOIEHbCFMVD4WbCFsOoEH2rcQFgq8qr533/NtUiKSplP2AT8TEbslrQE+L+mT+WNvjYg3N2mkaX92SrpUUj/fLgUeqT3LGGOOMCQ12uqIjCLPtibfRucIl6GpEv8N4M+At+ZP8s/5vtVlEEMKfNbdJ3XqG+w+SVU+Fuw+aTqBB9p3nxQKvKq+d3zlLpKiFU32OUHSzZX7W/NS2mVzUp9sTYXTgT/Py3O/EHiFpF8DbgZeHRHfYQmaulPuAy5q3HVjjDkCEbCCNSF2RsSoGlGLRMQCcJakY4GPSXoW8A7gjWSC+Y3AH7OMaF42iEt6TUT8kaQ/ZYTMj4hX1g6jRQbzC0MKfNbdJ3XqG+w+sftk8ivQQ/vuk0KBV9X3976VvsRTk1TJSomI70q6EXhBNRcu6Z3A9cudW6fEi6n2Ny97lDHGzADZjM1EbUlPBA7kAXwD8HzgDyVtKZaqBH4BuG25dpYN4hHx9/nNxyPibw/pwKrX+144sDCkwGfefVKjvqt9tftkPPUNdp809X5X+9qW+6RQ4FX1PVgoP1epSKjDtwDX5HnxHnBtRFwv6X2SziLLftwL/OZyjTS9sPk64G8b7DPGmCMYJaudEhG3As8dsf+lK2mnLif+QuAC4GRJf1J56BhgCelhjDFHKFO4KESdEn+QLB9+EZkNpmAX8NttdaopgwMLQ2mUWbcQ1qVQwBbCZDXAwRbChhN4oH0LYZFGqaZQjn5S+blLgSLQEmnG1aIuJ/5V4KuSPhARVt7GmJlHkT7PPg516ZRrI+KXgK9IqkolkU04enarvatFQwp81i2EdeobbCGcVPlYmB0LYZ36hvYthIUCr6rvTVueQFrioPd6GqhLp1yZ/72w7Y4YY0wnWEJ8rBZ16ZRCau4E9kTEQNLTgGcAn1z6zMnQX9cbUuCzbiGsVd9gC2HO2BN4wBbChhN4oH0LYaHAq+q7WDQmGTF9SrxpAazPAeslnQxsA14GvKetThljzLSiGDTaJkVTn7gi4nFJlwN/mk/F/0qbHWvCmo3rhxT4rLtPatV3pS9V7D5Z+QQesPuk6QQeaN99UijwqvoetWzjeMTSn4tVonEQl/Q84FeBy1d4rjHGHBkEU5dOaRqIX0U2Q/NjEXG7pKcCn22vW83orV07pMBn3X0yrvoGu0/sPklfPhbad58st2h6OuKg92IaaFqK9h+Bf5R0tKRNEXE3sKoVDI0xZjXolE+8QNKPAO8FNmd39TDwaxFxe5udq+/YsAKfefdJjfoGu09SlY8Fu0+aer+hfffJKAVeXTg9GV0M4sBfAr8TEZ8FkHQe8E7gJ1rqlzHGTB8RS4qj1aJpED+qCOAAEXGjpKOWO8EYY45EOplOAe6W9D+B9+X3LwXuWeb4iTDYt38ojTLzFsKaFArYQjipGuAwOxbCuhQKTM5CWE2hFCt/paO7k31+A3gi8NF8O4Fswo8xxswWMWi2TYi6Aljrgf9OthLz18hWXT6w3DmTZH7PviEFbgthiS2ELZePBVsIG07ggfYthIUCr6rvR+/aOfLYw2YKp93XpVOuAQ4A/wS8EHgmmWfcGGNmDtG9nPiZEfEjAJL+Gvhi+11qzuDAYEiBz7qFsFZ9gy2EBeNO4AFbCBtO4IH2LYSFAq+q70fvbCEnvjBd7pS6nPjib/vUi0JI6kv6iqTr8/ubJd0g6c78b+LyY8YYMybFtPuu5MSB50gqvjYFbMjvF4tCHLP0qbVcCdxBtl4nwFXAtoi4WtJV+f3XLtdAb643pMBn3X0yrvoGu0/sPklfPhbad58UCryqvvc8ln6yz7SlU5ZV4hHRj4hj8u3oiJir3D7sAC7pFOBFwF9Vdl9MloMn//viw23fGGPaITqnxNvibcBrgIrU5KRiEYqI2C7pxFEnSroCuALglGOPHlLgs+4+qVXfYPdJovKxYPdJU+83tO8+KRR4VX1vOKqFYlhdUuJtIOlCYEdEfOlwzo+IrRFxdkScffymDfUnGGNMKopp9022CbEaSvxc4CJJFwDrgWMkvR94SNKWXIVvAXbUNdRbt3ZIgc+8+6RGfYPdJ6nKx4LdJ02939C++6RQ4FX1fcLTN5eNlT82xiAO+jU9DUxciUfE6yLilIg4FbgE+ExEXApcB1yWH3YZ8IlJ980YY5YlSKbEJa2X9EVJX5V0u6Tfz/evyKk38SC+DFcDPyvpTuBn8/vGGDM1BEEsLDTaGrAP+JmIeA5wFvACSedQOvXOIFvT+KrlGlnVJdYi4kbgxvz2I8D5Kzlf6g2lUWwhrGAL4eKuNmqAgy2ETSfwQPsWwiKNUk2hbH56+brwyaGnWjlBspV9IiKAIie4Jt+CzKl3Xr7/GrIYuaTdepqUuDHGTDkrurB5gqSbK9sVh7aWT3q8hewa4A0RcROHOPWAkU69gk4vdjw4cGBIgc+8hbBGfYMthOWB403gAVsIm07ggfYthIUCr6rv9ceXz5+EWNGFzZ0RcfbyzcUCcJakY4GPSXrWSrvU6SBujDGTJYgW7IMR8V1JNwIvYIVOvU4H8cH+A0MKfNYthHXqG2whTFY+FmwhbDiBB9q3EBYKvKq+q7+AklC4UxIg6YnAgTyAbwCeD/whpVPvaho49TodxI0xZrJEsgubwBbgGkl9suuT10bE9ZL+H3CtpMuB+4CXLNdIp4N4DGJIgc+6+2Rc9Q12nzSdwAN2n6SawAPju08KBV59/fc+8r2h5x+LoKl9sL6piFuB547YvyKnXqeDuDHGTJburnY/lWiuN6TAZ919Uqe+s2PtPoHxvd9g90lT7ze07z4p3oOq+n78kfLXWhJW5k6ZCJ0O4sYYM1msxI0xprskdKekotNBvL9u7XAaZcYthHUpFLCFMFUNcLCFsOkEHmjfQlikUaoplMH+tAE3CCKdOyUJnQ7ixhgzUazE06J+f0iB20JYYgthuzXAwRbCabIQFgq8qr4HC6N/mR42EcSB9Ot2jkOng7gxxkyWpJN9ktDtIF75aWMLYXHc8uobbCFMVT4WbCEsqFPf0L6FsFDgVfVdfa2S4XSKMcZ0lGinANY4dDqIx8L8kAKfdfdJnfoGu0+SlY+t9GXW3Sd16hvad58UCrz6Wu28p/x/SIXdKcYY01UiiAUH8WTEYDCkwGfdfVKnvsHuk1TlY8Huk6beb2jffVK8VlX1fc9jaafIRwSDA0t8rleJTgdxY4yZKIGVeErU6w0p8Fl3n9Spb7D7JFX5WLD7pKn3G9p3nxQKvKq+U9vEwUHcGGM6S0QwSFRPPBUO4sYYswLsTkmI+nPDaZQZtxDWpVDAFsJUNcDBFsKmE3igfQthkUapplBO31R+LkhRWtzuFGOM6S52p6Sm3x9S4LNuIaxT32ALYarysWALYdMJPNC+hbA4tKq+Tz3n5HJgn/g6KRhYiRtjTEexxTA1GlLgs24hrFPfYAthqvKxYAth0wk80L6FsFDgVfV9/DOrSpzxcU7cGGO6S2B3SlpiMKTAZ919Mq76zo61+wTq1TfYfdJ0Ag+07z4pFHhVfVcL4iUhgsF+X9g0xphuEjCwEk9IxJACn3X3SZ36BrtPUpWPrfZ11t0ndeob2nefFAq8qr53P1B+LlIQTF9OfIncgDHGmCEi+8JtstUh6cmSPivpDkm3S7oy3/8GSQ9IuiXfLliunW4rcfWGFPisu0/q1DfYfZKqfCzYfdLU+w3tu08KBV5V34/eVX7G0hApL2zOA6+OiC9LOhr4kqQb8sfeGhFvbtLIxJX4Mt8+myXdIOnO/O9xdW0ZY8xEyX3iTbbapiK2R8SX89u7gDuAk5c/a5jVSKcU3z7PBM4BXi7pTOAqYFtEnAFsy+8bY8zUEBEs7J9vtAEnSLq5sl2xVLuSTgWeC9yU73qFpFslvatO0E48nRIR24Ht+e1dkopvn4uB8/LDrgFuBF67bGP9ueE0ii2Ei9hC2G4NcLCFsOkEnmws7VoIizRKNYWy9zvl+5aGFaVTdkbE2XUHSdoE/B3wqoj4vqR3AG8ks6W/Efhj4DeWOn9Vc+KHfPuclAd4ImK7pBOXOOcK4AqAp/zAyEOMMaYdEk+7l7SGLIB/ICI+ChARD1Uefydw/XJtrFoQH/Ht0+i8iNgKbAU4+1nPiCEFPuMWwjr1DbYQpiofC7YQNp3AA+1bCAsFXs6TVEkAAAtsSURBVFXfc0tc6D9sAiLRckHKgt5fA3dExFsq+7cUghb4BeC25dpZlSA+6tsHeKjovKQtwI7V6JsxxixFECmrGJ4LvBT4mqRb8n2vB35F0llk6ZR7gd9crpGJB/Glvn2A64DLgKvzv/XlaqRlFfgsWghr1TfYQpiofCzYQth0Ag+0byEsFHhVfW84rvIZS0FADNIo8Yj4PDAqBfEPK2lnNZT4Ut8+VwPXSrocuA94ySr0zRhjliQCFvbXT+SZJKvhTlnq2wfg/BU2NqTAZ919Mq76BrtPmk7gAbtPmk7ggfbdJ4UCr6rvQSLVvEhEspx4Kro9Y9MYYybMwEG8Hew+aej9BrtPEpWPBbtPmnq/oX33SaHAq+r7O3eXfUmCV/YxxpjuErSQohkTB3FjjGlKhC9sJkW9oTTKzFsIa1IoYAthqhrgYAth0wk80L6FsFDI1RTK7dsr6cUERMLJPqnodhA3xphJ4iCeluj1hxW4LYQlthAu7mujBjjYQth0Ag+0byEsFHhVff/r7vJ1SUPSGZtJ6HQQN8aYiZJwxmYquh3E1Vtegc+ihbBGfYMthKnKx4IthE0n8ED7FsJCgadX3yWBfeLGGNNdIg76kpsGuh3ExbACn3H3ybjqG+w+aTqBB+w+aTqBB9p3n4xS4KduLPvF40MPr5is0oeVuDHGdJaECyUn4cgJ4nafAA3UN9h9kqh8LNh90tT7DZNzn1TV93OeuLF84FtDh66cCCtxY4zpLPaJp0ZDCnzW3Se16hvsPklUPhbsPmnq/c7G2K77pFDgVfW9+YzytUqhxAMXwDLGmO4SwcJ+B3FjjOkkETBY4rraatHpIB69/lAaxRbCCrYQLu5rowY42ELYdAIPtG8hLNIo1RTK+tRrbLK0OWK16HQQN8aYSRIcfH1gGuh0EF8YxJACn3kLYY36BlsIU5WPBVsIm07ggfYthIUCr6rvXn+p5XwPHytxY4zpKIOA/S6AlY5BLK/AZ9JCWKO+wRbCVOVjwRbCphN4oH0LYaHAq+q7jVV4nE4xxpiOEoTTKSmJGH5BZ919Uqe+we6TSZWPhdlxn9Spb2jffVIo8Kr63l8ZawpSXtiU9GTgvcAPAANga0S8XdJm4MPAqcC9wC9FxHeWameJZXCMMcaMYiGabQ2YB14dEc8EzgFeLulM4CpgW0ScAWzL7y9Jp5V4FbtPmnm/we6TVOVjwe6Tpt5vaN99Uijwqvre9719Q+2PQ0Q6d0pEbAe257d3SboDOBm4GDgvP+wa4EbgtUu1c8QEcWOMaZtgRe6UEyTdXLm/NSK2jjpQ0qnAc4GbgJPyAE9EbJd04nJP0ukgvhAxpMBn3X1Sq77B7pNE5WPB7pOm3m9o331SKPCq+t773bKvKVhhTnxnRJxdd5CkTcDfAa+KiO9LK/O2dzqIG2PMpEnpTpG0hiyAfyAiPprvfkjSllyFbwF2LNeGL2waY0xDouFFzSZqXZnk/mvgjoh4S+Wh64DL8tuXAZ9Yrp1OK/HH9i8MpVFm3UJYl0IBWwhtIZz8CvTQvoWwSKNUUyj7d6e1GEJSJX4u8FLga5Juyfe9HrgauFbS5cB9wEuWa6TTQdwYYyZJkBm6k7QV8Xmy5d5HcX7TdjodxB/dtX9Igc+6hbBOfYMthKnKx4IthKkm8MD4FsJCgVfVd+ql1IJw7RRjjOkqmTvFQTwZe3fvHlLgM28hrFHfYAthqvKxYAth0wk80L6FsFDgVfW9pvJaJKH5bMyJMXXuFEkvkPRNSXdJWna6qTHGTJJCiTfZJsVUKXFJfeDPgZ8F7gf+RdJ1EfH1UccPDuwbUuCz7j6pU99g90mq8rFg90nTCTzQvvukUOBV9d1fU/kfS8S0KfGpCuLAjwF3RcTdAJI+RFZHYGQQN8aYSTJg+haFUExRkl7SLwIviIj/mt9/KfDjEfGKyjFXAFfkd58F3DbxjrbPCcDO2qO6xZE4JvC4usTTI+LocRqQ9Cmy16YJOyPiBeM8XxOmTYmP8kwe9C2TF5DZCiDp5ia1CbrGkTiuI3FM4HF1iUOKUR0WkwjKK2XaLmzeDzy5cv8U4MEljjXGmJln2oL4vwBnSDpN0lrgErI6AsYYY0YwVemUiJiX9Arg/wB94F0Rcfsyp4yszXsEcCSO60gcE3hcXeJIHNN0Xdg0xhizMqYtnWKMMWYFOIgbY0yH6WwQPxKm50t6sqTPSrpD0u2Srsz3b5Z0g6Q787/H1bU1jUjqS/qKpOvz+50el6RjJX1E0jfy9+x5XR8TgKTfzj9/t0n6oKT1XRyXpHdJ2iHptsq+Jcch6XV5/PimpJ9fnV6PTyeDeGV6/guBM4FfkXTm6vbqsJgHXh0RzwTOAV6ej+MqYFtEnAFsy+93kSuBOyr3uz6utwOfiohnAM8hG1unxyTpZOCVwNkR8SwyQ8EldHNc7wEO9XGPHEf+f3YJ8MP5OX+Rx5XO0ckgTmV6fkTsB4rp+Z0iIrZHxJfz27vIgsLJZGO5Jj/sGuDFq9PDw0fSKcCLgL+q7O7suCQdA/wU2XJaRMT+iPguHR5ThTlgg6Q5YCPZ3IzOjSsiPgc8esjupcZxMfChiNgXEfcAd5HFlc7R1SB+MvDtyv37832dRdKpwHOBm4CTImI7ZIEeOHH1enbYvA14DQcvhNLlcT0VeBh4d54i+itJR9HtMRERDwBvJlsGbDvwvYj4NB0fV4WlxnHExJCuBvHa6fldQtImshWvXxUR3687ftqRdCGwIyK+tNp9Scgc8KPAOyLiucBjdCPFsCx5jvhi4DTgScBRki5d3V5NhCMmhnQ1iB8x0/MlrSEL4B+IiI/mux+StCV/fAuwY7X6d5icC1wk6V6yVNfPSHo/3R7X/cD9EXFTfv8jZEG9y2MCeD5wT0Q8HBEHgI8CP0H3x1Ww1DiOmBjS1SB+REzPlySyHOsdEfGWykPXAZflty8DPjHpvo1DRLwuIk6JiFPJ3pvPRMSldHhcEfHvwLclPT3fdT5ZieTOjinnPuAcSRvzz+P5ZNdmuj6ugqXGcR1wiaR1kk4DzgC+uAr9G5+I6OQGXAD8K/BvwO+udn8Ocww/SfYT7lbglny7ADie7Er6nfnfzavd1zHGeB5wfX670+MCzgJuzt+vjwPHdX1M+bh+H/gGWVnn9wHrujgu4INkef0DZEr78uXGAfxuHj++Cbxwtft/uJun3RtjTIfpajrFGGMMDuLGGNNpHMSNMabDOIgbY0yHcRA3xpgO4yBupgZJC5Juyavp/a2kjSs8/0mSPpLfPkvSBZXHLupqtUtjlsMWQzM1SNodEZvy2x8AvhQHT4JaSVu/TlaZ7xUJu2jM1GElbqaVfwJOz+tBf1zSrZK+IOnZAJJ+Olftt+QFqY6WdGqu4tcCfwD8cv74L0v6dUl/lp/7g5K25W1uk/SUfP97JP2JpH+WdLekX1y10RvTEAdxM3XkJVFfCHyNbDbhVyLi2cDrgffmh/0P4OURcRbwH4E9xfmRlSf+X8CHI+KsiPjwIU/xZ8B78zY/APxJ5bEtZDNpLwSuTj02Y1LjIG6miQ2SbiGb2n4fWV2ZnySbCk5EfAY4XtITgP8LvEXSK4FjI2J+Bc/zPOBv8tvvy5+j4OMRMYiIrwMnjTUaYybA3Gp3wJgKe3JlvUhelOlQIiKulvS/yWrNfEHS84G9h/m81QtD+6pPf5jtGTMxrMTNtPM54FcBJJ0H7IyI70v6oYj4WkT8IZlyf8Yh5+0Cjl6izX8mq65I3vbnk/famAnhIG6mnTcAZ0u6lSxHXZQVfVV+EfOrZPnwTx5y3meBM4sLm4c89krgZXmbLyVbC9SYTmKLoTHGdBgrcWOM6TAO4sYY02EcxI0xpsM4iBtjTIdxEDfGmA7jIG6MMR3GQdwYYzrM/wdpze+20qWPcAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Positional encoding correlation\n",
    "corr = tf.matmul(pos_encoding, pos_encoding, transpose_b=True).numpy()[0]\n",
    "plt.pcolormesh(corr, cmap='RdBu')\n",
    "plt.xlabel('Position')\n",
    "plt.xlim((0, MAX_SEQUENCE_LENGTH))\n",
    "plt.ylabel('Position')\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1.2.2 - Euclidean distance\n",
    "\n",
    "You can also use the euclidean distance instead of the correlation for comparing the positional encoding vectors. In this case, your visualization will display a matrix in which the main diagonal is 0, and its off-diagonal values increase as they move away from the main diagonal.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 100)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWsAAAEKCAYAAADU7nSHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO2de7AkZ3nef8+cszdpV5dlJXmRcCRiBZAvyI7KAeM4BGEbCJGolIlFWY7AqlJShUGycUCyyyFJlasUx0WMjJ3SxlyELQOyzEXGMaBagzFxLLwyAiSEIiwZsWil1XKRVtJqL+e8+WO6d74903O6z0z3zPTp51c1NT09X3/9fTNn3nnOO8/7tSICY4wx801v1gMwxhhTjoO1Mca0AAdrY4xpAQ7WxhjTAhysjTGmBThYG2NMC2gsWEt6j6T9ku5O9m2XdLuk+7P705PnrpP0NUn3SfrppsZljDFtpEll/T7gFSv2XQvsjojzgd3ZYyRdAFwGfH92zO9JWmhwbMYY0yoaC9YR8Vng2yt2XwrclG3fBLwm2f/BiDgcEQ8CXwN+tKmxGWNM21ic8vnOioh9ABGxT9KZ2f6zgb9J2u3N9g0h6SrgKgAWNv7TxW1nZc/0KzFHVWRKvfz4wc5eb+j5UcTycv9+6ejxfctLxwbPJ9vH+1wYvLy9xQ1D+04g75/B+NMx9bLj1EvGn0x1eWkJgKWjh5MxLxWeSr2FE/pM92UnHj4m2XfCGPJzJa97Ppb0Nclfv/7x2XuRvj7J+fP+C4aRnSvvM5J9yXbJe7Gw2D9/b6H4PV9eWs7ukzklr2X6vhT1lQ57ORvjcjJWkrHmc+31Rry++VzT+aV/4lnTXun7c8KjoQ6SjwIL6VgKjkj7yse9kJx/oeD8Ken7mrZdzPsqOH9K+vym5HXfcOwZAA7vf+T4vkf2P3V8+wBHDkTEGasOroTeKecE2XnKiEPf+mRErMwujM20g/Uoit6TwqgbEbuAXQAbTv/eeNbL3gIMPkzLx44UnmBx4xYAFrJ7gIVNg+3e4sb+QHrF2Zdjh54E4JknHju+75nvPHp8+/DBlf9EwKZt249vb3nWswHYfGrx38rS4UND40/HuuW0fl+btmw4vi8NVk9+92D//pEHj+878uR3Cs+1cevpQ2NZ3LL1+Hb+GqSvxeKGwXY+hjQoHD08CJBPP/4EAM88Pnit8vnB4HVPz79l27bB+LL+eyM+9EvHlofOefjQ4Evq6FOPDx2z+dRnHd/etr1//pNP2VzY/1NP9D+MT3538KE88vTgQ79h80mr9tVbGIz7maf7X+6HDg7e16WlwRfXxk39j+CWrRuP79u0ZfCxzIP80cODL4v0+DxIb0yOyftMSb8sltPjs2C3ZfPgmG3Jdh4Yl5LjjyXbJ23s/12cumUw/qLj0z42Lg4C7KknDf6ezzh5U39fcnyv4Bt7ezLX804bvO5nfuc+AP7ht3/z+L7/fsNAA97I178+1NlaOfYMi8+7pFLTo3e9d8fE50uYdrB+VNLOTFXvBPZn+/cCz0nanQM8POWxGWPM6kgjBV3TTDtY3wZcAVyf3X8s2f9Hkt4BPBs4H/h8eXexqqJeTFV0rqwL1DQUK+pcTcNAUY+jpqFYUadqMx9/kZqGgZotUtMwUNRlajodS5GaTreL1HT/+b7aKVLTMFDURWo6PX+RmoZiRZ2r6fS846hpKFbUuZqGgaIuU9NpX0VqGgaKukhNw0BRF6lpGCjqIjUNA0VdpKbTvorUNAwUdZkaLlLTMFDUZcfDQFEXqWkYKOoiNQ0DRV2kpmGgqFM1/R/f/KLj2zfeMLmwBp0QN6ZJY8Fa0geAlwI7JO0F3k4/SN8i6UrgIeC1ABFxj6RbgK8Ax4A3RkRxwtUYY2bFelTWEfG6EU9dPKL9bwC/scZzDCnqIjUNA2U3jpqGgaIuUtMwUNTjqOl0rEVqGgaKukhNQ7GiLlLTMFDURWoaBoq6SE3DQNkWqel0jkVqGgaKehw1DQNFXaSm++fqK+px1DQMFHWZmoaBoi5S0zBQxEVqGgaKukhNp8cXqemV/eYU5aeL1DQMFPEoNZwr6iI1XeX4ovx0kZpeOcecovx0kZqGgaJO1fS517x10NkNHxrqf60I0MI6C9bGGLPukE5wLk2TdROs7fboU6amwW6Pqm6PMjUNdnsUHT8rt0euqFM1vf/05w31PynrLg1ijDHrjvWYszbGmPWG0PECt2nT6mAt9YbSH1235pWlPtJtW/MmK3QBW/OqFrpA89a8PP2Rpj4e/G61asPKWFkbY0w7cLAeA0lDirrr1rwyNQ225lW15pWpabA1r2qhy8o55tRpzcsVdaqmv31oeJ2YiZBs3TPGmHlHWFmPR683pKi7bs0rU9PpGGzNm6zQBWzNq6vQBSa35uWKunY1naIeC+ut3NwYY9YdsrIeC6m3qqLuotujTE3329jtsbKfcQpdwG6PqoUu0Lzbo0hRL49Y335chN0gxhjTChysx2TlC9d1t0eZmga7Pepa1hTs9qjqnYbpuT1SNf34Mw24QRysjTFm3nGwnhi7Pap5p8Fuj7qWNQW7PebJ7ZEr6lRNP/bU4cK24yKJ3ga7QYwxZr5xGsQYY9qBg/UYxPLyUPqj69a8stQH2JpX1xrUYGteTlnqA5q35uXpjzT18XiS6qqLUem/pumVNzHGGAP9nLV61W4V+nqepLuS2xOSrhnVvt3KeunokKLuujWvTE2nc7Q1b7JCl7SPrlvzytQ0NG/NyxV1qqaPJH+jdbGwUI/GjYj7gAsBJC0A3wQ+Mqp9q4O1McZMFVFJNY/BxcDfR8TXRzVodbBeXjq2qqLuojWvTE2DrXl1LWsKtuZVLXSB5q15uaJO1fRCzYG1v+pe5T53SNqTPN4VEbtGtL0M+MBqnbU6WBtjzHTRyJRQAQci4qLSHqWNwCXAdau1a3WwjqVjQ4q6626PMjWdjsFuj8kKXcBuj6qFLtC82yNX1KPy87XQTBrklcDfRcSjqzVqdbA2xphp00Cwfh0lKRBYR8Habo9q3mmw26OuZU1X9lvUV1fcHmVqGpp3exTl5w/WvJCTBAuL9QVrSScBPwn8+7K26yZYG2PMNFD1nHUpEfE08KzShrQ8WGthcUhRd93tUaamwW6PaS1r2h9rN9weZWoamnd75K9BqqYfPzT4u6gDSTOrYGx1sDbGmGnTkM+6FAdrY4xZAw7WY9Bb3LBq+qOL1ryy1AfYmlfXGtRpX1235pWlPqB5a16e/khTH08fGbzHtaDRP8Y2TauDtTHGTBMheouzWf+u1cFaC4tDirrr1rxJ1TTYmle10AVszaur0AUmt+blijpV04t1pyw0uyVSWx2sjTFm2tRp3VsL6yZY25pXrdAFbM2zNW/6VxyH5q15uaJO1XQzCznV2mVlZnJaSb8k6R5Jd0v6gKTNkrZLul3S/dn96eU9GWPMFMnSIFVudTN1ZS3pbODNwAURcUjSLfSXB7wA2B0R10u6FrgWeNuqnS0vDynqrrs9ytR0el67PSZT02C3R9VCF2je7ZEr6kYXckInvP/TZFaX9VoEtkhaBE4CHgYuBW7Knr8JeM2MxmaMMYWoS8o6Ir4p6beAh4BDwKci4lOSzoqIfVmbfZLOLDpe0lXAVQC9k04fUtR2ewyw26PZZU3Bbo95cntMYyEnmF1RzNSVdZaLvhQ4D3g2cLKky6seHxG7IuKiiLiot2lr+QHGGFMTUv9LocqtbmbhBnk58GBEPAYg6cPAjwGPStqZqeqdwP4ZjM0YY1aliUBchVkE64eAF2XruB6if6HIPcBTwBXA9dn9x8o6knpD6Y+uW/PKUh9ga15da1CDrXk5ZakPaN6aV3T8obrXs6YZ1VyFWeSs75B0K/B3wDHgC8AuYCtwi6Qr6Qf01057bMYYsxrSiV+m02QmRTER8Xbg7St2H6avsivTW1gcUtRdt+ZNqqbB1ryqhS5ga17VQhdo3pqXH5+q6fR9rQOpgRL2iqybCkZjjGka0a2cdW2opyFF3XVrXpma7p/L1jyYvNAFbM2rWugCzVvzckU96n2tBdWbs5Z0GvD7wA8AAfxCRPzforatDtbGGDNN+sq61i+AdwKfiIifkbSRfpFgIe0O1jGsqLvu9ihT02C3R13LmoLdHlULXaB5t0fR+9pEJWFdylrSKcBPAK8HiIgjwMiLRrY7WBtjzBTpSWtxg+yQtCd5vCsidiWPnws8BrxX0guBO4GrI+IpCmh1sF5eWhpS1HZ7DLDbo9llTcFuj6reaWje7ZG/r01fHGCh+nrWByLiolWeXwR+BHhTZml+J/0F7H69qPGsFnIyxpjWUXO5+V5gb0TckT2+lX7wLqTVynrp6OFVFXUX3R6Tqmmw26Oqdzrto+tujzI1DM27PYo+Q0cOF//HMAl15awj4hFJ35D0vIi4j36dyVdGtW91sDbGmGnSQFHMm4CbMyfIA8AbRjV0sDbGmIqINf3AWEpE3AWsltc+TquDdSwvDaU/um7NK0t9gK15da1BnfbRdWteWeoDpmfNS1MfR0b8cDouec56FrQ6WBtjzDRxuXkN2JpXrdAFbM2ra1nTtI+uW/PK1DQ0b83LFXWqptP/GGrBytoYY+afTq1nXSfqLQwpalvzBtia1+yypmBrXtVCl7SPpqx5uaJO57fQwJXIHayNMWbO6XXt4gN10VtYHFLUXXd7lKlpsNujrmVNwW6PqoUuaR9NuT3yOaZqesOmwXtcC85ZG2PM/CO0lrVBaqXVwVq9hSFF3XW3R5maHtWX3R5r906D3R5VvdNpH025PXJFnarpJhZ1GvX7Q9O0OlgbY8w0EbAwm1jd8mAtDSnqrrs9ytR02pfdHpN5p8Fuj6reaWje7ZEr6rSfwzVXMKLml2AdRbuDtTHGTBEBG+q9rFdlHKyNMaYiToNMwMr0R9eteWWpD7A1r641qFfOMaeL1ryy1Ac0b83L+0pTH4eeHHlJw/GQnAYxxph5R9gNMhaShhR11615ZWoabM2b1rKm0B1rXpmahuatebmiTtV0I1eKcRrEGGPmGwk2NLDeSBVaHazV05Ci7ro1r0xNg615dS1rCrbmVS10geatebmiTvuseyEnp0GMMaYl1JkGkfQPwEFgCTgWESMv8dX6YL1SUXfd7VGmpsFuj7qWNQW7PeoqdEnnOK7bI+83PX7LtsHfRR0INaGs/2VEHChr1PpgbYwxU8Or7o1HRAwp6q67PcrUNNjtYbfH9C9iC827PfLjUzW9OfnvqQ76OevKzXdI2pM83hURu1a0CeBTkgK4seD547Q6WBtjzDRZY7n5gdVy0BkviYiHJZ0J3C7pqxHx2aKGrQ7Wy0tLQ4q6626PMjUNdnvUtawp2O1R1TsNzbs9ckWdqunlpSYumFtfdxHxcHa/X9JHgB8FCoP1TAyDkk6TdKukr0q6V9KLJW2XdLuk+7P702cxNmOMGUVu3atyK+1LOlnStnwb+Cng7lHtZ+PuhncCn4iI5wMvBO4FrgV2R8T5wO7ssTHGzBH9K8VUuVXgLOBzkr4IfB74s4j4xKjGU0+DSDoF+Ang9QARcQQ4IulS4KVZs5uAzwBvW62vWDo2lP7oujWvLPUBtubVtQY12Jo3T9a8PP2Rpj7Sz1sd1FkUExEP0BerlZhFzvq5wGPAeyW9ELgTuBo4KyL2AUTEvizhPoSkq4CrAHpbnCkxxkyPfrl5d6x7i8CPAG+KiDskvZM1pDwya8sugMVTz46Virrr1rwyNQ225tW1rCnYmpdTpqaheWterqhTNX3w2wPDQV3MqNp8JjnrvcDeiLgje3wr/eD9qKSdANn9/hmMzRhjVqWHKt3qZurKOiIekfQNSc+LiPuAi4GvZLcrgOuz+4+V9aVeb0hRd92aV6amwda8nEkLXcDWvKqFLtC8NS//vKVq+ugzTxeOe1zE7JT1rHzWbwJulrQReAB4A32Vf4ukK4GHgNfOaGzGGDOSGVWbzyZYR8RdQFFlz8Vr6UcLi0OKuutujzI1nfZht8dkhS5gt0fVQpe0r6bcHrmiTtX0xpNOHhr/RGjOc9aS/k1WrPK4pCckHZT0RPmRxhizflC9Pus1UVVZ/ybwryPi3tpHMAG93sKQou6626NMTYPdHnUtawp2e1T1TkPzbo9cUadqeutpxZ/XSZj3NMij8xaojTFmFswoVlcO1nskfQj4KHBcFkTEhxsZlTHGzCFtuKzXKcDT9BcayQlgpsFaPa2a/uiiNa8s9QG25tW1BjXYmle10AWat+bl6Y809TEqbTkJc23di4g3ND0QY4xpA7Na/a5SsJZ0DvA7wEvoK+rPAVdHxN4Gx1ZhXMOKoOvWvDI1vbLfor5szaun0AW6Y80rU9Npv01Z83JF3YSazpFmd1mvql8S7wVuA54NnA38abbPGGM6hVTtVjdVc9ZnREQanN8n6Zr6hzM+tubVU+jSH6uteVCupsHWvKqFLmkfTVnzyv4TrgMxuzRI1fMekHS5pIXsdjnwrSYHZowx84ikSre6qaqsfwF4F/A/6Oes/zrbN1MihhV1190eZWo67ctuj8kKXcBuj6qFLjA9t0ejS6RqzotiIuIh4JKGx2KMMXONgBlde2D1YC3prRHxm5J+h76iPoGIeHNjI6tALMeQou6626NMTYPdHnZ7TP+K49C826NImT/zeP3Z2jpTHJIWgD3ANyPi1au1LVPWeYn5njoGZowxbaZfwVhrl1fTj7OnlDVcNVhHxJ9mm09HxB+nz0ma+XrTEbGqorbbw26PnCaWNQW7Pap6p/tjbdbtkffVhJpOqStWZ/Ur/wr4DeCXy9pXdYNcV3GfMcasY0RP1W4V+G3grcByWUMoz1m/EngVcLakG5KnTgGKfyo3xpj1ytoKXnZISlPIu7ILfiPp1cD+iLhT0kurdFaWs36Yfr76EuDOZP9B4JcqD7khYunYUPqj69a8stQH2JpX1xrUYGte1UIXaN6aV5T+2HDyqUP7JkERaHmpvGGfAxFRdEUs6C/dcYmkVwGbgVMk/WFEXD6qs7Kc9ReBL0q6OSKspI0xnUdRKWuxKhFxHVkqOVPWv7JaoIbyNMgtEfFvgS9ISq176p8vfmiyIdeHrXnVCl3A1ry6ljUFW/OqFrrA9Kx5qZretGXT0POTEVBDsB6HsjTI1dn9qv4/Y4zpDBHlbdbUXXwG+ExZu7I0yL5s8wBwKCKWJf0T4PnAn084xonpX938REXddWtemZoGW/NyJi10AVvzqha6QPPWvFxRp2p6w4j/RMcmZqesq1r3PgtslnQ2sBt4A/C+pgZljDHzimK50q1uqn7tKCKelnQl8DtZCfoXah/NGllY7K2qqLvo9ihT02C3R13LmoLdHlULXaB5t0euqFM1vbBYVY9WJWB5Nl6LysFa0ouBnwOuXOOxxhizPgjm9gfGnGvo20w+EhH3SHou8OnmhlWN3kJv6Ju5626PMjUMdnvUtawp2O1R1TsNzbs9ckVdv5pOCVie42AdEX8J/KWkbZK2RsQDwExX3DPGmFnQRD66ClUvmPuDwPuB7f2Hegz4dxFxT5ODWwt2e1TzToPdHnUtawp2e1T1TkPzbo8iRZ1+hmpjnoM1cCPwyxHxaThecfO/gB9raFzGGDN/RED1cvNaqRqsT84DNfRN3JJOXu0AY4xZj8x1GgR4QNKvA3+QPb4ceLCZIVVneWl5KP3RdWteWeoDbM2raw1qsDWvrkIXqM+al36Gjhw6OvT8ZMx/UcwvAGcAH85uO+gXxhhjTLeI5Wq3milbyGkz8B+A7wO+DLwlIur+qhqb5aUYUtRdt+aVqem0D1vzJit0SfvoujWvTE2P6qtOa17+GUrV9KGDBwvbjs0My83L0iA3AUeBvwJeCbyAvufaGGM6h5jfnPUFEfGDAJLeDXy++SFVZ3l5aUhRd92aV6am0z5szZus0CXto+vWvDI1nfbVlDUvV9Spmn7m8ccKzzU+AUuzcYOU5ayP/7XXffEBSQuSviDp49nj7ZJul3R/dn96neczxpiJycvN5y1nDbxQ0hPZtoAt2eP84gOll09fhZWXYL8W2B0R10u6Nnv8ttU6kHpDirrrbo8yNZ32YbfHZIUuaR92ewyYldsjV9Spml46PDh/XcwqDbKqso6IhYg4Jbtti4jFZHvsQJ1cgv33k92X0s+Rk92/Ztz+jTGmGWJulXVT5Jdg35bsOyu/2EFE7JN0ZtGBkq4CrgLYcMoZQ4q6626PKgs52e1Rz7KmYLfHyn5W9jVNt0euqFM1vbBp8BtWbcyjsm6C9BLs4xwfEbsi4qKIuGjxpHqvXGyMMauSl5tXudXMLJR14SXYgUcl7cxU9U5gf1lH6RKpdntU806D3R51LWsKdntU9U5D826PXFGnanrzqWcUjmV8gjhWT6lJVsfyWWAT/Vh8a0S8fVT7qSvriLguIs6JiHOBy4C/yC7BfhtwRdbsCuBj0x6bMcasSlCnsj4MvCwiXghcCLxC0otGNZ6nq71cD9ySXTrsIeC1Mx6PMcacQBBETT7riAjgyezhhuw2ck3XmQbr9BLsEfEt4OK1HC+G0x9dt+aVpT7A1ry61qAGW/OqFrpA89a8PP2Rpj5OOnUSd3EBwVquFLND0p7k8a6I2JU2kLQA3El/SY/fjYg7RnU2T8raGGPmnDWtZ30gIi5atbeIJeBCSacBH5H0AxFxd1HbVgfr5eUYUtRdt+aVqWmwNa+uZU3B1rx5sublijpV0xtGfF7HJur7gfHEbuO7kj4DvAIoDNZT/4HRGGPaSxDLS5VuZUg6I1PUSNoCvBz46qj2rVfWKxV11615ZWp65RxzbM2rv9AlneN6t+aVqWlo3pqXK+r0nFH3NRhzN0g97ARuyvLWPeCWiPj4qMatDtbGGDNdYi0/MK7eU8SXgB+u2r7dwTpiSFF33e0xqZoGuz2qFrqA3R51FbrA5G6P/Lypmj5c92W9gtqse2ul3cHaGGOmyprcILXS6mCtnoYUddfdHmVqGuz2qGtZU7Dbo6p3Gpp3e+SKOlXTx47WHFgbcoNUodXB2hhjpouVtTHGzD/1ukHWRKuDda+nofRH1615ZakPsDXP1rwB07riODRvzcvTH2nqo4rfeS0EQdTkBlkrrQ7WxhgzVaysx0M9DSlqW/MG2JrX7BrUYGvePFnzckWdqum6lTURxNEj5e0aoNXB2hhjpkt9RTFrpd3BOoYVddeteWVqGmzNq2tZU7A17/iYS9R0/1zNWvNyFZ2q6WOHnqR2nAYxxpg5J6L+1EpFWh2sI2JIUXfd7VGmpsFuj7qWNU376rrbo0xNQ/Nuj3w7VdPp+evCbhBjjJl3IoglB+s1EzGsqLvu9ihT02C3R86k3mmw26Oqdxqad3vkijo9/5EnvzM0/kmICJaPFn8umqbVwdoYY6ZKYGU9FhpW1F13e5SpabDbo65lTcFuj6reaWje7ZGPoW41vRIHa2OMmXMigmWvZ22MMfOP3SBj0JNWTX900ZpXlvoAW/PqWoM67bfr1ryy1Ac0b80rSn9s3Hp64bzGpkY3iKTnAO8HvgdYBnZFxDtHtW91sDbGmGlSsxvkGPCWiPg7SduAOyXdHhFfKWrc6mCdXikmp+vWvDI1Dbbm1bWsadpH1615ZWo6PW/T1rxUTW/9nvOOb397qOV4LNekrCNiH7Av2z4o6V7gbGD9BWtjjJkqa7Pu7ZC0J3m8KyJ2FTWUdC79K53fMaqzdROsbc2rVugCtubVtawp2JpXtdAFmrfm5Yo6VdNbT9tW2HZs1pazPhARF5U1krQV+BPgmoh4YlS7dROsjTGmaYJ63SCSNtAP1DdHxIdXa9vqYB0xrKi77vaYVE2D3R5VC13Abo+qhS7QvNsjV9SpmtaI/77GJoLlI/X8wChJwLuBeyPiHWXtWx2sjTFmqgQs16esXwL8PPBlSXdl+341Iv53UeOWB+sYUtRdd3uUqWmw26OuZU37Y7XbY+U5Z+X2yBW1TviPaXD+Ogjq81lHxOeAytK/5cHaGGOmSEC43HwcNKSou+72KFPTYLdHXcuagt0eVb3T/TbNuj10/D+mZH7frctdnRMzKzcv/tWhQSQ9R9KnJd0r6R5JV2f7t0u6XdL92X3NdaLGGDMhmc+6yq1uph6sGZRYvgB4EfBGSRcA1wK7I+J8YHf22Bhj5oaIYOnIsUq3upl6GmSVEstLgZdmzW4CPgO8bbW+er3h9IeteQNszWt2DWqwNa9qoUu63ZQ1Lx9DmvpYOjKYVz3MLg0y05z1ihLLs7JATkTsk3TmiGOuAq4C2Lz9rOkM1BhjoJtXillZYqkRinQlWW39LoDt570gVirirlvzytQ02JpX17KmYGte1UIXaN6alyvqVE33Fgfvay0ERPJ3MU1mEqxHlFg+Kmlnpqp3AvtnMTZjjBlFJLUd02bqwXqVEsvbgCuA67P7j5X2xbAi7ro1r0xNg615dS1rCrbmVS10ScfQlDUvV9Spmk7/Y6iFOPG1mCazUNaFJZb0g/Qtkq4EHgJeO4OxGWPMSCJg6UhHimJKSiwvXlNfDCvqrrs9ytQ02O1R17KmYLdH1UIXaN7tkSvq2tV0SkS3ctbGGNNWlh2s107EsKLuutujTE2D3R51LWsKdnvU5Z2Gyd0eRYo6HUstdNG6Z4wxbSM48ctymjhYG2NMVSK68wNjnfR6Gkp/dN2aV5b6AFvzprUGNXTHmleW+oDpWfPSsRz61sOFYxmX6FpRjDHGtBIH6/FYkIYUddeteWVqGGzNq2sN6lF9ddGaV6amoXlrXj6WVE0fPlj/etazqmCcxRKpxhjTTrIKxiq3MiS9R9J+SXdXOXW7lXVPQ4q669a8MjUNtubVtaxp2peteQNmZc3LFXX9anpAUKvP+n3Au4D3V2nc6mBtjDFTJYLlmtwgEfHZbJnoSrQ+WK9UlF13e5SpabDbw26P6V9xHJp3exQp6k3bth/fruM65xFrUtY7JO1JHu/Klngei9YHa2OMmSZruFLMgYi4qK7ztj5Yr1TUXXd7lKnptA+7PZpV09Adt0eZmk7H0LTbI1XTm08fXE3qyaGWYxDhtUGMMWbusc96PKRhRd11t0eZmk77sOpss6gAAAhwSURBVNtjMu802O1R1TsNzbs9ckWdqunNp5xR2HZcgvoWcpL0AfoXCd8haS/w9oh496j2rQ7WxhgzVSJYOlJPsI6I162lvYO1McZUJOLE/46nSauD9UJPQ+mPrlvzylIfaR+25g0Yp9AFbM2rWugCzVvz8vRHmvpY3LJ16JhJWXKwNsaY+SaAGf2+2O5gvdjTkKLuujWvykJOtuad2M/KvmzNG1DXsqbQvDUvV9RNqOkUK2tjjJlzlgOO+Eoxa2ehp1UVdReteWVqGmzNq2tZU7A1r2qhy8o5Fo1lUmtekaJOX4u6cBrEGGPmnCCcBhkHMayou+72KFPTYLfHtJY1he64PSZV01Cf2yOd//KxI0PPT4J/YDTGmJbgYD0hdntU806D3R51LWsKdntU9U6nY2nK7ZG/BqmaTl/3OoiwG8QYY+aewG6QsVjoaUhRd93tMamaTufYdbdHmZoGuz2qeqehebdHrqhTNZ2OtQ6cszbGmJbgNIgxxsw5/Zz1bM7d6mC9aaE3lP7oujWvLPUBtubZmrc+rXn5656O9VjNaRCwsjbGmLkngHpWs147rQ7WG449M6Sou27NK1PTYGteXcua9s9la146DpidNS8faxNq+vgYCLtBjDFm3um7QRys18zh/Y8MKWpb8wbYmtfssqZga17VQhdo3ppXpKjT/whqYYY/MBb/tc4QSa+QdJ+kr0m6dtbjMcaYnFxZV7mVsdZYN1fKWtIC8LvATwJ7gb+VdFtEfKWo/SP7nxpS1F13e5SpabDbo65lTcFuj6qFLjA9t0c6/l5vYej5SalDWa811sGcBWvgR4GvRcQDAJI+CFwKjJyAMcZMi2VqKzdfc6ybt2B9NvCN5PFe4J+lDSRdBVyVPTx8I1+/G+DGG77e33PDhxof5BTYARyY9SBqZj3OCeZwXsUad83UMq+jyfaTI1tNjeeVN1mdAxz55I18fUfF5psl7Uke74qIXdl2aaxbybwF6yLf2QlfY9lkdwFI2hMRF01jYNNkPc5rPc4JPK82sSJwjkVEvKKOsVAh1q1k3n5g3As8J3l8DvDwiLbGGNNW1hzr5i1Y/y1wvqTzJG0ELgNum/GYjDGmbtYc6+YqDRIRxyT9IvBJYAF4T0Tcs8ohu1Z5rs2sx3mtxzmB59Um5mZOY8Q6FDOqxjHGGFOdeUuDGGOMKcDB2hhjWkBrg/V6KEuX9BxJn5Z0r6R7JF2d7d8u6XZJ92f3p5f1NY9IWpD0BUkfzx63el6STpN0q6SvZu/Zi9s+JwBJv5T9/d0t6QOSNrdxXpLeI2m/pLuTfSPnIem6LH7cJ+mnZzPq6rQyWCelmq8ELgBeJ+mC2Y5qLI4Bb4mIFwAvAt6YzeNaYHdEnA/szh63kauBe5PHbZ/XO4FPRMTzgRfSn1ur5yTpbODNwEUR8QP0f+y6jHbO633ASh904Tyyz9llwPdnx/xeFlfml4ho3Q14MfDJ5PF1wHWzHlcN8/oY/bUC7gN2Zvt2AvfNemxjzOUc+h+OlwEfz/a1dl7AKcCDZD/KJ/tbO6dszHkl3Xb67rCPAz/V1nkB5wJ3l70/K2MGfVfGi2c9/tVurVTWFJdqnj2jsdSCpHOBHwbuAM6KiH0A2f2ZsxvZ2Pw28FZOvLBGm+f1XOAx4L1Zauf3JZ1Mu+dERHwT+C3gIWAf8HhEfIqWzyth1DxaF0PaGqzXXKo5z0jaCvwJcE1EPDHr8UyKpFcD+yPizlmPpUYWgR8B/mdE/DDwFO1IDaxKlsO9FDgPeDZwsqTLZzuqqdC6GNLWYL1uytIlbaAfqG+OiA9nux+VtDN7fiewf1bjG5OXAJdI+gfgg8DLJP0h7Z7XXmBvRNyRPb6VfvBu85wAXg48GBGPRcRR4MPAj9H+eeWMmkfrYkhbg/W6KEuXJODdwL0R8Y7kqduAK7LtK+jnsltDRFwXEedExLn035u/iIjLafG8IuIR4BuS8pXbLqa/nGVr55TxEPAiSSdlf48X0//htO3zyhk1j9uAyyRtknQecD7w+RmMrzqzTppP8EPCq4D/B/w98GuzHs+Yc/hx+v96fQm4K7u9CngW/R/n7s/ut896rBPM8aUMfmBs9byAC4E92fv1UeD0ts8pm9d/Ab4K3A38AbCpjfMCPkA/736UvnK+crV5AL+WxY/7gFfOevxlN5ebG2NMC2hrGsQYYzqFg7UxxrQAB2tjjGkBDtbGGNMCHKyNMaYFOFibuUHSkqS7stXf/ljSSWs8/tmSbs22L5T0quS5S9q6OqMx4CvFmDlC0pMRsTXbvhm4M04sFlpLX6+nv5LcL9Y4RGNmhpW1mVf+Cvi+bD3ij0r6kqS/kfRDAJL+RabC78oWVtom6dxMlW8E/ivws9nzPyvp9ZLelR37jyTtzvrcLel7s/3vk3SDpL+W9ICkn5nZ7I1ZgYO1mTskLdJfq/zL9KvrvhARPwT8KvD+rNmvAG+MiAuBfw4cyo+PiCPAfwI+FBEXRsSHVpziXcD7sz5vBm5InttJv7L01cD1dc/NmHFxsDbzxBZJd9Ev6X6I/ropP06/BJqI+AvgWZJOBf4P8A5JbwZOi4hjazjPi4E/yrb/IDtHzkcjYjkivgKcNdFsjKmRxVkPwJiEQ5lSPk62uNBKIiKul/Rn9NdS+RtJLweeGfO86Q83h9PTj9mfMbVjZW3mnc8CPwcg6aXAgYh4QtI/jogvR8R/o6/En7/iuIPAthF9/jX91QDJ+v5c7aM2pmYcrM2885+BiyR9iX4OOV/u8prsx8Qv0s9X//mK4z4NXJD/wLjiuTcDb8j6/Hn614o0Zq6xdc8YY1qAlbUxxrQAB2tjjGkBDtbGGNMCHKyNMaYFOFgbY0wLcLA2xpgW4GBtjDEt4P8DR0NjzeqvkVIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Positional encoding euclidean distance\n",
    "eu = np.zeros((MAX_SEQUENCE_LENGTH, MAX_SEQUENCE_LENGTH))\n",
    "print(eu.shape)\n",
    "for a in range(MAX_SEQUENCE_LENGTH):\n",
    "    for b in range(a + 1, MAX_SEQUENCE_LENGTH):\n",
    "        eu[a, b] = tf.norm(tf.math.subtract(pos_encoding[0, a], pos_encoding[0, b]))\n",
    "        eu[b, a] = eu[a, b]\n",
    "        \n",
    "plt.pcolormesh(eu, cmap='RdBu')\n",
    "plt.xlabel('Position')\n",
    "plt.xlim((0, MAX_SEQUENCE_LENGTH))\n",
    "plt.ylabel('Position')\n",
    "plt.colorbar()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nice work! You can use these visualizations as checks for any positional encodings you create. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='2'></a>\n",
    "## 2 - Semantic embedding\n",
    "<div style=\"text-align: right;\"><a href=\"#toc\" style=\"text-decoration:none;\">↑ Table of Contents</a></div>\n",
    "\n",
    "You have gained insight into the relationship positional encoding vectors have with other vectors at different positions by creating correlation and distance matrices. Similarly, you can gain a stronger intuition as to how positional encodings affect word embeddings by visualizing the sum of these vectors.  \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='2-1'></a>\n",
    "### 2.1 - Load pretrained embedding\n",
    "<div style=\"text-align: right;\"><a href=\"#toc\" style=\"text-decoration:none;\">↑ Table of Contents</a></div>\n",
    "\n",
    "To combine a pretrained word embedding with the positional encodings you created, start by loading one of the pretrained embeddings from the [glove](https://nlp.stanford.edu/projects/glove/) project. You will use the embedding with 100 features. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 400000 word vectors.\n",
      "d_model: %s (100,)\n"
     ]
    }
   ],
   "source": [
    "embeddings_index = {}\n",
    "GLOVE_DIR = \"glove\"\n",
    "f = open(os.path.join(GLOVE_DIR, 'glove.6B.100d.txt'))\n",
    "for line in f:\n",
    "    values = line.split()\n",
    "    word = values[0]\n",
    "    coefs = np.asarray(values[1:], dtype='float32')\n",
    "    embeddings_index[word] = coefs\n",
    "f.close()\n",
    "\n",
    "print('Found %s word vectors.' % len(embeddings_index))\n",
    "print('d_model: %s', embeddings_index['hi'].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** This embedding is composed of 400,000 words and each word embedding has 100 features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Consider the following text that only contains two sentences. Wait a minute - these sentences have no meaning! Instead, the sentences are engineered such that:\n",
    "* Each sentence is composed of sets of words, which have some semantic similarities among each groups.\n",
    "* In the first sentence similar terms are consecutive, while in the second sentence, the order is random. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "texts = ['king queen man woman dog wolf football basketball red green yellow',\n",
    "         'man queen yellow basketball green dog  woman football  king red wolf']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, run the following code cell to apply the tokenization to the raw text. Don't worry too much about what this step does - it will be explained in detail in later ungraded labs. A quick summary (not crucial to understanding the lab):\n",
    "\n",
    "* If you feed an array of plain text of different sentence lengths, and it will produce a matrix with one row for each sentence, each of them represented by an array of size `MAX_SEQUENCE_LENGTH`.\n",
    "* Each value in this array represents each word of the sentence using its corresponding index in a dictionary(`word_index`). \n",
    "* The sequences shorter than the `MAX_SEQUENCE_LENGTH` are padded with zeros to create uniform length. \n",
    "\n",
    "Again, this is explained in detail in later ungraded labs, so don't worry about this too much right now!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 11 unique tokens.\n",
      "(2, 100)\n",
      "[[ 1  2  3  4  5  6  7  8  9 10 11  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "   0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "   0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "   0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "   0  0  0  0]\n",
      " [ 3  2 11  8 10  5  4  7  1  9  6  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "   0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "   0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "   0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0  0\n",
      "   0  0  0  0]]\n"
     ]
    }
   ],
   "source": [
    "tokenizer = Tokenizer(num_words=MAX_NB_WORDS)\n",
    "tokenizer.fit_on_texts(texts)\n",
    "sequences = tokenizer.texts_to_sequences(texts)\n",
    "\n",
    "word_index = tokenizer.word_index\n",
    "print('Found %s unique tokens.' % len(word_index))\n",
    "\n",
    "data = pad_sequences(sequences, padding='post', maxlen=MAX_SEQUENCE_LENGTH)\n",
    "\n",
    "print(data.shape)\n",
    "\n",
    "print(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To simplify your model, you will only need to obtain the embeddings for the different words that appear in the text you are examining. In this case, you will filter out only the 11 words appearing in our sentences. The first vector will be an array of zeros and will codify all the unknown words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(12, 100)\n"
     ]
    }
   ],
   "source": [
    "embedding_matrix = np.zeros((len(word_index) + 1, EMBEDDING_DIM))\n",
    "for word, i in word_index.items():\n",
    "    embedding_vector = embeddings_index.get(word)\n",
    "    if embedding_vector is not None:\n",
    "        # words not found in embedding index will be all-zeros.\n",
    "        embedding_matrix[i] = embedding_vector\n",
    "print(embedding_matrix.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create an embedding layer using the weights extracted from the pretrained glove embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_layer = Embedding(len(word_index) + 1,\n",
    "                            EMBEDDING_DIM,\n",
    "                            embeddings_initializer=tf.keras.initializers.Constant(embedding_matrix),\n",
    "                            trainable=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transform the input tokenized data to the embedding using the previous layer. Check the shape of the embedding to make sure the last dimension of this matrix contains the embeddings of the words in the sentence. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(2, 100, 100)\n"
     ]
    }
   ],
   "source": [
    "embedding = embedding_layer(data)\n",
    "print(embedding.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='2-2'></a>\n",
    "### 2.2 - Visualization on a Cartesian plane\n",
    "<div style=\"text-align: right;\"><a href=\"#toc\" style=\"text-decoration:none;\">↑ Table of Contents</a></div>\n",
    "\n",
    "Now, you will create a function that allows you to visualize the encoding of our words in a Cartesian plane. You will use PCA to reduce the 100 features of the glove embedding to only 2 components. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "\n",
    "def plot_words(embedding, sequences, sentence):\n",
    "    pca = PCA(n_components=2)\n",
    "    X_pca_train = pca.fit_transform(embedding[sentence,0:len(sequences[sentence]),:])\n",
    "\n",
    "\n",
    "    fig, ax = plt.subplots(figsize=(12, 6)) \n",
    "    plt.rcParams['font.size'] = '12'\n",
    "    ax.scatter(X_pca_train[:, 0], X_pca_train[:, 1])\n",
    "    words = list(word_index.keys())\n",
    "    for i, index in enumerate(sequences[sentence]):\n",
    "        ax.annotate(words[index-1], (X_pca_train[i, 0], X_pca_train[i, 1]))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Nice! Now you can plot the embedding of each of the sentences. Each plot should disply the embeddings of the different words. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuEAAAFlCAYAAABFiTbBAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de1RXVf7/8dcWiFARFDABCUSLsu9UNJiWF6oxUTHHNK+lqeXUmjGzzEbFb2nTZGU287NmvpV+Ryt1tItpakpXyfJSKHQXR03ID94JFAO57d8f6OcrpXnhw/lweT7WYq3P2edz9n6fs4r18rDPPsZaKwAAAADOaeTtAgAAAICGhhAOAAAAOIwQDgAAADiMEA4AAAA4jBAOAAAAOIwQDgAAADjM1xuDhoaG2piYGG8MDQAAgAZk8+bNB621Yd6u4+e8EsJjYmKUnp7ujaEBAADQgBhjsr1dw6kwHQUAAABwGCEcAAAAcBghHAAAAHAYIRwAAABwGCEcAAAAcBghHAAAAHAYIRxAnZaRkaFrrrlGgYGBGjx4sIYMGaKpU6dq/vz56tKlS5XvGmO0fft2SdKxY8f00EMP6eKLL9ZFF12ke++9V0VFRe7vrly5UldffbWCg4N1/fXX68svv3Tvi4mJ0TPPPKMrr7xSQUFBGjx4sIqLi505YQBAvUAIB1BnlZSUqF+/fho+fLjy8vI0cOBAvfnmm2d17J///Gdt27ZNmZmZ2r59u1wulx577DFJ0pYtWzR69Gi9+OKLOnTokO655x717dtXx44dcx//2muvac2aNfr+++/15Zdfav78+TVxigCAeooQDqDO2rhxo0pLSzV+/Hj5+fnptttuU4cOHc54nLVWc+bM0d/+9je1aNFCgYGBmjJlihYvXixJmjNnju655x517NhRPj4+uvPOO+Xv76+NGze6+xg3bpwiIiLUokUL3XLLLcrMzKyx8wQA1D9eeWMmAFTHsgyXZqZmadv6NSpq1EzLM3PVLz5SkhQdHX3G4w8cOKCffvpJv/3tb91t1lqVl5dLkrKzs/Xyyy/rueeec+8vKSlRbm6ue7tVq1buz40bN66yDwCAMyGEA6hTlmW4NHnpVyoqLZdP0xYqLjigSW9WztfuFx+pnJwctW3bVk2aNNFPP/3kPm7v3r3uz6GhoQoICNA333yjyMjIX4wRFRWllJQUpaSk1PwJAQAaJKajAKhTZqZmqai08o61f+RlkvHR/o1v6el3vtXSpUv12WefSZKuuuoqffPNN8rMzFRxcbGmTZvm7qNRo0YaM2aMHnjgAe3fv1+S5HK5lJqaKkkaM2aMXnjhBW3atEnWWh09elSrVq3SkSNHnD1ZAEC9RQgHUKfk5v/fCibGx09ht05R4VcfaONjv9eSJUvUv39/SdKll16qRx55RN27d9cll1zyi5VSnnrqKbVr106dOnVSs2bN1L17d2VlZUmSEhISNGfOHI0dO1bNmzdXu3btePASAOBRxlrr+KAJCQk2PT3d8XEB1D0xMTGaO3euunfvLknq/OSHcuUXKXfuH9Wix7268OIrJUmRwQH6dNJNGjlypFq3bq3HH3/cm2UDAGoJY8xma22Ct+v4Oe6EA6hTJibFKcDPRxF3/9MdwAP8fDQxKc7LlQEAcPZ4MBNAnXJiFZSZqVnKzS9SRHCAJibFudsBAKgLuBMOoM7YunWr2rRpo+KsdXK9MFpzuvvr00k3KXP5HA0aNEgjRozQm2++qbfeeksnT3nbsmWL4uPjFRgYqIEDB2rw4MGaOnWqF88EANDQEcIB1AlbtmxRjx499Nxzz2nIkCG/2P/2229ryJAhys/PV9++fTV27FhJlet733rrrRo5cqTy8vI0dOhQvfXWW06XDwBAFYRwALXeunXr1LdvX7388svq06fPKb/TpUsX9e7dWz4+Pho+fLi++OILSZVv1SwrK9O4cePk5+en/v3769prr3WyfAAAfqHaIdwYc6Ex5jNjzBfGmG+MMdM9URgAnPDCCy/o+uuv14033nja7/z8DZbFxcUqKytTbm6uIiMjZYxx74+KiqrRegEAOBNP3Ak/Jukma+1Vkq6W1NMY08kD/QJooJZluNT5yQ/VZtIq7S0o1qiH/6qcnBw98MAD59xXeHi4XC6XTl6O9YcffvBkuQAAnLNqh3BbqfD4pt/xH+cXHwdQL5x4Lb0rv0hWUlmF1aItB3Tf0//Sxx9/rEmTJp1Tf9ddd518fHz0/PPPq6ysTMuXL3e/VRMAAG/xyBKFxhgfSZsltZP0D2vtplN85w+S/iBJF198sSeGBVAPnfxa+hOOlVfon+v36r333tONN94oPz+/s+7vggsu0NKlS3X33Xdr8uTJ6tWrl/r06SN/f39Plw4AwFnz6BszjTHBkt6SdJ+19uvTfY83ZgI4nTaTVp3yT2lG0vdPJntkjI4dO+ree+/VqFGjPNIfAKD2ahBvzLTW5ktaK6mnJ/sF0HBEBAecU/vZSEtL0969e1VWVqaXX35ZX375pXr25NcUAMB7PLE6StjxO+AyxgRI6i5pa3X7BdAwnXgt/cmq+1r6rKwsXXXVVQoKCtKsWbP0xhtvKDw8vLqlAgBw3qo9HcUYc6WklyX5qDLUv2atfezXjmE6CoBfsyzDxWvpAQAeUVuno1T7wUxr7ZeS4j1QCwBIkvrFRxK6AQD1Gm/MBAAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHvGTevHm65ZZb3Nvt2rXToEGD3NtRUVHKzMzU+vXr1aFDBwUFBalDhw5av369+zs33HCDpk6dquuvv15NmzbVLbfcokOHDun2229Xs2bN1KFDB+3atcv9/fvvv19RUVFq1qyZfvvb32rdunXufdOmTdOgQYM0YsQIBQYG6oorrlB6enrNXgQAABqoaodwY0yUMeYjY8x3xphvjDH3e6IwoL5LTEzUunXrVFFRoT179qi0tFSffvqpJGnnzp0qLCzUxRdfrOTkZI0bN06HDh3Sgw8+qOTkZB06dMjdz+LFi/Xqq6/K5XJpx44duu666zRq1Cjl5eXp8ssv1/Tp093f7dChgzIzM5WXl6dhw4Zp4MCBKi4udu9/++23NWTIEOXn56tv374aO3ascxcEAIAGxBN3wsskTbDWXi6pk6Q/GWPae6BfoF6LjY1VYGCgMjMzlZaWpqSkJEVGRmrr1q1KS0tT165dtWrVKl1yySUaPny4fH19NXToUF122WVasWKFu59Ro0apbdu2CgoKUq9evdS2bVt1795dvr6+GjhwoDIyMtzfveOOOxQSEiJfX19NmDBBx44dU1ZWlnt/ly5d1Lt3b/n4+Gj48OH64osvHL0mAAA0FL7V7cBau0fSnuOfjxhjvpMUKenb6vYN1DfLMlyamZql3PwiRQQHKPY3HbR27Vpt375diYmJCg4OVlpamjZs2KDExETl5uYqOjq6Sh/R0dFyuVzu7Ysuusj9OSAg4BfbhYWF7u1Zs2Zp7ty5ys3NlTFGhw8f1sGDB937W7Vq5f7cuHFjFRcXq6ysTL6+1f5VAQAATuLROeHGmBhJ8ZI2nWLfH4wx6caY9AMHDnhyWKBOWJbh0uSlX8mVXyQryZVfpG0mSouXr9G6deuUmJioxMREpaWlKS0tTYmJiYqIiFB2dnaVfnJychQZGXnO469bt05PPfWUXnvtNf3444/Kz89XUFCQrLUeOkMAAHC2PBbCjTFNJb0paby19vDP91trX7LWJlhrE8LCwjw1LFBnzEzNUlFpeZW2RpFXaPPGT1RUVKTWrVura9euWrNmjQ4dOqT4+Hj17t1b27Zt06JFi1RWVqYlS5bo22+/VZ8+fc55/CNHjsjX11dhYWEqKyvTY489psOHf/G/KgAAcIBHQrgxxk+VAXyhtXapJ/oE6pvc/KJftPm1iJTxu1Bdu3aVJDVr1kyxsbHq3LmzfHx8FBISopUrV2rWrFkKCQnR008/rZUrVyo0NPScx09KSlKvXr106aWXKjo6WhdeeKGioqKqfV4AAODcmer+KdoYYyS9LCnPWjv+bI5JSEiwLH2Ghqbzkx/KdYogHhkcoE8n3eSFigAAqP+MMZuttQneruPnPHEnvLOk4ZJuMsZkHv/p7YF+gXplYlKcAvx8qrQF+PloYlKclyoCAADe4onVUT6RZDxQC1Cv9YuvfJjy5NVRJibFudsBAEDDwbpjgIP6xUcSugEAAK+tBwAAAJxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIB+qwmJgYzZw5U1deeaWaNGmiu+66S/v27VOvXr0UGBio7t2768cff5QkDRw4UK1atVJQUJC6deumb775xt3PyJEj9ac//UnJyckKDAxUx44dtWPHDm+dFgAA9R4hHKjj3nzzTb333nvatm2bVqxYoV69eumJJ57QwYMHVVFRodmzZ0uSevXqpf/85z/av3+/rrnmGt1+++1V+vn3v/+tRx99VD/++KPatWunlJQUb5wOAAANgq+3CwBQPffdd58uuugiSVLXrl3VsmVLxcfHS5JuvfVWffDBB5Kk0aNHu4+ZNm2amjdvroKCAgUFBUmS+vfvr2uvvVaSdPvtt+vBBx908jQAAGhQCOFAHbMsw6WZqVnKzS/S3oJibT/i494XEBDgDuQntgsLC1VeXq6UlBS9/vrrOnDggBo1qvwj2MGDB90hvFWrVu7jGjdurMLCQofOCACAhofpKEAdsizDpclLv5Irv0hWUlmF1f+u+17LMly/etyiRYu0fPlyvf/++yooKNCuXbskSdbami8aAAD8AiEcqENmpmapqLS8Stux8grNTM361eOOHDkif39/hYSE6KefftKUKVNqskwAAHAGhHCgDsnNLzqn9hNGjBih6OhoRUZGqn379urUqVNNlAcAAM6S8cafoxMSEmx6errj4wJ1XecnP5TrFIE7MjhAn066yQsVAQBQuxljNltrE7xdx89xJxyoQyYmxSnAz6dKW4CfjyYmxXmpIgAAcD5YHQWoQ/rFR0qSe3WUiOAATUyKc7cDAIC6gRAO1DH94iMJ3QAA1HFMRwEAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABzmkRBujPmXMWa/MeZrT/QHAAAA1GeeuhM+X1JPD/UFAAAA1GseCeHW2o8l5XmiLwAAAKC+c2xOuDHmD8aYdGNM+oEDB5waFgAAAKh1HAvh1tqXrLUJ1tqEsLAwp4YFAAAAah1WRwEAAAAcRggHAAAAHOapJQr/LWmDpDhjzG5jzF2e6BcAAACoj3w90Ym1dqgn+gEAAAAaAqajAAAAAA4jhAMAAAAOI4QDAAAADiOEAwAAAA4jhAMAAAAOI4QDAAAADiOEAwAAAA4jhAMAAAAOI4QDAAAADiOEw+tGjhypqVOnersMAAAAxxDCAQAAAIcRwgEAAACHEcLhuIyMDF1zzTUKDAzU4MGDVVxc7N43Z84ctWvXTi1atFDfvn2Vm5vr3vfuu+8qLi5OQUFB+uMf/6jExETNnTvXG6cAAABQLYRwOKqkpET9+vXT8OHDlZeXp4EDB+rNN9+UJH344YeaPHmyXnvtNe3Zs0fR0dEaMmSIJOngwYO67bbbNGPGDB06dEhxcXFav369N08FAADgvBHC4aiNGzeqtLRU48ePl5+fn2677TZ16NBBkrRw4UKNHj1a11xzjfz9/TVjxgxt2LBBu3bt0jvvvKMrrrhC/fv3l6+vr8aNG6dWrVp5+WwAAADODyEcjsrNzVVkZKSMMe626Oho974TnyWpadOmCgkJkcvlUm5urqKiotz7jDFq3bq1c4UDAAB4kK+3C0D9tyzDpZmpWcrNL1KTvD3K3fWDrLXuIJ6Tk6O2bdsqIiJC2dnZ7uOOHj2qQ4cOKTIyUuHh4dq9e7d7n7W2yjYAAEBdwp1w1KhlGS5NXvqVXPlFspKOBLXRkZIK3f3wYyorK9PSpUv12WefSZKGDRumefPmKTMzU8eOHdOUKVPUsWNHxcTEKDk5WV999ZWWLVumsrIy/eMf/9DevXu9e3IAAADniRCOGjUzNUtFpeXubePjp9B+U7Rk4QI1b95cS5YsUf/+/SVJv/vd7/SXv/xFAwYMUHh4uHbs2KHFixdLkkJDQ/X666/r4YcfVkhIiL799lslJCTI39/fK+cFAABQHcZa6/igCQkJNj093fFx4bw2k1bpVP+FGUnfP5l83v1WVFSodevWWrhwoW688cbz7gcAANRvxpjN1toEb9fxc9wJR42KCA44p/Zfk5qaqvz8fB07dkxPPPGErLXq1KlTdUsEAABwHCEcNWpiUpwC/HyqtAX4+WhiUtw597Vhwwa1bdtWoaGhWrFihZYtW6aAgHMP8wAAAN7GdBTUuJNXR4kIDtDEpDj1i4/0dlkAAKABqK3TUViiEDWuX3wkoRsAAOAkTEcBAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRgiH19xwww2aO3eue3vq1KkKDQ1Vq1atvFgVAABAzeO19agVfvjhB82aNUvZ2dlq2bKlt8sBAACoUdwJR62QnZ2tkJAQAjgAAGgQCOE4L/PmzdMtt9zi3m7Xrp0GDRrk3o6KilJmZqbWr1+vDh06KCgoSB06dND69et/0df777+vm2++Wbm5uWratKlGjhzpxCkAAAB4DSEc5yUxMVHr1q1TRUWF9uzZo9LSUn366aeSpJ07d6qwsFAXX3yxkpOTNW7cOB06dEgPPvigkpOTdejQoSp9de/eXatXr1ZERIQKCws1f/58L5wRAACAcwjhOC+xsbEKDAxUZmam0tLSlJSUpMjISG3dulVpaWnq2rWrVq1apUsuuUTDhw+Xr6+vhg4dqssuu0wrVqzwdvkAAABexYOZOG+JiYlau3attm/frsTERAUHBystLU0bNmxQYmKicnNzFR0dXeWY6OhouVwuL1UMAABQOxDCcdaWZbg0MzVLuflFiggO0FVtrtTatWv1/fffa8qUKQoODtbChQu1YcMGjR07Vt99952ys7Or9JGTk6OePXt66QwAAABqB6aj4Kwsy3Bp8tKv5MovkpXkyi9S6qEWeu+DD1VUVKTWrVura9euWrNmjQ4dOqT4+Hj17t1b27Zt06JFi1RWVqYlS5bo22+/VZ8+fbx9OgAAAF7lkRBujOlpjMkyxmw3xkzyRJ+oXWamZqmotLxKW3mzcJX7+Ktr166SpGbNmik2NladO3eWj4+PQkJCtHLlSs2aNUshISF6+umntXLlSoWGhnrjFAAAAGoNY62tXgfG+EjaJulmSbslfS5pqLX229Mdk5CQYNPT06s1LpzVZtIqneq/FCPp+yeTnS4HAADgrBhjNltrE7xdx8954k74tZK2W2t3WmtLJC2W9HsP9ItaJCI44JzaAQAAcHqeCOGRkn44aXv38bYqjDF/MMakG2PSDxw44IFh4aSJSXEK8POp0hbg56OJSXFeqggAAKDu8kQIN6do+8XMBWvtS9baBGttQlhYmAeGhZP6xUdqRv/fKDI4QEZSZHCAZvT/jfrF/+LfWwAAADgDTyxRuFtS1EnbrSXleqBf1DL94iMJ3QAAAB7giTvhn0u6xBjTxhhzgaQhkt72QL8AAABAvVTtO+HW2jJjzFhJqZJ8JP3LWvtNtSsDAAAA6imPvDHTWvuOpHc80RcAAABQ3/HGTAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGGEcAAAAMBhhHAAAADAYYRwAAAAwGHVCuHGmIHGmG+MMRXGmARPFQUAAADUZ9W9E/61pP6SPvZALQAAAECD4Fudg62130mSMcYz1QAAAAANgGNzwo0xfzDGpBtj0g8cOODUsAAAAECtc8Y74caY9yW1OsWuFGvt8rMdyFr7kqSXJCkhIcGedYUAAABAPXPGEG6t7e5EIQAAAEBDwRKFAAAAgMOqu0ThrcaY3ZKuk7TKGJPqmbIAAACA+qu6q6O8JektD9UCAAAANAhMRwEAAAAcRggHAAAAHEYIBwAAABxGCAcAAAAcRggHAAAAHEYIBwAAABxGCAcAAECNi4mJ0fvvv++x/ubPn68uXbp4pC9jzHxjzOPneew0Y8yC459jjDHWGHPGZcAJ4QAAAGgwjDEjjTGfeLsOQjgAAADgMEI4AAAAHPH555+rffv2at68uUaNGqXi4mL9+OOP6tOnj8LCwtS8eXP16dNHu3fvdh8zf/58xcbGKjAwUG3atNHChQtP2ffEiRPVpUsXFRQUqKCgQHfddZfCw8Ml6UpjzOPGGB9jzOWSXpB0nTGm0BiTf1IXocaY94wxR4wxacaY6BM7jDH/zxjzgzHmsDFmszGma3WvBSEcAAAAjli4cKFSU1O1Y8cObdu2TY8//rgqKio0atQoZWdnKycnRwEBARo7dqwk6ejRoxo3bpxWr16tI0eOaP369br66qur9FlRUaExY8boyy+/1LvvvqugoCDdeeed8vX11fbt2yXpW0k9JN1trf1O0r2SNlhrm1prg0/q6nZJf5EUKilT0slp/3NJV0tqIWmRpNeNMRdW51oQwgEAAOCIsWPHKioqSi1atFBKSor+/e9/KyQkRAMGDFDjxo0VGBiolJQUpaWluY9p1KiRvv76axUVFSk8PFxXXHGFe19paamGDh2qvLw8rVixQo0bN9a+ffu0evVq/f3vf1eTJk0kqUzS3yQNOUN5q6y1H1trj0lKUeXd8ihJstYusNYestaWWWtnSfKXFFeda0EIBwAAgCOioqLcn6Ojo5Wbm6uffvpJ99xzj6Kjo9WsWTN169ZN+fn5Ki8vV5MmTbRkyRK98MILCg8PV3JysrZu3eruY/v27Vq+fLkeffRRXXDBBZKk7OxslZaWKjw8XMHBwVLlHewXJbU8Q3k/nPhgrS2UlCcpQpKMMROMMd8ZYwqOT2EJUuUd8/NGCAcAAECNWJbhUucnP1SbSau0t6BYK9Z/5d6Xk5OjiIgIzZo1S1lZWdq0aZMOHz6sjz/+WJJkrZUkJSUl6b333tOePXt02WWXacyYMe4+Lr/8cs2bN0+9evVSVlaWpMqg7+/vr4MHDyo/P1+SMq21zay1J26h29OU6/4XgjGmqSqnnuQen//9Z0mDJDU/PoWlQJKpzrUhhAMAAMDjlmW4NHnpV3LlF8lKKquwmj/nRc1d87ny8vL0xBNPaPDgwTpy5IgCAgIUHBysvLw8TZ8+3d3Hvn379Pbbb+vo0aPy9/dX06ZN5ePjU2WcoUOH6oknnlD37t21Y8cOhYeHq0ePHpowYYIOHz4sSTLGtDXGJJ7oVlJrY8wFPyu5tzGmy/H2v0jaZK39QVKgKqe0HJDka4x5RFKz6l4fQjgAAAA8bmZqlopKy6u0BVzeTePuvE2xsbGKjY3V1KlTNX78eBUVFSk0NFSdOnVSz5493d+vqKjQrFmzFBERoRYtWigtLU3//Oc/fzHWnXfeqUceeUQ33XSTdu3apVdeeUUlJSVq3769VDkd5Q1J4ce//qGkbyTtNcYcPKmbRZIeVeU0lN+q8kFNSUqVtFrSNknZkop10tSV82VO3Op3UkJCgk1PT3d8XAAAADijzaRVp5z3YSR9/2SyY3UYYzZbaxMcG/AscSccAAAAHhcRHHBO7Q0NIRwAAAAeNzEpTgF+VedvB/j5aGJStVb2qzd8vV0AAAAA6p9+8ZGSKueG5+YXKSI4QBOT4tztDR0hHAAAADWiX3wkofs0mI4CAAAAOIwQDgAAADiMEA4AAAA4jBAOAACAasnKylJ8fLwCAwM1e/Zsj/U7f/58denS5byOXbt2rVq3bu3eNsbsMsZ091Rt1cWDmQAAAKiWp59+WjfccIMyMjLOu49du3apTZs2Ki0tla9v/Y+o3AkHAABAtWRnZ+uKK67wdhl1CiEcAAAA5+2mm27SRx99pLFjx6pp06b64osvNGLECIWFhSk6OlqPP/64KioqJEkVFRV6/PHHFR0drZYtW2rEiBEqKCiQJHXr1k2SFBwcrKZNm2rDhg2SJGut7rvvPgUFBemyyy7TBx984B573rx5uvzyyxUYGKjY2Fi9+OKLDp/9+SOEAwAA4Lx9+OGH6tq1q55//nkVFhZq1qxZKigo0M6dO5WWlqZXXnlF8+bNk1Q5x3v+/Pn66KOPtHPnThUWFmrs2LGSpI8//liSlJ+fr8LCQl133XWSpE2bNik2Nn9pJaQAABCJSURBVFYHDx7U9OnT1b9/f+Xl5UmSWrZsqZUrV+rw4cOaN2+eHnjgAW3ZssULV+HcEcIBAADgEeXl5VqyZIlmzJihwMBAxcTEaMKECXr11VclSQsXLtSDDz6o2NhYNW3aVDNmzNDixYtVVlZ22j5btmyp8ePHy8/PT4MHD1ZcXJxWrVolSUpOTlbbtm1ljFFiYqJ69OihdevWOXKu1VX/Z70DAADA45ZluNyvpM/LydcVOT/qloMHVVJSoujoaPf3oqOj5XK5JEm5ubm/2FdWVqZ9+/addpzIyEgZY6ock5ubK0lavXq1pk+frm3btqmiokI//fSTfvOb33j6VGsEd8IBAABwTpZluDR56Vdy5RfJSjpWVq6lW1z65Idj8vPzU3Z2tvu7OTk5ioysfHV9RETEL/b5+vrqoosuqhK0T+ZyuWStrXJMRESEjh07pgEDBuihhx7Svn37lJ+fr969e1f5bm1GCAcAAMA5mZmapaLS8iptJeUVevb97Ro0aJBSUlJ05MgRZWdn69lnn9Udd9whSRo6dKj+9re/6fvvv1dhYaGmTJmiwYMHy9fXV2FhYWrUqJF27txZpd/9+/dr9uzZKi0t1euvv67vvvtOvXv3VklJiY4dO6awsDD5+vpq9erVevfddx27BtXFdBQAAACck9z8otO2r3zuOd13332KjY3VhRdeqDFjxmj06NGSpNGjRys3N1fdunVTcXGxkpKS9Nxzz0mSGjdurJSUFHXu3FmlpaVas2aNJKljx476z3/+o9DQUF100UV64403FBISIkmaPXu2Bg0apGPHjumWW25R3759HTh7zzDeuGWfkJBg09PTHR8XAAAA1df5yQ/lOkUQjwwO0KeTbvJCRadnjNlsrU3wdh0/x3QUAAAAnJOJSXEK8POp0hbg56OJSXFeqqjuYToKAAAAzkm/+MoHLU+sjhIRHKCJSXHudpwZIRwAAADnrF98JKG7GpiOAgAAADiMEA4AAAA4jBBeh/3aK14BAABQexHCa6EtW7YoPj5egYGBGjhwoAYPHqypU6dq7dq1at26tZ566im1atVKo0aNUkVFhZ588km1bdtWISEhGjRokPLy8tx9bdy4Uddff72Cg4N11VVXae3ate59N9xwg/77v/9bnTt3VmBgoHr06KGDBw964YwBAAAalmqFcGPMTGPMVmPMl8aYt4wxwZ4qrKEqKSnRrbfeqpEjRyovL09Dhw7VW2+95d6/d+9e5eXlKTs7Wy+99JJmz56tZcuWKS0tTbm5uWrevLn+9Kc/Sap8zWtycrKmTp2qvLw8PfPMMxowYIAOHDjg7m/RokWaN2+e9u/fr5KSEj3zzDOOnzMAAEBDU9074e9J+i9r7ZWStkmaXP2SGraNGzeqrKxM48aNk5+fn/r3769rr73Wvb9Ro0aaPn26/P39FRAQoBdffFF//etf1bp1a/n7+2vatGl64403VFZWpgULFqh3797q3bu3GjVqpJtvvlkJCQl655133P2NGjVKl156qQICAjRo0CBlZmZ647QBAAAalGotUWitffekzY2SbqteOQ3TsgyXe53NC7I3qHHzMBlj3PujoqLcn8PCwnThhRe6t7Ozs3XrrbeqUaP/+/eUj4+P9u3bp+zsbL3++utasWKFe19paaluvPFG93arVq3cnxs3bqzCwkKPnx8AAACq8uSc8NGSVp9upzHmD8aYdGNM+snTIRq6ZRkuTV76lVz5RbKSCkxT7crZrbe27HZ/54cffnB/PjmcS5UBffXq1crPz3f/FBcXKzIyUlFRURo+fHiVfUePHtWkSZOcOj0AAACcwhlDuDHmfWPM16f4+f1J30mRVCZp4en6sda+ZK1NsNYmhIWFeab6emBmapaKSsvd2/6Rl8mqkSZMe0plZWVavny5Pvvss9Mef++99yolJUXZ2dmSpAMHDmj58uWaNm2aPvvsM61YsUKpqakqLy9XcXGx1q5dq927d5+2PwAAANS8M05HsdZ2/7X9xpg7JfWR9DtrrfVUYQ1Fbn5RlW3j46ewW6do95rZCg6ep169eqlPnz7y9/c/5fH333+/rLXq0aOHcnNz1bJlSw0ePFgXXHCBmjRpouXLl+vhhx/W0KFD5ePjo2uvvVb/8z//48SpAQAA4DRMdXKzMaanpGclJVprz3qOSUJCgk1PTz/vceuTzk9+KNfPgrgkRQYH6NNJN0mSOnbsqDFjxujuu+8+636nTZum7du3a8GCBR6rFQAAoK4xxmy21iZ4u46fq+6c8OclBUp6zxiTaYx5wQM1NSgTk+IU4OdTpc3mfqO7f9tc0dHRGjRokD7//HP98Y9/1CeffHLaNb+///57JSYmKjAwUDfffDPrfQMAANRi1V0dpZ2nCmmo+sVHSpJ7dZSI4ABd5Ws16Y6eOnDggPbv369//etfuvnmm3XllVfq1VdfVc+ePfXBBx9owIAB2rp1q8LCwjRs2DBdd911evfdd7Vp0yYlJyfr97///RlGBwAAgDdUazrK+WI6ytmJiYnRI488otGjR+upp57S119/rVdffdW9PykpScOGDdONN96o2NhYFRQUqEmTJpKkYcOGqVGjRkxHAQAADVptnY5SrTvh8LyT1wzfW1CsnGMBkvSra36feFPmiQAuSdHR0VWWNgQAAEDtQQivRU6sGX5iycKyCqt/fbJLV3dyudf8njNnzi+Oy87O1o8//qijR4+6g3hOTs4v1hQHAABA7eDJl/Wgmn6+ZrgkHSuv0MzULN1xxx2nXfM7OjpaCQkJevTRR1VSUqJPPvmkyh1zAAAA1C6E8Frk52uGn9weFRWl5cuX64knnlBYWJiioqI0c+ZMVVRUSJIWLVqkTZs2qUWLFpo+fbpGjBjhZOkAAAA4BzyYWYuczZrhAAAAOHu19cFM7oTXIqdaMzzAz0cTk+K8VBEAAABqAg9m1iKnWjN8YlKcux0AAAD1AyG8lukXH0noBgAAqOeYjgIAAAA4jBAOAAAAOIwQDgAAADiMEA4AAAA4jBAOAAAAOIwQDgAAADiMEA4AAAA4jBAOAAAAOIwQfpbmz5+vLl26uLeNMdq+fbsXKwIAAEBdRQgHAAAAHEYIBwAAABzW4EL4zJkzNWDAgCpt9913n8aPH6+CggLdddddCg8PV2RkpKZOnary8vIz9llQUKARI0YoLCxM0dHRevzxx1VRUSFJio6O1ubNmyVJCxYskDFG3377rSRp7ty56tevn4fPEAAAALVdgwvhd9xxh9asWaP8/HxJUllZmZYsWaLhw4frzjvvlK+vr7Zv366MjAy9++67mjt37hn7vO+++1RQUKCdO3cqLS1Nr7zyiubNmydJSkxM1Nq1ayVJH3/8sWJjY5WWlubeTkxMrJkTBQAAQK3V4EJ4eHi4unXrptdff12StGbNGoWGhqp169ZavXq1/v73v6tJkyZq2bKlHnjgAS1evPhX+ysvL9eSJUs0Y8YMBQYGKiYmRhMmTNCrr74qqTKEnwjd69at0+TJk93baWlphHAAAIAGyNfbBThhWYZLM1OzlJtfpIjgAHVKvEULFizQmDFjtGDBAg0fPlzZ2dkqLS1VeHi4+7iKigpFRUX9at8HDx5USUmJoqOj3W3R0dFyuVySKkP4Qw89pL1796q8vFyDBw/W9OnTtWvXLhUUFOjqq6+umZMGAABArVXvQ/iyDJcmL/1KRaWVc7td+UVaaVrLlfGFvv76a61cuVJPP/20/Pz85O/vr4MHD8rX9+wvS2hoqPz8/JSdna327dtLknJychQZGSlJateunRo3bqzZs2erW7duCgwMVKtWrfTSSy+pS5cuatSowf0xAgAAoMGr9wlwZmqWO4CfcMz6qMllnTVs2DBde+21uvjiixUeHq4ePXpowoQJOnz4sCoqKrRjxw731JHT8fHx0aBBg5SSkqIjR44oOztbzz77rO644w73dxITE/X888+7p57ccMMNVbYBAADQsNT7EJ6bX3TK9kaXJuqrr77S8OHD3W2vvPKKSkpK1L59ezVv3ly33Xab9uzZc8YxnnvuOTVp0kSxsbHq0qWLhg0bptGjR7v3JyYm6siRI+rWrdsptwEAANCwGGut44MmJCTY9PR0R8bq/OSHcp0iiIfqsL6dfZf27t2rZs2aOVILAAAAnGWM2WytTfB2HT9X7++ET0yKU4CfT5W2C32NwnM+0JAhQwjgAAAAcFy9fzCzX3zlA5InVke5qLH0xZMD5dMmRv9cs8a7xQEAAKBBqvfTUQAAANBwMR0FAAAAgCRCOAAAAOA4QjgAAADgMEI4AAAA4DBCOAAAAOAwQjgAAADgMEI4AAAA4DBCOAAAAOAwQjgAAADgMEI4AAAA4DCvvLbeGHNAUrbjA3tPqKSD3i6iAeA61zyucc3jGtc8rnHN4xrXPK7x2Yu21oZ5u4if80oIb2iMMenW2gRv11HfcZ1rHte45nGNax7XuOZxjWse17juYzoKAAAA4DBCOAAAAOAwQrgzXvJ2AQ0E17nmcY1rHte45nGNax7XuOZxjes45oQDAAAADuNOOAAAAOAwQrhDjDEzjTFbjTFfGmPeMsYEe7um+sYYM9AY840xpsIYwxPjHmSM6WmMyTLGbDfGTPJ2PfWRMeZfxpj9xpivvV1LfWWMiTLGfGSM+e7474r7vV1TfWOMudAY85kx5ovj13i6t2uqr4wxPsaYDGPMSm/XgvNDCHfOe5L+y1p7paRtkiZ7uZ766GtJ/SV97O1C6hNjjI+kf0jqJam9pKHGmPberapemi+pp7eLqOfKJE2w1l4uqZOkP/Hfsscdk3STtfYqSVdL6mmM6eTlmuqr+yV95+0icP4I4Q6x1r5rrS07vrlRUmtv1lMfWWu/s9ZmebuOeuhaSduttTuttSWSFkv6vZdrqnestR9LyvN2HfWZtXaPtXbL8c9HVBlgIr1bVf1iKxUe3/Q7/sPDZx5mjGktKVnSXG/XgvNHCPeO0ZJWe7sI4CxFSvrhpO3dIrigjjPGxEiKl7TJu5XUP8enSWRK2i/pPWst19jz/i7pYUkV3i4E58/X2wXUJ8aY9yW1OsWuFGvt8uPfSVHln0QXOllbfXE21xgeZ07Rxp0t1FnGmKaS3pQ03lp72Nv11DfW2nJJVx9/9uktY8x/WWt51sFDjDF9JO231m42xtzg7Xpw/gjhHmSt7f5r+40xd0rqI+l3lrUhz8uZrjFqxG5JUSdtt5aU66VagGoxxvipMoAvtNYu9XY99Zm1Nt8Ys1aVzzoQwj2ns6S+xpjeki6U1MwYs8Bae4eX68I5YjqKQ4wxPSX9WVJfa+1P3q4HOAefS7rEGNPGGHOBpCGS3vZyTcA5M8YYSf8r6Ttr7bPerqc+MsaEnVj9yxgTIKm7pK3erap+sdZOtta2ttbGqPL38YcE8LqJEO6c5yUFSnrPGJNpjHnB2wXVN8aYW40xuyVdJ2mVMSbV2zXVB8cfKB4rKVWVD7K9Zq39xrtV1T/GmH9L2iApzhiz2xhzl7drqoc6Sxou6abjv4czj99NhOeES/rIGPOlKv8B/561liX0gFPgjZkAAACAw7gTDgAAADiMEA4AAAA4jBAOAAAAOIwQDgAAADiMEA4AAAA4jBAOAAAAOIwQDgAAADiMEA4AAAA47P8DOexqc8Ezk2sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_words(embedding, sequences, 0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Plot the word of embeddings of the second sentence. Recall that the second sentence contains the same words are the first sentence, just in a different order. You can see that the order of the words does not affect the vector representations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAuQAAAFoCAYAAAAfPu5bAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de1hVZf7//9ctMISKoIIKSCBadvhNRWFWHrA+Jnkc0zymppZTn8mcynE+mk5p42RlzszPmvlUOqPloexgmpraaSTLQ6FYViOOJ7CNmoqgKGfu7x/o/khppmy4EZ6P69rXtde91rrXe61ru6+Xi3vd21hrBQAAAMCNOq4LAAAAAGozAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADvm7LiAsLMzGxsa6LgMAAAA12KZNmw5Za8Nd13EmzgN5bGysUlJSXJcBAACAGswYk+66hrNhyAoAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5ABqjNTUVF1//fUKDg7WgAEDNHDgQE2aNElz585V+/bty21rjNGOHTskSQUFBfrd736nSy+9VE2bNtUDDzygvLw877bLly/Xddddp9DQUN1yyy366quvvOtiY2P13HPP6ZprrlFISIgGDBig/Pz8qjlhAECNQCAHUCMUFhaqd+/eGjp0qLKystSvXz+9/fbbP2vf//mf/9H27du1ZcsW7dixQx6PR08++aQkafPmzRo5cqReeuklHT58WPfff7969eqlgoIC7/5vvPGGVq1apd27d+urr77S3LlzK+MUAQA1FIEcQI2wYcMGFRUV6eGHH1ZAQIDuuusutWnT5pz7WWs1a9Ys/eUvf1GjRo0UHBysxx57TK+//rokadasWbr//vvVtm1b+fn56Z577lFgYKA2bNjg7WPMmDGKjIxUo0aN1LNnT23ZsqXSzhMAUPM4/6VOAKiIJakeTV+dpu3rVimvTgMt3ZKp3vFRkqSYmJhz7n/w4EGdOHFCN9xwg7fNWquSkhJJUnp6ul555RU9//zz3vWFhYXKzMz0Ljdr1sz7vm7duuXWAQBwLgRyABetJakeTVi8VXlFJfKr30j5OQc1/u2y8d2946OUkZGhli1bql69ejpx4oR3v/3793vfh4WFKSgoSN98842ioqJ+dIzo6GhNnDhREydOrPwTAgDUSgxZAXDRmr46TXlFZXeyA6OukIyfvt/wjp5971stXrxYn3/+uSTp2muv1TfffKMtW7YoPz9fkydP9vZRp04djRo1So888oi+//57SZLH49Hq1aslSaNGjdKLL76ojRs3ylqr48ePa8WKFTp27FjVniwAoMYikAO4aGVm/99MKMYvQOF3PqbcrR9pw5O/0qJFi9SnTx9J0uWXX67HH39cnTt31mWXXfajGVeeeeYZtWrVSjfddJMaNGigzp07Ky0tTZKUkJCgWbNmafTo0WrYsKFatWrFQ5sAAJ8y1lqnBSQkJNiUlBSnNQC4OMTGxmr27Nnq3LmzJKnd0x/Lk52nzNm/UaMuD+iSS6+RJEWFBumz8bdp+PDhat68uaZOneqybABANWCM2WStTXBdx5lwhxzARWtcUmsFBfgp8r6/e8N4UICfxiW1dlwZAAA/Hw91ArhonZpNZfrqNGVm5ykyNEjjklp72wEAuBhwhxzARWnbtm1q0aKF8tPWyvPiSM3qHKjPxt+mLUtnqX///ho2bJjefvttvfPOOzp9WNzmzZsVHx+v4OBg9evXTwMGDNCkSZMcngkAoLYjkAO46GzevFldunTR888/r4EDB/5o/bvvvquBAwcqOztbvXr10ujRoyWVzR9+5513avjw4crKytKgQYP0zjvvVHX5AACUQyAHcFFZu3atevXqpVdeeUU9evQ44zbt27dXt27d5Ofnp6FDh+rLL7+UVPZrnsXFxRozZowCAgLUp08f3XjjjVVZPgAAP+LTQG6MmW+M2WeMOWqM2W6Muc+X/QPAiy++qFtuuUW33nrrWbf54S9n5ufnq7i4WJmZmYqKipIxxrs+Ojq6UusFAOBcfH2HfJqkWGttA0m9JE01xtxwjn0A4KyWpHrU7umP1WL8Cu3PydeI3/9JGRkZeuSRR867r4iICHk8Hp0+3evevXt9WS4AAOfNp4HcWvuNtbbg1OLJV0tfHgNA7bEk1aMJi7fKk50nK6m41Grh5oN66Nl/6pNPPtH48ePPq7+bb75Zfn5+euGFF1RcXKylS5d6f80TAABXfD7toTHm75KGSwqSlCrpvTNs82tJv5akSy+91NclAKghpq9OU15RSbm2gpJS/X3dfn3wwQe69dZbFRAQ8LP7+8UvfqHFixfrvvvu04QJE9S1a1f16NFDgYGBvi4dAICfrVJ+qdMY4yfpZkmdJD1jrS0627b8UieAs2kxfoXO9A1lJO1+urtPjtG2bVs98MADGjFihE/6AwBUT7XulzqttSXW2k8lNZf035VxDAA1X2Ro0Hm1/xzJycnav3+/iouL9corr+irr77SHXfcccH9AQBQUZU97aG/GEMO4AKNS2qtoAC/cm1BAX4al9T6gvtMS0vTtddeq5CQEM2YMUNvvfWWIiIiKloqAAAXzGdDVowxTSTdJmm5pDxJnSUtljTYWrv0bPsxZAXAT1mS6tH01WnKzM5TZGiQxiW1Vu/4KNdlAQAuMtV5yIovH+q0Khue8qLK7rynS3r4p8I4AJxL7/goAjgAoEbzWSC31h6UlOir/gAAAIDaoLLHkAMAAAD4CQRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHCKQAwAAAA4RyAEAAACHCOQAAACAQwRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5EA1MGfOHPXs2dO73KpVK/Xv39+7HB0drS1btmjdunVq06aNQkJC1KZNG61bt867TadOnTRp0iTdcsstql+/vnr27KnDhw/r7rvvVoMGDdSmTRvt2bPHu/1vf/tbRUdHq0GDBrrhhhu0du1a77rJkyerf//+GjZsmIKDg3X11VcrJSWlci8CAAC1lM8CuTEm0BjzD2NMujHmmDEm1RjT1Vf9AzVZYmKi1q5dq9LSUu3bt09FRUX67LPPJEm7du1Sbm6uLr30UnXv3l1jxozR4cOH9eijj6p79+46fPiwt5/XX39d8+bNk8fj0c6dO3XzzTdrxIgRysrK0pVXXqkpU6Z4t23Tpo22bNmirKwsDR48WP369VN+fr53/bvvvquBAwcqOztbvXr10ujRo6vuggAAUIv48g65v6S9khIlhUj6g6Q3jDGxPjwGUCPFxcUpODhYW7ZsUXJyspKSkhQVFaVt27YpOTlZHTp00IoVK3TZZZdp6NCh8vf316BBg3TFFVdo2bJl3n5GjBihli1bKiQkRF27dlXLli3VuXNn+fv7q1+/fkpNTfVuO2TIEDVu3Fj+/v4aO3asCgoKlJaW5l3fvn17devWTX5+fho6dKi+/PLLKr0mAADUFv6+6shae1zS5NOalhtjdku6QdIeXx0HqCmWpHo0fXWaMrPzFBkapLhfttGaNWu0Y8cOJSYmKjQ0VMnJyVq/fr0SExOVmZmpmJiYcn3ExMTI4/F4l5s2bep9HxQU9KPl3Nxc7/KMGTM0e/ZsZWZmyhijo0eP6tChQ971zZo1876vW7eu8vPzVVxcLH9/n31tAAAAVeIYcmNMU0mXS/rmDOt+bYxJMcakHDx4sLJKAKqtJakeTVi8VZ7sPFlJnuw8bTfRen3pKq1du1aJiYlKTExUcnKykpOTlZiYqMjISKWnp5frJyMjQ1FRUed9/LVr1+qZZ57RG2+8oSNHjig7O1shISGy1vroDAEAwM9VKYHcGBMgaYGkV6y123643lr7srU2wVqbEB4eXhklANXa9NVpyisqKddWJ+pqbdrwqfLy8tS8eXN16NBBq1at0uHDhxUfH69u3bpp+/btWrhwoYqLi7Vo0SJ9++236tGjx3kf/9ixY/L391d4eLiKi4v15JNP6ujRo746PQAAcB58HsiNMXUkzZNUKImnwIAzyMzO+1FbQKMomYBL1KFDB0lSgwYNFBcXp3bt2snPz0+NGzfW8uXLNWPGDDVu3FjPPvusli9frrCwsPM+flJSkrp27arLL79cMTExuuSSSxQdHV3h8wIAAOfP+PJP1MYYI+mfkmIldbPW/jh1/EBCQoJlOjXUNu2e/lieM4TyqNAgfTb+NgcVAQBQsxljNllrE1zXcSa+vkP+v5KulNTz54RxoLYal9RaQQF+5dqCAvw0Lqm1o4oAAIArPpsuwRgTI+l+SQWS9pfdLJck3W+tXeCr4wA1Qe/4sgcxT59lZVxSa287AACoPXw57WG6JHPODQFIKgvlBHAAAFBp0x4CAAAAODcCOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHCKQAzVEbGyspk+frmuuuUb16tXTvffeqwMHDqhr164KDg5W586ddeTIEUlSv3791KxZM4WEhKhjx4765ptvvP0MHz5cDz74oLp3767g4GC1bdtWO3fudHVaAADUeARyoAZ5++239cEHH2j79u1atmyZunbtqqeeekqHDh1SaWmpZs6cKUnq2rWr/vOf/+j777/X9ddfr7vvvrtcP6+99pqeeOIJHTlyRK1atdLEiRNdnA4AALWCv+sCAPjOQw89pKZNm0qSOnTooCZNmig+Pl6SdOedd+qjjz6SJI0cOdK7z+TJk9WwYUPl5OQoJCREktSnTx/deOONkqS7775bjz76aFWeBgAAtQqBHLhILUn1aPrqNGVm5ykyNEgnCku8YVySgoKCfrScm5urkpISTZw4UW+++aYOHjyoOnXK/lB26NAhbyBv1qyZd7+6desqNze3is4KAIDahyErwEVoSapHExZvlSc7T1aSJztPR04U6rMdh86578KFC7V06VJ9+OGHysnJ0Z49eyRJ1trKLRoAAJwRgRy4CE1fnaa8opJybdZKb6TsPee+x44dU2BgoBo3bqwTJ07oscceq6wyAQDAz0AgBy5Cmdl5Z2w/nFt4zn2HDRummJgYRUVF6aqrrtJNN93k6/IAAMB5MK7/TJ2QkGBTUlKc1gBcbNo9/bE8ZwjlUaFB+mz8bQ4qAgCgejPGbLLWJriu40y4Qw5chMYltVZQgF+5tqAAP41Lau2oIgAAcKGYZQW4CPWOj5KkcrOsjEtq7W0HAAAXDwI5cJHqHR9FAAcAoAZgyAoAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4JBPA7kxZrQxJsUYU2CMmevLvgEAAICayN/H/WVKmiopSVKQj/sGAAAAahyfBnJr7WJJMsYkSGruy74BAACAmsjJGHJjzK9PDm1JOXjwoIsSAAAAgGrBSSC31r5srU2w1iaEh4e7KAEAAACoFphlBQAAAHCIQA4AAAA45NOHOo0x/if79JPkZ4y5RFKxtbbYl8cBAAAAagpf3yGfJClP0nhJQ06+n+TjYwAAAAA1hq+nPZwsabIv+wQAAABqMsaQAwAAAA4RyAEAAACHCOQAAACAQwRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikKNaGT58uCZN4sddAQBA7UEgBwAAABwikAMAAAAOEcjhVGpqqq6//noFBwdrwIABys/P966bNWuWWrVqpUaNGqlXr17KzMz0rnv//ffVunVrhYSE6De/+Y0SExM1e/ZsF6cAAABQIQRyOFNYWKjevXtr6NChysrKUr9+/fT2229Lkj7++GNNmDBBb7zxhvbt26eYmBgNHDhQknTo0CHdddddmjZtmg4fPqzWrVtr3bp1Lk8FAADgghHI4cyGDRtUVFSkhx9+WAEBAbrrrrvUpk0bSdKCBQs0cuRIXX/99QoMDNS0adO0fv167dmzR++9956uvvpq9enTR/7+/hozZoyaNWvm+GwAAAAuDIEczmRmZioqKkrGGG9bTEyMd92p95JUv359NW7cWB6PR5mZmYqOjvauM8aoefPmVVc4AACAD/m7LgC1y5JUj6avTlNmdp7qZe1T5p69stZ6Q3lGRoZatmypyMhIpaene/c7fvy4Dh8+rKioKEVEROi7777zrrPWllsGAAC4mHCHHFVmSapHExZvlSc7T1bSsZAWOlZYqvt+/6SKi4u1ePFiff7555KkwYMHa86cOdqyZYsKCgr02GOPqW3btoqNjVX37t21detWLVmyRMXFxfrb3/6m/fv3uz05AACAC0QgR5WZvjpNeUUl3mXjF6Cw3o9p0YL5atiwoRYtWqQ+ffpIkv7rv/5Lf/zjH9W3b19FRERo586dev311yVJYWFhevPNN/X73/9ejRs31rfffquEhAQFBgY6OS8AAICKMNZapwUkJCTYlJQUpzWgarQYv0Jn+rQZSbuf7n7B/ZaWlqp58+ZasGCBbr311gvuBwAA1FzGmE3W2gTXdZwJd8hRZSJDg86r/aesXr1a2dnZKigo0FNPPSVrrW666aaKlggAAFDlCOSoMuOSWisowK9cW1CAn8YltT7vvtavX6+WLVsqLCxMy5Yt05IlSxQUdP7BHgAAwDWGrKBKnT7LSmRokMYltVbv+CjXZQEAgBquOg9ZYdpDVKne8VEEcAAAgNMwZAUAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhAjmqhU6dOmj17tnd50qRJCgsLU7NmzRxWBQAAUPn8XRcA/NDevXs1Y8YMpaenq0mTJq7LAQAAqFTcIUe1k56ersaNGxPGAQBArUAgR4XNmTNHPXv29C63atVK/fv39y5HR0dry5YtWrdundq0aaOQkBC1adNG69at+1FfH374oW6//XZlZmaqfv36Gj58eFWcAgAAgDMEclRYYmKi1q5dq9LSUu3bt09FRUX67LPPJEm7du1Sbm6uLr30UnXv3l1jxozR4cOH9eijj6p79+46fPhwub46d+6slStXKjIyUrm5uZo7d66DMwIAAKg6BHJUWFxcnIKDg7VlyxYlJycrKSlJUVFR2rZtm5KTk9WhQwetWLFCl112mYYOHSp/f38NGjRIV1xxhZYtW+a6fAAAAKd4qBM+kZiYqDVr1mjHjh1KTExUaGiokpOTtX79eiUmJiozM1MxMTHl9omJiZHH43FUMQAAQPVAIMcFWZLq0fTVacrMzlNkaJCubXGN1qxZo927d+uxxx5TaGioFixYoPXr12v06NH697//rfT09HJ9ZGRk6I477nB0BgAAANUDQ1Zw3pakejRh8VZ5svNkJXmy87T6cCN98NHHysvLU/PmzdWhQwetWrVKhw8fVnx8vLp166bt27dr4cKFKi4u1qJFi/Ttt9+qR48erk8HAADAKZ8GcmNMI2PMO8aY48aYdGPMYF/2j+ph+uo05RWVlGsraRChEr9AdejQQZLUoEEDxcXFqV27dvLz81Pjxo21fPlyzZgxQ40bN9azzz6r5cuXKywszMUpAAAAVBvGWuu7zox5TWUh/15J10laIekWa+03Z9snISHBpqSk+KwGVL4W41foTJ8aI2n3092ruhwAAIBzMsZsstYmuK7jTHx2h9wYU09SX0l/sNbmWms/lfSupKG+Ogaqh8jQoPNqBwAAwNn5csjK5ZJKrLXbT2v7UtLVP9zQGPNrY0yKMSbl4MGDPiwBVWFcUmsFBfiVawsK8NO4pNaOKgIAALh4+TKQ15eU84O2HEnBP9zQWvuytTbBWpsQHh7uwxJQFXrHR2lan18qKjRIRlJUaJCm9fmlesdHuS4NAADgouPLaQ9zJTX4QVsDScd8eAxUE73jowjgAAAAPuDLO+TbJfkbYy47re1aSWd9oBMAAACo7XwWyK21xyUtlvSkMaaeMaadpF9JmuerYwAAAAA1ja9/GOg3koIkfS/pNUn//VNTHgIAAAC1nS/HkMtamyWpty/7BAAAAGoyX98hBwAAAHAeCOQAAACAQwRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHCKQAwAAAA4RyAEAAACHCOQAAACAQwRyAAAAwCECOQAAAOAQgRwAAABwiEAOAAAAOEQgBwAAABwikAMAAAAOEcgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAcIhADgAAADhEIAcAAAAcIpADAAAADhHIAQAAAId8EsiNMaONMSnGmAJjzFxf9AkAAADUBv4+6idT0lRJSZKCfNQnAAAAUOP5JJBbaxdLkjEmQVJzX/QJAAAA1AZOxpAbY359cohLysGDB12UAAAAAFQLTgK5tfZla22CtTYhPDzcRQkAAABAtXDOQG6MWWOMsWd5fVoVRQIAAAA11TnHkFtrO1VBHQAAAECt5JOHOo0x/if78pPkZ4y5RFKxtbbYF/0DAAAANZWvxpBPkpQnabykISffT/JR3wAAAECN5atpDydLmuyLvgAAAIDaxMksKwAAAADKEMgBAAAAhwjkAAAAgEMEcgAAAMAhAjkAAADgEIEcAAAAVSo2NlYffvihz/qbO3eu2rdv75O+jDFzjTFTL3DfycaY+Sffx578ZftzzmpIIAcAAECtZIwZboz51HUdBHIAAADAIQI5AAAAqtwXX3yhq666Sg0bNtSIESOUn5+vI0eOqEePHgoPD1fDhg3Vo0cPfffdd9595s6dq7i4OAUHB6tFixZasGDBGfseN26c2rdvr5ycHOXk5Ojee++VpGuMMR5jzFRjjJ8x5kpJL0q62RiTa4zJPq2LMGPMB8aYY8aYZGNMzKkVxpj/3xiz1xhz1BizyRjToaLXgkAOAACAKrdgwQKtXr1aO3fu1Pbt2zV16lSVlpZqxIgRSk9PV0ZGhoKCgjR69GhJ0vHjxzVmzBitXLlSx44d07p163TdddeV67O0tFSjRo3SV199pffff18hISG655575O/vL0lfS4qX1EXSfdbaf0t6QNJ6a219a23oaV3dLemPksIkbZF0evL/QtJ1khpJWijpTWPMJRW5FgRyAAAAVLnRo0crOjpajRo10sSJE/Xaa6+pcePG6tu3r+rWravg4GBNnDhRycnJ3n3q1Kmjr7/+Wnl5eYqIiNDVV1/tXVdUVKRBgwYpKytLy5YtU926dXXgwAGtXLlSf/3rXyWp1Fr7vaS/SBp4jvJWWGs/sdYWSJqosrvo0ZJkrZ1vrT1srS221s6QFCipdUWuBYEcAAAAVS46Otr7PiYmRpmZmTpx4oTuv/9+xcTEqEGDBurYsaOys7NVUlKievXqadGiRXrxxRcVERGh7t27a9u2bd4+duzYoaVLl+qJJ57QL37xC0lSenq6ioqKFBERIUnXnRyW8pKkJucob++pN9baXElZkiIlyRgz1hjzb2NMzsn+QlR2J/2CEcgBAABQ6ZaketTu6Y/VYvwK7c/J17J1W73rMjIyFBkZqRkzZigtLU0bN27U0aNH9cknn0iSrLWSpKSkJH3wwQfat2+frrjiCo0aNcrbx5VXXqk5c+aoa9euSktLk1QW+gMDA3Xo0CFJ2mKtDbXWNrDWnrq1bs9Srvd/C8aY+iobnpJ5crz4/0jqL6nhyWEuOZJMRa4NgRwAAACVakmqRxMWb5UnO09WUnGp1dxZL2n2qi+UlZWlp556SgMGDNCxY8cUFBSk0NBQZWVlacqUKd4+Dhw4oHfffVfHjx9XYGCg6tevLz8/v3LHGTRokJ566il17txZO3fuVEREhLp06aKxY8dKUh1jTB1jTEtjTOKpbiU1N8b84gcldzPGtD/Z/kdJG621eyUFSyqWdFCSvzHmcUkNKnp9COQAAACoVNNXpymvqKRcW9CVHTXmnrsUFxenuLg4TZo0SQ8//LDy8vIUFhamm266SXfccYd3+9LSUs2YMUORkZFq1KiRkpOT9fe///1Hx7rnnnv0+OOP67bbbtOePXv06quvqrCwUJL+P0lHJL0lKeLk5h9L+kbSfmPModO6WSjpCZUNVblBZQ95StJqSSslbZeULilfpw1vuVDm1J8AXElISLApKSlOawAAAEDlaTF+xRnHhhhJu5/uXiU1GGM2WWsTquRg54k75AAAAKhUkaFB59Ve2xDIAQAAUKnGJbVWUED58d5BAX4al1Sh2QJrDH/XBQAAAKBm6x0fJalsLHlmdp4iQ4M0Lqm1t722I5ADAACg0vWOjyKAnwVDVgAAAACHCOQAAACAQwRyAAAAwCECOQAAAHwmLS1N8fHxCg4O1syZM33W79y5c9W+ffsL2nfNmjWSdM2pZWPMHmNMZ99UVnE81AkAAACfefbZZ9WpUyelpqZecB979uxRixYtVFRUJH//mh9XuUMOAAAAn0lPT9fVV1/tuoyLCoEcAAAAPnHbbbfpX//6l0aPHq369evryy+/1LBhwxQeHq6YmBhNnTpVpaWlkqTS0lJNnTpVMTExatKkiYYNG6acnBxJUtfjSxMAABFYSURBVMeOHSVJoaGhql+/vtavXy9JstbqoYceUkhIiK644gp99NFH3mPPmTNHV155pYKDgxUXF6eXXnqpis/+whHIAQAA4BMff/yxOnTooBdeeEG5ubmaMWOGcnJytGvXLiUnJ+vVV1/VnDlzJJWNCZ87d67+9a9/adeuXcrNzdXo0aMlSZ988okkKTs7W7m5ubr55pslSRs3blRcXJwOHTqkKVOmqE+fPsrKypIkNWnSRMuXL9fRo0c1Z84cPfLII9q8ebODq3D+COQAAADwuZKSEi1atEjTpk1TcHCwYmNjNXbsWM2bN0+StGDBAj366KOKi4tT/fr1NW3aNL3++usqLi4+a59NmjTRww8/rICAAA0YMECtW7fWihUrJEndu3dXy5YtZYxRYmKiunTporVr11bJuVZUzR8lDwAAgEq1JNWj6avTlJmdp6yMbF2dcUQ9Dx1SYWGhYmJivNvFxMTI4/FIkjIzM3+0rri4WAcOHDjrcaKiomSMKbdPZmamJGnlypWaMmWKtm/frtLSUp04cUK//OUvfX2qlYI75AAAALhgS1I9mrB4qzzZebKSCopLtHizR5/uLVBAQIDS09O922ZkZCgqKkqSFBkZ+aN1/v7+atq0abnQfTqPxyNrbbl9IiMjVVBQoL59++p3v/udDhw4oOzsbHXr1q3cttUZgRwAAAAXbPrqNOUVlZRrKywp1Z8/3KH+/ftr4sSJOnbsmNLT0/XnP/9ZQ4YMkSQNGjRIf/nLX7R7927l5ubqscce04ABA+Tv76/w8HDVqVNHu3btKtfv999/r5kzZ6qoqEhvvvmm/v3vf6tbt24qLCxUQUGBwsPD5e/vr5UrV+r999+vsmtQUQxZAQAAwAXLzM47a/vy55/XQw89pLi4OF1yySUaNWqURo4cKUkaOXKkMjMz1bFjR+Xn5yspKUnPP/+8JKlu3bqaOHGi2rVrp6KiIq1atUqS1LZtW/3nP/9RWFiYmjZtqrfeekuNGzeWJM2cOVP9+/dXQUGBevbsqV69elXB2fuGcX0rPyEhwaakpDitAQAAABem3dMfy3OGUB4VGqTPxt/moKIzM8ZsstYmuK7jTBiyAgAAgAs2Lqm1ggL8yrUFBfhpXFJrRxVdfBiyAgAAgAvWO77sIc1Ts6xEhgZpXFJrbzvOjUAOAACACukdH0UArwCGrAAAAAAOEcgBAAAAhwjkNcRP/cwsAAAAqi8CeTW3efNmxcfHKzg4WP369dOAAQM0adIkrVmzRs2bN9czzzyjZs2aacSIESotLdXTTz+tli1bqnHjxurfv7+ysrK8fW3YsEG33HKLQkNDde2112rNmjXedZ06ddIf/vAHtWvXTsHBwerSpYsOHTrk4IwBAABqlwoHcmNMoDHmH8aYdGPMMWNMqjGmqy+Kq+0KCwt15513avjw4crKytKgQYP0zjvveNfv379fWVlZSk9P18svv6yZM2dqyZIlSk5OVmZmpho2bKgHH3xQUtlPzXbv3l2TJk1SVlaWnnvuOfXt21cHDx709rdw4ULNmTNH33//vQoLC/Xcc89V+TkDAADUNr64Q+4vaa+kREkhkv4g6Q1jTKwP+q7VNmzYoOLiYo0ZM0YBAQHq06ePbrzxRu/6OnXqaMqUKQoMDFRQUJBeeukl/elPf1Lz5s0VGBioyZMn66233lJxcbHmz5+vbt26qVu3bqpTp45uv/12JSQk6L333vP2N2LECF1++eUKCgpS//79tWXLFhenDQAAUKtUeNpDa+1xSZNPa1pujNkt6QZJeyraf22zJNXjncfzF+nrVbdhuIwx3vXR0dHe9+Hh4brkkku8y+np6brzzjtVp87//T/Lz89PBw4cUHp6ut58800tW7bMu66oqEi33nqrd7lZs2be93Xr1lVubq7Pzw8AAADl+XwMuTGmqaTLJX3zE9v82hiTYoxJOX3IRG23JNWjCYu3ypOdJyspx9TXnozv9M7m77zb7N271/v+9KAulYX1lStXKjs72/vKz89XVFSUoqOjNXTo0HLrjh8/rvHjx1fV6QEAAOAMfBrIjTEBkhZIesVau+1s21lrX7bWJlhrE8LDw31ZwkVt+uo05RWVeJcDo66QVR2NnfyMiouLtXTpUn3++edn3f+BBx7QxIkTlZ6eLkk6ePCgli5dqsmTJ+uLL77QsmXLtHr1apWUlCg/P19r1qzRd999d9b+AAAAUPnOGciNMWuMMfYsr09P266OpHmSCiWNrsSaa6zM7Lxyy8YvQOF3PqbvNq5QaGio5s+frx49eigwMPCM+//2t79Vr1691KVLFwUHB+umm27Sxo0bJZUNQVm6dKmeeuophYeHKzo6WtOnT1dpaWmlnxcAAADOzlhrK95J2diJf0qKldTNWpv303v8n4SEBJuSklLhGmqCdk9/LE/2jy9dVGiQPht/mySpbdu2GjVqlO67776f3e/kyZO1Y8cOzZ8/32e1AgAAXEyMMZustQmu6zgTXw1Z+V9JV0rqeT5hHOWNS2qtoAC/cm028xvdd0NDxcbGqn///vriiy/04IMP6tNPPz3rnOK7d+9WYmKigoODdfvttzOfOAAAQDVW4VlWjDExku6XVCBp/2kPGt5vrV1Q0f5rk97xUZLknWUlMjRI1/pbjR9yhw4ePKgDBw7on//8p26//XZdc801mjdvnu644w599NFH6tu3r7Zt26bw8HANHjxYN998s95//31t3LhR3bt3169+9SvHZwcAAIAz8cmQlYpgyMrPExsbq8cff1wjR47UM888o6+//lrz5s3zrk9KStLgwYN16623Ki4uTjk5OapXr54kafDgwapTpw5DVgAAQK1VnYesVPgOOSrP6XOS78/JV0ZBkCT95Jzip36h81QYl6SYmJhy0yUCAACg+iCQV1On5iQ/NQ1icanVPz/do+tu8njnFJ81a9aP9ktPT9eRI0d0/PhxbyjPyMj40ZzlAAAAqB58/sNA8I0fzkkuSQUlpZq+Ok1Dhgw565ziMTExSkhI0BNPPKHCwkJ9+umn5e6kAwAAoHohkFdTP5yT/PT26Ojon5xTfOHChdq4caMaNWqkKVOmaNiwYVVZOgAAAM4DD3VWUz9nTnIAAAD8PNX5oU7ukFdTZ5qTPCjAT+OSWjuqCAAAAJWBhzqrqTPNST4uqbW3HQAAADUDgbwa6x0fRQAHAACo4RiyAgAAADhEIAcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQH4B5s6dq/bt23uXjTHasWOHw4oAAABwsSKQAwAAAA4RyAEAAACHanUgnz59uvr27Vuu7aGHHtLDDz+snJwc3XvvvYqIiFBUVJQmTZqkkpKSc/aZk5OjYcOGKTw8XDExMZo6dapKS0slSTExMdq0aZMkaf78+TLG6Ntvv5UkzZ49W7179/bxGQIAAKC6q9WBfMiQIVq1apWys7MlScXFxVq0aJGGDh2qe+65R/7+/tqxY4dSU1P1/vvva/bs2efs86GHHlJOTo527dql5ORkvfrqq5ozZ44kKTExUWvWrJEkffLJJ4qLi1NycrJ3OTExsXJOFAAAANVWrQ7kERER6tixo958801J0qpVqxQWFqbmzZtr5cqV+utf/6p69eqpSZMmeuSRR/T666//ZH8lJSVatGiRpk2bpuDgYMXGxmrs2LGaN2+epLJAfiqAr127VhMmTPAuJycnE8gBAABqIX/XBVS1JakeTV+dpszsPEWGBummxJ6aP3++Ro0apfnz52vo0KFKT09XUVGRIiIivPuVlpYqOjr6J/s+dOiQCgsLFRMT422LiYmRx+ORVBbIf/e732n//v0qKSnRgAEDNGXKFO3Zs0c5OTm67rrrKuekAQAAUG3VqkC+JNWjCYu3Kq+obCy4JztPy01zeVK/1Ndff63ly5fr2WefVUBAgAIDA3Xo0CH5+//8SxQWFqaAgAClp6frqquukiRlZGQoKipKktSqVSvVrVtXM2fOVMeOHRUcHKxmzZrp5ZdfVvv27VWnTq3+gwUAAECtVKsS4PTVad4wfkqB9VO9K9pp8ODBuvHGG3XppZcqIiJCXbp00dixY3X06FGVlpZq586d3uElZ+Pn56f+/ftr4sSJOnbsmNLT0/XnP/9ZQ4YM8W6TmJioF154wTs8pVOnTuWWAQAAULvUqkCemZ13xvY6lydq69atGjp0qLft1VdfVWFhoa666io1bNhQd911l/bt23fOYzz//POqV6+e4uLi1L59ew0ePFgjR470rk9MTNSxY8fUsWPHMy4DAACgdjHWWqcFJCQk2JSUlCo5VrunP5bnDKE8TEf17cx7tX//fjVo0KBKagEAAEDVMcZsstYmuK7jTGrVHfJxSa0VFOBXru0Sf6OIjI80cOBAwjgAAACqXK16qLN3fNnDladmWWlaV/ry6X7yaxGrv69a5bY4AAAA1Eq1asgKAAAAaieGrAAAAAA4IwI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHCKQAwAAAA4RyAEAAACHCOQAAACAQwRyAAAAwCECOQAAAOCQsda6LcCYg5LSnRZRtcIkHXJdRA3HNa4aXOfKxzWufFzjysc1rnxc458nxlob7rqIM3EeyGsbY0yKtTbBdR01Gde4anCdKx/XuPJxjSsf17jycY0vfgxZAQAAABwikAMAAAAOEcir3suuC6gFuMZVg+tc+bjGlY9rXPm4xpWPa3yRYww5AAAA4BB3yAEAAACHCOQAAACAQwRyB4wxgcaYfxhj0o0xx4wxqcaYrq7rqmmMMaONMSnGmAJjzFzX9dQUxphGxph3jDHHT36GB7uuqSbhc1v5+A6uGsaY+caYfcaYo8aY7caY+1zXVFMZYy4zxuQbY+a7rgUXxt91AbWUv6S9khIlZUjqJukNY8wvrbV7XBZWw2RKmiopSVKQ41pqkr9JKpTUVNJ1klYYY7601n7jtqwag89t5eM7uGpMk3SvtbbAGHOFpDXGmFRr7SbXhdVAf5P0hesicOG4Q+6Atfa4tXaytXaPtbbUWrtc0m5JN7iurSax1i621i6RdNh1LTWFMaaepL6S/mCtzbXWfirpXUlD3VZWc/C5rXx8B1cNa+031tqCU4snXy0dllQjGWMGSsqW9JHrWnDhCOTVgDGmqaTLJXGHEdXd5ZJKrLXbT2v7UtLVjuoBKozv4MpjjPm7MeaEpG2S9kl6z3FJNYoxpoGkJyWNdV0LKoZA7pgxJkDSAkmvWGu3ua4HOIf6knJ+0JYjKdhBLUCF8R1cuay1v1HZ90MHSYslFfz0HjhPf5T0D2vtXteFoGII5JXAGLPGGGPP8vr0tO3qSJqnsvG4o50VfBH6udcYPpcrqcEP2hpIOuagFqBC+A6uGtbakpPD25pL+m/X9dQUxpjrJHWW9BfXtaDieKizElhrO51rG2OMkfQPlT0Y181aW1TZddUkP+cao1Jsl+RvjLnMWvufk23Xij/14yLDd7AT/mIMuS91khQrKaPs46z6kvyMMVdZa693WBcuAHfI3flfSVdK6mmtzXNdTE1kjPE3xlwiyU9lX1KXGGP4T2gFWGuPq+zPzk8aY+oZY9pJ+pXK7jLCB/jcVhm+gyuRMaaJMWagMaa+McbPGJMkaZCkj13XVoO8rLL/4Fx38vWipBUqm6EJFxkCuQPGmBhJ96vsH9B+Y0zuydfdjkuraSZJypM0XtKQk+8nOa2oZviNyqbj+17Sa5L+mykPfYrPbSXjO7hKWJUNT/lO0hFJz0l62Fq71GlVNYi19oS1dv+pl8qGFOZbaw+6rg3nz1hrXdcAAAAA1FrcIQcAAAAcIpADAAAADhHIAQAAAIcI5AAAAIBDBHIAAADAIQI5AAAA4BCBHAAAAHCIQA4AAAA4RCAHAAAAHPp/tivh0dFlDeAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_words(embedding, sequences, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a name='3'></a>\n",
    "## 3 - Semantic and positional embedding\n",
    "<div style=\"text-align: right;\"><a href=\"#toc\" style=\"text-decoration:none;\">↑ Table of Contents</a></div>\n",
    "\n",
    "Next, you will combine the original glove embedding with the positional encoding you calculated earlier. For this exercise, you will use a 1 to 1 weight ratio between the semantic and the positional embedding. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAssAAAFoCAYAAAChaLYGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXgUVb7G8feQxBAg0EAChBATdpBHJRpEAQk6SGQREWQVEPQ64AyiDnIvCM6AOoAyzMx1mauiAy444IIgIIKKICiigUS9KuGydSBhC6FDAtlz7h/RHqIUgnSnSfh+nqefJ3Wq6tSvjrH7pXK6ylhrBQAAAODnagS6AAAAAOBCRVgGAAAAHBCWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHAQHuoCIiAgbFxcX6DIAAABQjW3dujXLWht5rvsFPCzHxcUpOTk50GUAAACgGjPGuH/NfkzDAAAAABwQlgEAAAAHhGUAAADAAWEZkqSFCxeqW7du3mVjjHbu3BnAigAAAAKPsAwAAAA4ICwDAAAADgjL1czcuXM1aNCgCm333XefHnjgAeXk5Ojuu+9WVFSUoqOjNX36dJWWlv5inzk5ORo9erQiIyMVGxurxx9/XGVlZZKk2NhYbd26VZL02muvyRij7777TpL04osvasCAAT4+QwAAgMpDWK5mRo4cqffff18ej0eSVFJSoiVLlmjUqFG68847FRwcrJ07dyolJUVr167Viy+++It93nfffcrJydHu3bu1YcMGvfLKK1qwYIEkKTExUevXr5ckffLJJ2rRooU2bNjgXU5MTPTPiQIAAFQCwnI1ExUVpe7du+vNN9+UJL3//vuKiIhQs2bNtHr1av39739X7dq11ahRIz344INavHjxGfsrLS3VkiVLNHv2bIWHhysuLk6TJk3Sq6++Kqk8LP8Yjjdu3KipU6d6lzds2EBYBgAAVRphuZpYlpKhrnPWqfmUVdrtulp/f+4lSeVTI0aNGiW3263i4mJFRUXJ5XLJ5XJp3LhxOnz48Bn7zcrKUlFRkWJjY71tsbGxysjIkFQeljdu3KiDBw+qtLRUQ4cO1aeffqq9e/cqJydHHTt29N9JAwAA+FnAH3eN87csJUNTl36j/OLy+ccF0Vdr/7L/1n+/+ZFWrlypJ598UiEhIQoNDVVWVpaCg8/+P3tERIRCQkLkdrt12WWXSZLS09MVHR0tSWrVqpVq1aqlp556St27d1d4eLiaNGmiF154Qd26dVONGvx7DAAAVF0kmWpg7po0b1CWJBN8iWq16aKHJ96ja665RpdeeqmioqLUq1cvTZo0ScePH1dZWZl27drlnTLhJCgoSEOGDNG0adOUm5srt9utv/71rxo5cqR3m8TERD3zzDPeKRc9evSosAwAAFBVEZargUxP/s/aal/+G508uEejRo3ytr3yyisqKirSZZddpvr16+v222/XgQMHfrH/p59+WrVr11aLFi3UrVs3jRgxQnfddZd3fWJionJzc9W9e/fTLgMAAFRVxlob0AISEhJscnJyQGuo6rrOWaeMnwTmkuOHdeDFe3Us67Dq1q0boMoAAAAuDMaYrdbahHPdjyvL1cDkpLYKCwnyLltbppNbl6tH7wEEZQAAgPPAF/yqgQHx5V+2m7smTfsPH9P+Z0cqutmlWvjWhwGuDAAAoGpjGgYAAACqPaZhAAAAAD5GWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB34Jy8aY1saYAmPMa/7oHwAAAKgM/rqy/KykL/3UNwAAAFApfB6WjTHDJHkkfeTrvgEAAIDK5NOwbIypK+lRSZN+YbvfGmOSjTHJR44c8WUJAAAAgM/4+sryY5JestbuO9NG1toXrLUJ1tqEyMhIH5cAAAAA+EawrzoyxnSU1FNSvK/6BAAAAALJZ2FZUg9JcZLSjTGSVEdSkDHmMmvtVT48DgAAAFApfBmWX5C0+JTlh1Qenu/14TEAAACASuOzsGytPSnp5I/Lxpg8SQXWWr7BBwAAgCrJl1eWK7DWzvBX3wAAAEBl4HHXAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLuKCUlJQEugQAAAAvwjIqxbZt2xQfH6/w8HANHjxYQ4cO1fTp07V+/Xo1a9ZMTzzxhJo0aaKxY8eqrKxMc+bMUcuWLdWwYUMNGTJE2dnZ3r4+//xzdenSRS6XS1deeaXWr1/vXdejRw898sgj6tq1q8LDw9WrVy9lZWUF4IwBAEB1QFiG3xUVFem2227TmDFjlJ2dreHDh+udd97xrj948KCys7Pldrv1wgsv6KmnntKyZcu0YcMGZWZmqn79+vr9738vScrIyFDfvn01ffp0ZWdn6y9/+YsGDRqkI0eOePt7/fXXtWDBAh0+fFhFRUX6y1/+UunnDAAAqgfCMvzu888/V0lJiSZOnKiQkBANHDhQ11xzjXd9jRo1NHPmTIWGhiosLEzPP/+8/vznP6tZs2YKDQ3VjBkz9NZbb6mkpESvvfaa+vTpoz59+qhGjRq66aablJCQoPfee8/b39ixY9WmTRuFhYVpyJAhSk1NDcRpAwCAaiA40AWg+svMzFR0dLSMMd62mJgY78+RkZGqWbOmd9ntduu2225TjRr//rdcUFCQDh06JLfbrTfffFMrVqzwrisuLtYNN9zgXW7SpIn351q1aikvL8/n5wQAAC4OhGX4zbKUDM1dk6bdX+9VdtpuvbNtv267qpkkad++fWrZsqUkVQjRUnmQ/uc//6muXbv+rM+YmBiNGjVK8+fP9/8JAACAix7TMOAXy1IyNHXpN8rw5OuS6HYqtTU0/uE/6+0v3Vq+fLm++OILx33Hjx+vadOmye12S5KOHDmi5cuXS5JGjhypFStWaM2aNSotLVVBQYHWr1+v/fv3V8p5AQCAiwthGX4xd02a8otLJUkmKESRtz2sYylrNKRbO7322mvq16+fQkNDT7vv/fffr/79+6tXr14KDw/Xtddeqy1btkgqv7K8fPlyzZo1S5GRkYqJidHcuXNVVlZWaecGAAAuHsZaG9ACEhISbHJyckBrgO81n7JKp/vNMpL2zOmrzp07a/z48Ro7dmxllwYAAC5Cxpit1tqEc92PK8vwi6ausArLBenfqDTvmKLCL9HLL7+sr7/+WjfffHOAqgMAADg7fMEPfjE5qa2mLv3GOxWjODtDWe8+oSOlhZrXupXeeustRUVFBbhKAACAMyMswy8GxEdLKp+7nOnJV7set2ny7CnedgAAgKqAsAy/GRAfTTgGAABVGnOWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHBCWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHBCWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHBCWAQAAAAeEZQAAAMABYRkAAABwQFgGAAAAHBCWq5gZM2Zo5MiRgS4DAADgokBYBgAAABwQlgOspKQk0CUAAADAAWE5AOLi4vTEE0/oiiuuUO3atbVp0yZ16dJFLpdLV155pdavX+/dds+ePUpMTFR4eLhuuukmZWVlBa5wAACAi0xwoAu4WP3rX//SqlWrVKNGDV1xxRV69dVXdfPNN+ujjz7SoEGDtH37dkVGRmrEiBG67rrrtHbtWm3ZskV9+/bVrbfeGujyAQAALgpcWQ6QiRMnKiYmRq+99pr69OmjPn36qEaNGrrpppuUkJCg9957T+np6fryyy/12GOPKTQ0VN27d9ctt9wS6NIBAAAuGlxZriTLUjI0d02aMj35OphToPTCMEmS2+3Wm2++qRUrVni3LS4u1g033KDMzEzVr19ftWvX9q6LjY3Vvn37Kr1+AACAixFhuRIsS8nQ1KXfKL+4VJJUUmb1z0171fHaDMXExGjUqFGaP3/+z/Zzu906duyYTpw44Q3M6enpMsZUav0AAAAXK6ZhVIK5a9K8QflHhaVlmrsmTSNHjtSKFSu0Zs0alZaWqqCgQOvXr9f+/fsVGxurhIQE/elPf1JRUZE2bdpU4Qo0AAAA/IuwXAkyPfmO7TExMVq+fLlmzZqlyMhIxcTEaO7cuSorK5Mkvf7669qyZYsaNGigmTNnavTo0ZVZOgAAwEXNWGsDWkBCQoJNTk4OaA3+1nXOOmWcJjBHu8L06ZQbA1ARAADAxcUYs9Vam3Cu+3FluRJMTmqrsJCgCm1hIUGanNQ2QBUBAADgbPgsLBtjQo0xLxlj3MaYXGNMijGmt6/6r8oGxEdr9sDLFe0Kk1H5FeXZAy/XgPjoQJcGAACAM/Dl3TCCJe2TlCgpXVIfSW8YYy631u714XGqpAHx0YRjAACAKsZnYdlae0LSjFOaVhpj9ki6WtJeXx0HAAAAqCx+m7NsjGksqY2kb0+z7rfGmGRjTPKRI0f8VQIAAABwXvwSlo0xIZIWSXrZWrv9p+uttS9YaxOstQmRkZH+KAEAAAA4bz4Py8aYGpJelVQkaYKv+wcAAAAqi08fd23Kn8P8kqTGkvpYa4t92T8AAABQmXwaliX9j6T2knpaa0//2DoAAACgivDlfZZjJY2T1FHSQWNM3g+vO3x1DAAAAKAy+fLWcW5Jxlf9AQAAAIHG464BAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAwAFhGQAAAHBAWAYAAAAcEJYBAAAAB4RlAAAAVLqUlBRdddVVCg8P19ChQzVs2DBNnz5dCxcuVLdu3Spsa4zRzp07JUmFhYV66KGHdOmll6px48YaP3688vPzvduuXLlSHTt2lMvlUpcuXfT111+f2s9eY8xDxpivjTE5xpglxpiaZ6qTsAwAAIBKVVRUpAEDBmjUqFHKzs7W4MGD9fbbb5/Vvv/1X/+lHTt2KDU1VTt37lRGRoYeffRRSdK2bdt011136fnnn9fRo0c1btw49e/fX4WFhad2MUTSzZKaS7pC0pgzHY+wDAAAgEr1+eefq7i4WA888IBCQkJ0++23q1OnTr+4n7VW8+fP19/+9jc1aNBA4eHhevjhh7V48WJJ0vz58zVu3Dh17txZQUFBuvPOOxUaGqrPP//81G6estZmWmuzJa2Q1PFMxwz+9acJAAAAnJ1lKRmauyZNmZ58XeLerNr1G8kY410fGxv7i30cOXJEJ0+e1NVXX+1ts9aqtLRUkuR2u/Xyyy/r6aef9q4vKipSZmbmqd0cPOXnk5KanumYXFkGAACAXy1LydDUpd8ow5MvKynH1NHe9H16Z9t+7zbp6emSpNq1a+vkyZPe9oMH/51tIyIiFBYWpm+//VYej0cej0c5OTnKy8uTJMXExGjatGnedR6PRydPntTw4cN/de2EZQAAAPjV3DVpyi8u9S6HRreTNUH6w59mq6SkREuXLtUXX3whSbryyiv17bffKjU1VQUFBZoxY4Z3vxo1auiee+7Rgw8+qMOHD0uSMjIytGbNGknSPffco+eee05btmyRtVYnTpzQqlWrlJub+6trJywDAICLQlxcnD788MOftXfo0EHr16+v/IIuIpme/ArLJihEkbc9rMwv3lf9+vW1ZMkSDRw4UJLUpk0b/fGPf1TPnj3VunXrn90Z44knnlCrVq107bXXqm7duurZs6fS0tIkSQkJCZo/f74mTJig+vXrq1WrVlq4cOF51W6stefVwflKSEiwycnJAa0BAABUf3FxcXrxxRfVs2fPQJdy0ek6Z50yfhKYJSnaFaZPp9woSRozZoyaNWumxx9/3C81GGO2WmsTznU/riwDAADAryYntVVYSFCFtrCQIE1Oahugis4eYRkAAFx0tm/frubNm2vx4sUVpmfMmDFDQ4YM0ejRoxUeHq4OHTro1L+Ab9u2TfHx8QoPD9fgwYM1dOhQTZ8+PVCnUWUMiI/W7IGXK9oVJqPyK8qzB16uAfHRgS7tFxGWAQDARWXbtm3q1auXnn76aQ0bNuxn6999910NGzZMHo9H/fv314QJEySV34Lstttu05gxY5Sdna3hw4frnXfeqezyq6wB8dH6dMqN2jOnrz6dcuPPgvLChQv9NgXjfBCWAQDARWPjxo3q37+/Xn75ZfXr1++023Tr1k19+vRRUFCQRo0apa+++kpS+YM0SkpKNHHiRIWEhGjgwIG65pprKrN8BAAPJQEAANXWqQ/COJhToP9+5h/q9ZsbdMMNNzju06RJE+/PtWrVUkFBgUpKSpSZmano6OgKD9KIiYnxa/0IPK4sAwCAaumnD8IoKbOqc+N4ffX9Tj344IPn3F9UVJQyMjJ06p3E9u3b58OKcSEiLAMAgGrppw/CkKTioJpyDZyhTz75RFOmTDmn/q677joFBQXpmWeeUUlJiZYvX+59kAaqL6ZhAACAaumnD8L40eHCIG394APdcMMNCgkJOev+LrnkEi1dulT/8R//oalTp6p3797q16+fQkNDfVUyLkA8lAQAAFRLZ/MgjPPVuXNnjR8/XmPHjvVJf/AfHkoCAABwCn88CGPDhg06ePCgSkpK9PLLL+vrr7/WzTfffL6l4gLGNAwAAFAt/Xgf3x/vhtHUFabJSW3P60EYaWlpGjJkiPLy8tSyZUu99dZbioqK8lXJuAAxDQMAAADVHtMwAAAAAB8jLAMAAAAOCMsAAACAA8IyAAAA4MCnYdkY08AY844x5oQxxm2MGeHL/gEAAIDK5Otbxz0rqUhSY0kdJa0yxnxlrf3Wx8cBAAAA/M5nV5aNMbUlDZL0iLU2z1q7SdK7kkb56hgAAABAZfLlNIw2kkqttTtOaftKUgcfHgMAAACoNL4My3Uk5fykLUdS+E83NMb81hiTbIxJPnLkiA9LAAAAAHzHl2E5T1Ldn7TVlZT70w2ttS9YaxOstQmRkZE+LAEAAADwHV+G5R2Sgo0xrU9pu1ISX+4DAABAleSzsGytPSFpqaRHjTG1jTFdJd0q6VVfHQMAAADnJy4uTnPnztUVV1yh2rVr6+6779ahQ4fUu3dvhYeHq2fPnjp27JgkafDgwWrSpInq1aun7t2769tv/30NdMyYMfr973+vvn37Kjw8XJ07d9auXbsCdVp+4+uHkvxOUpikw5L+JelebhsHAABwYXn77bf1wQcfaMeOHVqxYoV69+6tWbNmad68eUpNTdVTTz0lSfrkk0/UpUsXHT58WFdddZXi4+OVmpqqzz77TCtXrtQ//vEP7dmzR6tWrVKrVq00bdo09ejRQ9OnT1eXLl1Up04d3XLLLTp69KjuuOMO1a1bV506ddLevXu9tdx///2KiYlR3bp1dfXVV2vjxo3edTNmzNCQIUM0evRohYeHq0OHDkpOTq7UsfJpWLbWZltrB1hra1trL7XWvu7L/gEAAHD+7rvvPjVu3FjR0dG6/vrr1blzZ8XHx6tnz546efKktm3bpgMHDqhmzZrasmWLQkNDNWbMGBUXF8vlcqlv375q3769xo4dq0ceeUS33nqrbrnlFqWmpkqSFi9erFdffVUZGRnatWuXrrvuOo0dO1bZ2dlq3769Zs6c6a2lU6dOSk1NVXZ2tkaMGKHBgweroKDAu/7dd9/VsGHD5PF41L9/f02YMKFSx4rHXQMAAFRzy1Iy1HXOOjWfskoHcwq0MzfIuy4sLEyNGzeWJLVo0UI1a9bUgQMH9PHHH6t+/fo6evSoateurS5dukiS3nrrLbVu3VotW7ZU06ZNNXz4cLVr106pqanKy8uTJI0dO1YtW7ZUvXr11Lt3b7Vs2VI9e/ZUcHCwBg8erJSUFO/xR44cqYYNGyo4OFiTJk1SYWGh0tLSvOu7deumPn36KCgoSKNGjdJXX31VGUPmRVgGAACoxpalZGjq0m+U4cmXlVRSZvXSxj1alpIhSdqXfVIvbdqj5lNWqeucdYqIjpPH49FLL72kw4cP684779S8efM0YMAASdKhQ4cUGxtb4RixsbHKysryLv8YvqWKYfzH5R9DtSTNmzdP7du3V7169eRyuZSTk1OhryZNmnh/rlWrlgoKClRSUuKbwTkLhGUAAIBqbO6aNOUXl1ZoKywt09w1aVqWkqHUfR7lFhTLSsrw5OuQaaCDR45q+/btcrlc6tmzp9atW6eVK1dKKg/Cbre7Qn/p6emKiIg459o2btyoJ554Qm+88YaOHTsmj8ejevXqyVr7q8/X1wjLAAAA1VimJ9+xfe6aNJWWVQymxhWl3ByPatasqdatW2vs2LF6++23VVxcLElKTEzUjh07tHv3bpWVlWnJkiX67rvvdN11151zbbm5uQoODlZkZKRKSkr06KOP6vjx4+d+kn5EWAYAAKjGmrrCKiw3u/efCovrqKauMGV68hVxy0NydbvDu97VZZiCapXfKm758uXKy8tTfHy8evToIWutOnXqpJUrV+rEiRN69tln9eSTT2rlypW69dZbtX///nOqLSkpSb1791abNm0UGxurmjVrKiYmxifn7Ssm0Je5ExISbGXfAgQAAOBi8eOc5VOnYoSFBGn2wMs1d02aMk5z5TnaFaZPp9xYmWX6nTFmq7U24Vz3C/ZHMQAAALgwDIiPllQ+dznTk6+mrjBNTmrrbT9dkJ6c1DYgtV6ICMsAAADV3ID4aG84/mm75BykQVgGAAC4qDkFaZTjC34AAACAA8IyAAAA4ICwDAAAADggLAMAAAAOCMsAAACAA8IyAAAA4ICwDAAAADggLAMAAAAOCMsAAACAA8IyAAAA4ICwDACAD4wZM0bTp08PdBkAfIywDAAAADggLAMAAAAOCMsAAPwKKSkpuuqqqxQeHq6hQ4eqoKDAu27+/Plq1aqVGjRooP79+yszM9O7bu3atWrbtq3q1aun3/3ud0pMTNSLL74YiFMAcBYIywAAnKOioiINGDBAo0aNUnZ2tgYPHqy3335bkrRu3TpNnTpVb7zxhg4cOKDY2FgNGzZMkpSVlaXbb79ds2fP1tGjR9W2bVt99tlngTwVAL+AsAwAwDn6/PPPVVxcrAceeEAhISG6/fbb1alTJ0nSokWLdNddd+mqq65SaGioZs+erc2bN2vv3r1677331KFDBw0cOFDBwcGaOHGimjRpEuCzAXAmhGUAAM5RZmamoqOjZYzxtsXGxnrX/fizJNWpU0cNGzZURkaGMjMzFRMT411njFGzZs0qr3AA54ywDADAWVqWkqGuc9bpwXf36qu0PXpn237vuvT0dElS06ZN5Xa7ve0nTpzQ0aNHFR0draioKO3f/+99rLUVlgFceAjLAACchWUpGZq69BtlePJ1SXQ7lamGxk99XG9/6dbSpUv1xRdfSJJGjBihBQsWKDU1VYWFhXr44YfVuXNnxcXFqW/fvvrmm2+0bNkylZSU6Nlnn9XBgwcDfGYAzoSwDADAWZi7Jk35xaWSJBMUosjbHtax1A80pFt7LVmyRAMHDpQk/eY3v9Fjjz2mQYMGKSoqSrt27dLixYslSREREXrzzTf1n//5n2rYsKG+++47JSQkKDQ0NGDnBeDMjLU2oAUkJCTY5OTkgNYAAMAvaT5llU73iWkk7ZnT91f1WVZWpmbNmmnRokW64YYbzqs+AGdmjNlqrU041/24sgwAwFlo6go7p3Yna9askcfjUWFhoWbNmiVrra699lpflAjADwjLAACchclJbRUWElShLSwkSJOT2p5TP5s3b1bLli0VERGhFStWaNmyZQoLO7fADaDyEJYB4ALSo0ePCk9zmz59uiIiIrgX7wVgQHy0Zg+8XNGuMBlJ0a4wzR54uQbER59TPzNmzNDRo0eVm5urLVu2qHPnzv4pGIBPBAe6AADA6e3bt0/z5s2T2+1Wo0aNAl0OVB6YzzUcA6jauLIMABcot9uthg0bEpQBIIAIywDgIwsWLNAtt9ziXW7VqpWGDBniXY6JiVFqaqo+++wzderUSfXq1VOnTp302Wef/ayvDz/8UDfddJMyMzNVp04djRkzpjJOAQDwE4RlAPCRxMREbdy4UWVlZTpw4ICKi4v16aefSpJ2796tvLw8XXrpperbt68mTpyoo0eP6g9/+IP69u2ro0ePVuirZ8+eWr16tZo2baq8vDwtXLgwAGcEACAsA4CPtGjRQuHh4UpNTdWGDRuUlJSk6Ohobd++XRs2bND111+vVatWqXXr1ho1apSCg4M1fPhwtWvXTitWrAh0+QCA0+ALfgDgQ4mJiVq/fr127typxMREuVwubdiwQZs3b1ZiYqIyMzMVGxtbYZ/Y2FhlZGQEqGIAwJkQlgHgPC1LydDcNWnK9OQr6GRj7Vj+vvKzD+jhhx+Wy+XSokWLtHnzZk2YMEHff/+93G53hf3T09N18803B6h6AMCZMA0DAM7DspQMTV36jTI8+bKS8iPaauvnm5TlyVWzZs10/fXX6/3339fRo0cVHx+vPn36aMeOHXr99ddVUlKiJUuW6LvvvlO/fv0CfSoAgNPgyjIAnIe5a9KUX1zqXQ5pEC0TUlOljdtJkurWrasWLVooMjJSQUFBatiwoVauXKn7779f9957r1q1aqWVK1cqIiIiUKcAADgDY60NaAEJCQk2OTk5oDUAwK/VfMoqne5d1EjaM6dvZZcDAHBgjNlqrU041/2YhgEA56GpK+yc2gEAVQthGQDOw+SktgoLCarQFhYSpMlJbQNUEQDAl5izDADnYUB8tCR574bR1BWmyUltve0AgKqNsAwA52lAfNKe5PcAABEHSURBVDThGACqKaZhAAAAAA4IywAAAIADwjIAAADggLAMAAAAOCAsAwAAAA4IywAAAIADwjIAAADggLAMAAAAOCAsAwAAAA4IywAAAIADwjIAAADg4LzDsjEm1BjzkjHGbYzJNcakGGN6+6I4AAAAIJB8cWU5WNI+SYmS6kl6RNIbxpg4H/QNAAAABEzw+XZgrT0hacYpTSuNMXskXS1p7/n2DwAAAASKz+csG2MaS2oj6dszbPNbY0yyMSb5yJEjvi4BAAAA8AmfhmVjTIikRZJettZud9rOWvuCtTbBWpsQGRnpyxIAAAAAn/nFsGyMWW+MsQ6vTadsV0PSq5KKJE3wY80AAABApfjFOcvW2h6/tI0xxkh6SVJjSX2stcXnXxoAAAAQWOf9Bb8f/I+k9pJ6WmvzfdQnAAAAEFC+uM9yrKRxkjpKOmiMyfvhdcd5VwcAAAAEkC9uHeeWZHxQCwAAAHBB4XHXAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICwDAAAADgjLAAAAgAPCMgAAAOCAsAwAAAA4ICw7iIuL04cffuiz/hYuXKhu3br5pK8xY8Zo+vTpv2rfGTNmaOTIkZKkvXv3yhijkpISn9QFAABQ3RCWL3C+DNkAAAA4N4RlAAAAwAFh+Qy+/PJLXXbZZapfv77Gjh2rgoICHTt2TP369VNkZKTq16+vfv36af/+/d59Fi5cqBYtWig8PFzNmzfXokWLTtv35MmT1a1bN+Xk5CgnJ0d33323oqKiFB0drenTp6u0tFTff/+9xo8fr82bN6tOnTpyuVze/bOysnTTTTcpPDxciYmJcrvd3nX333+/YmJiVLduXV199dXauHGj/wYJAACgGiMsn8GiRYu0Zs0a7dq1Szt27NDjjz+usrIyjR07Vm63W+np6QoLC9OECRMkSSdOnNDEiRO1evVq5ebm6rPPPlPHjh0r9FlWVqZ77rlHX3/9tdauXat69erpzjvvVHBwsHbu3KmUlBStXbtWL774otq3b6/nnntO1113nfLy8uTxeCrU9sgjjygrK0sdO3bUHXfc4V3XqVMnpaamKjs7WyNGjNDgwYNVUFBQOYMGAABQjRCWz2DChAmKiYlRgwYNNG3aNP3rX/9Sw4YNNWjQINWqVUvh4eGaNm2aNmzY4N2nRo0a+t///V/l5+crKipKHTp08K4rLi7W8OHDlZ2drRUrVqhWrVo6dOiQVq9erb///e+qXbu2GjVqpAcffFCLFy8+Y219+/ZV9+7dFRoaqj//+c/avHmz9u3bJ0kaOXKkGjZsqODgYE2aNEmFhYVKS0vzzyABAABUY4TlM4iJifH+HBsbq8zMTJ08eVLjxo1TbGys6tatq+7du8vj8ai0tFS1a9fWkiVL9NxzzykqKkp9+/bV9u3bvX3s3LlTy5cv15/+9CddcsklkiS3263i4mJFRUXJ5XLJ5XJp3LhxOnz48FnXVqdOHTVo0ECZmZmSpHnz5ql9+/aqV6+eXC6XcnJylJWV5cuhAQAAuCgQln+wLCVDXeesU/Mpq9R1zjqdLCr1XqmVpPT0dDVt2lTz5s1TWlqatmzZouPHj+uTTz6RJFlrJUlJSUn64IMPdODAAbVr10733HOPt4/27dtrwYIF6t27t/dKb0xMjEJDQ5WVlSWPxyOPx6Pjx4/r22+/lSQZY05b76m15eXlKTs7W02bNtXGjRv1xBNP6I033tCxY8fk8XhUr149b30AAAA4e4RllQflqUu/UYYnX1ZShidfx04Wafa8v2v//v3Kzs7WrFmzNHToUOXm5iosLEwul0vZ2dmaOXOmt59Dhw7p3Xff1YkTJxQaGqo6deooKCiowrGGDx+uWbNmqWfPntq1a5eioqLUq1cvTZo0ScePH1dZWZl27drlndrRuHFj7d+/X0VFRRX6ee+997Rp0yYVFRXpkUceUefOnRUTE6Pc3FwFBwcrMjJSJSUlevTRR3X8+HG/jyEAAEB1RFiWNHdNmvKLSyu0WSupZTf16tVLLVq0UIsWLTR9+nQ98MADys/PV0REhK699lrdfPPN3n3Kyso0b948NW3aVA0aNNCGDRv0j3/842fHu/POO/XHP/5RN954o/bu3atXXnlFRUVF3jtv3H777Tpw4IAk6cYbb1SHDh3UpEkTRUREePsYMWKEZs6cqQYNGmjr1q3eu24kJSWpd+/eatOmjWJjY1WzZs0KUzYAAABw9kyg/zyfkJBgk5OTA1pD8ymrdLpRMJL2zOlb2eUAAADAx4wxW621Cee6H1eWJTV1hZ1TOwAAAC4OhGVJk5PaKiyk4tzisJAgTU5qG6CKAAAAcCG4qMNyWlqa4uPjNap7O3Ut/ELRrjAZSdGuMM0eeLkGxEf/qn4XLlyobt26/ap9169fr2bNmnmX4+Li9OGHH/6qvgAAAHB+gn3ZmTGmtaRvJL1lrR3py7794cknn1SPHj2UkpLyq/vYu3evmjdvruLiYgUH+3Q4AQAAEGC+vrL8rKQvfdyn37jd7gpP2AMAAABO5bOwbIwZJskj6SNf9elPN954oz7++GNNmDBBderU0VdffaXRo0crMjJSsbGxevzxx1VWViap/JZwjz/+uGJjY9WoUSONHj1aOTk5kqTu3btLklwul+rUqaPNmzdLKn9IyX333ad69eqpXbt2+uijfw/LggUL1L59e4WHh6tFixZ6/vnnK/nsAQAAcDZ8EpaNMXUlPSppki/6qwzr1q3T9ddfr2eeeUZ5eXmaN2+ecnJytHv3bm3YsEGvvPKKFixYIKl8DvLChQv18ccfa/fu3crLy9OECRMkyfsEP4/Ho7y8PF133XWSpC1btqhFixbKysrSzJkzNXDgQGVnZ0uSGjVqpJUrV+r48eNasGCBHnzwQW3bti0AowAAAIAz8dWV5cckvWSt3feLW0oyxvzWGJNsjEk+cuSIj0r49UpLS7VkyRLNnj1b4eHhiouL06RJk/Tqq69KkhYtWqQ//OEPatGiherUqaPZs2dr8eLFKikpceyzUaNGeuCBBxQSEqKhQ4eqbdu2WrVqlSSpb9++atmypYwxSkxMVK9evbRx48ZKOVcAAACcvV8My8aY9cYY6/DaZIzpKKmnpL+d7UGttS9YaxOstQmRkZHnU/85W5aSoa5z1qn5lFVKSfcoJf2YsrKyVFRUpNjYWO92sbGxysjIkCRlZmb+bF1JSYkOHTrkeJzo6GgZYyrsk5mZKUlavXq1rr32WjVo0EAul0vvvfeesrKyfH2qAAAAOE+/GJattT2stcbh1U1SD0lxktKNMQclPSRpkDHmgptXsCwlQ1OXfqMMT76spMKSUi3dlqFN+woVEhIit9vt3TY9PV3R0eW3jmvatOnP1gUHB6tx48YVAvGpMjIydOrTEdPT09W0aVMVFhZq0KBBeuihh3To0CF5PB716dNHgX6SIgAAAH7OF9MwXpDUUlLHH17PSVolKckHffvU3DVpyi8urdBWVFqmv364U0OGDNG0adOUm5srt9utv/71rxo5svzud8OHD9ff/vY37dmzR3l5eXr44Yc1dOhQBQcHKzIyUjVq1NDu3bsr9Hv48GE99dRTKi4u1ptvvqnvv/9effr0UVFRkQoLCxUZGang4GCtXr1aa9eurbQxAAAAwNk77xsDW2tPSjr547IxJk9SgbU28JORfyLTk+/YvvLpp3XfffepRYsWqlmzpu655x7dddddkqS77rpLmZmZ6t69uwoKCpSUlKSnn35aklSrVi1NmzZNXbt2VXFxsd5//31JUufOnfV///d/ioiIUOPGjfXWW2+pYcOGkqSnnnpKQ4YMUWFhoW655Rb179+/Es4eAAAA58oE+s//CQkJNjk5uVKO1XXOOmWcJjBHu8L06ZQbK6UGAAAAVD5jzFZrbcK57ndRPe56clJbhYUEVWgLCwnS5KS2AaoIAAAAF7KL6vnMA+LLv7A3d02aMj35auoK0+Sktt52AAAA4FQXVViWygMz4RgAAABn46KahgEAAACcC8IyAAAA4ICwDAAAADggLAMAAAAOCMsAAACAA8IyAAAA4ICwDAAAADggLAMAAAAOCMsAAACAA8IyAAAA4MBYawNbgDFHJLkDWkTliJCUFegiqiHG1T8YV99jTP2DcfUPxtX3GFP/OJdxjbXWRp7rAQIeli8Wxphka21CoOuobhhX/2BcfY8x9Q/G1T8YV99jTP2jMsaVaRgAAACAA8IyAAAA4ICwXHleCHQB1RTj6h+Mq+8xpv7BuPoH4+p7jKl/+H1cmbMMAAAAOODKMgAAAOCAsAwAAAA4ICxXMmPMa8aYA8aY48aYHcaY/wh0TVWZMSbUGPOSMcZtjMk1xqQYY3oHuq7qwBgzwRiTbIwpNMYsDHQ9VZUxpoEx5h1jzIkffk9HBLqmqo7fTf/g/dQ/+Nz3L2NMa2NMgTHmNX8dI9hfHcPRbEl3W2sLjTHtJK03xqRYa7cGurAqKljSPkmJktIl9ZH0hjHmcmvt3kAWVg1kSnpcUpKksADXUpU9K6lIUmNJHSWtMsZ8Za39NrBlVWn8bvoH76f+wee+fz0r6Ut/HoAry5XMWvuttbbwx8UfXi0DWFKVZq09Ya2dYa3da60ts9aulLRH0tWBrq2qs9YutdYuk3Q00LVUVcaY2pIGSXrEWptnrd0k6V1JowJbWdXG76Z/8H7qH3zu+48xZpgkj6SP/HkcwnIAGGP+YYw5KWm7pAOS3gtwSdWGMaaxpDaSuGqHC0EbSaXW2h2ntH0lqUOA6gHOGu+nvsPnvu8ZY+pKelTSJH8fi7AcANba30kKl3S9pKWSCs+8B86GMSZE0iJJL1trtwe6HkBSHUk5P2nLUfn//8AFi/dT3+Jz3y8ek/SStXafvw9EWPYhY8x6Y4x1eG06dVtrbekPf5JtJunewFR84TvbMTXG1JD0qsrnhk4IWMFVxLn8ruK85Emq+5O2upJyA1ALcFZ4P/UPPvd9xxjTUVJPSX+rjOPxBT8fstb2+BW7BYu5S47OZkyNMUbSSyr/AlUfa22xv+uq6n7l7yrO3Q5JwcaY1tba//uh7UrxZ21coHg/rRR87p+/HpLiJKWX/8qqjqQgY8xl1tqrfH0wrixXImNMI2PMMGNMHWNMkDEmSdJwSesCXVsV9z+S2ku6xVqbH+hiqgtjTLAxpqakIJW/CdU0xvAP7HNgrT2h8j+5PmqMqW2M6SrpVpVftcOvxO+mX/F+6kN87vvNCyr/B0fHH17PSVql8jvk+BxhuXJZlf/pZb+kY5L+IukBa+3ygFZVhRljYiWNU/n/LAeNMXk/vO4IcGnVwXRJ+ZKmSBr5w8/TA1pR1fQ7ld/e7LCkf0m6l9vGnTd+N/2A91O/4HPfD6y1J621B398qXzKW4G19og/jmestf7oFwAAAKjyuLIMAAAAOCAsAwAAAA4IywAAAIADwjIAAADggLAMAAAAOCAsAwAAAA4IywAAAIADwjIAAADggLAMAAAAOPh/tOwhkh8Al3kAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsQAAAFoCAYAAABQY+2LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXRV5fn28eshiTHAIYEQIAkhISLjWyUaRJmCFI0MUkCZh4LWGXEqv6JghZaCirRd2MECFlRQRIsgIIJKCSA4BEGsVlCGBBKGDCQkGMj0vH8ApwSZDDs5Cfv7WeuslT3d+95nnXW83Dzn2cZaKwAAAMCtavi6AQAAAMCXCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNX9fN1C/fn0bExPj6zYAAABwGdu8eXOmtTbsbNt8HohjYmKUnJzs6zYAAABwGTPGpJxrG0MmAAAA4GqOBmJjzFpjzDFjTP7J13Yn6wMAAABOq4g7xGOstbVPvlpUQH0AAADAMQyZAHBZi4mJ0YcffuhYvXnz5qlTp06O1Bo1apQmTpxYrmMnTZqk4cOHS5L27NkjY4yKi4sd6QsA3KYiAvE0Y0ymMeZjY0zXCqgPANWOk0EaAOAspwPxbyTFSoqUNEvSMmPMVWfuZIy51xiTbIxJzsjIcLgFAAAA4OI5GoittZ9aa/Ostcetta9I+lhSz7PsN8taG2+tjQ8LO+t0cADgmM8//1ytW7dW3bp1NXr0aB07dkyHDx9W7969FRYWprp166p3797at2+f95h58+YpNjZWHo9HTZs21YIFC85ae9y4cerUqZNyc3OVm5uru+++W+Hh4YqMjNTEiRNVUlKi//73v7r//vu1adMm1a5dWyEhId7jMzMzdcstt8jj8SghIUEpKf+bFeiRRx5RVFSU6tSpo+uvv17r16+vuDcJAFysoscQW0mmgs8BAOe1YMECrVq1Sjt37tSOHTs0ZcoUlZaWavTo0UpJSVFqaqqCgoI0ZswYSdLRo0c1duxYrVy5Unl5edq4caPatm1bpmZpaanuuecebdu2TatXr1ZwcLB++ctfyt/fX99//722bNmi1atXa86cOWrVqpVeeukl3XTTTcrPz1dOTk6Z3p5++mllZmaqbdu2GjZsmHdbu3bttHXrVmVnZ2vo0KEaMGCAjh07VjlvGgC4iGOB2BgTYoxJNMZcaYzxN8YMk9RF0iqnzgEA5TFmzBhFRUWpXr16mjBhgt544w2FhobqjjvuUM2aNeXxeDRhwgQlJSV5j6lRo4b+85//qKCgQOHh4WrTpo13W1FRkYYMGaLs7GwtW7ZMNWvW1MGDB7Vy5Ur9+c9/Vq1atdSgQQM99thjWrhw4Xl769Wrl7p06aLAwED94Q9/0KZNm7R3715J0vDhwxUaGip/f3898cQTOn78uLZvZzZLAHCak0+qC5A0RVJLSSWSvpXU11rLtzeASrVkS5qmr9qu9JwCHcg9pn2FNb3boqOjlZ6erh9++EGPPfaY3n//fR0+fFiSlJeXp5KSEtWqVUtvvvmmXnjhBd19993q2LGjZsyYoZYtW0qSvv/+e3355Zf67LPPdMUVV0iSUlJSVFRUpPDwcO+5SktLFRUVdd5eT99eu3Zt1atXT+np6YqKitKMGTM0Z84cpaenyxijI0eOKDMz07H3CQBwgmN3iK21GdbadtZaj7U2xFp7o7X2A6fqA8DFWLIlTU8u/kppOQWykopLrea8/7mWbEmTJKWmpioiIkIzZszQ9u3b9emnn+rIkSNat26dJMlaK0lKTEzUBx98oP3796tly5a65557vOdo1aqV5s6dqx49enjv2EZFRSkwMFCZmZnKyclRTk6Ojhw5oq+//lqSZMzZR4+duhssSfn5+crOzlZERITWr1+v5557TosWLdLhw4eVk5Oj4OBgb38AAOcwDzGAy8r0VdtVUFRSZl3258s0ZdEGZWdna+rUqRo0aJDy8vIUFBSkkJAQZWdna/Lkyd79Dx48qHfffVdHjx5VYGCgateuLT8/vzI1hwwZoqlTp6p79+7auXOnwsPDdeutt+qJJ57QkSNHVFpaqp07d3qHYTRs2FD79u1TYWFhmTrvvfeeNmzYoMLCQj399NNq3769oqKilJeXJ39/f4WFham4uFi/+93vdOTIkQp61wDA3QjEAC4r6TkFP1pXq3WCts0ep9jYWMXGxmrixIl69NFHVVBQoPr16+vGG2/Ubbfd5t2/tLRUM2bMUEREhOrVq6ekpCT97W9/+1HdX/7yl/rtb3+rbt26ac+ePXr11VdVWFjondHizjvv1P79+yVJ3bp1U5s2bdSoUSPVr1/fW2Po0KGaPHmy6tWrp82bN3tns0hMTFSPHj3UvHlzRUdH68orr7zg8AsAQPkYX//zW3x8vE1OTvZpDwAuHx2fXaO0s4TiyJAgfTy+mw86AgBUBcaYzdba+LNt4w4xgMvKuMQWCgooO7whKMBP4xJb+KgjAEBV5+QsEwDgc33jIiXJO8tEREiQxiW28K4HAOBMBGIAl52+cZEEYADARWPIBAAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1SokEBtjrjbGHDPGzK+I+gAAAIBTKuoO8V8lfV5BtQEAAADHOB6IjTGDJeVI+sjp2gAAAIDTHA3Expg6kn4n6YkL7HevMSbZGJOckZHhZAsAAADAT+L0HeLfS3rZWrv3fDtZa2dZa+OttfFhYWEOtwAAAABcPH+nChlj2krqLinOqZoAAABARXMsEEvqKilGUqoxRpJqS/IzxrS21l7n4HkAAAAAxzgZiGdJWnja8q91IiA/4OA5AAAAAEc5NobYWvuDtfbAqZekfEnHrLX8ag4AAFyS7du3Ky4uTh6PRzNnznSs7rx589SpU6dyHbt27Vo1btzYuxwTE6MPP/zQqdZQiZy8Q1yGtXZSRdUGAADu8vzzz6tr167asmVLuWvs2bNHTZs2VVFRkfz9KywCoRri0c0AAKDKS0lJUZs2bXzdBi5TBGIAAFCldevWTf/+9781ZswY1a5dW19++aVGjhypsLAwRUdHa8qUKSotLZUklZaWasqUKYqOjlaDBg00cuRI5ebmSpK6dOkiSQoJCVHt2rW1adMmSZK1Vg8//LCCg4PVsmVLffTR/54tNnfuXLVq1Uoej0exsbH6xz/+UclXj8pAIAYAAFXamjVr1LlzZ/3lL39Rfn6+ZsyYodzcXO3atUtJSUl69dVXNXfuXEknxgTPmzdP//73v7Vr1y7l5+drzJgxkqR169ZJknJycpSfn6+bbrpJkvTpp58qNjZWmZmZmjx5svr376/s7GxJUoMGDbR8+XIdOXJEc+fO1WOPPaYvvvjCB+8CKhKBGAAAVBslJSV68803NW3aNHk8HsXExOiJJ57Qa6+9JklasGCBHn/8ccXGxqp27dqaNm2aFi5cqOLi4nPWbNCggR599FEFBARo0KBBatGihVasWCFJ6tWrl6666ioZY5SQkKBbb71V69evr5RrReUhEAMAgGojMzNThYWFio6O9q6Ljo5WWlqaJCk9Pf1H24qLi3Xw4MFz1oyMjNTJZyh4j0lPT5ckrVy5UjfeeKPq1aunkJAQvffee8rMzHT6suBjBGIAAFAlLdmSpo7PrlHT8Su0JTVHW1IPq379+goICFBKSop3v9TUVEVGRkqSIiIifrTN399fDRs2LBN6T5eWliZrbZljIiIidPz4cd1xxx369a9/rYMHDyonJ0c9e/Yssy8uDwRiAABQ5SzZkqYnF3+ltJwCWUnHi0u0+Is0Ldt2QAMHDtSECROUl5enlJQU/fGPf9Tw4cMlSUOGDNGf/vQn7d69W/n5+Xrqqac0aNAg+fv7KywsTDVq1NCuXbvKnOvQoUOaOXOmioqK9NZbb+m///2vevbsqcLCQh0/flxhYWHy9/fXypUrtXr1ah+8G6hoTMIHAACqnOmrtqugqKTMusKSUk1ftV3LX3xRDz/8sGJjY3XllVfqnnvu0V133SVJuuuuu5Senq4uXbro2LFjSkxM1IsvvihJqlmzpiZMmKCOHTuqqKhI77//viSpffv2+u6771S/fn01bNhQb7/9tkJDQyVJM2fO1MCBA3X8+HHdfvvt6tOnTyW+C6gsxte3/ePj421ycrJPewAAAFVL0/ErdLaEYiTtfrZXZbeDy4AxZrO1Nv5s2xgyAQAAqpyIkKCftB64FARiAABQ5YxLbKGgAL8y64IC/DQusYWPOsLljDHEAACgyukbd2LWiOmrtis9p0ARIUEal9jCux5wEoEYAABUSX3jIgnAqBQMmQAAAICrEYgBAADgagRiAAAAuBqBGAAAAK5GIAYAAICrEYgBAADgagRiAAAAuBqBGAAAAK5GIAYAAICrEYgBAADgagRiAAAAuBqBGAAAAK5GIAYAAICrEYgBAADgagRiAAAAuJqjgdgYM98Ys98Yc8QYs8MY8ysn6wMAAABOc/oO8TRJMdbaOpL6SJpijLne4XMAAAAAjnE0EFtrv7bWHj+1ePJ1lZPnAAAAAJzk+BhiY8zfjDE/SPpW0n5J7zl9DgAAAMApjgdia+2DkjySOktaLOn4mfsYY+41xiQbY5IzMjKcbgEAAAC4aBUyy4S1tsRau0FSY0kPnGX7LGttvLU2PiwsrCJaAAAAAC5KRU+75i/GEAMAAKAKcywQG2MaGGMGG2NqG2P8jDGJkoZIWuPUOQAAAACn+TtYy+rE8IiXdCJop0h61Fq71MFzAAAAAI5yLBBbazMkJThVDwAAAKgMPLoZAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAArkYgBgAAgKsRiAEAAOBqBGIAAAC4GoEYAAAAruZYIDbGBBpjXjbGpBhj8owxW4wxPZyqDwAAAFQEJ+8Q+0vaKylBUrCkpyUtMsbEOHgOAAAAwFH+ThWy1h6VNOm0VcuNMbslXS9pj1PnAQAAAJxUYWOIjTENJTWX9PVZtt1rjEk2xiRnZGRUVAsAAADABVVIIDbGBEhaIOkVa+23Z2631s6y1sZba+PDwsIqogUAAADgojgeiI0xNSS9JqlQ0hin6wMAAABOcmwMsSQZY4yklyU1lNTTWlvkZH0AAADAaY4GYkl/l9RKUndrbYHDtQEAAADHOTkPcbSk+yS1lXTAGJN/8jXMqXMAAAAATnNy2rUUScapegAAAEBl4NHNAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1RwNxMaYMcaYZGPMcWPMPCdrAwAAABXB3+F66ZKmSEqUFORwbQAAAMBxjgZia+1iSTLGxEtq7GRtAAAAoCIwhhgAAACu5pNAbIy59+RY4+SMjAxftAAAAABI8lEgttbOstbGW2vjw8LCfNECAAAAIIkhEwAAAHA5p6dd8zfGXCnJT5KfMeZKY4zTM1kAqIJiYmI0ffp0XXPNNapVq5buvvtuHTx4UD169JDH41H37t11+PBhSdKAAQPUqFEjBQcHq0uXLvr666+9dUaNGqWHHnpIvXr1ksfjUfv27bVz505fXRYAwAWcvkM8UVKBpPGShp/8e6LD5wBQRf3rX//SBx98oB07dmjZsmXq0aOHpk6dqszMTJWWlmrmzJmSpB49eui7777ToUOHdN1112nYsGFl6rzxxht65plndPjwYTVr1kwTJkzwxeUAAFzC6WnXJkma5GRNANXHww8/rIYNG0qSOnfurAYNGiguLk6S1K9fP3300UeSpLvuust7zKRJk1S3bl3l5uYqODhYktS/f3/dcMMNkqRhw4bp8ccfr8zLAAC4DMMZAJTbki1pmr5qu9JzCnQg95i+z/PzbgsKCvKG41PL+fn5Kikp0YQJE/TWW28pIyNDNWqc+IeqzMxMbyBu1KiR97iaNWsqPz+/kq4IAOBG/KgOQLks2ZKmJxd/pbScAllJxaVWL6/frSVb0s573Ouvv66lS5fqww8/VG5urvbs2SNJstZWfNMAAJwFgRhAuUxftV0FRSVl1h0vKdX0VdvPe1xeXp4CAwMVGhqqH374QU899VRFtgkAwAURiAGUS3pOwU9af8rIkSMVHR2tyMhItW7dWjfeeGNFtAcAwEUzvv5nyvj4eJucnOzTHgD8dB2fXaO0s4TfyJAgfTy+mw86AgDg3Iwxm6218Wfbxh1iAOUyLrGFggL8yqwLCvDTuMQWPuoIAIDyYZYJAOXSNy5SkryzTESEBGlcYgvvegAAqgsCMYBy6xsXSQAGAFR7DJkAAACAqxGIAQAA4GoEYgAAALgagRgAAACuRiAGAACAqxGIAQAA4GoEYgAAALgagRgAAACuRiAGAACAqxGIAQAA4GoEYgAAALgagRgAAACuRiAGAACAqxGIAQAA4GoEYgDwkblz5+r222/3Ljdr1kwDBw70LkdFRWnr1q3auHGj2rVrp+DgYLVr104bN2707tO1a1dNnDhRHTp0UO3atXX77bcrKytLw4YNU506ddSuXTvt2bPHu/8jjzyiqKgo1alTR9dff73Wr1/v3TZp0iQNHDhQI0eOlMfjUZs2bZScnFyxbwIAVAEEYgDwkYSEBK1fv16lpaXav3+/ioqK9PHHH0uSdu3apfz8fDVp0kS9evXS2LFjlZWVpccff1y9evVSVlaWt87ChQv12muvKS0tTTt37tRNN92k0aNHKzs7W61atdLkyZO9+7Zr105bt25Vdna2hg4dqgEDBujYsWPe7e+++64GDx6snJwc9enTR2PGjKm8NwQAfIRADAA+EhsbK4/Ho61btyopKUmJiYmKjIzUt99+q6SkJHXu3FkrVqzQ1VdfrREjRsjf319DhgxRy5YttWzZMm+d0aNH66qrrlJwcLCuv/56HThwQP369dOwYcO0e/duvffee5o3b546deqk4cOHKzQ0VP7+/vr1r3+tgoICbd++XcePH9fq1au99R566CENGDBAX375pSRp+fLlatu2rUJCQtShQwdt27bNe/6YmBi98MILuuaaaxQcHKxBgwaVCdkAUNURiAGgki3ZkqaOz65R0/ErdKx+C724YKnWrVunhIQEde3aVUlJSUpKSlJCQoLS09MVHR1d5vjo6GilpaV5lxs2bChJKiws1LvvvquWLVsqOztbAwYM0MaNG1VUVOTdd8aMGWrVqpWCg4MlSXl5ecrMzNRvfvMbZWVl6fbbb9f333+vtLQ0zZ49W8eOHdNnn32mu+66S//4xz+UlZWl++67T3369NHx48e9dRctWqT3339fu3fv1rZt2zRv3rwKfAcBwFkEYgCoREu2pOnJxV8pLadAVlJpw1Z6e9kqrfhgjRISEpSQkFAmEEdERCglJaVMjdTUVEVGRv6o9ieffKLS0lK1bt1aAQEBuvPOO9WiRQvv9tzcXD333HNatGiRDh8+LEnyeDwqLS3V7NmzlZiYqMDAQHk8Hj311FPeu9Avv/yy7rvvPrVv315+fn765S9/qcDAQH3yySfe2mPHjlVERITq1aun22+/XVu3bq2Adw8AKgaBGAAq0fRV21VQVOJdvrLJz3Q0ZZsOZeepcePG6ty5s95//31lZWUpLi5OPXv21I4dO/T666+ruLhYb775pr755hv17t37R7XT09Pl8XjKrDt191iSSkpK5O/vr7CwMBUXF0uS8vPzlZOTox9++EGzZs3SokWLFBISottuu03Z2dmSpJSUFM2YMUMhISHe1969e5Wenu6t3ahRI+/fNWvWVH5+vjNvGABUAn9fNwAAbpKeU1BmOaBepGoEXCn/iNaSpDp16ig2NlZhYWHy8/NTaGioli9frkceeUQPPPCAmjVrpuXLl6t+/freGltSD6vjs2u0a9seHcrKUWrWUe+2Q4cOSZJq1aqlwMBA9ejRQ82bN1dQUJAkKTw8XMHBwQoKCtL999+vQ4cOaf78+ZKkPXv2qGnTpoqKitKECRM0YcKECn1vAMBXHL1DbIypZ4x5xxhz1BiTYowZ6mR9AKjuIkKCfrSu8ZjXdM2wJ73LycnJWrlypXe5U6dO2rx5s3Jzc7V582Z16tTJu+3RPy3QGvszpeUU6IrIljKBtfV1Yaj+9XmKFi9erB07dujBBx/Utddeq2+++UYPP/ywDh06pH79+kmS1q5dq1tvvVX33HOP9u3bpz/+8Y+SpLS0NG3fvl3WWt1333166aWX9Omnn8paq6NHj2rFihXKy8urqLcJACqV00Mm/iqpUFJDScMk/d0Y08bhcwBV2qhRozRx4kRft4EqalxiCwUF+JVZFxTgp3GJLc5xxPmdPgTD+AUorN9TOrz1Aw3s1Epvvvmm+vfvL0lq3ry5fvvb36p79+66+uqry4RqSXruuefUrFkz3XjjjapTp466d++u7du3S5Li4+M1e/ZsjRkzRnXr1lWzZs340RyAy4qx1jpTyJhakg5L+n/W2h0n170mKc1aO/5cx8XHx1smfsflZNSoUWrcuLGmTJni61ZQRS3Zkqbpq7YrPadAESFBGpfYQn3jfvwjuYvRdPwKne1b3Eja/WwvPo8AcJIxZrO1Nv5s25wcQ9xcUsmpMHzSl5ISztLQvZLulaQmTZo42AIAVH194yLLHYDPFBESpLQzxiWfWg8AuDhODpmoLSn3jHW5kjxn7mitnWWtjbfWxoeFhTnYAlD5tmzZouuuu04ej+dHDySYPXu2mjVrpnr16qlPnz5lfpW/evVqtWjRQsHBwXrwwQeVkJCgOXPm+OISUI05PQQDANzIyUCcL6nOGevqSOJXF7hsFRYWqm/fvhoxYoT3QQj/+te/JElr1qzRk08+qUWLFmn//v2Kjo7W4MGDJUmZmZm68847NW3aNGVlZalFixbauHGjLy8F1VTfuEhN6/8zRYYEyUiKDAnStP4/896BnjdvHsMlAOACKmIMcRtr7Xcn170qKZ0xxLhcrVu3ToMHD1ZaWpqMMZKkDh06qFu3btq/f79CQ0P1/PPPSzox32vdunX13Xffad26dfr73/+uTZs2SZKstWrSpImeeeYZ/epXv/LZ9QAAcLmqlDHE1tqjxpjFkn5njPmVpLaSfiGpg1PnAKqC038QdUXKJtWq28AbhiV5H7Obnp6u6667zru+du3aCg0NVVpamtLT0xUVFeXdZoxR48aNK+8iAACAl9PTrj0oKUjSIUlvSHrAWvu1w+cAfObMx+7mmtrak7pX73yxz7tPamqqJP3okbtHjx5VVlaWIiMjFR4ern37/neMtbbMMgAAqDyOBmJrbba1tq+1tpa1tom19nUn6wO+duZjdwMjW8oaPz3+zDQVFxdr8eLF+uyzzyRJQ4cO1dy5c7V161YdP35cTz31lNq3b6+YmBj16tVLX331lZYsWaLi4mL99a9/1YEDB3x1WQAAuJrTd4iBy9qZj9099SCE9M/eV926dcs8COHnP/+5fv/73+uOO+5QeHi4du7cqYULF0qS6tevr7feekv/93//p9DQUH3zzTeKj49XYGBgpV8TAABu59iP6sqLH9WhOun47JqzzvkaGRKkj8d3K3fd0tJSNW7cWAsWLNDNN998KS0CAICzON+P6rhDDPwETs75umrVKuXk5Oj48eOaOnWqrLW68cYbnWoVAABcJCefVAdc9k7N7erEY3c3bdqkoUOHqrCwUK1bt9aSJUsUFMTTxQAAqGwMmfgJiouL5e/P/0MAAABUNwyZuIAvvvhCcXFx8ng8GjBggAYNGqSJEydq7dq1aty4sZ577jk1atRIo0ePVmlpqZ599lldddVVCg0N1cCBA5Wdne2t9cknn6hDhw4KCQnRtddeq7Vr13q3de3aVU8//bQ6duwoj8ejW2+9VZmZmT64YgAAAJzi+kBcWFiofv36adSoUcrOztaQIUP0zjvveLcfOHBA2dnZSklJ0axZszRz5kwtWbJESUlJSk9PV926dfXQQw9JktLS0tSrVy9NnDhR2dnZeuGFF3THHXcoIyPDW+/111/X3LlzdejQIRUWFuqFF16o9GsGAADA/7g+EH/yyScqLi7W2LFjFRAQoP79++uGG27wbq9Ro4YmT56swMBABQUF6R//+If+8Ic/qHHjxgoMDNSkSZP09ttvq7i4WPPnz1fPnj3Vs2dP1ahRQ7fccovi4+P13nvveeuNHj1azZs3V1BQkAYOHKitW7f64rIBAABwkusHxKanpysyMrLMo3dPf6RuWFiYrrzySu9ySkqK+vXrpxo1/vf/En5+fjp48KBSUlL01ltvadmyZd5tRUVFZabRatSokffvmjVrKj8/3/FrAgAAwMVzbSBesiVN01dt165te5S9fZfe+WKf+l3XWJK0d+9eXXXVVZJUJihLJ8LyP//5T3Xs2PFHNaOiojRixAjNnj274i8AAAAAjnDlkIklW9L05OKvlJZToCsiW6rE1tD9T/1B//o8RUuXLvU+evds7r//ft19992Kjz/xI8WMjAwZY/T9999r+PDhWrZsmVatWqWSkhIdO3ZMa9eu1b59+yrr0gAAAPATuTIQT1+1XQVFJZL+9+jdw1tWaWCnlpo/f6gYj6cAABI4SURBVL569+59zkfoPvLII4qLi9PXX38tj8dT5kEKUVFRWrp0qaZOnaqwsDBFRUVp+vTpKi0trZTrAgAAwE/nynmIm45fobNdtZG0+9leat++ve6//36NHj36rMfPmzdPc+bM0YYNG04cZ4y+++47NWvWrOKaBgAAQLkxD/EZIkJOPA0s99N/KeOdqTqW+pVK8g8r3HOFunfvrs2bN6tDhw66++67FR4ersjISE2cOFElJSUXrJ2bm6uRI0cqLCxM0dHRmjJlivcOcXR0tDZv3ixJmj9/vowx+uabbyRJc+bMUd++fSvoigEAAHAurgzE4xJbKCjAT7Xa3KyC3ZtVeHCn9s97WF9M+YXWrl2rGTNm6De/+Y38/f31/fffa8uWLVq9erXmzJlzwdoPP/ywcnNztWvXLiUlJenVV1/V3LlzJUkJCQneB3WsW7dOsbGxSkpK8i4nJCRU2DUDAADg7FwZiPvGRWpa/58punGkrmz8/1QvuI7eXv8fvf3WIjVv3lyDBw/WypUr9ec//1m1atVSgwYN9Nhjj2nhwoXnrVtSUqI333xT06ZNk8fjUUxMjJ544gm99tprkk4E4lMBeP369XryySe9y0lJSQRiAAAAH3BlIJZOhOKPx3fTP6f+WpFZyeobF6n58+drxIgRSklJUVFRkcLDwxUSEqKQkBDdd999OnTo0HlrZmZmqrCwUNHR0d510dHRSktLk3QiEK9fv14HDhxQSUmJBg0apI8//lh79uxRbm6u2rZtW6HXDAAAgB9z3TzEp+YfTs8pUERIkB7p2k7btj2g//znP1q+fLmef/55BQQEKDAwUJmZmfL3v/i3qH79+goICFBKSopat24tSUpNTVVkZKQkqVmzZqpZs6ZmzpypLl26yOPxqFGjRpo1a5Y6depU5mEfAAAAqByuSmCnzz9sJaXlFOiZFd8p/uYeGjp0qG644QY1adJE4eHhuvXWW/XEE0/oyJEjKi0t1c6dO73DG87Fz89PAwcO1IQJE5SXl6eUlBT98Y9/1PDhw737JCQk6C9/+Yt3eETXrl3LLAMAAKByuSoQnz7/8CkFRSU60KC9vvrqK40YMcK7/tVXX1VhYaFat26tunXr6s4779T+/fsveI4XX3xRtWrVUmxsrDp16qShQ4fqrrvu8m5PSEhQXl6eunTpctZlAAAAVC5XzUN8rvmHS44cUta8h3TgwAHVqVOnUnoBAABA5WEe4pNOzT98OmtLVfLlcg0ePJgwDAAA4EKuCsSn5h8+pbTwmPb+aaD8D/xHkydP9mFnAAAA8BVXzTLRN+7EbA+nZpmIalBXMz/93rseAAAA7uOqQCydCMUEYAAAAJziqiETAAAAwJkIxAAAAHA1AjEAAABcjUAMAAAAV3MkEBtjxhhjko0xx40x85yoCQAAAFQGp2aZSJc0RVKipB8//QIAAACoohwJxNbaxZJkjImX1NiJmgAAAEBl8MkYYmPMvSeHWCRnZGT4ogUAAABAko8CsbV2lrU23lobHxYW5osWAAAAAEkXEYiNMWuNMfYcrw2V0SQAAABQUS44htha27US+gAAAAB8wpEf1Rlj/E/W8pPkZ4y5UlKxtbbYifoAAABARXFqDPFESQWSxksafvLviQ7VBgAAACqMU9OuTZI0yYlaAAAAQGXi0c0AAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVCMQAAABwNQIxAAAAXI1ADAAAAFcjEAMAAMDVLjkQG2MCjTEvG2NSjDF5xpgtxpgeTjQHAAAAVDQn7hD7S9orKUFSsKSnJS0yxsQ4UBsAAACoUP6XWsBae1TSpNNWLTfG7JZ0vaQ9l1ofAAAAqEiOjyE2xjSU1FzS107XBgAAAJzmaCA2xgRIWiDpFWvtt+fZ715jTLIxJjkjI8PJFgAAAICf5IKB2Biz1hhjz/HacNp+NSS9JqlQ0pjz1bTWzrLWxltr48PCwi75IgAAAIDyuuAYYmtt1wvtY4wxkl6W1FBST2tt0aW3BgAAAFS8S/5R3Ul/l9RKUndrbYFDNQEAAIAK58Q8xNGS7pPUVtIBY0z+ydewS+4OAAAAqGBOTLuWIsk40AsAAABQ6Xh0MwAAACpVTEyMPvzwwx+tb9OmjdauXVvp/Tg1hhgAAAC4JF9/7ZvHWHCHGAAAAK5GIAYAAIDPfPvtt2ratKkWLlxYZijFpEmTNHDgQI0cOVIej0dt2rRRcnKy97gvvvhCcXFx8ng8GjBggAYNGqSJEyeWqwcCMQAAAHziiy++0K233qoXX3xRgwcP/tH2d999V4MHD1ZOTo769OmjMWNOPPutsLBQ/fr106hRo5Sdna0hQ4bonXfeKXcfBGIAAABUuvXr16tPnz565ZVX1Lt377Pu06lTJ/Xs2VN+fn4aMWKEvvzyS0nSJ598ouLiYo0dO1YBAQHq37+/brjhhnL3QiAGAABApXvppZfUoUMH3Xzzzefcp1GjRt6/a9asqWPHjqm4uFjp6emKjIzUiYclnxAVFVXuXgjEAAAAqHBLtqSp47Nr1HT8Ch3IPabR//cHpaam6rHHHvvJtcLDw5WWliZrrXfd3r17y90bgRgAAAAVasmWND25+Cul5RTISioutXr9iww9/Pw/tW7dOo0fP/4n1bvpppvk5+env/zlLyouLtbSpUv12Weflbs/5iEGAABAhZq+arsKikrKrDteUqq/bTygDz74QDfffLMCAgIuut4VV1yhxYsX61e/+pWefPJJ9ejRQ71791ZgYGC5+jOn32r2hfj4eHv6FBoAAAC4vDQdv0JnS5xG0u5nezlyjvbt2+v+++/X6NGjz7rdGLPZWht/tm0MmQCAy9ykSZM0fPhwX7cBwMUiQoJ+0vqLkZSUpAMHDqi4uFivvPKKtm3bpttuu61ctQjEAAAAqFDjElsoKMCvzLqgAD+NS2xR7prbt2/Xtddeq+DgYM2YMUNvv/22wsPDy1WLMcQAUM0UFxfL35+vbwDVR9+4SEknxhKn5xQoIiRI4xJbeNeXx7333qt7773Xkf64QwwA1UBMTIyee+45XXPNNapVq5Y2bNigDh06KCQkRNdee63Wrl3r3Xf37t1KSEiQx+PRLbfcoszMTN81DgAn9Y2L1Mfju2n3s7308fhulxSGncYtBgCoJt544w2tWLFCNWrU0DXXXKPXXntNt912mz766CPdcccd+vbbbxUWFqahQ4fqpptu0urVq/Xpp5+qV69e+sUvfuHr9gGgyuIOMQBUE2PHjlVUVJTmz5+vnj17qmfPnqpRo4ZuueUWxcfH67333lNqaqo+//xz/f73v1dgYKC6dOmi22+/3detA0CVxh1iAKiilmxJ8463O5B7TKnHT/waOyUlRW+99ZaWLVvm3beoqEg333yz0tPTVbduXdWqVcu7LTo6+pKe4AQAlzsCMQBUQaee6nRqIvviUqt/btijtjemKSoqSiNGjNDs2bN/dFxKSooOHz6so0ePekNxamqqjDGV2j8AVCcMmQCAKuhcT3Wavmq7hg8frmXLlmnVqlUqKSnRsWPHtHbtWu3bt0/R0dGKj4/XM888o8LCQm3YsKHMneSz6dq1q+bMmeNdnjhxourXr69GjRpVyLUBQFVDIAaAKig9p+Cc66OiorR06VJNnTpVYWFhioqK0vTp01VaWipJev311/Xpp5+qXr16mjx5skaOHHnR5927d69mzJihb775RgcOHHDkWgCgqmPIBABUQREhQUo7LRQ3fuCf3vXSiUeUJiUlnfXY2NhYrV+/vlznTUlJUWhoqBo0aFCu4wGgOuIOMQBUQRf7VKe5c+eWmUWiWbNmGjhwoHc5KipKW7du1caNG9WuXTsFBwerXbt22rhx44/O+eGHH+qWW25Renq6ateurVGjRjl7UQBQRRGIAaAK6hsXqWn9f6bIkCAZSZEhQZrW/2c/msg+ISFB69evV2lpqfbv36+ioiJ9/PHHkqRdu3YpPz9fTZo0Ua9evTR27FhlZWXp8ccfV69evZSVlVWmVvfu3bVy5UpFREQoPz9f8+bNq6SrBQDfYsgEAFRRfeMiL/gkp9jYWHk8Hm3dulU7duxQYmKitm7dqm+//VabNm1S586dtWLFCl199dUaMWKEJGnIkCGaOXOmli1bxl1gABCBGACqpdPnKD5Wv4VeXLBUQQUZSkhIUEhIiJKSkrRp0yYlJCQoPT1d0dHRZY6Pjo5WWlqaj7oHgKqFIRMAUM2cmqM4LadAVlJpw1Z6e9kqrfhgjRISEpSQkKCkpCQlJSUpISFBERERSklJKVMjNTVVkZHnv/sMAG5BIAaAaubMOYqvbPIzHU3ZpkPZeWrcuLE6d+6s999/X1lZWYqLi1PPnj21Y8cOvf766youLtabb76pb775Rr179/bhVQBA1cGQCQCoZs6cozigXqRqBFwp/4jWkqQ6deooNjZWYWFh8vPzU2hoqJYvX65HHnlEDzzwgJo1a6bly5erfv36vmgfAKocY6299CLGzJf0c0m1JB2Q9Ly1ds75jzohPj7eJicnX3IPAOAWHZ9dU2aO4lMiQ4L08fhuPugIAKo+Y8xma2382bY5NWRimqQYa20dSX0kTTHGXO9QbQDAaS52jmIAwMVxZMiEtfbr0xdPvq6StNmJ+gCA/zk1FdupWSYiQoI0LrHFBadoAwCcnSNDJiTJGPM3SaMkBUnaIqmLtTb/HPveK+leSWrSpMn1Z/76GQAAAHBSZQyZkLX2QUkeSZ0lLZZ0/Dz7zrLWxltr48PCwpxqAQAAAPjJLhiIjTFrjTH2HK8Np+9rrS2x1m6Q1FjSAxXVNAAAAOCUC44httZ2LWfdq8pxHAAAAFCpLnnIhDGmgTFmsDGmtjHGzxiTKGmIpDWX3h4AAABQsZyYZcLqxPCIl3QiYKdIetRau9SB2gAAAECFuuRAbK3NkJTgQC8AAABApXNslgkAAACgOiIQAwAAwNUIxAAAAHA1AjEAAABczbFHN5e7AWMydGJmigupLymzgtvB5YnPDsqDzw3Ki88OyoPPTcWLttae9RHJPg/EF8sYk3yu508D58NnB+XB5wblxWcH5cHnxrcYMgEAAABXIxADAADA1apTIJ7l6wZQbfHZQXnwuUF58dlBefC58aFqM4YYAAAAqAjV6Q4xAAAA4DgCMQAAAFyt2gZiY8zVxphjxpj5vu4FVZsxJtAY87IxJsUYk2eM2WKM6eHrvlA1GWPqGWPeMcYcPfmZGerrnlD18T2DS0Wu8a1qG4gl/VXS575uAtWCv6S9khIkBUt6WtIiY0yMD3tC1fVXSYWSGkoaJunvxpg2vm0J1QDfM7hU5BofqpaB2BgzWFKOpI983QuqPmvtUWvtJGvtHmttqbV2uaTdkq73dW+oWowxtSTdIelpa22+tXaDpHcljfBtZ6jq+J7BpSDX+F61C8TGmDqSfifpCV/3gurJGNNQUnNJX/u6F1Q5zSWVWGt3nLbuS0ncIcZPwvcMLha5pmqodoFY0u8lvWyt3evrRlD9GGMCJC2Q9Iq19ltf94Mqp7ak3DPW5Ury+KAXVFN8z+AnItdUAVUqEBtj1hpj7DleG4wxbSV1l/QnX/eKquNCn5vT9qsh6TWdGB86xmcNoyrLl1TnjHV1JOX5oBdUQ3zP4Kcg11Qd/r5u4HTW2q7n226MeVRSjKRUY4x04m6OnzGmtbX2ugpvEFXShT43kmROfGBe1okfSvW01hZVdF+olnZI8jfGXG2t/e7kumvFP3vjIvA9g3LoKnJNlVCtnlRnjKmpsndvfq0TH6QHrLUZPmkK1YIx5iVJbSV1t9bm+7ofVF3GmIWSrKRf6cRn5j1JHay1hGKcF98z+KnINVVHlbpDfCHW2h8k/XBq2RiTL+kYHxqcjzEmWtJ9ko5LOnDy/8Il6T5r7QKfNYaq6kFJ/5R0SFKWTvyHiTCM8+J7BuVBrqk6qtUdYgAAAMBpVepHdQAAAEBlIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABXIxADAADA1QjEAAAAcDUCMQAAAFyNQAwAAABX+/+YaZ5x4TsHfwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "embedding2 = embedding * 1.0 + pos_encoding[:,:,:] * 1.0\n",
    "\n",
    "plot_words(embedding2, sequences, 0)\n",
    "plot_words(embedding2, sequences, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Wow look at the big difference between the plots! Both plots have changed drastically compared to their original counterparts. Notice that in the second image, which corresponds to the sentence in which similar words are not together, very dissimilar words such as `red` and `wolf` appear more close.\n",
    "\n",
    "Now you can try different relative weights and see how this strongly impacts the vector representation of the words in the sentence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtMAAAFoCAYAAACVEXcaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3df3QW1YH/8fcloTFCIAKRHyEGAQVlraKhqCChiiKg1IIiKqhoVbpFFK3fRcEutopaStu17W6rdqFqLZSKWEFFqSWCiooCulihQAmQoBIggfA7ZL5/AE8BQcoYEhLer3NyTubemTt3MufJ88nNnfuEKIqQJEmSdPhqVXUHJEmSpOrKMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTMlV3YEv06hRo6hFixZV3Q1JkiTVcO+//35RFEUZh3vcUR2mW7Rowdy5c6u6G5IkSarhQgj5cY5zmockSZIUk2FakiRJiskwLUmSJMVkmJakY1TXrl158sknE9sjR46kUaNGNGnSpAp7JUnVy1H9AKIkqXKsXLmSsWPHkp+fz4knnljV3ZGkasORaUkS+fn5NGzY0CAtSYfJMC1J1dC4ceO4/PLLE9utW7emX79+ie2srCzmz5/PW2+9RYcOHahfvz4dOnTgrbfe+kJbM2bM4OKLL6awsJC6dety4403VsYlSFKNYJiWpGooNzeXWbNmUV5ezurVq9mxYwdvvvkmAMuWLaO0tJSTTjqJXr16MXToUNauXctdd91Fr169WLt27T5tdevWjZdffplmzZpRWlrK+PHjq+CKJKl6MkxLUjXUsmVL0tLSmD9/Pnl5eXTv3p3MzEw++eQT8vLyuOCCC5g2bRqnnHIKAwcOJDk5mWuuuYa2bdvy4osvVnX3JanG8AFESaompswrYMz0RRQWb6FZeiotz+jAzJkzWbJkCbm5uaSnp5OXl8fbb79Nbm4uhYWFZGdn79NGdnY2BQUFVXQFklTzODItSdXAlHkF3Dv5IwqKtxABBcVbWByymPDCK8yaNYvc3Fxyc3PJy8sjLy+P3NxcmjVrRn7+vp+Ou2LFCjIzM6vmIiSpBjJMS1I1MGb6Irbs2LlPWa3Mdrw/ZzZbtmyhefPmXHDBBbzyyiusXbuW9u3b07NnTxYvXsyzzz5LWVkZEydO5OOPP+ayyy6roquQpJrHaR6SVA0UFm/5QlntBpmE2sdxwQUXAFCvXj1atmxJRkYGSUlJNGzYkKlTp3LHHXfw3e9+l9atWzN16lQaNWpU2d2XpBorRFFU1X04qJycnGju3LlV3Q1JqnKdHnmdggME6sz0VN4cfmEV9EiSapYQwvtRFOUc7nFO85CkauCe7m1IrZ20T1lq7STu6d6minokSQKneUhStXBF+10PDe69msc93dskyiVJVcMwLUnVxBXtMw3PknSUcZqHJEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEk10I033sjIkSOruhtSjWeYlo4iixYton379qSlpfHYY49VWLvjx4+nc+fOsY6dOXMmzZs3T2y3aNGCGTNmVFTXJEmq1pKrugOS/unHP/4xXbt2Zd68ebHbWL58OSeffDI7duwgOdmXuCRJR5Ij09JRJD8/n3bt2lV1NyRVQ/PmzePss88mLS2Nq6++mq1btybqnnjiCVq3bk2DBg3o3bs3hYWFibpXX32VNm3aUL9+ff793/+d3Nxcnnzyyaq4BKlaMkxLR4kLL7yQv/71rwwZMoS6deuyYMECrr/+ejIyMsjOzubBBx+kvLwcgPLych588EGys7M58cQTuf766ykpKQGgS5cuAKSnp1O3bl3efvttAKIo4vbbb6d+/fq0bduWv/zlL4lzjxs3jtNOO420tDRatmzJb37zm0q+eklfxfbt27niiisYOHAg69at46qrruK5554D4PXXX+fee+/lj3/8I6tXryY7O5v+/fsDUFRUxJVXXsnDDz/M2rVradOmDW+99VZVXopU7RimpaPE66+/zgUXXMAvf/lLSktLGTt2LCUlJSxbtoy8vDyeeuopxo0bB+yaAz1+/Hj++te/smzZMkpLSxkyZAgAb7zxBgDFxcWUlpZy3nnnAfDOO+/QsmVLioqKeOCBB+jTpw/r1q0D4MQTT2Tq1Kls2LCBcePGMWzYMD744IMq+ClIimPOnDns2LGDO++8k9q1a3PllVfSoUMHAH7/+99z0003cfbZZ5OSksLDDz/M22+/zfLly3nppZdo164dffr0ITk5maFDh9KkSZMqvhqpejFMS0ehnTt3MnHiRB5++GHS0tJo0aIFd999N08//TSw683xrrvuomXLltStW5eHH36YCRMmUFZWdtA2TzzxxMQb7dVXX02bNm2YNm0aAL169aJVq1aEEMjNzeWSSy5h1qxZlXKtkr66wsJCMjMzCSEkyrKzsxN1e74HqFu3Lg0bNqSgoIDCwkKysrISdSGEfR44lnRoFRamQwgpIYTfhhDyQwgbQwjzQgg99qq/KITwSQhhcwjhryGE7C9rTzpWTJlXQKdHXufk4dOYt6KYeSvWU1RUxPbt2/d5A8zOzqagoAD44ptjdnY2ZWVlfPbZZwc9z4HeaPfMm3z55Zc599xzadCgAenp6bz00ksUFRVV9KVKqmB7fn8M+/NyFiz6B89/sCpRt2LFCgCaNWtGfn5+onzTpk2sXbuWzMxMmjZtyqpV/zwmiqJ9tiUdWkWOTCcDK4FcoD5wP/DHEEKLEEIjYPLusgbAXGBiBZ5bqpamzCvg3skfUVC8hQjYVraTyR8UMHvlNmrXrr3PG+CKFSvIzMwEvvjmuGLFCpKTk2ncuPE+gXlvBQUFRFG0zzHNmjVj27Zt9O3bl+9///t89tlnFBcX07Nnz332lXT02fv3x9cy21JOLQbf+yDPvZfP5MmTeffddwG49tprGTduHPPnz2fbtm3cd999dOzYkRYtWtCrVy8++ugjpkyZQllZGb/61a/49NNPq/jKpOqlwsJ0FEWboigaFUXR8iiKyqMomgr8AzgH6AMsjKJoUhRFW4FRwJkhhLYVdX6pOhozfRFbduzcp2z7znJ+OmMJ/fr1Y8SIEWzcuJH8/Hx++tOfMmDAAACuueYafvazn/GPf/yD0tJS7rvvPq6++mqSk5PJyMigVq1aLFu2bJ92P//8cx577DF27NjBpEmT+Nvf/kbPnj3Zvn0727ZtIyMjg+TkZF5++WVeffXVSvsZSIpn798fIak2Gd++j/XzX6Nf59OYOHEiffr0AeCiiy7iRz/6EX379qVp06YsXbqUCRMmANCoUSMmTZrE//t//4+GDRvy8ccfk5OTQ0pKSpVdl1TdHLFFaEMIjYFTgYXAd4EFe+qiKNoUQlgKtAM+OVJ9kI52hcVbDlo+9Re/4Pbbb6dly5Ycd9xx3HLLLdx0000A3HTTTRQWFtKlSxe2bt1K9+7d+cUvfgHA8ccfz4gRI+jUqRM7duzglVdeAaBjx478/e9/p1GjRjRu3Jg//elPNGzYEIDHHnuMfv36sW3bNi6//HJ69+5dCVcv6avY//dHStNTaDboMQIw8ZFe+9QNHjyYwYMHH7CdSy+9lMWLFwO7Vgpq3ry586alwxCOxL9yQwi1gZeBpVEU3RZC+C2wJoqi4Xvt8ybwRBRF4/c79lbgVoCTTjrpnL3/lS3VNJ0eeZ2CAwTqzPRU3hx+YRX0SFJ1UVG/P6ZPn07Hjh1JTU1lzJgx/OpXv2LZsmWkpqZWZHelo14I4f0oinIO97gKX80jhFALeBrYDgzZXVwK1Ntv13rAxv2Pj6Lo8SiKcqIoysnIyKjo7klHlXu6tyG1dtI+Zam1k7ine5sq6pGk6qKifn+8/fbbtGrVikaNGvHiiy8yZcoUg7R0GCp0ZDrsevLpf4EWQM8oirbsLr8VuCGKok67t+sAa4Czoyg66DSPnJycaO7cuRXWP+loNGVeAWOmL6KweAvN0lO5p3sbrmifWdXdklQN+PtDqjhxR6YrOkz/GjgL6BZFUele5RnAEuAmYBrwAJAbRdG5X9aeYVqSJEmVocqneexeN/o2doXpT0MIpbu/rouiaA3QF3gIWA90BPpX1LklSZKkqlBhq3lEUZQPHHiB2131MwCXwpMkSVKN4ceJS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxVWiYDiEMCSHMDSFsCyGM36/uohDCJyGEzSGEv4YQsivy3JIkSVJlq+iR6ULgQeB/9y4MITQCJgP3Aw2AucDECj63JEmSVKmSK7KxKIomA4QQcoDme1X1ARZGUTRpd/0ooCiE0DaKok8qsg+SJElSZamsOdPtgAV7NqIo2gQs3V0uSZIkVUuVFabrAiX7lZUAafvvGEK4dfe867lr1qyplM5JkiRJcVRWmC4F6u1XVg/YuP+OURQ9HkVRThRFORkZGZXSOUmSJCmOygrTC4Ez92yEEOoArXaXS5IkSdVSRS+NlxxCOA5IApJCCMeFEJKB54F/CyH03V3/A+BDHz6UJElSdVbRI9MjgS3AcGDA7u9HRlG0BugLPASsBzoC/Sv43JIkSVKlquil8UYBow5SNwNoW5HnkyRJkqqSHycuSZIkxWSYliRJkmIyTEuSJEkxGaaV0KJFC2bMmFFh7Y0fP57OnTtXSFs33ngjI0eOjHXsqFGjGDBgAADLly8nhEBZWVmF9EuSJB3bDNM66lRkCJckSTqSDNOSJElSTIZp7eO9997j9NNP54QTTmDQoEFs3bqV9evXc9lll5GRkcEJJ5zAZZddxqpVqxLHjB8/npYtW5KWlsbJJ5/M73//+wO2fc8999C5c2dKSkooKSnh5ptvpmnTpmRmZjJy5Eh27tzJ3/72NwYPHszbb79N3bp1SU9PTxxfVFTExRdfTFpaGrm5ueTn5yfq7rjjDrKysqhXrx7nnHMOs2bNOnI/JEmSpN0M09rH73//e6ZPn87SpUtZvHgxDz74IOXl5QwaNIj8/HxWrFhBamoqQ4YMAWDTpk0MHTqUl19+mY0bN/LWW29x1lln7dNmeXk5t9xyCx9++CGvvvoq9evX54YbbiA5OZklS5Ywb948Xn31VZ588klOO+00fv3rX3PeeedRWlpKcXHxPn27//77KSoq4qyzzuK6665L1HXo0IH58+ezbt06rr32Wq666iq2bt1aOT80SZJ0zDJMax9DhgwhKyuLBg0aMGLECP7whz/QsGFD+vbty/HHH09aWhojRowgLy8vcUytWrX4v//7P7Zs2ULTpk1p165dom7Hjh1cc801rFu3jhdffJHjjz+ezz77jJdffpmf//zn1KlThxNPPJFhw4YxYcKEL+1br1696NKlCykpKTz00EO8/fbbrFy5EoABAwbQsGFDkpOTufvuu9m2bRuLFi06Mj8kSZKk3Sr0ExBV/UyZV8CY6YsoLN7CpyVbWbX9+ERddnY2hYWFbN68mWHDhvHKK6+wfv16ADZu3MjOnTupU6cOEydO5Cc/+Qk333wznTp1YuzYsbRtu+vDLpcsWcKCBQt49913+drXvgZAfn4+O3bsoGnTpolzlZeXk5WV9aV93bu+bt26NGjQgMLCQrKyshg7dixPPvkkhYWFhBDYsGEDRUVFFfZzkiRJOhBHpo9hU+YVcO/kjygo3kIElJVHPPnKe0yZVwDAihUraNasGWPHjmXRokW88847bNiwgTfeeAOAKIoA6N69O6+99hqrV6+mbdu23HLLLYlznHbaaYwbN44ePXokRoqzsrJISUmhqKiI4uJiiouL2bBhAwsXLgQghHDA/u4ZhQYoLS1l3bp1NGvWjFmzZvHoo4/yxz/+kfXr11NcXEz9+vUT/ZMk6Wg3btw4Lr/88sR269at6devX2I7KyuL+fPn89Zbb9GhQwfq169Phw4deOuttxL7dO3alZEjR3L++edTt25dLr/8ctauXct1111HvXr16NChA8uXL0/s/2XPG40aNYp+/fpx/fXXk5aWRrt27Zg7d+6R/SFUU4bpY9iY6YvYsmPnPmXr3nuRB/84m3Xr1jF69GiuvvpqNm7cSGpqKunp6axbt44HHnggsf9nn33Gn//8ZzZt2kRKSgp169YlKSlpnzavueYaRo8eTbdu3Vi6dClNmzblkksu4e6772bDhg2Ul5ezdOnSxNSRxo0bs2rVKrZv375POy+99BKzZ89m+/bt3H///XTs2JGsrCw2btxIcnIyGRkZlJWV8cMf/pANGzYcoZ+aJEkVLzc3l1mzZlFeXs7q1avZsWMHb775JgDLli2jtLSUk046iV69ejF06FDWrl3LXXfdRa9evVi7dm2inQkTJvD0009TUFDA0qVLOe+88xg0aBDr1q3jtNNO2+c9/FDPG/35z3+mf//+FBcX07t378TzUtqXYfoYVli85QtldU7P5cMn7qFly5a0bNmSkSNHcuedd7JlyxYaNWrEueeey6WXXprYv7y8nLFjx9KsWTMaNGhAXl4e//3f//2Fdm+44QZ+8IMfcOGFF7J8+XKeeuoptm/fnlg55Morr2T16tUAXHjhhbRr144mTZrQqFGjRBvXXnstDzzwAA0aNOD9999PrBrSvXt3evTowamnnkp2djbHHXfcIaeMSJJ0NNmzKtb8+fPJy8uje/fuZGZm8sknn5CXl8cFF1zAtGnTOOWUUxg4cCDJyclcc801tG3blhdffDHRzqBBg2jVqhX169enR48etGrVim7dupGcnMxVV13FvHnzEvse6nmjzp0707NnT5KSkhg4cCALFiyo1J9JdRGO5n+F5+TkRP5L4cjp9MjrFBwgUGemp/Lm8AuroEeSJB27BgwYwNlnn82SJUvo1KkTCxYsoFWrVrz99tucccYZlJWVMXfuXCZNmpQ4pn///pxxxhmMGDGCrl27MmDAAL7zne8AMHLkSFatWsX48eMBmDFjBoMHD2bJkiUAB3ze6LXXXuOiiy5i1KhRLFmyhGeeeQbY9QnCJ598Mjt27CA5uWY+chdCeD+KopzDPc6R6WPYPd3bkFp73ykZqbWTuKd7myrqkSRJx5Yp8wro9MjrnDx8GnM2N2bCC68wa9YscnNzyc3NJS8vj7y8PHJzc2nWrNk+n7EAu55vyszMPOzz+rxRxTFMH8OuaJ/Jw33OIDM9lcCuEemH+5zBFe0P/0UpSZIOz/4LAWxp1Ib358ymqHgjzZs354ILLuCVV15h7dq1tG/fnp49e7J48WKeffZZysrKmDhxIh9//DGXXXbZYZ/b540qTs0cp9e/7Ir2mYZnSZKqwP4LAdRukEmofRw7G+9aXrZevXq0bNmSjIwMkpKSaNiwIVOnTuWOO+7gu9/9Lq1bt2bq1Kn7PF/0r9r7eaM6deowbNgwnzeKyTnTkiRJVeDk4dM4UAoLwD8e6VXZ3TnmOWdakiSpGmmWnnpY5To6GaYlSZKqgAsB1AzOmZYkSaoCe55ZGjN9EYXFW2iWnso93dv4LFM1Y5iWJEmqIi4EUP05zUOSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiqtQwHUJoEEJ4PoSwKYSQH0K4tjLPL0mSJFWk5Eo+36+A7UBj4CxgWghhQRRFCyu5H5IkSdJXVmkj0yGEOkBf4P4oikqjKJoN/BkYWFl9kCRJkipSZU7zOBXYGUXR4r3KFgDt9t4phHBrCGFuCGHumjVrKrF7kiRJ0uGpzDBdFyjZr6wESNu7IIqix6MoyomiKCcjI6PSOidJkiQdrsoM06VAvf3K6gEbK7EPkiRJUoWpzDC9GEgOIZyyV9mZgA8fSpIkqVqqtDAdRdEmYDLwwxBCnRBCJ+BbwNOV1QdJkiSpIlX2h7b8O5AKfA78Afiuy+JJkiSpuqrUdaajKFoHXFGZ55QkSZKOFD9OXJIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpo+gUaNGMWDAgKruhiRJko4Qw7QkSZKqTIsWLRgzZgxf//rXqVOnDjfffDOfffYZPXr0IC0tjW7durF+/XoArrrqKpo0aUL9+vXp0qULCxcuTLRz44038r3vfY9evXqRlpZGx44dWbp06RHvv2H6MJSVlVV1FyRJkmqc5557jtdee43Fixfz4osv0qNHD0aPHk1RURHl5eU89thjAPTo0YO///3vfP7555x99tlcd911+7Tzhz/8gf/8z/9k/fr1tG7dmhEjRhzxvhumD6FFixY8+uijib+WZs+ezfnnn096ejpnnnkmM2fOTOz7j3/8g9zcXNLS0rj44ospKiqquo5LkiRVE7fffjuNGzcmMzOTCy64gI4dO9K+fXtSUlL49re/zbx58wC46aabSEtLIyUlhVGjRrFgwQJKSkoS7fTp04dvfOMbJCcnc9111zF//vwj3vfkI36GGuAPf/gD06ZNo1atWnz961/n6aef5tJLL+Uvf/kLffv25ZNPPiEjI4Nrr72W8847j1dffZV33nmHXr168a1vfauquy9JknTUmDKvgDHTF1FYvIVm6als3r6Txo0bJ+pTU1O/sF1aWsrOnTsZMWIEkyZNYs2aNdSqtWtMuKioiPr16wPQpEmTxHHHH388paWlR/x6HJn+FwwdOpSsrCyeeeYZevbsSc+ePalVqxYXX3wxOTk5vPTSS6xYsYL33nuPH/3oR6SkpNClSxcuv/zyqu66JEnSUWPKvALunfwRBcVbiICC4i2s37ydN5cc+r/5zz77LC+88AIzZsygpKSE5cuXAxBF0ZHt9CEYpg9gyrwCOj3yOicPn8anJVtZsS0VgPz8fCZNmkR6enria/bs2axevZrCwkJOOOEE6tSpk2gnOzu7qi5BkiTpqDNm+iK27Ni5T1kUwR/nrjzksRs3biQlJYWGDRuyefNm7rvvviPVzcNimN7P/n8xlZVH/O/s5UyZV0BWVhYDBw6kuLg48bVp0yaGDx9O06ZNWb9+PZs2bUq0tWLFiqq7EEmSpKNMYfGWA5avLd1+yGOvv/56srOzyczM5PTTT+fcc8+t6O7FEqp6aPzL5OTkRHPnzq3Uc3Z65HUK9rrRq/7nJhr2GErrs85jwnWn0KFDB373u9/RrVs3duzYwZw5c2jdujXNmzfn3HPPpXPnzowePZp3332Xnj170rt3b5555plKvQZJkqSj0f45a4/M9FTeHH5hFfTon0II70dRlHO4xzkyvZ+D/cVUWLyFrKwsXnjhBUaPHk1GRgZZWVmMGTOG8vJyYNdcnnfeeYcGDRrwwAMPcP3111dm1yVJko5q93RvQ2rtpH3KUmsncU/3NlXUo6/Oken9HM1/MUmSJFV3+6/mcU/3NlzRPrOquxV7ZNql8fZzT/c23Dv5o30mx1f3v5gkSZKOFle0zzwqwnNFMUzvZ8/NPRr/YpIkSdLRxTB9ADXtLyZJkiQdGT6AKEmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhRThYTpEMKQEMLcEMK2EML4A9RfFEL4JISwOYTw1xBCdkWcV5IkSapKFTUyXQg8CPzv/hUhhEbAZOB+oAEwF5hYQeeVJEmSqkxyRTQSRdFkgBBCDtB8v+o+wMIoiibt3mcUUBRCaBtF0ScVcX5JkiSpKlTGnOl2wII9G1EUbQKW7i7/ghDCrbunjMxds2ZNJXRPkiRJiqcywnRdoGS/shIg7UA7R1H0eBRFOVEU5WRkZBzxzkmSJElxHTJMhxBmhhCig3zN/hfOUQrU26+sHrAxToclSZKko8Uh50xHUdT1K55jIXDDno0QQh2g1e5ySZIkqdqqqKXxkkMIxwFJQFII4bgQwp6g/jzwbyGEvrv3+QHwoQ8fSpIkqbqrqDnTI8V196gAABH6SURBVIEtwHBgwO7vRwJEUbQG6As8BKwHOgL9K+i8kiRJUpWpqKXxRgGjvqR+BtC2Is4lSZIkHS38OHFJkiQpJsO0JEmSFJNhWpIkSYrJMC1JkiTFZJiWJEmSYjJMS5IkSTEZpiVJkqSYDNOSJElSTIZpSZIkKSbDtCRJkhSTYVqSJEmKyTAtSZIkxWSYliRJkmIyTEuSJEkxGaYlSZKkmAzTkiRJUkyGaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhukKUlZWVtVdkCRJUiUzTB/CBx98QPv27UlLS+Oqq67i6quvZuTIkcycOZPmzZvz6KOP0qRJEwYNGkR5eTmPPPIIrVq1omHDhvTr149169Yl2pozZw7nn38+6enpnHnmmcycOTNR17VrV+6//346depEWloal1xyCUVFRVVwxZIkSfpXGaa/xPbt2/n2t7/NjTfeyLp167jmmmt4/vnnE/Wffvop69atIz8/n8cff5zHHnuMKVOmkJeXR2FhISeccALf+973ACgoKKBXr16MHDmSdevW8ZOf/IS+ffuyZs2aRHvPPvss48aN4/PPP2f79u385Cc/qfRrliRJqmrz5s3j7LPPJi0tjauvvpr+/fszcuRIxo8fT+fOnffZN4TAkiVLANi2bRvf//73Oemkk2jcuDGDBw9my5YtiX2nTp3KWWedRXp6Oueffz4ffvjh3k2dEUL4fgjhwxBCSQhhYgjhuEP11TD9JebMmUNZWRlDhw6ldu3a9OnTh2984xuJ+lq1avHAAw+QkpJCamoqv/nNb3jooYdo3rw5KSkpjBo1ij/96U+UlZXxzDPP0LNnT3r27EmtWrW4+OKLycnJ4aWXXkq0N2jQIE499VRSU1Pp168f8+fPr4rLliRJqjLbt2/niiuuYODAgaxbt46rrrqK55577l869j/+4z9YvHgx8+fPZ8mSJRQUFPDDH/4Q2DXb4KabbuI3v/kNa9eu5bbbbqN3795s27Zt7yb6AZcCJwNfB2481DkN0/uZMq+ATo+8zsnDp3HL/7zK8SdkEEJI1GdlZSW+z8jI4Ljj/vkHS35+Pt/+9rdJT08nPT2d0047jaSkJD777DPy8/OZNGlSoi49PZ3Zs2ezevXqxPFNmjRJfH/88cdTWlp6hK9WkiTp6DJnzhx27NjBnXfeSe3atbnyyivp0KHDIY+LoognnniCn/3sZzRo0IC0tDTuu+8+JkyYAMATTzzBbbfdRseOHUlKSuKGG24gJSWFOXPm7N3MY1EUFUZRtA54ETjrUOc1TO9lyrwC7p38EQXFW4iAklCX5StW8fwHqxL7rFy5MvH93iEbdgXtl19+meLi4sTX1q1byczMJCsri4EDB+5Tt2nTJoYPH15ZlydJko5RLVq0YMaMGV8ob9eu3T7PcFWlPQOafX8ylQ216vHC/MJEXXZ29iGPX7NmDZs3b+acc85JDFxeeumliSm1+fn5jB07dp+BzZUrV1JYWLh3M5/u9f1moO6hzmuY3suY6YvYsmNnYjslsy0Rtbh71KOUlZXxwgsv8O677x70+MGDBzNixAjy8/OBXTf1hRdeAGDAgAG8+OKLTJ8+nZ07d7J161ZmzpzJqlWrDtqeJEnSkbRw4UK6du1a1d3YZ0AzqW4DtpasYfhzHzJlXgEAK1asAKBOnTps3rw5cdynn/4z+zZq1IjU1FQWLlyYGLgsKSlJ/Kc/KyuLESNG7DOwuXnzZq655pqv1HfD9F4Ki7fssx2SapPx7ftY9c400tPTeeaZZ7jssstISUk54PF33HEHvXv35pJLLiEtLY1zzz2Xd955B9h1A1944QVGjx5NRkYGWVlZjBkzhvLy8iN+XZIkSUezvQc0UzLbQkji8znP8+OXPmby5MmJwcwzzzyThQsXMn/+fLZu3cqoUaMSbdSqVYtbbrmFYcOG8fnnnwO7FoCYPn06ALfccgu//vWveeedd4iiiE2bNjFt2jQ2btz4lfpumN5Ls/TUL5SlND2FDsOepLS0lEmTJlFQUEDz5s3p2rXrF0aVa9WqxV133cWiRYvYuHEjS5cuZfTo0Yn6jh07kpeXx7p161izZg3Tpk3jpJNOAmDmzJl85zvfSex74403Mnv27CN0pZIk6Vj1ySefcPLJJzNhwoR9pn+MGjWKfv36cf3115OWlka7du2YO3du4riDLRdcEfYe0NwzmFn60V+Y88NvMXHiRPr06QPAqaeeyg9+8AO6devGKaec8oWVPR599FFat27NueeeS7169ejWrRuLFi0CICcnhyeeeIIhQ4Zwwgkn0Lp1a8aPH/+V+26Y3ss93duQWjtpn7KocCHfOecEysrK+N3vfseHH37IpZdeWkU9lCRJiu+DDz7gkksu4Re/+AX9+/f/Qv2f//xn+vfvT3FxMb1792bIkCHAoZcL/qr2H9BMaXoKzQY9xnkPTGXixImkpv6zfsSIERQVFbFy5UoGDBhAFEW0bt0agOOOO47Ro0ezbNkyNmzYwN/+9jeGDh2aOPbSSy/lvffeo7i4mNWrVzNp0iTS0tL2VH8URVFiYnkURaOiKBpwqL4bpvdyRftMHu5zBpnpqQQgMz2VXidFDB9wKfXr12fs2LH86U9/omnTplXdVUmSpMMya9Ysevfuze9+9zsuu+yyA+7TuXNnevbsSVJSEgMHDmTBggXAoZcL/qoONKCZWjuJe7q3qbBzHCnJVd2Bo80V7TO5on3mXiUX8t8P/keV9UeSJCmOKfMKGDN9EYXFW/i0ZCv/9cv/5pKLvsk3v/nNgx6z/zK9W7dupaysjMLCQjIzMw+6XPBXtSd77elvs/RU7uneZr9MdnQyTEuSJNUwe1bH2PNQX1l5RN1ug1nwt2kMGzaMn/3sZ4fVXtOmTSkoKCCKokSgXrlyJa1ataqwPn9xQPOfKmJu85HiNA9JkqQaZv/lfgF2JB1Hep9RvPHGG4f9ORfnnXceSUlJ/PKXv/yXlgs+ljgyLUmSVMPsv9zvHp9vS+L9117jm9/8JrVr1/6X2/va177G5MmT+c53vsO9995Ljx49vnS54GNJiKKoqvtwUDk5OdHeS7JIkiTp0Do98joFBwjUmempvDn8wgo5R8eOHRk8eDCDBg2qkPaqWgjh/SiKcg73OKd5SJIk1TBHYnWMvLw8Pv30U5cL3o9hupoZP378PguUhxBYsmRJFfZIkiQdbQ603O/Dfc74SqtjLFq0iDPPPNPlgvfjnGlJkqQa6MtWx4jj1ltv5dZbb62w9moKR6YlSZKkmAzTVWTMmDH07dt3n7Lbb7+dO++8k5KSEm6++WaaNm1KZmYmI0eOZOfOnQdp6Z9KSkq4/vrrycjIIDs7mwcffJDy8nIAsrOzef/99wF45plnCCHw8ccfA/Dkk09yxRVXVPAVSpIk1XyG6SoyYMAAXnnlFYqLiwEoKytj4sSJDBw4kBtuuIHk5GSWLFnCvHnzePXVV3nyyScP2ebtt99OSUkJy5YtIy8vj6eeeopx48YBkJuby8yZMwF44403aNmyJXl5eYnt3NzcI3OhkiRJNZhhuoo0bdqULl26MGnSJABeeeUVGjVqRPPmzXn55Zf5+c9/Tp06dTjxxBMZNmwYEyZM+NL2du7cycSJE3n44YdJS0ujRYsW3H333Tz99NPArjC9JzzPmjWLe++9N7Gdl5dnmJYkSYrBMF3JpswroNMjr3Py8GksSz+Hn//6t8CuqRcDBw4kPz+fHTt20LRpU9LT00lPT+e2227j888//9J2i4qK2L59O9nZ2Ymy7OxsCgoKgF1hetasWXz66afs3LmTq6++mjfffJPly5dTUlLCWWeddeQuWpIkqYZyNY9KNGVeAfdO/ijx8Z5bM89h1ZT/4r8m/YWpU6fy4x//mNq1a5OSkkJRURHJyf/67WnUqBG1a9cmPz+f008/HYAVK1aQmbnrKd7WrVtz/PHH89hjj9GlSxfS0tJo0qQJjz/+OJ07d6ZWLf+ukiRJOlwmqEo0ZvqiRJAGCMlf4/hTz+e+obfwjW98g5NOOommTZtyySWXcPfdd7NhwwbKy8tZunRpYkrGwSQlJdGvXz9GjBjBxo0byc/P56c//SkDBgxI7JObm8svf/nLxJSOrl277rMtSZKkw2OYrkSFB/hYzzpnXMTmT//BwIEDE2VPPfUU27dv5/TTT+eEE07gyiuvZPXq1Yds/xe/+AV16tShZcuWdO7cmWuvvZabbropUZ+bm8vGjRvp0qXLAbclSZJ0eEIURVXdh4PKycmJ5s6dW9XdqDCdHnmdgv0CddmGz1n95HdZX/Q59erVq6KeSZIkHdtCCO9HUZRzuMc5Ml2J7unehtTaSYntKCpn8/sv0LXHFQZpSZKkasgHECvRno/0HDN9Eas+X8+qXw0gs/lJjP/TjCrumSRJkuJwmockSZKOeU7zkCRJkirZVw7TIYSUEMJvQwj5IYSNIYR5IYQe++1zUQjhkxDC5hDCX0MI2QdrT5IkSaouKmJkOhlYCeQC9YH7gT+GEFoAhBAaAZN3lzcA5gITK+C8kiRJUpX6yg8gRlG0CRi1V9HUEMI/gHOA5UAfYGEURZMAQgijgKIQQtsoij75queXJEmSqkqFz5kOITQGTgUW7i5qByzYU787fC/dXS5JkiRVWxUapkMItYHfA7/ba9S5LlCy364lQNpB2rg1hDA3hDB3zZo1Fdk9SZIkqUIdMkyHEGaGEKKDfM3ea79awNPAdmDIXk2UAvt/Ikk9YOOBzhdF0eNRFOVEUZSTkZFx2BckSZIkVZZDzpmOoqjrofYJIQTgt0BjoGcURTv2ql4I3LDXvnWAVvxzGogkSZJULVXUNI//AU4DLo+iaMt+dc8D/xZC6BtCOA74AfChDx9KkiSpuquIdaazgduAs4BPQwilu7+uA4iiaA3QF3gIWA90BPp/1fNKkiRJVe2o/jjxEMIaIL+q+3EMaAQUVXUndMR4f2s272/N5b2t2by/R5/sKIoO+4G9ozpMq3KEEObG+Sx6VQ/e35rN+1tzeW9rNu9vzVHh60xLkiRJxwrDtCRJkhSTYVoAj1d1B3REeX9rNu9vzeW9rdm8vzWEc6YlSZKkmByZliRJkmIyTEuSJEkxGaaPUSGElBDCb0MI+SGEjSGEeSGEHvvtc1EI4ZMQwuYQwl93f0CPqokQwpAQwtwQwrYQwvgD1Ht/q7EQQoMQwvMhhE27X8fXVnWfFM+XvVZ9nVZvh3qv9f7WDIbpY1cysBLIBeoD9wN/DCG0AAghNAIm7y5vAMwFJlZFRxVbIfAg8L/7V3h/a4RfAduBxsB1wP+EENpVbZcU0wFfq75Oa4SDvtd6f2sOH0BUQgjhQ+CBKIqeCyHcCtwYRdH5u+vqsOuTmtpHUfRJVfZThyeE8CDQPIqiG/cq8/5WY7vv13rg36IoWry77GmgIIqi4VXaOcW2/2vV12nNtOe9FmiI97dGcGRaAIQQGgOnAgt3F7UDFuypj6JoE7B0d7mqP+9v9XYqsHNPkN5tAd6/msbXaQ2z33ut97eGMEyLEEJt4PfA7/b6a7guULLfriVAWmX2TUeM97d68/4dG7zPNcgB3mu9vzWEYbqGCiHMDCFEB/mavdd+tYCn2TX3csheTZQC9fZrth6w8Yh3Xof0r97fL+H9rd68f8cG73MNcZD3Wu9vDWGYrqGiKOoaRVE4yFdngBBCAH7LrgeY+kZRtGOvJhYCZ+7Z2D2XqxX/nAaiKvSv3N9D8P5Wb4uB5BDCKXuVnYn3r6bxdVoDfMl7rfe3hjBMH9v+BzgNuDyKoi371T0P/FsIoW8I4TjgB8CHPhRRfYQQknffuyQgKYRwXAgheXe197ca2z23cjLwwxBCnRBCJ+Bb7Br5UjXzJa9VX6c1w8Hea72/NYRh+hi1ey3L24CzgE9DCKW7v64DiKJoDdAXeIhdqwZ0BPpXVX8Vy0hgCzAcGLD7+5Hg/a0h/h1IBT4H/gB8N4oiR7SqpwO+Vn2dVn9f9l7r/a05XBpPkiRJismRaUmSJCkmw7QkSZIUk2FakiRJiskwLUmSJMVkmJYkSZJiMkxLkiRJMRmmJUmSpJgM05IkSVJMhmlJkiQppv8PknoilZ+8Gd8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAswAAAFoCAYAAABDtK1/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de3gV1b3/8fcioRAgECCRS4hBQFFpq9ggKmjwioBaBLkKFuyxtS1qrbVHBVu03pWe/rQ99UIL3kUsYkEFay0RFLUoiEcrFpQgiSghJBAMhJD5/QHsAsIoNCQE36/nyfPsWWv2zHf2uM2HyZo1IYoiJEmSJO1evdouQJIkSTqQGZglSZKkGAZmSZIkKYaBWZIkSYphYJYkSZJiGJglSZKkGMm1XUCc9PT0qH379rVdhiRJkg5yb775ZlEURRm76zugA3P79u1ZsGBBbZchSZKkg1wIIX9PfQ7JkCRJkmIYmCVJkqQYBmZJkiQphoFZkg5Co0aNYty4cbVdhiQdFAzMkiRJUgwDsyRJkhTDwCxJB4GFCxdy3HHHkZqaypAhQ9i4cWOi74EHHqBTp060aNGC8847j8LCwkTfCy+8QOfOnWnWrBk//vGPyc3NZeLEibVxCJJ0wDIwS1IdV1FRQf/+/Rk5ciTFxcUMGjSIP//5zwC89NJLXHvttTz55JN88sknZGdnM3ToUACKioq44IILuPXWW1mzZg2dO3fm1Vdfrc1DkaQDkoFZkuq41157jc2bN/PTn/6U+vXrc8EFF9CtWzcAHn30US6++GKOO+44GjRowK233sr8+fNZvnw5zz33HF26dGHAgAEkJydz+eWX07p161o+Gkk68BiYJamOmr6wgB63vcTAu2ayrl5Tnln076EW2dnZABQWFiZeAzRp0oSWLVtSUFBAYWEhWVlZib4QAu3atau5A5CkOsLALEl10PSFBVw77R0KSspJatKCjaWruebPi5m+sACAFStWANC2bVvy8//9tNcNGzawZs0aMjMzadOmDStXrkz0RVG007IkaSsDsyTVQXfOXkL55i0ANMg8EkISn732NHc89x7Tpk3jjTfeAGD48OFMmjSJRYsWsWnTJq677jq6d+9O+/bt6devH++88w7Tp0+nsrKS3//+96xatao2D0uSDkgGZkmqgwpLyhOvQ1J9Ms6/jrJ3/sZrN36XKVOmMGDAAABOP/10fv3rXzNw4EDatGnDsmXLeOKJJwBIT09n6tSp/OIXv6Bly5a899575OTk0KBBg1o5Jkk6UIUoimq7hj3KycmJFixYUNtlSNIBp8dtL1GwQ2jeLjMthVeuOW2ftllVVUW7du149NFHOfXUU//TEiWpTgkhvBlFUc7u+rzCLEl10NW9O5NSP2mntpT6SVzdu/NebWf27NmUlJSwadMmbrnlFqIo4oQTTqjOUiWpzkuu7QIkSXuvf9dMYOtY5sKSctqmpXB1786J9q9q/vz5DB8+nIqKCo4++mimT59OSkrK/ihZkuosh2RIkiTpa88hGZIkSdI+MjBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDs/Q1UVlZWdslSJJUJxmYpTrurbfeomvXrqSmpjJo0CCGDBnCuHHjmDNnDu3ateP222+ndevWjB49mqqqKm677TY6duxIy5YtGTx4MMXFxYltvfbaa5x00kmkpaVxzDHHMGfOnERfr169uP766+nRowepqamcddZZFBUV1cIRS5JUswzMUh1WUVHB+eefz6hRoyguLmbYsGE8/fTTif5Vq1ZRXFxMfn4+999/P3fffTfTp08nLy+PwsJCmjdvzk9+8hMACgoK6NevH+PGjaO4uJi77rqLgQMHsnr16sT2HnvsMSZNmsRnn31GRUUFd911V40fsyRJNc3ALNVhr732GpWVlVx++eXUr1+fAQMGcPzxxyf669Wrxw033ECDBg1ISUnhvvvu4+abb6Zdu3Y0aNCA8ePH89RTT1FZWckjjzxC37596du3L/Xq1ePMM88kJyeH5557LrG90aNHc8QRR5CSksLgwYNZtGhRbRy2JEk1Krm2C5C07woLC8nMzCSEkGjLyspKvM7IyKBhw4aJ5fz8fM4//3zq1fv3v5WTkpL49NNPyc/PZ+rUqcyYMSPRt3nzZk499dTEcuvWrROvGzVqRFlZWbUfkyRJBxoDs1THTF9YwJ2zl1BYUk7j4k/4ZPkKoihKhOaPP/6Yjh07AuwUpGFrmP7Tn/5Ejx49vrDdrKwsRo4cyQMPPLD/D0KSpDrEIRlSHTJ9YQHXTnuHgpJyImB9s8NYt6mKH1xzE5WVlTzzzDO88cYbe3z/pZdeytixY8nPzwdg9erVPPPMMwCMGDGCGTNmMHv2bLZs2cLGjRuZM2cOK1eurIlDkyTpgGVgluqQO2cvoXzzlsRySKpPev/reOzhB0lLS+ORRx7hnHPOoUGDBrt9/xVXXMF5553HWWedRWpqKieccAKvv/46sPUK8zPPPMMtt9xCRkYGWVlZ3HnnnVRVVdXIsUmSdKAKURTVdg17lJOTEy1YsKC2y5AOGIdd8yy7+8YG4KPb+gHQvXt3Lr30UkaPHl2jtUmSVJeFEN6Moihnd31eYZbqkLZpKV9o27jiHdKTyqmsrOTBBx9k8eLFnH322bVQnSRJB6dqC8whhAYhhD+GEPJDCOtDCAtDCH126D89hPB+COHzEMLfQwjZ1bVv6evi6t6dSamftFNbKC3kX/f+iGbNmjFhwgSeeuop2rRpU0sVSpJ08KnOWTKSgY+BXGAF0Bd4MoTwLaAMmAb8FzAD+DUwBTihGvcvHfT6d80ESMyS0TYthatv+m/6d727liuTJOngtV/HMIcQFgM3AC2BUVEUnbStvTFQBHSNouj9Pb3fMcySJEmqCbUyhjmE0Ao4AngX6AK8vb0viqINwLJt7ZIkSdIBa78E5hBCfeBR4MFtV5CbAKW7rFYKpO7mvT8IISwIISxYvXr1/ihPkiRJ+sqqPTCHEOoBDwMVwJhtzWVA011WbQqs3/X9URTdH0VRThRFORkZGdVdniRJkrRXqjUwh63P4f0j0AoYGEXR5m1d7wLH7LBeY6DjtnZJkiTpgFXdV5j/ABwFnBtFUfkO7U8D3wwhDAwhNAR+CSyOu+FPkiTFmzRpEueee25iuVOnTgwePDixnJWVxaJFi3j11Vfp1q0bzZo1o1u3brz66quJdXr16sW4ceM46aSTaNKkCeeeey5r1qzhwgsvpGnTpnTr1o3ly5cn1r/iiivIysqiadOmfOc732Hu3LmJvvHjxzN48GAuuugiUlNT6dKlC968r4NBdc7DnA38EDgWWBVCKNv2c2EURauBgcDNwFqgOzC0uvYtSdLXUW5uLnPnzqWqqopPPvmEzZs388orrwDw4YcfUlZWxqGHHkq/fv24/PLLWbNmDT/72c/o168fa9asSWzniSee4OGHH6agoIBly5Zx4oknMnr0aIqLiznqqKO44YYbEut269aNRYsWUVxczPDhwxk0aBAbN25M9P/lL39h6NChlJSUcN555zFmzBikuq7aAnMURflRFIUoihpGUdRkh59Ht/W/GEXRkVEUpURR1CuKouXVtW9Jkr6OOnToQGpqKosWLSIvL4/evXuTmZnJ+++/T15eHieffDLPPvsshx9+OCNHjiQ5OZlhw4Zx5JFHMmPGjMR2Ro8eTceOHWnWrBl9+vShY8eOnHHGGSQnJzNo0CAWLlyYWHfEiBG0bNmS5ORkrrrqKjZt2sSSJUsS/T179qRv374kJSUxcuRI3n77baS6rjofXCJJkmrA9IUFiQcYbUzvzD2PPkNK+Wpyc3NJS0sjLy+P+fPnk5ubS2FhIdnZOz9cNzs7m4KCgsRyq1atEq9TUlK+sFxWVpZYnjBhAhMnTqSwsJAQAuvWraOoqCjR37p168TrRo0asXHjRiorK0lONnKo7tpv8zBLkqTqN31hAddOe4eCknIioKrVUTw1YzbP/vUlcnNzyc3NJS8vj7y8PHJzc2nbti35+fk7bWPFihVkZmbu9b7nzp3L7bffzpNPPsnatWspKSmhWbNm7M+HoEkHAgOzJEl1yJ2zl1C+eUtiueGh32JD/mI+K15Pu3btOPnkk5k1axZr1qyha9eu9O3blw8++IDHHnuMyspKpkyZwnvvvcc555yz1/tev349ycnJZGRkUFlZyY033si6deuq8/CkA5KBWZKkOqSwpHyn5fotMqlXvyHJbY8GoGnTpnTo0IEePXqQlJREy5YtmTlzJhMmTKBly5bccccdzJw5k/T09L3ed+/evenTpw9HHHEE2dnZNGzYkKysrGo5LulAFg7kP6Pk5ORETkcjSdK/9bjtJQp2Cc0AmWkpvHLNabVQkXRwCCG8GUVRzu76vMIsSVIdcnXvzqTUT9qpLaV+Elf37lxLFUkHP29ZlSSpDunfdevNettnyWiblsLVvTsn2iVVPwOzJEl1TP+umQZkqQY5JEOSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBjVGphDCGNCCAtCCJtCCJN36Ts9hPB+COHzEMLfQwjZ1blvSZIkaX+o7ivMhcBNwJ92bAwhpAPTgOuBFsACYEo171uSJEmqdsnVubEoiqYBhBBygHY7dA0A3o2iaOq2/vFAUQjhyCiK3q/OGiRJkqTqVFNjmLsAb29fiKJoA7BsW7skSZJ0wKqpwNwEKN2lrRRI3XXFEMIPto2DXrB69eoaKU6SJEnak5oKzGVA013amgLrd10xiqL7oyjKiaIoJyMjo0aKkyRJkvakpgLzu8Ax2xdCCI2BjtvaJUmSpANWdU8rlxxCaAgkAUkhhIYhhGTgaeCbIYSB2/p/CSz2hj9JkiQd6Kr7CvM4oBy4Bhix7fW4KIpWAwOBm4G1QHdgaDXvW5IkSap21T2t3Hhg/B76XgSOrM79SZIkSfubj8aWJEmSYhiY9ZW1b9+eF198sdq2N3nyZHr27Fkt2xo1ahTjxo3bp/eOHz+eESNGALB8+XJCCFRWVlZLXZIkqe4zMKvOqc6gLUmS9GUMzJIkSVIMA7P2yj/+8Q+OPvpomjdvzujRo9m4cSNr167lnHPOISMjg+bNm3POOeewcuXKxHsmT55Mhw4dSE1N5bDDDuPRRx/d7bavvvpqevbsSWlpKaWlpXz/+9+nTZs2ZGZmMm7cOLZs2cI///lPLr30UubPn0+TJk1IS0tLvL+oqIgzzzyT1NRUcnNzyc/PT/RdccUVZGVl0bRpU77zne8wd+7c/fchSZKkg4qBWXvl0UcfZfbs2SxbtowPPviAm266iaqqKkaPHk1+fj4rVqwgJSWFMWPGALBhwwYuv/xynn/+edavX8+rr77Kscceu9M2q6qquOSSS1i8eDEvvPACzZo143vf+x7JycksXbqUhQsX8sILLzBx4kSOOuoo7r33Xk488UTKysooKSnZqbbrr7+eoqIijj32WC688MJEX7du3Vi0aBHFxcUMHz6cQYMGsXHjxpr50CRJUp1mYNZeGTNmDFlZWbRo0YKxY8fy+OOP07JlSwYOHEijRo1ITU1l7Nix5OXlJd5Tr149/u///o/y8nLatGlDly5dEn2bN29m2LBhFBcXM2PGDBo1asSnn37K888/z29/+1saN27MIYccwpVXXskTTzwRW1u/fv045ZRTaNCgATfffDPz58/n448/BmDEiBG0bNmS5ORkrrrqKjZt2sSSJUv2z4ckSdJXsGTJErp27Upqaip33313tW33P7nXZ86cObRr1y6xXN03/NdV1ToPsw5+WVlZidfZ2dkUFhby+eefc+WVVzJr1izWrl0LwPr169myZQuNGzdmypQp3HXXXXz/+9+nR48eTJgwgSOP3Dol99KlS3n77bd54403+MY3vgFAfn4+mzdvpk2bNol9VVVV7bTvL6utSZMmtGjRgsLCQrKyspgwYQITJ06ksLCQEALr1q2jqKio2j4XSZL21h133EGvXr1YuHDhPm9j+fLlHHbYYWzevJnkZGPd/uIVZsWavrCAHre9xGHXPMuq0o3MePWdRN+KFSto27YtEyZMYMmSJbz++uusW7eOl19+GYAoigDo3bs3f/3rX/nkk0848sgjueSSSxLbOOqoo5g0aRJ9+vRJXPHNysqiQYMGFBUVUVJSQklJCevWrePdd98FIISw21q3X00GKCsro7i4mLZt2zJ37lxuv/12nnzySdauXUtJSQnNmjVL1CdJUm3Iz8/f6a+uOnAZmLVH0xcWcO20dygoKScCKqsiJj9wHxNn/YPi4mJuueUWhgwZwvr160lJSSEtLY3i4mJuuOGGxDY+/fRT/vKXv7BhwwYaNGhAkyZNSEpK2mk/w4YN45ZbbuGMM85g2bJltGnThrPOOourrrqKdevWUVVVxbJlyxLDPFq1asXKlSupqKjYaTvPPfcc8+bNo6Kiguuvv57u3buTlZXF+vXrSU5OJiMjg8rKSm688UbWrVu33z8/SZL25LTTTuPvf/87Y8aMoUmTJrz99ttcdNFFZGRkkJ2dnbhHCLb+lfWmm24iOzubQw45hIsuuojS0lIATjnlFADS0tJo0qQJ8+fPB7ZetLrsssto1qwZRx55JH/7298S+540aRJHHXUUqampdOjQgfvuu6+Gj77uMTBrj+6cvYTyzVt2aks56hQu/94FdOjQgQ4dOjBu3Dh++tOfUl5eTnp6OieccAJnn312Yv2qqiomTJhA27ZtadGiBXl5efzv//7vF/b1ve99j1/+8pecdtppLF++nIceeoiKiorEjBwXXHABn3zyCbD1fzJdunShdevWpKenJ7YxfPhwbrjhBlq0aMGbb76ZmI2jd+/e9OnThyOOOILs7GwaNmz4pcM7JEnan1566SVOPvlkfve731FWVsaECRMoLS3lww8/JC8vj4ceeohJkyYBW8ckT548mb///e98+OGHlJWVJW6u3/5X3ZKSEsrKyjjxxBMBeP311+nQoQNFRUXccMMNDBgwgOLiYgAOOeQQZs6cybp165g0aRJXXnklb731Vi18CnVHOJD/LJ2TkxMtWLCgtsv42jrsmmfZ3X8dAfjotn41XY4kSQeVXr16MWLECEaPHk2jRo1YuHAhRx99NAD33Xcfjz/+OHPmzOH0009n4MCB/PjHPwa23iz4zW9+k/LyclauXPmFMcyTJ0/muuuuo6CgIDGM8fjjj+eyyy5j5MiRX6ijf//+nHrqqVxxxRXMmTOHESNGJKaHbd++PRMnTuSMM86oiY+kVoUQ3oyiKGd3fV5h1h61TUvZq3ZJkrT3ioqKqKioIDs7O9GWnZ1NQUEBAIWFhV/oq6ys5NNPP93jNjMzM3e652f7jfoAzz//PCeccAItWrQgLS2N5557zhvhv4SBWXt0de/OpNTfebxxSv0kru7duZYqkiSpbtvxZvqFK0pYuGIt6enp1K9ff6cHbq1YsYLMzEwA2rZt+4W+5ORkWrVqtccb4QsKCna6uX37jfqbNm1i4MCB/PznP+fTTz+lpKSEvn37eiP8lzAwa4/6d83k1gHfIjMthQBkpqVw64Bv0b9rZm2XJklSnbPrzfSbKrcw7a0CZixexeDBgxk7dizr168nPz+f3/zmN4wYMQLYenP8//zP//DRRx9RVlbGddddx5AhQxI3tNerV48PP/xwp3199tln3H333WzevJmpU6fyz3/+k759+1JRUcGmTZvIyMggOTmZ559/nhdeeKEWPo26xQn7FKt/10wDsiRJ1WB3N9NXbKniztlLmHnPPVx22WV06NCBhg0bcskll3DxxRcDcPHFF1NYWMgpp5zCxo0b6d27N/fccw8AjRo1YuzYsfTo0YPNmzcza9YsALp3786//vUv0tPTadWqFU899RQtW7YE4O6772bw4MFs2rSJc889l/POO68GP4W6yZv+JEmSaoA30x/YvOlPkiSplnkzfd1lYJYkSaoB3kxfdzmGWZIkqQZsvyfoztlLKCwpp21aClf37uy9QnWAgVmSJKmGeDN93eSQDEmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKYWCWJEmSYhiYJUmSpBgGZkmSJCmGgVmSJEmKUaOBOYTQIoTwdAhhQwghP4QwvCb3L0mSJO2t5Bre3++BCqAVcCzwbAjh7SiK3q3hOiRJkqSvpMauMIcQGgMDgeujKCqLomge8BdgZE3VIEmSJO2tmhyScQSwJYqiD3ZoexvosuNKIYQfhBAWhBAWrF69ugbLkyRJkr6oJgNzE6B0l7ZSIHXHhiiK7o+iKCeKopyMjIwaK06SJEnanZoMzGVA013amgLra7AGSZIkaa/UZGD+AEgOIRy+Q9sxgDf8SZIk6YBVY4E5iqINwDTgxhBC4xBCD+C7wMM1VYMkSZK0t2r6wSU/BlKAz4DHgR85pZwkSZIOZDU6D3MURcVA/5rcpyRJkvSf8NHYkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDDXgsmTJ9OzZ8/EcgiBpUuX1mJFkiRJ2hMDsyRJkhTDwCxJkiTFMDD/B+68804GDhy4U9tll13GT3/6U0pLS/n+979PmzZtyMzMZNy4cWzZsuVLt1laWspFF11ERkYG2dnZ3HTTTVRVVQGQnZ3Nm2++CcAjjzxCCIH33nsPgIkTJ9K/f/9qPkJJkiQZmP8DI0aMYNasWZSUlABQWVnJlClTGDlyJN/73vdITk5m6dKlLFy4kBdeeIGJEyd+6TYvu+wySktL+fDDD8nLy+Ohhx5i0qRJAOTm5jJnzhwAXn75ZTp06EBeXl5iOTc3d/8cqCRJ0teYgfk/0KZNG0455RSmTp0KwKxZs0hPT6ddu3Y8//zz/Pa3v6Vx48YccsghXHnllTzxxBOx29uyZQtTpkzh1ltvJTU1lfbt23PVVVfx8MMPA1sD8/aAPHfuXK699trEcl5enoFZkiRpPzAw74PpCwvocdtLHHbNs3yY9h1+e+8fga3DJEaOHEl+fj6bN2+mTZs2pKWlkZaWxg9/+EM+++yz2O0WFRVRUVFBdnZ2oi07O5uCggJga2CeO3cuq1atYsuWLQwZMoRXXnmF5cuXU1payrHHHrv/DlqSJOlrKrm2C6hrpi8s4Npp71C+eet45I2Z32Hl9P/H/5v6N2bOnMkdd9xB/fr1adCgAUVFRSQnf/WPOD09nfr165Ofn8/RRx8NwIoVK8jMzASgU6dONGrUiLvvvptTTjmF1NRUWrduzf3330/Pnj2pV89//0iSJFU3E9ZeunP2kkRYBgjJ36DRESdx3eWXcPzxx3PooYfSpk0bzjrrLK666irWrVtHVVUVy5YtSwyf2JOkpCQGDx7M2LFjWb9+Pfn5+fzmN79hxIgRiXVyc3P53e9+lxh+0atXr52WJUmSVL0MzHupsKT8C22Nv3U6n6/6iJEjRybaHnroISoqKjj66KNp3rw5F1xwAZ988smXbv+ee+6hcePGdOjQgZ49ezJ8+HAuvvjiRH9ubi7r16/nlFNO2e2yJElSXde+fXtefPHFL7R36dIlMQFCTQpRFNX4Tr+qnJycaMGCBbVdxk563PYSBbuE5sp1n/HJxB+xtugzmjZtWkuVSZIkHRzat2/PxIkTOeOMM2psnyGEN6Moytldn1eY99LVvTuTUj8psRxFVXz+5jP06tPfsCxJknQQMjDvpf5dM7l1wLfITEshqtjIyt8OpknRP5n8uztruzRJkqSDzvvvv89hhx3GE088sdNQjfHjxzN48GAuuugiUlNT6dKlCzuOTHjrrbfo2rUrqampDBo0iCFDhjBu3Lh9qsHAvA/6d83klWtOI/83A9myqZwVy5aQlZVV22VJkiQdVN566y3OOuss7rnnHoYOHfqF/r/85S8MHTqUkpISzjvvPMaMGQNARUUF559/PqNGjaK4uJhhw4bx9NNP73MdBmZJkiQdcObOnct5553Hgw8+yDnnnLPbdXr27Enfvn1JSkpi5MiRvP322wC89tprVFZWcvnll1O/fn0GDBjA8ccfv8+1GJglSZJ0wLn33ns56aSTOPXUU/e4TuvWrROvGzVqxIc0EO8AABDcSURBVMaNG6msrKSwsJDMzExCCIn+/2Q0gIFZkiRJtW7HJymvKt3I6F/czIoVK7jyyiv3eltt2rShoKCAHWeD+/jjj/e5NgOzJEmSatX2JykXlJQTAZVVEY+9tZrL7vgTL7/8Mtdcc81ebe/EE08kKSmJ3/3ud1RWVvLMM8/wxhtv7HN9PhpbkiRJtWrXJykDbNpSxf++uoq//vWvnHrqqdSvX/8rb+8b3/gG06ZN47/+67+49tpr6dOnD+eccw4NGjTYp/p8cIkkSZJq1WHXPMvuEmkAPrqtX7Xso3v37lx66aWMHj16t/0+uESSJEkHrLZpKXvV/lXk5eWxatUqKisrefDBB1m8eDFnn332Pm3LwCxJkqRateuTlAFS6idxde/O+7zNJUuWcMwxx9CsWTMmTJjAU089RZs2bfZpWw7JkCRJUq2bvrCAO2cvobCknLZpKVzduzP9u2bW2P7jhmR4058kSZJqXf+umTUakPeGQzIkSZKkGAZmSZIkKYaBWZIkSYphYJYkSZJiGJglSZKkGAZmSZIkKYaBWZIkSYphYJYkSZJiVEtgDiGMCSEsCCFsCiFM3k3/6SGE90MIn4cQ/h5CyK6O/UqSJEn7W3VdYS4EbgL+tGtHCCEdmAZcD7QAFgBTqmm/kiRJ0n5VLY/GjqJoGkAIIQdot0v3AODdKIqmbltnPFAUQjgyiqL3q2P/kiRJ0v5SE2OYuwBvb1+IomgDsGxb+xeEEH6wbXjHgtWrV9dAeZIkSdKe1URgbgKU7tJWCqTubuUoiu6PoigniqKcjIyM/V6cJEmSFOdLA3MIYU4IIdrDz7yvsI8yoOkubU2B9ftSsCRJklSTvnQMcxRFvf7DfbwLfG/7QgihMdBxW7skSZJ0QKuuaeWSQwgNgSQgKYTQMISwPYw/DXwzhDBw2zq/BBZ7w58kSZLqguoawzwOKAeuAUZsez0OIIqi1cBA4GZgLdAdGFpN+5UkSZL2q+qaVm48MD6m/0XgyOrYlyRJklSTfDS2JEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJEmqk8aPH8+IESP2+34MzJIkSVIMA7MkSZIOCJWVlbVdwm4ZmCVJklRr2rdvz+233863v/1tGjduzLx58zjppJNIS0vjmGOOYc6cOYl1P/roI3Jzc0lNTeXMM8+kqKioRmpMrpG9SJIkSXvw+OOP8+yzz1KvXj2+/e1v8/DDD3P22Wfzt7/9jYEDB/L++++TkZHB8OHDOfHEE3nhhRd4/fXX6devH9/97nf3e31eYZYkSVKtuvzyy8nKyuKRRx6hb9++9O3bl3r16nHmmWeSk5PDc889x4oVK/jHP/7Br3/9axo0aMApp5zCueeeWyP1GZglSZK0R7169WLixImJ5XHjxpGenk7r1q33eZvTFxbQ47aXOOyaZ1lVupEVm1IAyM/PZ+rUqaSlpSV+5s2bxyeffEJhYSHNmzencePGie1kZ2fv+4HtBYdkSJIk6Sv5+OOPmTBhAvn5+RxyyCH7tI3pCwu4dto7lG/eAkBlVcSf5i3n2BMKyMrKYuTIkTzwwANfeF9+fj5r165lw4YNidC8YsUKQgj7fkBfkVeYJUmS9JXk5+fTsmXLfQ7LAHfOXpIIy9tt2lLFnbOXMGLECGbMmMHs2bPZsmULGzduZM6cOaxcuZLs7GxycnL41a9+RUVFBfPmzWPGjBn/6SF9JQZmSZKkg9SkSZN2GufbqVMnBg8enFjOyspi0aJFvPrqq3Tr1o1mzZrRrVs3Xn311S9s68UXX+TMM8+ksLCQJk2aMGrUqH2qqbCkfI/tWVlZPPPMM9xyyy1kZGSQlZXFnXfeSVVVFQCPPfYYr7/+Oi1atOCGG27goosu2qca9pZDMiRJkg5Subm5XHnllVRVVfHpp5+yefNmXnnlFQA+/PBDysrKOPTQQ+nYsSN33303w4YNY+rUqfTr14+lS5fSsmXLxLbOOOMMnn/+eUaMGMHKlSv3uaa2aSkU7BCa2/3oT4l2gO7du5OXl7fb93bo0IG5c+fu8773lVeYJUmSDlIdOnQgNTWVRYsWkZeXR+/evcnMzOT9998nLy+Pk08+mWeffZbDDz+ckSNHkpyczLBhwzjyyCP323CHq3t3JqV+0k5tKfWTuLp35/2yv+rgFWZJkqSDWG5uLnPmzGHp0qXk5uaSlpZGXl4e8+fPJzc3l8LCwi/MNpGdnU1BQcF+qad/10xg61jmwpJy2qalcHXvzon2A5GBWZIk6SAzfWFBIpAmfd6KD56ZRXnxJ1x33XWkpaXx6KOPMn/+fMaMGcM///lP8vPzd3r/ihUrOPvss/dbff27Zh7QAXlXDsn4EgsXLuS4444jNTWVIUOGMHToUMaNG8fkyZPp2bPnTuuGEFi6dCkAmzZt4uc//zmHHnoorVq14tJLL6W8/N/jdWbOnMmxxx5LWloaJ510EosXL070tW/fnrvuuotvf/vbNGvWjCFDhrBx48aaOWBJklSnbZ+2raCknAgoT+/Mm6/No6hkPe3atePkk09m1qxZrFmzhq5du9K3b18++OADHnvsMSorK5kyZQrvvfce55xzTm0fygHDwByjoqKC/v37M3LkSIqLixk0aBB//vOfv9J7//u//5sPPviARYsWsXTpUgoKCrjxxhsBeOutt7j44ou57777WLNmDT/84Q8577zz2LRpU+L9Tz75JLNmzeKjjz5i8eLFTJ48eX8coiRJOsjsOm1b/RaZhPoN2dLqSACaNm1Khw4d6NGjB0lJSbRs2ZKZM2cyYcIEWrZsyR133MHMmTNJT0+vrUM44IQoimq7hj3KycmJFixYUGv7f/nllxk6dCgFBQWJSbFPOukkTjvtNDp16sTEiROZN29eYv0QAv/617/o2LEjTZo0YfHixXTs2BGA+fPnM3z4cD766CN+9KMfkZ6ezq9//evEezt37sz9999Pbm4u7du356abbmLEiBEA/OIXv2DdunXce++9NXj0kiSpLjrsmmfZXboLwEe39avpcuqMEMKbURTl7K7PMcy72HHMzzfy59O4+SE7PUHmqzyCcfXq1Xz++ed85zvfSbRFUcSWLVv/tZefn8+DDz7IPffck+ivqKigsLAwsbzj4yYbNWq0U58kSdKe7Dpt247t2jcOydjBrmN+SkMTlq/4mKff+vdcgytWrACgcePGfP7554n2VatWJV6np6eTkpLCu+++S0lJCSUlJZSWllJWVgZsnSR87Nixib6SkhI+//xzhg0bVjMHKkmSDlp1cdq2A52BeQe7jvlpkHkkUUjiZ7+6lcrKSqZNm8Ybb7wBwDHHHMO7777LokWL2LhxI+PHj0+8r169elxyySVceeWVfPbZZwAUFBQwe/ZsAC655BLuvfdeXn/9daIoYsOGDTz77LOsX7++5g5WkiQdlPp3zeTWAd8iMy2FAGSmpXDrgG/VqVkpDjQG5h3s+qjGkFSfjPOvo/CNWTRv3pwpU6YwYMAAAI444gh++ctfcsYZZ3D44Yd/YcaM22+/nU6dOnHCCSfQtGlTzjjjDJYsWQJATk4ODzzwAGPGjKF58+Z06tTJm/okSVK16d81k1euOY2PbuvHK9ecZlj+D3nT3w563PbSbsf8ZKal8Mo1pwEwatQo2rVrx0033VQt+2zfvj0/+clPePjhh1m2bBlDhw7llltuYdSoUcybN4/u3bszdepUmjdvzqBBg5g7dy7l5eUcc8wx/OEPf6BLly6Juho3bszy5ct5+eWXOfroo3nssccSNx1KkiRpz+Ju+vMK8w5qa8zPn//8Z/7617/ywQcfMGPGDPr06cMtt9xCUVERVVVV3H333QD06dOHf/3rX3z22Wccd9xxXHjhhTtt5/HHH+dXv/oVa9eupVOnTowdO3a/1i1JkvR14CwZO6itRzVedtlltGrVCoCTTz6ZQw45hK5duwJw/vnn87e//Q2Aiy++OPGe8ePH07x5c0pLS2nWrBkAAwYM4Pjjjwfgwgsv5Gc/+9l+rVuSJOnrwMC8iy97VGN1jDXeceq6VaUbWbr+31e1U1JSEuF5+3JZWRlbtmxh7NixTJ06ldWrV1Ov3tY/DhQVFSUC865T0W2flUOSJEn7ziEZNWzXqesqqyL+OPcjpi8siH3fY489xjPPPMOLL75IaWkpy5cvB7bO7yxJkqT9x8Bcw3adug5g05Yq7py9JPZ969evp0GDBrRs2ZLPP/+c6667bn+WKUmSpG0MzDVs16nrvqx9u4suuojs7GwyMzM5+uijOeGEE/ZHeZIkSdqF08rVsK8ydZ0kSZJqltPKHUB8XKUkSVLd4iwZNay2pq6TJEnSvjEw14Ivm7pOkiRJBw6HZEiSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSDAOzJEmSFMPALEmSJMUwMEuSJEkxDMySJElSjBBFUW3XsEchhNVAfm3X8TWWDhTVdhGqdp7Xg5Pn9eDkeT04eV4PTNlRFGXsruOADsyqXSGEBVEU5dR2HapenteDk+f14OR5PTh5Xuseh2RIkiRJMQzMkiRJUgwDs+LcX9sFaL/wvB6cPK8HJ8/rwcnzWsc4hlmSJEmK4RVmSZIkKYaBWZIkSYphYNZOQggNQgh/DCHkhxDWhxAWhhD67LLO6SGE90MIn4cQ/h5CyK6tevXVhRDGhBAWhBA2hRAm76bf81oHhRBahBCeDiFs2Pa9HV7bNWnvxX0//W7WTV/2+9TzWrcYmLWrZOBjIBdoBlwPPBlCaA8QQkgHpm1rbwEsAKbURqHaa4XATcCfdu3wvNZpvwcqgFbAhcAfQghdarck7YPdfj/9btZpe/x96nmte7zpT18qhLAYuCGKoj+HEH4AjIqi6KRtfY3Z+rSirlEUvV+bdeqrCSHcBLSLomjUDm2e1zpo23laC3wziqIPtrU9DBREUXRNrRanfbLr99Pv5sFl++9ToCWe1zrFK8yKFUJoBRwBvLutqQvw9vb+KIo2AMu2tavu8rzWTUcAW7aH5W3exvN2MPG7eZDY5fep57WOMTBrj0II9YFHgQd3+BdvE6B0l1VLgdSarE3VzvNaN3neDn6e44PAbn6fel7rGAPz10wIYU4IIdrDz7wd1qsHPMzWsZFjdthEGdB0l802Bdbv9+K1R1/1vMbwvNZNnreDn+e4jtvD71PPax1jYP6aiaKoVxRFYQ8/PQFCCAH4I1tvIhoYRdHmHTbxLnDM9oVt46468u8hG6oFX+W8fgnPa930AZAcQjh8h7Zj8LwdTPxu1mExv089r3WMgVm78wfgKODcKIrKd+l7GvhmCGFgCKEh8EtgsTcpHPhCCMnbzlkSkBRCaBhCSN7W7Xmtg7aNe5wG3BhCaBxC6AF8l61Xs1SHxHw//W7WbXv6fep5rWMMzNrJtnkgfwgcC6wKIZRt+7kQIIqi1cBA4Ga23p3fHRhaW/Vqr4wDyoFrgBHbXo8Dz2sd92MgBfgMeBz4URRFXqWqe3b7/fS7WXfF/T71vNY9TisnSZIkxfAKsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTDwCxJkiTFMDBLkiRJMQzMkiRJUgwDsyRJkhTj/wOfuY8g173t4AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "W1 = 1 # Change me\n",
    "W2 = 10 # Change me\n",
    "embedding2 = embedding * W1 + pos_encoding[:,:,:] * W2\n",
    "plot_words(embedding2, sequences, 0)\n",
    "plot_words(embedding2, sequences, 1)\n",
    "\n",
    "# For reference\n",
    "#['king queen man woman dog wolf football basketball red green yellow',\n",
    "# 'man queen yellow basketball green dog  woman football  king red wolf']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you set `W1 = 1` and `W2 = 10`, you can see how arrangement of the words begins to take on a clockwise or anti-clockwise order depending on the position of the words in the sentence. Under these parameters, the positional encoding vectors have dominated the embedding. \n",
    "\n",
    "Now try inverting the weights to `W1 = 10` and `W2 = 1`. Observe that under these parameters, the plot resembles the original embedding visualizations and there are only a few changes between the positions of the plotted words.\n",
    "\n",
    "In the previous Transformer assignment, the word embedding is multiplied by `sqrt(EMBEDDING_DIM)`. In this case, it will be equivalent using `W1 = sqrt(EMBEDDING_DIM) = 10` and `W2 = 1`. \n",
    "\n",
    "### Congratulations!\n",
    " \n",
    "You've completed this notebook, and have a better sense of the inputs of the Transformer network! \n",
    "\n",
    "By now, you've: \n",
    "\n",
    "* Created positional encoding matrices to visualize the relational properties of the vectors\n",
    "* Plotted embeddings and positional encodings on a Cartesian plane to observe how they affect each other \n",
    "\n",
    "\n",
    "<font color='blue'><b>What you should remember</b>:\n",
    "- Positional encodings can be expressed as linear functions of each other, which allow the model to learn according to the relative positions of words.\n",
    "- Positional encodings can affect the word embeddings, but if the relative weight of the positional encoding is small, the sum will retain the semantic meaning of the words.  \n"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "Transformer Assignment - Subclass.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}