optimizing _language_modelling_nn.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Language Modeling From Scratch - Deep Dive Into Activations, Gradients and BatchNormalization\n",
    "\n",
    "In previous article we implemented a Multi Layer Perceptron with 2 Hidden Layers. We observed an improvement over our original model which made use of probability distribution to generate random names. In this article we'll take a deeper look into activations, gradients and batch normalization. This is a separate article in itself because it's important to develop a solid understanding of these topics before we mode onto more complex architectures like RNN and Transformers.\n",
    "\n",
    "Starting code for this article is same as previous article, just a cleaned out version of it. As usual, we'll start by importing all the required modules."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn.functional as F\n",
    "import matplotlib.pyplot as plt # for making figures\n",
    "import numpy as np\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we'll import our dataset of names and build vocabulary out of it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(32033,\n",
       " 27,\n",
       " {1: 'a',\n",
       "  2: 'b',\n",
       "  3: 'c',\n",
       "  4: 'd',\n",
       "  5: 'e',\n",
       "  6: 'f',\n",
       "  7: 'g',\n",
       "  8: 'h',\n",
       "  9: 'i',\n",
       "  10: 'j',\n",
       "  11: 'k',\n",
       "  12: 'l',\n",
       "  13: 'm',\n",
       "  14: 'n',\n",
       "  15: 'o',\n",
       "  16: 'p',\n",
       "  17: 'q',\n",
       "  18: 'r',\n",
       "  19: 's',\n",
       "  20: 't',\n",
       "  21: 'u',\n",
       "  22: 'v',\n",
       "  23: 'w',\n",
       "  24: 'x',\n",
       "  25: 'y',\n",
       "  26: 'z',\n",
       "  0: '.'},\n",
       " {'a': 1,\n",
       "  'b': 2,\n",
       "  'c': 3,\n",
       "  'd': 4,\n",
       "  'e': 5,\n",
       "  'f': 6,\n",
       "  'g': 7,\n",
       "  'h': 8,\n",
       "  'i': 9,\n",
       "  'j': 10,\n",
       "  'k': 11,\n",
       "  'l': 12,\n",
       "  'm': 13,\n",
       "  'n': 14,\n",
       "  'o': 15,\n",
       "  'p': 16,\n",
       "  'q': 17,\n",
       "  'r': 18,\n",
       "  's': 19,\n",
       "  't': 20,\n",
       "  'u': 21,\n",
       "  'v': 22,\n",
       "  'w': 23,\n",
       "  'x': 24,\n",
       "  'y': 25,\n",
       "  'z': 26,\n",
       "  '.': 0})"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# read in all the words\n",
    "words = open('names.txt', 'r').read().splitlines()\n",
    "# build the vocabulary of characters and mappings to/from integers\n",
    "chars = sorted(list(set(''.join(words))))\n",
    "stoi = {s:i+1 for i,s in enumerate(chars)}\n",
    "stoi['.'] = 0\n",
    "itos = {i:s for s,i in stoi.items()}\n",
    "vocab_size = len(itos)\n",
    "len(words), vocab_size, itos, stoi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding a utility method to create train, test and validation dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "block_size = 3 # context length: how many characters do we take to predict the next one?\n",
    "def build_dataset(words):  \n",
    "  X, Y = [], []\n",
    "  \n",
    "  for w in words:\n",
    "    context = [0] * block_size\n",
    "    for ch in w + '.':\n",
    "      ix = stoi[ch]\n",
    "      X.append(context)\n",
    "      Y.append(ix)\n",
    "      context = context[1:] + [ix] # crop and append\n",
    "\n",
    "  X = torch.tensor(X)\n",
    "  Y = torch.tensor(Y)\n",
    "  print(X.shape, Y.shape)\n",
    "  return X, Y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Creating training, testing and validation dataset using the above method. We'll use 80%, 10% and 10% respectively"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([182625, 3]) torch.Size([182625])\n",
      "torch.Size([22655, 3]) torch.Size([22655])\n",
      "torch.Size([22866, 3]) torch.Size([22866])\n"
     ]
    }
   ],
   "source": [
    "import random\n",
    "random.seed(42)\n",
    "random.shuffle(words)\n",
    "n1 = int(0.8*len(words))\n",
    "n2 = int(0.9*len(words))\n",
    "\n",
    "Xtr,  Ytr  = build_dataset(words[:n1])     # 80%\n",
    "Xdev, Ydev = build_dataset(words[n1:n2])   # 10%\n",
    "Xte,  Yte  = build_dataset(words[n2:])     # 10%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similar to the neural network discussed in previous article, we'll create two hidden layers, HL1 and HL2\n",
    "\n",
    "![https://cs231n.github.io/assets/nn1/neural_net2.jpeg](https://cs231n.github.io/assets/nn1/neural_net2.jpeg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11697\n"
     ]
    }
   ],
   "source": [
    "n_embd = 10 # the dimensionality of the character embedding vectors\n",
    "n_hidden = 200 # the number of neurons in the hidden layer of the MLP\n",
    "\n",
    "g = torch.Generator().manual_seed(2147483647) # for reproducibility\n",
    "C  = torch.randn((vocab_size, n_embd),            generator=g)\n",
    "W1 = torch.randn((n_embd * block_size, n_hidden), generator=g) \n",
    "W2 = torch.randn((n_hidden, vocab_size),          generator=g)\n",
    "b2 = torch.randn(vocab_size,                      generator=g)\n",
    "\n",
    "parameters = [C, W1, W2, b2]\n",
    "print(sum(p.nelement() for p in parameters)) # number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we'll setup the training loop with forward pass, backward pass and parameter update using learning rate, same as last article. We'll also keep track of loss per step to later plot the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# same optimization as last time\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "def run_training_loop(break_on_first=False):\n",
    "\n",
    "  for i in range(max_steps):\n",
    "    \n",
    "    # minibatch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,), generator=g)\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # batch X,Y\n",
    "    \n",
    "    # forward pass\n",
    "    emb = C[Xb] # embed the characters into vectors\n",
    "    embcat = emb.view(emb.shape[0], -1) # concatenate the vectors\n",
    "    # Linear layer\n",
    "    hpreact = embcat @ W1 #+ b1 # hidden layer pre-activation\n",
    "    # Non-linearity\n",
    "    h = torch.tanh(hpreact) # hidden layer\n",
    "    logits = h @ W2 + b2 # output layer\n",
    "    loss = F.cross_entropy(logits, Yb) # loss function\n",
    "    \n",
    "    # backward pass\n",
    "    for p in parameters:\n",
    "      p.grad = None\n",
    "    loss.backward()\n",
    "    \n",
    "    # update\n",
    "    lr = 0.1 if i < 100000 else 0.01 # step learning rate decay\n",
    "    for p in parameters:\n",
    "      p.data += -lr * p.grad\n",
    "\n",
    "    # track stats\n",
    "    if i % 10000 == 0: # print every once in a while\n",
    "      print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    lossi.append(loss.log10().item())\n",
    "\n",
    "    if break_on_first:\n",
    "      return logits, h, hpreact\n",
    "      break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 29.8979\n",
      "  10000/ 200000: 2.6601\n",
      "  20000/ 200000: 2.7236\n",
      "  30000/ 200000: 2.2986\n",
      "  40000/ 200000: 2.5010\n",
      "  50000/ 200000: 2.0478\n",
      "  60000/ 200000: 2.4736\n",
      "  70000/ 200000: 2.4584\n",
      "  80000/ 200000: 2.4450\n",
      "  90000/ 200000: 2.1317\n",
      " 100000/ 200000: 2.3553\n",
      " 110000/ 200000: 2.4058\n",
      " 120000/ 200000: 1.6513\n",
      " 130000/ 200000: 1.9658\n",
      " 140000/ 200000: 2.1240\n",
      " 150000/ 200000: 2.0506\n",
      " 160000/ 200000: 2.0208\n",
      " 170000/ 200000: 2.3377\n",
      " 180000/ 200000: 2.1702\n",
      " 190000/ 200000: 2.0824\n"
     ]
    }
   ],
   "source": [
    "run_training_loop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Adding a method to calculate loss on train and validation split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [],
   "source": [
    "@torch.no_grad() # this decorator disables gradient tracking\n",
    "def split_loss(split):\n",
    "  x,y = {\n",
    "    'train': (Xtr, Ytr),\n",
    "    'val': (Xdev, Ydev),\n",
    "    'test': (Xte, Yte),\n",
    "  }[split]\n",
    "  emb = C[x] # (N, block_size, n_embd)\n",
    "  embcat = emb.view(emb.shape[0], -1) # concat into (N, block_size * n_embd)\n",
    "  hpreact = embcat @ W1 # + b1\n",
    "  h = torch.tanh(hpreact) # (N, n_hidden)\n",
    "  logits = h @ W2 + b2 # (N, vocab_size)\n",
    "  loss = F.cross_entropy(logits, y)\n",
    "  print(split, loss.item())\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "split_loss('train')\n",
    "split_loss('val')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x124cf4650>]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAGdCAYAAADJ6dNTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABQqUlEQVR4nO3deXxM5+IG8GeyTRKyIJsQYt8FQRpbqRCqqsttFbdU0VJul2hpaitqaava++tVWqV0sbVXl1tqC6ktthC72BKxZBFkJeu8vz8iYyazT2YyJ5nn+/nkQ86cc+Y9M8mcJ+8qE0IIEBEREUmEg60LQERERKSK4YSIiIgkheGEiIiIJIXhhIiIiCSF4YSIiIgkheGEiIiIJIXhhIiIiCSF4YSIiIgkxcnWBTCGQqHArVu34OHhAZlMZuviEBERkRGEEMjNzUVgYCAcHIyvD6kW4eTWrVsICgqydTGIiIjIDNevX0fDhg2N3r9ahBMPDw8AZRfn6elp49IQERGRMXJychAUFKS8jxurWoST8qYcT09PhhMiIqJqxtQuGewQS0RERJLCcEJERESSwnBCREREksJwQkRERJLCcEJERESSwnBCREREksJwQkRERJLCcEJERESSwnBCREREksJwQkRERJLCcEJERESSwnBCREREklItFv6zllX7k3D97n281D0IrQO4oCAREZEU2HXNyZZTt7DmYDJS7ty3dVGIiIjoIbsOJ0RERCQ9DCdEREQkKQwnAIStC0BERERKdh1OZDKZrYtAREREFdh1OCEiIiLpYTghIiIiSWE4ISIiIklhOAEg2COWiIhIMuw6nLA7LBERkfTYdTghIiIi6WE4ISIiIklhOAHAadiIiIikg+GEiIiIJMWuwwkniCUiIpIek8PJ3r17MXToUAQGBkImk+G3334z+tgDBw7AyckJnTp1MvVpiYiIyE6YHE7y8/MREhKCZcuWmXRcVlYWRo8ejf79+5v6lERERGRHnEw9YPDgwRg8eLDJTzRx4kSMHDkSjo6OJtW2VAVOwkZERCQdVdLn5LvvvsPVq1cxZ84co/YvLCxETk6O2pc1yDgNGxERkeRYPZxcunQJ77//Pn788Uc4ORlXUbNo0SJ4eXkpv4KCgqxcSiIiIpIKq4aT0tJSjBw5EnPnzkXLli2NPi46OhrZ2dnKr+vXr1uxlERERCQlJvc5MUVubi6OHTuGEydOYMqUKQAAhUIBIQScnJywY8cOPPHEExrHyeVyyOVyaxZNDbucEBERSYdVw4mnpydOnz6ttu2rr77C7t278csvv6BJkybWfHoiIiKqhkwOJ3l5ebh8+bLy+6SkJCQkJKBu3bpo1KgRoqOjcfPmTXz//fdwcHBA+/bt1Y738/ODq6urxnabYH9YIiIiyTE5nBw7dgz9+vVTfh8VFQUAGDNmDNasWYPU1FSkpKRYroRERERkV2RCSH+Wj5ycHHh5eSE7Oxuenp4WO++LX8fhSNJdLBvZBUM61rfYeYmIiMj8+7ddr61DRERE0mPX4YRdToiIiKTHrsMJERERSQ/DCREREUkKwwkAwWnYiIiIJIPhhIiIiCTFrsOJjD1iiYiIJMeuwwkRERFJD8MJAOlPQ0dERGQ/GE6IiIhIUuw6nMg4DRsREZHk2HU4ISIiIulhOCEiIiJJYTgBOAUbERGRhDCcEBERkaTYdTjhJGxERETSY9fhhIiIiKSH4QSA4CxsREREksFwQkRERJLCcEJERESSYtfhhB1iiYiIpMeuwwkRERFJD8MJERERSQrDCREREUmKXYcTrkpMREQkPXYdToiIiEh6GE4AcA42IiIi6WA4ISIiIklhOCEiIiJJsetwwknYiIiIpMeuw0k5AXY6ISIikgqGEyIiIpIUhhMiIiKSFIYTIiIikhSGEyIiIpIUhhNwEjYiIiIpYTghIiIiSWE4ISIiIkkxOZzs3bsXQ4cORWBgIGQyGX777Te9+2/evBkDBgyAr68vPD09ER4eju3bt5tbXouScRY2IiIiyTE5nOTn5yMkJATLli0zav+9e/diwIAB2Lp1K+Lj49GvXz8MHToUJ06cMLmw1sI+J0RERNLhZOoBgwcPxuDBg43e/4svvlD7fuHChfj999/xv//9D507dzb16YmIiKiGMzmcVJZCoUBubi7q1q2rc5/CwkIUFhYqv8/JyamKohEREZEEVHmH2CVLliAvLw8vvviizn0WLVoELy8v5VdQUJBVysIeJ0RERNJTpeFk3bp1mDt3LjZt2gQ/Pz+d+0VHRyM7O1v5df369SosJREREdlSlTXrbNiwAePHj8fPP/+MiIgIvfvK5XLI5fIqKhm4JjEREZGEVEnNyfr16zF27FisX78eQ4YMqYqnJCIiomrK5JqTvLw8XL58Wfl9UlISEhISULduXTRq1AjR0dG4efMmvv/+ewBlTTljxozBv//9b4SFhSEtLQ0A4ObmBi8vLwtdBhEREdUUJtecHDt2DJ07d1YOA46KikLnzp0xe/ZsAEBqaipSUlKU+3/zzTcoKSnB5MmTUb9+feXXW2+9ZaFLMB/nYCMiIpIek2tO+vbtC6Fn1rI1a9aofR8bG2vqU1Q5fddDREREVYtr6xAREZGkMJwQERGRpNh1OGGXEyIiIumx63BSjj1OiIiIpIPhhIiIiCSF4YSIiIgkheGEiIiIJMWuw4mMs7ARERFJjl2HEyX2iCUiIpIMhhMiIiKSFLsOJ7svZAAAtp9Ns3FJiIiIqJxdh5NyMQ9DChEREdkewwkRERFJCsMJERERSQrDCREREUkKwwkRERFJCsMJERERSQrDCREREUkKwwkRERFJCsMJERERSQrDCREREUkKwwkRERFJCsMJERERSQrDCREREUkKwwkRERFJCsMJERERSQrDCREREUkKwwkRERFJCsMJERERSQrDCREREUkKwwkRERFJCsMJERERSQrDCREREUkKwwkRERFJCsMJERERSQrDCREREUkKwwkRERFJisnhZO/evRg6dCgCAwMhk8nw22+/GTwmNjYWXbp0gVwuR/PmzbFmzRozikpERET2wORwkp+fj5CQECxbtsyo/ZOSkjBkyBD069cPCQkJePvttzF+/Hhs377d5MISERFRzedk6gGDBw/G4MGDjd5/xYoVaNKkCT777DMAQJs2bbB//358/vnniIyMNPXpiYiIqIazep+TuLg4REREqG2LjIxEXFyctZ+aiIiIqiGTa05MlZaWBn9/f7Vt/v7+yMnJwYMHD+Dm5qZxTGFhIQoLC5Xf5+TkWLuYREREJBGSHK2zaNEieHl5Kb+CgoJsXSQiIiKqIlYPJwEBAUhPT1fblp6eDk9PT621JgAQHR2N7Oxs5df169etXUwiIiKSCKs364SHh2Pr1q1q23bu3Inw8HCdx8jlcsjlcmsXjYiIiCTI5JqTvLw8JCQkICEhAUDZUOGEhASkpKQAKKv1GD16tHL/iRMn4urVq5g2bRouXLiAr776Cps2bcI777xjmSsgIiKiGsXkcHLs2DF07twZnTt3BgBERUWhc+fOmD17NgAgNTVVGVQAoEmTJtiyZQt27tyJkJAQfPbZZ/j22285jJiIiIi0kgkhhK0LYUhOTg68vLyQnZ0NT09Pi503+P0tyv8nLx5isfMSERGR+fdvSY7WISIiIvvFcPLQg6JSWxeBiIiIwHCi9NeZVFsXgYiIiMBwQkRERBLDcEJERESSwnBCREREksJw8pBMZusSEBEREcBwQkRERBLDcEJERESSwnDykAxs1yEiIpIChpOHiksVti4CERERgeFESfILDBEREdkJhhMiIiKSFIYTIiIikhSGk3Js1yEiIpIEhhMiIiKSFIYTIiIikhSGk4dyCoptXQQiIiICw4nSR1vO27oIREREBIYTIiIikhiGEyIiIpIUhhMiIiKSFIYTIiIikhSGEyIiIpIUhhMiIiKSFIYTIiIikhSGEyIiIpIUhhMiIiKSFIYTIiIikhSGEyIiIpIUhhMiIiKSFIYTIiIikhSGEyIiIpIUhhMiIiKSFIYTIiIikhSGExWX0nNtXQQiIiK7x3Ci4sDlTFsXgYiIyO4xnBAREZGkmBVOli1bhuDgYLi6uiIsLAxHjhzRu/8XX3yBVq1awc3NDUFBQXjnnXdQUFBgVoGJiIioZjM5nGzcuBFRUVGYM2cOjh8/jpCQEERGRiIjI0Pr/uvWrcP777+POXPm4Pz581i1ahU2btyIDz74oNKFJyIioprH5HCydOlSTJgwAWPHjkXbtm2xYsUKuLu7Y/Xq1Vr3P3jwIHr27ImRI0ciODgYAwcOxIgRIwzWttiCTCazdRGIiIjsnknhpKioCPHx8YiIiHh0AgcHREREIC4uTusxPXr0QHx8vDKMXL16FVu3bsWTTz6p83kKCwuRk5Oj9kVERET2wcmUnTMzM1FaWgp/f3+17f7+/rhw4YLWY0aOHInMzEz06tULQgiUlJRg4sSJept1Fi1ahLlz55pSNCIiIqohrD5aJzY2FgsXLsRXX32F48ePY/PmzdiyZQvmz5+v85jo6GhkZ2crv65fv27tYhIREZFEmFRz4uPjA0dHR6Snp6ttT09PR0BAgNZjZs2ahZdffhnjx48HAHTo0AH5+fl47bXXMGPGDDg4aOYjuVwOuVxuStEsgl1OiIiIbM+kmhMXFxeEhoYiJiZGuU2hUCAmJgbh4eFaj7l//75GAHF0dAQACCFMLa9VSaw4REREdsnkZp2oqCisXLkSa9euxfnz5zFp0iTk5+dj7NixAIDRo0cjOjpauf/QoUOxfPlybNiwAUlJSdi5cydmzZqFoUOHKkOKVPz3+A1bF4GIiMjumdSsAwDDhw/H7du3MXv2bKSlpaFTp07Ytm2bspNsSkqKWk3JzJkzIZPJMHPmTNy8eRO+vr4YOnQoFixYYLmrsJBTN7JtXQQiIiK7JxNSa1vRIicnB15eXsjOzoanp6fFzhv8/haNbcmLh1js/ERERPbM3Ps319YhIiIiSWE4ISIiIklhOCEiIiJJYTghIiIiSWE4ISIiIklhOKngaPJdWxeBiIjIrjGcVLB6f5Kti0BERGTXGE6IiIhIUhhOKuDif0RERLbFcEJERESSwnBSgQysOiEiIrIlhpOKmE2IiIhsiuGEiIiIJIXhpIItp1JtXQQiIiK7xnBCREREksJwQkRERJLCcEJERESSwnBCREREksJwQkRERJLCcEJERESSwnBCREREksJwosW3+65CCGHrYhAREdklhhMtPtpyHvsvZ9q6GERERHaJ4USHW1kPbF0EIiIiu8RwQkRERJJi1+FExhWIiYiIJMeuw0nXxnV0Pjb9v6dRXKqowtIQERERYOfhxJCY8xm2LgIREZHdsetwIoP+dh3WnBAREVU9uw4nPh4ueh//1/oTWHc4pYpKQ0RERICdh5N2gV4G9/ng19NVUBIiIiIqZ9fhxFJKFQJFJWwCIiIisgS7DieWGko8YOnfCP1oJwpLSi1zQiIiIjtm1+HEUq5m5iO3oARXMvJtXRQiIqJqj+HECk5ez8Kc388g+36xrYtCRERU7TjZugC2ZGgocbmjyXdRx90Zzf08jNp/2LIDAID8olIseSHE7PIRERHZI/sOJ0b2OXlhRRwAIHnxEJPOfzkjz9QiERER2T27btYxtT9scmY+bucW4kERO74SERFZi1nhZNmyZQgODoarqyvCwsJw5MgRvftnZWVh8uTJqF+/PuRyOVq2bImtW7eaVWBLMnW0Tt8lsei2YBdCP9pplfMTERGRGc06GzduRFRUFFasWIGwsDB88cUXiIyMRGJiIvz8/DT2LyoqwoABA+Dn54dffvkFDRo0wLVr1+Dt7W2J8tvEfdacEBERWY3J4WTp0qWYMGECxo4dCwBYsWIFtmzZgtWrV+P999/X2H/16tW4e/cuDh48CGdnZwBAcHBw5UpdTbDihIiIyHQmNesUFRUhPj4eERERj07g4ICIiAjExcVpPeaPP/5AeHg4Jk+eDH9/f7Rv3x4LFy5Eaanu2ofCwkLk5OSofVmDsaN1dBFCYOvpVAuVhoiIiAATw0lmZiZKS0vh7++vtt3f3x9paWlaj7l69Sp++eUXlJaWYuvWrZg1axY+++wzfPTRRzqfZ9GiRfDy8lJ+BQUFmVLMKrP3Uibe+Om4zsdl7HRCRERkMquP1lEoFPDz88M333yD0NBQDB8+HDNmzMCKFSt0HhMdHY3s7Gzl1/Xr161dTLOcup5l6yIQERHVOCb1OfHx8YGjoyPS09PVtqenpyMgIEDrMfXr14ezszMcHR2V29q0aYO0tDQUFRXBxcVF4xi5XA65XG5K0cxSS165aV5Ehe8VQn2LpepNhBCYsu4E3F0c8SkndSMiohrOpJoTFxcXhIaGIiYmRrlNoVAgJiYG4eHhWo/p2bMnLl++DIXi0aq9Fy9eRP369bUGk6r0fGgDi57vqS/3o1TxKKBYqlXnxr0H2HI6FT/H3+DigkREVOOZ3KwTFRWFlStXYu3atTh//jwmTZqE/Px85eid0aNHIzo6Wrn/pEmTcPfuXbz11lu4ePEitmzZgoULF2Ly5MmWuwozyZ0cDe9kopv3Hij/X9kOt+Uq1sgQERHVZCa3awwfPhy3b9/G7NmzkZaWhk6dOmHbtm3KTrIpKSlwcHiUeYKCgrB9+3a888476NixIxo0aIC33noL06dPt9xV2EDk53vRoaGXrYtRacWlCmTdL4avh/Wb0YiIiIxhVqeLKVOmYMqUKVofi42N1dgWHh6OQ4cOmfNUkpWYnovE9FyN7WsOJj/65mHFiUIhsCcxAx0aesHPw7VSz2vpSpShX+7HhbRc7IrqY/TChjVBQXEpHB1kcHa06xUciIgkiZ/MFrb6QJLy/0mZ+QCATceuY9zaY+j/2d9mndNSzUPaXEgrC1h/nrKf+VoKikvRfs529Plkj62LQkREWjCcWNHt3EIoFALbz5bNAZNbUGL0sSWlClxMz4VCIdQCD1Xe5Yw8lCgEUrMLbF0UIiLSonJjackgAWBP4m31bQ/bZvRN0vbGT8ex41w62tT3xPlU68yQWx0t/usCDifdwYbXHrNKh2YiIrI91pxUMSEERqw8hH+siFOGFG12nCubS6ZiMCkqVeB2bqHO40oVAi9+HYeojQkWKa/UrPj7Ck6kZGGLHTVDERHZG4YTK6sYQHILS3Do6l3EX7uHGyrDjo0V8dnf6LZgF67dydf6+KkbWTiSdBebT9w06bzW7NdiDSUKDq8mIqqpGE5saK3qyB6U9TM5fPUOCop1T7SW8bDWZNf5DK2Pq86JEnfljt5aFlXZD4oRvfk0jiXfNWp/IiIia2GfEyub/t/TOh/7++JtzFT5/tPtifh671VEtvPXeUw5XU1CqptHrDwEJwcZLi98EsmZ+fByc0adWtpn5S3vdLv+SAouzB8EV2f25zCkqEQBFyfmeyIiS+Mnq5X99/gNte87frhD577f7i8LCNvPpuvcx1QlCoGbWQ/Qd0ksOs/fadQxlhxiK4RA9v1ii52vsvYkZuC5rw5W+jyL/7qAljP/wpmb2RYolX76+iZVN7GJGbikZX4gIiJVDCc2dje/CHsuZOBOXqHaujzm0naGhJQsk86RUaEpSKEQ+PXEDVy9nWdyeT784yxC5u3A7gvqgau4VIGXVx3GZzsS1bbnFZZgxDeHNJq8LGXsd0dRVKowvKMBK/6+AgBYUqH8lnb6Rja6LdiFTcekuTI3APx0+Bp+OnzN4H5nbmbjle+OYsDne6ugVERUnbFZx4YEymZovZllesfYii6l5yIzrwiODpbv2PrHyVt4Z+NJAEDy4iEAgAdFpYjalID+bfyx42waUu7ex/fjuqvNgCuEwNq4spvWJ9sS8UTrR81VO8+lY9+lTOy7lImpA1spt6/en4S4q3cQd/UOxvQIVm4/mnwX+y9lWvzapG7K+uPIzCvCtF9O4cWuQRY/f/aDYni6Oukd1q5PbkExZvx6BgDwdEggPFydde57jkPiichIDCc2lJr1APlF5q0yvHDreZxPzcWSFzpCJpMp/xrV1l9FqNSn7L+UiTb1PXAiJQv9WvsZfJ7s+8X4ZNsFje2rDyThrzNp+OtMmnLbS98cwu6pfQEA+YUleOrL/Y/KUKFKp6hEe+1FXqH2iepeWBGndfvJ61lwdXZEq4CyqffvF5XAQSYzus/MusMpytesXm3prS+kWpsmhMA3e6+iXaAXerXwqfS5z9zMxlNf7segdgFY8XKoWedQfR91vadERKZis44NmRtMAEAhyvqzXLmtPqTYUH+Vf646jMH/3ofx3x/DD3HJBp9n5LeHcEtlJtUl28uaMe7lF2nse1WlLJtP3FRO3w+oByRLuH73Ps7czMawZQcQ+UVZMCsoLkXb2dvRaV5Zv5607ALsPJcOhZ7msg9+PY3Qj3Yh9KNdKCpRYNPR6+i5eDcupufizM1srD2YrDz+4JVM9Pp4N/ZevK1xnrv5RTh9w/L9T1Rv+LsvZGDRXxfwz1WHLXLu1Q/7OG07m2ZgTyKiqsWak2ruP7sv4c3+LfTucyz5ntr35X1KfqnQWVebs7fUq+L/s+cyXg5vrHP/lDv30aieu0Ynzjt5RRBCKJsPLuroFGls588vd1/Gl7svq227fvc+AKCgWIGDVzIxcmXZTXzJCyH4R2hDg+fMLSjGtP+eAgC89/NJnHwYNjxcnfBcl4bK841efUR5THljSPcFu1CiENj8Rg90aVTHqGswhmr/H3PmxTFWbkGx3iYZY9ScbrtEZGusOanmfku4hScMLCi4Rkfn0jM3dfcBuJtfhJwC7aNsnvvqoM5OpX0+3YP4a/c0tt/JL8LbKrPWfhV7Retz/q2lVsIcY1QChLaaDm1UK1gKVWosEtN0jy6RyWQQQignhdt3UXu/mBv37uPHQ9f0zmFTGYUlpUYHu32XbqP7gl1qE/V1+HAH5v95TmPf9UdS8O7PJ5XNSxfScjB100n8eeoWJnx/DMl37ut8nozcArXrrV7T/BGRLTGckFZd5u/UOez5ZtYD/O/kLZ3HPr/8oNYb0e8Jt3Dj3n2UVAg2Px+7jpJSBSK/2IuL6eojggpLjL+Zq3bqLC41/e/4not3a93+9d6rSM7UPiMvAMz9n+ZNvaKBn+/FzN/O4Itdl0wuVzldfVbv5heh3eztGLf2mNr2yxm5GP51HA5eUQ9ML686ojEiCwBW7ddcYDJ682n8En8D2x72LXr6ywP47/EbmLLuBHaeS8dr3z96zgVbziuXFbhx7z66L4hBr4/1D0svKC7F9rNpyNURhGui7WfTsOeC9kkUqeqlZRcgRU/IJttgOCGz3DMwd0mMjg/fXh/vwchv1ftMvPfLKXy6I1FjNtvtZ9PQauY2rDFiVeb7RSWVHvWkb4hx3yWxWrfLoLtmStX9h/2LDlx+FBQKiksx7ZeTyht/uVKFwH92X8L6Iykaz6XNllO3UKIQ2H0hAxfScpQ1KBO+j8fhpLvK5qjKiLmQjsKSUo3X6I5K36NfT9zE5HXHAQD7Ho6syswre09zCorVmn1KShWIv3YXs38/g9d/iMeE79WDVWXdyy/CgKV/Y9mey4Z3rkJZ94vw+g/xGLvmKIotMKSdKu+xRTHo8+kenTXFZBvsc0JWEZuouynlSJLmFPlf/31VY9vrP8QDAD40omai7eztOh+7mJ6LradT8WSH+gbPU+6CnqYcVRVv1p/vuoinQuqjmW9trfufVpm07fu4ZGw6dgObjt3Aljd74crtfOy9eBu/xBvuCwQAV27n4cM/ziK4Xi3ltkFf7MMn/+iIF7sG4aZKH5WC4tJKzfq7+fhN7Dxn3OSAH/5xVi2wnU/NweB/71Pb5+NtF7By36PQeeiqZZdN+GbfVVzKyMOn2xMxuV9zi567MnILHo1GK1UIGPOWFD9c7DPQ282KJaNbWQ/gGVC5fldkOaw5oRrvQlou3vjpuN6mKHOVd8JV1V+lD9DF9Fx88Kv2JQz2qczbMuT/9uPN9Sf0BhPVmoeNR1PQ/7O/se9SJn44pD4B2g8P55ZRDU6v/xCP9JwCGFJYUop8HcO5VW+s+lSsSaoYTACoBZPKyissURsZBgDFNWhY83NfHUSPxbu19uWqji6m5+LQ1TtV9nw1aYZle8JwQnbjX+tPWPycujqEbn84PHfg53ux7nCKxuP38ovUwokxtp5OVf5f35pNp29mY3yF/id/X7yNsIUxSM3W3/TVfUEM2s3ZrjOgSFHvj3ej35JYnL1l/aUErCkzrxDPfnUAG4+q/7yU17b9esK4GjWpG/j5Xrz0zaEq6eex7Uwqui3YhbgrlglDJaUKs5t/Eq5n4YoZs2zbK7sPJ23qe9q6CFQDvf5DvM41ZM7czFb2zTCFKU0fu85rb4I5mqz/r+/sB2UfvCdvZOFoNVmhurz/k6FOpgXFxo9oUnU7txBzfj+DC2nWneF2yfZEnEjJ0hs8rUEIgcNX72idu8iaku/o7mRuKRN/LJth2ZS5gdYeTMbkn45rdNwHymoBO364AxlG1EKqSs8pwDPLDqjVqpJ+dh9OXBw5wJGsQ9caMk99uR8HLfSXnKmMbe54/Yd4nbPyVkeXM/LQetY2RG06afKxU38+ibVx1zDoC83mKVWmjjiqmJN0zY5cTmalwdhbT6dh+DeHELG08jdOIQSm/XISX+y6aIGSWU75UPhTN7Kw/kiK3pA654+z2HI6Ff87pdkMfCmjrOZDX586bYztw0aP2H2HWLZGkj2Z+rNxN2dj+5dUF+XDpH89cROfD+9k0rHnjGguKu8E/P2r3dGnpa85RbTZZ9GOc2VNkHcsUHNy9lYONh0ra356O6Jlpc9nrvVHUnDqRpby+/Jh+E//5wAAwLe2HBFtNZf6UJVXUIKfDl/DbyduIrJdAHw9zFve4ljyXbV5l8g4dl9zwr5SRNKgUAhk3Tf/BplX+GhOHNVf62/3aY4EM0ZhSSne2nACmXnay3Qz6wFGfHMIu86lKzsBf7Jdcx0qSym/wWbfL8Z3B5I0ht5bUkmpAiv3XsWZm6b14zFlXiJd9C03Yazozaex/ojulbwvZmjWZOQVlCj7igFlTaAzfj2Do8n38NGW83hrQ4LB5z19IxtPLInFhiMp+Nf6Ezh4OROL/9L8mUi5cx8Dlv5dqdXGt5xKRfTmUzV2SDrDCetOiCTh1bVH0WneTrUbYmZeId5cfwLj1x5FRq7+dv7yDrE/HrqmNqHcR1vOm1WedYdT8HuCetV+eT+i/MIS9Fy8G3FX72B8Jedo2ZOYgfBFMTh4WX8H6fJGncgv9mLu/85hyP/pb2Z6UIm1u9YfScGCrefVFu805G6FmpfjKffwfzGXdN48F24te1+u3cnH6z8cQ8L1LGTdL0K3Bbsw7Rfjm99S7tzH+iMpeheerNgg9sm2RI0pDSb+GK+cvgAoW41dl8KSUiyPvaKxDMdrPxzD1cx8vL/5NP538pbGnE7lZv1+Bpcy8jDtl1PKbdfv3sf8P88ZPV/T5HXHsf7IdWw8an7AkTKGE2YTIkkob8f/UWVo9NjvjuKPk7ew63wGoo3sKDrztzN6Hze246e2ZQ+G/qfsZr3WiEUzgbIwozoq5ZbKjaf8D6Ox3x1FanaBwYkNd55LR//PYpH2sDNm+Sy/N7MeYMj/7cN/VYahz//zHNrM3mbUkN2KN+57+UWY9ftZnftnPyjG5J+OY/K648pr+3bfVXSZvxPfqgwRf+6rg1i686Ly/azYz6O8H8aE749h+9l0PLPsAP4dcwl38ouUTUOlCoE/Tt7Se8Pu8+keRG8+jeWxV0yqdXnxa/U+VbpqyLRZtucKPt52AQM/34uei3fjREpZR/PUbM0ArW1mZ23LWIxYeQir9ieh5+Ldak1ShlizBs2WGE4YTogkZcPR61j81wUcuJypNmldzIUMs+esUL1paQsvc34/g8jP92LV/iRl09IeLZ0eC4rLJkT7ZFui1udR7bSafb8YAz7fiz6f7sH/Tt7CCysO4pSRK1drG8p9K7tAYxVyoKy/y9lbOcr+REUlCmXN0Rs/Hcec389gxsO5doxpAlCtPdDmsx2J2HI6FVtOpaLPp2XLE5TXTv11RnOF64vpebh+9z66L4zBv7Us36A6HP+7A8lqj607koI3159An0/0L4MAlE2AWDFwlJPpWvvBTGkqo3VuZj3QO8NxxY7MV2/naQSWwpJStYU9n/7PAQghcPpGtloNmBACG46kqK2AfvZWNp5ZdgD/O3kL2SoBVwiBKeuOY8EWw5NYXr2dhxv3pDWFv913iCUi6Vnx9xWs+Ftzcchd5zMwwEBHRm02qrTtn7qZpfH42ocT183/8xzm/3kOSYue1HmuGTom1asoNefRzaZ8jh3Vodwx5zMwNCRQ67Ht5mzHkQ/6I86Imo+Kc9ecVPmr+25+kfLafjqcAldnB+yb9gR8PeRIuXMfDeu4aXR+PqJjCPmirechd3bUmAxu7HeGO3su/usCbucW4nNto3j05M1ZD4NkqUJg1f4k/KNLQ3i5657F9ZiOiepKFQKbrNj8kZlXhPOp2oeaV3w9n/jsbzSu5662TVvZNh27rhxWnrx4CICyn5n3N6v//O06XzaEvvxnrHzfs7dy8OfDta5mDGmLnIJibI6/AUcHGXo091HOYp39oFi5eGz5sVJg9+GkhX9tnNPxQ0VE0jLh+2NIXjwEqdkPMG7NMbXfXYUQRk3Edv3uA/x56hZiE29j4bMd4OKkWYHcbo7u5RB26JnGXyYr60z69d6rBufC+Nf6E3pH9mw/l668OeuSX1iisbq4vptwQbECI1ceQudG3th07AZe6hakcx2scmduZsPXQ46v92rvWKythkmdgMKMGq/nlx9U+37+n+ew/WwaFj3XAX+dTkXfVn5o38DL6PNN++8pjW0VZxauDG0zIetyrcIEdDlaRsdpm+8mUcfcSaouZ+SiuZ+HxtIa7//3FLaeflSz9UJoQ7wzoKVkR+bZfTjhehVE1UtsYgZe+e6oxvYDl+/gte/1N0mUm7Ku7K/M27mFmDaolcbj9yvRmfT5FXE4eT3LqH2n6pt3xYgbesUb+IOiUvxsYG2mSxl5yvk6NlQIMq+u0Xxdn/pyP/ZN62ewLLoIYV7zubbp+o8k3VVOZLZkh/a5VEwJCa//YNkFJ63l3Z9PYuaQNrhmxMR1o749jPnD2uO1Cs1zqsEEAH6Ov4GrmflY+GwHi5bVUmSiGiw8kJOTAy8vL2RnZ8PT07Izun687QKWx2pWHxMR2VJL/9q4mC6N6c73TeuH3kb0+9ClttxJ6yRzc4a2xVwjFvasyZIXD8GyPZfx6Xbt/ZiszUPuhNyH7401mnXMvX/bfYdYIiIpkkowsQRds9/aezCRglyJrqPFcEJERHpVptaE9Pt231XEJurv92OP7L7PibMj8xkREdmGuZME1nR2f2ce17OJrYtARERkc1Lqgmr34UTfmHkiIiJ7kf3AtJW1rcnuwwkRERGpz9ZrawwnRERExGYdIiIikhbpRBMzw8myZcsQHBwMV1dXhIWF4cgRw2srAMCGDRsgk8nwzDPPmPO0REREZCV3TViZ2dpMDicbN25EVFQU5syZg+PHjyMkJASRkZHIyNA/Tjs5ORnvvvsuevfubXZhiYiIyDp0LZxoCyaHk6VLl2LChAkYO3Ys2rZtixUrVsDd3R2rV6/WeUxpaSlGjRqFuXPnomnTppUqMBEREVmekFDDjknhpKioCPHx8YiIiHh0AgcHREREIC4uTudx8+bNg5+fH8aNG2fU8xQWFiInJ0fti4iIiKxIOtnEtHCSmZmJ0tJS+Pv7q2339/dHWlqa1mP279+PVatWYeXKlUY/z6JFi+Dl5aX8CgoKMqWYREREVI1ZdbRObm4uXn75ZaxcuRI+Pj5GHxcdHY3s7Gzl1/Xr1w0fRERERGaTUMWJaWvr+Pj4wNHREenp6Wrb09PTERAQoLH/lStXkJycjKFDhyq3KRSKsid2ckJiYiKaNWumcZxcLodcLjelaERERFQJ1XaeExcXF4SGhiImJka5TaFQICYmBuHh4Rr7t27dGqdPn0ZCQoLy6+mnn0a/fv2QkJDA5hoiIiLSYPKqxFFRURgzZgy6du2K7t2744svvkB+fj7Gjh0LABg9ejQaNGiARYsWwdXVFe3bt1c73tvbGwA0thMREZHtSKjixPRwMnz4cNy+fRuzZ89GWloaOnXqhG3btik7yaakpMDBgRPPEhERVScSyiaQCSk1MumQk5MDLy8vZGdnw9PT0+Ln/z4uGbN/P2vx8xIREVUX43o1wayn2lr0nObev1nFAWB0eDDmD2tn62IQERERGE6U+rfxN7wTERFRDSWldhSGk4cCvd1sXQQiIiKbqbbT1xMRERFZG8MJERERsVmHiIiISBeGEyIiIpIUhhMiIiKqvmvr1HSv9mxi6yIQERHZhHSiCcOJmlpyR1sXgYiIyO4xnBARERFH6xAREZG0cBI2iXJzYbMOERHZJ9acSNSY8GBbF4GIiMjuMZyoqCV3snURiIiIbEJCFScMJ/q82b+FrYtARERUJdisU0009HaDixNfIiIioqrEO28FP4zrrvz/s10a2LAkREREVefq7TxbF0GJ4aSCOu4uyv87Ozpg2cguNiwNERFR1ci6X2zrIigxnBgwoK2/rYtARERkVxhOKvDzlNu6CERERFWOk7BJmJ+HK34Y1x3/ndRDuW3mkDY2LBEREZH1KaSTTRhOtOndwhehjesovx/fu6kNS0NERGRfGE6M9PrjpgeUoLpuVigJERGR5SkkNNEJw4mR3oloiY+f72DSMc19a1upNERERJYloWzCcGIsV2dHDO/WyOLnjZn6uMXPSUREVJ0xnFjIiO5BGts6NvQ2eJyrM1dCJiIi2/OtLZ3RqgwnlTAy7FFNSmS7ALQO8FB7vFcLH4PnqM3FBomISAJCgrxsXQQlhhMztanviYXPPuqDIpPJTO40u2ZsN3i5OVu6aHarqU8tWxeBiKjaYp+TGkhmxjF9W/lZvBz2bOZTnI+GiMhcEsomDCeWJKsQUSoGlm9Hd626wtghLzcXwzsREZHkMZyYqWLwMJQ42zfwRISF1uk5P28QLi0YjDHhjY0+5tWeTbBufBhcHGvuWy4zp/qKiIgAAK0q9Ju0pZp7p5IAAWD+M+3xWNO6+GViD4P7G8vNxRHOjg6Y/ERz5bYWfvrnVHk3siV6NPfBgmfb691vRPdGaOBdPSePYzYhIjKfo4T+wmM4saBG9dw1tr38WGNseC3cYkOGAzxdlf/383CFq3PZW/ifkV3w9cuh6B5cV/m4agdRd5eyUUH/CG2o9/wfPNkaHz2jP8BUNNqEGhxjffNyqMXPSURE1QPDiZkCvFw1tnVpVAefPN9R+b22DDq4fQAAYEiH+mY9r3+F5z06IwJ73u2LVgEeiGwXAF+VVZW3vd0H3u7OaOb7KKTIDCRjD1dn9GttWkfdHs0MD5k2lY+H6ePtDV1bRe0CPU1+DiIisj6GExP9MK47Itr4qQ0jBqCc4+TFbpqTsan67MUQrPhnF3z6Qke9+xnLw9UZTVRqSIZ2LAs9jeq6w8XJAUdnRGDHO9abhXbVmK4YoKMvjSl9YlSN69XErOpFU45Y+GwH/DGll8nPQebxcOV8PkRSJ6XROvzEMFHvFr7o3cJX+f2RGf2RW1ACf0/NmhRt3F2cMKi9ebUmANDKX3/fksh2AfhjSk80fbiuj7OeDrA+teXIzCs0uyy7pz6ufJ7pg1rj420X1B6fO6w91sZdM/m8rQI8tHZubeZbC1du5+s8TkLNpdWWp6sTcgpKLH5ef09X5BbkWfy8RGQ5UhovIaGiVE9+Hq5oVmGBv/oPm17amtlsMHNIG7zVv4XWx2YMaav3WJlMho4NvY2aebZjQy+1xQxVm3/eH9za4PH1VKY6ljtZ7kdJBsDJQfN8FZttjs8aoPZ96wD119tBBp2LNVozyPia0SRlivciW1nt3MM6NVD+P6xJXT17EhFZj1l3lGXLliE4OBiurq4ICwvDkSNHdO67cuVK9O7dG3Xq1EGdOnUQERGhd/+aYO+0fjg/b5CyE6qpRocH450BLbU+ZukZZYd3a4TLCwbj6IwI/PVWH+X21/s0xZ//6qU2csfSN8XDH/RHv1a+GtvbN/BCm/oeiGjjj1EqSwRUzBN1a6nPa+Li5IB/v9Tp0f4yGYZ3awQXHcFJmDEd4jOdAg3uY855yxkT8joHeZt9/vnD2ul9fMaQRxPZVXx9dZn38JzlnbXH9WpiZumIiMqYHE42btyIqKgozJkzB8ePH0dISAgiIyORkZGhdf/Y2FiMGDECe/bsQVxcHIKCgjBw4EDcvHmz0oWXKmdHB7i5VH50Tq/mZR1NVUfgWIOTowN8PeRqN3GZTIb2Dbzg5PgoErzQVX2kT2VrHyo2he19rx82vvYY2tT3hEwmw7djumLBs9prPiqa3K8ZAO01LtqYW/QvXupscJ/G9cyfRt/UTrr1jAwQxnCQqS9E6eggw+kPBxo8bnR4MI7M6I+46CdwcvZAzHpKe+3eT+PDLFZWIrK8ihOJ2pLJ4WTp0qWYMGECxo4di7Zt22LFihVwd3fH6tWrte7/008/4Y033kCnTp3QunVrfPvtt1AoFIiJial04Wu6NWO74eD7T+D7cd0xOrwxfhjX3abl8fNwNbmqf0T3ILjpGUb9VMeymoigum5oVM8dYU3rGXVeh4e/Q4ue64DX+jTFuwPLanX6t3k00mjl6LLhyO213PBb+FtnsqHvX+2ORc+VBaomPrUM1lQAwDsRj2rJTB1xtOSFEKP3dXFywJCOumt+dk/tq7HNw9W4mjo/D1fIZDJ4uWvfXwiBns0tP6qLiGomk8JJUVER4uPjERER8egEDg6IiIhAXFycUee4f/8+iouLUbeu7ptcYWEhcnJy1L7sSfn9ycnRAYHebnB1dsS8Ye3VOuJaQtfgOobLUuH7Da89ht4tfPBEaz94GNGvZdFzHfX+9f1clwbY9Ho4/vxXb4PnUpUwp+ycI7o3wgdPtlHe1F2dHZG06ElcXjAYT7QuG0W0bFQXjAprhB3v9MHWN3tj+aguCG2see0dG3rhUHR/k8pRUZ+Wvmjp74Fz8yKxe+rjesMAAMwZ2hZvRbRQzhUzdaD25jxVjg6P3hVjm14A4MyHkahbywUTemtvdgl+OOrrvchW8PWQY/qgsn5HwSrz95z6cCDa1rfsEOxOJjZTvR3RQm2+HyIp6MXwbVEmhZPMzEyUlpbC31996Ki/vz/S0tKMOsf06dMRGBioFnAqWrRoEby8vJRfQUH6h+fWNNauWNvzbl/MH9YO43uZtooyUPaX/Q/jwrD6lW5G/5XvpKcLuEwmQ/cmdY3qS6P6dJ56/qKXyWRqz1nfyw0Lnu2Alv4eaBvoicEP55hxdJChZ/NHNTXTB7XWOn+NOdxdnHS+PoPaBSj/P7ZnWVCYN6w9zsyNRI9mPlj6ou7akMHtA9C9SV083tIXT7T2Qx33R+FE9elUA0SXRt7476Qeyma7GUPaonuFGrANrz2m/P/kfs1x5IP+CKpbFkp2RT2OWU+1xdY3e+t93VU19TWuaUsmAzZP6qE1LOri4eqMnVF9DO9oghHda85nTFPfWpzE0AY++YdlpoewJSmNeKzS0TqLFy/Ghg0b8Ouvv8LVVfdNIDo6GtnZ2cqv69evV2Epq96WN9Xn29B3M7eEJj618HJ4sM6Ooqqe71LWz6RDAy+rlskWZDIZfhz3qB+Eao2EPvOfaQ8HGYy6oWrrHPuSjhth+Qir57o0xLl5kVg2sova44Ferlj+z1DIZDKsfbX7w4D46HHVzsuqz7r5jZ4aZVW90uTFQ/BYheY01WDl5OiAcb2aKEefvdhV/yzDAOBcoe+Pri7CgV5ucHCQKWc6NpaxzU3GMmeF8L/f62vRMljKM50aqPXnCrRQ4LY1XYHXp7ZlF/wMqls9l++oaUz6RPDx8YGjoyPS09PVtqenpyMgIEDHUWWWLFmCxYsXY8eOHejYUX/ClMvl8PT0VPuqydoFeqGlgflLbGVS32b4/tXuWDdBf2dGY4YuV5a+OVvMpXoT9tbRX6Jc34cji15+rDEufjQYPZsZ1z9G33Pq4u7ihCc7BGD72+bVEFjz52l0eLDFz1kVHfEsfdPRFe61vfb9TZx1uTIUFQLxvulPVNlzW9NXo7po3X74A9218Pum9VP73tlRZrDvnrk/i8b+cUPGMenT3sXFBaGhoWqdWcs7t4aHh+s87pNPPsH8+fOxbds2dO3a1fzSUpVzcnRAn5a+Bv9SfaZzA0S08ceE3k3QwNsNHzxpeJ4UY73Rtxm6BdfBk2ZO+W/IkhdC8F5kK415UsoNaOuPmUPa4IvhnZTbnBwdUMuIQFaZGRdlMpnaKqHaQo3q6BoHlcc/HNoOo8Mb448pPStRAu0cHGSSnvFV24zFUwe0xJY3e+us7bLkbSWizaPnb1jHDTFTH8fXVdjMUnEdL303zZOz1fuDGVMrZq5xvZqgYR3zAmLvFj46fz/1XV9502S5v97qg94tfPX21VL9NUtePASXFgw2qox+euY3+nmi7vtjRX/+q2pmro6Z+jiGhhieGsFWTP5TNCoqCitXrsTatWtx/vx5TJo0Cfn5+Rg7diwAYPTo0YiOjlbu//HHH2PWrFlYvXo1goODkZaWhrS0NOTlcbbImsTFyQHfjumKGUPa4sD7T+C1Ps0sdu5pg1rj54k98GLXsuaQJyz8V+g/Qhticr/mOh/39ZBjfO+m8HZX/0AbHR6M3i1M6wQX6OUKZwv+heXrIce0Qa0wc0gbtTlS6tRywbxh7dGxobfW46TUtgxA2fn29cdN7wdVvl4VALwQ2hArR2v+ATS5X3N4ujor+/NoG7JdPsqqsoZXWMKimW9tvU21wfXc0bGh5ZpNHWRAm/qe8PeUI8TAeSuOrhoVZvySE6qvuzEGtQ/AfjNrcVoZGF1XPvGlal+q8hosudoUCWX/Vlx+RNXsh0PhJz5e9hlmbI2tTCbDTJV5glR1C66LaCMmtgSgMalnZakuAGvoeSozDYKlmRxOhg8fjiVLlmD27Nno1KkTEhISsG3bNmUn2ZSUFKSmpir3X758OYqKivCPf/wD9evXV34tWbLEcldRAyx6rgMcZMbNzGqvfD3kSPxoEFaNkUbtm5uLI34YF4aINrrDknuF+W5+HB+GsKb10Kelr8UmK3ujb3OM7236Td2a+lacXM9AFVKHhl64+NFgRA/W/uGuzWcvhKBHs3pY+GwHfD48BN2b1MW0Qdp/fxweBsLG9WohefEQbHlTc3SYrg9xU5ybF6nzA765n+bNYOLjzRD7Xj+sm/AYlo/qgtf7NFVbPNQYr/QIVvte7uQIFycHHJj+BH59Q3fN2aS+6n9ANPB2U5vXyJB5wzRXL//+1UdNJlN1TCRpDl0/Ps4Py7t3Wj/sm9YPm14Px+pXuqJjQy9lny3VGs9yA9v66/wjp38bf5z+cKDWz+J3VUbTrRnbTfkcukKJOSwxR5aq2npqOSvWgJo6cs6azKqbnTJlCqZMmaL1sdjYWLXvk5OTzXkKuxPauC4ufjTY6p1hbeXwB/2RnJmP4d8cqtR55E6W/cW1BH0di91dnLD21e6QAejRrJ7y/VX9EDeWoZlnPxjSBmO/O4rxRoSehnXcAdw1uQzGemdAS3y996pJx5S/jrOeaou9F28jLbsAiem5Ovd/PrQhng8ta4Z4tnNDPNvZ/CaJ1gGeSM1+oLatgbcb2gV6Yse5dK3HyJ0cUcfdGffuFyu3VZwVWrWG6sdxYXhz/QkcSS573b3dnZUBobbcCYM71FeOJGtcz13td8XV2QEFxQoAZbUD0YNb49mvDqK23AlzhrbFmoPJAMpew/KaG0OfJdMqzPgshFDrbyGTAfp+5LSNsOvT8lEojWjrj892XlQ5f9m/MVMfR15BCZ796gAUFlppztnRQdmE80Rrf+U0AoD2m72DgwyrX+mGZ786gBMpWRqPG9PhumdzHzg7OuCJ1oNMDhQNvN1wM+uBxvYFz2oGvooGtPXHTpWfyeB67ki+c1/rvg6yshqrUzeyAQBfvxyK13+Ix5CHC8ROHdAS6w6nAACe69xA6zlspWbeCaupmhpMgLLZYI2dYK26+eDJNmjiU0s5jXtFj7f0RZ+WvlZ/f/u18sPpDwdipo4ZWlXNeLINnu3cAOvMnbXVwE3F1dnR7KaKcb2aYO2r3dVCn2qHUhcj/ro/OiMCY3sGG/V8EW380aieu1qfnh3v9MG+af2UzQVA2ZDqiDb+6Nq4DuY+3Q51a7kgfuYAbafUKsDLFZtU+h5seO0xnUPow5rWUxve7efxqBw/jOuOzo3q4Py8QYifFQGZTIY//9UL/xnZGRc/GqzR50SbVWO6au3DVDFMAbqXzHBxctAbhIXQ3hzTzLc2QoK8EWfCnEKVWBEC7U0YaVjHQKd4ba+ZOTUd29/R7Oj+WNO6BpvVkhY9qdFs2VJHk1dzv9o4P38Q6qo0R0e2C8DpDwfiPyPKZrquV1uOIx/0x4Jn2+MjI4JRVaq5d0OSpJEP18qpWKVcnTWs44497/a1yigWVcaM8jF2iG2dWi74fHgn9LDixFFBdR51RvTUcYPTFyCESgJapjJSw5jXwddDrlwx25DyENW5kTda+XtgYFt/tPT3gIODTDkPzbBOgWjuVxvfjumKXyb1wJiHTSkOKv2HjO0LVV6VHmygfV91eLfqMNry2kM3F0fl/9s38FLOtqzNjgo3w/4qnXbbNyjrf/N0J/W/nHs298GWN3th//R+OpdJmPlUW53LFQDA7yodsuvWUv8ZMHYl98ryVvnZq/iTo/r9B0+2xh9TLN8ZVdvPhbbRjbV0rMXWQ2VUYPnP/pSHfeRmDmmDhVr6Sj0dEojVY7pB7uSIoSGBaODtppwWwsPVWe13yM/TFaPCGpu9Fpy1SKs0VOPNH9Yeo8IaoY2OnvdSsG5CGL4/eA3bzho3sSBpN3dYO+y9eBu5hSX4vEK7/0vdgjAyrJHR8+eo1gYY+0e0qX1+nR0dsO1t9b4owT61cGH+IKMWZPT31BytoW2tp2MzI6BQaI6q0WbZyC74+2IG5j/THh/+cRZtA82rjWrp74FuwXVwNPmexmM/jXsMh5LuoF8rP1zNVB+o0O7h85nbgdrV2RHLRnZBRm4BmvuZtmSEs6MMxaXq73av5j44nHRHY7slGNOJ39BK89qC80vdTJ/gr6V/bVxMz0Mrfw/41Nb8uZo6sCX++VhjrZNGjuvVRC0w1pI7Yd+0fmpBujpgOKEq5eggU37gSVWPZj7o0cwHwe9vsXVRqjWf2nKcnhup9bGyZh9vvcc3rlsLZ25af+kK1Y9sQ8O19QnwfDRM9rMXQvDxtgv4coTmQpGm9Jsa0rG+sn/AoucqNwNpeDMfHE2+B6cKNykvd2dEPpy1WLXPiRBC5fUw/cZWPhKovPyV0eLhyJsfxnVHUakCrWZuM/pYS8SYwx/0x528IrUw4GBkYtPWERoomyl6/+VM5BWWaDy2Zmx3/HDoGkaHN8Z/dl/WeFwmk2kNJrHv9lUuQ6GqugUTgOGEqNow1CG2Kvl4yJGr5UPVkKgBLfFz/HW80c/wX6lzh7WDi5MDRnRvpP6Aka9DVQ2XXv1KV2w/k47X+jwaMfV8aEM816WByQs5WtPkfs3g7ylHHz1rdJlT3IqHrPhnF+Q8KFGbsbiyyqcRkMlkleoUX/H9MPb98fd0VTZDrRlb1lxiaNK1XVGP43Zuoc7mxeX/7IJShUDzGX9pPBbo7aYcXh81oCUupedprAqv6u/3+iIzr0hrMKmuGE6IdKgtd0JeYUmVzu5ZXXzzcig++PU03o4wbbjom/1b4M3+LYza16e2XKM5yBTGzvRZ2fxQcXTIo/NKJ5gAZTU2psxjIoRxr03Fe/Sg9ubVlIzr1QQyAN/uTwIA9G/tj21n09Dcr7bVZl8d0yMY8dfumTRfkbFLHTT3q61Wa1JxvpSyNcBk+OdjjfDjoRSdvxf1asvVOlJr07heLUnNUWIJDCdEOuyd1g+XM/LQzYjVm6uClG52Lfw98PPEHlX6nK/0CMbfF2/juS7Wm8XU3un6Cav4o9dMpYPu86EN8d3BZPQzY32ict+O7oqIhzP7loeTAC9XnJw9EO5y3TUlxgRQJwcZfGq7ILdAszbn6ZBAtAv0RKMKM8maq0sjb52PDe8WhM0nbmgE2Y+e6YAZT7a1+Pwm1R3DCZEOdWu5aKzeS7bz4dPtKvSD0M/YLCel0Gdrqn0TdDWeTejdRDlaCSgb/RH7bl+zXsduwXWQmJaLHiqrgytrLNv4acxgaw6ZTIa46P4oVQitcxJZckbWzo3qYN2EMLWRauVqyZ3w5780J/8DLD/xWk3AcEJE1QaDhHU19amFfq184eXmrLMpZcYQzaHD5r4vm14PR4lCqDV57J/eD0mZ+ejcyHCNpbZOodo4OzrAyH7NldajmfWG59sThhMiqpEYY0wnk8nw3VjN2Yut9VrKZDLlFPTlvN1d0LmR7oX5AOCXieH4d8wl5To4VPNwEjYiiSufxKniGiqkn7F/zDeuZ5n+BlR1ugbXxQ/jwtDCwIKAVH2x5oRI4lb8MxSJabnKmTzJMja89hhOpGThSTNHlxCR9TCcEEmci5MDOpi5To09MzSS47Gm9dSmiCfd2NWHqhqbdYioZuINlajaYjghIiIiSWE4IaIaiRUnlmPsbLtElsJwQkQ1krbVXImoemA4IaIaqW8rX4zv1QT/fqmTrYtS7fVrXbZYoL8nAx9VDY7WIaIaSSaTYSYn6bKIWU+1Rdv6nsr1b4isjeGEiIj0cndxwsvhwbYuBtkRNusQERGRpDCcEBERkaQwnBAREZGkMJwQERGRpDCcEBERkaQwnBAREZGkMJwQERGRpDCcEBERkaQwnBAREZGkMJwQERGRpDCcEBERkaQwnBAREZGkMJwQERGRpFSLVYmFEACAnJwcG5eEiIiIjFV+3y6/jxurWoST3NxcAEBQUJCNS0JERESmys3NhZeXl9H7y4SpccYGFAoFbt26BQ8PD8hkMoudNycnB0FBQbh+/To8PT0tdl4pqenXyOur/mr6NfL6qr+afo3WvD4hBHJzcxEYGAgHB+N7klSLmhMHBwc0bNjQauf39PSskT9wqmr6NfL6qr+afo28vuqvpl+jta7PlBqTcuwQS0RERJLCcEJERESSYtfhRC6XY86cOZDL5bYuitXU9Gvk9VV/Nf0aeX3VX02/RileX7XoEEtERET2w65rToiIiEh6GE6IiIhIUhhOiIiISFIYToiIiEhS7DqcLFu2DMHBwXB1dUVYWBiOHDli6yJh0aJF6NatGzw8PODn54dnnnkGiYmJavv07dsXMplM7WvixIlq+6SkpGDIkCFwd3eHn58f3nvvPZSUlKjtExsbiy5dukAul6N58+ZYs2aNRnks/Rp9+OGHGmVv3bq18vGCggJMnjwZ9erVQ+3atfH8888jPT29WlxbueDgYI1rlMlkmDx5MoDq9/7t3bsXQ4cORWBgIGQyGX777Te1x4UQmD17NurXrw83NzdERETg0qVLavvcvXsXo0aNgqenJ7y9vTFu3Djk5eWp7XPq1Cn07t0brq6uCAoKwieffKJRlp9//hmtW7eGq6srOnTogK1bt5pcFlOur7i4GNOnT0eHDh1Qq1YtBAYGYvTo0bh165baObS954sXL5bE9Rm6RgB45ZVXNMo/aNAgtX2q63sIQOvvo0wmw6effqrcR8rvoTH3BSl9dhpTFoOEndqwYYNwcXERq1evFmfPnhUTJkwQ3t7eIj093ablioyMFN999504c+aMSEhIEE8++aRo1KiRyMvLU+7z+OOPiwkTJojU1FTlV3Z2tvLxkpIS0b59exERESFOnDghtm7dKnx8fER0dLRyn6tXrwp3d3cRFRUlzp07J7788kvh6Ogotm3bptzHGq/RnDlzRLt27dTKfvv2beXjEydOFEFBQSImJkYcO3ZMPPbYY6JHjx7V4trKZWRkqF3fzp07BQCxZ88eIUT1e/+2bt0qZsyYITZv3iwAiF9//VXt8cWLFwsvLy/x22+/iZMnT4qnn35aNGnSRDx48EC5z6BBg0RISIg4dOiQ2Ldvn2jevLkYMWKE8vHs7Gzh7+8vRo0aJc6cOSPWr18v3NzcxNdff63c58CBA8LR0VF88skn4ty5c2LmzJnC2dlZnD592qSymHJ9WVlZIiIiQmzcuFFcuHBBxMXFie7du4vQ0FC1czRu3FjMmzdP7T1V/Z215fUZukYhhBgzZowYNGiQWvnv3r2rtk91fQ+FEGrXlZqaKlavXi1kMpm4cuWKch8pv4fG3Bek9NlpqCzGsNtw0r17dzF58mTl96WlpSIwMFAsWrTIhqXSlJGRIQCIv//+W7nt8ccfF2+99ZbOY7Zu3SocHBxEWlqactvy5cuFp6enKCwsFEIIMW3aNNGuXTu144YPHy4iIyOV31vjNZozZ44ICQnR+lhWVpZwdnYWP//8s3Lb+fPnBQARFxcn+WvT5a233hLNmjUTCoVCCFG937+KH/wKhUIEBASITz/9VLktKytLyOVysX79eiGEEOfOnRMAxNGjR5X7/PXXX0Imk4mbN28KIYT46quvRJ06dZTXJ4QQ06dPF61atVJ+/+KLL4ohQ4aolScsLEy8/vrrRpfF1OvT5siRIwKAuHbtmnJb48aNxeeff67zGKlcnxDar3HMmDFi2LBhOo+pae/hsGHDxBNPPKG2rTq9hxXvC1L67DSmLMawy2adoqIixMfHIyIiQrnNwcEBERERiIuLs2HJNGVnZwMA6tatq7b9p59+go+PD9q3b4/o6Gjcv39f+VhcXBw6dOgAf39/5bbIyEjk5OTg7Nmzyn1Ur798n/Lrt+ZrdOnSJQQGBqJp06YYNWoUUlJSAADx8fEoLi5We87WrVujUaNGyueU+rVVVFRUhB9//BGvvvqq2qKV1fn9U5WUlIS0tDS15/Hy8kJYWJjae+bt7Y2uXbsq94mIiICDgwMOHz6s3KdPnz5wcXFRu57ExETcu3fPqGs2piyWkJ2dDZlMBm9vb7XtixcvRr169dC5c2d8+umnatXl1eH6YmNj4efnh1atWmHSpEm4c+eOWvlrynuYnp6OLVu2YNy4cRqPVZf3sOJ9QUqfncaUxRjVYuE/S8vMzERpaanamwQA/v7+uHDhgo1KpUmhUODtt99Gz5490b59e+X2kSNHonHjxggMDMSpU6cwffp0JCYmYvPmzQCAtLQ0rddW/pi+fXJycvDgwQPcu3fPKq9RWFgY1qxZg1atWiE1NRVz585F7969cebMGaSlpcHFxUXjQ9/f399guaVwbdr89ttvyMrKwiuvvKLcVp3fv4rKy6PteVTL6ufnp/a4k5MT6tatq7ZPkyZNNM5R/lidOnV0XrPqOQyVpbIKCgowffp0jBgxQm2BtDfffBNdunRB3bp1cfDgQURHRyM1NRVLly6tFtc3aNAgPPfcc2jSpAmuXLmCDz74AIMHD0ZcXBwcHR1r1Hu4du1aeHh44LnnnlPbXl3eQ233BSl9dhpTFmPYZTipLiZPnowzZ85g//79attfe+015f87dOiA+vXro3///rhy5QqaNWtW1cU0yeDBg5X/79ixI8LCwtC4cWNs2rQJbm5uNiyZdaxatQqDBw9GYGCgclt1fv/sWXFxMV588UUIIbB8+XK1x6KiopT/79ixI1xcXPD6669j0aJFkpoSXJeXXnpJ+f8OHTqgY8eOaNasGWJjY9G/f38blszyVq9ejVGjRsHV1VVte3V5D3XdF2oau2zW8fHxgaOjo0bv4fT0dAQEBNioVOqmTJmCP//8E3v27EHDhg317hsWFgYAuHz5MgAgICBA67WVP6ZvH09PT7i5uVXZa+Tt7Y2WLVvi8uXLCAgIQFFREbKysnQ+Z3W6tmvXrmHXrl0YP3683v2q8/tXfi59zxMQEICMjAy1x0tKSnD37l2LvK+qjxsqi7nKg8m1a9ewc+dOg8vKh4WFoaSkBMnJyXrLrlpuW15fRU2bNoWPj4/az2R1fw8BYN++fUhMTDT4OwlI8z3UdV+Q0menMWUxhl2GExcXF4SGhiImJka5TaFQICYmBuHh4TYsWdkwsylTpuDXX3/F7t27NaoRtUlISAAA1K9fHwAQHh6O06dPq32YlH+gtm3bVrmP6vWX71N+/VX1GuXl5eHKlSuoX78+QkND4ezsrPaciYmJSElJUT5ndbq27777Dn5+fhgyZIje/arz+9ekSRMEBASoPU9OTg4OHz6s9p5lZWUhPj5euc/u3buhUCiUwSw8PBx79+5FcXGx2vW0atUKderUMeqajSmLOcqDyaVLl7Br1y7Uq1fP4DEJCQlwcHBQNoVI+fq0uXHjBu7cuaP2M1md38Nyq1atQmhoKEJCQgzuK6X30NB9QUqfncaUxShGd52tYTZs2CDkcrlYs2aNOHfunHjttdeEt7e3Wk9mW5g0aZLw8vISsbGxakPa7t+/L4QQ4vLly2LevHni2LFjIikpSfz++++iadOmok+fPspzlA8ZGzhwoEhISBDbtm0Tvr6+WoeMvffee+L8+fNi2bJlWoeMWfo1mjp1qoiNjRVJSUniwIEDIiIiQvj4+IiMjAwhRNkQtEaNGondu3eLY8eOifDwcBEeHl4trk1VaWmpaNSokZg+fbra9ur4/uXm5ooTJ06IEydOCABi6dKl4sSJE8rRKosXLxbe3t7i999/F6dOnRLDhg3TOpS4c+fO4vDhw2L//v2iRYsWasNQs7KyhL+/v3j55ZfFmTNnxIYNG4S7u7vGME0nJyexZMkScf78eTFnzhytwzQNlcWU6ysqKhJPP/20aNiwoUhISFD7nSwf4XDw4EHx+eefi4SEBHHlyhXx448/Cl9fXzF69GhJXJ+ha8zNzRXvvvuuiIuLE0lJSWLXrl2iS5cuokWLFqKgoKDav4flsrOzhbu7u1i+fLnG8VJ/Dw3dF4SQ1menobIYw27DiRBCfPnll6JRo0bCxcVFdO/eXRw6dMjWRRIAtH599913QgghUlJSRJ8+fUTdunWFXC4XzZs3F++9957aPBlCCJGcnCwGDx4s3NzchI+Pj5g6daooLi5W22fPnj2iU6dOwsXFRTRt2lT5HKos/RoNHz5c1K9fX7i4uIgGDRqI4cOHi8uXLysff/DggXjjjTdEnTp1hLu7u3j22WdFampqtbg2Vdu3bxcARGJiotr26vj+7dmzR+vP5JgxY4QQZcMjZ82aJfz9/YVcLhf9+/fXuO47d+6IESNGiNq1awtPT08xduxYkZubq7bPyZMnRa9evYRcLhcNGjQQixcv1ijLpk2bRMuWLYWLi4to166d2LJli9rjxpTFlOtLSkrS+TtZPm9NfHy8CAsLE15eXsLV1VW0adNGLFy4UO3GbsvrM3SN9+/fFwMHDhS+vr7C2dlZNG7cWEyYMEEjxFbX97Dc119/Ldzc3ERWVpbG8VJ/Dw3dF4SQ1menMWUxRPbwwomIiIgkwS77nBAREZF0MZwQERGRpDCcEBERkaQwnBAREZGkMJwQERGRpDCcEBERkaQwnBAREZGkMJwQERGRpDCcEBERkaQwnBAREZGkMJwQERGRpDCcEBERkaT8P8Uw/TcCEwn5AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(lossi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that we have the initial setup done, let's look into the optimizations that we can perform.\n",
    "\n",
    "## Overconfident Softmax\n",
    "\n",
    "The first problem that the above model has is the \"hockeystick\" nature of loss over time, as is evident from the above graph. During first iteration we are seeing loss of 29.8979, which is way-too-high. \n",
    "\n",
    "### What should be the expected loss for first iteration?\n",
    "For the first iteration, we'd expect the model to have no prediction ability, so it should assign equal probability to each of the character which would be 1/27. Let's look at what the loss corresponding to this value comes out to be.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "tensor(3.2958)"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "-torch.tensor(1/27.0).log()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the loss reported by the neural network on first iteration should be around 3.29, Let's try to look at what the logits reported by HL2 after first iteration are"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 29.8979\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([ -7.0724,   4.4451,  10.1186,  13.6337,   8.1399,   2.1134,   3.3501,\n",
       "         12.6121, -10.0490,  -6.7101,  -9.5156, -17.8216,   7.2949,  -1.5835,\n",
       "         10.2453, -14.9260,  -4.5385, -16.9217,   5.0395, -11.4323,  17.1238,\n",
       "         12.3386,  16.0655,   0.0758,   1.6998,  28.4674,   6.6073],\n",
       "       grad_fn=<SelectBackward0>)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Reinitialize weights and then run the training loop again\n",
    "logits, _, _ = run_training_loop(break_on_first=True)\n",
    "logits[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see, the logits are diverging a lot, thus making the softmax overconfidently predicit a few character where it should have almost same values for each character. We can solve this problem by multiplying W2 by a constant, say 0.01 which would reduce the value of weights, hence the value of logits reported by (W2 * input + b2), let's look at the logits reported by changing W2 to W2 * 0.0.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11697\n"
     ]
    }
   ],
   "source": [
    "n_embd = 10 # the dimensionality of the character embedding vectors\n",
    "n_hidden = 200 # the number of neurons in the hidden layer of the MLP\n",
    "\n",
    "g = torch.Generator().manual_seed(2147483647) # for reproducibility\n",
    "C  = torch.randn((vocab_size, n_embd),            generator=g)\n",
    "W1 = torch.randn((n_embd * block_size, n_hidden), generator=g)\n",
    "W2 = torch.randn((n_hidden, vocab_size),          generator=g) * 0.01\n",
    "b2 = torch.randn(vocab_size,                      generator=g)\n",
    "\n",
    "parameters = [C, W1, W2, b2]\n",
    "print(sum(p.nelement() for p in parameters)) # number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After running the training loop for first iteration, let's check the value of logits again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.8155\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([ 0.5781, -0.8984, -0.1784,  1.5460, -1.6654, -1.3676,  0.2092,  0.8925,\n",
       "        -1.1660,  0.3242,  0.0055,  0.4046,  1.0528, -0.4510,  0.5271, -0.6357,\n",
       "         2.5709,  0.9907, -0.1271, -0.9460, -0.7640,  1.0952,  0.0044, -0.3396,\n",
       "        -1.4397, -0.7380,  1.3038], grad_fn=<SelectBackward0>)"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "logits, _, _ = run_training_loop(break_on_first=True)\n",
    "logits[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the values are closer to each other now. Now let's run the entire training loop and check the final reported loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.8155\n",
      "  10000/ 200000: 2.1191\n",
      "  20000/ 200000: 2.6264\n",
      "  30000/ 200000: 2.1872\n",
      "  40000/ 200000: 2.4961\n",
      "  50000/ 200000: 1.8205\n",
      "  60000/ 200000: 2.1090\n",
      "  70000/ 200000: 2.3420\n",
      "  80000/ 200000: 2.3301\n",
      "  90000/ 200000: 2.0682\n",
      " 100000/ 200000: 2.2838\n",
      " 110000/ 200000: 2.2106\n",
      " 120000/ 200000: 1.6535\n",
      " 130000/ 200000: 1.8890\n",
      " 140000/ 200000: 2.0644\n",
      " 150000/ 200000: 1.9606\n",
      " 160000/ 200000: 1.9712\n",
      " 170000/ 200000: 2.3565\n",
      " 180000/ 200000: 2.0956\n",
      " 190000/ 200000: 2.1049\n"
     ]
    }
   ],
   "source": [
    "lossi = []\n",
    "# reinitialize the weights and run the full training loop again\n",
    "run_training_loop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train 2.0693342685699463\n",
      "val 2.1324031352996826\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x124c207d0>]"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAGdCAYAAADJ6dNTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABUbElEQVR4nO3deVxUVf8H8M+wg7KoyKIiuO+CghKWS0muT9r2RGZpVFamv8eixWhRq6e0LLPMtMdS27VsT9MSl1xQE8Vd3MWFRVRAUQGZ8/tDGRlmu3fmzsyd4fN+vXi99M6de8/l6tzvnPM936MRQggQERERqYSHsxtAREREVBODEyIiIlIVBidERESkKgxOiIiISFUYnBAREZGqMDghIiIiVWFwQkRERKrC4ISIiIhUxcvZDZBCq9Xi9OnTCAwMhEajcXZziIiISAIhBC5cuIAmTZrAw0N6f4hLBCenT59GVFSUs5tBREREVjhx4gSaNWsmeX+XCE4CAwMBXLu4oKAgJ7eGiIiIpCgtLUVUVJTuOS6VVcHJ7NmzMX36dOTn5yM2NhazZs1Cz549Te5fXFyMl19+GT/++CPOnTuH6OhozJw5E0OGDJF0vuqhnKCgIAYnRERELkZuSobs4GTx4sVIS0vD3LlzkZiYiJkzZ2LgwIHIyclBWFiYwf4VFRW4/fbbERYWhiVLlqBp06Y4fvw4QkJC5J6aiIiI6gCN3FWJExMT0aNHD3z00UcAriWrRkVF4f/+7//w4osvGuw/d+5cTJ8+Hfv374e3t7dVjSwtLUVwcDBKSkrYc0JEROQirH1+y5pKXFFRgaysLCQnJ984gIcHkpOTkZmZafQ9v/76K5KSkjBu3DiEh4ejc+fOeOutt1BVVWXyPOXl5SgtLdX7ISIiorpBVnBSVFSEqqoqhIeH620PDw9Hfn6+0fccOXIES5YsQVVVFZYtW4ZXX30V7733Hv773/+aPM/UqVMRHBys++FMHSIiorrD7kXYtFotwsLC8L///Q/x8fFISUnByy+/jLlz55p8T3p6OkpKSnQ/J06csHcziYiISCVkJcSGhobC09MTBQUFetsLCgoQERFh9D2RkZHw9vaGp6enbluHDh2Qn5+PiooK+Pj4GLzH19cXvr6+cppGREREbkJWz4mPjw/i4+ORkZGh26bVapGRkYGkpCSj77n55ptx6NAhaLVa3bYDBw4gMjLSaGBCREREdZvsYZ20tDTMmzcPn3/+Ofbt24exY8eirKwMqampAIBRo0YhPT1dt//YsWNx7tw5TJgwAQcOHMDSpUvx1ltvYdy4ccpdBREREbkN2XVOUlJScObMGUyaNAn5+fmIi4vD8uXLdUmyubm5evXzo6KisGLFCjzzzDPo2rUrmjZtigkTJmDixInKXQURERG5Ddl1TpyBdU6IiIhcj0PqnBARERHZG4MTIiIiUpU6HZzsyyvFp+uO4GqV1vLORERE5BBWrUrsLgZ/sA4A4OmhQerNLZzcGiIiIgLqeM9JtT2nuXYPERGRWjA4ISIiIlVhcEJERESqwuAEQJemwc5uAhEREV1Xp4OT7s1DAAARwX7ObQgRERHp1OnghIiIiNSHwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIiIiFSFwQkAIZzdAiIiIqpWp4MTjUbj7CYQERFRLXU6OCEiIiL1YXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLgxAaVVVq8sGQHfsk+5eymEBERuQ0GJzZYknUS3209iQmLsp3dFCIiIrfB4MQG58oqnN0ElyaEcHYTiIhIhep0cFJWfhUAH5LOcKWyCv3fW4vnv9/h7KYQEZHK1OngZH/+BQDA4q0nFDtm5uGz2HioSLHjuasVe/JxpKgM32eddHZTVCnz8FkM+2g9dp4sdnZTiIgcrk4HJ9XW5JxR5DhXKqswYt4mPPDpZly83itDxrGzyrwR8zZh58kSjPx0s7ObQkTkcAxOFFReqdX9+RKDE1LAhSv8d0REdQ+DEyO0WoE5aw5jy9Fzzm4KkcuZvfoQJv2ym7lcRGQ1L2c3QI1+23kaby/fDwA4Nm2ok1tD5Fqmr8gBAIxMjEa7iEAnt4aIXBF7Tow4cqZM9nvevf6BTETXXKmscnYTiMhFMThRyEerD+Fw0UWnnFsIgYcXbMHITzfZpSv9+60n8NfeAqOvVWkFnlmcjS83HVf8vK5i/vqjWLQl19nNICJyGxzWUdBDTppZcaH8qm7GUV7JFTQJ8Vfs2CfOXcLzS3YCMD7E9cfuPPy0/RR+2n4KD90Urdh5XUV+yRW8/vteAMB9CVHw8NA4uUVERK6PPScWfPeP9BooZRX278ZenVOImSsPOCzZ0FIV3Lo+m4RTxomIlMfgxIiaj/0XftipqrHz1AX/YObKg1ixJ9/ZTSEVqKzSWt7JiCqtwKUKBlZEpE5WBSezZ89GTEwM/Pz8kJiYiC1btpjcd+HChdBoNHo/fn5+VjfYnq5UVuF08WWD7VVadUyJPHHuku7Pp4uvOLEl5EhXTQQgW4+dQ5uX/8DctYdlH/PO2RvQcdIKnOf6UESkQrKDk8WLFyMtLQ2TJ0/Gtm3bEBsbi4EDB6KwsNDke4KCgpCXl6f7OX5cncmT7V9djl7TVuHDjIPObopRA97/29lNICfoMGk5so6fN9j+4o+7AADT/tgv+5i7TpUAANYeUKY6sjEapt8QkZVkByczZszAmDFjkJqaio4dO2Lu3LkICAjA/PnzTb5Ho9EgIiJC9xMeHm5To+2hsNS+PRG/7zyNiUt2ouKqdd3wAHDZycNLT3y5FQ99tpnFtRysskog/cedzm4GqYip3jQidyErOKmoqEBWVhaSk5NvHMDDA8nJycjMzDT5vosXLyI6OhpRUVEYPnw49uzZY/Y85eXlKC0t1fuxt49WH7Lr8cd/sx2Lt56QtcigWoaTqq3YU4B1B4tw8vxllJVfReqCLfhOwUUTXZO67pGrKLlcqTdMSdL9tbcArV/+A0u4aCa5MVnBSVFREaqqqgx6PsLDw5GfbzxBs127dpg/fz5++eUXfPXVV9BqtejVqxdOnjT9H2vq1KkIDg7W/URFRclpplXMdQb0nb4GZy+WK3IeqcdZsOEo2r/6B/45dg5CCLv37Mj12fqjWJ1zBttzi3XbNh85i+EOWEn3ckUV9uWVqq4Hh8MY0sW+9id6v7MaJ88zQJFrzBdbAQDPfb/DyS0hsh+7z9ZJSkrCqFGjEBcXh759++LHH39E48aN8cknn5h8T3p6OkpKSnQ/J07Y/9u5MPMNuOhiudGkQ3s+jF77bS8qqwSe/W4H/jVrPXq+leGwc0tRernSYFvK/zZhx8kSjPjfJrue+545GzH4g3VYvrtuz1hSW3BWpRWyg3hjuTRERLKKsIWGhsLT0xMFBfrVQgsKChARESHpGN7e3ujWrRsOHTI9jOLr6wtfX185TbPZV5vMV/h05hDvntP2H9ZSUnW9l++2noCvlweGxzVV9Ph78679Pn7YdgqDu0Qqemy1UlkcYtTo+Vuw/lCRs5tBRG5AVs+Jj48P4uPjkZFx41u8VqtFRkYGkpKSJB2jqqoKu3btQmSkez9UXOBZovNF5jHM+OuAoscsLL2CF5bsxIRF2UZrcSjV81NyudJkvY7yq45NIL6qshwhR6sdmGjAcS4iso7sYZ20tDTMmzcPn3/+Ofbt24exY8eirKwMqampAIBRo0YhPT1dt//rr7+OP//8E0eOHMG2bdvw4IMP4vjx43jssceUuwoHuFhuOIzhTPk1clBsfQRM+mUPPsw4iMNnlFsbaFGNyrrWJvZmHT+HMV9sNZk4WXq5ErGv/YmOk1YYvLZ8dz7avbIcCzYctercUtXs0WCCIhGRMmQHJykpKXj33XcxadIkxMXFITs7G8uXL9clyebm5iIvL0+3//nz5zFmzBh06NABQ4YMQWlpKTZu3IiOHTsqdxUO8N1W/QfP+bIKzMpQbobP+oNFeO77HSi9Ii0I+mTtEd10Qlu+r9cMHMoklmKX0uthLCdFrnvmZOKvvQUY/802o69vOXbO5Hv/8+12ANdydxwlr8SxSctK9dOUlV/FJ2sP41iR/NW4baVxdvIUEamSVQv/jR8/HuPHjzf62po1a/T+/v777+P999+35jSq9uz3OyTXHTG1om9ND352bdHA+r5emDKsk6TjPvTZFnz7+E2S9jVl6IfrbHq/I5w8b1i115nySi4jIsjPbR6s0/7Yjy83Hce7f+bg4JtD7HKOsvKrCPDxdJvfGRHZF9fWsdIGGYl/1VP/pJDzIM48ctbs63/sysMv2afM7rM//4Lk89VFVyqrkHX8nK6H6bt/TiBp6iq8+stuAICSaSbfbT2B295bY7QHw57ZLJuPXvt3VFlln7McKLiATpNXYJyJHjAiotoYnLipxf/kYuzX2zBhUbZN66eYelzVldTPcV9vwz1zMvH04mxUaQXeWZED4Mbsrh+26Q/3CSHw+87TVuXvvLBkJ46cKcMrP+82eO1QoXL5QI62YMMxAMCyXXV76jcRScfgxE1N/GGX7s+Xag0/CSGMltGvTu7Uaq+9nnX8PO6cvcGu7XQ0IQQ+W38U6w9K6/nK2H9tzajfdpzG80sMi179mn1a7+8r9xVi/Dfb0f+9tbptX206jsEfrJNcSM/WWUYr9uRj0My/caBAfq/YC0t2YLOFHjmppIzgcJCHiIxhcOJgQgAVV7VYubfAaQWonvwqCx0mLceZC4YFs65WadHypWVo+8ofGD3f+GrTUvIGau/y5558vL18P7QOmm5bYaIwzcbDZ/HG73t1OT5y/LjNcIhMr3ifEMg+YXhPX/l5N/bllWL69V4XxZj4VT7xZRb2518wmUhcU+18qO+2nkSKFUX08kqkD0e6ci8QETmGVQmxZL1P1x3BBzVWPT42bahNxzt+tgwN6vnIes+KPdceSD9vN3zY/rbzRk/ARROzd6R826292u3jX2YBADo3CcbQrpEovqT81Oyc/AuY8usePDugrcl9nFku3dELN5aVWz7f0l15aBLsZ/O5Vhip1jvxh5344P5uettOnLuE5BlrDfYlIqqJPSd2MvTDdUaXo6+uniqVqQCh2r9mrbd4jJz8UqT/uAv5JVcsrop8vkyZoOFAwY1vxzVrgVTXZ5n8643FH5VaYfXhBVuQeeQs7p1rehHK2r7IPGYwxFNyqRID3/8bs2oEkVKoNQ9n58liPPV1Fo6fNT5V+NwleTlJh89cxMs/7cJGC0nhe06X4t65G2u1pUTWuYiobmLPiQxPL9qONuGBGHdra4v7Fl2swOj5W2zuGTlnIZn1whXD4GXdQf2g6JGF12YLHSy4gJwaeQhaldREb//qcjzepyVeGNTepuPky1wcccvRc5j0y7UgqeZ9+mz9EeQUXEDOX86ZyWTqtkz5dQ/6tWuMfu3Cru1nZJ99eYZLHQz76Fre0JEzZVj+dB+D169UygsOq/Npvt6ca/Hft6UeMs4sJiJj2HMiw8/Zp3V5A2r6UK3dlIc+M54rkpV7Xi+YmfrHfqvON3/9UVlrvdTMy7hiZGjjqlbg4zWGCyvKZapNfxvpwQKAU8XGh3gq7DSl1lYLNx7Dwwv+0f3d2MJ/5pJgj5noOSEiUhv2nFgh6/h52d82pXPegzF14T8We2oA4NP1R9EmrL5V55i+IsfsrJWKq1q8uXQvujYL0W2r/o3klVzGhG+zZZ9z1Pwt2P/GIPh5e+ptL7pw41oLL1zB2K+2YWRic1nHrrl+zKxVylQM3sqVeomc5lLFVby1bB8Gd47Eza1Dnd2cOos9J1Z4eIHxnglr/Lz9FG57b43ettyz1idtmstRsdTbISUwqZZrYr0bKT7PPG7ytW+35OLzzON49nvDabuTf9ljtmS9OeVGgsk3l+3T/Xnasv3IOn4ead8ZnteUQ4UXZQ8lEck17Y/9SJqaYVO9IpLu49WH8dWmXIz8VP6MPlIOgxMrGMvzsNbTi7Nx5MyN7vaT5y+jz/TVso5RM+Z45SfDAl6u5LSZKaklZtbrsVQJt5qpaaylNe7p3LXShpjkzjr5fWee5PWLnO3TdUdk7c+y9PZxqeIq5q49jLySK4r1zJF5tnzxIuUwOLEzuSW7bS0nb23PgqubsChb0n7z1tm2SnHRRcPaMHIoWetEygCgtTOh/rt0n+WdFKCxMDF9x4li/FCHV3vOr7GYpKnA9mqVFr/vPI0CF+nFu1xRxV4gsog5J3a2dGee5Z1ckDpTRk3bdaoEK/dZXoDR3rbl6ueTCCFQeuUqgv29ZR0nY18BjtcY/nv159148KZovX20QqDfu2usbisAfLL2MNYdLMKnoxMMcnYcYfj1CsVNQvyR1KqRw8/vChZuPIb/Lt2HQD8v7Joy0NnNsSj+v3/hUkUVtr96u+waTVR3sOeE7E4NM5Yf/GwzFm48ZvJ1Z02rfmZxNmJf+xNbZfZ4Pfq5/mKSX246jsEf/K23La/kis0rOk/9Yz/WHyoyWEPI0Y4UsaqsKatzri2xoORwsz1dul7radcp96t5s/bAGTy9aLvJIehDhReReViZ5SHcHXtO3IAtCbSOIPWxv3DDURSUGHZNK1WkzZxV19fQcbSfr6/NI6dwnClaAbxpZjhGq73WU2NNfkj17LRimQXbSH2qtAIeGuYJ2UP1kh9B/t54fXhng9er89Qynu2LVo2tm/Eox2frj6LwwhWkD+5g93MpjT0nbkBKlVilWao0a40pv+3VPaxrKnWRb4S1xby4FDEvLnXoOQuNrJdUraJKixbpy2w6/vzrKwzL8e2WXJOv/bT9FF79eTeqHLTmkrXOl1Vg9upDOF1sW0+Us5VeqUSPN1fi/77d7uymuLU8I1+yajpYYNgTuPnIWQz5YJ2ia6698ftefLL2CA5asQioszE4IbtTashE3Y8v+d5ebl0RvF0OKgG/Yo/hejnGCr8B1hclXLmvAF9uOo7lRtbmUZO077IxfUUO7rdiUURLdp8qQeqCLdifb1jdtyYlOjqW7szDubIK/O4iuXCr9xdi7FdZDkugrV2K4WDBBQz/aD3+9/dhzMo4iEsV9vuilPK/TdibZ7jkgxIuyVw2RQ04rEN213XKn85ugirNsbIq7h0fOaan7InrizUC0hZ7lMLU6sVy1/dxtHXX11+yxzTTuz/eiIoqLbblFmPH5AGKH1+tpARbqQuvVUQO8vPG2/d2tWt75qw5bPCF4amvt+Fg4UXsuP6F4GxZBaYM62TXdqghR08N2HNCLoMj5Op1SmLirSOmKBeWXsGSrJNGl0pQo4rrOVXm6vjUdYUX7D9N2lhPZnGte5J9otju7XAGIYTJXlFnYXBC5GBarZBcNM4e3v/rgOz3VA/NGRuiK7xwBZ/8La9omwEjxz1Ro5dCzufm0Fnr8dz3O/CBzFWlzbkqMyfm+NkyvPTTLhwrct56Rhn7CnDztFXYfISzQ8i0Kq3AnbM34JGF/1je2YEYnBA52Jy1hyUXjbMHax7a/126D//9fa/BUNRvO06j55sZNrdpmpFFKP9t5QymM9eTglcrNAPLmkBy5Keb8c3mXDwwT/kcFake/XwrThVfxgMsw05mHCy8gB0nS7A6x/gCqc7C4IRcxnmV5yVI9fFq1yxD/un6o6jdgfDOCuuSemsrM5KwV3Pdon+cUPm4skqL3adK8Ppve2W/t7q+zGkLszbksDYh1l4zoU6ev4QNh4qsfr+l6sDmZgQWX6rAbe+uwYy/DuBoURk+33gM5VetG8bLOn4e6w5KezBLHfrYebJYL8H5xLlLWLVf2SKQV6u0iiQKq2w0R4cJseQSDp+5iANGpt9R3fBL9ml8cH83h57z6cXZblvhWQm3vH1tDbDvnkhCzxYNFT/+E19uNfnaZ+uP4khRGT7MOIgPr/cEll6uxP/1b6O335XKKmzPLUZCTAN4e974Ln7i3CU8vTgbj/dpqZf4bUlllbQneUFpOQbNXIdj04YCAHq/U3u9NNsjgrvnbMTOkyVY9WxftHRAzRRHY88JuYQ7r5cxJ3U5cc41636UXqlE2uJsrMkxPfSjvsBEPSnhNXsQlKzLUZO5YQZjvUFbjbQj7btsjJi3CVOX6ffwvbBkJ7KOn5cVmKjNzusziN79U7n1utSEwQm5BFcpzU32s+WockM7M/86iB+3n8LDCwyTAK9Wae1aCfd8WYXJmTm1hw1U2uOOn52Y0C1FYekV7DpZgmW7rtXPmb9Bf8HP2rNwrJWxr8BgvSxHq77G2qQOQXFYh4hU+0GgFluPnUN4kB+iGgYYvPbiDzux6rl+Jt+76chZbD4iLYAxV+m12xt/2S0YvlJZhW5v/AUAOPLWEHh46PeG9Jq2CqOSYjC2XyvJxzRWbdQe8kouw1OjQViQH/7ccyN/YknWCYzt1woVV7Xw8ZL+fdee1fN7vmV7krYxAjeKIAb7e+vWuKoevlGLz9YfxWwXzW2rxp4TIge6cKXSaPInAfvySnHv3Ewj4/Pm7TpZgvUHi3D//zbh/ZXmp0lvzz2PT9cdMVu12J69dGdqLC9QYWTNqLySK7IrB9desiD37CXJNV4KS68gdcEWZFhYsftyRRWSpq5Cz7cyDIZUDp8pw5Rf96DtK38gJ98+ZdJr3y1HLQt0ptbvtryyCnd8tB53fLQeh+28GOWxojL8fcC6GTRv/L4X5xxUVdde2HNC5EDHVL5IozNZKstfu7R4NTkVc+/62LA0+MXyq6jv64VLFVex8ZBtNUFOnLskq/dADikP5H+OncO/52aiXXigpGNO+W0PVuecweqcM2a//dd8SFcaCaqqV/z+cNVBzH6gu6Rz17T12DnMXXsEk+/oaLTXTEm2FBsrq1G+/lK5bV8yKqu0WJNzBj1jGiI4wNvg9X7vrgEA/DC2F+KjG9h0LlfEnhMiUp25aw1L+5tb1NAYqavuLrq+MOF/vs3GY1+YniFiycXyq+j9zmok2mlIQcrV/LjtWi5IjomF3mo/l2v3DOjvK3Dru2tw+/WVdO3p3rmZWLmvAOO/2WbTcbRaYTZBd8qve3DL26tRqoJqvB+sPIgxX2zF/RZq4ew8Waz39w2HinDyvLwvOTNXHkC+gtPaHYHBCRGpQs06NsaKslUruliOuz7egMX/mF7tWI4qrcCSrJNYaWFow5J8E+sG2dP3W0/Y7di7T5XiaFEZDhZexOEz+kMY5uK+rcfOGR3eqT3UZOwQJyUug2CqRsqXm47jnjmmF85buPEYThVfxikHri59qNDwd1FRJfDR9ZyQfXnmF3ysacvRcxj56WbdNG6pVuecwcMLthh9Tag07ZrDOkSkClPNBCQ1vbsiB9tzi7E9txgpPZqb3K9Kq8Wfe/IRH90Ajer74vnvdxjdb/PRc1ilUDVZpVkagXh+yU7Fz/n3gTOYu/Yw7o1vptsm9Zt6YekV3Hu9sm/NYaK9p0sx5MN1GJlo+n5JMf6bbRACiAk1HPrRaIBvtygTsEoxTmIvz0s/7TbYNk/Gcg+v/bYXxZcq8cztbbFsl/XT2/fXChgrq7Q2D03ZE3tOiMilmMo9qe1AwUU8/mUWhn10rUbO91knje5Xu1dATVbsMT5N1J5Gzd+CjYfPIu0748GcOf8cMz6kMmvVtUJpX2++ETw88OlmbDwsvcJsyaVK/L4zD0t35aHognOSPS1VtTXGWE2WXafM51fV9kHGQVyquKrL7TFH6v+P22esRezrf6KwVN5wqaMwOCEit+bILnylHSq8ETjZMkNFCIGNh4vw0k+7FGiVch6YJ33dn5rDD84ainDmEMgHK6WtiWUuj6im6uT8tVbOCLI3BidE5DIGf7AOR61Y6dfce45bMYPqyJmLeHv5fknTNf89dyMKSpVJRjRVvM2Sbm/8ZTEQWJJ10mRCau1Vma3pQbDFyfOX9R6idbFe0AETSc4AcOu7a8xWO65298cbMMNFKsoy54SIXIac5MGa/idjjF+KIR+uw5VKLQ4WXMS0e7pgxZ58/L7DeD7AP8fOo9/0Ndj3xiDJxz9QcAHPfrcDueduBE5VWoHu1wu4mWIq56L4kuWg5jkTOTkAsPgf+yXe1qz8ayrmOFR4UW8l7192nDa6X82pvo5mrwUWpThaVIaHF/yDY9OGmg1gt+UWY1tuMSJD/B3YOuswOCEil2VLzQpbXKm8Vutje+553D5jLc5bePhfvj5T5c+90mYEjft6Gw4W6ufCfLvFfgGCJTWDJKnDS1VaAc/rFXDN3ab7PsmU3R5jKxYfKypz2lpP6T/uMhoYniq+bNXaQ3ISZmt75WfLQ3fpP6preM8YDusQkcuSWsvkqpHCYUqoqNJaDExqeuP3vbo/t391ObQmvm1bO3wjx3t/5lg/PCLh197qpWX4IvOYlSeQz5kFDk31WN08bZVVx3tz2T6r3nf4zEXsPmVd76LaMDghIrdnaqaOrWwtdb9st/GhILkF56wxa9Uhoyv5SiE142TSL3sQ8+JSyce1Z09YmcRZLOYUqHRmS7X+79m/YJ6jMDghIpLhxDnlvqFLLTqmJvPXH0O5kWEVpZReqcQjCw1Xi7ZVp8krFD8m2Q9zToiIZJC7MGG13TJrW6hJzQ4NuQsTAvKm4L6zfL9qi+LZy8INRy3uI3UIUy5n5W1Zwp4TInJZp12ohsm/ZklfoFBtbK3vkbFPerDx1SbHVXlViym/7bW4j6uvMiwXgxMiclm9rEw4VAvHVguxnq1frmvXSTF5HttO49ayTxTb5bhHVbpSOoMTIiIn2XD4rLObQHXc36wQS0RENan1wVCbndIdDEgpFkd1A4MTIiIiUhUGJ0REZFZ1RVwiR2FwQkRERKrC4ISIiIhUhcEJERERqQqDEyIiIlIVBidERESE8qtVzm6CDoMTIiIiwtoc9dTdYXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERITyq+pZpoDBCREREeFAwQVnN0GHwQkRERGpCoMTIiIiUhWrgpPZs2cjJiYGfn5+SExMxJYtWyS9b9GiRdBoNLjzzjutOS0RERHZicbZDahBdnCyePFipKWlYfLkydi2bRtiY2MxcOBAFBYWmn3fsWPH8Nxzz6F3795WN5aIiIjcn+zgZMaMGRgzZgxSU1PRsWNHzJ07FwEBAZg/f77J91RVVWHkyJF47bXX0LJlS5saTERERO5NVnBSUVGBrKwsJCcn3ziAhweSk5ORmZlp8n2vv/46wsLC8Oijj0o6T3l5OUpLS/V+iIiIqG6QFZwUFRWhqqoK4eHhetvDw8ORn59v9D3r16/HZ599hnnz5kk+z9SpUxEcHKz7iYqKktNMIiIikkujnqwTu87WuXDhAh566CHMmzcPoaGhkt+Xnp6OkpIS3c+JEyfs2EoiIiJST2gCeMnZOTQ0FJ6enigoKNDbXlBQgIiICIP9Dx8+jGPHjuGOO+7QbdNqr1Wg8/LyQk5ODlq1amXwPl9fX/j6+sppGhEREbkJWT0nPj4+iI+PR0ZGhm6bVqtFRkYGkpKSDPZv3749du3ahezsbN3PsGHDcOuttyI7O5vDNURERGRAVs8JAKSlpWH06NFISEhAz549MXPmTJSVlSE1NRUAMGrUKDRt2hRTp06Fn58fOnfurPf+kJAQADDYTkRERARYEZykpKTgzJkzmDRpEvLz8xEXF4fly5frkmRzc3Ph4cHCs0RERK5ERfmw0AghhLMbYUlpaSmCg4NRUlKCoKAgxY4b8+JSxY5FRETkyp5JbosJyW0UPaa1z292cRAREZGqMDghIiIiVWFwQkRERKrKOWFwQkRERKrC4ISIiIhUhcEJERERqap8PYMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiLWOSEiIiJ10agoOmFwQkRERFDTOsAMToiIiEhVGJwQERGRqjA4ISIiIuacEBEREZnC4ISIiIhUhcEJERERqQqDEyIiIlIVBidERETECrFEREREpjA4ISIiIlVhcEJERETQQD3jOgxOiIiIiDknRERERKYwOCEiIiJVYXBCREREqO/r5ewm6DA4ISIiIrQND3R2E3QYnBAREZGqMDghIiIiztYhIiIiMoXBCREREakKgxMiIiJSFQYnREREpCoMToiIiEhVGJwQERGRqjA4ISIiIhWtSczghIiIiFSGwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERERf+IyIiInURwtktuIHBCREREakKgxMiIiKClj0nRERERMYxOCEiIiImxBIREZG6qCg2YXBCRERE6sLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIiIU4mJiIiITGFwQkRERKrC4ISIiIhUxargZPbs2YiJiYGfnx8SExOxZcsWk/v++OOPSEhIQEhICOrVq4e4uDh8+eWXVjeYiIiI3Jvs4GTx4sVIS0vD5MmTsW3bNsTGxmLgwIEoLCw0un/Dhg3x8ssvIzMzEzt37kRqaipSU1OxYsUKmxtPRERE7kd2cDJjxgyMGTMGqamp6NixI+bOnYuAgADMnz/f6P79+vXDXXfdhQ4dOqBVq1aYMGECunbtivXr19vceCIiInI/soKTiooKZGVlITk5+cYBPDyQnJyMzMxMi+8XQiAjIwM5OTno06ePyf3Ky8tRWlqq90NERET2pJ65xLKCk6KiIlRVVSE8PFxve3h4OPLz802+r6SkBPXr14ePjw+GDh2KWbNm4fbbbze5/9SpUxEcHKz7iYqKktNMIiIicmEOma0TGBiI7Oxs/PPPP3jzzTeRlpaGNWvWmNw/PT0dJSUlup8TJ044oplERESkAl5ydg4NDYWnpycKCgr0thcUFCAiIsLk+zw8PNC6dWsAQFxcHPbt24epU6eiX79+Rvf39fWFr6+vnKYRERGRm5DVc+Lj44P4+HhkZGTotmm1WmRkZCApKUnycbRaLcrLy+WcmoiIiOoIWT0nAJCWlobRo0cjISEBPXv2xMyZM1FWVobU1FQAwKhRo9C0aVNMnToVwLX8kYSEBLRq1Qrl5eVYtmwZvvzyS8yZM0fZKyEiIiK3IDs4SUlJwZkzZzBp0iTk5+cjLi4Oy5cv1yXJ5ubmwsPjRodMWVkZnnrqKZw8eRL+/v5o3749vvrqK6SkpCh3FURERGQTNS38pxFCCGc3wpLS0lIEBwejpKQEQUFBih33/77djt92nFbseERERK7qnXu64r4eys6Otfb5XafX1pl+b1dnN4GIiEgVLpZfdXYTdOp0cOLn7ensJhAREVEtdTo4ISIiIvVhcEJERESqwuCEiIiIVDVbh8EJERERqQqDEyIiIlLRmsQMToiIiAiAmoqeMTghIiIiVWFwQkRERKrC4ISIiIiYc0JERERkCoMTIiIiUhUGJ0RERASNiqqw1fngZFN6f2c3gYiIiGqo88FJRLCfs5tARERENdT54ISIiIjUhcEJERERceE/IiIiIlMYnNQSEuDt7CYQERHVaQxOahmZ2NzZTSAiInI4FY3qMDghIiIidWFwQkRERKrC4ISIiIhUNV2HwYkRXz7a09lNICIiqrMYnBjRu01jZzeBiIiozmJwQkRERJyto2atGtcHANzeMdzJLSEiIqqbGJzUcmdcUwDA+ylxeLhXjHMbQ0REVAcxOKnFw+Nax1Z9Xy88eksLJ7eGiIjIMVQ0WYfBCREREakLgxMiIiJSFQYnZqipi4uIiKiuYHBCRERE0KhoMjGDEyIiIlIVBidERESkqlQGBidERESkKgxOaoiPbqD394ggPye1hIiIqO5icFJDbLMQvb97efLXQ0RE5Gh8+tbQsnE9ZzeBiIiozvNydgPU4IexSdhw6Czu7xHl7KYQERHVeQxOAMRHN0R8dENnN4OIiIjAYR0iIiICVFSCjcGJLK3D6ju7CURERG6PwYlEzw9shxVP93F2M4iIiNwegxOJohoGwNNDTZ1eRERE7onBCREREbF8PREREZEpDE6IiIgIGhXN12FwIpF6bhkREZF7Y3DiAE/0bYkNL97m7GYQERG5BAYndhYS4I30wR3QNMRfb3tMowAAwJAuEXZvg5qSnOR69JYWzm4CERE5GIMTB3rrri66Py9/ug9WPdsXKT2aO7FF6teovo+zm0BERA7G4MSBhnaNhK+XB25pHQo/b0+0bOyYirMu3HGiqgQtIiK3pqKPWwYnCnr7ni5mXw/298auKQPx5aM9dduEEPZuFhERkUUqik0YnFirZWg9g22+Xp4W3+fj5QGNnZNA3KmSrSvnyxARkXUYnFipTbh6FwFcP/FWvb978AlPKhAS4O3sJhCRi2Bw4oa8PPRvK2MT1xHszwc4ERGDE4XcG99M7+9xUSEAgDvjmjqhNUB8dAOnnFdpUuOquQ92t2s76poEO/z7YXoVEUnF4EQh/+oaqff3z1N7Ys7I7nhxcHuHt6W+r5dNibb7Xh+kYGscQ0q+D0nXsJ4PhnaJtLyjDHd1c06gTkSux6rgZPbs2YiJiYGfnx8SExOxZcsWk/vOmzcPvXv3RoMGDdCgQQMkJyeb3V/tqoupDbHwwR0c4I3BXSLh5+24h+Yv427Gz+Nuhr+P/jnlTset/X5nsseQlLFkZjJC4d/9E31bKntAIlKUvSdryCE7OFm8eDHS0tIwefJkbNu2DbGxsRg4cCAKCwuN7r9mzRqMGDECq1evRmZmJqKiojBgwACcOnXK5sY7UvUMmD+e7o0fxiZhWGwTg30CrHioW+rf6BEjvXs9NipEN5zkLnq1ClX8mKue66f4MV1Fr1aNnHbuer5eTjs3EVmmntDEiuBkxowZGDNmDFJTU9GxY0fMnTsXAQEBmD9/vtH9v/76azz11FOIi4tD+/bt8emnn0Kr1SIjI8PmxjvCw71i0LVZMPp3CAMABPl5Iz66odEIs3+HcPyrayReGdrB0c10W52bBkvbUU3/q2xg7y8ugzpH4J+Xk+17EiNaNa6HID8m+xKRNLK+ylRUVCArKwvp6em6bR4eHkhOTkZmZqakY1y6dAmVlZVo2LChyX3Ky8tRXl6u+3tpaamcZipqyrBOkvf19NDgowfUl5h5f88ofJF5HADQIMAb5y9VOrlF0tyX0MzyTiRb40Bfh5+T09mJSA5ZPSdFRUWoqqpCeHi43vbw8HDk5+dLOsbEiRPRpEkTJCeb/vY2depUBAcH636ioqLkNNOtKPEgGd0rBsPjmmDGfbFYmNrT8hts9NKQ9vjpqV42H6d7c8MhrTAzv48dkwdIDmgWpPawul2uYt6oBGc3gYjIKg6drTNt2jQsWrQIP/30E/z8/Ezul56ejpKSEt3PiRMnHNhK9YgI8sOUO6T33NRUM5fFx9MDH9zfDXd3b4ZYiTkpd1+fWTH/YfkPOA006GYksJDjnXu64t8JhkGpl4nqtx4aDYL9vdGnbWNJx7+1XZhN7XMmKbVQvD01uL1juMX9nM3HkxMGiciQrE+G0NBQeHp6oqCgQG97QUEBIiIizL733XffxbRp0/Dnn3+ia9euZvf19fVFUFCQ3k9d9NnDCQgLMh3E1RQZbHo/a3rU37svFttfvR23tZf/gBPXQ6NZI7qZ3KdDpPl7el+PKMll+OOiQnCzExM9HW3Vs30Ntu2YNEDWMW5qaXpYFQC6K1znxFTid9920oJJIrI/NY2+ygpOfHx8EB8fr5fMWp3cmpSUZPJ977zzDt544w0sX74cCQnsatZjZrqOqVIlD97U3GDbX2mGDyxTFj9+E0YmGh7joZui8f2T1+6jRqNBg3o+ko9pzB1GZjRV++mpXljxdB90bSYx4fW62r+SlqH18PO4m+El8Rv4E33UPZ3V38qp58G1SsPXrhIs16O3tLDp/UTkelw2OAGAtLQ0zJs3D59//jn27duHsWPHoqysDKmpqQCAUaNG6SXMvv3223j11Vcxf/58xMTEID8/H/n5+bh48aJyV1HHvD6sM1Y/10+vqFV9GdM0E1s2wpt3Ga6g/MadndEjxvw3aqX4eXuiXUSg7QeS+Z8pRuU1Tny8lBnmsHXxR28OtxCRE8kuPJCSkoIzZ85g0qRJyM/PR1xcHJYvX65Lks3NzYVHjW9tc+bMQUVFBe69916940yePBlTpkyxrfV1UGSwHzw8NGgRWs/sc7mulgo3d92jk6Lx73jOAFITN1pAm4gUZFVVpPHjx2P8+PFGX1uzZo3e348dO2bNKciEPyb0tstx/byt/6b82egEbMs9j9mrD8t+ryODqNeGdza6vWmIP04VX5Z0jD2vDUTRxXL8tP0UZq48qGTz6qQ7Yptg1f5CVFbV0WiaiIxi362LCQmQlgfiyI/6/h3C0aie42tnAMrUXpMzJFbP1wvRjerh6eS2ZvezthBfPR9lqqg+3CtGkeNYQ0pvSMvQepj7YHfF1+8hIvfA4EQhSs9uUJJS6yV8aGb2TU0tQ+tLPqaaErCU9Fhv+Ym37SMC8eGIOIv7WQo8k1o2wjO3XwueJv2ro95r/dradwp1x8ggNDASQNdeiDKpVSMM6hypqrU8iOq6rs1CnN0EHQYnCkhJiHL50txShlcCaswkMVVvBICu1L8jjLEiCHC01JtjJO23/Ok+aB1me5Jwj5gGuoTYR2rMunllaAc0bxRg8f2BXAOHqE5q1Vj6F0t7Y3CigABf61fxrT0F1FWYi2UsfRuOkfCANKZjrdooK9P6IqWHfqG20PrOGV4yp6eDZkBZIqVmzsyUOFnT0p/q18qWJhERGcXgxMm6udAqwkr1wKfebF0NjSf66veStA6rbxAI3dSyISb0b2N129Tu9/+7xWIvly31ae7s1hQRZgr61dY6zPCbllpHaobHma67Y43ebZRfMZvs6zUZa6WRczE4cTJnjrmbq+BqT52bWl/x9+7u12q7mCrDr9FodPkW7qZZA3+zqzR/OKIb7u7eFA8YKbBnDUf9y5z/sLLrHFX/G6lp7oPxmJkSp+h53jJSK4jUpZGNhSTJeRicKMBZ+SZJ5kq2S0giMVfB1RhzBcKkxljx0Q0QH31jmMNYM83N9ng6uS3mP5yALx9VbgFDtX7Tl2tYbBPMuC8Ovl7WDzPai7l/jb3bNEaCggnlM+6LM9gW5O/F5FsJesSoN7HfGoufMF25nNSNwYkNpt/bFf3aNcbjCpVE79mioayA4Z7uzfDxyO5YP/FWs/vZ+pH8f7e1xgOJzc0mS90b3wzNGwZYnMJ6W3vzybJrn++H7MnG14kR4lrl0tvah7t8ArI13PnZ6mHvamx2mFvvjvfj8T7qzSH64hF5X0hC6/uidVh9DuW4KAYnNvh3QhQWpvZEPYVmN0wc1B7tZZR09/DQYEiXSDRrYF2CqVTPDmiHt+7qovdh/Or1Oh5jel/LHwn088ba5/thio0fBNGN6jk88GgTrkAZfQv6Xl8tWYluZuHQKjaOYe/nvPv9xuoeqSuO16amGSgkHYMTlTG2IN99CeoruX53fDNsfSUZLw25UWxMSrd57XoXzvTb+Fvw+vBOuKOr/QuBff5IT2x48TZseqm/3c/lfK7XpdA23PkPsFu5QrNJ1hQVdHTP1icPxWPcrerteXI1DE5URmoFWGfQ1HrohNb3tRiQWBrDdmbXeJdmwRiVFGN0McB1L5gfKpMitP61e9mjxbUcm6Yh/lxQT6XSbm9n1+O/PMRyxWA1TINX05cHZ4lq6G/V+wZ2ijBagJCsw09KVVHug+FelSxwVzugsYW9hjPahgdi7oPd9bZFNbR9qGzDi7dh+6u32/TQyXol2eZ2kDKkVkiubdK/OmKMQnlpZH+e7phM5IIYnLipO7sZTqd0Chf5fz6oc6TidSt8vTxtqjkCAI1U8G1aEW7whbyBFQUToxr661XpJeuoPV54gsGn4hic1HE9rw853KOSnhZSD3eaesvRCvfwzr1dnd0EA3d1a4p0CcN2ctzeMVzR47kiBid1gLlnzKejEzD3we4GC8Qp5ZFa68rUfkjwoSGdrb8rUzPB/nqmD25q2RDfjrlJ9jGVmgnhinGQEsHbqmdvLBUQ1zzE5uMZ88NY96n1EdPIMD/M3u62sRfamho+nZpYX6iyJlcuQsfgpI4L8vPGoM6R8PO2XLirQb0b3doBEvYHrg2XrH2+n7XNIwWserYvvhmTiLYmpky3CQ/EoseTzBf1MyE2KgSzH7iRr8OS7tfqD0nVsnF9rEzri6l3d8H9PZSp7FtbzaKH1fpaOS3XlSgV74YG3hhaNVWZWndOIwGrNcN6Sn1p25h+G14ZqmyvjqMwOCHJfL08se3V27H91dvhJWPWSbQTvu24s0b1fGXlP7RsXB+9WikTNBgrODi0ayQ2vHgb3ryrs8suHaBkz82jMh9GrcPqY0TP5rqVpB1hRE/7BEI1KTHjrTZnzCaqeVfG9jWcKjywk/khGB8nztDz9fJEoJ9rrjLO4ERFwgKNL7impmqoDev52JzkWVOaih5m1WuytKvVwxAR5IewQF98MybRGc0y4OmhwZaXk+FrZjkBe3lpSAejXc5NQ/wxMjFaUg+cu+uoUJe8s9ladyVSxgKSrmrJk0kY2CnC2c2wyoz7Yp3dBLMYnKjA908m4X8PxZucvvp/t7VBr1aNMF2FyWBy1f7eY24hO0e7M64pfhl3M358qpfe9v4dwrDl5WTFeh/+eTkZS560LQ/A29PDpWqmhFgx08Ua1bVlrDHljht5V0OtLMzXOFD67Kq371H3woFNG1hX78Oc14dbV0G6Y6S0gK9rM/OfJ/UVquZdLSGmoaTcI7m9J0qUMrDkbhnDj87gOp9ubqxHTEMMMBN9Bwd445sxN+HfCVGSj1nzGywLA0mj0WgQGxWi2HIEpjQO9EVCjGEegFwdIq/18DhwNECSfxupaGxsMb6a5NTDsWbaZkSw5aBheNyNxEdru+LlJCCm2CnHpNorQztg8ePykpztPWgyKikGft7yf7dS83g8PTS6Hp+HayXjA8CwuCYY2CkcAxw4G0YA+CutD14a0l5y0OvtqcG+1wfhxcHtddt6KvCZYY4198WeXHMwiizy9vTAuhduhVYIl+lqt/TBzpk9+maN6I4PVx3E6KQYuxy/R0wD/LbjtOz3PZPcFj1iGmDjobP4dP1RNKrnY1CFt/aXTTk5H9b8M2gdZnn9pAb1fPC/h+Lh4+WBpTvzrDgLMHtkd8s72VnvNqEYlRRj83RUe/1/Gx7bFIu3npD1HjkLQ859KB57T5citlmIwWvenh745KEEbDxchD/3Fhi8/uBNzbH+YJH+Rr1TW/dLiW5UD4/3aYUXluyQ/B5/H0/93lE7fwlRQ4XimtQVKpGiohoGqD4ZteZD6avH1JHT4WzPD5RWSj0i2A9v3dUF7WQsFinHAz2b4517uyKxhbxvbD5e11aOfmFQe3xwfxz+mNDb6jYY+4Zrz6TIAZ0i0K+d+ZWzq9V+VnRqEmQwtbpmj9Dgzo7JTbildaiq62TIXRy0S42hXylftHy9PNGteQPZK11veak/3hjeWdZ7DM/NR6pS+Jskp2pUzwcDO4VjYKdwWSsyuwNT0xI9VFL0w8vTA/clRFldW8LHywPD45oiLEheYuQPY2/k/CQ76CEbX6sWhalp17b4eKT19YQiavwOB1lIwLT3sKSt/H3k9eTWnFnUtVkw7u9xY3hbyf8qYUF+RvNHGpvoUQg08nu+N74ZEqIb4Jlk44n+UnOSpMTfv46/WdKxXBWDE3Ko2v/pNBoNPnkoAZ88lGDXiqRhMhIV5QqycqreT2N7Wd6pDqodKNSmdMfJY7e0MOjSHt0rBhP6t0GshQRLOTQaDUYlRWP8ra3xjcxeQp8a38jnPhRvdJ83hndCcodwozk/taktT0kqjUaDafcoPzGgj4m6L/+Ob4YHb4qWfBw/b08sGdsLE5LbGH19bL/WGNo10mAtL2t0NTJs9cbwThZXcH4gsTm+elT9vdTqDrGJFLApvb9dZ4v0bNEIb9/TBfH/XSnrfXK7nekapQd1jE2N9/HywDO3t0XJ5UrsOFmi2Lm8PD3wnMRhO7keSorBQxLzj/q1C0OnJkHo3CRYdv6HtZ4b4JyyAdXDQvVM9Nisea4fmpmYmTT937EmhxGt+XdY39dLr2ihJb2uF0aU+r0tPMgP8dENsXDjMZP7vHWXumeJVWPPCbm9iGA/uyQFV9dxGNQ5wn0W6JOgegpi56bK1fNQyUiWATn5La3DlCnl7wg+Xh5Y+p/eeNuO5Qlq94T2bHGjArGpqcGv2riMhrFZPcEB3siedDuyXr3d6HtiQuvJKirpCNUrsHeIDMKfz/TBdhNtV/ScKptwoK47QmSG2v7z/DGhN74Zk2jz2htSqeX6H+4Vg2/H3IRFj7vPmi3WqH07AnzqRke01Gm9tQO77jXWDvrqsUS892/DImAjEw2nVwuJfRRv3NkZD5iofBsS4CP5C4raApW24YEICfDRy7UxJzTQPUpHqOsuECnkzbuuZd2/rMBqoQ1NTHEOCfBBr1ahigzP1EwGlvph7CyeHhoktWqkeEEro8S1lWijGhrvdpc6s6ladZE2Y8mMcvRs0RBNgv1cemE1a713X6zJIRJThnSJ0HvoN6znY7AS+heP9LSphzMhWv4MHWM+uD8OkcF+RoMne/L2NN/2u7o1RT8JVXvDAv2wMLUHvrex0KOzMTghh3i4VwxC6/vioSTpyWW1yXlkj0yMxo7JAzDGioJd1WaN6IZBnSLwpJH1NOoSXycXZ7ovIQpfPGI8gW90rxhsk9Hl/e2YmzCoUwS+t3Gl3sWP3wSNRqP3ABjTW/4Cb0ox1uNgjQCJQcfiJ5KQ1LIRfhl3s6ShGClF9kwlpZo/7g1yqqomRDdEdKMAow/7Tk2CkZne3yB4srfebW60xdhsMY1Ggw9HdEPvNqF4x0JScL92YeghoWjbKBs+j+2tbvRDktNNGdYJk/7V0aFJoMH+tiXB3hHbBHfENlGoNeoyZVgn3P3xRoy/rbXFff/Tvw02HzknaRaII9Rc9NDXy0NWD06b8ECTs13kqM6naNm4Po5NG4ryq1Xw9VI2r2l4XBP8ki2tCN6tEmuzWJIQ3RDPDWiLer5eKCu/anK/zk2D8e316rO55y4pcm5r7Zg0ABVVWln/Dny8PLD62X6qynXSAPjrmT44VXwZnZoYnyUW5OeNL2XOtIlqYDpoe314Z3yReVzW8RyFwQk5DGenGGpUzwdnyypwa3tlHi5StY8Iwq4pAyWthBta3xcrnumj6Plrf5NuFhIA4KyZ/W/w8/bE8qd7w1Oj0VXQfGVoB/x36T5Md3BXfDWlAxMA+OD+bpKDEyUGAn28PODv44nxt12bBnulsgpHiy7hh20nzb5vaJdI7DhRjAOFF/H3gTMKtESeYCtn4qnx86hNeCDaKFxjJ6lVI8s7qRCDEyInWvqf3lh38AyGxTm+h0ZKYOIo6UPao6JKazLZsvY33PYR+rM9HuvdEg8lRdslSFADR3zD79NGf4jDz9sT790XazE48fDQ4JV/dcS3W3LtEpyoJRHcVWk0GjRr4I+T5y+b3c+elZetweCEyIkigv1kLejorkICfPB+SpykfU2Vl7d3YGJN1VhTibxytQsPxPGz9hk++fOZPlj8zwk81c94blV4kC8KSsvtcm5b2bNwozFqr77rTvibJlJYAzsWfCPlHvhS1Hz4hQf54a9n+iDQz/L9XfJkEhZsOIZX/mX7bDHgWmD08M0xCL9exr53m1CcuVCO/fkXFDm2uaTWpf/pjQSZBQYNqKeTTjaNRoPp93bFxfKraBJy499eWFDdqW3kDAxOVKh78xAUlJbbbUE3so9vxiTi7T/2400XqcDoqhzZ+zzu1tb4c0++rndLaj5AQkxDJCi8xH2vVqG6P3/xSE8IAbR8aZmi5zAmtL4vmob441SxpWEBuzfFgKNinur7f/jMRd22uQ/anlhNpjE4UaEfxvaCVqgrJ4As69UqFL+Mv8XZzXAr1TVfpExFtYfGgb7Y8OJtDh8+sESj0ahqpkld1CJU3Su+uzoGJyqk0WhgoR5PnaS2hC2qG9QWmBDVBSzCRkTkoqorCye2VHYISU0GdopwdhNsEt1IenE4W7VsbP36Tmr76seeEyKV8eJwHkm09D+9UXFVC3+Z5eTlsqbX0tYep9FJ0XhxcAe7X5u9/Dr+ZuSXXLFqlpc1nuzbyqUWn7SEwQmRyozo2Rw/ZJ3C7R3Dnd0Uu5E7o6me740HlAeHWXQ8PTQu+/C2RKOxfG1q/qfQtVkIujqoqPK1QK69Y07mIAxOiFQm0M9b8YqsavHB/XFYujMPT8hcr6hRfV+8fU8X+Hh5wMfLfUej/bw9cKVS6+xmqII732eyjHefiBxmeFxT/G9UglXFrFJ6NMdd3dSxvo+9fDvmJrSPCMQ3j8lbP8WdvDi4PdqFB2Ksmy+4+fzAdrildajlHSXw9nS/R7n7XRERkQQdI6+VwB/aJdLJLbmhW/MGWP50H/Sq9dDq2CTIxDvUI8hfmY74J/u2wopn+qBBPR9FjqdW425tja9sDELTB7dH2/D6eOpWywt4WqK2yZAc1iEi1RnaNRKbDp/FYDsGDr+OvxkXrlxV9UPwjwm9setkCQZ3tm3GyoCO4fhzb4FCrTJucOdIxEYdxY4TxQavBVmoqhvbzPgqvGTeE31byR4idRXsOSGXwXUt6o6PRnTDlpeTLT7UTFnyZBL6tWtsdh8vTw9VByYA0CEyCPf1iLJ55svskd3RU+GKtbV5emgw4z79VaHfT4nFTS0b4rkBbc2+9+lk86/XFnN9em7txQrtzVPNGbhuhp/2pHqvDeuEbbnnXb7eAUlnayHChJiGGNolEmtylF8l1xV5e3pgfmoPzF59yKHDWHd1ayYtT0jmvf728Zvwa/ZppPRw7KKZ0Y0CMKRLBAJ9vV0qz+OxW1pgym97kdzBdWYAMjgh1RvdKwaje8U4uxnkYljZVV99Xy9MHOQe000jg/2dMpyh0Wjw8UjXW1NndK8YJLZs5FJ1UBicEBERuTGNRoMOkepPqq7JdfqliIiISFHV5fX7dwhzckv0seeEiIiojvr+ySSs3l+IYbFNnd0UPQxOiIhIEVENAuDr5YFAPy9wiSjXEBboh5QezZ3dDAMc1iFSueoM+0duaeHklhCZ5+PlgR2TByAzvT8TkiUad2sreHlo8Pygds5uiqqw54RI5eY82B0HCy6iQ6RjVjclsoWft/yFCEPr+dqhJa7h+YHt8UxyW3i50NRkR2BwQqRy3p4eLlG+nEiuj0d2x8nzl9CljleIZWBiiMEJERGZdXPrUHyfdRINFa6oO0RF6xqRujA4ISIisyYP64R2EYEYZOMaP0RSMTghIrfUsnE9ZzfBbdT39cJjvVs6uxlUhzA4ISK31L15A3xwfxyiGzFIIXI1DE6IyG0Nj1NXYSkikoYpwkRERKQqVgUns2fPRkxMDPz8/JCYmIgtW7aY3HfPnj245557EBMTA41Gg5kzZ1rbViIiIqoDZAcnixcvRlpaGiZPnoxt27YhNjYWAwcORGFhodH9L126hJYtW2LatGmIiGCmNxEREZknOziZMWMGxowZg9TUVHTs2BFz585FQEAA5s+fb3T/Hj16YPr06bj//vvh61t3qwASERGRNLKCk4qKCmRlZSE5OfnGATw8kJycjMzMTMUaVV5ejtLSUr0fIiIiqhtkBSdFRUWoqqpCeHi43vbw8HDk5+cr1qipU6ciODhY9xMVFaXYsYmIiEjdVDlbJz09HSUlJbqfEydOOLtJRERE5CCy6pyEhobC09MTBQUFetsLCgoUTXb19fVlfgoREVEdJavnxMfHB/Hx8cjIyNBt02q1yMjIQFJSkuKNIyIiorpHdoXYtLQ0jB49GgkJCejZsydmzpyJsrIypKamAgBGjRqFpk2bYurUqQCuJdHu3btX9+dTp04hOzsb9evXR+vWrRW8FCIiInIHsoOTlJQUnDlzBpMmTUJ+fj7i4uKwfPlyXZJsbm4uPDxudMicPn0a3bp10/393Xffxbvvvou+fftizZo1tl8BERERuRWNEEI4uxGWlJaWIjg4GCUlJQgKCnJ2c4iIiEgCa5/fqpytQ0RERHWXS6xKXN25w2JsRERErqP6uS13kMYlgpMLFy4AAIuxERERuaALFy4gODhY8v4ukXOi1Wpx+vRpBAYGQqPRKHbc0tJSREVF4cSJE26by+Lu18jrc33ufo28Ptfn7tdoz+sTQuDChQto0qSJ3mQZS1yi58TDwwPNmjWz2/GDgoLc8h9cTe5+jbw+1+fu18jrc33ufo32uj45PSbVmBBLREREqsLghIiIiFSlTgcnvr6+mDx5sluv4+Pu18jrc33ufo28Ptfn7teoxutziYRYIiIiqjvqdM8JERERqQ+DEyIiIlIVBidERESkKgxOiIiISFXqdHAye/ZsxMTEwM/PD4mJidiyZYuzm4SpU6eiR48eCAwMRFhYGO68807k5OTo7dOvXz9oNBq9nyeffFJvn9zcXAwdOhQBAQEICwvD888/j6tXr+rts2bNGnTv3h2+vr5o3bo1Fi5caNAepX9HU6ZMMWh7+/btda9fuXIF48aNQ6NGjVC/fn3cc889KCgocIlrqxYTE2NwjRqNBuPGjQPgevfv77//xh133IEmTZpAo9Hg559/1ntdCIFJkyYhMjIS/v7+SE5OxsGDB/X2OXfuHEaOHImgoCCEhITg0UcfxcWLF/X22blzJ3r37g0/Pz9ERUXhnXfeMWjL999/j/bt28PPzw9dunTBsmXLZLdFzvVVVlZi4sSJ6NKlC+rVq4cmTZpg1KhROH36tN4xjN3zadOmqeL6LF0jADz88MMG7R80aJDePq56DwEY/f+o0Wgwffp03T5qvodSngtq+uyU0haLRB21aNEi4ePjI+bPny/27NkjxowZI0JCQkRBQYFT2zVw4ECxYMECsXv3bpGdnS2GDBkimjdvLi5evKjbp2/fvmLMmDEiLy9P91NSUqJ7/erVq6Jz584iOTlZbN++XSxbtkyEhoaK9PR03T5HjhwRAQEBIi0tTezdu1fMmjVLeHp6iuXLl+v2scfvaPLkyaJTp056bT9z5ozu9SeffFJERUWJjIwMsXXrVnHTTTeJXr16ucS1VSssLNS7vr/++ksAEKtXrxZCuN79W7ZsmXj55ZfFjz/+KACIn376Se/1adOmieDgYPHzzz+LHTt2iGHDhokWLVqIy5cv6/YZNGiQiI2NFZs2bRLr1q0TrVu3FiNGjNC9XlJSIsLDw8XIkSPF7t27xbfffiv8/f3FJ598ottnw4YNwtPTU7zzzjti79694pVXXhHe3t5i165dstoi5/qKi4tFcnKyWLx4sdi/f7/IzMwUPXv2FPHx8XrHiI6OFq+//rrePa35f9aZ12fpGoUQYvTo0WLQoEF67T937pzePq56D4UQeteVl5cn5s+fLzQajTh8+LBuHzXfQynPBTV9dlpqixR1Njjp2bOnGDdunO7vVVVVokmTJmLq1KlObJWhwsJCAUCsXbtWt61v375iwoQJJt+zbNky4eHhIfLz83Xb5syZI4KCgkR5ebkQQogXXnhBdOrUSe99KSkpYuDAgbq/2+N3NHnyZBEbG2v0teLiYuHt7S2+//573bZ9+/YJACIzM1P112bKhAkTRKtWrYRWqxVCuPb9q/3Br9VqRUREhJg+fbpuW3FxsfD19RXffvutEEKIvXv3CgDin3/+0e3zxx9/CI1GI06dOiWEEOLjjz8WDRo00F2fEEJMnDhRtGvXTvf3++67TwwdOlSvPYmJieKJJ56Q3Ba512fMli1bBABx/Phx3bbo6Gjx/vvvm3yPWq5PCOPXOHr0aDF8+HCT73G3ezh8+HBx22236W1zpXtY+7mgps9OKW2Rok4O61RUVCArKwvJycm6bR4eHkhOTkZmZqYTW2aopKQEANCwYUO97V9//TVCQ0PRuXNnpKen49KlS7rXMjMz0aVLF4SHh+u2DRw4EKWlpdizZ49un5rXX71P9fXb83d08OBBNGnSBC1btsTIkSORm5sLAMjKykJlZaXeOdu3b4/mzZvrzqn2a6utoqICX331FR555BG9RStd+f7VdPToUeTn5+udJzg4GImJiXr3LCQkBAkJCbp9kpOT4eHhgc2bN+v26dOnD3x8fPSuJycnB+fPn5d0zVLaooSSkhJoNBqEhITobZ82bRoaNWqEbt26Yfr06Xrd5a5wfWvWrEFYWBjatWuHsWPH4uzZs3rtd5d7WFBQgKVLl+LRRx81eM1V7mHt54KaPjultEUKl1j4T2lFRUWoqqrSu0kAEB4ejv379zupVYa0Wi2efvpp3HzzzejcubNu+wMPPIDo6Gg0adIEO3fuxMSJE5GTk4Mff/wRAJCfn2/02qpfM7dPaWkpLl++jPPnz9vld5SYmIiFCxeiXbt2yMvLw2uvvYbevXtj9+7dyM/Ph4+Pj8GHfnh4uMV2q+HajPn5559RXFyMhx9+WLfNle9fbdXtMXaemm0NCwvTe93LywsNGzbU26dFixYGx6h+rUGDBiavueYxLLXFVleuXMHEiRMxYsQIvQXS/vOf/6B79+5o2LAhNm7ciPT0dOTl5WHGjBkucX2DBg3C3XffjRYtWuDw4cN46aWXMHjwYGRmZsLT09Ot7uHnn3+OwMBA3H333XrbXeUeGnsuqOmzU0pbpKiTwYmrGDduHHbv3o3169frbX/88cd1f+7SpQsiIyPRv39/HD58GK1atXJ0M2UZPHiw7s9du3ZFYmIioqOj8d1338Hf39+JLbOPzz77DIMHD0aTJk1021z5/tVllZWVuO+++yCEwJw5c/ReS0tL0/25a9eu8PHxwRNPPIGpU6eqqiS4Kffff7/uz126dEHXrl3RqlUrrFmzBv3793diy5Q3f/58jBw5En5+fnrbXeUemnouuJs6OawTGhoKT09Pg+zhgoICREREOKlV+saPH4/ff/8dq1evRrNmzczum5iYCAA4dOgQACAiIsLotVW/Zm6foKAg+Pv7O+x3FBISgrZt2+LQoUOIiIhARUUFiouLTZ7Tla7t+PHjWLlyJR577DGz+7ny/as+lrnzREREoLCwUO/1q1ev4ty5c4rc15qvW2qLtaoDk+PHj+Ovv/6yuKx8YmIirl69imPHjplte812O/P6amvZsiVCQ0P1/k26+j0EgHXr1iEnJ8fi/0lAnffQ1HNBTZ+dUtoiRZ0MTnx8fBAfH4+MjAzdNq1Wi4yMDCQlJTmxZdemmY0fPx4//fQTVq1aZdCNaEx2djYAIDIyEgCQlJSEXbt26X2YVH+gduzYUbdPzeuv3qf6+h31O7p48SIOHz6MyMhIxMfHw9vbW++cOTk5yM3N1Z3Tla5twYIFCAsLw9ChQ83u58r3r0WLFoiIiNA7T2lpKTZv3qx3z4qLi5GVlaXbZ9WqVdBqtbrALCkpCX///TcqKyv1rqddu3Zo0KCBpGuW0hZrVAcmBw8exMqVK9GoUSOL78nOzoaHh4duKETN12fMyZMncfbsWb1/k658D6t99tlniI+PR2xsrMV91XQPLT0X1PTZKaUtkkhOnXUzixYtEr6+vmLhwoVi79694vHHHxchISF6mczOMHbsWBEcHCzWrFmjN6Xt0qVLQgghDh06JF5//XWxdetWcfToUfHLL7+Ili1bij59+uiOUT1lbMCAASI7O1ssX75cNG7c2OiUseeff17s27dPzJ492+iUMaV/R88++6xYs2aNOHr0qNiwYYNITk4WoaGhorCwUAhxbQpa8+bNxapVq8TWrVtFUlKSSEpKcolrq6mqqko0b95cTJw4UW+7K96/CxcuiO3bt4vt27cLAGLGjBli+/btutkq06ZNEyEhIeKXX34RO3fuFMOHDzc6lbhbt25i8+bNYv369aJNmzZ601CLi4tFeHi4eOihh8Tu3bvFokWLREBAgME0TS8vL/Huu++Kffv2icmTJxudpmmpLXKur6KiQgwbNkw0a9ZMZGdn6/2frJ7hsHHjRvH++++L7OxscfjwYfHVV1+Jxo0bi1GjRqni+ixd44ULF8Rzzz0nMjMzxdGjR8XKlStF9+7dRZs2bcSVK1dc/h5WKykpEQEBAWLOnDkG71f7PbT0XBBCXZ+dltoiRZ0NToQQYtasWaJ58+bCx8dH9OzZU2zatMnZTRIAjP4sWLBACCFEbm6u6NOnj2jYsKHw9fUVrVu3Fs8//7xenQwhhDh27JgYPHiw8Pf3F6GhoeLZZ58VlZWVevusXr1axMXFCR8fH9GyZUvdOWpS+neUkpIiIiMjhY+Pj2jatKlISUkRhw4d0r1++fJl8dRTT4kGDRqIgIAAcdddd4m8vDyXuLaaVqxYIQCInJwcve2ueP9Wr15t9N/k6NGjhRDXpke++uqrIjw8XPj6+or+/fsbXPfZs2fFiBEjRP369UVQUJBITU0VFy5c0Ntnx44d4pZbbhG+vr6iadOmYtq0aQZt+e6770Tbtm2Fj4+P6NSpk1i6dKne61LaIuf6jh49avL/ZHXdmqysLJGYmCiCg4OFn5+f6NChg3jrrbf0HuzOvD5L13jp0iUxYMAA0bhxY+Ht7S2io6PFmDFjDIJYV72H1T755BPh7+8viouLDd6v9nto6bkghLo+O6W0xRLN9QsnIiIiUoU6mXNCRERE6sXghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUpX/B9AhtpTcBkp1AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "split_loss('train')\n",
    "split_loss('val')\n",
    "plt.plot(lossi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can make two observations here\n",
    "- Both training loss and validation loss are lower than previously reported\n",
    "- The \"hockey shape\" of the loss over iteration is gone as we are now initializing W2 with more reasonable values resulting in better logits values and softmax not being wrong confidently. \n",
    "This also results in model spending more iterations in actually reducing the loss rather than spending those cycles in trying to squash loss from very high values to a more reasonable one.\n",
    "\n",
    "## TanH Saturation\n",
    "The plot below shows what a tanh function looks like, as we can see the function acts as a bounding method for the input and limits value in the range of [-1, 1]. For very hight values, tanH value remains constant at 1 and for very low values tanH value remains constant at -1. For both of these scenarios gradiant is 0. This is true for other squashing functions like sigmoid and relu."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "x = np.linspace(-100, 100, 100)\n",
    "y = np.tanh(x)\n",
    "plt.plot(x, y, label='y = tang(x)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's take a look at the activations from first hidden layer i.e hpreact and the corresponding h values and plot them out to see what values are being used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.8508\n"
     ]
    }
   ],
   "source": [
    "logits, h, hpreact = run_training_loop(break_on_first=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([32, 200])"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "h.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([2125.,  153.,  117.,   76.,   65.,   51.,   44.,   60.,   20.,\n",
       "          26.,   17.,   25.,   22.,   30.,   30.,   20.,   28.,   16.,\n",
       "          19.,   18.,   29.,   24.,   14.,   12.,   25.,   29.,   16.,\n",
       "          20.,   28.,   21.,   24.,   18.,   14.,   22.,    6.,   38.,\n",
       "          27.,   29.,   49.,   17.,   22.,   16.,   29.,   44.,   44.,\n",
       "          39.,  109.,  114.,  174., 2385.]),\n",
       " array([-1.  , -0.96, -0.92, -0.88, -0.84, -0.8 , -0.76, -0.72, -0.68,\n",
       "        -0.64, -0.6 , -0.56, -0.52, -0.48, -0.44, -0.4 , -0.36, -0.32,\n",
       "        -0.28, -0.24, -0.2 , -0.16, -0.12, -0.08, -0.04,  0.  ,  0.04,\n",
       "         0.08,  0.12,  0.16,  0.2 ,  0.24,  0.28,  0.32,  0.36,  0.4 ,\n",
       "         0.44,  0.48,  0.52,  0.56,  0.6 ,  0.64,  0.68,  0.72,  0.76,\n",
       "         0.8 ,  0.84,  0.88,  0.92,  0.96,  1.  ]),\n",
       " <BarContainer object of 50 artists>)"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAGiCAYAAAD5t/y6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAo90lEQVR4nO3de3hU9Z3H8U8IZLjOhAjJkDWkXJRruIgSYhW05CFgpLiyz8qlXFoKKxvsQiwCu4gI3QWBVVuK0na52KdQxD6ILvcIDSgG0CwRCJAHaDC4OgGJZAhiIOS3f3Rz6kiAJEwuv/B+Pc88Mud8z5nfd36ZzMcz50xCjDFGAAAAFmlQ2wMAAACoLAIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALBOpQLMggUL9MADD6hFixaKjIzUE088oZycnICaRx55RCEhIQG3p59+OqAmLy9PycnJatq0qSIjIzV9+nSVlJQE1KSnp+u+++6Ty+VSx44dtXr16qp1CAAA6p1KBZjdu3crJSVF+/btU1pamq5evapBgwbp0qVLAXUTJ07UF1984dwWLVrkrLt27ZqSk5N15coVffjhh3rjjTe0evVqzZkzx6nJzc1VcnKyHn30UWVlZWnq1Kn66U9/qu3bt99muwAAoD4IuZ0/5nju3DlFRkZq9+7d6t+/v6S/HoHp1auXXn311XK32bp1qx5//HF9/vnnioqKkiQtX75cM2bM0Llz5xQWFqYZM2Zo8+bNOnLkiLPdiBEjdOHCBW3btq2qwwUAAPVEw9vZuLCwUJIUERERsHzNmjX6wx/+IK/Xq6FDh+r5559X06ZNJUkZGRmKi4tzwoskJSUlafLkycrOzlbv3r2VkZGhxMTEgH0mJSVp6tSpNxxLcXGxiouLnfulpaUqKCjQXXfdpZCQkNtpEwAA1BBjjC5evKjo6Gg1aHDjD4qqHGBKS0s1depUff/731f37t2d5aNGjVJsbKyio6N16NAhzZgxQzk5OdqwYYMkyefzBYQXSc59n8930xq/36/Lly+rSZMm141nwYIFevHFF6vaDgAAqEPOnDmju++++4brqxxgUlJSdOTIEX3wwQcByydNmuT8Oy4uTm3atNHAgQN16tQpdejQoaoPd0uzZs1Samqqc7+wsFBt27bVmTNn5Ha7q+1xAQBA8Pj9fsXExKhFixY3ratSgJkyZYo2bdqkPXv23DQdSVJ8fLwk6eTJk+rQoYO8Xq8OHDgQUJOfny9J8nq9zn/Lln27xu12l3v0RZJcLpdcLtd1y91uNwEGAADL3Or0j0pdhWSM0ZQpU/T2229r165dateu3S23ycrKkiS1adNGkpSQkKDDhw/r7NmzTk1aWprcbre6du3q1OzcuTNgP2lpaUpISKjMcAEAQD1VqQCTkpKiP/zhD1q7dq1atGghn88nn8+ny5cvS5JOnTql+fPnKzMzU6dPn9a7776rsWPHqn///urRo4ckadCgQeratavGjBmjTz75RNu3b9fs2bOVkpLiHEF5+umn9Ze//EXPPfecjh8/rtdee03r16/XtGnTgtw+AACwUaUuo77R4ZxVq1Zp/PjxOnPmjH70ox/pyJEjunTpkmJiYvT3f//3mj17dsDHOJ9++qkmT56s9PR0NWvWTOPGjdPChQvVsOHfPtFKT0/XtGnTdPToUd199916/vnnNX78+Ao35vf75fF4VFhYyEdIAABYoqLv37f1PTB1GQEGAAD7VPT9m7+FBAAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWaXjrEgAAcCf53szNt6w5vTC5BkZyYxyBAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALBOw9oegI2+N3PzLWtOL0yugZEAAHBn4ggMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOtUKsAsWLBADzzwgFq0aKHIyEg98cQTysnJCaj55ptvlJKSorvuukvNmzfX8OHDlZ+fH1CTl5en5ORkNW3aVJGRkZo+fbpKSkoCatLT03XffffJ5XKpY8eOWr16ddU6BAAA9U6lAszu3buVkpKiffv2KS0tTVevXtWgQYN06dIlp2batGn67//+b7311lvavXu3Pv/8cz355JPO+mvXrik5OVlXrlzRhx9+qDfeeEOrV6/WnDlznJrc3FwlJyfr0UcfVVZWlqZOnaqf/vSn2r59exBaBgAAtgsxxpiqbnzu3DlFRkZq9+7d6t+/vwoLC9W6dWutXbtW//AP/yBJOn78uLp06aKMjAz169dPW7du1eOPP67PP/9cUVFRkqTly5drxowZOnfunMLCwjRjxgxt3rxZR44ccR5rxIgRunDhgrZt21ahsfn9fnk8HhUWFsrtdle1xXJ9b+bmW9acXpgc1McEAKCm1Ob7XEXfv2/rHJjCwkJJUkREhCQpMzNTV69eVWJiolPTuXNntW3bVhkZGZKkjIwMxcXFOeFFkpKSkuT3+5Wdne3UfHsfZTVl+yhPcXGx/H5/wA0AANRPVQ4wpaWlmjp1qr7//e+re/fukiSfz6ewsDCFh4cH1EZFRcnn8zk13w4vZevL1t2sxu/36/Lly+WOZ8GCBfJ4PM4tJiamqq0BAIA6rsoBJiUlRUeOHNG6deuCOZ4qmzVrlgoLC53bmTNnantIAACgmjSsykZTpkzRpk2btGfPHt19993Ocq/XqytXrujChQsBR2Hy8/Pl9XqdmgMHDgTsr+wqpW/XfPfKpfz8fLndbjVp0qTcMblcLrlcrqq0AwAALFOpIzDGGE2ZMkVvv/22du3apXbt2gWs79Onjxo1aqSdO3c6y3JycpSXl6eEhARJUkJCgg4fPqyzZ886NWlpaXK73eratatT8+19lNWU7QMAANzZKnUEJiUlRWvXrtU777yjFi1aOOeseDweNWnSRB6PRxMmTFBqaqoiIiLkdrv1zDPPKCEhQf369ZMkDRo0SF27dtWYMWO0aNEi+Xw+zZ49WykpKc4RlKefflq//vWv9dxzz+knP/mJdu3apfXr12vz5lufFQ0AAOq/Sh2Bef3111VYWKhHHnlEbdq0cW5vvvmmU/PKK6/o8ccf1/Dhw9W/f395vV5t2LDBWR8aGqpNmzYpNDRUCQkJ+tGPfqSxY8dq3rx5Tk27du20efNmpaWlqWfPnvrP//xP/dd//ZeSkpKC0DIAALDdbX0PTF3G98AAAFA19f57YAAAAGoDAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOpUOMHv27NHQoUMVHR2tkJAQbdy4MWD9+PHjFRISEnAbPHhwQE1BQYFGjx4tt9ut8PBwTZgwQUVFRQE1hw4d0sMPP6zGjRsrJiZGixYtqnx3AACgXqp0gLl06ZJ69uypZcuW3bBm8ODB+uKLL5zbH//4x4D1o0ePVnZ2ttLS0rRp0ybt2bNHkyZNctb7/X4NGjRIsbGxyszM1OLFizV37lz99re/rexwAQBAPdSwshsMGTJEQ4YMuWmNy+WS1+std92xY8e0bds2ffTRR7r//vslSUuXLtVjjz2mJUuWKDo6WmvWrNGVK1e0cuVKhYWFqVu3bsrKytLLL78cEHQAAMCdqVrOgUlPT1dkZKQ6deqkyZMn6/z58866jIwMhYeHO+FFkhITE9WgQQPt37/fqenfv7/CwsKcmqSkJOXk5Oirr74q9zGLi4vl9/sDbgAAoH4KeoAZPHiwfv/732vnzp166aWXtHv3bg0ZMkTXrl2TJPl8PkVGRgZs07BhQ0VERMjn8zk1UVFRATVl98tqvmvBggXyeDzOLSYmJtitAQCAOqLSHyHdyogRI5x/x8XFqUePHurQoYPS09M1cODAYD+cY9asWUpNTXXu+/1+QgwAAPVUtV9G3b59e7Vq1UonT56UJHm9Xp09ezagpqSkRAUFBc55M16vV/n5+QE1ZfdvdG6Ny+WS2+0OuAEAgPqp2gPMZ599pvPnz6tNmzaSpISEBF24cEGZmZlOza5du1RaWqr4+HinZs+ePbp69apTk5aWpk6dOqlly5bVPWQAAFDHVTrAFBUVKSsrS1lZWZKk3NxcZWVlKS8vT0VFRZo+fbr27dun06dPa+fOnRo2bJg6duyopKQkSVKXLl00ePBgTZw4UQcOHNDevXs1ZcoUjRgxQtHR0ZKkUaNGKSwsTBMmTFB2drbefPNN/fKXvwz4iAgAANy5Kh1gPv74Y/Xu3Vu9e/eWJKWmpqp3796aM2eOQkNDdejQIf3whz/UvffeqwkTJqhPnz56//335XK5nH2sWbNGnTt31sCBA/XYY4/poYceCviOF4/Hox07dig3N1d9+vTRs88+qzlz5nAJNQAAkFSFk3gfeeQRGWNuuH779u233EdERITWrl1705oePXro/fffr+zwAADAHYC/hQQAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgnUoHmD179mjo0KGKjo5WSEiINm7cGLDeGKM5c+aoTZs2atKkiRITE3XixImAmoKCAo0ePVput1vh4eGaMGGCioqKAmoOHTqkhx9+WI0bN1ZMTIwWLVpU+e4AAEC9VOkAc+nSJfXs2VPLli0rd/2iRYv0q1/9SsuXL9f+/fvVrFkzJSUl6ZtvvnFqRo8erezsbKWlpWnTpk3as2ePJk2a5Kz3+/0aNGiQYmNjlZmZqcWLF2vu3Ln67W9/W4UWAQBAfdOwshsMGTJEQ4YMKXedMUavvvqqZs+erWHDhkmSfv/73ysqKkobN27UiBEjdOzYMW3btk0fffSR7r//fknS0qVL9dhjj2nJkiWKjo7WmjVrdOXKFa1cuVJhYWHq1q2bsrKy9PLLLwcEHQAAcGcK6jkwubm58vl8SkxMdJZ5PB7Fx8crIyNDkpSRkaHw8HAnvEhSYmKiGjRooP379zs1/fv3V1hYmFOTlJSknJwcffXVV8EcMgAAsFClj8DcjM/nkyRFRUUFLI+KinLW+Xw+RUZGBg6iYUNFREQE1LRr1+66fZSta9my5XWPXVxcrOLiYue+3++/zW4AAEBdVW+uQlqwYIE8Ho9zi4mJqe0hAQCAahLUAOP1eiVJ+fn5Acvz8/OddV6vV2fPng1YX1JSooKCgoCa8vbx7cf4rlmzZqmwsNC5nTlz5vYbAgAAdVJQA0y7du3k9Xq1c+dOZ5nf79f+/fuVkJAgSUpISNCFCxeUmZnp1OzatUulpaWKj493avbs2aOrV686NWlpaerUqVO5Hx9JksvlktvtDrgBAID6qdIBpqioSFlZWcrKypL01xN3s7KylJeXp5CQEE2dOlW/+MUv9O677+rw4cMaO3asoqOj9cQTT0iSunTposGDB2vixIk6cOCA9u7dqylTpmjEiBGKjo6WJI0aNUphYWGaMGGCsrOz9eabb+qXv/ylUlNTg9Y4AACwV6VP4v3444/16KOPOvfLQsW4ceO0evVqPffcc7p06ZImTZqkCxcu6KGHHtK2bdvUuHFjZ5s1a9ZoypQpGjhwoBo0aKDhw4frV7/6lbPe4/Fox44dSklJUZ8+fdSqVSvNmTOHS6gBAIAkKcQYY2p7ENXB7/fL4/GosLAw6B8nfW/m5lvWnF6YHNTHBACgptTm+1xF37/rzVVIAADgzkGAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALBO0APM3LlzFRISEnDr3Lmzs/6bb75RSkqK7rrrLjVv3lzDhw9Xfn5+wD7y8vKUnJyspk2bKjIyUtOnT1dJSUmwhwoAACzVsDp22q1bN7333nt/e5CGf3uYadOmafPmzXrrrbfk8Xg0ZcoUPfnkk9q7d68k6dq1a0pOTpbX69WHH36oL774QmPHjlWjRo30H//xH9UxXAAAYJlqCTANGzaU1+u9bnlhYaFWrFihtWvX6gc/+IEkadWqVerSpYv27dunfv36aceOHTp69Kjee+89RUVFqVevXpo/f75mzJihuXPnKiwsrDqGDAAALFIt58CcOHFC0dHRat++vUaPHq28vDxJUmZmpq5evarExESntnPnzmrbtq0yMjIkSRkZGYqLi1NUVJRTk5SUJL/fr+zs7Bs+ZnFxsfx+f8ANAADUT0EPMPHx8Vq9erW2bdum119/Xbm5uXr44Yd18eJF+Xw+hYWFKTw8PGCbqKgo+Xw+SZLP5wsIL2Xry9bdyIIFC+TxeJxbTExMcBsDAAB1RtA/QhoyZIjz7x49eig+Pl6xsbFav369mjRpEuyHc8yaNUupqanOfb/fT4gBAKCeqvbLqMPDw3Xvvffq5MmT8nq9unLlii5cuBBQk5+f75wz4/V6r7sqqex+eefVlHG5XHK73QE3AABQP1V7gCkqKtKpU6fUpk0b9enTR40aNdLOnTud9Tk5OcrLy1NCQoIkKSEhQYcPH9bZs2edmrS0NLndbnXt2rW6hwsAACwQ9I+Qfv7zn2vo0KGKjY3V559/rhdeeEGhoaEaOXKkPB6PJkyYoNTUVEVERMjtduuZZ55RQkKC+vXrJ0kaNGiQunbtqjFjxmjRokXy+XyaPXu2UlJS5HK5gj1cAABgoaAHmM8++0wjR47U+fPn1bp1az300EPat2+fWrduLUl65ZVX1KBBAw0fPlzFxcVKSkrSa6+95mwfGhqqTZs2afLkyUpISFCzZs00btw4zZs3L9hDBQAAlgp6gFm3bt1N1zdu3FjLli3TsmXLblgTGxurLVu2BHtoAACgnuBvIQEAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1CDAAAMA6BBgAAGAdAgwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1iHAAAAA6xBgAACAdQgwAADAOgQYAABgHQIMAACwDgEGAABYhwADAACsQ4ABAADWIcAAAADrEGAAAIB1Gtb2AAAAQM353szNtT2EoOAIDAAAsA4BBgAAWIePkKpJRQ7RnV6YXAMjAQCg/uEIDAAAsA4BBgAAWIcAAwAArEOAAQAA1uEk3lrEib4AgGCqL9/xUhEEGAAALHAnhZOK4CMkAABgHQIMAACwDgEGAABYhwADAACsw0m8dRxXKgEAcD2OwAAAAOtwBKYeqOmjNBwVAgDUNo7AAAAA63AE5g5R0S9A4sgJgNrC0V1UBgEGAfimRwCouGD9ziSYVR4BBgBwQxwVqRn8z2Pl1ekAs2zZMi1evFg+n089e/bU0qVL1bdv39oeFoKkrv1irMn/k7Kx97r2JlWTY7bx+alJ9fXNt772VV/U2QDz5ptvKjU1VcuXL1d8fLxeffVVJSUlKScnR5GRkbU9PNxCTb7w69ovmbrWu41v4nXtOazJ/dg4X3XNndz7nSTEGGNqexDliY+P1wMPPKBf//rXkqTS0lLFxMTomWee0cyZM2+5vd/vl8fjUWFhodxud1DHVtfeMAGbBOsIFW4fc4HbUV0hsKLv33XyCMyVK1eUmZmpWbNmOcsaNGigxMREZWRklLtNcXGxiouLnfuFhYWS/vpEBFtp8ddB3ydwp2g77a3aHgL+H3OB21Ed76/f3u+tjq/UyQDz5Zdf6tq1a4qKigpYHhUVpePHj5e7zYIFC/Tiiy9etzwmJqZaxggAwJ3M82r17v/ixYvyeDw3XF8nA0xVzJo1S6mpqc790tJSFRQU6K677lJISEjQHsfv9ysmJkZnzpwJ+kdTdUV975H+7Fffe6zv/Un1v0f6qzpjjC5evKjo6Oib1tXJANOqVSuFhoYqPz8/YHl+fr68Xm+527hcLrlcroBl4eHh1TVEud3uevlD+W31vUf6s19977G+9yfV/x7pr2puduSlTJ38UwJhYWHq06ePdu7c6SwrLS3Vzp07lZCQUIsjAwAAdUGdPAIjSampqRo3bpzuv/9+9e3bV6+++qouXbqkH//4x7U9NAAAUMvqbIB56qmndO7cOc2ZM0c+n0+9evXStm3brjuxt6a5XC698MIL131cVZ/U9x7pz371vcf63p9U/3ukv+pXZ78HBgAA4Ebq5DkwAAAAN0OAAQAA1iHAAAAA6xBgAACAdQgw5fj3f/93Pfjgg2ratGmFvwzPGKM5c+aoTZs2atKkiRITE3XixImAmoKCAo0ePVput1vh4eGaMGGCioqKqqGDm6vsOE6fPq2QkJByb2+99be/pVLe+nXr1tVESwGq8jw/8sgj14396aefDqjJy8tTcnKymjZtqsjISE2fPl0lJSXV2coNVbbHgoICPfPMM+rUqZOaNGmitm3b6mc/+5nzN8PK1NYcLlu2TN/73vfUuHFjxcfH68CBAzetf+utt9S5c2c1btxYcXFx2rJlS8D6irwea1plevzd736nhx9+WC1btlTLli2VmJh4Xf348eOvm6vBgwdXdxs3VJn+Vq9efd3YGzduHFBj+xyW9zslJCREycl/+wOIdWUO9+zZo6FDhyo6OlohISHauHHjLbdJT0/XfffdJ5fLpY4dO2r16tXX1VT2dV1pBteZM2eOefnll01qaqrxeDwV2mbhwoXG4/GYjRs3mk8++cT88Ic/NO3atTOXL192agYPHmx69uxp9u3bZ95//33TsWNHM3LkyGrq4sYqO46SkhLzxRdfBNxefPFF07x5c3Px4kWnTpJZtWpVQN23+68pVXmeBwwYYCZOnBgw9sLCQmd9SUmJ6d69u0lMTDQHDx40W7ZsMa1atTKzZs2q7nbKVdkeDx8+bJ588knz7rvvmpMnT5qdO3eae+65xwwfPjygrjbmcN26dSYsLMysXLnSZGdnm4kTJ5rw8HCTn59fbv3evXtNaGioWbRokTl69KiZPXu2adSokTl8+LBTU5HXY02qbI+jRo0yy5YtMwcPHjTHjh0z48ePNx6Px3z22WdOzbhx48zgwYMD5qqgoKCmWgpQ2f5WrVpl3G53wNh9Pl9Aje1zeP78+YD+jhw5YkJDQ82qVaucmroyh1u2bDH/9m//ZjZs2GAkmbfffvum9X/5y19M06ZNTWpqqjl69KhZunSpCQ0NNdu2bXNqKvt8VQUB5iZWrVpVoQBTWlpqvF6vWbx4sbPswoULxuVymT/+8Y/GGGOOHj1qJJmPPvrIqdm6dasJCQkx//u//xv0sd9IsMbRq1cv85Of/CRgWUV+8KtbVfsbMGCA+Zd/+Zcbrt+yZYtp0KBBwC/Z119/3bjdblNcXByUsVdUsOZw/fr1JiwszFy9etVZVhtz2LdvX5OSkuLcv3btmomOjjYLFiwot/4f//EfTXJycsCy+Ph480//9E/GmIq9HmtaZXv8rpKSEtOiRQvzxhtvOMvGjRtnhg0bFuyhVkll+7vV79b6OIevvPKKadGihSkqKnKW1aU5LFOR3wHPPfec6datW8Cyp556yiQlJTn3b/f5qgg+QgqC3Nxc+Xw+JSYmOss8Ho/i4+OVkZEhScrIyFB4eLjuv/9+pyYxMVENGjTQ/v37a2yswRhHZmamsrKyNGHChOvWpaSkqFWrVurbt69Wrlx5yz+HHmy309+aNWvUqlUrde/eXbNmzdLXX38dsN+4uLiAL1JMSkqS3+9XdnZ28Bu5iWD9LBUWFsrtdqthw8Dvs6zJObxy5YoyMzMDXjsNGjRQYmKi89r5royMjIB66a9zUVZfkddjTapKj9/19ddf6+rVq4qIiAhYnp6ersjISHXq1EmTJ0/W+fPngzr2iqhqf0VFRYqNjVVMTIyGDRsW8Dqqj3O4YsUKjRgxQs2aNQtYXhfmsLJu9RoMxvNVEXX2m3ht4vP5JOm6bwmOiopy1vl8PkVGRgasb9iwoSIiIpyamhCMcaxYsUJdunTRgw8+GLB83rx5+sEPfqCmTZtqx44d+ud//mcVFRXpZz/7WdDGfytV7W/UqFGKjY1VdHS0Dh06pBkzZignJ0cbNmxw9lve/Jatq0nBmMMvv/xS8+fP16RJkwKW1/Qcfvnll7p27Vq5z+3x48fL3eZGc/Ht11rZshvV1KSq9PhdM2bMUHR0dMAbwuDBg/Xkk0+qXbt2OnXqlP71X/9VQ4YMUUZGhkJDQ4Paw81Upb9OnTpp5cqV6tGjhwoLC7VkyRI9+OCDys7O1t13313v5vDAgQM6cuSIVqxYEbC8rsxhZd3oNej3+3X58mV99dVXt/0zXxF3TICZOXOmXnrppZvWHDt2TJ07d66hEQVXRfu7XZcvX9batWv1/PPPX7fu28t69+6tS5cuafHixUF586vu/r79Rh4XF6c2bdpo4MCBOnXqlDp06FDl/VZGTc2h3+9XcnKyunbtqrlz5wasq845RNUsXLhQ69atU3p6esCJriNGjHD+HRcXpx49eqhDhw5KT0/XwIEDa2OoFZaQkBDwh3kffPBBdenSRb/5zW80f/78WhxZ9VixYoXi4uLUt2/fgOU2z2FdcMcEmGeffVbjx4+/aU379u2rtG+v1ytJys/PV5s2bZzl+fn56tWrl1Nz9uzZgO1KSkpUUFDgbH87Ktrf7Y7jT3/6k77++muNHTv2lrXx8fGaP3++iouLb/vvZdRUf2Xi4+MlSSdPnlSHDh3k9XqvO4M+Pz9fkoIyf1LN9Hjx4kUNHjxYLVq00Ntvv61GjRrdtD6Yc1ieVq1aKTQ01Hkuy+Tn59+wF6/Xe9P6irwea1JVeiyzZMkSLVy4UO+995569Ohx09r27durVatWOnnyZI2++d1Of2UaNWqk3r176+TJk5Lq1xxeunRJ69at07x58275OLU1h5V1o9eg2+1WkyZNFBoaets/ExUStLNp6qHKnsS7ZMkSZ1lhYWG5J/F+/PHHTs327dtr7STeqo5jwIAB1125ciO/+MUvTMuWLas81qoI1vP8wQcfGEnmk08+Mcb87STeb59B/5vf/Ma43W7zzTffBK+BCqhqj4WFhaZfv35mwIAB5tKlSxV6rJqYw759+5opU6Y4969du2b+7u/+7qYn8T7++OMByxISEq47ifdmr8eaVtkejTHmpZdeMm6322RkZFToMc6cOWNCQkLMO++8c9vjrayq9PdtJSUlplOnTmbatGnGmPozh8b89X3E5XKZL7/88paPUZtzWEYVPIm3e/fuActGjhx53Um8t/MzUaGxBm1P9cinn35qDh486FwqfPDgQXPw4MGAS4Y7depkNmzY4NxfuHChCQ8PN++88445dOiQGTZsWLmXUffu3dvs37/ffPDBB+aee+6ptcuobzaOzz77zHTq1Mns378/YLsTJ06YkJAQs3Xr1uv2+e6775rf/e535vDhw+bEiRPmtddeM02bNjVz5syp9n6+q7L9nTx50sybN898/PHHJjc317zzzjumffv2pn///s42ZZdRDxo0yGRlZZlt27aZ1q1b1+pl1JXpsbCw0MTHx5u4uDhz8uTJgMs2S0pKjDG1N4fr1q0zLpfLrF692hw9etRMmjTJhIeHO1d8jRkzxsycOdOp37t3r2nYsKFZsmSJOXbsmHnhhRfKvYz6Vq/HmlTZHhcuXGjCwsLMn/70p4C5KvsddPHiRfPzn//cZGRkmNzcXPPee++Z++67z9xzzz01Hqir0t+LL75otm/fbk6dOmUyMzPNiBEjTOPGjU12drZTY/sclnnooYfMU089dd3yujSHFy9edN7nJJmXX37ZHDx40Hz66afGGGNmzpxpxowZ49SXXUY9ffp0c+zYMbNs2bJyL6O+2fMVDASYcowbN85Iuu725z//2anR/39fRpnS0lLz/PPPm6ioKONyuczAgQNNTk5OwH7Pnz9vRo4caZo3b27cbrf58Y9/HBCKasqtxpGbm3tdv8YYM2vWLBMTE2OuXbt23T63bt1qevXqZZo3b26aNWtmevbsaZYvX15ubXWrbH95eXmmf//+JiIiwrhcLtOxY0czffr0gO+BMcaY06dPmyFDhpgmTZqYVq1amWeffTbgEuSaVNke//znP5f7My3J5ObmGmNqdw6XLl1q2rZta8LCwkzfvn3Nvn37nHUDBgww48aNC6hfv369uffee01YWJjp1q2b2bx5c8D6irwea1pleoyNjS13rl544QVjjDFff/21GTRokGndurVp1KiRiY2NNRMnTgzqm0NlVaa/qVOnOrVRUVHmscceM//zP/8TsD/b59AYY44fP24kmR07dly3r7o0hzf6/VDWz7hx48yAAQOu26ZXr14mLCzMtG/fPuD9sMzNnq9gCDGmhq9zBQAAuE18DwwAALAOAQYAAFiHAAMAAKxDgAEAANYhwAAAAOsQYAAAgHUIMAAAwDoEGAAAYB0CDAAAsA4BBgAAWIcAAwAArEOAAQAA1vk/dSl73bIBJzoAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(h.view(-1).tolist(), bins=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the above figure we can clearly see that the -1 and 1 values occur most frequently. Which would mean for all of these values when the network backpropagates, there would be no gradiant and associated neuron would not learn anything. Let's check one more thing\n",
    "\n",
    "Now we'll check how many examples have h > 0.99"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x121c60a50>"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABj0AAAElCAYAAAC/JSDoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA6s0lEQVR4nO3df5BdZX04/vcNIQtq9mKAJKQE5IcCFaEtSog/KEpKiB0rgq0irUApWhvoQLQqHX/ROg1Kq1TlR6ejoFMjlhnQEac4gBLqGKiGYai2ZISiwJBExckuxLJEcr9/9Mt+XEnuye6zT57nnn29ZnYm2XvP8+s8v85937On0+v1egEAAAAAADDgZpUuAAAAAAAAwHQQ9AAAAAAAAFpB0AMAAAAAAGgFQQ8AAAAAAKAVBD0AAAAAAIBWEPQAAAAAAABaQdADAAAAAABoBUEPAAAAAACgFQQ9AAAAAACAVphdugC/bvv27fHYY4/F3Llzo9PplC4OAAAAAABQUK/XiyeeeCIWLVoUs2b1v5cjW9DjyiuvjMsvvzw2bdoUxx57bHz605+O448/vvG4xx57LBYvXpyrWAAAAAAAwAB65JFH4sADD+z7nixBjy9/+cuxatWquOaaa2LJkiVxxRVXxPLly2PDhg0xf/78vsfOnTs3Iv6v8MPDw5POu9vt9n19ZGQk6fiUtHMqWe+apda73/E1t3nOvpha75lqUPta6vnMWe8mbe2LOdsld5uXXGvaOsZq1q9dSs4tudPOWe+cah2fuyKlzWvej83UPddMnVMHda/YpOR1SU72yDtWsmw194eax3fOvHMa5Hmv1r2ivrZjbS1b7nrl6uejo6OxePHi8fhBP51er9ebck47sWTJknjFK14Rn/nMZyLi//5k1eLFi+PCCy+M97///X2PHR0djW63GyMjI1MKejT9Saym6qb8Sa0MTbnLSta7Zqn17nd8zW2esy+m1numGtS+lno+c9a7SVv7Ys52yd3mJdeato6xmvVrl5JzS+60c9Y7p1rH565IafOa92Mzdc81U+fUQd0rNil5XZKTPfKOlSxbzf2h5vGdM++cBnneq3WvqK/tWFvLlrteufr5ZOIG0/4g86effjrWr18fy5Yt+3+ZzJoVy5Yti3Xr1j3n/WNjYzE6OjrhBwAAAAAAYLKmPejxs5/9LJ555plYsGDBhN8vWLAgNm3a9Jz3r169Orrd7viP53kAAAAAAABTMe1Bj8m65JJLYmRkZPznkUceKV0kAAAAAABgAE37g8z322+/2GOPPWLz5s0Tfr958+ZYuHDhc94/NDQUQ0ND010MAAAAAABghpn2Oz3mzJkTxx13XNx+++3jv9u+fXvcfvvtsXTp0unODgAAAAAAICIy3OkREbFq1ao4++yz4+Uvf3kcf/zxccUVV8TWrVvj3HPPzZEdAAAAAABAnqDHW97ylvjpT38aH/rQh2LTpk3xW7/1W3HLLbc85+HmAAAAAAAA06XT6/V6pQvxq0ZHR6Pb7cbIyEgMDw9P+vhOp9P39abqNh2fknZOJetds9R69zu+5jbP2RdT6z1TDWpfSz2fOevdpK19MWe75G7zkmtNW8dYzfq1S8m5JXfaOeudU63jc1ektHnN+7GZuueaqXPqoO4Vm5S8LsnJHnnHSpat5v5Q8/jOmXdOgzzv1bpX1Nd2rK1ly12vXP18MnGDaX+mBwAAAAAAQAmCHgAAAAAAQCsIegAAAAAAAK0g6AEAAAAAALTC7NIF2Jlut7vT1/o98KTmh6jWbFAftJSq5noParsO6sNlm44v+VC+1L5Q88MKB/VBxzkN8kO1c+Y9qOczot4xmLvNcz44L2eb1tyXSo7vnOu3/jD9eQ/yg46Nwd2fds69Rc7zXfNesWS9a/7Mw5w6+fRrfiB3qkG9FszZ5jU/XLrmea3mNXZQr89rXkt2lTs9AAAAAACAVhD0AAAAAAAAWkHQAwAAAAAAaAVBDwAAAAAAoBUEPQAAAAAAgFYQ9AAAAAAAAFpB0AMAAAAAAGiF2aULsLv1er3SRSii0+n0fb2pXfq9npp2k37p5z6fKfWeqXKek9znu63nNKVeKXNDm+We91I05V3rnJrz2Ny0+Y6l1DvnGEud60vumQZ5jR3UvGue73PKWe+cbZ77fLV1DDaptWy55/OceeeUc/1OVXJ857wmalLznJpTyTW05nbpp+ZrgyY5x3fOvHOnXWs/r3Vtn07u9AAAAAAAAFpB0AMAAAAAAGgFQQ8AAAAAAKAVBD0AAAAAAIBWEPQAAAAAAABaQdADAAAAAABoBUEPAAAAAACgFWaXLsDOjIyMxPDw8KSP63Q6fV/v9XpJx6ekXVLOejcpmXZqvVPOaWrebe2LJZVsl5S+lrOf5paz3k1ytkvJtHO2S+42z7keNMl5ztpar5Jyrt+5+2m/42s+XyX7caqUNq/5fOfs54NsUMdYk5J7xZzjv+ZzUvJ6q+Y9ckrdSvalmj93mKl51zz+eS5ryeCp+XOmGvZr7vQAAAAAAABaQdADAAAAAABoBUEPAAAAAACgFQQ9AAAAAACAVhD0AAAAAAAAWkHQAwAAAAAAaAVBDwAAAAAAoBVmT3eCH/nIR+LSSy+d8Lsjjjgi7r///unOaod6vV7f1zudzm4pR9s0tWtOJc9Zv7xT+1rNfTWl3m3NO1VK2XP3tZxK1jtF7vGb85zUPLfkrHdKvVLLVfMa2a9suftpyvEl19Ccaedu85Q1NGfeOY+NKDsG+2lrvSLy7tfaWu+S+8ycfTE17ZR5Mfe8lXMNTTlnNV8T1byO1bo3SM27pJqvQ5uUnNdKpl1yf57r2Ii843tQ+3GqWj+TnIxpD3pERLz0pS+N22677f9lMjtLNgAAAAAAAOOyRCNmz54dCxcuzJE0AAAAAADADmV5pscPf/jDWLRoURx66KFx1llnxcMPP7zT946NjcXo6OiEHwAAAAAAgMma9qDHkiVL4rrrrotbbrklrr766njooYfiNa95TTzxxBM7fP/q1auj2+2O/yxevHi6iwQAAAAAAMwAnV7mJ7Js2bIlDj744PjEJz4R55133nNeHxsbi7GxsfH/j46OxuLFi2NkZCSGh4envTxtfcDUID90t0mtDxyr+UHmJR8gl6rWvFOVfMBUzQ/eSpGz3jXPqSXnliY1rxXGwfQem1vNa2jNY6ita2iTnHNqSTXPWyX7WkklH2Resp/nXEtKrkU1r6GDOp/XXO+a9waD2s+b1Fqv1LRzKtnXaj7fTUrOW4Nq0Pv5rsQNsj9hfJ999omXvOQl8cADD+zw9aGhoRgaGspdDAAAAAAAoOWyBz2efPLJePDBB+NP/uRPJnVct9udUn5N0aKav53bpF/eNUdUc7Z507ElvzlUc19LGSe5o/8l+7Jvg+x+tc5dbf4G3aC2ecl+XPMdESXlbJdB7uf95C5XW+vd1rtrcu5ja/72/Uy9Y3FQ97ipadd8fZ2Sfsn5vM13Qw7qOKn5joa2zms51Vyvmq+ZBlXNn2k2qXnOzLXWjI6O7nLMYNqf6fGe97wn1q5dGz/60Y/iO9/5TrzpTW+KPfbYI84888zpzgoAAAAAAGDctN/p8eijj8aZZ54Zjz/+eOy///7x6le/Ou66667Yf//9pzsrAAAAAACAcdMe9Lj++uunO0kAAAAAAIBG0/7nrQAAAAAAAEoQ9AAAAAAAAFpB0AMAAAAAAGgFQQ8AAAAAAKAVpv1B5tNlZGQkhoeHd3u+vV5vysd2Op2ktJuOb3o9Je8UqfUqWbaccvaH3PVKyTuln+ZWc9lyqvmc5eznNc89OcdYznrlzDv3GpoiZ18suU6l5l2y7Exe7rmj5PjPKWfZah5DbS1b6lrRL++S61hbx8iupJ8z735y9qXcah7fte6ZUtV6vdVkkK/HSl63NKn1M7CaP2dKVev8UfNaUHLPNF3c6QEAAAAAALSCoAcAAAAAANAKgh4AAAAAAEArCHoAAAAAAACtIOgBAAAAAAC0gqAHAAAAAADQCoIeAAAAAABAK8wuXYCp6HQ62dLu9XrZ8s5Z7lQ5652qX95N5S6Zd2rZUo4f5LxT2jy1n/ZLvynt1LKVzDsl7ZxqnjOb1LxWlGzXlH6eknZpKWXLOf5zzlttlrPebT3fufvSoO4VS+Zdcs6ted+S0lcHeU6sud459w6DukduUnItaVKy3Wpex1L6ec3XwCXlPN+pebd1v1byM4+a27zWfU3uzzRzpT06OhrdbneX3utODwAAAAAAoBUEPQAAAAAAgFYQ9AAAAAAAAFpB0AMAAAAAAGgFQQ8AAAAAAKAVBD0AAAAAAIBWEPQAAAAAAABaYXbpAky3Xq/X9/VOp5M1/RSpZUtJO6VebW7zfnmntmlKu+Rsk4j+Zcudd870c5c9Rck2LylnvZvGWK1jMHXuyLmW1FzvnFLLlnONzX18ipxjrOT63SRlXss5DlLHUK1zZmr6Ne9TU/LO3c/7KTlfNyk5vku2S87+kPuaqNZ9cJuvBVPmliZtnZvaPO+Vyjt3m9baHwZ1ToxIm/cGuV45j0/tD7WOwd11vt3pAQAAAAAAtIKgBwAAAAAA0AqCHgAAAAAAQCsIegAAAAAAAK0g6AEAAAAAALSCoAcAAAAAANAKsyd7wJ133hmXX355rF+/PjZu3Bg33XRTnHbaaeOv93q9+PCHPxz//M//HFu2bIlXvepVcfXVV8eLX/zi6Sz3TnU6nb6v93q9bMen5p0itV5Nr6dIrXdK2Uq2eerxOc9JipL9nKlJ6Ws1z6kl1Vy2FDnP564cn1Nb+9ogr3P95Oxrg7xOlSx7zeMkRc49dO49cL/0c8/ntcpdr5LtkpJ3yeuxkn2trfNWSSX7Usm0mwzq3JDboI7BQb4myrl3SFXy89RBvS6p+TPsQdhLTvpOj61bt8axxx4bV1555Q5f//jHPx6f+tSn4pprrom77747nv/858fy5cvjqaeeSi4sAAAAAADAzkz6To8VK1bEihUrdvhar9eLK664Ij7wgQ/EG9/4xoiI+MIXvhALFiyIr3zlK/HWt741rbQAAAAAAAA7Ma3P9HjooYdi06ZNsWzZsvHfdbvdWLJkSaxbt246swIAAAAAAJhg0nd69LNp06aIiFiwYMGE3y9YsGD8tV83NjYWY2Nj4/8fHR2dziIBAAAAAAAzxLTe6TEVq1evjm63O/6zePHi0kUCAAAAAAAG0LQGPRYuXBgREZs3b57w+82bN4+/9usuueSSGBkZGf955JFHprNIAAAAAADADDGtQY9DDjkkFi5cGLfffvv470ZHR+Puu++OpUuX7vCYoaGhGB4envADAAAAAAAwWZN+pseTTz4ZDzzwwPj/H3roobj33ntj3rx5cdBBB8VFF10UH/3oR+PFL35xHHLIIfHBD34wFi1aFKeddtq0FbrX601bWpNNu9PpTPnYVLnTL6VfmzZJOV+7cnyptGvOO3ebp/SHJillS23TmsdvznGQouT4rllb69Uk59xRcl5rOnaQ652i5rLVuk7lVvOeKec5SZG7XOaWyaedqmSbp+Rd836tZN6Der1W65wXkT4OcubdJKVdc57vQV37d+X4Wue1JiXPSc2fcdW8j835WW7N9c5pEOo16aDH9773vXjta187/v9Vq1ZFRMTZZ58d1113Xbz3ve+NrVu3xjve8Y7YsmVLvPrVr45bbrkl9tprr+krNQAAAAAAwK/p9CoLzYyOjka3242RkZEq/9TVoEYHS37Dpklbv0lWc7uk5D3Id3o0qWw6HAip56vmMchzlRzfgzzfl9w79NPWb2tF1HunxyC3eYqa90xtNVPnlpJ9bZDbPEXN81pb8655zhvUdSqi7s9M2qrWzx1yn69a+1rJOz1qXr9LfubRpOY2z2UycYNpfaYHAAAAAABAKYIeAAAAAABAKwh6AAAAAAAArSDoAQAAAAAAtIKgBwAAAAAA0AqzSxdguqU+fb7p+JyaytZPyXqXbLPc9U45J01KtluTfvXO3WY527yklHYrOb6b0m7r3FKz1HZJ6Wsl5SzbTN075E67X7vUPNfXPOfmVPP4T1Fy/OYcQ9ORfq1p1zyGSo7RWveKg3w9VfMeeVDHd859au68U+Q83yU/85ip1/4l+3GtbTLoBnVOHWQ1XAu60wMAAAAAAGgFQQ8AAAAAAKAVBD0AAAAAAIBWEPQAAAAAAABaQdADAAAAAABoBUEPAAAAAACgFQQ9AAAAAACAVphdugBT0el0sqXd6/WmnHdquZryTtFUtpx5NylZtpS0U8tVss1TpIyRXTk+Z9ol546SaZds85JjrGRfbNIv75rn65rnvaZ26/d6zfXKKfecmrOfl5zXmrR1P1dyzszZ11IN6vgfZCX7Yoqa90w1j+9+Sl6XpCo5X+dsl9z795x591Pz3FHyGrnk9XXN1/4l+0uKts6JbTao8/XuWp/d6QEAAAAAALSCoAcAAAAAANAKgh4AAAAAAEArCHoAAAAAAACtIOgBAAAAAAC0gqAHAAAAAADQCrNLF2C69Xq9atPvdDpJr/fLO3e9+0kp9668npJ3k5LtliK1zVOPrzXvJjnTzn1Oak27Sc7znbPNa54bSo6hknk3STnfueuVulb1k3O+rjnvFCXPZ8k2L9nPc65TqWqu9yDv71PUujdoOj7ntV6T3NdEOfdMNe+5+il5HZq7zXOe75LzfVuVnNdK9oecYzD3fF7rOKj5s72S17g52QtOLe9d5U4PAAAAAACgFQQ9AAAAAACAVhD0AAAAAAAAWkHQAwAAAAAAaAVBDwAAAAAAoBUEPQAAAAAAgFYQ9AAAAAAAAFph9mQPuPPOO+Pyyy+P9evXx8aNG+Omm26K0047bfz1c845Jz7/+c9POGb58uVxyy23TCqfbre709d6vd6k0pqMTqfT9/WUvHOWO1VTvftpqldqm9Zctpz6lS21XCnHp5yPXck7pd6pZeunKe+cfaXN57tk2v2OLzk3pKadUvaa16kmNa8VOcdgStlqHt85y5Z7fLd1bklJv+a+1iTn3iI170HuL7Uq2aYlr0Nz9vOar7dyqvk6tOT5blJrfyh5PZaq1jaNKPv5Ws5rh5rl/JwpRc2f9TTJef3d1jm1Sb9yj46O9o0Z/KpJ3+mxdevWOPbYY+PKK6/c6XtOPfXU2Lhx4/jPl770pclmAwAAAAAAMCmTvtNjxYoVsWLFir7vGRoaioULF065UAAAAAAAAJOV5Zked9xxR8yfPz+OOOKIeNe73hWPP/74Tt87NjYWo6OjE34AAAAAAAAma9qDHqeeemp84QtfiNtvvz0+9rGPxdq1a2PFihXxzDPP7PD9q1evjm63O/6zePHi6S4SAAAAAAAwA3R6CU816XQ6z3mQ+a/7n//5nzjssMPitttui5NPPvk5r4+NjcXY2Nj4/0dHRxsDH4P6IPOa1fwA2JrLllOtD5jyIPM8Zur5rlXNc0OTQS57irauFSUf6J2SdqqaH2Se8+HxxueOpbTLIO9DU9OfiWqer2ueW2bqGlrz3FLrGmtem1navG8p+SDzktf+tX7u0KTmz3pKrjVNSq5zuTz7IPORkZEYHh7u+94sf97qVx166KGx3377xQMPPLDD14eGhmJ4eHjCDwAAAAAAwGRlD3o8+uij8fjjj8cBBxyQOysAAAAAAGAGmz3ZA5588skJd2089NBDce+998a8efNi3rx5cemll8YZZ5wRCxcujAcffDDe+973xuGHHx7Lly+ftkLnvK0pRe5bf2q9Da3mW3tz9pVBvoW9Zjn/FEtK3qlqnbci6v2TBjWf75pvM819/KCq9c+CNKn5T5I0qXXf0qTNa2xOJf9MREraJfeKJeWc90qOkZrHZ839oeYxmFK2mv/EVMn1vWa1/tmf3Gr9s5ypar3O3JXXc+adknaTlHqXvP6u+XqsSc1rxaDuc6erTScd9Pje974Xr33ta8f/v2rVqoiIOPvss+Pqq6+O++67Lz7/+c/Hli1bYtGiRXHKKafE3/7t38bQ0NC0FBgAAAAAAGBHJh30OOmkk/pGXL7xjW8kFQgAAAAAAGAqsj/TAwAAAAAAYHcQ9AAAAAAAAFpB0AMAAAAAAGgFQQ8AAAAAAKAVBD0AAAAAAIBWmF26ADszMjISw8PDpYvxHJ1OZ6ev9Xq9bGlPR/op+uXdVO6UtHMb1LxrLndTf0jp56l55zTI46CfQe5rgzrGal4LSpYtdYzlXL9T+mqtYz8if7vkzDtn2il9seS8llvOvWJKuwxym9ZsUNut5v5Qcs4s2S4paecud63rd8n1ua3XPBF177FrVbLcOT8bqHlOzdnmNV9/D/J1aM59asl619qPf5U7PQAAAAAAgFYQ9AAAAAAAAFpB0AMAAAAAAGgFQQ8AAAAAAKAVBD0AAAAAAIBWEPQAAAAAAABaYXbpAky3TqfT9/Ver5d0fE5NZesntd6p6ec6NlXq+U5tt5S8U5Qsd868m9Scd84xVPO8lmJQy50q9/nsl37uNq91PRjkMVZy3mvSr11y70tK7qlS+kPJ81nznJvS5iXHb839fFDnjiZN9cpZ75Lnu+ZrQfPajuU8301yzh1NZSu5fueU83OHmvtxzeM75ZzkHmMp+5aaP+upeR3LqeR+rOY2z3ktuKvc6QEAAAAAALSCoAcAAAAAANAKgh4AAAAAAEArCHoAAAAAAACtIOgBAAAAAAC0gqAHAAAAAADQCoIeAAAAAABAK8wuXYCd6Xa7Uzqu1+tNc0l2n06n0/f1nHVLSbup3Kn6lS0175R6Nx2bej77HZ8775xy9pea65Wzr6Uen3sMT1Xu81lrX6x5/JZch2qeU5u0tT80qXXfklvJfUut81puKW2es14l547cau1ruc/3oO7P29oXa27zJiWvQ5vUOr5zS1lLBrmvpcxrOQ3q52O506/12n2QlWzT1H4+U8fornKnBwAAAAAA0AqCHgAAAAAAQCsIegAAAAAAAK0g6AEAAAAAALSCoAcAAAAAANAKgh4AAAAAAEArCHoAAAAAAACtMHsyb169enXceOONcf/998fee+8dr3zlK+NjH/tYHHHEEePveeqpp+Ld7353XH/99TE2NhbLly+Pq666KhYsWDCpgo2MjMTw8PCkjomI6HQ6fV/v9XqTTnN3KVm2pnYrKWfZUvpLal9LqZd+Xp/U811rXysptdxN7VJrX8x9PlPqnXPuSa1XzWOs3+uD2k93RUq9U9KejvRT5Kx3k5xjMOdakrM/lFwj29zPc+adck5qHmM594q58671nOTeO/RTcnzn3tuXXEtqXb+blBzfTUrukVOUbNPc11u1XhsM6ucGqewVpyZnP58uk7rTY+3atbFy5cq466674tZbb41t27bFKaecElu3bh1/z8UXXxxf+9rX4oYbboi1a9fGY489Fqeffvq0FxwAAAAAAOBXdXoJ4ZWf/vSnMX/+/Fi7dm2ceOKJMTIyEvvvv3+sWbMm3vzmN0dExP333x9HHXVUrFu3Lk444YTGNEdHR6Pb7Ra706PWb7E0manfBG8yU799X+s3RWayQe1rg/wttrb2xZztUvJbTbm1dYzVbFDv9MjZH2o+37WOz13hTo92malz6qDuFZu09VvJ9sg7VvMdyTkN8ucOtd7p0WSQ571a94r62o61tWy565Wrn08mbpD0TI+RkZGIiJg3b15ERKxfvz62bdsWy5YtG3/PkUceGQcddFCsW7duh2mMjY3F6OjohB8AAAAAAIDJmnLQY/v27XHRRRfFq171qjj66KMjImLTpk0xZ86c2GeffSa8d8GCBbFp06YdprN69erodrvjP4sXL55qkQAAAAAAgBlsykGPlStXxve///24/vrrkwpwySWXxMjIyPjPI488kpQeAAAAAAAwM82eykEXXHBB3HzzzXHnnXfGgQceOP77hQsXxtNPPx1btmyZcLfH5s2bY+HChTtMa2hoKIaGhqZSDAAAAAAAgHGTutOj1+vFBRdcEDfddFN885vfjEMOOWTC68cdd1zsueeecfvtt4//bsOGDfHwww/H0qVLp6fEAAAAAAAAOzCpOz1WrlwZa9asia9+9asxd+7c8ed0dLvd2HvvvaPb7cZ5550Xq1atinnz5sXw8HBceOGFsXTp0jjhhBOyVODXNT0Bvunp9CnpN6Wdu2wpaecsW1PaTXK2S0q9U+uVs81Tz3eKnGMsVUq7pLZpSr1y97UUOftaznLXLPd8nXNey2mmli3nGMs9X+dsl5zrd81zbpOS47vWcTDI+5KSSu4lm5S8HrNXfK7c/XhQ9y01X3+X7Oclr+1Lqnl817wW5ZSzn+dU8/V3zv35IO/nar0mqtl0lXtSQY+rr746IiJOOumkCb+/9tpr45xzzomIiE9+8pMxa9asOOOMM2JsbCyWL18eV1111bQUFgAAAAAAYGc6vcrC4aOjo9HtdmNkZCSGh4enPf2ZeqdHk7be6ZEz79xDp9ZvweTup7VG4Gv+tkdJ2mX61fzNwFS1lq3mfjzId3qUNKjfDE7lG3STP7ZJzXd61NrmJfMe5Ds9BlXueg/qfF7zfm5Qxxh51Lo/b1Jyj1xyXhvkMTSo+9Sa92tNBvXz1F0p967EDSb1TA8AAAAAAIBaCXoAAAAAAACtIOgBAAAAAAC0gqAHAAAAAADQCrNLF2C65X6wVs4HqZV80FLOh1OXfEB7k5of4pai5IOSan5wVkrZcj7EqSn9mh+EWPJBiU1q7ov9lHyIW81t3taypY6xFCUfXF9yTs2ddsmH7rb1QYk52zxl798k9xqaU81raK3XYzN1r5hbzn1Lipr3yDnXyJr7Us39oebxXesYyy2lbiUfJl7ymqbmfl5SzfvvQR3f/co9Ojoa3W53l9JxpwcAAAAAANAKgh4AAAAAAEArCHoAAAAAAACtIOgBAAAAAAC0gqAHAAAAAADQCoIeAAAAAABAKwh6AAAAAAAArTC7dAGmotPpZEu71+sVyzsl7aZy5zy+qdypZeuXfsl6p+ZdOv1+Uto8Z39IHX85x3dT2iX7Wsm+lDPvnGnnntdqzTv3Gliy3XKqeQ3NmXfK8YNc7yYl611yj9xP7vGZs81Typ6a96C2eW6DOuemqvWaqEnNbV6yL5X83CFFyTk1t1r32LnHd85r/yYl1++UvJuU/Nww5+cOJfPOKfe8VOvna7mvO2rYU7nTAwAAAAAAaAVBDwAAAAAAoBUEPQAAAAAAgFYQ9AAAAAAAAFpB0AMAAAAAAGgFQQ8AAAAAAKAVBD0AAAAAAIBWmF26ADvT7XandFyv15vmkkxf+p1OJyntpuNzHVta7nPaT792y12ukuesX91S+3FOqWMoZ9lTzmfuetVctpS0m/TLu2S9Sp7v1DFQ8/lOqXfJNh/kvpbr2F2Rc89Ucp1rUnPZ+hnU9blJzvW3Kf2a57UmJfctTQZ1f56SdlP6Tcfm7IuDvG/JOa8N8t6/1rkl93yeU8o5yd2Pa12Da55bShrU/ViTkuO75Gc5uZXcrz3LnR4AAAAAAEArCHoAAAAAAACtIOgBAAAAAAC0gqAHAAAAAADQCoIeAAAAAABAKwh6AAAAAAAArSDoAQAAAAAAtMLsybx59erVceONN8b9998fe++9d7zyla+Mj33sY3HEEUeMv+ekk06KtWvXTjjune98Z1xzzTWTKtjIyEgMDw/v8LVOp7PT4/q9FhHR6/UmVY7JpJ+adk45y9bU5rnPSUreTVLKlpp3iqZyp5Qtdz9PKVtqX0sZ3yX7eWra/Y7PPYZKjpOc4yClP+TuSyXz7qfk+l2y3k1q3luUVHKdq3kNrVXO8Vs675T0B3lOHdQ9cpOax3fJ69BBnbtKXhs0qbmvDercVHJeS5VzTq35eqzmz4JqVfN8XPP4Ljnnlvy8pZ+ar4Gna/xO6k6PtWvXxsqVK+Ouu+6KW2+9NbZt2xannHJKbN26dcL7zj///Ni4ceP4z8c//vFpKSwAAAAAAMDOTOpOj1tuuWXC/6+77rqYP39+rF+/Pk488cTx3z/vec+LhQsXTk8JAQAAAAAAdkHSMz1GRkYiImLevHkTfv/FL34x9ttvvzj66KPjkksuiV/84hc7TWNsbCxGR0cn/AAAAAAAAEzWpO70+FXbt2+Piy66KF71qlfF0UcfPf77t73tbXHwwQfHokWL4r777ov3ve99sWHDhrjxxht3mM7q1avj0ksvnWoxAAAAAAAAIiKi05vik0ne9a53xb/927/Ft7/97TjwwAN3+r5vfvObcfLJJ8cDDzwQhx122HNeHxsbi7GxsfH/j46OxuLFi6f8IPMmJR9kXvOD1FKUfPBOk5ofCpRTzQ/dbpKz3dr6IPOcSj5cumaD/CDzftqc96DuHUoa5HnNg8zbpdbxOx3p51JzuQd5j5ySdqpBXUua5OyrNV9/t3Veq/mzgZxq/txhkOe9FG0dYzWr+Rq41jW0zdffKXZl/PaLGzxrSnd6XHDBBXHzzTfHnXfe2TfgERGxZMmSiIidBj2GhoZiaGhoKsUAAAAAAAAYN6mgR6/XiwsvvDBuuummuOOOO+KQQw5pPObee++NiIgDDjhgSgXcWTl2JjXCXvJb5inHl4zup6r1m/1NBrnNS36TpEnJaHJK3jV/839Qv+UyyN8UK9mXUtNPybvknNtkUPcOJdX8japB/WZwzWrel9Q8t9Taz2v+FmnN+5aav9VY616xSer47qfN63db98g51byO1fy5Q86+llPOdsk9xtx1sPvV+hlYzddEqVL6eb/XR0dHo9vt7lIZJhX0WLlyZaxZsya++tWvxty5c2PTpk0REdHtdmPvvfeOBx98MNasWROvf/3rY99994377rsvLr744jjxxBPjmGOOmUxWAAAAAAAAkzKpoMfVV18dEREnnXTShN9fe+21cc4558ScOXPitttuiyuuuCK2bt0aixcvjjPOOCM+8IEPTFuBAQAAAAAAdmTSf96qn8WLF8fatWuTCgQAAAAAADAVs0oXAAAAAAAAYDoIegAAAAAAAK0g6AEAAAAAALSCoAcAAAAAANAKk3qQ+e7U7XZLF2GHmh7m3k+n00lKu+n4WvNOabOmvFPLnVq2FDn7Usl658w7pR+m5p0qJe/c/bzf8altnqLk+UqVs91qnq9T0s+9TuVcQ5uk1Dsl7VSp5yTn+c45DpqU3I+lqLmvpcrZ11IMcj8vec1T8/VWrfN5aj+u+To017ERdV+3pBjU+brJIK9jNc/ng/o5U0m559x+au4rNc89g9rXchrka/9d5U4PAAAAAACgFQQ9AAAAAACAVhD0AAAAAAAAWkHQAwAAAAAAaAVBDwAAAAAAoBUEPQAAAAAAgFYQ9AAAAAAAAFphdukCTEWv1yuWdqfTmfKxOaWUe1der7luKcc21Ttn2rX246bjc/eFlHNSUlvPd+r5KDmn5qx3atlKtnmKnGXLOV83yT2fp/Tz1Hq3dd+Sc2/QpOY9Uz+DXO+c+5aSc0uTkvvUQW3zVDXvW3JqKnut81qqkvvzJin9oa3zWpOS/bTm/XtOJa9DS+4lS/bjmbp/r3nuGNT+kHtvX8N1qDs9AAAAAACAVhD0AAAAAAAAWkHQAwAAAAAAaAVBDwAAAAAAoBUEPQAAAAAAgFYQ9AAAAAAAAFpB0AMAAAAAAGiF2aULsDMjIyMxPDy8w9c6nc6U0+31elM+NlVT3in1yp13zrI1STlnOcudmvagtmlJucs9qHNLk5Jla2rTfmVLHSM1z7m5ji0tZ5vnbJeUfpo771QpYyy13rWuc4Nc75x7oqa0S+adkn7OepVWsp+nSF0rcu4dSs5bKXLveVLGWEraqXL2tVQ590S599C5ji1tUNeSkue7ZNolx2/J68ycam7TJiltXnKP3CTn+M49b9WwT3WnBwAAAAAA0AqCHgAAAAAAQCsIegAAAAAAAK0g6AEAAAAAALSCoAcAAAAAANAKgh4AAAAAAEArzJ7Mm6+++uq4+uqr40c/+lFERLz0pS+ND33oQ7FixYqIiHjqqafi3e9+d1x//fUxNjYWy5cvj6uuuioWLFgw6YJ1u91JHxMR0ev1pnTcszqdzpTTTzl2V15vSj/l2JS8c7d5St45y5aadoqUvpCafmo/zX3OUtR6TkvOayXlrveg5l3z+cw5Nw1yvWst2yD3tRS5yz2o7TLIcrZ5ybUk5fiax3fqWlHymihFar1zXofmbJea27xkvVLOSclrwSYl99BNUs5J7javde+Qe/wOal+r+TONknNuWz/LqblNc36e2iSlXXZXX5nUnR4HHnhgXHbZZbF+/fr43ve+F6973evijW98Y/zgBz+IiIiLL744vva1r8UNN9wQa9eujcceeyxOP/30LAUHAAAAAAD4VZ1eYnhl3rx5cfnll8eb3/zm2H///WPNmjXx5je/OSIi7r///jjqqKNi3bp1ccIJJ+xSeqOjo1O+yyNisO/0aFLy25olo4P95I4ODuqdHoPc12aqWiPwg/ytprZ+c6jmb0yWVPLbmjXPe7XuHVLV2qYR9a5zNc8dNY+xmvepKQa1TXOrud7u9Kgr75q/AV9SzfXOeb5TlSxbrZ9xNRnkvHOq+TPNQVVzu9Q8hnK1y7Nxg5GRkRgeHu773ik/0+OZZ56J66+/PrZu3RpLly6N9evXx7Zt22LZsmXj7znyyCPjoIMOinXr1k01GwAAAAAAgF0yqWd6RET853/+ZyxdujSeeuqpeMELXhA33XRT/OZv/mbce++9MWfOnNhnn30mvH/BggWxadOmnaY3NjYWY2Nj4/8fHR2dbJEAAAAAAAAmf6fHEUccEffee2/cfffd8a53vSvOPvvs+K//+q8pF2D16tXR7XbHfxYvXjzltAAAAAAAgJlr0kGPOXPmxOGHHx7HHXdcrF69Oo499tj4x3/8x1i4cGE8/fTTsWXLlgnv37x5cyxcuHCn6V1yySUxMjIy/vPII49MuhIAAAAAAABTfqbHs7Zv3x5jY2Nx3HHHxZ577hm33377+GsbNmyIhx9+OJYuXbrT44eGhmJ4eHjCDwAAAAAAwGRN6pkel1xySaxYsSIOOuigeOKJJ2LNmjVxxx13xDe+8Y3odrtx3nnnxapVq2LevHkxPDwcF154YSxdujROOOGESRes31PYU55O36Tp6fL98k45dlc0pZ/r2Kbjm+rVlHdq2fop2eZNUtotZ7ma0s85/tosdZzUKrXcKf0pdW6pdS3JLWUdS61XSn9pSrvkGKt5/Na6d2hS8nynpj2oe8WceZecO1LlPGc52zxnP96V43MdG5F371Byfa55vk6Z15ianNdjg3rOcs8dJa+/S16XpBw7yH2xZN4559Sa5/Oc63eK3PvQkp95pMi9l9wdJhX0+MlPfhJvf/vbY+PGjdHtduOYY46Jb3zjG/F7v/d7ERHxyU9+MmbNmhVnnHFGjI2NxfLly+Oqq67KUnAAAAAAAIBf1elVFuYfHR2Nbrc75Ts9SkYm2xwF76fmb+fV3OY1t1s/NbdpzUqe75S8B/lbDyl5D/JaUvLbPSW/fd+kZD9n8mq+42FQ7/TI/W2rkm1e0qCe70G++8adHrtfyX1LiprvCkzR5uuxQd2npqr1To8mbe6LOQ3qdWjOvJsM6lqxK+nnzDtF7Xd69IsbPCv5mR4AAAAAAAA1EPQAAAAAAABaQdADAAAAAABoBUEPAAAAAACgFWaXLsCve/ZBKKOjo1M6fqrHTYfceZesW4pBLXdEu/tTKW2tV6pB7WuDWu6SaefOu+bzOajtat4aPLXOD8ZYfWnnVPP5LrnWlFRzuWsuWz+D3Jdm6rxVc9n6qXlOrVnN9aq5bKW0dU5MNVPbxTnZuV15yHunV/JR8Dvw6KOPxuLFi0sXAwAAAAAAqMgjjzwSBx54YN/3VBf02L59ezz22GMxd+7c6HQ6MTo6GosXL45HHnkkhoeHSxcPijAOwDgAYwCMA4gwDiDCOABjgJmo1+vFE088EYsWLYpZs/o/taO6P281a9asHUZqhoeHDWJmPOMAjAMwBsA4gAjjACKMAzAGmGm63e4uvc+DzAEAAAAAgFYQ9AAAAAAAAFqh+qDH0NBQfPjDH46hoaHSRYFijAMwDsAYAOMAIowDiDAOwBiA/qp7kDkAAAAAAMBUVH+nBwAAAAAAwK4Q9AAAAAAAAFpB0AMAAAAAAGgFQQ8AAAAAAKAVqg96XHnllfGiF70o9tprr1iyZEn8x3/8R+kiQRarV6+OV7ziFTF37tyYP39+nHbaabFhw4YJ7znppJOi0+lM+PnzP//zQiWG6feRj3zkOX38yCOPHH/9qaeeipUrV8a+++4bL3jBC+KMM86IzZs3FywxTL8XvehFzxkHnU4nVq5cGRHWAtrpzjvvjDe84Q2xaNGi6HQ68ZWvfGXC671eLz70oQ/FAQccEHvvvXcsW7YsfvjDH054z89//vM466yzYnh4OPbZZ58477zz4sknn9yNtYCp6zcGtm3bFu973/viZS97WTz/+c+PRYsWxdvf/vZ47LHHJqSxo/Xjsssu2801galrWgvOOeec5/TxU089dcJ7rAUMuqZxsKPrhE6nE5dffvn4e6wHUHnQ48tf/nKsWrUqPvzhD8c999wTxx57bCxfvjx+8pOflC4aTLu1a9fGypUr46677opbb701tm3bFqecckps3bp1wvvOP//82Lhx4/jPxz/+8UIlhjxe+tKXTujj3/72t8dfu/jii+NrX/ta3HDDDbF27dp47LHH4vTTTy9YWph+3/3udyeMgVtvvTUiIv7wD/9w/D3WAtpm69atceyxx8aVV165w9c//vGPx6c+9am45ppr4u67747nP//5sXz58njqqafG33PWWWfFD37wg7j11lvj5ptvjjvvvDPe8Y537K4qQJJ+Y+AXv/hF3HPPPfHBD34w7rnnnrjxxhtjw4YN8Qd/8AfPee/f/M3fTFgfLrzwwt1RfJgWTWtBRMSpp546oY9/6UtfmvC6tYBB1zQOfrX/b9y4MT73uc9Fp9OJM844Y8L7rAfMdLNLF6CfT3ziE3H++efHueeeGxER11xzTXz961+Pz33uc/H+97+/cOlget1yyy0T/n/dddfF/PnzY/369XHiiSeO//55z3teLFy4cHcXD3ab2bNn77CPj4yMxGc/+9lYs2ZNvO51r4uIiGuvvTaOOuqouOuuu+KEE07Y3UWFLPbff/8J/7/sssvisMMOi9/93d8d/521gLZZsWJFrFixYoev9Xq9uOKKK+IDH/hAvPGNb4yIiC984QuxYMGC+MpXvhJvfetb47//+7/jlltuie9+97vx8pe/PCIiPv3pT8frX//6+Pu///tYtGjRbqsLTEW/MdDtdscD4M/6zGc+E8cff3w8/PDDcdBBB43/fu7cudYHBla/cfCsoaGhnfZxawFt0DQOfr3/f/WrX43Xvva1ceihh074vfWAma7aOz2efvrpWL9+fSxbtmz8d7NmzYply5bFunXrCpYMdo+RkZGIiJg3b96E33/xi1+M/fbbL44++ui45JJL4he/+EWJ4kE2P/zhD2PRokVx6KGHxllnnRUPP/xwRESsX78+tm3bNmFdOPLII+Oggw6yLtBaTz/9dPzLv/xL/Omf/ml0Op3x31sLmEkeeuih2LRp04T5v9vtxpIlS8bn/3Xr1sU+++wz/iFXRMSyZcti1qxZcffdd+/2MkNuIyMj0el0Yp999pnw+8suuyz23Xff+O3f/u24/PLL45e//GWZAkImd9xxR8yfPz+OOOKIeNe73hWPP/74+GvWAmaazZs3x9e//vU477zznvOa9YCZrto7PX72s5/FM888EwsWLJjw+wULFsT9999fqFSwe2zfvj0uuuiieNWrXhVHH330+O/f9ra3xcEHHxyLFi2K++67L973vvfFhg0b4sYbbyxYWpg+S5Ysieuuuy6OOOKI2LhxY1x66aXxmte8Jr7//e/Hpk2bYs6cOc+5uF+wYEFs2rSpTIEhs6985SuxZcuWOOecc8Z/Zy1gpnl2jt/RdcGzr23atCnmz58/4fXZs2fHvHnzrBG0zlNPPRXve9/74swzz4zh4eHx3//lX/5l/M7v/E7MmzcvvvOd78Qll1wSGzdujE984hMFSwvT59RTT43TTz89DjnkkHjwwQfjr//6r2PFihWxbt262GOPPawFzDif//znY+7cuc/5k8/WA6g46AEz2cqVK+P73//+hGcZRMSEv0X6spe9LA444IA4+eST48EHH4zDDjtsdxcTpt2v3sZ7zDHHxJIlS+Lggw+Of/3Xf4299967YMmgjM9+9rOxYsWKCX+OwVoAMHNt27Yt/uiP/ih6vV5cffXVE15btWrV+L+POeaYmDNnTrzzne+M1atXx9DQ0O4uKky7t771reP/ftnLXhbHHHNMHHbYYXHHHXfEySefXLBkUMbnPve5OOuss2Kvvfaa8HvrAVT8563222+/2GOPPWLz5s0Tfr9582Z/k45Wu+CCC+Lmm2+Ob33rW3HggQf2fe+SJUsiIuKBBx7YHUWD3W6fffaJl7zkJfHAAw/EwoUL4+mnn44tW7ZMeI91gbb68Y9/HLfddlv82Z/9Wd/3WQtou2fn+H7XBQsXLoyf/OQnE17/5S9/GT//+c+tEbTGswGPH//4x3HrrbdOuMtjR5YsWRK//OUv40c/+tHuKSDsZoceemjst99+43sgawEzyb//+7/Hhg0bGq8VIqwHzEzVBj3mzJkTxx13XNx+++3jv9u+fXvcfvvtsXTp0oIlgzx6vV5ccMEFcdNNN8U3v/nNOOSQQxqPuffeeyMi4oADDshcOijjySefjAcffDAOOOCAOO6442LPPfecsC5s2LAhHn74YesCrXTttdfG/Pnz4/d///f7vs9aQNsdcsghsXDhwgnz/+joaNx9993j8//SpUtjy5YtsX79+vH3fPOb34zt27ePBwZhkD0b8PjhD38Yt912W+y7776Nx9x7770xa9as5/y5H2iLRx99NB5//PHxPZC1gJnks5/9bBx33HFx7LHHNr7XesBMVPWft1q1alWcffbZ8fKXvzyOP/74uOKKK2Lr1q1x7rnnli4aTLuVK1fGmjVr4qtf/WrMnTt3/G+Odrvd2HvvvePBBx+MNWvWxOtf//rYd99947777ouLL744TjzxxDjmmGMKlx6mx3ve8554wxveEAcffHA89thj8eEPfzj22GOPOPPMM6Pb7cZ5550Xq1atinnz5sXw8HBceOGFsXTp0jjhhBNKFx2m1fbt2+Paa6+Ns88+O2bP/n/bNWsBbfXkk09OuFvpoYceinvvvTfmzZsXBx10UFx00UXx0Y9+NF784hfHIYccEh/84Adj0aJFcdppp0VExFFHHRWnnnpqnH/++XHNNdfEtm3b4oILLoi3vvWtE/48HNSq3xg44IAD4s1vfnPcc889cfPNN8czzzwzfq0wb968mDNnTqxbty7uvvvueO1rXxtz586NdevWxcUXXxx//Md/HC984QtLVQsmpd84mDdvXlx66aVxxhlnxMKFC+PBBx+M9773vXH44YfH8uXLI8JaQDs07Yki/u/LHzfccEP8wz/8w3OOtx7A/69XuU9/+tO9gw46qDdnzpze8ccf37vrrrtKFwmyiIgd/lx77bW9Xq/Xe/jhh3snnnhib968eb2hoaHe4Ycf3vurv/qr3sjISNmCwzR6y1ve0jvggAN6c+bM6f3Gb/xG7y1veUvvgQceGH/9f//3f3t/8Rd/0XvhC1/Ye97zntd705ve1Nu4cWPBEkMe3/jGN3oR0duwYcOE31sLaKtvfetbO9wHnX322b1er9fbvn1774Mf/GBvwYIFvaGhod7JJ5/8nPHx+OOP984888zeC17wgt7w8HDv3HPP7T3xxBMFagOT128MPPTQQzu9VvjWt77V6/V6vfXr1/eWLFnS63a7vb322qt31FFH9f7u7/6u99RTT5WtGExCv3Hwi1/8onfKKaf09t9//96ee+7ZO/jgg3vnn39+b9OmTRPSsBYw6Jr2RL1er/dP//RPvb333ru3ZcuW5xxvPYD/0+n1er3skRUAAAAAAIDMqn2mBwAAAAAAwGQIegAAAAAAAK0g6AEAAAAAALSCoAcAAAAAANAKgh4AAAAAAEArCHoAAAAAAACtIOgBAAAAAAC0gqAHAAAAAADQCoIeAAAAAABAKwh6AAAAAAAArSDoAQAAAAAAtIKgBwAAAAAA0Ar/H0bIUyoqoFkSAAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 2000x1000 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(20, 10))\n",
    "plt.imshow(h.abs() > 0.99, cmap='gray', interpolation='nearest') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the plot above, we see that a lot of it is white, which means lot of the neurons are in the the flat area of tanh. We would be in a lot of trouble if for any of these neurons we have an entire column white. That would be what we call a \"dead neuron\".\n",
    "\n",
    "A dead neuron is a neuron for which none of the examples land in active and non-flat area of tanh. Such a neuron never learns anything from the examples and is the machine learning equivalent of the biological term - \"brain dead\"\n",
    "\n",
    "Another interesting thing to note here is that, the neurons can become dead during optimization as well. For instance when learning rate is very high, it can result in preactivation(hpreact) values to tip over to the flat zone, thereby making the neuron dead from that moment onwards.\n",
    "\n",
    "---\n",
    "\n",
    "As we have seen the issue arises due to high values of hpreact, so we can make use of the approach we used earlier and decrease the Weights W1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11697\n"
     ]
    }
   ],
   "source": [
    "n_embd = 10 # the dimensionality of the character embedding vectors\n",
    "n_hidden = 200 # the number of neurons in the hidden layer of the MLP\n",
    "\n",
    "g = torch.Generator().manual_seed(2147483647) # for reproducibility\n",
    "C  = torch.randn((vocab_size, n_embd),            generator=g)\n",
    "W1 = torch.randn((n_embd * block_size, n_hidden), generator=g) * 0.2\n",
    "W2 = torch.randn((n_hidden, vocab_size),          generator=g) * 0.01\n",
    "b2 = torch.randn(vocab_size,                      generator=g) \n",
    "\n",
    "parameters = [C, W1, W2, b2]\n",
    "print(sum(p.nelement() for p in parameters)) # number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.3189\n"
     ]
    }
   ],
   "source": [
    "logits, h, hpreact = run_training_loop(break_on_first=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([160., 168., 172., 157., 165., 139., 146., 123., 116., 122., 108.,\n",
       "        105.,  91., 108., 104.,  97., 110.,  83.,  99., 114., 106.,  97.,\n",
       "         98.,  90., 113., 108., 111., 120., 104.,  93., 104., 130.,  98.,\n",
       "        117., 108., 118., 109., 109., 121., 106., 133., 126., 140., 161.,\n",
       "        156., 171., 196., 207., 238., 225.]),\n",
       " array([-9.99249458e-01, -9.59265115e-01, -9.19280772e-01, -8.79296429e-01,\n",
       "        -8.39312086e-01, -7.99327743e-01, -7.59343400e-01, -7.19359057e-01,\n",
       "        -6.79374714e-01, -6.39390371e-01, -5.99406028e-01, -5.59421685e-01,\n",
       "        -5.19437342e-01, -4.79452999e-01, -4.39468656e-01, -3.99484313e-01,\n",
       "        -3.59499969e-01, -3.19515626e-01, -2.79531283e-01, -2.39546940e-01,\n",
       "        -1.99562597e-01, -1.59578254e-01, -1.19593911e-01, -7.96095681e-02,\n",
       "        -3.96252251e-02,  3.59117985e-04,  4.03434610e-02,  8.03278041e-02,\n",
       "         1.20312147e-01,  1.60296490e-01,  2.00280833e-01,  2.40265176e-01,\n",
       "         2.80249519e-01,  3.20233862e-01,  3.60218205e-01,  4.00202549e-01,\n",
       "         4.40186892e-01,  4.80171235e-01,  5.20155578e-01,  5.60139921e-01,\n",
       "         6.00124264e-01,  6.40108607e-01,  6.80092950e-01,  7.20077293e-01,\n",
       "         7.60061636e-01,  8.00045979e-01,  8.40030322e-01,  8.80014665e-01,\n",
       "         9.19999008e-01,  9.59983351e-01,  9.99967694e-01]),\n",
       " <BarContainer object of 50 artists>)"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAGdCAYAAAA44ojeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAiqklEQVR4nO3de3DU1f3/8VcuZLnuxhiSkBqQm1wkXIQSQqlQyZCEiDjQqSBjg2WgpUGrUZS0CoJ2QGXUkUGxDhedEak4CoqAIgqoDSARFLkNoUGgmCCkZJMggZDz+6Pf7M81AbLJJnuyPB8zO5P9fM5+9rz3ZLOvnM9lQ4wxRgAAABYJDXQHAAAAfo6AAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwTnigO1AfVVVVOnnypNq1a6eQkJBAdwcAANSBMUalpaWKj49XaOiV50iaZUA5efKkEhISAt0NAABQD8ePH9cNN9xwxTbNMqC0a9dO0v8KdDqdAe4NAACoC7fbrYSEBM/n+JU0y4BSvVvH6XQSUAAAaGbqcngGB8kCAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWCc80B0AAAD+c+OsD67a5uiCjCboScMwgwIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrhAe6AwAAoGndOOuDq7Y5uiCjCXpyecygAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA64QHugMAAODqbpz1QaC70KSYQQEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB2fAsr8+fP1y1/+Uu3atVNMTIzuvPNOHTp0yKvN+fPnlZWVpeuvv15t27bV+PHjVVRU5NXm2LFjysjIUOvWrRUTE6OZM2eqsrKy4dUAAICg4FNA2bp1q7KysrR9+3Zt2rRJFy9e1KhRo1ReXu5p8+CDD+r999/X6tWrtXXrVp08eVLjxo3zrL906ZIyMjJ04cIF/etf/9Jrr72mFStWaPbs2f6rCgAANGshxhhT3wf/8MMPiomJ0datW3XrrbeqpKRE7du318qVK/Xb3/5WknTw4EH16tVLubm5GjJkiDZs2KDbb79dJ0+eVGxsrCRpyZIlevTRR/XDDz8oIiLiqs/rdrvlcrlUUlIip9NZ3+4DANBsNPV1UI4uyPD7Nn35/G7QMSglJSWSpKioKElSXl6eLl68qJSUFE+bnj17qmPHjsrNzZUk5ebmKjEx0RNOJCk1NVVut1v79u2r9XkqKirkdru9bgAAIHjVO6BUVVXpgQce0K9+9Sv16dNHklRYWKiIiAhFRkZ6tY2NjVVhYaGnzU/DSfX66nW1mT9/vlwul+eWkJBQ324DAIBmoN4BJSsrS99++61WrVrlz/7UKicnRyUlJZ7b8ePHG/05AQBA4NTru3hmzJihdevWadu2bbrhhhs8y+Pi4nThwgWdPXvWaxalqKhIcXFxnjY7d+702l71WT7VbX7O4XDI4XDUp6sAAKAZ8mkGxRijGTNm6N1339Unn3yizp07e60fOHCgWrRooc2bN3uWHTp0SMeOHVNycrIkKTk5WXv37tWpU6c8bTZt2iSn06nevXs3pBYAABAkfJpBycrK0sqVK7V27Vq1a9fOc8yIy+VSq1at5HK5NGXKFGVnZysqKkpOp1P33XefkpOTNWTIEEnSqFGj1Lt3b91zzz165plnVFhYqMcee0xZWVnMkgAAAEk+BpSXX35ZkjRixAiv5cuXL9fkyZMlSc8//7xCQ0M1fvx4VVRUKDU1VS+99JKnbVhYmNatW6fp06crOTlZbdq0UWZmpubNm9ewSgAAQNBo0HVQAoXroAAAgklTX+OkLpr1dVAAAAAaAwEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKwTHugOBLMbZ31w1TZHF2Q0QU8AAGheCCgAADSiuvyziprYxQMAAKxDQAEAANYhoAAAAOsQUAAAgHU4SLaeOOgJAIDGwwwKAACwDgEFAABYh4ACAACswzEozQBXpAUAXGuYQQEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB3O4gEAoBacQRlYzKAAAADrMINSC75nBwCAwGIGBQAAWIeAAgAArENAAQAA1uEYlADjeBcAAGpiBgUAAFiHGRQAAOqJWfDGwwwKAACwDgEFAABYh4ACAACs43NA2bZtm8aMGaP4+HiFhIRozZo1XusnT56skJAQr1taWppXm+LiYk2aNElOp1ORkZGaMmWKysrKGlQIAAAIHj4HlPLycvXr10+LFy++bJu0tDR9//33ntubb77ptX7SpEnat2+fNm3apHXr1mnbtm2aNm2a770HAABByeezeNLT05Wenn7FNg6HQ3FxcbWuO3DggDZu3Kgvv/xSgwYNkiQtWrRIo0eP1sKFCxUfH+9rlwAAQJBplGNQtmzZopiYGPXo0UPTp0/XmTNnPOtyc3MVGRnpCSeSlJKSotDQUO3YsaPW7VVUVMjtdnvdAABA8PJ7QElLS9Prr7+uzZs36+mnn9bWrVuVnp6uS5cuSZIKCwsVExPj9Zjw8HBFRUWpsLCw1m3Onz9fLpfLc0tISPB3twEAgEX8fqG2CRMmeH5OTExU37591bVrV23ZskUjR46s1zZzcnKUnZ3tue92uwkpAAAEsUY/zbhLly6Kjo5Wfn6+JCkuLk6nTp3yalNZWani4uLLHrficDjkdDq9bgAAIHg1ekA5ceKEzpw5ow4dOkiSkpOTdfbsWeXl5XnafPLJJ6qqqlJSUlJjdwcAADQDPu/iKSsr88yGSFJBQYH27NmjqKgoRUVFae7cuRo/frzi4uJ05MgRPfLII+rWrZtSU1MlSb169VJaWpqmTp2qJUuW6OLFi5oxY4YmTJjAGTwAAEBSPWZQdu3apQEDBmjAgAGSpOzsbA0YMECzZ89WWFiYvvnmG91xxx266aabNGXKFA0cOFCfffaZHA6HZxtvvPGGevbsqZEjR2r06NEaNmyY/vGPf/ivKgAA0Kz5PIMyYsQIGWMuu/7DDz+86jaioqK0cuVKX58aDVSXb908uiCjCXoCAMCV8V08AADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1/P5txgAA2K4uF65EYDGDAgAArMMMSpDgvwEAQDBhBgUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4XagMANBt1uSjl0QUZTdATNDZmUAAAgHUIKAAAwDoEFAAAYB2OQQGAJsLxE0DdMYMCAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6nMUDAAgqdTlbCvZjBgUAAFiHgAIAAKxDQAEAANbhGBR44UqXAAAbMIMCAACswwwKGgUzMQCAhiCgAAAaHf+0wFfs4gEAANZhBgVA0OK/dqD5YgYFAABYh4ACAACswy4eALhGsQsMNiOgAEAzQ7DAtYBdPAAAwDrMoMBnfJU5AKCxMYMCAACsQ0ABAADWYRcP0IxxsCSAYEVAAdCkCFUA6oJdPAAAwDoEFAAAYB128QBoljjdHQhuzKAAAADrMIMC+BkHgQJAwxFQAACXxa40BAq7eAAAgHWYQUHANOWuEHa7NC/8195wvIZo7phBAQAA1iGgAAAA67CLBwhy7N5Cc8FuKfwUAQVW48MVAK5NBBTg/xCGrk3+Gnf++wf8i2NQAACAdXwOKNu2bdOYMWMUHx+vkJAQrVmzxmu9MUazZ89Whw4d1KpVK6WkpOjw4cNebYqLizVp0iQ5nU5FRkZqypQpKisra1AhAAAgePgcUMrLy9WvXz8tXry41vXPPPOMXnzxRS1ZskQ7duxQmzZtlJqaqvPnz3vaTJo0Sfv27dOmTZu0bt06bdu2TdOmTat/FQAAIKj4fAxKenq60tPTa11njNELL7ygxx57TGPHjpUkvf7664qNjdWaNWs0YcIEHThwQBs3btSXX36pQYMGSZIWLVqk0aNHa+HChYqPj29AOQAAIBj49RiUgoICFRYWKiUlxbPM5XIpKSlJubm5kqTc3FxFRkZ6wokkpaSkKDQ0VDt27Kh1uxUVFXK73V43AAAQvPwaUAoLCyVJsbGxXstjY2M96woLCxUTE+O1Pjw8XFFRUZ42Pzd//ny5XC7PLSEhwZ/dBgAAlmkWZ/Hk5OSopKTEczt+/HiguwQAABqRX6+DEhcXJ0kqKipShw4dPMuLiorUv39/T5tTp055Pa6yslLFxcWex/+cw+GQw+HwZ1cB63FdDQDXMr8GlM6dOysuLk6bN2/2BBK3260dO3Zo+vTpkqTk5GSdPXtWeXl5GjhwoCTpk08+UVVVlZKSkvzZHQB+xIXsmgbBFPgfnwNKWVmZ8vPzPfcLCgq0Z88eRUVFqWPHjnrggQf01FNPqXv37urcubMef/xxxcfH684775Qk9erVS2lpaZo6daqWLFmiixcvasaMGZowYQJn8AAAAEn1CCi7du3Sb37zG8/97OxsSVJmZqZWrFihRx55ROXl5Zo2bZrOnj2rYcOGaePGjWrZsqXnMW+88YZmzJihkSNHKjQ0VOPHj9eLL77oh3KAxhWs/90Ga10Ami+fA8qIESNkjLns+pCQEM2bN0/z5s27bJuoqCitXLnS16cGAADXiGZxFg8AALi2EFAAAIB1CCgAAMA6BBQAAGAdv14HBQCCEWc5AU2PGRQAAGAdAgoAALAOu3jQ7DH9DgDBh4ACwG8IiwD8hV08AADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdLnUPBACXhAeAK2MGBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANbxe0B54oknFBIS4nXr2bOnZ/358+eVlZWl66+/Xm3bttX48eNVVFTk724AAIBmrFFmUG6++WZ9//33ntvnn3/uWffggw/q/fff1+rVq7V161adPHlS48aNa4xuAACAZiq8UTYaHq64uLgay0tKSrR06VKtXLlSt912myRp+fLl6tWrl7Zv364hQ4Y0RncAAEAz0ygzKIcPH1Z8fLy6dOmiSZMm6dixY5KkvLw8Xbx4USkpKZ62PXv2VMeOHZWbm3vZ7VVUVMjtdnvdAABA8PJ7QElKStKKFSu0ceNGvfzyyyooKNCvf/1rlZaWqrCwUBEREYqMjPR6TGxsrAoLCy+7zfnz58vlcnluCQkJ/u42AACwiN938aSnp3t+7tu3r5KSktSpUye99dZbatWqVb22mZOTo+zsbM99t9tNSAEAIIg1+mnGkZGRuummm5Sfn6+4uDhduHBBZ8+e9WpTVFRU6zEr1RwOh5xOp9cNAAAEr0YPKGVlZTpy5Ig6dOiggQMHqkWLFtq8ebNn/aFDh3Ts2DElJyc3dlcAAEAz4fddPA8//LDGjBmjTp066eTJk5ozZ47CwsI0ceJEuVwuTZkyRdnZ2YqKipLT6dR9992n5ORkzuABAAAefg8oJ06c0MSJE3XmzBm1b99ew4YN0/bt29W+fXtJ0vPPP6/Q0FCNHz9eFRUVSk1N1UsvveTvbgAAgGYsxBhjAt0JX7ndbrlcLpWUlDTK8Sg3zvrA79sEAKA5Obogw+/b9OXzm+/iAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANYhoAAAAOsQUAAAgHUIKAAAwDoEFAAAYB0CCgAAsA4BBQAAWIeAAgAArENAAQAA1iGgAAAA6xBQAACAdQgoAADAOgQUAABgHQIKAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFgnoAFl8eLFuvHGG9WyZUslJSVp586dgewOAACwRMACyj//+U9lZ2drzpw5+uqrr9SvXz+lpqbq1KlTgeoSAACwRMACynPPPaepU6fq3nvvVe/evbVkyRK1bt1ay5YtC1SXAACAJcID8aQXLlxQXl6ecnJyPMtCQ0OVkpKi3NzcGu0rKipUUVHhuV9SUiJJcrvdjdK/qopzjbJdAACai8b4jK3epjHmqm0DElBOnz6tS5cuKTY21mt5bGysDh48WKP9/PnzNXfu3BrLExISGq2PAABcy1wvNN62S0tL5XK5rtgmIAHFVzk5OcrOzvbcr6qqUnFxsa6//nqFhIT47XncbrcSEhJ0/PhxOZ1Ov23XJsFeY7DXJwV/jdTX/AV7jcFen9R4NRpjVFpaqvj4+Ku2DUhAiY6OVlhYmIqKiryWFxUVKS4urkZ7h8Mhh8PhtSwyMrLR+ud0OoP2l65asNcY7PVJwV8j9TV/wV5jsNcnNU6NV5s5qRaQg2QjIiI0cOBAbd682bOsqqpKmzdvVnJyciC6BAAALBKwXTzZ2dnKzMzUoEGDNHjwYL3wwgsqLy/XvffeG6guAQAASwQsoNx111364YcfNHv2bBUWFqp///7auHFjjQNnm5LD4dCcOXNq7E4KJsFeY7DXJwV/jdTX/AV7jcFen2RHjSGmLuf6AAAANCG+iwcAAFiHgAIAAKxDQAEAANYhoAAAAOtccwHl73//u4YOHarWrVvX+WJvxhjNnj1bHTp0UKtWrZSSkqLDhw97tSkuLtakSZPkdDoVGRmpKVOmqKysrBEquDJf+3H06FGFhITUelu9erWnXW3rV61a1RQl1VCf13rEiBE1+v+nP/3Jq82xY8eUkZGh1q1bKyYmRjNnzlRlZWVjllIrX+srLi7Wfffdpx49eqhVq1bq2LGj7r//fs93VlUL5BguXrxYN954o1q2bKmkpCTt3Lnziu1Xr16tnj17qmXLlkpMTNT69eu91tflPdmUfKnv1Vdf1a9//Wtdd911uu6665SSklKj/eTJk2uMVVpaWmOXcVm+1LdixYoafW/ZsqVXG9vGT/Ktxtr+noSEhCgjI8PTxqYx3LZtm8aMGaP4+HiFhIRozZo1V33Mli1bdMstt8jhcKhbt25asWJFjTa+vq99Zq4xs2fPNs8995zJzs42LperTo9ZsGCBcblcZs2aNebrr782d9xxh+ncubP58ccfPW3S0tJMv379zPbt281nn31munXrZiZOnNhIVVyer/2orKw033//vddt7ty5pm3btqa0tNTTTpJZvny5V7uf1t+U6vNaDx8+3EydOtWr/yUlJZ71lZWVpk+fPiYlJcXs3r3brF+/3kRHR5ucnJzGLqcGX+vbu3evGTdunHnvvfdMfn6+2bx5s+nevbsZP368V7tAjeGqVatMRESEWbZsmdm3b5+ZOnWqiYyMNEVFRbW2/+KLL0xYWJh55plnzP79+81jjz1mWrRoYfbu3etpU5f3ZFPxtb67777bLF682OzevdscOHDATJ482bhcLnPixAlPm8zMTJOWluY1VsXFxU1Vkhdf61u+fLlxOp1efS8sLPRqY9P4GeN7jWfOnPGq79tvvzVhYWFm+fLlnjY2jeH69evN3/72N/POO+8YSebdd9+9Yvt///vfpnXr1iY7O9vs37/fLFq0yISFhZmNGzd62vj6mtXHNRdQqi1fvrxOAaWqqsrExcWZZ5991rPs7NmzxuFwmDfffNMYY8z+/fuNJPPll1962mzYsMGEhISY//znP37v++X4qx/9+/c3f/jDH7yW1eWXuinUt8bhw4ebv/zlL5ddv379ehMaGur1h/Tll182TqfTVFRU+KXvdeGvMXzrrbdMRESEuXjxomdZoMZw8ODBJisry3P/0qVLJj4+3syfP7/W9r/73e9MRkaG17KkpCTzxz/+0RhTt/dkU/K1vp+rrKw07dq1M6+99ppnWWZmphk7dqy/u1ovvtZ3tb+tto2fMQ0fw+eff960a9fOlJWVeZbZNIY/VZe/A4888oi5+eabvZbdddddJjU11XO/oa9ZXVxzu3h8VVBQoMLCQqWkpHiWuVwuJSUlKTc3V5KUm5uryMhIDRo0yNMmJSVFoaGh2rFjR5P11R/9yMvL0549ezRlypQa67KyshQdHa3Bgwdr2bJldfq6bH9rSI1vvPGGoqOj1adPH+Xk5OjcuXNe201MTPS6UGBqaqrcbrf27dvn/0Iuw1+/SyUlJXI6nQoP974WY1OP4YULF5SXl+f1/gkNDVVKSorn/fNzubm5Xu2l/41Fdfu6vCebSn3q+7lz587p4sWLioqK8lq+ZcsWxcTEqEePHpo+fbrOnDnj177XRX3rKysrU6dOnZSQkKCxY8d6vYdsGj/JP2O4dOlSTZgwQW3atPFabsMY1sfV3oP+eM3qoll8m3EgFRYWSlKNK9zGxsZ61hUWFiomJsZrfXh4uKKiojxtmoI/+rF06VL16tVLQ4cO9Vo+b9483XbbbWrdurU++ugj/fnPf1ZZWZnuv/9+v/W/Lupb4913361OnTopPj5e33zzjR599FEdOnRI77zzjme7tY1x9bqm4o8xPH36tJ588klNmzbNa3kgxvD06dO6dOlSra/twYMHa33M5cbip++36mWXa9NU6lPfzz366KOKj4/3+mOflpamcePGqXPnzjpy5Ij++te/Kj09Xbm5uQoLC/NrDVdSn/p69OihZcuWqW/fviopKdHChQs1dOhQ7du3TzfccINV4yc1fAx37typb7/9VkuXLvVabssY1sfl3oNut1s//vij/vvf/zb4974ugiKgzJo1S08//fQV2xw4cEA9e/Zsoh75V13ra6gff/xRK1eu1OOPP15j3U+XDRgwQOXl5Xr22Wf99uHW2DX+9MM6MTFRHTp00MiRI3XkyBF17dq13tutq6YaQ7fbrYyMDPXu3VtPPPGE17rGHkP4bsGCBVq1apW2bNnidSDphAkTPD8nJiaqb9++6tq1q7Zs2aKRI0cGoqt1lpyc7PWlr0OHDlWvXr30yiuv6MknnwxgzxrH0qVLlZiYqMGDB3stb85jaIugCCgPPfSQJk+efMU2Xbp0qde24+LiJElFRUXq0KGDZ3lRUZH69+/vaXPq1Cmvx1VWVqq4uNjz+Iaoa30N7cfbb7+tc+fO6fe///1V2yYlJenJJ59URUWFX76roalqrJaUlCRJys/PV9euXRUXF1fjCPSioiJJajZjWFpaqrS0NLVr107vvvuuWrRoccX2/h7D2kRHRyssLMzzWlYrKiq6bD1xcXFXbF+X92RTqU991RYuXKgFCxbo448/Vt++fa/YtkuXLoqOjlZ+fn6Tfrg1pL5qLVq00IABA5Sfny/JrvGTGlZjeXm5Vq1apXnz5l31eQI1hvVxufeg0+lUq1atFBYW1uDfizrx29EszYyvB8kuXLjQs6ykpKTWg2R37drlafPhhx8G7CDZ+vZj+PDhNc78uJynnnrKXHfddfXua33567X+/PPPjSTz9ddfG2P+/0GyPz0C/ZVXXjFOp9OcP3/efwVcRX3rKykpMUOGDDHDhw835eXldXquphrDwYMHmxkzZnjuX7p0yfziF7+44kGyt99+u9ey5OTkGgfJXuk92ZR8rc8YY55++mnjdDpNbm5unZ7j+PHjJiQkxKxdu7bB/fVVfer7qcrKStOjRw/z4IMPGmPsGz9j6l/j8uXLjcPhMKdPn77qcwRyDH9KdTxItk+fPl7LJk6cWOMg2Yb8XtSpr37bUjPx3Xffmd27d3tOpd29e7fZvXu31ym1PXr0MO+8847n/oIFC0xkZKRZu3at+eabb8zYsWNrPc14wIABZseOHebzzz833bt3D9hpxlfqx4kTJ0yPHj3Mjh07vB53+PBhExISYjZs2FBjm++995559dVXzd69e83hw4fNSy+9ZFq3bm1mz57d6PXUxtca8/Pzzbx588yuXbtMQUGBWbt2renSpYu59dZbPY+pPs141KhRZs+ePWbjxo2mffv2ATvN2Jf6SkpKTFJSkklMTDT5+flepzVWVlYaYwI7hqtWrTIOh8OsWLHC7N+/30ybNs1ERkZ6zpi65557zKxZszztv/jiCxMeHm4WLlxoDhw4YObMmVPracZXe082FV/rW7BggYmIiDBvv/2211hV/w0qLS01Dz/8sMnNzTUFBQXm448/Nrfccovp3r17k4bl+tY3d+5c8+GHH5ojR46YvLw8M2HCBNOyZUuzb98+Txubxs8Y32usNmzYMHPXXXfVWG7bGJaWlno+6ySZ5557zuzevdt89913xhhjZs2aZe655x5P++rTjGfOnGkOHDhgFi9eXOtpxld6zfzhmgsomZmZRlKN26effuppo/+7XkS1qqoq8/jjj5vY2FjjcDjMyJEjzaFDh7y2e+bMGTNx4kTTtm1b43Q6zb333usVeprK1fpRUFBQo15jjMnJyTEJCQnm0qVLNba5YcMG079/f9O2bVvTpk0b069fP7NkyZJa2zYFX2s8duyYufXWW01UVJRxOBymW7duZubMmV7XQTHGmKNHj5r09HTTqlUrEx0dbR566CGv03Sbiq/1ffrpp7X+TksyBQUFxpjAj+GiRYtMx44dTUREhBk8eLDZvn27Z93w4cNNZmamV/u33nrL3HTTTSYiIsLcfPPN5oMPPvBaX5f3ZFPypb5OnTrVOlZz5swxxhhz7tw5M2rUKNO+fXvTokUL06lTJzN16lS//uH3lS/1PfDAA562sbGxZvTo0earr77y2p5t42eM77+jBw8eNJLMRx99VGNbto3h5f5GVNeUmZlphg8fXuMx/fv3NxEREaZLly5en4nVrvSa+UOIMQE4VxQAAOAKuA4KAACwDgEFAABYh4ACAACsQ0ABAADWIaAAAADrEFAAAIB1CCgAAMA6BBQAAGAdAgoAALAOAQUAAFiHgAIAAKxDQAEAANb5fzVyMQ3EmwU2AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(h.view(-1).tolist(), bins=50)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x126356b50>"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABj0AAAElCAYAAAC/JSDoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAmSUlEQVR4nO3df5BV5X0/8M/FlRV195IFdpctP4KYqI1CW9Rlx4TGsBWwY4Niq8ZM0FLT2IUOEKuhEzW2ma7V1joxRvuHQjMNJnVGdDQTHESBZrLQBMtY07ojlAQc2DUh5V5Yy4ru+f6Rr7deflxY2N27e/b1mjkz7DnPvfdz7vnxnHvfnPtkkiRJAgAAAAAAYIgbUe4CAAAAAAAA+oLQAwAAAAAASAWhBwAAAAAAkApCDwAAAAAAIBWEHgAAAAAAQCoIPQAAAAAAgFQQegAAAAAAAKkg9AAAAAAAAFJB6AEAAAAAAKRCRbkLOFJPT0/s2bMnqqqqIpPJlLscAAAAAACgjJIkiQMHDkRDQ0OMGFH6Xo5+Cz0effTRePDBB6OjoyOmT58ejzzySFx++eUnfNyePXti4sSJ/VUWAAAAAAAwBO3evTsmTJhQsk2/hB7f+973Yvny5fH4449HY2NjPPzwwzFnzpxob2+P2trako+tqqrqj5IAAAAAAKBP5HK5ksuz2ewAVTK8nEx+kEmSJOnrF25sbIzLLrssvvnNb0bEr3+yauLEibFkyZL4yle+UvKx+XzeDgEAAAAAwKB1oq/VDd3QP3K5XFRXV5ds0+cDmb/77ruxdevWaG5u/r8XGTEimpubo62t7aj23d3dkc/niyYAAAAAAIDe6vPQ45e//GW8//77UVdXVzS/rq4uOjo6jmrf2toa2Wy2MBnPAwAAAAAAOBV9Hnr01ooVKyKXyxWm3bt3l7skAAAAAABgCOrzgczHjh0bZ5xxRnR2dhbN7+zsjPr6+qPaV1ZWRmVlZV+XAQAAAAAADDN9fqfHyJEjY8aMGbF+/frCvJ6enli/fn00NTX19csBAAAAAABERD/c6RERsXz58li4cGFceumlcfnll8fDDz8cXV1dceutt/bHywEAAAAAwIDJZDLlLoHj6JfQ44Ybbohf/OIXcc8990RHR0f81m/9Vqxdu/aowc0BAAAAAAD6SiZJkqTcRXxYPp+PbDZb7jIAAAAAAIBBJJfLRXV1dck2fT6mBwAAAAAAQDkIPQAAAAAAgFQQegAAAAAAAKkg9AAAAAAAAFKhotwFAAD/J0mSksszmcwAVcIHbBNwHFCs1P5gXxh8HL8AROi/+4M+dvBypwcAAAAAAJAKQg8AAAAAACAVhB4AAAAAAEAqCD0AAAAAAIBUEHoAAAAAAACpIPQAAAAAAABSQegBAAAAAACkQkW5CwAGXpIkJZdnMpkBqgQ4kuNv8BnM28T5nIFiXxp8Sh3/thcfZn8AIEJ/0B+8p4OXOz0AAAAAAIBUEHoAAAAAAACpIPQAAAAAAABSQegBAAAAAACkgtADAAAAAABIBaEHAAAAAACQCkIPAAAAAAAgFSrKXcBgkyRJyeWZTCaVr83wYl/iw0qde+wr/eNE5/tSbBM+bKjuD655Tk1/vm9DeZsM136snNt7sL6vQ3k/BhhqnHMZLE5nX0zzfjxcr5E/4E4PAAAAAAAgFYQeAAAAAABAKgg9AAAAAACAVBB6AAAAAAAAqSD0AAAAAAAAUkHoAQAAAAAApILQAwAAAAAASIU+Dz2+9rWvRSaTKZouvPDCvn6ZfnNk7UdOaX1tYPhy3hl4Jzrf2yaknf381PTn+zaUt8lQrbuchvL2LiWt6zWYJUlScgLSyzmXweJ09sU078dpXa+TVdEfT/qJT3wiXnrppf97kYp+eRkAAAAAAICCfkkjKioqor6+vj+eGgAAAAAA4Jj6ZUyPN998MxoaGuK8886Lm2++OXbt2nXctt3d3ZHP54smAAAAAACA3urz0KOxsTFWrVoVa9eujcceeyx27twZn/rUp+LAgQPHbN/a2hrZbLYwTZw4sa9LAgAAAAAAhoFM0s+ji+3fvz8mT54cDz30UCxatOio5d3d3dHd3V34O5/PCz4AAABgkDvR1wnDZbBUAGDg5HK5qK6uLtmm30cYHz16dHz84x+P7du3H3N5ZWVlVFZW9ncZAAAAAABAyvXLmB4fdvDgwdixY0eMHz++v18qIn79P01KTQw+thkny74CAACDRyaTKTkx9Pi8BUAa9Hnocccdd8TGjRvjZz/7WfzoRz+Ka6+9Ns4444y46aab+vqlAAAAAAAACvr8563eeuutuOmmm2Lfvn0xbty4+OQnPxmbN2+OcePG9fVLAQAAAAAAFPT7QOa9lc/nI5vNnvLjDaQ29NhmnCz7CgAAQP8p9ZnL5y0ABoOTGci838f0AAAAAAAAGAhCDwAAAAAAIBWEHgAAAAAAQCoIPQAAAAAAgFSoKHcBfc3AWkOPbcbJsq+QFgaIBICBV6r/jdAHn4rTfU9tk8HHew5AGrjTAwAAAAAASAWhBwAAAAAAkApCDwAAAAAAIBWEHgAAAAAAQCoIPQAAAAAAgFQQegAAAAAAAKkg9AAAAAAAAFKhotwFMDQkSXLcZZlMZgArARj6ynnedD4HSAfn897zvvQ97ykAMBi50wMAAAAAAEgFoQcAAAAAAJAKQg8AAAAAACAVhB4AAAAAAEAqCD0AAAAAAIBUEHoAAAAAAACpIPQAAAAAAABSoaLcBTA0ZDKZfnvuJEnK9toAw41zKmng2oGTleZ9ZSjXPhyleV88HcN1vQGA/uVODwAAAAAAIBWEHgAAAAAAQCoIPQAAAAAAgFQQegAAAAAAAKkg9AAAAAAAAFJB6AEAAAAAAKRCr0OPTZs2xTXXXBMNDQ2RyWTi2WefLVqeJEncc889MX78+Bg1alQ0NzfHm2++2Vf1kkKZTKbkBADwYa4dOFn2FQYL+yIAwMDpdejR1dUV06dPj0cfffSYyx944IH4xje+EY8//nhs2bIlzjnnnJgzZ04cOnTotIsFAAAAAAA4nkySJMkpPziTiTVr1sT8+fMj4td3eTQ0NMSXv/zluOOOOyIiIpfLRV1dXaxatSpuvPHGEz5nPp+PbDZ7qiUBAAAAAAAplMvlorq6umSbPh3TY+fOndHR0RHNzc2FedlsNhobG6Otra0vXwoAAAAAAKBIRV8+WUdHR0RE1NXVFc2vq6srLDtSd3d3dHd3F/7O5/N9WRIAAAAAADBM9OmdHqeitbU1stlsYZo4cWK5SwIAAAAAAIagPg096uvrIyKis7OzaH5nZ2dh2ZFWrFgRuVyuMO3evbsvSwIAAAAAAIaJPg09pkyZEvX19bF+/frCvHw+H1u2bImmpqZjPqaysjKqq6uLJgAAAAAAgN7q9ZgeBw8ejO3btxf+3rlzZ2zbti1qampi0qRJsXTp0vj6178eH/vYx2LKlClx9913R0NDQ8yfP78v6wYAAAAAACjS69DjJz/5SVx55ZWFv5cvXx4REQsXLoxVq1bFnXfeGV1dXfHFL34x9u/fH5/85Cdj7dq1cdZZZ/Vd1QAAAAAAAEfIJEmSlLuID8vn85HNZstdBgAAAAAAMIjkcrkTDpHRp2N6AAAAAAAAlIvQAwAAAAAASAWhBwAAAAAAkApCDwAAAAAAIBWEHgAAAAAAQCpUlLuA4yk1CnsmkxngagZGkiQll6d1vYcq2wsA4MTKec3keo3Bwr4IAPSGa4fT404PAAAAAAAgFYQeAAAAAABAKgg9AAAAAACAVBB6AAAAAAAAqSD0AAAAAAAAUkHoAQAAAAAApILQAwAAAAAASIWKchdwPNlsttwlDLhMJlPuEugF2wsA4MTKec3keq1/JEly3GXe82PzvgAAveHa4fS40wMAAAAAAEgFoQcAAAAAAJAKQg8AAAAAACAVhB4AAAAAAEAqCD0AAAAAAIBUEHoAAAAAAACpUFHuAgAAABg6MplMuUsAAIDjcqcHAAAAAACQCkIPAAAAAAAgFYQeAAAAAABAKgg9AAAAAACAVBB6AAAAAAAAqSD0AAAAAAAAUkHoAQAAAAAApEKvQ49NmzbFNddcEw0NDZHJZOLZZ58tWn7LLbdEJpMpmubOndtX9QIAAAAAZZQkSckJoJx6HXp0dXXF9OnT49FHHz1um7lz58bevXsL01NPPXVaRQIAAAAAAJxIRW8fMG/evJg3b17JNpWVlVFfX3/KRQEAAAAAAPRWv4zpsWHDhqitrY0LLrggbr/99ti3b99x23Z3d0c+ny+aAAAAAAAAeqvPQ4+5c+fGt7/97Vi/fn387d/+bWzcuDHmzZsX77///jHbt7a2RjabLUwTJ07s65IAAAAAAIBhIJOcxuhCmUwm1qxZE/Pnzz9um//+7/+OqVOnxksvvRSzZ88+anl3d3d0d3cX/s7n84IPAAAAABikTvR1YiaTGaBKgOEml8tFdXV1yTb98vNWH3beeefF2LFjY/v27cdcXllZGdXV1UUTAAAAAABAb/V76PHWW2/Fvn37Yvz48f39UgAAAAAAwDBW0dsHHDx4sOiujZ07d8a2bduipqYmampq4r777osFCxZEfX197NixI+688844//zzY86cOX1aOAAAAAAw8Px8FTCY9XpMjw0bNsSVV1551PyFCxfGY489FvPnz49///d/j/3790dDQ0NcddVV8dd//ddRV1d3Us+fz+cjm832piQAAAAAACDlTmZMj9MayLw/CD0AAAAAAIAjDYqBzAEAAAAAAAaC0AMAAAAAAEgFoQcAAAAAAJAKQg8AAAAAACAVhB4AAAAAAEAqVJS7AAAAAAAAGEqSJCm5PJPJDFAlHMmdHgAAAAAAQCoIPQAAAAAAgFQQegAAAAAAAKkg9AAAAAAAAFJB6AEAAAAAAKSC0AMAAAAAAEiFinIXAMDpSZKk5PJMJjNAlXCySm0z2wsAGEiuJYH+4NzCcGA/Hrzc6QEAAAAAAKSC0AMAAAAAAEgFoQcAAAAAAJAKQg8AAAAAACAVhB4AAAAAAEAqCD0AAAAAAIBUEHoAAAAAAACpUFHuAoaTJElKLs9kMgNUCZAmzh2Dj/N973nPAKA89LGkgWvJwcd73nv2Y+g77vQAAAAAAABSQegBAAAAAACkgtADAAAAAABIBaEHAAAAAACQCkIPAAAAAAAgFYQeAAAAAABAKgg9AAAAAACAVOhV6NHa2hqXXXZZVFVVRW1tbcyfPz/a29uL2hw6dChaWlpizJgxce6558aCBQuis7OzT4seqjKZTMmJgZckSckprYbresNAcb4/tlLnHe8ZAACnyrUkaWA/Hnp8vzZ49Sr02LhxY7S0tMTmzZtj3bp1cfjw4bjqqquiq6ur0GbZsmXx/PPPx9NPPx0bN26MPXv2xHXXXdfnhQMAAAAAAHxYJjmN2OkXv/hF1NbWxsaNG2PWrFmRy+Vi3LhxsXr16rj++usjIuKNN96Iiy66KNra2mLmzJknfM58Ph/ZbPZUS4JeOdHun9YkfbiuN1Bepc49zjsAAAAMJb5fK49cLhfV1dUl25zWmB65XC4iImpqaiIiYuvWrXH48OFobm4utLnwwgtj0qRJ0dbWdszn6O7ujnw+XzQBAAAAAAD01imHHj09PbF06dK44oor4uKLL46IiI6Ojhg5cmSMHj26qG1dXV10dHQc83laW1sjm80WpokTJ55qSQAAAAAAwDB2yqFHS0tLvP766/Hd7373tApYsWJF5HK5wrR79+7Tej4AAAAAAGB4qjiVBy1evDheeOGF2LRpU0yYMKEwv76+Pt59993Yv39/0d0enZ2dUV9ff8znqqysjMrKylMpAwAAAAAAoKBXd3okSRKLFy+ONWvWxMsvvxxTpkwpWj5jxow488wzY/369YV57e3tsWvXrmhqauqbigEAAAAAAI6hV3d6tLS0xOrVq+O5556Lqqqqwjgd2Ww2Ro0aFdlsNhYtWhTLly+PmpqaqK6ujiVLlkRTU1PMnDmzX1YATkcmkyl3CWUxXNcbKC/nHoB0SJLkuMuc6wGA4cJ1z+CVSUpdsR7Z+DgbcuXKlXHLLbdERMShQ4fiy1/+cjz11FPR3d0dc+bMiW9961vH/XmrI+Xz+chmsydbEgAAAANI6AEAQLnkcrmorq4u2aZXocdAEHoAAAAMXkIPAADK5WRCj16N6QEAAAAAADBYCT0AAAAAAIBUEHoAAAAAAACpIPQAAAAAAABSoaLcBQBQWqnBQiMMGAoADCzXHsBg4zNTutie6VNqm9qe9Ad3egAAAAAAAKkg9AAAAAAAAFJB6AEAAAAAAKSC0AMAAAAAAEgFoQcAAAAAAJAKQg8AAAAAACAVhB4AAAAAAEAqVJS7AABKy2Qy5S4BAABg0PKZKV1sz/SxTRlo7vQAAAAAAABSQegBAAAAAACkgtADAAAAAABIBaEHAAAAAACQCkIPAAAAAAAgFYQeAAAAAABAKgg9AAAAAACAVKgodwF9LUmSksszmcwAVTK4DOb3ZTDXNlyV2ia2Bwxtjm8AAI5nuF4rnuh7iVLS/L4ADFXu9AAAAAAAAFJB6AEAAAAAAKSC0AMAAAAAAEgFoQcAAAAAAJAKQg8AAAAAACAVhB4AAAAAAEAqCD0AAAAAAIBU6FXo0draGpdddllUVVVFbW1tzJ8/P9rb24vafPrTn45MJlM0felLX+rToks58rWPnIarwfy+DObahivbA9LL8Q0AwPEM1WvFJElKTidyou8lhur7AjBc9Sr02LhxY7S0tMTmzZtj3bp1cfjw4bjqqquiq6urqN1tt90We/fuLUwPPPBAnxYNAAAAAABwpIreNF67dm3R36tWrYra2trYunVrzJo1qzD/7LPPjvr6+r6pEAAAAAAA4CSc1pgeuVwuIiJqamqK5n/nO9+JsWPHxsUXXxwrVqyId95557jP0d3dHfl8vmgCAAAAAADorV7d6fFhPT09sXTp0rjiiivi4osvLsz/3Oc+F5MnT46GhoZ47bXX4q677or29vZ45plnjvk8ra2tcd99951qGQAAAAAAABERkUlOZkSnY7j99tvjBz/4Qfzwhz+MCRMmHLfdyy+/HLNnz47t27fH1KlTj1re3d0d3d3dhb/z+XxMnDjxVEoCAAAAYJg50VdbBhwHSI9cLhfV1dUl25zSnR6LFy+OF154ITZt2lQy8IiIaGxsjIg4buhRWVkZlZWVp1IGAAAAAABAQa9CjyRJYsmSJbFmzZrYsGFDTJky5YSP2bZtW0REjB8//pQKBPqe/wUDkA7O5zB8lTr++/vYL+drAxyLcw8AH9ar0KOlpSVWr14dzz33XFRVVUVHR0dERGSz2Rg1alTs2LEjVq9eHVdffXWMGTMmXnvttVi2bFnMmjUrpk2b1i8rAAAAAAAAENHLMT2Ol5yvXLkybrnllti9e3d8/vOfj9dffz26urpi4sSJce2118ZXv/rVE/7O1gfy+Xxks9mTLQk4Bf5nMEA6OJ/D8OVODwAAhqOTGdPjlAcy7y9CD+h/viQDSAfncxi+hB4AAAxHJxN6jBigWgAAAAAAAPqV0AMAAAAAAEgFoQcAAAAAAJAKQg8AAAAAACAVKspdADDwDDAJw5fBZ9PFNoPhe14r57qV87WH6/ZOq1LbM8I2BYYX50ToO+70AAAAAAAAUkHoAQAAAAAApILQAwAAAAAASAWhBwAAAAAAkApCDwAAAAAAIBWEHgAAAAAAQCoIPQAAAAAAgFSoKHcBAMDAyWQy5S4BoE85rw0vtne62J4A/8c5cehJkuSUH2t79y93egAAAAAAAKkg9AAAAAAAAFJB6AEAAAAAAKSC0AMAAAAAAEgFoQcAAAAAAJAKQg8AAAAAACAVhB4AAAAAAEAqVJS7AAAAAAAAGEoymUy5S+A43OkBAAAAAACkgtADAAAAAABIBaEHAAAAAACQCkIPAAAAAAAgFYQeAAAAAABAKgg9AAAAAACAVOhV6PHYY4/FtGnTorq6Oqqrq6OpqSl+8IMfFJYfOnQoWlpaYsyYMXHuuefGggULorOzs8+LBgAAAACAckmSpORE+fQq9JgwYULcf//9sXXr1vjJT34Sn/nMZ+Kzn/1s/PSnP42IiGXLlsXzzz8fTz/9dGzcuDH27NkT1113Xb8UDgAAAAAA8GGZ5DRjp5qamnjwwQfj+uuvj3HjxsXq1avj+uuvj4iIN954Iy666KJoa2uLmTNnntTz5fP5yGazp1MSAAAAAAD0mxN9rZ7JZAaokuEll8tFdXV1yTanPKbH+++/H9/97nejq6srmpqaYuvWrXH48OFobm4utLnwwgtj0qRJ0dbWdqovAwAAAAAAcFIqevuA//iP/4impqY4dOhQnHvuubFmzZr4zd/8zdi2bVuMHDkyRo8eXdS+rq4uOjo6jvt83d3d0d3dXfg7n8/3tiQAAAAAAIDe3+lxwQUXxLZt22LLli1x++23x8KFC+M///M/T7mA1tbWyGazhWnixImn/FwAAAAAAMDwddpjejQ3N8fUqVPjhhtuiNmzZ8f//M//FN3tMXny5Fi6dGksW7bsmI8/1p0egg8AAAAAAAYrY3qUR7+O6fGBnp6e6O7ujhkzZsSZZ54Z69evLyxrb2+PXbt2RVNT03EfX1lZGdXV1UUTAAAAAABAb/VqTI8VK1bEvHnzYtKkSXHgwIFYvXp1bNiwIV588cXIZrOxaNGiWL58edTU1ER1dXUsWbIkmpqaYubMmf1VPwOkVHKZ5tRyuK43A8++BgAAAACnr1ehx9tvvx1f+MIXYu/evZHNZmPatGnx4osvxu/93u9FRMQ//MM/xIgRI2LBggXR3d0dc+bMiW9961v9UjgAAAAAAMCHnfaYHn0tn89HNpstdxkcYbj+L/Thut4MPPsaAAAAwNBhTI/yGJAxPQAAAAAAAAYDoQcAAAAAAJAKQg8AAAAAACAVhB4AAAAAAEAqVJS7gCMNsnHV+f/y+Xy5SyiL4breDDz7GgAAAMDQ4buc8jiZ/GDQhR4HDhwodwkcQzabLXcJZTFc15uBZ18DAAAAGDp8l1MeBw4cOOF7n0kG2a0VPT09sWfPnqiqqopMJhP5fD4mTpwYu3fvjurq6nKXB2XhOADHATgGwHEAEY4DiHAcgGOA4ShJkjhw4EA0NDTEiBGlR+0YdHd6jBgxIiZMmHDU/Orqagcxw57jABwH4BgAxwFEOA4gwnEAjgGGm5O9u8ZA5gAAAAAAQCoIPQAAAAAAgFQY9KFHZWVl3HvvvVFZWVnuUqBsHAfgOADHADgOIMJxABGOA3AMQGmDbiBzAAAAAACAUzHo7/QAAAAAAAA4GUIPAAAAAAAgFYQeAAAAAABAKgg9AAAAAACAVBj0ocejjz4aH/3oR+Oss86KxsbG+Ld/+7dylwT9orW1NS677LKoqqqK2tramD9/frS3txe1+fSnPx2ZTKZo+tKXvlSmiqHvfe1rXztqH7/wwgsLyw8dOhQtLS0xZsyYOPfcc2PBggXR2dlZxoqh7330ox896jjIZDLR0tISEfoC0mnTpk1xzTXXRENDQ2QymXj22WeLlidJEvfcc0+MHz8+Ro0aFc3NzfHmm28WtfnVr34VN998c1RXV8fo0aNj0aJFcfDgwQFcCzh1pY6Bw4cPx1133RWXXHJJnHPOOdHQ0BBf+MIXYs+ePUXPcaz+4/777x/gNYFTd6K+4JZbbjlqH587d25RG30BQ92JjoNjfU7IZDLx4IMPFtroD2CQhx7f+973Yvny5XHvvffGq6++GtOnT485c+bE22+/Xe7SoM9t3LgxWlpaYvPmzbFu3bo4fPhwXHXVVdHV1VXU7rbbbou9e/cWpgceeKBMFUP/+MQnPlG0j//whz8sLFu2bFk8//zz8fTTT8fGjRtjz549cd1115WxWuh7P/7xj4uOgXXr1kVExB/+4R8W2ugLSJuurq6YPn16PProo8dc/sADD8Q3vvGNePzxx2PLli1xzjnnxJw5c+LQoUOFNjfffHP89Kc/jXXr1sULL7wQmzZtii9+8YsDtQpwWkodA++88068+uqrcffdd8err74azzzzTLS3t8cf/MEfHNX2r/7qr4r6hyVLlgxE+dAnTtQXRETMnTu3aB9/6qmnipbrCxjqTnQcfHj/37t3bzz55JORyWRiwYIFRe30Bwx3FeUuoJSHHnoobrvttrj11lsjIuLxxx+P73//+/Hkk0/GV77ylTJXB31r7dq1RX+vWrUqamtrY+vWrTFr1qzC/LPPPjvq6+sHujwYMBUVFcfcx3O5XDzxxBOxevXq+MxnPhMREStXroyLLrooNm/eHDNnzhzoUqFfjBs3rujv+++/P6ZOnRq/+7u/W5inLyBt5s2bF/PmzTvmsiRJ4uGHH46vfvWr8dnPfjYiIr797W9HXV1dPPvss3HjjTfGf/3Xf8XatWvjxz/+cVx66aUREfHII4/E1VdfHX/3d38XDQ0NA7YucCpKHQPZbLYQgH/gm9/8Zlx++eWxa9eumDRpUmF+VVWV/oEhq9Rx8IHKysrj7uP6AtLgRMfBkfv/c889F1deeWWcd955RfP1Bwx3g/ZOj3fffTe2bt0azc3NhXkjRoyI5ubmaGtrK2NlMDByuVxERNTU1BTN/853vhNjx46Niy++OFasWBHvvPNOOcqDfvPmm29GQ0NDnHfeeXHzzTfHrl27IiJi69atcfjw4aJ+4cILL4xJkybpF0itd999N/75n/85/viP/zgymUxhvr6A4WTnzp3R0dFRdP7PZrPR2NhYOP+3tbXF6NGjC19yRUQ0NzfHiBEjYsuWLQNeM/S3XC4XmUwmRo8eXTT//vvvjzFjxsRv//Zvx4MPPhjvvfdeeQqEfrJhw4aora2NCy64IG6//fbYt29fYZm+gOGms7Mzvv/978eiRYuOWqY/YLgbtHd6/PKXv4z3338/6urqiubX1dXFG2+8UaaqYGD09PTE0qVL44orroiLL764MP9zn/tcTJ48ORoaGuK1116Lu+66K9rb2+OZZ54pY7XQdxobG2PVqlVxwQUXxN69e+O+++6LT33qU/H6669HR0dHjBw58qgP93V1ddHR0VGegqGfPfvss7F///645ZZbCvP0BQw3H5zjj/W54INlHR0dUVtbW7S8oqIiampq9BGkzqFDh+Kuu+6Km266Kaqrqwvz//zP/zx+53d+J2pqauJHP/pRrFixIvbu3RsPPfRQGauFvjN37ty47rrrYsqUKbFjx474y7/8y5g3b160tbXFGWecoS9g2Pmnf/qnqKqqOuonn/UHMIhDDxjOWlpa4vXXXy8ayyAiin6L9JJLLonx48fH7NmzY8eOHTF16tSBLhP63Idv4502bVo0NjbG5MmT41/+5V9i1KhRZawMyuOJJ56IefPmFf0cg74AYPg6fPhw/NEf/VEkSRKPPfZY0bLly5cX/j1t2rQYOXJk/Omf/mm0trZGZWXlQJcKfe7GG28s/PuSSy6JadOmxdSpU2PDhg0xe/bsMlYG5fHkk0/GzTffHGeddVbRfP0BDOKftxo7dmycccYZ0dnZWTS/s7PTb9KRaosXL44XXnghXnnllZgwYULJto2NjRERsX379oEoDQbc6NGj4+Mf/3hs37496uvr49133439+/cXtdEvkFY///nP46WXXoo/+ZM/KdlOX0DafXCOL/W5oL6+Pt5+++2i5e+991786le/0keQGh8EHj//+c9j3bp1RXd5HEtjY2O899578bOf/WxgCoQBdt5558XYsWML10D6AoaTf/3Xf4329vYTflaI0B8wPA3a0GPkyJExY8aMWL9+fWFeT09PrF+/PpqamspYGfSPJEli8eLFsWbNmnj55ZdjypQpJ3zMtm3bIiJi/Pjx/VwdlMfBgwdjx44dMX78+JgxY0aceeaZRf1Ce3t77Nq1S79AKq1cuTJqa2vj93//90u20xeQdlOmTIn6+vqi838+n48tW7YUzv9NTU2xf//+2Lp1a6HNyy+/HD09PYVgEIayDwKPN998M1566aUYM2bMCR+zbdu2GDFixFE/9wNp8dZbb8W+ffsK10D6AoaTJ554ImbMmBHTp08/YVv9AcPRoP55q+XLl8fChQvj0ksvjcsvvzwefvjh6OrqiltvvbXcpUGfa2lpidWrV8dzzz0XVVVVhd8czWazMWrUqNixY0esXr06rr766hgzZky89tprsWzZspg1a1ZMmzatzNVD37jjjjvimmuuicmTJ8eePXvi3nvvjTPOOCNuuummyGazsWjRoli+fHnU1NREdXV1LFmyJJqammLmzJnlLh36VE9PT6xcuTIWLlwYFRX/d7mmLyCtDh48WHS30s6dO2Pbtm1RU1MTkyZNiqVLl8bXv/71+NjHPhZTpkyJu+++OxoaGmL+/PkREXHRRRfF3Llz47bbbovHH388Dh8+HIsXL44bb7yx6OfhYLAqdQyMHz8+rr/++nj11VfjhRdeiPfff7/wWaGmpiZGjhwZbW1tsWXLlrjyyiujqqoq2traYtmyZfH5z38+PvKRj5RrtaBXSh0HNTU1cd9998WCBQuivr4+duzYEXfeeWecf/75MWfOnIjQF5AOJ7omivj1f/54+umn4+///u+Perz+AP6/ZJB75JFHkkmTJiUjR45MLr/88mTz5s3lLgn6RUQcc1q5cmWSJEmya9euZNasWUlNTU1SWVmZnH/++clf/MVfJLlcrryFQx+64YYbkvHjxycjR45MfuM3fiO54YYbku3btxeW/+///m/yZ3/2Z8lHPvKR5Oyzz06uvfbaZO/evWWsGPrHiy++mERE0t7eXjRfX0BavfLKK8e8Dlq4cGGSJEnS09OT3H333UldXV1SWVmZzJ49+6jjY9++fclNN92UnHvuuUl1dXVy6623JgcOHCjD2kDvlToGdu7cedzPCq+88kqSJEmydevWpLGxMclms8lZZ52VXHTRRcnf/M3fJIcOHSrvikEvlDoO3nnnneSqq65Kxo0bl5x55pnJ5MmTk9tuuy3p6Ogoeg59AUPdia6JkiRJ/vEf/zEZNWpUsn///qMerz+AX8skSZL0e7ICAAAAAADQzwbtmB4AAAAAAAC9IfQAAAAAAABSQegBAAAAAACkgtADAAAAAABIBaEHAAAAAACQCkIPAAAAAAAgFYQeAAAAAABAKgg9AAAAAACAVBB6AAAAAAAAqSD0AAAAAAAAUkHoAQAAAAAApILQAwAAAAAASIX/ByvINQUDfv74AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 2000x1000 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(20, 10))\n",
    "plt.imshow(h.abs() > 0.99, cmap='gray', interpolation='nearest') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So what we did above is essentially decrease the value of preactivations such that the values fall in the non-flat region of tanh. The image above shows that very few of the neurons are activated with the examples for the first iteration. Let's try to use this approach for the entire training set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.3189\n",
      "  10000/ 200000: 2.0652\n",
      "  20000/ 200000: 2.5255\n",
      "  30000/ 200000: 2.0096\n",
      "  40000/ 200000: 2.2400\n",
      "  50000/ 200000: 1.8046\n",
      "  60000/ 200000: 2.0230\n",
      "  70000/ 200000: 2.2471\n",
      "  80000/ 200000: 2.3160\n",
      "  90000/ 200000: 2.1416\n",
      " 100000/ 200000: 2.1507\n",
      " 110000/ 200000: 2.1757\n",
      " 120000/ 200000: 1.5639\n",
      " 130000/ 200000: 1.8486\n",
      " 140000/ 200000: 2.1428\n",
      " 150000/ 200000: 1.9197\n",
      " 160000/ 200000: 2.0423\n",
      " 170000/ 200000: 2.3999\n",
      " 180000/ 200000: 2.0102\n",
      " 190000/ 200000: 2.0702\n"
     ]
    }
   ],
   "source": [
    "run_training_loop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train 2.0361340045928955\n",
      "val 2.102936267852783\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x12631a050>]"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAGdCAYAAADJ6dNTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABRz0lEQVR4nO3deVxU9foH8M+wi2wqu6KAC4oLKCrhbqJotltpedOorExvdimv2qJlC1ZmlpmWZbZri1a/q+GC4kqiKIpLuAsuIC4sssOc3x/IOAOznVmYMzOf9+s1L2XmzJnvYYY5z/kuzyMTBEEAERERkUQ4WLoBRERERMoYnBAREZGkMDghIiIiSWFwQkRERJLC4ISIiIgkhcEJERERSQqDEyIiIpIUBidEREQkKU6WboA+5HI5Ll26BE9PT8hkMks3h4iIiPQgCAJKS0sRHBwMBwf9+0OsIji5dOkSQkJCLN0MIiIiMkBeXh7atWun9/ZWEZx4enoCqD84Ly8vC7eGiIiI9FFSUoKQkBDFeVxfVhGcNAzleHl5MTghIiKyMmKnZHBCLBEREUkKgxMiIiKSFAYnREREJCkMToiIiEhSGJwQERGRpDA4ISIiIklhcEJERESSwuCEiIiIJIXBCREREUkKgxMiIiKSFAYnREREJCkMToiIiEhS7Do4OX65BF/uPIPaOrmlm0JERES3WEVVYnMZ8/FOAICDTIYnB4VZuDVEREQE2HnPSYMjl4ot3QQiIiK6hcEJERERSQqDEyIiIpIUBidEREQkKQxOiIiISFIYnBAREZGkMDgBAMHSDSAiIqIGDE6IiIhIUhicEBERkaQwOCEiIiJJYXBCREREksLghIiIiCSFwQkRERFJCoMTcCUxERGRlDA4ISIiIklhcEJERESSwuCEiIiIJMWg4GTp0qUIDQ2Fm5sbYmNjkZGRoXX7oqIiTJs2DUFBQXB1dUWXLl2wYcMGgxpsDjJLN4CIiIgUnMQ+Yc2aNUhKSsLy5csRGxuLxYsXIyEhATk5OfD392+yfXV1NUaOHAl/f3/8+uuvaNu2Lc6fPw8fHx9TtJ+IiIhsjOjgZNGiRZgyZQoSExMBAMuXL8f69euxcuVKzJ49u8n2K1euxPXr17Fnzx44OzsDAEJDQ41rNREREdksUcM61dXVyMzMRHx8/O0dODggPj4e6enpap/z559/Ii4uDtOmTUNAQAB69OiBd999F3V1dRpfp6qqCiUlJSo3c+JSYiIiIukQFZxcvXoVdXV1CAgIULk/ICAA+fn5ap9z5swZ/Prrr6irq8OGDRvw+uuv48MPP8Tbb7+t8XWSk5Ph7e2tuIWEhIhpJhEREVkxs6/Wkcvl8Pf3xxdffIGYmBiMHz8er776KpYvX67xOXPmzEFxcbHilpeXZ+5mEhERkUSImnPi6+sLR0dHFBQUqNxfUFCAwMBAtc8JCgqCs7MzHB0dFfd169YN+fn5qK6uhouLS5PnuLq6wtXVVUzTiIiIyEaI6jlxcXFBTEwMUlNTFffJ5XKkpqYiLi5O7XMGDhyIU6dOQS6XK+47ceIEgoKC1AYmREREZN9ED+skJSVhxYoV+Oabb3D8+HFMnToVZWVlitU7kyZNwpw5cxTbT506FdevX8eMGTNw4sQJrF+/Hu+++y6mTZtmuqMgIiIimyF6KfH48eNRWFiIuXPnIj8/H9HR0UhJSVFMks3NzYWDw+2YJyQkBBs3bsR//vMf9OrVC23btsWMGTMwa9Ys0x2FkQSB63WIiIikQiZYwZm5pKQE3t7eKC4uhpeXl8n2Gzp7PQDg/uhgLJ7Q22T7JSIiIsPP36ytQ0RERJLC4ISIiIgkhcEJERERSQqDEyIiIpIUBicAWrVkvhUiIiKpsOvgpH9YawBATIdWFm4JERERNbDr4ERm6QYQERFRE3YdnBAREZH0MDghIiIiSWFwQkRERJLC4ISIiIgkhcEJERERSQqDEwDSL31IRERkP+w6OJFxLTEREZHk2HVwQkRERNLD4ISIiIgkhcEJERERSQqDEyIiIpIUBidEREQkKQxOAHAlMRERkXTYdXAiY11iIiIiybHr4ISIiIikh8EJERERSQqDEyIiIpIUBidEREQkKQxOiIiISFIYnAAQWJaYiIhIMuw6OGFVYiIiIumx6+CEiIiIpIfBCREREUkKgxMiIiKSFAYnREREJCkMToiIiEhSGJwQERGRpNh1cMKlxERERNJj18EJERERSQ+DEyIiIpIUBidkFeRyAeXVtZZuBhERNQMGJ2QVJnzxNyLnbsSV0kpLN4VMrLi8hvWtiEgFgxOyChnnrgMAUo7kW7glZEqHLxQhav4mPPNdpqWbQkQSwuAEgK6LtvLqWizffhpnCm82T4OI7MTXu88BADYfK7BsQ4hIUuw6OJFBv7XE76fkYMFf/+DOD7ebuUVERERk18GJvjLP37B0E4iIiOwGgxMishjmQSQidew6OMm+WAwA+HBzDu76eCeKyqst3CIiIiKy6+CkuKIGAJB3vQLHLpdg+fYzFm4RkZ1p1HVy+EIRBr23FRuyL1umPUQkCXYdnDRWXSu3dBOI7Npz32Xiwo0KPP/DAUs3hYgsiMGJHqy1QOCpKzdx95Kd2HjUdnKDWOlbQRoor5hbsy8Xl4qbJtnbkH0ZKUcuo7i8BpeLK5qzeURkIQYFJ0uXLkVoaCjc3NwQGxuLjIwMjduuWrUKMplM5ebm5mZwg0l/L645iCMXS/AsE1yRFZj1W3aT+0ora/D8Dwfw3PcHEDV/E+KSt+LqzapmbVdxRQ1+ysjlnDSiZiQ6OFmzZg2SkpIwb948HDhwAFFRUUhISMCVK1c0PsfLywuXL19W3M6fP29Uo0k/56+WW7oJREapqK5rct/xyyXN2oYZqw9iztpsTP2eQ01EzUV0cLJo0SJMmTIFiYmJiIyMxPLly+Hu7o6VK1dqfI5MJkNgYKDiFhAQYFSjzUWA6ep71NbJceRiMerklqsZUlrFQnkkbdYwZJqWUwgASD9zzcItIbIfooKT6upqZGZmIj4+/vYOHBwQHx+P9PR0jc+7efMmOnTogJCQENx33304evSo1tepqqpCSUmJyq057DhRiK93nzXJvub9eRR3L9mFBX8dN8n+iIiI7IWo4OTq1auoq6tr0vMREBCA/Hz1ky4jIiKwcuVK/PHHH/j+++8hl8sxYMAAXLhwQePrJCcnw9vbW3ELCQkR00yDnS4sw5v/dwz7bxWZA4BX12Xj8IVi0fv6YW8uAGDFTtMEO1SPtWttixV0nBCRBZh9tU5cXBwmTZqE6OhoDB06FGvXroWfnx8+//xzjc+ZM2cOiouLFbe8vDxzN1PFZaUVAw1BBhERETUPUcGJr68vHB0dUVCgWkG0oKAAgYGBeu3D2dkZvXv3xqlTpzRu4+rqCi8vL5WbNdtxolCR8I2Mwytt22INc06sSXFFDfacvgq5Bee6EZmCqODExcUFMTExSE1NVdwnl8uRmpqKuLg4vfZRV1eH7OxsBAUFiWupBeTkl5pkP5NWZmDcsj0GPXfLsQL8mql5CIyIpKmqtq7Zlz3f++kuPLZiL37JbN7eZiJTEz2sk5SUhBUrVuCbb77B8ePHMXXqVJSVlSExMREAMGnSJMyZM0ex/fz587Fp0yacOXMGBw4cwL/+9S+cP38eTz/9tOmOwkwe/2qv6OfkXS/HKjWTak9duSl6X3VyAU9/ux8v/3IIF27Y5rLgypo67Dl1FTV1zM5L1qmgpBKC0LSn4s6F29H37S3Iu679b7emTm6y7NTnr9W/1vps20m8KFU1dXJsyL7c7AGovRAdnIwfPx4LFy7E3LlzER0djaysLKSkpCgmyebm5uLy5dt1MW7cuIEpU6agW7duuOuuu1BSUoI9e/YgMjLSdEdhJldKxX/oRn60HW/83zGjX1sQBIxctF3xc1G56YeFjl0qwT1LdmHHiUKT71tfL/x0EI99uRfv/fWPxdpAliOz8oG6FTvOIPbdVHy05WSTxy4W1Wezff6HA8i9pj5AkcsFxCVvRcxbmxmgW5nPtp3G8z8cwH2f7rZ0U2ySQRNip0+fjvPnz6Oqqgp79+5FbGys4rG0tDSsWrVK8fNHH32k2DY/Px/r169H7969jW64OR3IvWHwcytrxH/BHL9c0qRnpaKmDmeulhncDn089c0+ZF8sxqSVmjP8mtumY/Xzl77ec85ibTClU1duYs2+XLX5bfafu47Zvx1mplE9Hb0kfpWcLpuO5mOTEeUcahsFEO9sqE8V8Elq0+CkQfbFYgz5YJvaxypvDf2UVtWioKRp6n6SrpRbn6OGIJRMi7V11Ph697lmS55WUlmDMR/vRPyi7TonsR2+UKSSHbO6Vo7yasMTrd3gSdLk4hdtx6zfsvHL/qZj/g8tT8fqfXl4Z73xuW+qautwxQZOZtomxG46WqD2/hsG9iKWV9fime8y8cx3mSgzMEHhXZ/sNOh5UpV7rRxLt51CSSUn7JO0MDjRIObtzVofLyytwvCFaVi6TfOqI31cVRo6kqsZt25QUlGDez/djTEf71QEMXckpyJy7kaDv2g1uVJaiV/256GypmnqcE2OXirGkYumv9K1Vll5RRofO6+hi1+M+EXb0f/dVJw1c++auVUbMJTxwk8HDXot5V7NChGfbWUnCsTPHZOysZ/sxAcbc/DGH9oTYxI1NwYnGmia49HQc/Hp1pM4e7UMH2zMMenrnrpyE8VqXvtq2e1ejoYg5vqt+/4x0aqiBg9+tgczfz2M91P0O7by6lqM/WQX7l6yy6ieHL1w7SkAIO96fVfylmPqexesxe8HLzbL6yzecgL93tlikn3lXS9XOwHWGjWUuNh79rqOLYmaF4MTkRquemvUDMGk5WgufqiPU4U3Eb9oO3q/tcmo/RiqsqYOYz/ZiQs3bp34jut34nvws9vLpD/cdMIsbSPbpG0k05Sn/8VbTppsqHbw+9uwaHPzfM5PF97EoPe24oe9t4ulllXVIunnLKTq+fdJ5tEcAereM9fw+Fd7cabQtnrs9MHgxEQyz9/AE1/vM2ofe07VFxZT9x1aaoYx4cZ/W38euoSjl8TXMVLuuflqF9P1W6NLRRX4YOM/yC+2/nks+jD2vLJka9PhXFMPrwLAa+uO4MKNCry67ojivmVpp7H2wEU89c1+jc8T2794sagCHzVTwEX6G//F39h58iqe+z7T0k1pdgxODNT4y+3whSLj96nlMeUvJ3OorKlDbZ1qCwQIWLQpB9N+OCAq4+QPe8/jm1urb5SHeQRBMDpfy5nCm3hx9UGcuqI6lFVZU6d2OMxSpNrrr+lq74mvM7B022k88bXlVm6ZSk5+Kf48dKnZh15e/930f6Pqlhdf1hBAVtXenkdTZEBG6o+1rDgiyyoosb9cKgxOTCDz/HW8aWBuE5lJ5lA0/RL+I0vcWH7KkabLK/OuV+CTraewPvsydpzUPxfKq+uOYN6fR/Feyj+InLtR0ZbX/ziCQe9tU+miFuvr3efwe9YlPPL53yr393tnC6Lmb5JUgCI1lTV1GPXRDsxZm93ksYaJnqaev2QJCYt34IWfDmLHyasatzmUV4RCNXmMisqrDU79nmLEEmVj5V4rR8RrKYqfD+UV4aaOnpxsAwqaEjUXBicGUs76+OgX4jPJGuPOD7errIxRF1jMWJ2ldR+518pRJSIrpZiVOw2WpZ0GALz08yEAwPd/1xdR1HeirTbXy1SXQZdW1n8RH9KjB6u2Tt4kX4UxtF2hS2ni5Maj+Th55SZ+yrCOYpYVRk6uztbyWXj62/3o984WbD9RqAhI/skvQfT8zfiXAZmh9aHvZ+H7v88j4aMduFysf/6MlWqyUqtbzq7snk936b1/oP5vjjV7qLkwODGAIAjYder2VZm+yyEb/rCVT/SNT7L6yL1ejinf3h5vviRynsC5q2Uak0LZOrlcwLCFaRjy/jaVCZKZ52/g/qW7cVBkAr7VGbno+/YWtcuop3y7H2M/2aU2EHr+h0w8+NluoydpCiKmjUooTtLLip1nDZoD1WDhphMY9N5WZJ7XvBJl8soMRM/fjLuX7MLqjPqT+Z7T1wx+TW2OXFQ9lqOXivHZttNNtnvt9yPIKSjFgkZZk/93+JKo1zPl/K+DuTfQ563NeOob4+bV5V0vN+hCx97Z4yJFBici/d+hS+j4ygaDnhs1fxNCZ69H19dTkHFr6Z5yQUAx9TXKqzX/gesKeHaf1tzdrY2pv1SUu5Xr5AL+PmOak8L1smq8u+E4ThY0HaIorqjBhRsVuFRciWtKNTHGLduDrLwiPLQ8XdRrzV6bjWtl1XhxTVaTxzYfK8CxyyU43DhwkQEbsvNxILcIx4w4+dqDZdubnry1qWj0d3HhRgXGLdP9nh67rP59MOXQR50goKK6Dpnnb0AuFzD2k134VEuepOpauUroOf1HcfldLtyoUPs3W1lTJ7qWz9e7zwEAtuUYXuoi+0IxBr+/DWM+tq1EdmIJgsCkd3pgcCLS+uzLWpc/atMw9AAAj3yejl2NxsTfSzFNfRlDc69ouwr/7cBFdH09BatNMCQgCAIEQWjSrTzhi7+xLeeKUSuTBACzfjuML3acwciPdoh+fp1cwJy1hxVDUnq/rtB0MnGDxhc9GUo5JcT0fJB2FdV16DY3RfeGerheVo23/ndM1NBHeXWdzqGbx7/ai3HL9uDb9HMGtetmVa2o8hq1cgFPfJ2Bd9bXz4mrrKlD5NwUDFiQquOZmgmCgMzzN3BDZK9vQ8+PtScONNbLvxxGrzc2qXwPUFMMTixI29j2k6tEdJ82+j40R+2WzbeSfc1Wmkx5paQSoxeLDwAAzZk2E7/eh0Qjl2Sru9otLK3CxqP5qNNjbOOnjDyTBYpkHH2HPY9cLMbCTaZLiDhn7WGDhkW2aymiebmoAvvP1wcWq/dpnw+iybjP9jQ5uaceL8AqDbWpMs5eQ1pOIVbsrD+Ws1fLIBeAqzcN/44Im7MB45btwVA7HRo21m8HLgCA1l6zxuxwVIfBiVT9fUZ7VK18hbY++zJ+1jH5zZQaXvuDjTkGr+7QdvXX8AVuqHw1NWcSFu/As99l4rt0w1cKnSm8ifuX7rbarKzW2EvTuCCmJncv2WXSORYbNdT10eVtLXWT5qy7Hdirq2ulz5ygHDVDldrynZRUmC9jc0llLe54N1XnCrnC0iok/ZxlVEFVW6G8kAKo78lSnmR87FKJ2oRrhq7qrKypw+ItJ6yytAiDExvx318PK/6v7Usu5Ui+0TlT+r+bigs3yg2uT1JeXYfP0vS/ahBTRVZTt3rDFfjWfwzP4vvimixk5RXh6W81nwwa/Lz/gl771HeSaub5G0g5clm/jY009pOd+Cv7Mhb89Q8+3qKa+6JOLiAnv9Tiq5D+75C4yaH6MMXJs7C0Cu9uOI7kv5oGKcolMdTlrZizNrvJiSnTyEBdedn4b5n6fSaB29WXT10pxcHcGxrLUuSXVCJxlfbcOK+uy8baAxex7xyDk8Hv3+5tulFWja6vp2DCir8VP9/1yU7c+eF2FJRUqmQcN7TnZMnWk1i85STuXiJuZZYUOFm6AWSYmjrDTg6myDRYWFolKk1945bWygVFbRhNQmevR/KDPfFo//Z45jv926wud4UuuXoW4tNUb6k5NEyc3pI0FJ38PbRu+8Pe8wht0xIDO/lq3Ka4ogbeLZzVPnb0Ugmm/nBA8fPUYR3h4lR/HTP3jyP4YW8uXhjRGUkju4g9DJP5908HUV5di/H92ptsn4dFTn4tUNNDV1xRgy92nDHo9XeduopHV9zO33PFgM9yY8oXEC/9cggfjY/S+ZwVO87gvZR/sObZOJUJ+5ocyC3S+vi5a9KcY1InF3DkYjG6B3vByVH8dbqx8Xn2rd6MhrknF4tufycOWLDVJOUWjFntZmnsObFShvZaaCKmJwNonvwdc9Zmi540NlOpB0kfF4sqrGpZdeMMu8pvQ3WtHGsPXMCr645g4peq85kuFVXgP2sOKX6+c2Ga3q8pQEDutXJUVNfhh731E6I/0TOb6JGLxZi8MsMsq5Jm/dY0mVxzin3X8Emlmij3qBjba6LOpSLdaQfe2XActXIBs38T97dkbd5Zfxz3Ld2NuX9KryKzqepAWTMGJzZEW0bIsZ/sxO5TmpcQnyi4abY/iDq5gO/+NmyuxyOfi1vaK5bYvCaaaEsvLXa8WBAEtWnLdXn8q71I+vmQ2seSG+XMuCZipcXRSyUY8sG2Jithtt3qdt5+ohCPf7W3yXg6ADy4bA+2nyjEhC/M+z6SfkxdRV2qGn+X/e/wpSbffw2J637cK/2khMxzQlYt/1ZGSXUTH49eKmlyNd3Yt0ZMFtXFHHVHjE1VLzYU0zah9GZVLZaKmH2vrQ2TVmagz1ubUVpZg1fWZeNrNdk/1dFW9r5KTU/bP/n1vRm6esHmayjN8M2ecziYewOTV2Zg58mrmPlr08CoIZ9GSaV5JmZ+vOUkrpTaR7HC5nZSz8nIjZVV1eJg7g3RvatyuYBdJ68atNpQeVLpuatl6PnGRnywsT4gz7tejuk/HtT5/WdJcrmAT9UUkzSWtSVeVMbghBT0XRkhFQPf24qq2jpRdYSMuQLRNU9G21VpQyCgj50nr6K0shbvrD+OH/fmGly3SZfRi+uTYb3+h/bAMSuvSONjD3x2e05C4zkS6lYI7D51VWvGVrE+2nLCLMMrJI5yIc5xy/bggc/2YO0B7X+X23KuYMFf/yh6OX7en4d/fbUXYz8RN3mzpLIGscmpePmX+uD4w80nUF5dh6XbTmPLsQJRdcEsJeVovkVrM0kRgxMbsv5wPga9txVbjhu+IkVfUgjIb1bV4t31x3XWETKFo5dUT7QpR/L1nqdz4PwN0dk9AcNzYTS4drMKk1ZmYP1hzat8auvkippHptZ4hcC1m1WY+OVejFuWbtI5S9Z8dSg1hg7txi+6ne+oIb3AuoP1wYlMw1qTxK/3Yfn20/j91nbrs+s/p8oTQ/Wx7sBFFJZW4Vc1q5Ge/na/2Sq6q+tJPVlQiitqJkrrom5IVFXT3+HNqlrF76qmTo7nf8g0OLmfFHG1jg35aIv+K2hsxTcih6K0pf1vLL+4En6erth75hoea9QlLGbV06WiCpTrqBD74aYc/J51EX9OG6R1OwH1qykaZF8sxtPf7MfsMV2bbPteyj/YcaIQO04UYmRkgNr9Df0gTWf7TUXMPBeyDLmRkZ6+JSjOKSWSE1Pg0BTkcsEsK4guFlUoslKfWzDWpPtW1+Pb9+3NqKyRY+d/h2Pv2evYkJ2PDdn5mBQXatLXthQGJ2Qwa7xgVR66Onu1TOvE0zuSU+Hm7IDKGuMrGNdquCJt6EFYcmu8ebmOWjLrDlzEn0o5Pv53q1ckp6DpsNH1sttzcg5pGJoRe5WqLM2IOitPrtqHF0Z0Nvj5ZB7qKpzra3VGrkoGaW2GiVgt1lhFdR2q6+Qal8LrMnvtYb3zEInR3HWyGr6X/j5zTeNiCGv8jm7A4ITs1oQv/ta5jSkCk7+O5Ouds+JzHTky/tSQfEzXfBhT5MzQpVJEr9S2nEKjishRvUtGBJfq/Psn8cOPDRoHJrtOXRVV4E5dp83RS8VYvOUkZiZEoEuAJwAg6s1NqK6T48ibCfBwFX8KM0VgUlsn11iCwxxkqJ9cfr2sGoHebno/T93w6adbT+J0YRkWPRJlcObZ5sA5J2SQP7Iu4bSVTaC1FGN6J6zJpWLtY+3M3WB6hq6oaS4v/XxI5yR0mUyGZWmnsUtNqoMHP9uDzccKVFbaVN/q7WyoOt74BFymZQjV2BV+DRrnU6qq1RyYy+UCftmfpzYtvRj3L92NO5JTcfhCkVH7WbjpBNYdvKh1dZ8UMDghgxlaV4fskzmXqturySu1p463tM161qFqXGizYQJ61a2l6IWlVXpXM9ZWouLZ7zWXnqipk+PXzAuKi4nqWjkqNPQGNkz2bbB0m+bh2PBXNmDmr4dx54fbtTVbq4qaOhy7XD9sNEePobObVbX4cW8urmkp8FiplF7g94MXTVqbyhQYnJDNqzRxNl1TqpMLGueDGGvL8eYvUJiVV4SHl+9Ru/x43UHTj/OTbRr7ya4mk2uHq5mnciivSGVVW0NeHU20FVT9cudZvPzLIQy7lTE69t0t6DY3ReX7Q9P+9+o5EdhQpUp5gtSlpG88evP670fwyrpsRUCjjvJTXlyThbf+d8zo3h1T4pwTsnlVOr6wLGn8F3/b1HDH/Ut3q/yrzBTzd8h+qJsT1niI9L5Gn7OVeiYsVGfP6fphpYa6ZTduDQEt2nwCL8Z3xo4ThXju+wN4b1xPvfepbzVgsd8AzynVGysqr8E7G1QLTW40MGdKUYXl6oc1xuCEbNqHm3IUK1qkyJYCEyJDbD+h/8To53Us4c/SUYTQEF/sOIMbZdX45VYeFTE1ncxVDVg5YVvjwESTQ3lFiArxuX2HxL96OKxDNm3J1lN6j1UTkenpyjwtprjnIR2Vo82VZfUXNQnetCkur1HUnpKK+5bublIaoKZOrnEFoKWx54SIiMxGU44fYxWacHl8w5BOg8YZoXU5XXhTkekWAKLmbzJJuwwx5uOdGpNNRs/frPi/AAErdp7B+ynSLAbJ4ISIiKzOM9/pn6VZl8dW7IWn2+3Todj6PldvVivS71vacS2TYJW9/MthdPRraebWGI7DOkREZPdKzVQ5Wx+GTmA1xnU15SSkVKeKwQkREZEFHTTDRF597Dt3wyKvqw8GJ0RERCQpDE6IiIhIUhicEBERkaQwOCEiIiJJYXBCREREkFLaWAYnREREJCkMToiIiEhSGJwQERGRpDA4ISIiIklhcEJERESSwuCEiIiIWFuHiIiIpCXzvHRq7TA4ISIiIhSWVlm6CQoMToiIiEhSGJwQERGRpDA4ISIiIkkxKDhZunQpQkND4ebmhtjYWGRkZOj1vNWrV0Mmk+H+++835GWJiIjITCS0WEd8cLJmzRokJSVh3rx5OHDgAKKiopCQkIArV65ofd65c+fw8ssvY/DgwQY3loiIiMzDqpcSL1q0CFOmTEFiYiIiIyOxfPlyuLu7Y+XKlRqfU1dXh4kTJ+LNN99EeHi4UQ0mIiIi2yYqOKmurkZmZibi4+Nv78DBAfHx8UhPT9f4vPnz58Pf3x9PPfWUXq9TVVWFkpISlRsRERHZB1HBydWrV1FXV4eAgACV+wMCApCfn6/2Obt27cJXX32FFStW6P06ycnJ8Pb2VtxCQkLENJOIiIismFlX65SWluLxxx/HihUr4Ovrq/fz5syZg+LiYsUtLy/PjK0kIiIiKXESs7Gvry8cHR1RUFCgcn9BQQECAwObbH/69GmcO3cO99xzj+I+uVxe/8JOTsjJyUHHjh2bPM/V1RWurq5imkZERERGECS0XkdUz4mLiwtiYmKQmpqquE8ulyM1NRVxcXFNtu/atSuys7ORlZWluN17770YPnw4srKyOFxDREQkEVJarSOq5wQAkpKSMHnyZPTt2xf9+/fH4sWLUVZWhsTERADApEmT0LZtWyQnJ8PNzQ09evRQeb6Pjw8ANLmfiIiICDAgOBk/fjwKCwsxd+5c5OfnIzo6GikpKYpJsrm5uXBwYOJZIiIiMoxMEKTUkaNeSUkJvL29UVxcDC8vL5PtN3T2epPti4iIyJo9MSAUb9zb3aT7NPT8zS4OIiIigpT6KhicEBERkYTW6jA4ISIiIolhcEJERESSwuCEiIiIJIXBCREREUkKgxMiIiKSVIZYBidERERkvbV1iIiIiMyNwQkRERFJCoMTIiIikhQGJ0RERMQJsURERCQtEopNGJwQERGRtDA4ISIiIklhcEJEREScc0JERERSI53oxK6DkzDflpZuAhERETVi18EJERER1eOwDhEREZEGDE6IiIhIUuw6OBGk1IdFREREAOw8OCEiIqJ6UrpeZ3BCREREkmLXwYlMJrN0E4iIiCRBYJ4TaeCcEyIiIumx7+DE0g0gIiKiJuw6OCEiIqJ6UhpMYHBCREREksLghIiIiCTFroMTKXVhERERWZJcQudEuw5OiIiISHoYnBAREZGk2HVwwhxsRERE9ZiEjYiIiEgDuw5O7o0KtnQTiIiIqBG7Dk76dGhl6SYQERFRI3YdnBAREZH0MDghIiIiSRWcs+/gREJvBBERkSVJ6ZRo38EJERERSY59ByfMc0JERCQ59h2cEBERkeQwOCEiIiIIEqqGa9/BiXTeByIiIrrFvoMTIiIiAiCt63UGJ0RERISyqjpLN0GBwQkRERGhsobBiSRIqTw0ERER1TMoOFm6dClCQ0Ph5uaG2NhYZGRkaNx27dq16Nu3L3x8fNCyZUtER0fju+++M7jBREREZHpSumAXHZysWbMGSUlJmDdvHg4cOICoqCgkJCTgypUrardv3bo1Xn31VaSnp+Pw4cNITExEYmIiNm7caHTjjSVjFjYiIiLJER2cLFq0CFOmTEFiYiIiIyOxfPlyuLu7Y+XKlWq3HzZsGB544AF069YNHTt2xIwZM9CrVy/s2rXL6MYTERGRaUgozYm44KS6uhqZmZmIj4+/vQMHB8THxyM9PV3n8wVBQGpqKnJycjBkyBCN21VVVaGkpETlRkRERPZBVHBy9epV1NXVISAgQOX+gIAA5Ofna3xecXExPDw84OLigrFjx2LJkiUYOXKkxu2Tk5Ph7e2tuIWEhIhppt6kNL5GRERE9ZpltY6npyeysrKwb98+vPPOO0hKSkJaWprG7efMmYPi4mLFLS8vrzmaSURERBLgJGZjX19fODo6oqCgQOX+goICBAYGanyeg4MDOnXqBACIjo7G8ePHkZycjGHDhqnd3tXVFa6urmKaRkREREaw2jknLi4uiImJQWpqquI+uVyO1NRUxMXF6b0fuVyOqqoqMS9NREREZiSlqQ6iek4AICkpCZMnT0bfvn3Rv39/LF68GGVlZUhMTAQATJo0CW3btkVycjKA+vkjffv2RceOHVFVVYUNGzbgu+++w7Jly0x7JAZo3ZK9M0RERFIjOjgZP348CgsLMXfuXOTn5yM6OhopKSmKSbK5ublwcLjdIVNWVobnn38eFy5cQIsWLdC1a1d8//33GD9+vOmOwkDRIT6WbgIRERE1IhMEKY0yqVdSUgJvb28UFxfDy8vLpPsOnb3epPsjIiKyRneEt8bqZ/SfoqEPQ8/fdl1bh4iIiOpJqauCwQkRERFJaDosgxMiIiKSGAYnREREJKmuEwYnREREJCkMToiIiAitW7pYugkKDE6IiIgIIa1bWLoJCgxOiIiISFIYnBARERFkMpmlm6DA4OSWB3u3tXQTiIiICAxOFFyc+KsgIiL7JaVqNjwjExERkaQwOLlFQkNtREREzU5CHScMToiIiEhaGJwo6ezvYekmEBER2T27D07cnOt/BcMi/C3cEiIiIgIYnGDXrDvx45RYjIoMsHRTiIiILEZCU07gZOkGWJqvhyt8PVwt3QwiIiK6xe57ToiIiEhaGJwQERERlxJL1R3hbSzdBCIiIosQJDTrhMGJktljulq6CURERHaPwYmSlq52Pz+YiIjslAzSSZXO4ISIiIgkhcEJERERcc4JERERkSYMToiIiEhSGJwQERGRpDA4acTfk6nsiSwl87V4SzeBiCSAwUkj/l4MTogspQ3rXBFZDDPEStiSR/tYuglERER2jcFJI2G+LRHk7WbpZhDZnSWP9rZ0E4hIIhicqPHkwDBLN4HI7twTFWzpJhCRRDA4UeOpQWH4c/pAlfvWPT/AQq0hIiKyLwxO1HBwkKFXOx+V+7oGelmmMURERM1AkNCMWAYnenLgb4qIiKhZ8JSrJ1cnR3T297B0M4iIiMxCJmNVYqv0ckKE4v/dgznMQ0REtoPDOlZiaBc/jY85SCjCJCIiMpZ0QhMGJwYb1NnX0k0gIiKySQxOtNCWjG3GiM7N2BIi+9HWp4Wlm0BEFsbgRIvZY7ri7l5BWJXYr8ljbs6OFmgRke1bMK6npZtARBbmZOkGSJmPuws+fex2rZ1AL6a1JyIiMjcGJyJEhfhg3j2RaN/a3dJNISIislkc1hEpcWAYRnQLsHQziGyWu4v+Q6YbXxxixpYQkaWw54SIJKVP+1aY0C8Eq/flqdz/4cNRaNuqBb7efRZPDAiDVwsnRAR6WqiVRLZHQmlO2HNiTuG+LRX/f/Pe7hZsCZH1kMlkWDCuV5P7o0J8cEd4G3z+eF/EdWyD7sHeFmgdke0SJJTphMGJER7o3Vbr436eror/c3kkkThTBoep/NyJ5SOI7IZBwcnSpUsRGhoKNzc3xMbGIiMjQ+O2K1aswODBg9GqVSu0atUK8fHxWre3JoseidL6eEd+mRIZzMfdxdJNILIrVj2ss2bNGiQlJWHevHk4cOAAoqKikJCQgCtXrqjdPi0tDY8++ii2bduG9PR0hISEYNSoUbh48aLRjbc0XUWSZo3uqvi/8nv+vpouayIiIqonOjhZtGgRpkyZgsTERERGRmL58uVwd3fHypUr1W7/ww8/4Pnnn0d0dDS6du2KL7/8EnK5HKmpqUY3XspmJkTAu4Wz2sccHJq3Lg/zsxARkTURFZxUV1cjMzMT8fHxt3fg4ID4+Hikp6frtY/y8nLU1NSgdevW4lpq5SxZ7XHDjMEWe20iIrIOUqpnKyo4uXr1Kurq6hAQoJrnIyAgAPn5+XrtY9asWQgODlYJcBqrqqpCSUmJys0apM+509JNUKt1S47dExGR9WjW1ToLFizA6tWrsW7dOri5aR5qSE5Ohre3t+IWEhLSjK00XJA3V+QQmQqHI4mal9VOiPX19YWjoyMKCgpU7i8oKEBgYKDW5y5cuBALFizApk2b0KuX9gmhc+bMQXFxseKWl5endXtrI6GeMyLJul/HUv0G2qqHE5H+JBSbiAtOXFxcEBMTozKZtWFya1xcnMbnvf/++3jrrbeQkpKCvn376nwdV1dXeHl5qdysnZTedCJr4Oggw9E3E/CvO9rjxymxGrdrwQrhRDZHdPr6pKQkTJ48GX379kX//v2xePFilJWVITExEQAwadIktG3bFsnJyQCA9957D3PnzsWPP/6I0NBQxdwUDw8PeHgwD4it6h/WGhlnr1u6GWTlWro64e37e2rfiF2RRDZH9JyT8ePHY+HChZg7dy6io6ORlZWFlJQUxSTZ3NxcXL58WbH9smXLUF1djYceeghBQUGK28KFC013FFamd3sfSzdBtENzR4nanucLkqIAL1fdGxGRxRlU+G/69OmYPn262sfS0tJUfj537pwhL2GT9r0aj2tlVQj3s74eI2939TlbiCyNgTCR7WFtnWYS1c4Hfp6u6Boobv7MPVHBZmqRbXj97khLN4GIiEyMwYmZtG/tDgA4NG8Uds0ajkADVxQsebS3KZtlc2LD7CuZHxnuq8m6J+NLzd29gizdBCKLYHBiYj9OicXMhAiM7Vn/peLdwhntWrlbuFXAg3ouy9Tm4wnRorZf+LD2wojK/n1nJ5GtsU5c9mp6umpcNRjRLUD3RhLj6WbQyLtG/PyRtWBwYqTBnX1Vfh7Q0RfThndq9vo5jbV0UV1euWh8tMH7Gt29PofNfdHiApyHYtrh3IKxmJkQoXPbGSM6G9Q2a/P04HBLN0GSugZ6mmW/c8Z01b2RHXF25Fc+aRYX3sbSTVDgJ9VIr9zVzWT7aqVh0ukf0wbitbHd8Oa93U32WgCw87/D9drO0VF8oKWc18XPQ/sKiWPzE+BkJ1+anLzZPN68tzs+fDgKwyL8Ld0USenkb32T8an5tGslnSzn9nFGsBIpLw7BnV2bfplGhfjg6cHhmDwg1KSvF9Ja83BTRIAJr2R1nJHdnJom0ZrQzzpKFpA0TR4QinEx7SzdDIXM1zTXEmtODI7JWjA4kZAALzesfKKf1i7unm29VX4+Nj8B5xaMxb5X4/HF4zGK+3WNwy//Vx/jGmtC6rLnPqTnicXLjUuc7V1om5aWboJWM0Z0RhsdvYfNqVuQ9WfcJtvH4MTK9Gir+sXi7lI/Yc7P0xWjumuvb6QsKsTHoNcP9zXNiUBXMix90/239nDROFG3Ob6E7+qp/+/c1n0oYgK0KSU/qCODrAW9dV93TJfYZG8XA4ZpiZobgxMrozwptUMbzcMygo7ykroqKGvqeNn4nyE4/MYoeLoatorgtbHdEBfeBl9N7mfQ8xtzcpCp/E6eGhSm+H9zfAV/+mgfrHzC+paomoO/hbKv+nlqft1O/h7oGuiJAR0tM9FvWIQ/J6GS1ZBSDTj+1RjJVCWm37q/h17b3aE0mzpBRE+JqTg7OsDLzRk7Z2mfTKspMHh6cDh+euYOtHAxTbE2t0ZF35yacZXUo/3bw8FBBgc9l7JKzfi+5p3XI4V5Q44OMmx4YTB+eLpp4cBnhoRjZKRllxePUDPHzJyiDewxJWpuDE4k4vE7Oui9bepLQzF7TFf8J76Lxm3UzTnpI6Kmj3LQFawmN4KPu4ve+9L9WuaP1x0bBS33miDzbkPPlXLruwdbz3j+lCFhujcyQuPAURtDkxSqsypRtVfOwUGm9u/hlbu6YeqwjiZ7XXVatbz9d6JupUxzxrWv3x2JZ4ZyKTtZBwYnVqijnweeG9pRa++DKb7zvnmyPx6KaYcXzJCDxFzfycpd6OF+t+fHDOykmo9mwTjj5ik8OTAMiQNDm9y//oXBRu3XGrwwojO+nGTaoSxTlSHw93TVunx40SPRAOqHF5uDh9LwZ98OrZrlNZUdfmMUUl8aigOvj8RTg8Lg6uSoMSLakjSkmVtHpBmDE9JoaBc/LHw4Cp5WsCJmZkIEIgI8MUUpyVmHNu748elYbHyx6Zduw0RiQ829J7L+i14kKeURMJSrkwPi1QyHxIg8+foo5fXx1zJvxJQGdvLFyXfGKJLhmSPvR/9QU5ZUMC6M93JzRkc/D7Ruqbuns5O/eRLhERmCwYmFpL401OAVM4Ya0sUPAOBuovke5uTTQlxANG14J2z8z5Am1ZMHdPJFhJmyj4rVPdjL4nMczMndxUkx8bThs6ZNF39PvDSyC2YmRDRrAKzcu9Z4KXqolknmRLauo590kvQxOLGQjn4emBjb3mz791VzJfr8sE54/6Fe2JI01OSvp20+i69H06s25TkG6iaUdg7wxCt3dTXZHA4pTFl9bmhHvevAWKtlE2OQ/GBPLJmgX8HKf4/ojGnDpbHUNiLAEwvG9Wpyv6mWz5tialXqS0Nx8p0xxu+ISA1vkReF5mTaqlIkGXf1DMTNylr0VepidnFywCNmWqGx+pk4XCyqwPCFaU0eGx7hj2eGhKOHUgI5dxcn7H1lBBwdZBrrED0zpH6y4tFLJWZpsz3o7O8BmQw4UXCzWV7P290Zj/bXL+h++wH9Vqg1l2AfN7VB7DsP9ISbswMmr8zAikl9Mf6LvzXu468ZgzH3jyMmbJVqRCOlK1sic2JwYkHmvIZ2lMnw5n3N9+Xv4uSAMKUrTOWvVJlMprYGUYCXNCqkdvb3wLlrZaipk9Iqf+1aujiirLpO53av3x2JIV38EDp7fTO0SpwupiyRYGa927fC4TcSdG5nzsR/L8bbR3FMe+Xn6YrC0ipLN0MyOKxj51yYIArDzZRronEhx1Zqll9/8XhMk8rW+tj/2ki95nWEmWhIwl64qln+LJgxNZW6JfljewWp3fZFpdQBvz4Xh5dHdVHJWzKwk+GJ5tR152sqRGqoKYPNu3Rdl/n3mbZwqjno+i4wV/VuKeKZyc6lvmT6+Se25McpsXhqUBheuaur6Ocqzy95Zki42pPHqO6B+O6ppgnCdNGVxC71paH4beoArcUdqamodt66NzKh2WO6YmCnNmiv9D5Ft/PR+by+oa0x/c7OTfL3mNLu2Xfqve37DzWdq9Pg3qhgvH53JJJGRmjdR1y4ebP4ism7Yylv3ttd68oqc77fUsPgxM5J/eRlzNBPm5b6L0+VaRhkG9DRF6/fbdiyYWWv3NXNoMmwHzzUC5v+MwT/vDUazw3VL2FY+9bu6OjnIWpp77dP9kePtl54oVEdmHatWjRZYWTLc3plMhnS59yJN+9tnqtsXw9X/PD0HbhbqbfE3L9ffXcvZrm9v6crMl4ZgdiwpsuoX7mrG54aFKYzoFZXkV0fpkioKBXhfh5aK1ib4rOx9DHpFH3VhsEJSdrdvYIxZXAYlv8rRvfGt3z6WG+M69MOE++4PTFTzB/1mB71ZQGUu1h7KV1RP9yoYnKfkPogQJ9cEmI93DcEXQI8RV313R8t/st6cGdf/O/fg9GnUUDz6l3dsEJDwjVdV9b6ToyVmiDvFpg8IFTv7Z1ZSA8A4O/lhjXPxmFsT9VhKX2z/xpy4n12aDjefbAnHuzdVvfGVsKcK/rG9Wln1PBfc+KEWAlqhmzuOnWRyNimo4MMr44Vlz307l7BuLuX4VdTHzwchVHdA3Bn19s9Br3bt8L3T8UipHULdGijOo/D290Zh+aNgpuzdcX66nqWBnf2w8jIAGw+VqDz+W19bieUU3fVK5X8MuYWHeKDSXGhKkMz6vQK8caa/XlqH5PAn7xRLJlc0MPVCYvGR2PtwYtat2MIady8pOZmXd+mNkYqOS9WTOqLYRGqkyvnN1O3tmQovRUerk54oHe7JpMEB3X2bRKYNPBu4dxk6OfdW0tlNU1w1OW3qQO0Pm5sd3arli74cUos1j4/QPFZdHSQqfSU6PsR1aeQoyETf6Xopyl3qPw8a3RX3BMVrDOp4oR+7fG2ngU+b9P/O8KYixpjvorC/VpaTXZZdRPEH5BIr4spLkq1Vao31Ws0FwYnhJGRAViV2F/x84R+ISoFy0ic9Dl3IvWloRjdIwgHXx+JTx/VLyFZY7rmjIzro/5LVd33T5CGrvUBHX3Rp735a764Ojngu6dirSI7cWONV1nFdbx99fnvOzup5BLSxtFBhn+JKPCpD1MXzZw9RvzE74ZhUGvQWV3xRTO8zswE7ZN/dXF1MuzUvH2m9mrxgOb5dVLD4MRWSaRXxlqY8rcV5N1CkSyrVUsXUT1k+k56BcT1vP3v34OwYlJfky8PtWUfT4jGnDFdteYuUTcXSLnYnyEahiTFFgo0xZ/8oEYFMvVZri4VAV7NU59JH8Zmtt768jDTNMSKMTghszBHaPSEiEmKYpj66tMY2k4wHf0Mz1nSxsMVIyMD4OjAP3l93RfdFs9qCBaH3jppP6im9+qt+3ugVztvfDwh2qDXDfR2w9E3E/Dzs3EGPd+UVkyKwfoXBpll3w81mliubHiEnkGR0p+utfQI6EN5PpcpOVnR5G1+U5HVmHdPJNJeHmZQkPK4ibvTLeHBPu0wMyECvzzXfCctdb0zYk4CDXOZTD2cYWmrEvvh+PzRCPJuehJp18odf04fhPuiDZ/L0NLVSWNZB03McXJ2dXJE92CRuV/0bMYLd3YWPQ+pa6An3n2gp7j2KDF17RixlbhNyZAVSqM1DMH1bu+Dt0TPhzIvBicW1NIKx98tSSaTIdTAjKcjukmnGnBDIcSebcV96Ts6yDBteCf003OOgzEei22PHm29MDzCuOy5n03sg2+f7I9Zo+vnMkiok8ooMplMr0nAUtQwZ0ZM7PPhw1EY2KkNFjxYHxiYapm4k1IjYsN0ryQJ822Jx/QomKppYuisMV1xR3hrk7T/2aHhOietq+NjoqHVReOj1Q79fZ3YT+Nz1OVrcnF0wLrnB0ruAo7BiQWNjAzAXT0D8aqaujPWTornIKnko/jluQF4YkAovpikf+4WU1n0SBQA4LWx2j9z7z7QE//792C4GDgxr4G7ixOGdPEzej+27v5bvSymqsKtzYwRnfHmvd2x7eVhihVG9+vo5RkX0w4/PH0HJvRvj5PvjEHyg4b3XmgSpsewpb4TTVcl9leb78fXwxWrn4kzKBeQKfQPa41ds/TLvOvlpnvukrphmuER/hbt0TEV5jkxkq+n4atanBwd8NnEpicorxbW9ba8PKqL7o30MCmuA75NP2+2uSXaNOf84TDflnjDjEu1tfVODOnihxNvj2GwIDERgZ7Y92q8ya6qtXFzdlQkmfs2sT92n74qKjurc6N6XOYYTnp+eCdsyylUuW9AxzYI11KVOdTXHfkllQDq/8YWT+iNTccKUK6jQGZzXkh19GsJD1cnvPtAT5RW1mBByj9Kf6+qLWncLjE9XU8PCkPm+RsYHuHX5PdoLfgNZSR/TzesSuxn0nkAHzwUhah23lj+L+tIM2wqc++OxJ/TB+L1u8UlXbMlHZqhnIAhgYmnkStQSDc/T9cmJ35z83Z3xl09gyxSd0bbBUFbnxbw81RdfaOuroxyz8GiR6Jxd68gjUMtzZ1X6u85IzQ+9lhse42TrTWJFNGrNqZnEHbMHK4xu7OyLoGaAz5LYnBiAsMi/E06DyDUtyX+mD4Io3sYlrzLWGLr7dxhooJdTo4O6NXOx66KWzX4fdpALJvYB121LFtVJ/LW9kPNvORz/+ua632Q+TT0aGian2bqq/6ugZ7o5O9h9iJ8+nhyoGoV40ClOlszEyLQJcADUwaHK+4L9mmBTx/rI5khDXVp+x/so3mFkqm1b+MOJx3B7pAufqJKgzQnXg6Rwo9TYpF6/AqeGqRfafOMV0cg73o5YjqYf4Kmufm0cEF+TaXFXj86xAfRIT7IyisS9bxVT/bDn1mX8FBMO0TP32yexkH9RDoyv/hu/lj9zB1qk4eZg5OjAza9OKTZhjl7tfPROOzw7JBwxIa3xvWb1fjj0CXMUZqbN214J0wb3knt85RJ7TLHmItYQQAcGr0xj98Rir/PXDd4nwsf6gV/I4qrmhODE1IY0NEXAzrqv7TP39MN/p7N/8E2x3yJr57oi//+ehj/HS0+Q6Yl+Xu64Wmlq0dzCm3jjnPXyjGqu+ErnwRJTpVuXj3aeuHIxRJFr5c2MpnMZD2T+hK7hNkYU4d1hJuzI+7s6g9HpROvk6MMDg4yRfbi+EjprLZTuPVR/u/oCPy4NxcXblQ02SQiwBM5BaVma8LYXkH4Zk9rZJwzPECRKgYnZHWeG9oRqccLTNZFGuzTAt2DvbH+hcEm2Z+ltfEwT+mBTf8ZiqKKaosEpLbkq8n98FNGrtVWbTYlN2dHTB12e+7F88M6orpWblWfseeHdcLQLn4Y+8kuxX39w+p7SNZNG4DIuRsN2m//0NZI/eeKzu3C/VoyOCGSgtYtXZD60jDRz6tfVXD7yv3bJ/tj+4lCTIyV1vp+Q302sQ/Scq7olQfCEC5ODlZ10pCqAC83vBhvmhVutsbaei4b+DSqv+Tu4qT4N8y3Jc5eLRO9zw8ejsKqPefwSepJk7TR2jA4Ibs1pIufVdUO0eWunkG4q6dlJlGT/dI2P8XYOkPWwtB089qW/bdu6YKkkV3MGpxIeZCVq3VslD4JfIhsRcMKjUf6hli4JaRsS9JQi76+8vLh2DD1k1FNlfhOU3Xmlq76TSaPl1AWaylgcGJj3h/XCwndA2yulolJSG3qvhphBqbntxbmSl//zZP98c2T/TEjvrPB+zB13RVSv5zWUjSlSJg8IFRrxuQnB4bB18MVYb4tcfrduxT392m0ZFnTXK/F46PRLcgLyyZqz1s19x7z5ncKltB7oQ9eXtuYR/qF4JF+vHq0Vt4tnJHxyggu3RXJw9XJ4FwvnzzaGz/uPY9XdKT0l6LBnf1wMLfI0s0wSDPnRNPI2dEBTw8Ox6LNJ5pkkw3zbYm590Ti1VufDUcHGXbMHI5jl0swqtEKopdGRuBSUWWTasud/D3x1wzdk+0b5qk0FuTthsvFlYjvFoCtaibI6vt7fLihV1Eiv3ddGJwQSYxU8w7YqnujgnFvlGVqrRhr2vCOkpswGWiDn1/lxJDt27ijvZrCgq1aumDlE5qL7hnqz+mDkH7mGkZ3D1QbnNgqDuuQSd3dq35CpvLyQCIyDyn2sM2I74z7ooOx8gndqdPNrVe7+srf1lxLys/TFfdGBVv1MRiCPSdkUkse7Y1593RvUheDiOyDl5szPp7Q29LNAAAsnhCNZWmnMbHR8np9v5+6Bnqao1mkBwYnZFIymYyBCZEduCcqGP936JLZ8uqYgr+nG+bd07QCeLifBz58OAq+St9VylMxNrwwGD/vz8MLIwyfYE3Gsa9+IrJrVjIPzKrMGNEZnm5OmDkqQq/tm7vqrj3x9XBp1vken0yIRs7boxHkbViOj+auEtzYuJh2KpOoldsTGeyFN+7tjtYtzZNt2VCdjKixFO5nXSsB2XNCRAb7z8gumDGis971WFYl9sO0Hw/gDTVXs2QcN2dH7Jo1HIs2n8BnaafN/noymUySc15s2dy7I9HCxbHJiiBtfpsah8zzN3BPL+ua9M3ghIiMIqZQXN/Q1tj7SrwZW2N/Xr87Ej9l5CJpZBc4OTrAqRkL91HzatXSBe8+0FPUc2I6tLbKyvEMToiIrNhTg8Lw1KAwxc9STklOpC8OABMRUbOzl7o7ZBiDgpOlS5ciNDQUbm5uiI2NRUZGhsZtjx49inHjxiE0NBQymQyLFy82tK1ERGQD7okKlly5AKm1x96JDk7WrFmDpKQkzJs3DwcOHEBUVBQSEhJw5Yr6zHXl5eUIDw/HggULEBiovjASERHZj+ER0qsG/uXkvujR1gurEk2f5dW8bHOOkejgZNGiRZgyZQoSExMRGRmJ5cuXw93dHStXrlS7fb9+/fDBBx9gwoQJcHVl/gsiIpKebkFe+N+/B2NYhL+lm0IQGZxUV1cjMzMT8fG3Z9s7ODggPj4e6enpJmtUVVUVSkpKVG5ERERkH0QFJ1evXkVdXR0CAlSrMQYEBCA/P99kjUpOToa3t7fiFhLCKrtERET2QpKrdebMmYPi4mLFLS8vz9JNIhsglRLtREQAML5v/YX3gI5tLNwS6RG1lsvX1xeOjo4oKChQub+goMCkk11dXV05P4WIiGzam/d1x7AIPwzq7Ntsr+np6oSWLo6okQtoI7H0/MpE9Zy4uLggJiYGqampivvkcjlSU1MRFxdn8sYRERHZKjdnR4zpGQRPt+ZbxuzgIMOBuSNxeN4oOEm41pXoLDhJSUmYPHky+vbti/79+2Px4sUoKytDYmIiAGDSpElo27YtkpOTAdRPoj127Jji/xcvXkRWVhY8PDzQqVMnEx4KERER6WINNZFEByfjx49HYWEh5s6di/z8fERHRyMlJUUxSTY3NxcODrejsUuXLqF3796KnxcuXIiFCxdi6NChSEtLM/4IiIiIyKYYlD94+vTpmD59utrHGgccoaGhEARWeyAionruLtK/cifLYnEDIiJqFq+N7YaDuUUYGcls4aQdgxMiImoWTw8Ot3QTbM6zQ8LxU0YuHu1vW/nAGJyQ3Wjh7IjKGrmlm0FEZDKhvi1x4u0xcHGS7sobQ9jW0RBpsfKJfmjf2h2fPx5j6aYQEZmMrQUmAHtOyI70bt8KO/473NLNICIiHWwv3CIiIiKrxuCEiIiIJIXBCREREUkKgxMiIiKSFAYnREQ2pHuwl6WbQGQ0rtYhIrIhCd0D8f64XujZztvSTSEyGIMTIiIbIpPJ8Eg/28oWSvaHwzpEREQkKQxOiIiISFIYnBAREZGkMDghIiIiSWFwQkRERJLC4ISIiIgkhcEJERERSQqDEyIiIpIUBidEREQkKQxOiIiISFIYnBAREZGkMDghIiIiSWFwQkRERJJiFVWJBUEAAJSUlFi4JURERKSvhvN2w3lcX1YRnJSWlgIAQkJYBpyIiMjalJaWwtvbW+/tZYLYcMYC5HI5Ll26BE9PT8hkMpPtt6SkBCEhIcjLy4OXl5fJ9isltn6MPD7rZ+vHyOOzfrZ+jOY8PkEQUFpaiuDgYDg46D+TxCp6ThwcHNCuXTuz7d/Ly8smP3DKbP0YeXzWz9aPkcdn/Wz9GM11fGJ6TBpwQiwRERFJCoMTIiIikhS7Dk5cXV0xb948uLq6WropZmPrx8jjs362fow8Putn68coxeOzigmxREREZD/suueEiIiIpIfBCREREUkKgxMiIiKSFAYnREREJCl2HZwsXboUoaGhcHNzQ2xsLDIyMizdJLzxxhuQyWQqt65duyoer6ysxLRp09CmTRt4eHhg3LhxKCgoUNlHbm4uxo4dC3d3d/j7+2PmzJmora1V2SYtLQ19+vSBq6srOnXqhFWrVjVpiyl+Pzt27MA999yD4OBgyGQy/P777yqPC4KAuXPnIigoCC1atEB8fDxOnjypss3169cxceJEeHl5wcfHB0899RRu3rypss3hw4cxePBguLm5ISQkBO+//36Ttvzyyy/o2rUr3Nzc0LNnT2zYsEF0Www5xieeeKLJezp69GirOMbk5GT069cPnp6e8Pf3x/3334+cnByVbaT0mdSnLYYc47Bhw5q8h88995xVHOOyZcvQq1cvRYKtuLg4/PXXX6L2J9Vj0/cYrfn9U2fBggWQyWR48cUXRe3Xmo4Rgp1avXq14OLiIqxcuVI4evSoMGXKFMHHx0coKCiwaLvmzZsndO/eXbh8+bLiVlhYqHj8ueeeE0JCQoTU1FRh//79wh133CEMGDBA8Xhtba3Qo0cPIT4+Xjh48KCwYcMGwdfXV5gzZ45imzNnzgju7u5CUlKScOzYMWHJkiWCo6OjkJKSotjGVL+fDRs2CK+++qqwdu1aAYCwbt06lccXLFggeHt7C7///rtw6NAh4d577xXCwsKEiooKxTajR48WoqKihL///lvYuXOn0KlTJ+HRRx9VPF5cXCwEBAQIEydOFI4cOSL89NNPQosWLYTPP/9csc3u3bsFR0dH4f333xeOHTsmvPbaa4Kzs7OQnZ0tqi2GHOPkyZOF0aNHq7yn169fV9lGqseYkJAgfP3118KRI0eErKws4a677hLat28v3Lx5U7GNlD6Tutpi6DEOHTpUmDJlisp7WFxcbBXH+Oeffwrr168XTpw4IeTk5AivvPKK4OzsLBw5csQm3j99jtGa37/GMjIyhNDQUKFXr17CjBkz9N6vNR2jIAiC3QYn/fv3F6ZNm6b4ua6uTggODhaSk5Mt2Kr64CQqKkrtY0VFRYKzs7Pwyy+/KO47fvy4AEBIT08XBKH+ROng4CDk5+crtlm2bJng5eUlVFVVCYIgCP/973+F7t27q+x7/PjxQkJCguJnc/x+Gp+45XK5EBgYKHzwwQcqx+jq6ir89NNPgiAIwrFjxwQAwr59+xTb/PXXX4JMJhMuXrwoCIIgfPbZZ0KrVq0UxycIgjBr1iwhIiJC8fMjjzwijB07VqU9sbGxwrPPPqt3Www5RkGoD07uu+8+jc+xpmO8cuWKAEDYvn274vlS+Uzq0xZDjlEQ6k9uyieCxqztGFu1aiV8+eWXNvn+NT5GQbCd96+0tFTo3LmzsHnzZpVjssX30S6Hdaqrq5GZmYn4+HjFfQ4ODoiPj0d6eroFW1bv5MmTCA4ORnh4OCZOnIjc3FwAQGZmJmpqalTa3bVrV7Rv317R7vT0dPTs2RMBAQGKbRISElBSUoKjR48qtlHeR8M2Dftort/P2bNnkZ+fr/I63t7eiI2NVTkeHx8f9O3bV7FNfHw8HBwcsHfvXsU2Q4YMgYuLi8rx5OTk4MaNG3odsz5tMUZaWhr8/f0RERGBqVOn4tq1a4rHrOkYi4uLAQCtW7cGIK3PpD5tMeQYG/zwww/w9fVFjx49MGfOHJSXlyses5ZjrKurw+rVq1FWVoa4uDibfP8aH2MDW3j/pk2bhrFjxzZphy2+j1ZR+M/Url69irq6OpU3CQACAgLwzz//WKhV9WJjY7Fq1SpERETg8uXLePPNNzF48GAcOXIE+fn5cHFxgY+Pj8pzAgICkJ+fDwDIz89Xe1wNj2nbpqSkBBUVFbhx40az/H4a2qPudZTb6u/vr/K4k5MTWrdurbJNWFhYk300PNaqVSuNx6y8D11tMdTo0aPx4IMPIiwsDKdPn8Yrr7yCMWPGID09HY6OjlZzjHK5HC+++CIGDhyIHj16KPYplc+kPm0x5BgB4LHHHkOHDh0QHByMw4cPY9asWcjJycHatWut4hizs7MRFxeHyspKeHh4YN26dYiMjERWVpbNvH+ajhGw/vcPAFavXo0DBw5g3759TR6ztb9DwE6DEykbM2aM4v+9evVCbGwsOnTogJ9//hktWrSwYMvIUBMmTFD8v2fPnujVqxc6duyItLQ0jBgxwoItE2fatGk4cuQIdu3aZemmmI2mY3zmmWcU/+/ZsyeCgoIwYsQInD59Gh07dmzuZooWERGBrKwsFBcX49dff8XkyZOxfft2SzfLpDQdY2RkpNW/f3l5eZgxYwY2b94MNzc3SzenWdjlsI6vry8cHR2bzB4uKChAYGCghVqlno+PD7p06YJTp04hMDAQ1dXVKCoqUtlGud2BgYFqj6vhMW3beHl5oUWLFs32+2nYl7bXCQwMxJUrV1Qer62txfXr101yzMqP62qLqYSHh8PX1xenTp1SvLbUj3H69On43//+h23btqFdu3aK+6X0mdSnLYYcozqxsbEAoPIeSvkYXVxc0KlTJ8TExCA5ORlRUVH4+OOPber903SM6ljb+5eZmYkrV66gT58+cHJygpOTE7Zv345PPvkETk5OCAgIsJn3sYFdBicuLi6IiYlBamqq4j65XI7U1FSVMUopuHnzJk6fPo2goCDExMTA2dlZpd05OTnIzc1VtDsuLg7Z2dkqJ7vNmzfDy8tL0cUZFxenso+GbRr20Vy/n7CwMAQGBqq8TklJCfbu3atyPEVFRcjMzFRss3XrVsjlcsUXTFxcHHbs2IGamhqV44mIiECrVq30OmZ92mIqFy5cwLVr1xAUFCT5YxQEAdOnT8e6deuwdevWJkNLUvpM6tMWQ45RnaysLABQeQ+lfIyNyeVyVFVV2cT7p+sY1bG292/EiBHIzs5GVlaW4ta3b19MnDhR8X+bex/1njprY1avXi24uroKq1atEo4dOyY888wzgo+Pj8pMZkt46aWXhLS0NOHs2bPC7t27hfj4eMHX11e4cuWKIAj1S7Tat28vbN26Vdi/f78QFxcnxMXFKZ7fsFxs1KhRQlZWlpCSkiL4+fmpXS42c+ZM4fjx48LSpUvVLhczxe+ntLRUOHjwoHDw4EEBgLBo0SLh4MGDwvnz5wVBqF/a6uPjI/zxxx/C4cOHhfvuu0/tUuLevXsLe/fuFXbt2iV07txZZZltUVGREBAQIDz++OPCkSNHhNWrVwvu7u5Nltk6OTkJCxcuFI4fPy7MmzdP7TJbXW0Re4ylpaXCyy+/LKSnpwtnz54VtmzZIvTp00fo3LmzUFlZKfljnDp1quDt7S2kpaWpLMMsLy9XbCOlz6SuthhyjKdOnRLmz58v7N+/Xzh79qzwxx9/COHh4cKQIUOs4hhnz54tbN++XTh79qxw+PBhYfbs2YJMJhM2bdpkE++frmO09vdPk8YrkGzhfVRmt8GJIAjCkiVLhPbt2wsuLi5C//79hb///tvSTRLGjx8vBAUFCS4uLkLbtm2F8ePHC6dOnVI8XlFRITz//PNCq1atBHd3d+GBBx4QLl++rLKPc+fOCWPGjBFatGgh+Pr6Ci+99JJQU1Ojss22bduE6OhowcXFRQgPDxe+/vrrJm0xxe9n27ZtAoAmt8mTJwuCUL+89fXXXxcCAgIEV1dXYcSIEUJOTo7KPq5duyY8+uijgoeHh+Dl5SUkJiYKpaWlKtscOnRIGDRokODq6iq0bdtWWLBgQZO2/Pzzz0KXLl0EFxcXoXv37sL69etVHtenLWKPsby8XBg1apTg5+cnODs7Cx06dBCmTJnSJMiT6jGqOy4AKp8XKX0m9WmL2GPMzc0VhgwZIrRu3VpwdXUVOnXqJMycOVMlT4aUj/HJJ58UOnToILi4uAh+fn7CiBEjFIGJvvuT6rHpc4zW/v5p0jg4sYX3UZlMEARB/34WIiIiIvOyyzknREREJF0MToiIiEhSGJwQERGRpDA4ISIiIklhcEJERESSwuCEiIiIJIXBCREREUkKgxMiIiKSFAYnREREJCkMToiIiEhSGJwQERGRpDA4ISIiIkn5f3H0aJ8cuQe4AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "split_loss('train')\n",
    "split_loss('val')\n",
    "plt.plot(lossi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Summarizing the results so far\n",
    "\n",
    "- Initial results\n",
    "```\n",
    "train 2.135850667953491\n",
    "val 2.1770708560943604\n",
    "```\n",
    "\n",
    "- Results after softmax fix\n",
    "```\n",
    "train 2.0693342685699463\n",
    "val 2.1324031352996826\n",
    "```\n",
    "\n",
    "- Results after tanh fix\n",
    "```\n",
    "train 2.0361340045928955\n",
    "val 2.102936267852783\n",
    "```\n",
    "\n",
    "We've managed to improve the results to a validation loss of 2.109.\n",
    "\n",
    "## Autocalculating the scale-down factor\n",
    "\n",
    "In the example above we use 0.2 to scale down W1, this might seem random and it is. To make use of a semi-principled approach, let's look into how much we need to scale down W1 with an example\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.0082) tensor(0.9978) tensor(-0.0081) tensor(3.2544)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABkwAAAGsCAYAAACfEYteAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABLHUlEQVR4nO39f5RV9X0v/j/5EQZ/MWoIM0DQQSWiVRkLMsGYqM3UwdIoifGiKzcg10tWjBjtJBrwIxBr0kFFS6JUqr3+SFojdbXaVCyJnQZbl6NGkJWq0UaXFBVnEHNhFBuwzPn+ka/jnTAoZ5zxyJzHY629YPZ57/d57b3Omtnv9Tzv/R5QKBQKAQAAAAAAKGMDS10AAAAAAABAqQlMAAAAAACAsicwAQAAAAAAyp7ABAAAAAAAKHsCEwAAAAAAoOwJTAAAAAAAgLInMAEAAAAAAMre4FIX0Bs6OjqycePGHHDAARkwYECpywEAgD5XKBTy+uuvZ9SoURk40PegeG/GTQAAlJOejJn6RWCycePGjBkzptRlAADAB+7FF1/Mxz/+8VKXwV7AuAkAgHJUzJipXwQmBxxwQJLfnviwYcNKXA0AAPS99vb2jBkzpvNeGN6LcRMAAOWkJ2OmfhGYvD2dfNiwYW78AQAoKx6txJ4ybgIAoBwVM2bysGMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoewITAAAAAACg7AlMAAAAAACAsicwAQAAAAAAyp7ABAAAAAAAKHsCEwAAAAAAoOwJTAAAAAAAgLInMAEAAAAAAMqewAQAAAAAACh7AhMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoe4NLXQAAe7+aeSuLPmb94ml9UAkAAEBxih3PGMsA9F9mmAAAAAAAAGVPYAIAAAAAAJS9Hj2Sa9myZbn22mvT2tqaCRMm5IYbbsjkyZPf87i77ror5557bs4888zce++9nfsLhUIWLVqUW265JVu2bMmnPvWp3HTTTRk3blxPygMAAACAPuGRxAD9V9EzTFasWJHGxsYsWrQoa9euzYQJE9LQ0JBNmza963Hr16/PN7/5zXz605/e5bVrrrkm3//+97N8+fI8+uij2W+//dLQ0JDf/OY3xZYHAAAAAABQtKJnmFx//fWZM2dOZs+enSRZvnx5Vq5cmVtvvTXz5s3r9pidO3fmS1/6Uq688sr827/9W7Zs2dL5WqFQyNKlS3PFFVfkzDPPTJL84Ac/SFVVVe69996cc845PTgtAD7sLKwIAAAAwIdJUTNMduzYkTVr1qS+vv6dDgYOTH19fVpaWnZ73J/+6Z9mxIgROf/883d57YUXXkhra2uXPisrK1NXV7fbPrdv35729vYuGwAAAAAAQE8VFZhs3rw5O3fuTFVVVZf9VVVVaW1t7faYhx56KP/n//yf3HLLLd2+/vZxxfTZ1NSUysrKzm3MmDHFnAYAAAAAAEAXRa9hUozXX389X/7yl3PLLbdk+PDhvdbv/Pnzs3Xr1s7txRdf7LW+AQAAAACA8lPUGibDhw/PoEGD0tbW1mV/W1tbqqurd2n//PPPZ/369fnc5z7Xua+jo+O3bzx4cJ599tnO49ra2jJy5MgufdbW1nZbR0VFRSoqKoopHQAAAAAAYLeKmmEyZMiQTJw4Mc3NzZ37Ojo60tzcnClTpuzSfvz48fn3f//3rFu3rnM744wzcuqpp2bdunUZM2ZMxo4dm+rq6i59tre359FHH+22TwAAAAAAgN5W1AyTJGlsbMysWbMyadKkTJ48OUuXLs22bdsye/bsJMnMmTMzevToNDU1ZejQoTnmmGO6HH/ggQcmSZf9l1xySb7zne9k3LhxGTt2bBYsWJBRo0Zl+vTpPT8zAAAAAACAPVR0YDJjxoy8+uqrWbhwYVpbW1NbW5tVq1Z1Ltq+YcOGDBxY3NIol112WbZt25avfOUr2bJlS0466aSsWrUqQ4cOLbY8AAAAAACAog0oFAqFUhfxfrW3t6eysjJbt27NsGHDSl0OQNmpmbeyz99j/eJpff4eSc/O5YOqDeD/5R6YYvnMAHSvP41nAHhHT+5/i55hAgB7iw9i4AMAAABA/1Dcs7MAAAAAAAD6IYEJAAAAAABQ9gQmAAAAAABA2ROYAAAAAAAAZU9gAgAAAAAAlD2BCQAAAAAAUPYEJgAAAAAAQNkTmAAAAAAAAGVvcKkLAIA9UTNvZalLAAAA6JGejGfWL57WB5UA8G7MMAEAAAAAAMqewAQAAAAAACh7AhMAAAAAAKDsCUwAAAAAAICyJzABAADoZcuWLUtNTU2GDh2aurq6PPbYY7tt+9RTT+Wss85KTU1NBgwYkKVLl+7SpqmpKSeccEIOOOCAjBgxItOnT8+zzz7bh2cAAADlR2ACAADQi1asWJHGxsYsWrQoa9euzYQJE9LQ0JBNmzZ12/7NN9/MYYcdlsWLF6e6urrbNg8++GAuvPDCPPLII3nggQfy1ltv5bTTTsu2bdv68lQAAKCsDC51AQAAAP3J9ddfnzlz5mT27NlJkuXLl2flypW59dZbM2/evF3an3DCCTnhhBOSpNvXk2TVqlVdfr799tszYsSIrFmzJp/5zGd6+QwAAKA8mWECAADQS3bs2JE1a9akvr6+c9/AgQNTX1+flpaWXnufrVu3JkkOPvjg3bbZvn172tvbu2wAAMDuCUwAAAB6yebNm7Nz585UVVV12V9VVZXW1tZeeY+Ojo5ccskl+dSnPpVjjjlmt+2amppSWVnZuY0ZM6ZX3h8AAPorj+QCAADYi1x44YV58skn89BDD71ru/nz56exsbHz5/b2dqEJ0O/VzFtZ6hIA2IsJTAAAAHrJ8OHDM2jQoLS1tXXZ39bWttsF3Ysxd+7c3HffffnXf/3XfPzjH3/XthUVFamoqHjf7wkAAOXCI7kAAAB6yZAhQzJx4sQ0Nzd37uvo6Ehzc3OmTJnS434LhULmzp2be+65J//yL/+SsWPH9ka5AADA/8MMEwAAgF7U2NiYWbNmZdKkSZk8eXKWLl2abdu2Zfbs2UmSmTNnZvTo0Wlqakry24Xin3766c7/v/zyy1m3bl3233//HHHEEUl++xiuO++8M//wD/+QAw44oHM9lMrKyuyzzz4lOEsAAOh/BCYAdOGZvwDw/syYMSOvvvpqFi5cmNbW1tTW1mbVqlWdC8Fv2LAhAwe+M9l/48aNOf744zt/XrJkSZYsWZKTTz45q1evTpLcdNNNSZJTTjmly3vddtttOe+88/r0fAAAoFwITAD6MeEHAJTG3LlzM3fu3G5fezsEeVtNTU0KhcK79vderwMAAO+fNUwAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoewITAAAAAACg7AlMAAAAAACAsicwAQAAAAAAyt7gUhcAwJ6pmbey1CUAAAAAQL9lhgkAAAAAAFD2BCYAAAAAAEDZ61FgsmzZstTU1GTo0KGpq6vLY489ttu2f//3f59JkyblwAMPzH777Zfa2tr88Ic/7NLmvPPOy4ABA7psU6dO7UlpAAAAAAAARSt6DZMVK1aksbExy5cvT11dXZYuXZqGhoY8++yzGTFixC7tDz744Px//9//l/Hjx2fIkCG57777Mnv27IwYMSINDQ2d7aZOnZrbbrut8+eKiooenhIAfLB6sr7M+sXT+qASAAAAAHqq6Bkm119/febMmZPZs2fn6KOPzvLly7Pvvvvm1ltv7bb9Kaecks9//vM56qijcvjhh+fiiy/Occcdl4ceeqhLu4qKilRXV3duBx10UM/OCAAAAAAAoEhFBSY7duzImjVrUl9f/04HAwemvr4+LS0t73l8oVBIc3Nznn322XzmM5/p8trq1aszYsSIHHnkkbngggvy2muv7baf7du3p729vcsGAAAAAADQU0U9kmvz5s3ZuXNnqqqquuyvqqrKM888s9vjtm7dmtGjR2f79u0ZNGhQ/uIv/iJ/+Id/2Pn61KlT84UvfCFjx47N888/n8svvzynn356WlpaMmjQoF36a2pqypVXXllM6QAAAAAAALtV9BomPXHAAQdk3bp1eeONN9Lc3JzGxsYcdthhOeWUU5Ik55xzTmfbY489Nscdd1wOP/zwrF69Op/97Gd36W/+/PlpbGzs/Lm9vT1jxozp8/MAAAAAAAD6p6ICk+HDh2fQoEFpa2vrsr+trS3V1dW7PW7gwIE54ogjkiS1tbX55S9/maamps7A5HcddthhGT58eJ577rluA5OKigqLwgMAAAAAAL2mqDVMhgwZkokTJ6a5ublzX0dHR5qbmzNlypQ97qejoyPbt2/f7esvvfRSXnvttYwcObKY8gAAAAAAAHqk6EdyNTY2ZtasWZk0aVImT56cpUuXZtu2bZk9e3aSZObMmRk9enSampqS/Ha9kUmTJuXwww/P9u3bc//99+eHP/xhbrrppiTJG2+8kSuvvDJnnXVWqqur8/zzz+eyyy7LEUcckYaGhl48VQAAAAAAgO4VHZjMmDEjr776ahYuXJjW1tbU1tZm1apVnQvBb9iwIQMHvjNxZdu2bfna176Wl156Kfvss0/Gjx+fv/7rv86MGTOSJIMGDcovfvGL3HHHHdmyZUtGjRqV0047LVdddZXHbgEAAAAAAB+IAYVCoVDqIt6v9vb2VFZWZuvWrRk2bFipywHoEzXzVpa6BHrR+sXTSl0CsJdzD0yxfGaActCfxk3GDADvT0/uf4tawwQAAAAAAKA/EpgAAAAAAABlT2ACAAAAAACUPYEJAAAAAABQ9gaXugAAAAAAoKueLGBvoXiA98cMEwAAAAAAoOwJTAAAAAAAgLInMAEAAAAAAMqewAQAAAAAACh7AhMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoewITAAAAAACg7AlMAAAAAACAsje41AUAAAAAwO+qmbey1CUAUGbMMAEAAAAAAMqewAQAAAAAACh7AhMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoewITAAAAAACg7AlMAAAAAACAsicwAQAAAAAAyp7ABAAAoJctW7YsNTU1GTp0aOrq6vLYY4/ttu1TTz2Vs846KzU1NRkwYECWLl36vvsEAACKJzABAADoRStWrEhjY2MWLVqUtWvXZsKECWloaMimTZu6bf/mm2/msMMOy+LFi1NdXd0rfQIAAMUTmAAAAPSi66+/PnPmzMns2bNz9NFHZ/ny5dl3331z6623dtv+hBNOyLXXXptzzjknFRUVvdInAABQPIEJAABAL9mxY0fWrFmT+vr6zn0DBw5MfX19WlpaPtA+t2/fnvb29i4bAACwewITAACAXrJ58+bs3LkzVVVVXfZXVVWltbX1A+2zqakplZWVnduYMWN69P4AAFAuBCYAAAD90Pz587N169bO7cUXXyx1SQAA8KE2uNQFAAAA9BfDhw/PoEGD0tbW1mV/W1vbbhd076s+KyoqdrsmCgAAsCszTAAAAHrJkCFDMnHixDQ3N3fu6+joSHNzc6ZMmfKh6RMAANiVGSYAAAC9qLGxMbNmzcqkSZMyefLkLF26NNu2bcvs2bOTJDNnzszo0aPT1NSU5LeLuj/99NOd/3/55Zezbt267L///jniiCP2qE8AAOD969EMk2XLlqWmpiZDhw5NXV1dHnvssd22/fu///tMmjQpBx54YPbbb7/U1tbmhz/8YZc2hUIhCxcuzMiRI7PPPvukvr4+v/rVr3pSGgAAQEnNmDEjS5YsycKFC1NbW5t169Zl1apVnYu2b9iwIa+88kpn+40bN+b444/P8ccfn1deeSVLlizJ8ccfn//9v//3HvcJAAC8fwMKhUKhmANWrFiRmTNnZvny5amrq8vSpUtz991359lnn82IESN2ab969er83//7fzN+/PgMGTIk9913X77xjW9k5cqVaWhoSJJcffXVaWpqyh133JGxY8dmwYIF+fd///c8/fTTGTp06HvW1N7ensrKymzdujXDhg0r5nQA9ho181aWugR60frF00pdArCXcw9MsXxmgL2NMVDxjDMA3tGT+9+iZ5hcf/31mTNnTmbPnp2jjz46y5cvz7777ptbb7212/annHJKPv/5z+eoo47K4YcfnosvvjjHHXdcHnrooSS/nV2ydOnSXHHFFTnzzDNz3HHH5Qc/+EE2btyYe++9t9jyAAAAAAAAilZUYLJjx46sWbMm9fX173QwcGDq6+vT0tLynscXCoU0Nzfn2WefzWc+85kkyQsvvJDW1tYufVZWVqaurm63fW7fvj3t7e1dNgAAAAAAgJ4qatH3zZs3Z+fOnbs8J7eqqirPPPPMbo/bunVrRo8ene3bt2fQoEH5i7/4i/zhH/5hkqS1tbWzj9/t8+3XfldTU1OuvPLKYkoH+NAxvRwAAAAAPjx6tOh7sQ444ICsW7cuP//5z/Pd7343jY2NWb16dY/7mz9/frZu3dq5vfjii71XLAAAAAAAUHaKmmEyfPjwDBo0KG1tbV32t7W1pbq6erfHDRw4MEcccUSSpLa2Nr/85S/T1NSUU045pfO4tra2jBw5skuftbW13fZXUVGRioqKYkoHgA+VYmcYWbwRAAAAoG8VNcNkyJAhmThxYpqbmzv3dXR0pLm5OVOmTNnjfjo6OrJ9+/YkydixY1NdXd2lz/b29jz66KNF9QkAAAAAANBTRc0wSZLGxsbMmjUrkyZNyuTJk7N06dJs27Yts2fPTpLMnDkzo0ePTlNTU5LfrjcyadKkHH744dm+fXvuv//+/PCHP8xNN92UJBkwYEAuueSSfOc738m4ceMyduzYLFiwIKNGjcr06dN770wBAAAAAAB2o+jAZMaMGXn11VezcOHCtLa2pra2NqtWrepctH3Dhg0ZOPCdiSvbtm3L1772tbz00kvZZ599Mn78+Pz1X/91ZsyY0dnmsssuy7Zt2/KVr3wlW7ZsyUknnZRVq1Zl6NChvXCKAAAAAAAA725AoVAolLqI96u9vT2VlZXZunVrhg0bVupyAPZIsWtYUN6sYQL8LvfAFMtnBtjbGDMVz7gB4B09uf8tag0TAAAAAACA/khgAgAAAAAAlD2BCQAAAAAAUPYEJgAAAAAAQNkTmAAAAAAAAGVPYAIAAAAAAJQ9gQkAAAAAAFD2BCYAAAAAAEDZG1zqAgAAAACA969m3sqij1m/eFofVAKwdzLDBAAAAAAAKHsCEwAAAAAAoOwJTAAAAAAAgLInMAEAAAAAAMqewAQAAAAAACh7AhMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoewITAAAAAACg7A0udQEA/UHNvJWlLgEAAAAAeB/MMAEAAAAAAMqeGSYAsBfoySym9Yun9UElAABQPLPyAdgbmGECAAAAAACUPYEJAAAAAABQ9gQmAAAAAABA2ROYAAAAAAAAZU9gAgAAAAAAlD2BCQAAAAAAUPYEJgAAAAAAQNkTmAAAAAAAAGVPYAIAAAAAAJQ9gQkAAAAAAFD2BCYAAAAAAEDZE5gAAAAAAABlr0eBybJly1JTU5OhQ4emrq4ujz322G7b3nLLLfn0pz+dgw46KAcddFDq6+t3aX/eeedlwIABXbapU6f2pDQAAAAAAICiFR2YrFixIo2NjVm0aFHWrl2bCRMmpKGhIZs2beq2/erVq3PuuefmZz/7WVpaWjJmzJicdtppefnll7u0mzp1al555ZXO7Uc/+lHPzggAAAAAAKBIRQcm119/febMmZPZs2fn6KOPzvLly7Pvvvvm1ltv7bb93/zN3+RrX/taamtrM378+PzVX/1VOjo60tzc3KVdRUVFqqurO7eDDjqoZ2cEAABQYsXMyk+Su+++O+PHj8/QoUNz7LHH5v777+/y+htvvJG5c+fm4x//ePbZZ5/OsRgAANB7igpMduzYkTVr1qS+vv6dDgYOTH19fVpaWvaojzfffDNvvfVWDj744C77V69enREjRuTII4/MBRdckNdee223fWzfvj3t7e1dNgAAgA+DYmflP/zwwzn33HNz/vnn54knnsj06dMzffr0PPnkk51tGhsbs2rVqvz1X/91fvnLX+aSSy7J3Llz8+Mf//iDOi0AAOj3igpMNm/enJ07d6aqqqrL/qqqqrS2tu5RH9/61rcyatSoLqHL1KlT84Mf/CDNzc25+uqr8+CDD+b000/Pzp07u+2jqakplZWVnduYMWOKOQ0AAIA+U+ys/O9973uZOnVqLr300hx11FG56qqr8vu///u58cYbO9s8/PDDmTVrVk455ZTU1NTkK1/5SiZMmPCuM1d80QwAAIrTo0Xfe2rx4sW56667cs8992To0KGd+88555ycccYZOfbYYzN9+vTcd999+fnPf57Vq1d328/8+fOzdevWzu3FF1/8gM4AAABg93oyK7+lpaVL+yRpaGjo0v7EE0/Mj3/847z88sspFAr52c9+lv/4j//IaaedtttafNEMAACKU1RgMnz48AwaNChtbW1d9re1taW6uvpdj12yZEkWL16cn/70pznuuOPete1hhx2W4cOH57nnnuv29YqKigwbNqzLBgAAUGo9mZXf2tr6nu1vuOGGHH300fn4xz+eIUOGZOrUqVm2bFk+85nP7LYWXzQDAIDiFBWYDBkyJBMnTuyyYPvbC7hPmTJlt8ddc801ueqqq7Jq1apMmjTpPd/npZdeymuvvZaRI0cWUx4AAEC/dMMNN+SRRx7Jj3/846xZsybXXXddLrzwwvzzP//zbo/xRTMAACjO4GIPaGxszKxZszJp0qRMnjw5S5cuzbZt2zJ79uwkycyZMzN69Og0NTUlSa6++uosXLgwd955Z2pqajq/JbX//vtn//33zxtvvJErr7wyZ511Vqqrq/P888/nsssuyxFHHJGGhoZePFUAAIC+1ZNZ+dXV1e/a/r/+679y+eWX55577sm0adOSJMcdd1zWrVuXJUuW7PI4LwAAoGeKXsNkxowZWbJkSRYuXJja2tqsW7cuq1at6pxCvmHDhrzyyiud7W+66abs2LEjX/ziFzNy5MjObcmSJUmSQYMG5Re/+EXOOOOMfOITn8j555+fiRMn5t/+7d9SUVHRS6cJAADQ93oyK3/KlCld2ifJAw880Nn+rbfeyltvvZWBA7sO3wYNGpSOjo5ePgMAAChfRc8wSZK5c+dm7ty53b72uwu1r1+//l372mefffKTn/ykJ2UAAAB86BQ7K//iiy/OySefnOuuuy7Tpk3LXXfdlccffzw333xzkmTYsGE5+eSTc+mll2afffbJoYcemgcffDA/+MEPcv3115fsPAEAoL/pUWACAABA92bMmJFXX301CxcuTGtra2pra3eZlf//zhY58cQTc+edd+aKK67I5ZdfnnHjxuXee+/NMccc09nmrrvuyvz58/OlL30pv/71r3PooYfmu9/9br761a9+4OcHAAD91YBCoVAodRHvV3t7eyorK7N161YLGQIlUTNvZalLgF2sXzyt1CUAfcg9MMXymQFKyZjpw8u4AeivenL/W/QaJgAAAAAAAP2NwAQAAAAAACh7AhMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoe4NLXQAA0Ddq5q0s+pj1i6f1QSUAAAAAH35mmAAAAAAAAGVPYAIAAAAAAJQ9j+QCAAAAgDJV7KN8PcYX6M/MMAEAAAAAAMqeGSYAv6MnC2UDAAAAAHs3M0wAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoewITAAAAAACg7AlMAAAAAACAsicwAQAAAAAAyp7ABAAAAAAAKHsCEwAAAAAAoOwJTAAAAAAAgLInMAEAAAAAAMre4FIXAAB8eNTMW1n0MesXT+uDSgAAAAA+WGaYAAAAAAAAZU9gAgAAAAAAlD2P5AIAAABgj/XkMa4AsDcwwwQAAAAAACh7AhMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoewITAAAAAACg7PUoMFm2bFlqamoydOjQ1NXV5bHHHttt21tuuSWf/vSnc9BBB+Wggw5KfX39Lu0LhUIWLlyYkSNHZp999kl9fX1+9atf9aQ0AAAAAACAohUdmKxYsSKNjY1ZtGhR1q5dmwkTJqShoSGbNm3qtv3q1atz7rnn5mc/+1laWloyZsyYnHbaaXn55Zc721xzzTX5/ve/n+XLl+fRRx/Nfvvtl4aGhvzmN7/p+ZkBAAAAAADsoaIDk+uvvz5z5szJ7Nmzc/TRR2f58uXZd999c+utt3bb/m/+5m/yta99LbW1tRk/fnz+6q/+Kh0dHWlubk7y29klS5cuzRVXXJEzzzwzxx13XH7wgx9k48aNuffee9/XyQEAAAAAAOyJogKTHTt2ZM2aNamvr3+ng4EDU19fn5aWlj3q480338xbb72Vgw8+OEnywgsvpLW1tUuflZWVqaur222f27dvT3t7e5cNAAAAAACgpwYX03jz5s3ZuXNnqqqquuyvqqrKM888s0d9fOtb38qoUaM6A5LW1tbOPn63z7df+11NTU258soriykdKFM181aWugQAAAAAYC/Qo0Xfe2rx4sW56667cs8992To0KE97mf+/PnZunVr5/biiy/2YpUAAAAAAEC5KWqGyfDhwzNo0KC0tbV12d/W1pbq6up3PXbJkiVZvHhx/vmf/znHHXdc5/63j2tra8vIkSO79FlbW9ttXxUVFamoqCimdAAAAAAAgN0qaobJkCFDMnHixM4F25N0LuA+ZcqU3R53zTXX5KqrrsqqVasyadKkLq+NHTs21dXVXfpsb2/Po48++q59AgAAAAAA9JaiZpgkSWNjY2bNmpVJkyZl8uTJWbp0abZt25bZs2cnSWbOnJnRo0enqakpSXL11Vdn4cKFufPOO1NTU9O5Lsn++++f/fffPwMGDMgll1yS73znOxk3blzGjh2bBQsWZNSoUZk+fXrvnSkAAAAAAMBuFB2YzJgxI6+++moWLlyY1tbW1NbWZtWqVZ2Ltm/YsCEDB74zceWmm27Kjh078sUvfrFLP4sWLcq3v/3tJMlll12Wbdu25Stf+Uq2bNmSk046KatWrXpf65wAAAAAAADsqQGFQqFQ6iLer/b29lRWVmbr1q0ZNmxYqcsBPkRq5q0sdQnQ761fPK3UJUBZcg9MsXxmgN5inFXe3P8De4ue3P8WtYYJAAAAAABAfyQwAQAA6GXLli1LTU1Nhg4dmrq6ujz22GPv2v7uu+/O+PHjM3To0Bx77LG5//77d2nzy1/+MmeccUYqKyuz33775YQTTsiGDRv66hQAAKDsCEwAAAB60YoVK9LY2JhFixZl7dq1mTBhQhoaGrJp06Zu2z/88MM599xzc/755+eJJ57I9OnTM3369Dz55JOdbZ5//vmcdNJJGT9+fFavXp1f/OIXWbBggXUfAQCgF1nDBOjXPFsX+p5nGENpuAf+8Kqrq8sJJ5yQG2+8MUnS0dGRMWPG5KKLLsq8efN2aT9jxoxs27Yt9913X+e+T37yk6mtrc3y5cuTJOecc04+8pGP5Ic//GGP6/KZAXqLcVZ5c/8P7C2sYQIAAFBCO3bsyJo1a1JfX9+5b+DAgamvr09LS0u3x7S0tHRpnyQNDQ2d7Ts6OrJy5cp84hOfSENDQ0aMGJG6urrce++971rL9u3b097e3mUDAAB2T2ACAADQSzZv3pydO3emqqqqy/6qqqq0trZ2e0xra+u7tt+0aVPeeOONLF68OFOnTs1Pf/rTfP7zn88XvvCFPPjgg7utpampKZWVlZ3bmDFj3ufZAQBA/yYwAQAA+BDr6OhIkpx55pn5kz/5k9TW1mbevHn54z/+485HdnVn/vz52bp1a+f24osvflAlAwDAXmlwqQsAAADoL4YPH55Bgwalra2ty/62trZUV1d3e0x1dfW7th8+fHgGDx6co48+ukubo446Kg899NBua6moqEhFRUVPTgMAAMqSGSYAAAC9ZMiQIZk4cWKam5s793V0dKS5uTlTpkzp9pgpU6Z0aZ8kDzzwQGf7IUOG5IQTTsizzz7bpc1//Md/5NBDD+3lMwAAgPJlhgkAAEAvamxszKxZszJp0qRMnjw5S5cuzbZt2zJ79uwkycyZMzN69Og0NTUlSS6++OKcfPLJue666zJt2rTcddddefzxx3PzzTd39nnppZdmxowZ+cxnPpNTTz01q1atyj/+4z9m9erVpThFAADolwQmAAAAvWjGjBl59dVXs3DhwrS2tqa2tjarVq3qXNh9w4YNGTjwncn+J554Yu68885cccUVufzyyzNu3Ljce++9OeaYYzrbfP7zn8/y5cvT1NSUr3/96znyyCPzd3/3dznppJM+8PMDAID+akChUCiUuoj3q729PZWVldm6dWuGDRtW6nKAD5GaeStLXQL0e+sXTyt1CVCW3ANTLJ8ZoLcYZ5U39//A3qIn97/WMAEAAAAAAMqeR3IBew3fYgIAAAAA+ooZJgAAAAAAQNkTmAAAAAAAAGXPI7kAAAAAgD3Sk8dlWyge2FuYYQIAAAAAAJQ9gQkAAAAAAFD2BCYAAAAAAEDZE5gAAAAAAABlT2ACAAAAAACUPYEJAAAAAABQ9gQmAAAAAABA2ROYAAAAAAAAZU9gAgAAAAAAlD2BCQAAAAAAUPYEJgAAAAAAQNkTmAAAAAAAAGVPYAIAAAAAAJS9waUuAADYu9XMW1n0MesXT+uDSgAA6Ime3M8BQH8kMAEAPnDFDsoFLAAAAEBf80guAAAAAACg7AlMAAAAAACAstejwGTZsmWpqanJ0KFDU1dXl8cee2y3bZ966qmcddZZqampyYABA7J06dJd2nz729/OgAEDumzjx4/vSWkAAAAAAABFKzowWbFiRRobG7No0aKsXbs2EyZMSENDQzZt2tRt+zfffDOHHXZYFi9enOrq6t32+3u/93t55ZVXOreHHnqo2NIAAAAAAAB6pOjA5Prrr8+cOXMye/bsHH300Vm+fHn23Xff3Hrrrd22P+GEE3LttdfmnHPOSUVFxW77HTx4cKqrqzu34cOH77bt9u3b097e3mUDAAAAAADoqaICkx07dmTNmjWpr69/p4OBA1NfX5+Wlpb3VcivfvWrjBo1Kocddli+9KUvZcOGDbtt29TUlMrKys5tzJgx7+u9AQAAAACA8lZUYLJ58+bs3LkzVVVVXfZXVVWltbW1x0XU1dXl9ttvz6pVq3LTTTflhRdeyKc//em8/vrr3bafP39+tm7d2rm9+OKLPX5vAAAAAACAwaUuIElOP/30zv8fd9xxqaury6GHHpq//du/zfnnn79L+4qKind9vBcAAAAAAEAxipphMnz48AwaNChtbW1d9re1tb3rgu7FOvDAA/OJT3wizz33XK/1CQAAAAAAsDtFBSZDhgzJxIkT09zc3Lmvo6Mjzc3NmTJlSq8V9cYbb+T555/PyJEje61PAAAAAACA3Sn6kVyNjY2ZNWtWJk2alMmTJ2fp0qXZtm1bZs+enSSZOXNmRo8enaampiS/XSj+6aef7vz/yy+/nHXr1mX//ffPEUcckST55je/mc997nM59NBDs3HjxixatCiDBg3Kueee21vnCQAAAAAAsFtFByYzZszIq6++moULF6a1tTW1tbVZtWpV50LwGzZsyMCB70xc2bhxY44//vjOn5csWZIlS5bk5JNPzurVq5MkL730Us4999y89tpr+djHPpaTTjopjzzySD72sY+9z9MDAAAAAAB4bz1a9H3u3LmZO3dut6+9HYK8raamJoVC4V37u+uuu3pSBgAAAAAAQK8oag0TAAAAAACA/khgAgAAAAAAlD2BCQAAAAAAUPYEJgAAAAAAQNkTmAAAAAAAAGVPYAIAAAAAAJQ9gQkAAAAAAFD2BCYAAAAAAEDZE5gAAAAAAABlT2ACAAAAAACUPYEJAAAAAABQ9gQmAAAAAABA2Rtc6gIAAAD6m2XLluXaa69Na2trJkyYkBtuuCGTJ0/ebfu77747CxYsyPr16zNu3LhcffXV+aM/+qNu2371q1/NX/7lX+bP//zPc8kll/TRGQBA76mZt7LoY9YvntYHlQC8OzNMAAAAetGKFSvS2NiYRYsWZe3atZkwYUIaGhqyadOmbts//PDDOffcc3P++efniSeeyPTp0zN9+vQ8+eSTu7S955578sgjj2TUqFF9fRoAAFB2zDABSqIn3y4BANgbXH/99ZkzZ05mz56dJFm+fHlWrlyZW2+9NfPmzdul/fe+971MnTo1l156aZLkqquuygMPPJAbb7wxy5cv72z38ssv56KLLspPfvKTTJvmW7cAANDbzDABAADoJTt27MiaNWtSX1/fuW/gwIGpr69PS0tLt8e0tLR0aZ8kDQ0NXdp3dHTky1/+ci699NL83u/93h7Vsn379rS3t3fZAACA3ROYAAAA9JLNmzdn586dqaqq6rK/qqoqra2t3R7T2tr6nu2vvvrqDB48OF//+tf3uJampqZUVlZ2bmPGjCniTAAAoPwITAAAAD7E1qxZk+9973u5/fbbM2DAgD0+bv78+dm6dWvn9uKLL/ZhlQAAsPcTmAAAAPSS4cOHZ9CgQWlra+uyv62tLdXV1d0eU11d/a7t/+3f/i2bNm3KIYccksGDB2fw4MH5z//8z3zjG99ITU3NbmupqKjIsGHDumwAAMDuCUwAAAB6yZAhQzJx4sQ0Nzd37uvo6Ehzc3OmTJnS7TFTpkzp0j5JHnjggc72X/7yl/OLX/wi69at69xGjRqVSy+9ND/5yU/67mQAAKDMDC51AQAAAP1JY2NjZs2alUmTJmXy5MlZunRptm3bltmzZydJZs6cmdGjR6epqSlJcvHFF+fkk0/Oddddl2nTpuWuu+7K448/nptvvjlJ8tGPfjQf/ehHu7zHRz7ykVRXV+fII4/8YE8OAAD6MYEJAABAL5oxY0ZeffXVLFy4MK2tramtrc2qVas6F3bfsGFDBg58Z7L/iSeemDvvvDNXXHFFLr/88owbNy733ntvjjnmmFKdAgAAlCWBCQAAQC+bO3du5s6d2+1rq1ev3mXf2WefnbPPPnuP+1+/fn0PKwMAAHbHGiYAAAAAAEDZM8MEAAAAoJ+ombey1CUAwF7LDBMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnjVMgF7hObkAAAAAwN7MDBMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnsAEAAAAAAAoexZ9BwA+9GrmrSz6mPWLp/VBJQAAAEB/1aMZJsuWLUtNTU2GDh2aurq6PPbYY7tt+9RTT+Wss85KTU1NBgwYkKVLl77vPgEAAAAAAHpT0YHJihUr0tjYmEWLFmXt2rWZMGFCGhoasmnTpm7bv/nmmznssMOyePHiVFdX90qfAAAAAAAAvanowOT666/PnDlzMnv27Bx99NFZvnx59t1339x6663dtj/hhBNy7bXX5pxzzklFRUWv9Ll9+/a0t7d32QAAAAAAAHqqqMBkx44dWbNmTerr69/pYODA1NfXp6WlpUcF9KTPpqamVFZWdm5jxozp0XsDAAAAAAAkRQYmmzdvzs6dO1NVVdVlf1VVVVpbW3tUQE/6nD9/frZu3dq5vfjiiz16bwAAAAAAgCQZXOoCeqKiomK3j/cCAAAAAAAoVlEzTIYPH55Bgwalra2ty/62trbdLuheij4BAAAAAACKUVRgMmTIkEycODHNzc2d+zo6OtLc3JwpU6b0qIC+6BMAAAAAAKAYRT+Sq7GxMbNmzcqkSZMyefLkLF26NNu2bcvs2bOTJDNnzszo0aPT1NSU5LeLuj/99NOd/3/55Zezbt267L///jniiCP2qE8AAAAAAIC+VHRgMmPGjLz66qtZuHBhWltbU1tbm1WrVnUu2r5hw4YMHPjOxJWNGzfm+OOP7/x5yZIlWbJkSU4++eSsXr16j/oEAAAAAADoSz1a9H3u3LmZO3dut6+9HYK8raamJoVC4X31CQAAAAAA0Jd6FJgAAHzY1cxbWfQx6xdP64NKAACAYrmfB0pBYALsoic3JQAAAAAAe7OB790EAAAAAACgfxOYAAAAAAAAZU9gAgAAAAAAlD2BCQAAAAAAUPYEJgAAAAAAQNkTmAAAAAAAAGVPYAIAAAAAAJQ9gQkAAAAAAFD2Bpe6AAAAAAB2VTNvZalLAICyYoYJAAAAAABQ9gQmAAAAAABA2ROYAAAAAAAAZU9gAgAAAAAAlD2BCQAAAAAAUPYEJgAAAAAAQNkTmAAAAAAAAGVvcKkLAPpWzbyVpS4BYK/Rk9+Z6xdP64NKAAAAgA+aGSYAAAAAAEDZE5gAAAAAAABlT2ACAAAAAACUPYEJAAAAAABQ9gQmAAAAvWzZsmWpqanJ0KFDU1dXl8cee+xd2999990ZP358hg4dmmOPPTb3339/52tvvfVWvvWtb+XYY4/Nfvvtl1GjRmXmzJnZuHFjX58GAACUFYEJAABAL1qxYkUaGxuzaNGirF27NhMmTEhDQ0M2bdrUbfuHH3445557bs4///w88cQTmT59eqZPn54nn3wySfLmm29m7dq1WbBgQdauXZu///u/z7PPPpszzjjjgzwtAADo9wYUCoVCqYt4v9rb21NZWZmtW7dm2LBhpS4HPlRq5q0sdQkA/I71i6eVugT6AffAH151dXU54YQTcuONNyZJOjo6MmbMmFx00UWZN2/eLu1nzJiRbdu25b777uvc98lPfjK1tbVZvnx5t+/x85//PJMnT85//ud/5pBDDtmjunxmYO9jPAd9z7059F89uf81wwQAAKCX7NixI2vWrEl9fX3nvoEDB6a+vj4tLS3dHtPS0tKlfZI0NDTstn2SbN26NQMGDMiBBx642zbbt29Pe3t7lw0AANg9gQkAAEAv2bx5c3bu3Jmqqqou+6uqqtLa2trtMa2trUW1/81vfpNvfetbOffcc9/1m3JNTU2prKzs3MaMGVPk2QAAQHkRmAAAAOwl3nrrrfyP//E/UigUctNNN71r2/nz52fr1q2d24svvvgBVQkAAHunwaUuAAAAoL8YPnx4Bg0alLa2ti7729raUl1d3e0x1dXVe9T+7bDkP//zP/Mv//Iv7/kc5oqKilRUVPTgLAAAoDyZYQIAANBLhgwZkokTJ6a5ublzX0dHR5qbmzNlypRuj5kyZUqX9knywAMPdGn/dljyq1/9Kv/8z/+cj370o31zAgAAUMbMMAEAAOhFjY2NmTVrViZNmpTJkydn6dKl2bZtW2bPnp0kmTlzZkaPHp2mpqYkycUXX5yTTz451113XaZNm5a77rorjz/+eG6++eYkvw1LvvjFL2bt2rW57777snPnzs71TQ4++OAMGTKkNCcKAAD9jMAEAACgF82YMSOvvvpqFi5cmNbW1tTW1mbVqlWdC7tv2LAhAwe+M9n/xBNPzJ133pkrrrgil19+ecaNG5d77703xxxzTJLk5Zdfzo9//OMkSW1tbZf3+tnPfpZTTjnlAzkv4P2pmbey1CUAAO9BYAIAANDL5s6dm7lz53b72urVq3fZd/bZZ+fss8/utn1NTU0KhUJvlgcAAHSjR2uYLFu2LDU1NRk6dGjq6ury2GOPvWv7u+++O+PHj8/QoUNz7LHH5v777+/y+nnnnZcBAwZ02aZOndqT0gAAAAAAAIpWdGCyYsWKNDY2ZtGiRVm7dm0mTJiQhoaGbNq0qdv2Dz/8cM4999ycf/75eeKJJzJ9+vRMnz49Tz75ZJd2U6dOzSuvvNK5/ehHP+rZGQEAAAAAABSp6MDk+uuvz5w5czJ79uwcffTRWb58efbdd9/ceuut3bb/3ve+l6lTp+bSSy/NUUcdlauuuiq///u/nxtvvLFLu4qKilRXV3duBx100G5r2L59e9rb27tsAAAAAAAAPVVUYLJjx46sWbMm9fX173QwcGDq6+vT0tLS7TEtLS1d2idJQ0PDLu1Xr16dESNG5Mgjj8wFF1yQ1157bbd1NDU1pbKysnMbM2ZMMacBAAAAAADQRVGByebNm7Nz585UVVV12V9VVZXW1tZuj2ltbX3P9lOnTs0PfvCDNDc35+qrr86DDz6Y008/PTt37uy2z/nz52fr1q2d24svvljMaQAAAAAAAHQxuNQFJMk555zT+f9jjz02xx13XA4//PCsXr06n/3sZ3dpX1FRkYqKig+yRAAAAAAAoB8rKjAZPnx4Bg0alLa2ti7729raUl1d3e0x1dXVRbVPksMOOyzDhw/Pc889121gAuWqZt7KUpcAAAAAANAvFfVIriFDhmTixIlpbm7u3NfR0ZHm5uZMmTKl22OmTJnSpX2SPPDAA7ttnyQvvfRSXnvttYwcObKY8gAAAAAAAHqkqMAkSRobG3PLLbfkjjvuyC9/+ctccMEF2bZtW2bPnp0kmTlzZubPn9/Z/uKLL86qVaty3XXX5Zlnnsm3v/3tPP7445k7d26S5I033sill16aRx55JOvXr09zc3POPPPMHHHEEWloaOil0wQAAAAAANi9otcwmTFjRl599dUsXLgwra2tqa2tzapVqzoXdt+wYUMGDnwnhznxxBNz55135oorrsjll1+ecePG5d57780xxxyTJBk0aFB+8Ytf5I477siWLVsyatSonHbaabnqqqusUwIAAAAA9JliH3++fvG0PqoE+DDo0aLvc+fO7Zwh8rtWr169y76zzz47Z599drft99lnn/zkJz/pSRkAAAAAAAC9ouhHcgEAAAAAAPQ3AhMAAAAAAKDs9eiRXMD7V+wzMgEAAAAA6DsCEwAAAIAi+AIcAPRPHskFAAAAAACUPYEJAAAAAABQ9jySCwBgL1Dsoz/WL57WR5UAAABA/yQwgV7iGbYA7Cl/MwAAAODDxyO5AAAAAACAsicwAQAAAAAAyp7ABAAAAAAAKHsCEwAAAAAAoOxZ9B0AAAAAYA/UzFtZ9DHrF0/rg0qAvmCGCQAAAAAAUPYEJgAAAAAAQNkTmAAAAAAAAGXPGiYAAABA2erJegQAQP9khgkAAAAAAFD2zDABAOiHevJt2fWLp/VBJQAAALB3MMMEAAAAAAAoewITAAAAAACg7HkkFwAAAABAH/G4XNh7mGECAAAAAACUPYEJAAAAAABQ9jySi36vJ9MeAQAA2DsZAwIAPWWGCQAAAAAAUPbMMAEAAAAA+BCxUDyUhsAEAIAkBmUAAACUN4EJexXPogUAACgfxoAAwAfJGiYAAAAAAEDZM8MEAIAP1AfxbWGPCgMAAKBYAhMAAAAAgL2cNQnh/ROYUDKeRQsAez9/zwHYU/5mAAAfdgITAAAAAIAyZFYKdNWjwGTZsmW59tpr09ramgkTJuSGG27I5MmTd9v+7rvvzoIFC7J+/fqMGzcuV199df7oj/6o8/VCoZBFixbllltuyZYtW/KpT30qN910U8aNG9eT8gAAKHMf1LeYDRbZHWMm+juzRQCA/qjowGTFihVpbGzM8uXLU1dXl6VLl6ahoSHPPvtsRowYsUv7hx9+OOeee26ampryx3/8x7nzzjszffr0rF27Nsccc0yS5Jprrsn3v//93HHHHRk7dmwWLFiQhoaGPP300xk6dOj7P8sPqQ/zQN7NLwAA9IwxE3sb4z8AilHs3w1fMmJvMqBQKBSKOaCuri4nnHBCbrzxxiRJR0dHxowZk4suuijz5s3bpf2MGTOybdu23HfffZ37PvnJT6a2tjbLly9PoVDIqFGj8o1vfCPf/OY3kyRbt25NVVVVbr/99pxzzjm79Ll9+/Zs37698+etW7fmkEMOyYsvvphhw4YVczoldcyin3wg7/PklQ1FH/NB1QYAsDfryX1Wb2lvb8+YMWOyZcuWVFZWlqwOdvVhGDMl/WfcRPGM5wDY25XyPpv+oydjpqJmmOzYsSNr1qzJ/PnzO/cNHDgw9fX1aWlp6faYlpaWNDY2dtnX0NCQe++9N0nywgsvpLW1NfX19Z2vV1ZWpq6uLi0tLd3e/Dc1NeXKK6/cZf+YMWOKOZ2yUbm01BUAAPRPH4b7rNdff11g8iHyYRkzJcZNAMDe68Nwn03/UcyYqajAZPPmzdm5c2eqqqq67K+qqsozzzzT7TGtra3dtm9tbe18/e19u2vzu+bPn99lQNHR0ZFf//rX+ehHP5oBAwYUc0pJ3kmafNOq77jGfc817nuucd9zjT8YrnPfc437nmv82zUtXn/99YwaNarUpfD/+LCMmZLeHzeVE79j+p5r3Pdc477nGvc917jvucZ9zzXue7u7xj0ZM/Vo0fdSq6ioSEVFRZd9Bx544Pvud9iwYT60fcw17nuucd9zjfuea/zBcJ37nmvc98r9GptZwrvpq3FTOSn33zEfBNe477nGfc817nuucd9zjfuea9z3urvGxY6ZBhbTePjw4Rk0aFDa2tq67G9ra0t1dXW3x1RXV79r+7f/LaZPAACADyNjJgAA2HsVFZgMGTIkEydOTHNzc+e+jo6ONDc3Z8qUKd0eM2XKlC7tk+SBBx7obD927NhUV1d3adPe3p5HH310t30CAAB8GBkzAQDA3qvoR3I1NjZm1qxZmTRpUiZPnpylS5dm27ZtmT17dpJk5syZGT16dJqampIkF198cU4++eRcd911mTZtWu666648/vjjufnmm5MkAwYMyCWXXJLvfOc7GTduXMaOHZsFCxZk1KhRmT59eu+d6buoqKjIokWLdpmuTu9xjfuea9z3XOO+5xp/MFznvuca9z3XmA+z/jhmKjd+x/Q917jvucZ9zzXue65x33ON+55r3Pd68xoPKBQKhWIPuvHGG3PttdemtbU1tbW1+f73v5+6urokySmnnJKamprcfvvtne3vvvvuXHHFFVm/fn3GjRuXa665Jn/0R3/U+XqhUMiiRYty8803Z8uWLTnppJPyF3/xF/nEJz7xvk8QAADgg2bMBAAAe58eBSYAAAAAAAD9SVFrmAAAAAAAAPRHAhMAAAAAAKDsCUwAAAAAAICyJzABAAAAAADKnsBkN7Zv357a2toMGDAg69atK3U5/coZZ5yRQw45JEOHDs3IkSPz5S9/ORs3bix1Wf3G+vXrc/7552fs2LHZZ599cvjhh2fRokXZsWNHqUvrV7773e/mxBNPzL777psDDzyw1OX0G8uWLUtNTU2GDh2aurq6PPbYY6UuqV/513/913zuc5/LqFGjMmDAgNx7772lLqlfaWpqygknnJADDjggI0aMyPTp0/Pss8+Wuqx+56abbspxxx2XYcOGZdiwYZkyZUr+6Z/+qdRlAf3EntzjDRgwYJftrrvu+mAL3YvtyTXesGFDpk2bln333TcjRozIpZdemv/+7//+YAvtZ2pqanb53C5evLjUZe3VjF36zre//e1dPq/jx48vdVl7vfcaDxYKhSxcuDAjR47MPvvsk/r6+vzqV78qTbF7qfe6xuedd94un+2pU6eWpti90J6MuX/zm9/kwgsvzEc/+tHsv//+Oeuss9LW1lbU+whMduOyyy7LqFGjSl1Gv3Tqqafmb//2b/Pss8/m7/7u7/L888/ni1/8YqnL6jeeeeaZdHR05C//8i/z1FNP5c///M+zfPnyXH755aUurV/ZsWNHzj777FxwwQWlLqXfWLFiRRobG7No0aKsXbs2EyZMSENDQzZt2lTq0vqNbdu2ZcKECVm2bFmpS+mXHnzwwVx44YV55JFH8sADD+Stt97Kaaedlm3btpW6tH7l4x//eBYvXpw1a9bk8ccfzx/8wR/kzDPPzFNPPVXq0oB+YE/v8W677ba88sorndv06dM/mAL7gfe6xjt37sy0adOyY8eOPPzww7njjjty++23Z+HChR9wpf3Pn/7pn3b53F500UWlLmmvZezS937v936vy+f1oYceKnVJe733Gg9ec801+f73v5/ly5fn0UcfzX777ZeGhob85je/+YAr3XvtyZh76tSpXT7bP/rRjz7ACvduezLm/pM/+ZP84z/+Y+6+++48+OCD2bhxY77whS8U90YFdnH//fcXxo8fX3jqqacKSQpPPPFEqUvq1/7hH/6hMGDAgMKOHTtKXUq/dc011xTGjh1b6jL6pdtuu61QWVlZ6jL6hcmTJxcuvPDCzp937txZGDVqVKGpqamEVfVfSQr33HNPqcvo1zZt2lRIUnjwwQdLXUq/d9BBBxX+6q/+qtRlAP3Iu93j+RvaO3Z3je+///7CwIEDC62trZ37brrppsKwYcMK27dv/wAr7F8OPfTQwp//+Z+Xuox+w9ilby1atKgwYcKEUpfRr/3u37KOjo5CdXV14dprr+3ct2XLlkJFRUXhRz/6UQkq3Pt1d78wa9aswplnnlmSevqj3x1zb9mypfCRj3ykcPfdd3e2+eUvf1lIUmhpadnjfs0w+R1tbW2ZM2dOfvjDH2bfffctdTn93q9//ev8zd/8TU488cR85CMfKXU5/dbWrVtz8MEHl7oM2K0dO3ZkzZo1qa+v79w3cODA1NfXp6WlpYSVQc9t3bo1Sfz+7UM7d+7MXXfdlW3btmXKlCmlLgcoIxdeeGGGDx+eyZMn59Zbb02hUCh1Sf1GS0tLjj322FRVVXXua2hoSHt7u9mE79PixYvz0Y9+NMcff3yuvfZajznrIWOXD8avfvWrjBo1Kocddli+9KUvZcOGDaUuqV974YUX0tra2uVzXVlZmbq6Op/rXrZ69eqMGDEiRx55ZC644IK89tprpS5pr/W7Y+41a9bkrbfe6vI5Hj9+fA455JCiPseDe7fMvVuhUMh5552Xr371q5k0aVLWr19f6pL6rW9961u58cYb8+abb+aTn/xk7rvvvlKX1G8999xzueGGG7JkyZJSlwK7tXnz5uzcubPLwDhJqqqq8swzz5SoKui5jo6OXHLJJfnUpz6VY445ptTl9Dv//u//nilTpuQ3v/lN9t9//9xzzz05+uijS10WUCb+9E//NH/wB3+QfffdNz/96U/zta99LW+88Ua+/vWvl7q0fqG1tbXbe8K3X6Nnvv71r+f3f//3c/DBB+fhhx/O/Pnz88orr+T6668vdWl7HWOXvldXV5fbb789Rx55ZF555ZVceeWV+fSnP50nn3wyBxxwQKnL65fe/v3a3efa797eM3Xq1HzhC1/I2LFj8/zzz+fyyy/P6aefnpaWlgwaNKjU5e1Vuhtzt7a2ZsiQIbuskVbs57gsZpjMmzev24X5/t/tmWeeyQ033JDXX3898+fPL3XJe509vcZvu/TSS/PEE0/kpz/9aQYNGpSZM2f6VtZ7KPYaJ8nLL7+cqVOn5uyzz86cOXNKVPneoyfXGKA7F154YZ588kmLAPeRI488MuvWrcujjz6aCy64ILNmzcrTTz9d6rKAD6nevsdbsGBBPvWpT+X444/Pt771rVx22WW59tpr+/AMPvzcR5dGMde9sbExp5xySo477rh89atfzXXXXZcbbrgh27dvL/FZwK5OP/30nH322TnuuOPS0NCQ+++/P1u2bMnf/u3flro0eF/OOeecnHHGGTn22GMzffr03Hffffn5z3+e1atXl7q0vU5fjrnLYobJN77xjZx33nnv2uawww7Lv/zLv6SlpSUVFRVdXps0aVK+9KUv5Y477ujDKvdue3qN3zZ8+PAMHz48n/jEJ3LUUUdlzJgxeeSRRzxO410Ue403btyYU089NSeeeGJuvvnmPq6ufyj2GtN7hg8fnkGDBqWtra3L/ra2tlRXV5eoKuiZuXPn5r777su//uu/5uMf/3ipy+mXhgwZkiOOOCJJMnHixPz85z/P9773vfzlX/5liSsDPoz6+h6vrq4uV111VbZv377LWLJc9OY1rq6uzmOPPdZl39v3iO4Lu3o/172uri7//d//nfXr1+fII4/sg+r6L2OXD96BBx6YT3ziE3nuuedKXUq/9fZnt62tLSNHjuzc39bWltra2hJV1f8ddthhGT58eJ577rl89rOfLXU5e43djbmrq6uzY8eObNmypcssk2J/P5dFYPKxj30sH/vYx96z3fe///185zvf6fx548aNaWhoyIoVK1JXV9eXJe719vQad6ejoyNJfLPlPRRzjV9++eWceuqpmThxYm677bYMHFgWk8net/fzOeb9GTJkSCZOnJjm5uZMnz49yW9/NzQ3N2fu3LmlLQ72UKFQyEUXXZR77rknq1evztixY0tdUtno6OhwHwHsVl/f461bty4HHXRQ2YYlSe9e4ylTpuS73/1uNm3alBEjRiRJHnjggQwbNszjF3/H+7nu69aty8CBAzuvMXvO2OWD98Ybb+T555/Pl7/85VKX0m+NHTs21dXVaW5u7gxI2tvbO2d00zdeeumlvPbaa11CKnbvvcbcEydOzEc+8pE0NzfnrLPOSpI8++yz2bBhQ1Ff0i+LwGRPHXLIIV1+3n///ZMkhx9+uG+I9pJHH300P//5z3PSSSfloIMOyvPPP58FCxbk8MMPN7ukl7z88ss55ZRTcuihh2bJkiV59dVXO1/zbZfes2HDhvz617/Ohg0bsnPnzqxbty5JcsQRR3T+7qA4jY2NmTVrViZNmpTJkydn6dKl2bZtW2bPnl3q0vqNN954o8u3sl544YWsW7cuBx988C5/AynehRdemDvvvDP/8A//kAMOOKDzGamVlZXZZ599Slxd/zF//vycfvrpOeSQQ/L666/nzjvvzOrVq/OTn/yk1KUB/cB73eP94z/+Y9ra2vLJT34yQ4cOzQMPPJA/+7M/yze/+c3SFr4Xea9rfNppp+Xoo4/Ol7/85VxzzTVpbW3NFVdckQsvvLCsQ6n3o6WlJY8++mhOPfXUHHDAAWlpacmf/Mmf5H/+z/+Zgw46qNTl7ZWMXfrWN7/5zXzuc5/LoYcemo0bN2bRokUZNGhQzj333FKXtld7r/HgJZdcku985zsZN25cxo4dmwULFmTUqFGdwSDv7d2u8cEHH5wrr7wyZ511Vqqrq/P888/nsssuyxFHHJGGhoYSVr33eK8xd2VlZc4///w0Njbm4IMPzrBhw3LRRRdlypQp+eQnP7nnb1Rgt1544YVCksITTzxR6lL6jV/84heFU089tXDwwQcXKioqCjU1NYWvfvWrhZdeeqnUpfUbt912WyFJtxu9Z9asWd1e45/97GelLm2vdsMNNxQOOeSQwpAhQwqTJ08uPPLII6UuqV/52c9+1u3ndtasWaUurV/Y3e/e2267rdSl9Sv/63/9r8Khhx5aGDJkSOFjH/tY4bOf/Wzhpz/9aanLAvqJ97rH+6d/+qdCbW1tYf/99y/st99+hQkTJhSWL19e2LlzZ2kL34vsyX30+vXrC6effnphn332KQwfPrzwjW98o/DWW2+Vrui93Jo1awp1dXWFysrKwtChQwtHHXVU4c/+7M8Kv/nNb0pd2l7N2KXvzJgxozBy5MjCkCFDCqNHjy7MmDGj8Nxzz5W6rL3ee40HOzo6CgsWLChUVVUVKioqCp/97GcLzz77bGmL3su82zV+8803C6eddlrhYx/7WOEjH/lI4dBDDy3MmTOn0NraWuqy9xp7Mub+r//6r8LXvva1wkEHHVTYd999C5///OcLr7zySlHvM+D//2YAAAAAAABly8IGAAAAAABA2ROYAAAAAAAAZU9gAgAAAAAAlD2BCQAAAAAAUPYEJgAAAAAAQNkTmAAAAAAAAGVPYAIAAAAAAJQ9gQkAAAAAAFD2BCYAAAAAAEDZE5gAAAAAAABlT2ACAAAAAACUvf8fF3HFEt00vbgAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 2000x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = torch.randn(1000, 10)\n",
    "w = torch.randn(10, 200) \n",
    "y = x @ w\n",
    "print(x.mean(), x.std(), y.mean(), y.std())\n",
    "plt.figure(figsize=(20, 5))\n",
    "plt.subplot(121)\n",
    "plt.hist(x.view(-1).tolist(), bins=50, density=True);\n",
    "plt.subplot(122)\n",
    "plt.hist(y.view(-1).tolist(), bins=50, density=True);\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The figure on left shows `x` with std 0.9978 and figure on right shows `y` with std 3.2554. `y` takes on more extreme values. Let's see what happens when we multiply x by 0.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.0256) tensor(1.0056) tensor(-0.0015) tensor(0.6493)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABkoAAAGsCAYAAACSD/sZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABBDElEQVR4nO3df3DddZ0v/mcTbEIpLWBp0najaQtae7GN29JsFQXXSHC5Sr3KVMZray633gXqwOaqtK60IlxToNZcsVLFrfxQlq5exZ0tt+hmrLuMka7tdlB+rSC1BUxo8baBOKZOku8fDvEb22JPaHJozuMx8xl73uf9eZ/X5zMxnrfPvD/vMf39/f0BAAAAAAAoQWXFLgAAAAAAAKBYBCUAAAAAAEDJEpQAAAAAAAAlS1ACAAAAAACULEEJAAAAAABQsgQlAAAAAABAyRKUAAAAAAAAJeuEYhdwLPT19eWZZ57JySefnDFjxhS7HAAAGHb9/f15/vnnM3Xq1JSV+fsn/jTzJgAASkkhc6ZREZQ888wzqampKXYZAAAw4vbs2ZM/+7M/K3YZHAfMmwAAKEVHM2caFUHJySefnOT3FzxhwoQiVwMAAMOvq6srNTU1A9+F4U8xbwIAoJQUMmcaFUHJi8vGJ0yY4As/AAAlxSOUOFrmTQAAlKKjmTN5mDEAAAAAAFCyBCUAAAAAAEDJEpQAAAAAAAAlS1ACAAAAAACULEEJAAAAAABQsgQlAAAAAABAyRKUAAAAAAAAJUtQAgAAAAAAlCxBCQAAAAAAULIEJQAAAAAAQMkSlAAAAAAAACVLUAIAAAAAAJSsIQUl69evT21tbSorK1NfX59t27Yd1Xl33313xowZk0WLFg1q7+/vz6pVqzJlypSceOKJaWhoyM9//vOhlAYAAAAAAHDUCg5KNm3alObm5qxevTo7duzI3Llz09jYmGefffYlz9u1a1c+9rGP5a1vfesh79144435whe+kA0bNuSBBx7ISSedlMbGxvz2t78ttDwAAAAAAICjVnBQsm7duixbtixNTU2ZPXt2NmzYkHHjxmXjxo1HPKe3tzcf/OAHc+2112bGjBmD3uvv709ra2s+9alP5aKLLsqcOXNyxx135Jlnnsk999xT8AUBAAAAAAAcrYKCkoMHD2b79u1paGj4wwBlZWloaEh7e/sRz/vMZz6TyZMn59JLLz3kvSeffDIdHR2Dxpw4cWLq6+uPOGZPT0+6uroGHQAAAAAAAIU6oZDO+/btS29vb6qqqga1V1VV5dFHHz3sOffff3/+7u/+Ljt37jzs+x0dHQNj/PGYL773x1paWnLttdcWUjoAw6h2xeaCz9m15sJhqAQAAGB4mf8AjD5D2sz9aD3//PP50Ic+lFtvvTWTJk06ZuOuXLkyBw4cGDj27NlzzMYGAAAAAABKR0ErSiZNmpTy8vJ0dnYOau/s7Ex1dfUh/Z944ons2rUr7373uwfa+vr6fv/BJ5yQxx57bOC8zs7OTJkyZdCYdXV1h62joqIiFRUVhZQOAAAAAABwiIJWlIwdOzbz5s1LW1vbQFtfX1/a2tqycOHCQ/rPmjUrP/3pT7Nz586B4z3veU/e/va3Z+fOnampqcn06dNTXV09aMyurq488MADhx0TAAAAAADgWCloRUmSNDc3Z+nSpZk/f34WLFiQ1tbWdHd3p6mpKUmyZMmSTJs2LS0tLamsrMxZZ5016PxTTjklSQa1X3XVVbn++utz5plnZvr06bnmmmsyderULFq0aOhXBsCQDOV5uwAAAABwvCo4KFm8eHH27t2bVatWpaOjI3V1ddmyZcvAZuy7d+9OWVlhW5984hOfSHd3dz7ykY9k//79Oeecc7Jly5ZUVlYWWh4AAAAAAMBRG9Pf399f7CJerq6urkycODEHDhzIhAkTil0OwHFtpFaU7Fpz4Yh8DsBo5TswhfIzA3BsmDMBHB8K+f5b8IoSAI4fHqMFAAAAAC+tsGdkAQAAAAAAjCKCEgAAAAAAoGQJSgAAAAAAgJIlKAEAAAAAAEqWoAQAAAAAAChZghIAAAAAAKBkCUoAAAAAAICSJSgBAAAAAABKlqAEAAAAAAAoWYISAAAAAACgZAlKAAAAAACAkiUoAQAAAAAAStYJxS4AAAAAAIqldsXmYpcAQJFZUQIAAAAAAJQsQQkAAAAAAFCyBCUAAAAAAEDJskcJAEUxEs8B3rXmwmH/DAAAAACOb1aUAAAAAAAAJcuKEoDjxEiswAAAAACAUmNFCQAAAAAAULIEJQAAAAAAQMkSlAAAABTB+vXrU1tbm8rKytTX12fbtm0v2X///v254oorMmXKlFRUVOR1r3td7r333hGqFgAARi97lAAAAIywTZs2pbm5ORs2bEh9fX1aW1vT2NiYxx57LJMnTz6k/8GDB/POd74zkydPzre+9a1MmzYtv/zlL3PKKaeMfPEAADDKCEoAAABG2Lp167Js2bI0NTUlSTZs2JDNmzdn48aNWbFixSH9N27cmF//+tf50Y9+lFe96lVJktra2pf8jJ6envT09Ay87urqOnYXAAAAo4hHbwEAAIyggwcPZvv27WloaBhoKysrS0NDQ9rb2w97zj/+4z9m4cKFueKKK1JVVZWzzjorn/3sZ9Pb23vEz2lpacnEiRMHjpqammN+LQAAMBoISgAAAEbQvn370tvbm6qqqkHtVVVV6ejoOOw5v/jFL/Ktb30rvb29uffee3PNNdfkc5/7XK6//vojfs7KlStz4MCBgWPPnj3H9DoAAGC08OgtAACAV7i+vr5Mnjw5X/nKV1JeXp558+bl6aefzk033ZTVq1cf9pyKiopUVFSMcKUAHCu1KzYXfM6uNRcOQyUAo5+gBAAAYARNmjQp5eXl6ezsHNTe2dmZ6urqw54zZcqUvOpVr0p5eflA2xve8IZ0dHTk4MGDGTt27LDWDAAAo5lHbwEAAIygsWPHZt68eWlraxto6+vrS1tbWxYuXHjYc97ylrfk8ccfT19f30Dbf/zHf2TKlClCEgAAeJkEJQAAACOsubk5t956a26//fY88sgjueyyy9Ld3Z2mpqYkyZIlS7Jy5cqB/pdddll+/etf58orr8x//Md/ZPPmzfnsZz+bK664oliXAAAAo4ZHbwEAAIywxYsXZ+/evVm1alU6OjpSV1eXLVu2DGzwvnv37pSV/eHv2mpqanLfffflb/7mbzJnzpxMmzYtV155Za6++upiXQIAAIwaghIAAIAiWL58eZYvX37Y97Zu3XpI28KFC/PjH/94mKsCAIDS49FbAAAAAABAyRKUAAAAAAAAJWtIQcn69etTW1ubysrK1NfXZ9u2bUfs++1vfzvz58/PKaeckpNOOil1dXW58847B/X58Ic/nDFjxgw6LrjggqGUBgAAAAAAcNQK3qNk06ZNaW5uzoYNG1JfX5/W1tY0Njbmsccey+TJkw/pf9ppp+Vv//ZvM2vWrIwdOzb/9E//lKampkyePDmNjY0D/S644IJ87WtfG3hdUVExxEsCAAAAAAA4OgWvKFm3bl2WLVuWpqamzJ49Oxs2bMi4ceOycePGw/Y/77zz8t73vjdveMMbMnPmzFx55ZWZM2dO7r///kH9KioqUl1dPXCceuqpQ7siAAAAAACAo1RQUHLw4MFs3749DQ0NfxigrCwNDQ1pb2//k+f39/enra0tjz32WN72trcNem/r1q2ZPHlyXv/61+eyyy7Lc889d8Rxenp60tXVNegAAAAAAAAoVEGP3tq3b196e3tTVVU1qL2qqiqPPvroEc87cOBApk2blp6enpSXl+dLX/pS3vnOdw68f8EFF+S//Jf/kunTp+eJJ57IJz/5ybzrXe9Ke3t7ysvLDxmvpaUl1157bSGlAwAAAAAAHKLgPUqG4uSTT87OnTvzwgsvpK2tLc3NzZkxY0bOO++8JMkHPvCBgb5vfOMbM2fOnMycOTNbt27NO97xjkPGW7lyZZqbmwded3V1paamZtivAwAAAAAAGF0KCkomTZqU8vLydHZ2Dmrv7OxMdXX1Ec8rKyvLGWeckSSpq6vLI488kpaWloGg5I/NmDEjkyZNyuOPP37YoKSiosJm78Bxr3bF5mKXAAAAAAAlr6A9SsaOHZt58+alra1toK2vry9tbW1ZuHDhUY/T19eXnp6eI77/1FNP5bnnnsuUKVMKKQ8AAAAAAKAgBT96q7m5OUuXLs38+fOzYMGCtLa2pru7O01NTUmSJUuWZNq0aWlpaUny+/1E5s+fn5kzZ6anpyf33ntv7rzzztxyyy1JkhdeeCHXXntt3ve+96W6ujpPPPFEPvGJT+SMM85IY2PjMbxUAAAAAACAwQoOShYvXpy9e/dm1apV6ejoSF1dXbZs2TKwwfvu3btTVvaHhSrd3d25/PLL89RTT+XEE0/MrFmz8vWvfz2LFy9OkpSXl+fBBx/M7bffnv3792fq1Kk5//zzc91113m8FgAAAAAAMKzG9Pf39xe7iJerq6srEydOzIEDBzJhwoRilwNwVOxRMvx2rbmw2CUADBvfgSmUnxmAwxtNczNzIIA/KOT7b0F7lAAAAAAAAIwmghIAAAAAAKBkCUoAAAAAAICSJSgBAAAAAABK1gnFLgAAhstQNmW0+SEAABy/RtPG7ACMHEEJAPz/CFcAAAAASotHbwEAAAAAACVLUAIAAAAAAJQsQQkAAAAAAFCyBCUAAAAAAEDJEpQAAAAAAAAlS1ACAAAAAACULEEJAAAAAABQsgQlAAAAAABAyRKUAAAAAAAAJUtQAgAAAAAAlCxBCQAAAAAAULIEJQAAAAAAQMkSlAAAAAAAACVLUAIAAAAAAJQsQQkAAAAAAFCyBCUAAAAAAEDJEpQAAAAAAAAl64RiFwDwSlO7YnPB5+xac+EwVAIAAAAADDcrSgAAAAAAgJIlKAEAAAAAAEqWoAQAAAAAAChZ9igBAAAAgFHAnpsAQ2NFCQAAAAAAULIEJQAAAAAAQMkSlAAAAAAAACVLUAIAAFAE69evT21tbSorK1NfX59t27Ydse9tt92WMWPGDDoqKytHsFoAABi9BCUAAAAjbNOmTWlubs7q1auzY8eOzJ07N42NjXn22WePeM6ECRPyq1/9auD45S9/OYIVAwDA6CUoAQAAGGHr1q3LsmXL0tTUlNmzZ2fDhg0ZN25cNm7ceMRzxowZk+rq6oGjqqpqBCsGAIDRS1ACAAAwgg4ePJjt27enoaFhoK2srCwNDQ1pb28/4nkvvPBCXvva16ampiYXXXRRHnrooZf8nJ6ennR1dQ06AACAQw0pKCnkWbrf/va3M3/+/Jxyyik56aSTUldXlzvvvHNQn/7+/qxatSpTpkzJiSeemIaGhvz85z8fSmkAAACvaPv27Utvb+8hK0KqqqrS0dFx2HNe//rXZ+PGjfnud7+br3/96+nr68ub3/zmPPXUU0f8nJaWlkycOHHgqKmpOabXAQAAo0XBQUmhz9I97bTT8rd/+7dpb2/Pgw8+mKampjQ1NeW+++4b6HPjjTfmC1/4QjZs2JAHHnggJ510UhobG/Pb3/526FcGAAAwSixcuDBLlixJXV1dzj333Hz729/O6aefni9/+ctHPGflypU5cODAwLFnz54RrBgAAI4fBQclhT5L97zzzst73/vevOENb8jMmTNz5ZVXZs6cObn//vuT/H41SWtraz71qU/loosuypw5c3LHHXfkmWeeyT333HPYMS0hBwAAjleTJk1KeXl5Ojs7B7V3dnamurr6qMZ41atelTe96U15/PHHj9inoqIiEyZMGHQAAACHKigoGeqzdF/U39+ftra2PPbYY3nb296WJHnyySfT0dExaMyJEyemvr7+iGNaQg4AAByvxo4dm3nz5qWtrW2gra+vL21tbVm4cOFRjdHb25uf/vSnmTJlynCVCQAAJaOgoGQoz9JNkgMHDmT8+PEZO3ZsLrzwwtx888155zvfmSQD5xUypiXkAADA8ay5uTm33nprbr/99jzyyCO57LLL0t3dnaampiTJkiVLsnLlyoH+n/nMZ/K9730vv/jFL7Jjx4781//6X/PLX/4y//2///diXQIAAIwaJ4zEh5x88snZuXNnXnjhhbS1taW5uTkzZszIeeedN6TxKioqUlFRcWyLBHgZaldsLnYJAMBxZPHixdm7d29WrVqVjo6O1NXVZcuWLQN/QLZ79+6Ulf3h79r+3//7f1m2bFk6Ojpy6qmnZt68efnRj36U2bNnF+sSAABg1CgoKBnqs3TLyspyxhlnJEnq6uryyCOPpKWlJeedd97AeZ2dnYOWjXd2dqaurq6Q8gAAAI4by5cvz/Llyw/73tatWwe9/vznP5/Pf/7zI1AVAACUnoIevXUsnqX74jk9PT1JkunTp6e6unrQmF1dXXnggQcKGhMAAAAAAKBQBT96q7m5OUuXLs38+fOzYMGCtLa2HvIs3WnTpqWlpSXJ7zdenz9/fmbOnJmenp7ce++9ufPOO3PLLbckScaMGZOrrroq119/fc4888xMnz4911xzTaZOnZpFixYduysFAAAAAAD4IwUHJYU+S7e7uzuXX355nnrqqZx44omZNWtWvv71r2fx4sUDfT7xiU+ku7s7H/nIR7J///6cc8452bJlSyorK4/BJQIAAAAAABzemP7+/v5iF/FydXV1ZeLEiTlw4EAmTJhQ7HKA45yN2SnUrjUXFrsEoAT5Dkyh/MwApcB8rnDmM8BoVcj334L2KAEAAAAAABhNBCUAAAAAAEDJEpQAAAAAAAAlS1ACAAAAAACULEEJAAAAAABQsgQlAAAAAABAyTqh2AUAAAAAwB+rXbG52CUAUCKsKAEAAAAAAEqWoAQAAAAAAChZghIAAAAAAKBk2aMEAF6moTw7edeaC4ehEgAAAAAKZUUJAAAAAABQsgQlAAAAAABAyRKUAAAAAAAAJUtQAgAAAAAAlCxBCQAAAAAAULIEJQAAAAAAQMkSlAAAAAAAACVLUAIAAAAAAJQsQQkAAAAAAFCyBCUAAAAAAEDJEpQAAAAAAAAlS1ACAAAAAACULEEJAAAAAABQsgQlAAAAAABAyRKUAAAAAAAAJUtQAgAAAAAAlCxBCQAAAAAAULJOKHYBAAAAAEBx1K7YXFD/XWsuHKZKAIpHUAKMaoV+4QMAAAAASotHbwEAAAAAACVLUAIAAAAAAJQsQQkAAAAAAFCyBCUAAAAAAEDJGlJQsn79+tTW1qaysjL19fXZtm3bEfveeuuteetb35pTTz01p556ahoaGg7p/+EPfzhjxowZdFxwwQVDKQ0AAAAAAOCoFRyUbNq0Kc3NzVm9enV27NiRuXPnprGxMc8+++xh+2/dujWXXHJJfvCDH6S9vT01NTU5//zz8/TTTw/qd8EFF+RXv/rVwPH3f//3Q7siAAAAAACAo1RwULJu3bosW7YsTU1NmT17djZs2JBx48Zl48aNh+3/jW98I5dffnnq6uoya9asfPWrX01fX1/a2toG9auoqEh1dfXAceqppw7tigAAAAAAAI5SQUHJwYMHs3379jQ0NPxhgLKyNDQ0pL29/ajG+M1vfpPf/e53Oe200wa1b926NZMnT87rX//6XHbZZXnuueeOOEZPT0+6uroGHQAAAAAAAIUqKCjZt29fent7U1VVNai9qqoqHR0dRzXG1VdfnalTpw4KWy644ILccccdaWtryw033JAf/vCHede73pXe3t7DjtHS0pKJEycOHDU1NYVcBgAAAAAAQJLkhJH8sDVr1uTuu+/O1q1bU1lZOdD+gQ98YODfb3zjGzNnzpzMnDkzW7duzTve8Y5Dxlm5cmWam5sHXnd1dQlLAAAAAACAghW0omTSpEkpLy9PZ2fnoPbOzs5UV1e/5Llr167NmjVr8r3vfS9z5sx5yb4zZszIpEmT8vjjjx/2/YqKikyYMGHQAQAAAAAAUKiCgpKxY8dm3rx5gzZif3Fj9oULFx7xvBtvvDHXXXddtmzZkvnz5//Jz3nqqafy3HPPZcqUKYWUBwAAcNxYv359amtrU1lZmfr6+mzbtu2ozrv77rszZsyYLFq0aHgLBACAElHwo7eam5uzdOnSzJ8/PwsWLEhra2u6u7vT1NSUJFmyZEmmTZuWlpaWJMkNN9yQVatW5a677kptbe3AXibjx4/P+PHj88ILL+Taa6/N+973vlRXV+eJJ57IJz7xiZxxxhlpbGw8hpcKAK8ctSs2F9R/15oLh6kSAIph06ZNaW5uzoYNG1JfX5/W1tY0Njbmsccey+TJk4943q5du/Kxj30sb33rW0ewWgAAGN0KWlGSJIsXL87atWuzatWq1NXVZefOndmyZcvABu+7d+/Or371q4H+t9xySw4ePJj3v//9mTJlysCxdu3aJEl5eXkefPDBvOc978nrXve6XHrppZk3b17+9V//NRUVFcfoMgEAAF451q1bl2XLlqWpqSmzZ8/Ohg0bMm7cuGzcuPGI5/T29uaDH/xgrr322syYMWMEqwUAgNFtSJu5L1++PMuXLz/se1u3bh30eteuXS851oknnpj77rtvKGUAAAAcdw4ePJjt27dn5cqVA21lZWVpaGhIe3v7Ec/7zGc+k8mTJ+fSSy/Nv/7rv/7Jz+np6UlPT8/A666urpdXOAAAjFIFrygBAABg6Pbt25fe3t6BVfkvqqqqGnhU8R+7//7783d/93e59dZbj/pzWlpaMnHixIGjpqbmZdUNAACjlaAEAADgFez555/Phz70odx6662ZNGnSUZ+3cuXKHDhwYODYs2fPMFYJAADHryE9egsAAIChmTRpUsrLy9PZ2TmovbOzM9XV1Yf0f+KJJ7Jr1668+93vHmjr6+tLkpxwwgl57LHHMnPmzEPOq6iosO8jAAAcBStKAAAARtDYsWMzb968tLW1DbT19fWlra0tCxcuPKT/rFmz8tOf/jQ7d+4cON7znvfk7W9/e3bu3OmRWgAA8DJZUQIAADDCmpubs3Tp0syfPz8LFixIa2truru709TUlCRZsmRJpk2blpaWllRWVuass84adP4pp5ySJIe0AwAAhROUAAAAjLDFixdn7969WbVqVTo6OlJXV5ctW7YMbPC+e/fulJV5AAAAAIwEQQkAAEARLF++PMuXLz/se1u3bn3Jc2+77bZjXxAAAJQof6IEAAAAAACULEEJAAAAAABQsgQlAAAAAABAyRKUAAAAAAAAJUtQAgAAAAAAlKwTil0AAAAAAKNb7YrNxS4BAI7IihIAAAAAAKBkWVECAMeBofwF3q41Fw5DJQAAAACjixUlAAAAAABAyRKUAAAAAAAAJUtQAgAAAAAAlCxBCQAAAAAAULIEJQAAAAAAQMkSlAAAAAAAACVLUAIAAAAAAJQsQQkAAAAAAFCyTih2AQBHq3bF5mKXAAAAAACMMlaUAAAAAAAAJUtQAgAAAAAAlCxBCQAAAAAAULIEJQAAAAAAQMkSlAAAAAAAACVLUAIAAAAAAJQsQQkAAAAAAFCyBCUAAAAAAEDJEpQAAAAAAAAlS1ACAAAAAACUrBOKXQAAAAAAcHyoXbG54HN2rblwGCoBOHaGFJSsX78+N910Uzo6OjJ37tzcfPPNWbBgwWH73nrrrbnjjjvys5/9LEkyb968fPaznx3Uv7+/P6tXr86tt96a/fv35y1veUtuueWWnHnmmUMpDwCICQwAAADA0Sj40VubNm1Kc3NzVq9enR07dmTu3LlpbGzMs88+e9j+W7duzSWXXJIf/OAHaW9vT01NTc4///w8/fTTA31uvPHGfOELX8iGDRvywAMP5KSTTkpjY2N++9vfDv3KAAAAAAAA/oSCg5J169Zl2bJlaWpqyuzZs7Nhw4aMGzcuGzduPGz/b3zjG7n88stTV1eXWbNm5atf/Wr6+vrS1taW5PerSVpbW/OpT30qF110UebMmZM77rgjzzzzTO65556XdXEAAAAAAAAvpaCg5ODBg9m+fXsaGhr+MEBZWRoaGtLe3n5UY/zmN7/J7373u5x22mlJkieffDIdHR2Dxpw4cWLq6+uPOGZPT0+6uroGHQAAAAAAAIUqKCjZt29fent7U1VVNai9qqoqHR0dRzXG1VdfnalTpw4EIy+eV8iYLS0tmThx4sBRU1NTyGUAAAAAAAAkGcKjt16ONWvW5O677853vvOdVFZWDnmclStX5sCBAwPHnj17jmGVAAAAAABAqTihkM6TJk1KeXl5Ojs7B7V3dnamurr6Jc9du3Zt1qxZk3/+53/OnDlzBtpfPK+zszNTpkwZNGZdXd1hx6qoqEhFRUUhpQMAAAAAAByioBUlY8eOzbx58wY2Yk8ysDH7woULj3jejTfemOuuuy5btmzJ/PnzB703ffr0VFdXDxqzq6srDzzwwEuOCQAAAAAA8HIVtKIkSZqbm7N06dLMnz8/CxYsSGtra7q7u9PU1JQkWbJkSaZNm5aWlpYkyQ033JBVq1blrrvuSm1t7cC+I+PHj8/48eMzZsyYXHXVVbn++utz5plnZvr06bnmmmsyderULFq06NhdKQAAAAAAwB8pOChZvHhx9u7dm1WrVqWjoyN1dXXZsmXLwGbsu3fvTlnZHxaq3HLLLTl48GDe//73Dxpn9erV+fSnP50k+cQnPpHu7u585CMfyf79+3POOedky5YtL2sfEwAAAAAAgD9lTH9/f3+xi3i5urq6MnHixBw4cCATJkwodjnAMKldsbnYJcCot2vNhcUuAThKvgNTKD8zQDGZz5U28wygGAr5/lvQHiUAAAAAAACjiaAEAAAAAAAoWYISAAAAAACgZAlKAAAAAACAkiUoAQAAAAAASpagBAAAAAAAKFmCEgAAgCJYv359amtrU1lZmfr6+mzbtu2Ifb/97W9n/vz5OeWUU3LSSSelrq4ud9555whWCwAAo5egBAAAYIRt2rQpzc3NWb16dXbs2JG5c+emsbExzz777GH7n3baafnbv/3btLe358EHH0xTU1Oamppy3333jXDlAAAw+ghKAAAARti6deuybNmyNDU1Zfbs2dmwYUPGjRuXjRs3Hrb/eeedl/e+9715wxvekJkzZ+bKK6/MnDlzcv/99x/xM3p6etLV1TXoAAAADiUoAQAAGEEHDx7M9u3b09DQMNBWVlaWhoaGtLe3/8nz+/v709bWlsceeyxve9vbjtivpaUlEydOHDhqamqOSf0AADDaCEoAAABG0L59+9Lb25uqqqpB7VVVVeno6DjieQcOHMj48eMzduzYXHjhhbn55pvzzne+84j9V65cmQMHDgwce/bsOWbXAAAAo8kJxS4AAACAP+3kk0/Ozp0788ILL6StrS3Nzc2ZMWNGzjvvvMP2r6ioSEVFxcgWCQAAxyFBCVAUtSs2F7sEAICimDRpUsrLy9PZ2TmovbOzM9XV1Uc8r6ysLGeccUaSpK6uLo888khaWlqOGJQAAABHx6O3AAAARtDYsWMzb968tLW1DbT19fWlra0tCxcuPOpx+vr60tPTMxwlAgBASbGiBAAAYIQ1Nzdn6dKlmT9/fhYsWJDW1tZ0d3enqakpSbJkyZJMmzYtLS0tSX6/Mfv8+fMzc+bM9PT05N57782dd96ZW265pZiXAQAAo4KgBAAAYIQtXrw4e/fuzapVq9LR0ZG6urps2bJlYIP33bt3p6zsDw8A6O7uzuWXX56nnnoqJ554YmbNmpWvf/3rWbx4cbEuAShhHqUMwGgjKAEAACiC5cuXZ/ny5Yd9b+vWrYNeX3/99bn++utHoCoAACg99igBAAAAAABKlqAEAAAAAAAoWYISAAAAAACgZAlKAAAAAACAkiUoAQAAAAAASpagBAAAAAAAKFmCEgAAAAAAoGQJSgAAAAAAgJIlKAEAAAAAAEqWoAQAAAAAAChZghIAAAAAAKBkCUoAAAAAAICSJSgBAAAAAABKlqAEAAAAAAAoWYISAAAAAACgZAlKAAAAAACAknVCsQsARofaFZuLXQIAAAAAQMGsKAEAAAAAAErWkIKS9evXp7a2NpWVlamvr8+2bduO2Pehhx7K+973vtTW1mbMmDFpbW09pM+nP/3pjBkzZtAxa9asoZQGAAAAAABw1AoOSjZt2pTm5uasXr06O3bsyNy5c9PY2Jhnn332sP1/85vfZMaMGVmzZk2qq6uPOO5/+k//Kb/61a8Gjvvvv7/Q0gAAAAAAAApS8B4l69aty7Jly9LU1JQk2bBhQzZv3pyNGzdmxYoVh/Q/++yzc/bZZyfJYd8fKOSEE14ySAEAAAAAjj9D2dd015oLh6ESgMMraEXJwYMHs3379jQ0NPxhgLKyNDQ0pL29/WUV8vOf/zxTp07NjBkz8sEPfjC7d+8+Yt+enp50dXUNOgAAAAAAAApVUFCyb9++9Pb2pqqqalB7VVVVOjo6hlxEfX19brvttmzZsiW33HJLnnzyybz1rW/N888/f9j+LS0tmThx4sBRU1Mz5M8GAAAAAABK15A2cz/W3vWud+Xiiy/OnDlz0tjYmHvvvTf79+/PP/zDPxy2/8qVK3PgwIGBY8+ePSNcMQAAAAAAMBoUtEfJpEmTUl5ens7OzkHtnZ2dx3R/kVNOOSWve93r8vjjjx/2/YqKilRUVByzzwMAAAAAAEpTQStKxo4dm3nz5qWtrW2gra+vL21tbVm4cOExK+qFF17IE088kSlTphyzMQEAAAAAAP5YQStKkqS5uTlLly7N/Pnzs2DBgrS2tqa7uztNTU1JkiVLlmTatGlpaWlJ8vsN4B9++OGBfz/99NPZuXNnxo8fnzPOOCNJ8rGPfSzvfve789rXvjbPPPNMVq9enfLy8lxyySXH6joBAAAAAAAOUXBQsnjx4uzduzerVq1KR0dH6urqsmXLloEN3nfv3p2ysj8sVHnmmWfypje9aeD12rVrs3bt2px77rnZunVrkuSpp57KJZdckueeey6nn356zjnnnPz4xz/O6aef/jIvDwAAAAAA4MgKDkqSZPny5Vm+fPlh33sx/HhRbW1t+vv7X3K8u+++eyhlAAAAAAAAvCwF7VECAAAAAAAwmgxpRQkAMDrVrthc8Dm71lw4DJUAAAAAjAwrSgAAAAAAgJIlKAEAAAAAAEqWoAQAAAAAAChZghIAAAAAAKBkCUoAAAAAAICSJSgBAAAAAABKlqAEAAAAAAAoWYISAAAAAACgZAlKAAAAAACAknVCsQsAXnlqV2wudgnAcWQovzN2rblwGCoBAGAozAEBKHVWlAAAAAAAACVLUAIAAAAAAJQsQQkAAAAAAFCyBCUAAABFsH79+tTW1qaysjL19fXZtm3bEfveeuuteetb35pTTz01p556ahoaGl6yPwAAcPQEJQAAACNs06ZNaW5uzurVq7Njx47MnTs3jY2NefbZZw/bf+vWrbnkkkvygx/8IO3t7ampqcn555+fp59+eoQrBwCA0UdQAgAAMMLWrVuXZcuWpampKbNnz86GDRsybty4bNy48bD9v/GNb+Tyyy9PXV1dZs2ala9+9avp6+tLW1vbCFcOAACjj6AEAABgBB08eDDbt29PQ0PDQFtZWVkaGhrS3t5+VGP85je/ye9+97ucdtppR+zT09OTrq6uQQcAAHAoQQkAAMAI2rdvX3p7e1NVVTWovaqqKh0dHUc1xtVXX52pU6cOClv+WEtLSyZOnDhw1NTUvKy6AQBgtBKUAAAAHEfWrFmTu+++O9/5zndSWVl5xH4rV67MgQMHBo49e/aMYJUAAHD8OKHYBQAAAJSSSZMmpby8PJ2dnYPaOzs7U11d/ZLnrl27NmvWrMk///M/Z86cOS/Zt6KiIhUVFS+7XgAAGO2sKAEAABhBY8eOzbx58wZtxP7ixuwLFy484nk33nhjrrvuumzZsiXz588fiVIBAKAkWFECAAAwwpqbm7N06dLMnz8/CxYsSGtra7q7u9PU1JQkWbJkSaZNm5aWlpYkyQ033JBVq1blrrvuSm1t7cBeJuPHj8/48eOLdh0AADAaCEoAAABG2OLFi7N3796sWrUqHR0dqaury5YtWwY2eN+9e3fKyv7wAIBbbrklBw8ezPvf//5B46xevTqf/vSnR7J0AAAYdQQlAAAARbB8+fIsX778sO9t3bp10Otdu3YNf0EAAFCiBCUAAAAAwCtK7YrNBZ+za82Fw1AJUAps5g4AAAAAAJQsQQkAAAAAAFCyBCUAAAAAAEDJEpQAAAAAAAAlS1ACAAAAAACULEEJAAAAAABQsgQlAAAAAABAyRKUAAAAAAAAJWtIQcn69etTW1ubysrK1NfXZ9u2bUfs+9BDD+V973tfamtrM2bMmLS2tr7sMQEAAAAAAI6FEwo9YdOmTWlubs6GDRtSX1+f1tbWNDY25rHHHsvkyZMP6f+b3/wmM2bMyMUXX5y/+Zu/OSZjAgDHt9oVmwvqv2vNhcNUCQAAAFDqCg5K1q1bl2XLlqWpqSlJsmHDhmzevDkbN27MihUrDul/9tln5+yzz06Sw74/lDGBo1fo/xkJAAAAAFBKCnr01sGDB7N9+/Y0NDT8YYCysjQ0NKS9vX1IBQxlzJ6ennR1dQ06AAAAAAAAClVQULJv37709vamqqpqUHtVVVU6OjqGVMBQxmxpacnEiRMHjpqamiF9NgAAAAAAUNqGtJl7sa1cuTIHDhwYOPbs2VPskgAAAAAAgONQQXuUTJo0KeXl5ens7BzU3tnZmerq6iEVMJQxKyoqUlFRMaTPAwAAAAAAeFFBK0rGjh2befPmpa2tbaCtr68vbW1tWbhw4ZAKGI4xAQAAAAAAjkZBK0qSpLm5OUuXLs38+fOzYMGCtLa2pru7O01NTUmSJUuWZNq0aWlpaUny+83aH3744YF/P/3009m5c2fGjx+fM84446jGBAAAAAAAGA4FByWLFy/O3r17s2rVqnR0dKSuri5btmwZ2Ix99+7dKSv7w0KVZ555Jm9605sGXq9duzZr167Nueeem61btx7VmAAAAAAAAMOh4KAkSZYvX57ly5cf9r0Xw48X1dbWpr+//2WNCQAAAAAAMByGFJQAAAAA8MpTu2JzsUsAgONOQZu5AwAAAAAAjCaCEgAAAAAAoGQJSgAAAAAAgJIlKAEAAAAAAEqWoAQAAAAAAChZghIAAAAAAKBkCUoAAAAAAICSJSgBAAAAAABKlqAEAAAAAAAoWYISAAAAAACgZAlKAAAAAACAknVCsQsAAPhTaldsLvicXWsuHIZKAAAAgNFGUALHkaH8H4UAAAAAAByZoAQAAAAAOO5ZiQ4MlT1KAAAAAACAkiUoAQAAAAAASpagBAAAAAAAKFmCEgAAAAAAoGQJSgAAAAAAgJIlKAEAAAAAAEqWoAQAAAAAAChZghIAAAAAAKBkCUoAAAAAAICSJSgBAAAAAABK1gnFLgAAAKAUrV+/PjfddFM6Ojoyd+7c3HzzzVmwYMFh+z700ENZtWpVtm/fnl/+8pf5/Oc/n6uuumpkCwZGXO2KzcUuAQBKghUlAAAAI2zTpk1pbm7O6tWrs2PHjsydOzeNjY159tlnD9v/N7/5TWbMmJE1a9akurp6hKsFAIDRTVACAAAwwtatW5dly5alqakps2fPzoYNGzJu3Lhs3LjxsP3PPvvs3HTTTfnABz6QioqKEa4WAABGN4/egiKxhBpgeA3l9+yuNRcOQyUAgx08eDDbt2/PypUrB9rKysrS0NCQ9vb2Y/Y5PT096enpGXjd1dV1zMYGAIDRxIoSAACAEbRv37709vamqqpqUHtVVVU6OjqO2ee0tLRk4sSJA0dNTc0xGxsAAEYTQQkAAMAotHLlyhw4cGDg2LNnT7FLAgCAVySP3gIAABhBkyZNSnl5eTo7Owe1d3Z2HtON2isqKuxnAgAAR8GKEgAAgBE0duzYzJs3L21tbQNtfX19aWtry8KFC4tYGQAAlCYrSgAAAEZYc3Nzli5dmvnz52fBggVpbW1Nd3d3mpqakiRLlizJtGnT0tLSkuT3G8A//PDDA/9++umns3PnzowfPz5nnHFG0a4DAI53tSs2F9R/15oLh6kSoJgEJQAAACNs8eLF2bt3b1atWpWOjo7U1dVly5YtAxu87969O2Vlf3gAwDPPPJM3velNA6/Xrl2btWvX5txzz83WrVtHunwAABhVhvTorfXr16e2tjaVlZWpr6/Ptm3bXrL/N7/5zcyaNSuVlZV54xvfmHvvvXfQ+x/+8IczZsyYQccFF1wwlNIAAACOC8uXL88vf/nL9PT05IEHHkh9ff3Ae1u3bs1tt9028Lq2tjb9/f2HHEISAAB4+QoOSjZt2pTm5uasXr06O3bsyNy5c9PY2Jhnn332sP1/9KMf5ZJLLsmll16af//3f8+iRYuyaNGi/OxnPxvU74ILLsivfvWrgePv//7vh3ZFAAAAAAAAR6ngoGTdunVZtmxZmpqaMnv27GzYsCHjxo3Lxo0bD9v/f//v/50LLrggH//4x/OGN7wh1113Xf78z/88X/ziFwf1q6ioSHV19cBx6qmnHrGGnp6edHV1DToAAAAAAAAKVdAeJQcPHsz27duzcuXKgbaysrI0NDSkvb39sOe0t7enubl5UFtjY2PuueeeQW1bt27N5MmTc+qpp+Yv//Ivc/311+fVr371YcdsaWnJtddeW0jpMOwK3fwLAAAAAIDiK2hFyb59+9Lb2zuwweCLqqqq0tHRcdhzOjo6/mT/Cy64IHfccUfa2tpyww035Ic//GHe9a53pbe397Bjrly5MgcOHBg49uzZU8hlAAAAAAAAJClwRclw+cAHPjDw7ze+8Y2ZM2dOZs6cma1bt+Yd73jHIf0rKipSUVExkiUCAAAAAACjUEErSiZNmpTy8vJ0dnYOau/s7Ex1dfVhz6muri6of5LMmDEjkyZNyuOPP15IeQAAAAAAAAUpKCgZO3Zs5s2bl7a2toG2vr6+tLW1ZeHChYc9Z+HChYP6J8n3v//9I/ZPkqeeeirPPfdcpkyZUkh5AAAAAAAABSkoKEmS5ubm3Hrrrbn99tvzyCOP5LLLLkt3d3eampqSJEuWLBm02fuVV16ZLVu25HOf+1weffTRfPrTn85PfvKTLF++PEnywgsv5OMf/3h+/OMfZ9euXWlra8tFF12UM844I42NjcfoMgEAAAAAAA5V8B4lixcvzt69e7Nq1ap0dHSkrq4uW7ZsGdiwfffu3Skr+0P+8uY3vzl33XVXPvWpT+WTn/xkzjzzzNxzzz0566yzkiTl5eV58MEHc/vtt2f//v2ZOnVqzj///Fx33XX2IQEAAABGhdoVm4tdAgBwBEPazH358uUDK0L+2NatWw9pu/jii3PxxRcftv+JJ56Y++67byhlAAAAAAAAvCxDCkoAAEajofyl5641Fw5DJQAAAMBIKXiPEgAAAAAAgNHCihIAgJfBKhQAAAA4vllRAgAAAAAAlCxBCQAAAAAAULI8egsAAAAA4Ch49C6MTlaUAAAAAAAAJUtQAgAAAAAAlCyP3oLDGMoySgAAAAAAjj9WlAAAAAAAACVLUAIAAAAAAJQsQQkAAAAAAFCy7FECAAAAUAD7WgLA6GJFCQAAAAAAULIEJQAAAAAAQMkSlAAAAAAAACXLHiUAACNsKM8137XmwmGoBAAAGG6+/8MrnxUlAAAAAABAyRKUAAAAAAAAJcujtziuWKoIAAAAAMCxZEUJAAAAAABQsgQlAAAAAABAyfLoLQAAAKBkDeURzwDA6CIoAQAAAAB4BbFPL4wsQQmjnr8OAmA0KPR/z0ySAAAA4OjYowQAAAAAAChZghIAAAAAAKBkCUoAAAAAAICSZY8SAIBRyOaPAJQq+1QCAIUSlAAAAAAAHOf8sRQMnUdvAQAAAAAAJcuKEg5rJBJoy6EBAAAAACg2QQkAAEks1Qfglccf2AEAI0FQAgDAkAlXAADg+OX7PPyeoOQ445cXAAAAAAAcO0MKStavX5+bbropHR0dmTt3bm6++eYsWLDgiP2/+c1v5pprrsmuXbty5pln5oYbbshf/dVfDbzf39+f1atX59Zbb83+/fvzlre8JbfcckvOPPPMoZTHHxmppcqWRAMAwNE71vMqeKUzZwQYHQr9fe6PuDkeFByUbNq0Kc3NzdmwYUPq6+vT2tqaxsbGPPbYY5k8efIh/X/0ox/lkksuSUtLS/7zf/7Pueuuu7Jo0aLs2LEjZ511VpLkxhtvzBe+8IXcfvvtmT59eq655po0Njbm4YcfTmVl5cu/ylcoXxIBgFL0Sl0h+0qti9FpOOZVMJLMZwE4Wr5nczwY09/f31/ICfX19Tn77LPzxS9+MUnS19eXmpqafPSjH82KFSsO6b948eJ0d3fnn/7pnwba/uIv/iJ1dXXZsGFD+vv7M3Xq1PzP//k/87GPfSxJcuDAgVRVVeW2227LBz7wgUPG7OnpSU9Pz8DrAwcO5DWveU327NmTCRMmFHI5x8xZq+8ryucCAHCon13bWPA5Q/k+N5TPOVa6urpSU1OT/fv3Z+LEiUWrg6E51vOqw3klzpsYGeanAJSiYn4355WpkDlTQStKDh48mO3bt2flypUDbWVlZWloaEh7e/thz2lvb09zc/OgtsbGxtxzzz1JkieffDIdHR1paGgYeH/ixImpr69Pe3v7YYOSlpaWXHvttYe019TUFHI5AACMUhNbR9fnvJTnn39eUHKcGY551eGYNwEApeSV8N2cV6ajmTMVFJTs27cvvb29qaqqGtReVVWVRx999LDndHR0HLZ/R0fHwPsvth2pzx9buXLloElCX19ffv3rX+fVr351xowZU8glcQQvpm3+2mz4uMfDy/0dXu7v8HOPh5f7O/zc4+H14v19+OGHM3Xq1GKXQ4GGY151OOZNx57fbcPPPR4Z7vPIcJ+Hn3s8Mtzn4eceD4/+/v48//zzRzVnGtJm7sVWUVGRioqKQW2nnHJKcYoZ5SZMmOC/nMPMPR5e7u/wcn+Hn3s8vNzf4eceD69p06alrKys2GXwCmXeNHz8bht+7vHIcJ9Hhvs8/NzjkeE+Dz/3+Ng72tX3Bc2qJk2alPLy8nR2dg5q7+zsTHV19WHPqa6ufsn+L/5nIWMCAAAcr4ZjXgUAAAxdQUHJ2LFjM2/evLS1tQ209fX1pa2tLQsXLjzsOQsXLhzUP0m+//3vD/SfPn16qqurB/Xp6urKAw88cMQxAQAAjlfDMa8CAACGruBHbzU3N2fp0qWZP39+FixYkNbW1nR3d6epqSlJsmTJkkybNi0tLS1JkiuvvDLnnntuPve5z+XCCy/M3XffnZ/85Cf5yle+kiQZM2ZMrrrqqlx//fU588wzM3369FxzzTWZOnVqFi1adOyulIJUVFRk9erVhyzV59hxj4eX+zu83N/h5x4PL/d3+LnHw8v9Pf4d63kVI8N/94afezwy3OeR4T4PP/d4ZLjPw889Lr4x/f39/YWe9MUvfjE33XRTOjo6UldXly984Qupr69Pkpx33nmpra3NbbfdNtD/m9/8Zj71qU9l165dOfPMM3PjjTfmr/7qrwbe7+/vz+rVq/OVr3wl+/fvzznnnJMvfelLed3rXvfyrxAAAOAV6FjPqwAAgKEZUlACAAAAAAAwGhS0RwkAAAAAAMBoIigBAAAAAABKlqAEAAAAAAAoWYISAAAAAACgZAlKOCrvec978prXvCaVlZWZMmVKPvShD+WZZ54pdlmjwq5du3LppZdm+vTpOfHEEzNz5sysXr06Bw8eLHZpo8b/+l//K29+85szbty4nHLKKcUuZ1RYv359amtrU1lZmfr6+mzbtq3YJY0a//Iv/5J3v/vdmTp1asaMGZN77rmn2CWNKi0tLTn77LNz8sknZ/LkyVm0aFEee+yxYpc1atxyyy2ZM2dOJkyYkAkTJmThwoX5v//3/xa7rFFrzZo1GTNmTK666qpilwIlyRxp+JkrjQzzpeFhzjS8zJuGn7nTyDCHeuUQlHBU3v72t+cf/uEf8thjj+X//J//kyeeeCLvf//7i13WqPDoo4+mr68vX/7yl/PQQw/l85//fDZs2JBPfvKTxS5t1Dh48GAuvvjiXHbZZcUuZVTYtGlTmpubs3r16uzYsSNz585NY2Njnn322WKXNip0d3dn7ty5Wb9+fbFLGZV++MMf5oorrsiPf/zjfP/738/vfve7nH/++enu7i52aaPCn/3Zn2XNmjXZvn17fvKTn+Qv//Ivc9FFF+Whhx4qdmmjzr/927/ly1/+cubMmVPsUqBkmSMNP3OlkWG+dOyZMw0/86bhZ+40MsyhXjnG9Pf39xe7CI4///iP/5hFixalp6cnr3rVq4pdzqhz00035ZZbbskvfvGLYpcyqtx222256qqrsn///mKXclyrr6/P2WefnS9+8YtJkr6+vtTU1OSjH/1oVqxYUeTqRpcxY8bkO9/5ThYtWlTsUkatvXv3ZvLkyfnhD3+Yt73tbcUuZ1Q67bTTctNNN+XSSy8tdimjxgsvvJA///M/z5e+9KVcf/31qaurS2tra7HLgpJnjjQyzJWGj/nSsWPONLLMm0aGudPIMYcqDitKKNivf/3rfOMb38ib3/xmE4BhcuDAgZx22mnFLgMOcfDgwWzfvj0NDQ0DbWVlZWloaEh7e3sRK4OhOXDgQJL4nTsMent7c/fdd6e7uzsLFy4sdjmjyhVXXJELL7xw0O9ioLjMkUaOuRKvdOZMjFbmTsPPHKq4BCUctauvvjonnXRSXv3qV2f37t357ne/W+ySRqXHH388N998c/7H//gfxS4FDrFv37709vamqqpqUHtVVVU6OjqKVBUMTV9fX6666qq85S1vyVlnnVXsckaNn/70pxk/fnwqKiry13/91/nOd76T2bNnF7usUePuu+/Ojh070tLSUuxSgJgjjTRzJY4H5kyMRuZOw8sc6pVBUFLCVqxYkTFjxrzk8eijjw70//jHP55///d/z/e+972Ul5dnyZIl8eS2Iyv0/ibJ008/nQsuuCAXX3xxli1bVqTKjw9Dub8A/39XXHFFfvazn+Xuu+8udimjyutf//rs3LkzDzzwQC677LIsXbo0Dz/8cLHLGhX27NmTK6+8Mt/4xjdSWVlZ7HJgVDJHGhnmSsPPfAk4lsydhpc51CuDPUpK2N69e/Pcc8+9ZJ8ZM2Zk7Nixh7Q/9dRTqampyY9+9CNLwY6g0Pv7zDPP5Lzzzstf/MVf5LbbbktZmRzzpQzl59czd1++gwcPZty4cfnWt7416PmvS5cuzf79+/0V5THmWbvDZ/ny5fnud7+bf/mXf8n06dOLXc6o1tDQkJkzZ+bLX/5ysUs57t1zzz1573vfm/Ly8oG23t7ejBkzJmVlZenp6Rn0HlA4c6SRYa40/MyXisecaeSZNw0vc6eRZw5VHCcUuwCK5/TTT8/pp58+pHP7+vqSJD09PceypFGlkPv79NNP5+1vf3vmzZuXr33ta774H4WX8/PL0I0dOzbz5s1LW1vbwJfQvr6+tLW1Zfny5cUtDo5Cf39/PvrRj+Y73/lOtm7d6ov+COjr6/N94Rh5xzvekZ/+9KeD2pqamjJr1qxcffXVQhI4BsyRRoa50vAzXyoecyZGC3On4jGHKg5BCX/SAw88kH/7t3/LOeeck1NPPTVPPPFErrnmmsycOdNfSh0DTz/9dM4777y89rWvzdq1a7N3796B96qrq4tY2eixe/fu/PrXv87u3bvT29ubnTt3JknOOOOMjB8/vrjFHYeam5uzdOnSzJ8/PwsWLEhra2u6u7vT1NRU7NJGhRdeeCGPP/74wOsnn3wyO3fuzGmnnZbXvOY1RaxsdLjiiity11135bvf/W5OPvnkgedET5w4MSeeeGKRqzv+rVy5Mu9617vymte8Js8//3zuuuuubN26Nffdd1+xSxsVTj755EOeCf3i3gieFQ0jyxxpZJgrjQzzpWPPnGn4mTcNP3OnkWEO9QrSD3/Cgw8+2P/2t7+9/7TTTuuvqKjor62t7f/rv/7r/qeeeqrYpY0KX/va1/qTHPbg2Fi6dOlh7+8PfvCDYpd23Lr55pv7X/Oa1/SPHTu2f8GCBf0//vGPi13SqPGDH/zgsD+vS5cuLXZpo8KRft9+7WtfK3Zpo8J/+2//rf+1r31t/9ixY/tPP/30/ne84x393/ve94pd1qh27rnn9l955ZXFLgNKjjnSyDBXGhnmS8PDnGl4mTcNP3OnkWEO9cphjxIAAAAAAKBkebgnAAAAAABQsgQlAAAAAABAyRKUAAAAAAAAJUtQAgAAAAAAlCxBCQAAAAAAULIEJQAAAAAAQMkSlAAAAAAAACVLUAIAAAAAAJQsQQkAAAAAAFCyBCUAAAAAAEDJEpQAAAAAAAAl6/8DRVc9KqBiG2YAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 2000x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "x = torch.randn(1000, 10) \n",
    "w = torch.randn(10, 200) * 0.2\n",
    "y = x @ w\n",
    "print(x.mean(), x.std(), y.mean(), y.std())\n",
    "plt.figure(figsize=(20, 5))\n",
    "plt.subplot(121)\n",
    "plt.hist(x.view(-1).tolist(), bins=50, density=True);\n",
    "plt.subplot(122)\n",
    "plt.hist(y.view(-1).tolist(), bins=50, density=True);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "y now has std of 0.6493 and takes on less extreme values. \n",
    "\n",
    "Based on the discussion in section TanH saturation, we know that we want preactivation values to be in range of 1 and -1, so we want the standar deviation of y same as that of x. Based on mathematical calculations that number comes out be 1/(sqrt(fan_in)) where fan_in is the number of inputs to the layer. Let's test this theory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(0.0028) tensor(1.0039) tensor(0.0020) tensor(1.0034)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABkoAAAGsCAYAAACSD/sZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA510lEQVR4nO3df3DX9Z0n8GcShgRUUMuSCJtuQN3lXAvZgmTpamuvqaHLdctd66HTKZjp0DktTr3UWmgr1Opd8Md6OZXCLl0qte3Jda+1d8VLt5c5etNZKl1cpq1VpnrmQN1E8IZE42zikNwfHeNlAcs3/Ajk83jMfKZ8P9/3+53XZ/JtzHueeb/fZUNDQ0MBAAAAAAAooPKxLgAAAAAAAGCsCEoAAAAAAIDCEpQAAAAAAACFJSgBAAAAAAAKS1ACAAAAAAAUlqAEAAAAAAAoLEEJAAAAAABQWBPGuoCTYXBwMC+99FLOO++8lJWVjXU5AABwyg0NDeXVV1/NjBkzUl7u75/47cybAAAoklLmTOMiKHnppZdSW1s71mUAAMBpt3///vzu7/7uWJfBWcC8CQCAIjqeOdO4CErOO++8JL954ClTpoxxNQAAcOr19vamtrZ2+Hdh+G3MmwAAKJJS5kzjIih5c9n4lClT/MIPAECh2EKJ42XeBABAER3PnMlmxgAAAAAAQGEJSgAAAAAAgMISlAAAAAAAAIUlKAEAAAAAAApLUAIAAAAAABSWoAQAAAAAACgsQQkAAAAAAFBYghIAAAAAAKCwBCUAAAAAAEBhCUoAAAAAAIDCEpQAAAAAAACFJSgBAAAAAAAKS1ACAAAAAAAUlqAEAAAAAAAoLEEJAAAAAABQWIISAAAAAACgsCaMdQEAnFnqVm8vuU/n+iWnoBIAAIDxwTwL4MxmRQkAAAAAAFBYVpQAAAAAwHEazeoQAM5sVpQAAAAAAACFJSgBAAAAAAAKS1ACAAAAAAAUlqAEAAAAAAAoLEEJAAAAAABQWIISAAAAAACgsAQlAAAAAABAYU0Y6wIAOHXqVm8f6xIAAAAA4IwmKAEAAACAM8xo/vCtc/2SU1AJwPg3qq23NmzYkLq6ulRVVaWhoSG7du06rn6PPvpoysrKsnTp0hH3h4aGsnbt2lx00UWZNGlSGhsb8+tf/3o0pQEAAAAAABy3koOSbdu2paWlJevWrcuTTz6ZefPmpampKS+//PLb9uvs7Mytt96aq6666oj37rnnnjzwwAPZtGlTnnjiiZxzzjlpamrKP/7jP5ZaHgAAAAAAwHErOSi5//77s3LlyjQ3N+eyyy7Lpk2bMnny5GzZsuWYfQ4fPpyPf/zjueOOOzJ79uwR7w0NDaWtrS1f+tKX8pGPfCRz587NN77xjbz00kt57LHHSn4gAAAAAACA41VSUDIwMJDdu3ensbHxrQHKy9PY2JidO3ces99XvvKVTJ8+PZ/85CePeO/5559PV1fXiDGnTp2ahoaGY47Z39+f3t7eERcAAAAAAECpSgpKDh48mMOHD6e6unrE/erq6nR1dR21z09+8pP81V/9VTZv3nzU99/sV8qYra2tmTp16vBVW1tbymMAAAAAAAAkGeVh7sfr1VdfzSc+8Yls3rw506ZNO2njrlmzJj09PcPX/v37T9rYAAAAAABAcUwopfG0adNSUVGR7u7uEfe7u7tTU1NzRPvnnnsunZ2d+fCHPzx8b3Bw8DdfeMKE7N27d7hfd3d3LrroohFj1tfXH7WOysrKVFZWllI6AAAAAADAEUpaUTJx4sTMnz8/HR0dw/cGBwfT0dGRRYsWHdF+zpw5+cUvfpE9e/YMX3/2Z3+W97///dmzZ09qa2sza9as1NTUjBizt7c3TzzxxFHHBAAAAAAAOFlKWlGSJC0tLVmxYkUWLFiQhQsXpq2tLX19fWlubk6SLF++PDNnzkxra2uqqqpy+eWXj+h//vnnJ8mI+7fcckvuuuuuXHrppZk1a1Zuv/32zJgxI0uXLh39kwFw2tSt3l5yn871S05BJQAAAABQmpKDkmXLluXAgQNZu3Zturq6Ul9fn/b29uHD2Pft25fy8tKOPrntttvS19eXT33qUzl06FCuvPLKtLe3p6qqqtTyAAAAAAAAjlvZ0NDQ0FgXcaJ6e3szderU9PT0ZMqUKWNdDsAZYzQrPU4XK0oATozfgSmVzwzAyWGeBXB2KOX335JXlAAwNs7kX8YBAAAA4GxV2h5ZAAAAnBQbNmxIXV1dqqqq0tDQkF27dh1Xv0cffTRlZWXOdAQAgJNEUAIAAHCabdu2LS0tLVm3bl2efPLJzJs3L01NTXn55Zfftl9nZ2duvfXWXHXVVaepUgAAGP8EJQAAAKfZ/fffn5UrV6a5uTmXXXZZNm3alMmTJ2fLli3H7HP48OF8/OMfzx133JHZs2f/1q/R39+f3t7eERcAAHAkQQkAAMBpNDAwkN27d6exsXH4Xnl5eRobG7Nz585j9vvKV76S6dOn55Of/ORxfZ3W1tZMnTp1+KqtrT3h2gEAYDwSlAAAAJxGBw8ezOHDh1NdXT3ifnV1dbq6uo7a5yc/+Un+6q/+Kps3bz7ur7NmzZr09PQMX/v37z+hugEAYLyaMNYFAAAAcGyvvvpqPvGJT2Tz5s2ZNm3acferrKxMZWXlKawMAADGB0EJAADAaTRt2rRUVFSku7t7xP3u7u7U1NQc0f65555LZ2dnPvzhDw/fGxwcTJJMmDAhe/fuzcUXX3xqiwYYx+pWbx/rEgAYY7beAgAAOI0mTpyY+fPnp6OjY/je4OBgOjo6smjRoiPaz5kzJ7/4xS+yZ8+e4evP/uzP8v73vz979uxx9ggAAJwgK0oAAABOs5aWlqxYsSILFizIwoUL09bWlr6+vjQ3NydJli9fnpkzZ6a1tTVVVVW5/PLLR/Q///zzk+SI+wAAQOkEJQAAAKfZsmXLcuDAgaxduzZdXV2pr69Pe3v78AHv+/btS3m5DQAAAOB0EJQAAACMgVWrVmXVqlVHfW/Hjh1v2/fhhx8++QUBAEBB+RMlAAAAAACgsAQlAAAAAABAYQlKAAAAAACAwhKUAAAAAAAAhSUoAQAAAAAACktQAgAAAAAAFJagBAAAAAAAKCxBCQAAAAAAUFiCEgAAAAAAoLAmjHUBAEVVt3r7WJcAAAAAAIVnRQkAAAAAAFBYghIAAAAAAKCwBCUAAAAAAEBhCUoAAAAAAIDCEpQAAAAAAACFJSgBAAAAAAAKS1ACAAAAAAAUlqAEAAAAAAAoLEEJAAAAAABQWIISAAAAAACgsAQlAAAAAABAYY0qKNmwYUPq6upSVVWVhoaG7Nq165htv/vd72bBggU5//zzc84556S+vj6PPPLIiDY33HBDysrKRlyLFy8eTWkAAAAAAADHbUKpHbZt25aWlpZs2rQpDQ0NaWtrS1NTU/bu3Zvp06cf0f7CCy/MF7/4xcyZMycTJ07MD37wgzQ3N2f69Olpamoabrd48eJ8/etfH35dWVk5ykcCAAAAAAA4PiUHJffff39WrlyZ5ubmJMmmTZuyffv2bNmyJatXrz6i/dVXXz3i9Wc+85ls3bo1P/nJT0YEJZWVlampqSm1HIAzQt3q7WNdAgAAAAAwCiVtvTUwMJDdu3ensbHxrQHKy9PY2JidO3f+1v5DQ0Pp6OjI3r178973vnfEezt27Mj06dPzB3/wB7nxxhvzyiuvHHOc/v7+9Pb2jrgAAAAAAABKVdKKkoMHD+bw4cOprq4ecb+6ujrPPPPMMfv19PRk5syZ6e/vT0VFRb761a/mgx/84PD7ixcvzr/6V/8qs2bNynPPPZcvfOEL+dCHPpSdO3emoqLiiPFaW1tzxx13lFI6AAAAAIxro9ntoHP9klNQCcDZpeStt0bjvPPOy549e/Laa6+lo6MjLS0tmT179vC2XNddd91w23e9612ZO3duLr744uzYsSMf+MAHjhhvzZo1aWlpGX7d29ub2traU/4cAAAAAADA+FJSUDJt2rRUVFSku7t7xP3u7u63PV+kvLw8l1xySZKkvr4+Tz/9dFpbW484v+RNs2fPzrRp0/Lss88eNSiprKx02DsAAAAAAHDCSjqjZOLEiZk/f346OjqG7w0ODqajoyOLFi067nEGBwfT399/zPdfeOGFvPLKK7noootKKQ8AAAAAAKAkJW+91dLSkhUrVmTBggVZuHBh2tra0tfXl+bm5iTJ8uXLM3PmzLS2tib5zXkiCxYsyMUXX5z+/v48/vjjeeSRR7Jx48YkyWuvvZY77rgjH/3oR1NTU5Pnnnsut912Wy655JI0NTWdxEcFAAAAAAAYqeSgZNmyZTlw4EDWrl2brq6u1NfXp729ffiA93379qW8/K2FKn19fbnpppvywgsvZNKkSZkzZ06++c1vZtmyZUmSioqK/PznP8/WrVtz6NChzJgxI9dcc03uvPNO22sBAAAAAACnVNnQ0NDQWBdxonp7ezN16tT09PRkypQpY10OUEB1q7ePdQlnnc71S8a6BICzmt+BKZXPDMDRFX0+Z24GjFel/P5b0hklAAAAAAAA44mgBAAAAAAAKCxBCQAAAAAAUFglH+YOAAAAAGeiop83AsDoWFECAAAAAAAUlhUlAIyJUv/Sq3P9klNUCQAAAABFZkUJAAAAAABQWIISAAAAAACgsAQlAAAAAABAYQlKAAAAAACAwhKUAAAAAAAAhSUoAQAAAAAACktQAgAAAAAAFJagBAAAAAAAKCxBCQAAAAAAUFiCEgAAAAAAoLAEJQAAAAAAQGEJSgAAAAAAgMISlAAAAAAAAIU1YawLAIDjUbd6e8l9OtcvOQWVAAAAADCeWFECAAAAAAAUlqAEAAAAAAAoLEEJAAAAAABQWIISAAAAAACgsAQlAAAAAABAYQlKAAAAAACAwhKUAAAAAAAAhSUoAQAAAAAACktQAgAAAAAAFJagBAAAAAAAKCxBCQAAAAAAUFiCEgAAAAAAoLAEJQAAAAAAQGGNKijZsGFD6urqUlVVlYaGhuzateuYbb/73e9mwYIFOf/883POOeekvr4+jzzyyIg2Q0NDWbt2bS666KJMmjQpjY2N+fWvfz2a0gAAAAAAAI5byUHJtm3b0tLSknXr1uXJJ5/MvHnz0tTUlJdffvmo7S+88MJ88YtfzM6dO/Pzn/88zc3NaW5uzg9/+MPhNvfcc08eeOCBbNq0KU888UTOOeecNDU15R//8R9H/2QAAAAAAAC/RdnQ0NBQKR0aGhpyxRVX5KGHHkqSDA4Opra2NjfffHNWr159XGO8+93vzpIlS3LnnXdmaGgoM2bMyGc/+9nceuutSZKenp5UV1fn4YcfznXXXXdE//7+/vT39w+/7u3tTW1tbXp6ejJlypRSHgfgpKhbvX2sS+AoOtcvGesSAE6Z3t7eTJ061e/AHDefGaAIzM1OPfMs4GxRyu+/Ja0oGRgYyO7du9PY2PjWAOXlaWxszM6dO39r/6GhoXR0dGTv3r1573vfmyR5/vnn09XVNWLMqVOnpqGh4Zhjtra2ZurUqcNXbW1tKY8BAAAAAACQpMSg5ODBgzl8+HCqq6tH3K+urk5XV9cx+/X09OTcc8/NxIkTs2TJkjz44IP54Ac/mCTD/UoZc82aNenp6Rm+9u/fX8pjAAAAAAAAJEkmnI4vct5552XPnj157bXX0tHRkZaWlsyePTtXX331qMarrKxMZWXlyS0SAAAAAAAonJKCkmnTpqWioiLd3d0j7nd3d6empuaY/crLy3PJJZckSerr6/P000+ntbU1V1999XC/7u7uXHTRRSPGrK+vL6U8AAAAAACAkpS09dbEiRMzf/78dHR0DN8bHBxMR0dHFi1adNzjDA4ODh/GPmvWrNTU1IwYs7e3N0888URJYwIAAAAAAJSq5K23WlpasmLFiixYsCALFy5MW1tb+vr60tzcnCRZvnx5Zs6cmdbW1iS/OXh9wYIFufjii9Pf35/HH388jzzySDZu3JgkKSsryy233JK77rorl156aWbNmpXbb789M2bMyNKlS0/ekwIcp7rV28e6BACgADZs2JB77703XV1dmTdvXh588MEsXLjwqG2/+93v5t//+3+fZ599Nm+88UYuvfTSfPazn80nPvGJ01w1AACMPyUHJcuWLcuBAweydu3adHV1pb6+Pu3t7cOHse/bty/l5W8tVOnr68tNN92UF154IZMmTcqcOXPyzW9+M8uWLRtuc9ttt6Wvry+f+tSncujQoVx55ZVpb29PVVXVSXhEAACAM8u2bdvS0tKSTZs2paGhIW1tbWlqasrevXszffr0I9pfeOGF+eIXv5g5c+Zk4sSJ+cEPfpDm5uZMnz49TU1NY/AEAAAwfpQNDQ0NjXURJ6q3tzdTp05NT09PpkyZMtblAGc5K0rGj871S8a6BIBTxu/AZ7eGhoZcccUVeeihh5L8Znvi2tra3HzzzVm9evVxjfHud787S5YsyZ133nlc7X1mgCIwnzv1zLOAs0Upv/+WdEYJAAAAJ2ZgYCC7d+9OY2Pj8L3y8vI0NjZm586dv7X/0NBQOjo6snfv3rz3ve89Zrv+/v709vaOuAAAgCMJSgAAAE6jgwcP5vDhw8PbF7+puro6XV1dx+zX09OTc889NxMnTsySJUvy4IMP5oMf/OAx27e2tmbq1KnDV21t7Ul7BgAAGE8EJQAAAGeB8847L3v27MnPfvaz/Lt/9+/S0tKSHTt2HLP9mjVr0tPTM3zt37//9BULAABnkZIPcwcAAGD0pk2bloqKinR3d4+4393dnZqammP2Ky8vzyWXXJIkqa+vz9NPP53W1tZcffXVR21fWVmZysrKk1Y3AACMV1aUAAAAnEYTJ07M/Pnz09HRMXxvcHAwHR0dWbRo0XGPMzg4mP7+/lNRIgAAFIoVJQAAAKdZS0tLVqxYkQULFmThwoVpa2tLX19fmpubkyTLly/PzJkz09ramuQ3540sWLAgF198cfr7+/P444/nkUceycaNG8fyMQAAYFwQlAAAAJxmy5Yty4EDB7J27dp0dXWlvr4+7e3twwe879u3L+Xlb20A0NfXl5tuuikvvPBCJk2alDlz5uSb3/xmli1bNlaPAAAA44agBAAAYAysWrUqq1atOup7//SQ9rvuuit33XXXaagKAACKxxklAAAAAABAYQlKAAAAAACAwhKUAAAAAAAAhSUoAQAAAAAACktQAgAAAAAAFNaEsS4AAAAAAP6putXbx7oEAArCihIAAAAAAKCwBCUAAAAAAEBhCUoAAAAAAIDCEpQAAAAAAACFJSgBAAAAAAAKS1ACAAAAAAAUlqAEAAAAAAAoLEEJAAAAAABQWBPGugCAU6lu9faxLgEAAAAAOINZUQIAAAAAABSWoAQAAAAAACgsQQkAAAAAAFBYghIAAAAAAKCwBCUAAAAAAEBhCUoAAAAAAIDCEpQAAAAAAACFJSgBAAAAAAAKS1ACAAAAAAAU1qiCkg0bNqSuri5VVVVpaGjIrl27jtl28+bNueqqq3LBBRfkggsuSGNj4xHtb7jhhpSVlY24Fi9ePJrSAAAAAAAAjtuEUjts27YtLS0t2bRpUxoaGtLW1pampqbs3bs306dPP6L9jh07cv311+c973lPqqqqcvfdd+eaa67JU089lZkzZw63W7x4cb7+9a8Pv66srBzlIwHAb9St3l5yn871S05BJQAAAACcqUpeUXL//fdn5cqVaW5uzmWXXZZNmzZl8uTJ2bJly1Hbf+tb38pNN92U+vr6zJkzJ1/72tcyODiYjo6OEe0qKytTU1MzfF1wwQWjeyIAAAAAAIDjVFJQMjAwkN27d6exsfGtAcrL09jYmJ07dx7XGK+//nreeOONXHjhhSPu79ixI9OnT88f/MEf5MYbb8wrr7xyzDH6+/vT29s74gIAAAAAAChVSUHJwYMHc/jw4VRXV4+4X11dna6uruMa4/Of/3xmzJgxImxZvHhxvvGNb6SjoyN33313fvzjH+dDH/pQDh8+fNQxWltbM3Xq1OGrtra2lMcAAAAAAABIMoozSk7E+vXr8+ijj2bHjh2pqqoavn/dddcN//td73pX5s6dm4svvjg7duzIBz7wgSPGWbNmTVpaWoZf9/b2CksAAAAAAICSlbSiZNq0aamoqEh3d/eI+93d3ampqXnbvvfdd1/Wr1+fv/mbv8ncuXPftu3s2bMzbdq0PPvss0d9v7KyMlOmTBlxAQAAAAAAlKqkoGTixImZP3/+iIPY3zyYfdGiRcfsd8899+TOO+9Me3t7FixY8Fu/zgsvvJBXXnklF110USnlAQAAAAAAlKSkoCRJWlpasnnz5mzdujVPP/10brzxxvT19aW5uTlJsnz58qxZs2a4/d13353bb789W7ZsSV1dXbq6utLV1ZXXXnstSfLaa6/lc5/7XH7605+ms7MzHR0d+chHPpJLLrkkTU1NJ+kxAQAAAAAAjlTyGSXLli3LgQMHsnbt2nR1daW+vj7t7e3DB7zv27cv5eVv5S8bN27MwMBAPvaxj40YZ926dfnyl7+cioqK/PznP8/WrVtz6NChzJgxI9dcc03uvPPOVFZWnuDjAQAAAAAAHNuoDnNftWpVVq1addT3duzYMeJ1Z2fn2441adKk/PCHPxxNGQAAAAAAACek5K23AAAAAAAAxgtBCQAAAAAAUFiCEgAAAAAAoLBGdUYJAAAAAFA8dau3l9ync/2SU1AJwMljRQkAAAAAAFBYghIAAAAAAKCwBCUAAAAAAEBhCUoAAAAAAIDCEpQAAAAAAACFJSgBAAAAAAAKS1ACAAAAAAAUlqAEAAAAAAAoLEEJAAAAAABQWIISAAAAAACgsAQlAAAAAABAYQlKAAAAAACAwhKUAAAAAAAAhTVhrAsAAAAAYHyrW719rEsAgGOyogQAAAAAACgsK0qAs4a/QAIAAAAATjYrSgAAAAAAgMISlAAAAAAAAIUlKAEAAAAAAApLUAIAAAAAABSWoAQAAAAAACgsQQkAAAAAAFBYghIAAAAAAKCwBCUAAAAAAEBhCUoAAAAAAIDCEpQAAAAAAACFJSgBAAAAAAAKS1ACAAAAAAAUlqAEAAAAAAAorFEFJRs2bEhdXV2qqqrS0NCQXbt2HbPt5s2bc9VVV+WCCy7IBRdckMbGxiPaDw0NZe3atbnooosyadKkNDY25te//vVoSgMAAAAAADhuE0rtsG3btrS0tGTTpk1paGhIW1tbmpqasnfv3kyfPv2I9jt27Mj111+f97znPamqqsrdd9+da665Jk899VRmzpyZJLnnnnvywAMPZOvWrZk1a1Zuv/32NDU15Ve/+lWqqqpO/CkB4DjVrd5ecp/O9UtOQSUAAAAAnA4lryi5//77s3LlyjQ3N+eyyy7Lpk2bMnny5GzZsuWo7b/1rW/lpptuSn19febMmZOvfe1rGRwcTEdHR5LfrCZpa2vLl770pXzkIx/J3Llz841vfCMvvfRSHnvssRN6OAAAAAAAgLdTUlAyMDCQ3bt3p7Gx8a0BysvT2NiYnTt3HtcYr7/+et54441ceOGFSZLnn38+XV1dI8acOnVqGhoajjlmf39/ent7R1wAAAAAAAClKikoOXjwYA4fPpzq6uoR96urq9PV1XVcY3z+85/PjBkzhoORN/uVMmZra2umTp06fNXW1pbyGAAAAAAAAElGeZj7aK1fvz6PPvpovve9753Q2SNr1qxJT0/P8LV///6TWCUAAAAAAFAUJQUl06ZNS0VFRbq7u0fc7+7uTk1Nzdv2ve+++7J+/fr8zd/8TebOnTt8/81+pYxZWVmZKVOmjLgAAADOJhs2bEhdXV2qqqrS0NCQXbt2HbPt5s2bc9VVV+WCCy7IBRdckMbGxrdtDwAAHL+SgpKJEydm/vz5wwexJxk+mH3RokXH7HfPPffkzjvvTHt7exYsWDDivVmzZqWmpmbEmL29vXniiSfedkwAAICz1bZt29LS0pJ169blySefzLx589LU1JSXX375qO137NiR66+/Pv/zf/7P7Ny5M7W1tbnmmmvy4osvnubKAQBg/Cl5662WlpZs3rw5W7duzdNPP50bb7wxfX19aW5uTpIsX748a9asGW5/99135/bbb8+WLVtSV1eXrq6udHV15bXXXkuSlJWV5ZZbbsldd92V//pf/2t+8YtfZPny5ZkxY0aWLl16cp4SAADgDHL//fdn5cqVaW5uzmWXXZZNmzZl8uTJ2bJly1Hbf+tb38pNN92U+vr6zJkzJ1/72teG/2gNAAA4MRNK7bBs2bIcOHAga9euTVdXV+rr69Pe3j58GPu+fftSXv5W/rJx48YMDAzkYx/72Ihx1q1bly9/+ctJkttuuy19fX351Kc+lUOHDuXKK69Me3v7CZ1jAgAAcCYaGBjI7t27R/yBWXl5eRobG7Nz587jGuP111/PG2+8kQsvvPCYbfr7+9Pf3z/8ure3d/RFAwDAOFZyUJIkq1atyqpVq4763o4dO0a87uzs/K3jlZWV5Stf+Uq+8pWvjKYcAACAs8bBgwdz+PDh4T82e1N1dXWeeeaZ4xrj85//fGbMmJHGxsZjtmltbc0dd9xxQrUCAEARlLz1FgAAAGNn/fr1efTRR/O9733vbVfhr1mzJj09PcPX/v37T2OVAABw9hjVihIAAABGZ9q0aamoqEh3d/eI+93d3ampqXnbvvfdd1/Wr1+f//E//kfmzp37tm0rKytTWVl5wvUCAMB4Z0UJAADAaTRx4sTMnz9/xEHsbx7MvmjRomP2u+eee3LnnXemvb09CxYsOB2lAgBAIVhRAgAAcJq1tLRkxYoVWbBgQRYuXJi2trb09fWlubk5SbJ8+fLMnDkzra2tSZK77747a9euzbe//e3U1dWlq6srSXLuuefm3HPPHbPnAACA8UBQAgAAcJotW7YsBw4cyNq1a9PV1ZX6+vq0t7cPH/C+b9++lJe/tQHAxo0bMzAwkI997GMjxlm3bl2+/OUvn87SAQBg3BGUAAAAjIFVq1Zl1apVR31vx44dI153dnae+oIA4BSpW7295D6d65ecgkoAjs4ZJQAAAAAAQGEJSgAAAAAAgMISlAAAAAAAAIUlKAEAAAAAAApLUAIAAAAAABTWhLEuACimutXbx7oEAAAAAAArSgAAAAAAgOISlAAAAAAAAIUlKAEAAAAAAApLUAIAAAAAABSWoAQAAAAAACgsQQkAAAAAAFBYE8a6AAA429Wt3l5yn871S05BJQAAAACUyooSAAAAAACgsKwoAQAAAOC4jWZFNQCcyawoAQAAAAAACktQAgAAAAAAFJagBAAAAAAAKCxBCQAAAAAAUFiCEgAAAAAAoLAEJQAAAAAAQGEJSgAAAAAAgMISlAAAAAAAAIUlKAEAAAAAAApLUAIAAAAAABTWhLEuAACKqG719pLad65fcooqAQAAACg2K0oAAAAAAIDCGlVQsmHDhtTV1aWqqioNDQ3ZtWvXMds+9dRT+ehHP5q6urqUlZWlra3tiDZf/vKXU1ZWNuKaM2fOaEoDAAAAAAA4biUHJdu2bUtLS0vWrVuXJ598MvPmzUtTU1Nefvnlo7Z//fXXM3v27Kxfvz41NTXHHPcP//AP8w//8A/D109+8pNSSwMAAAAAAChJyUHJ/fffn5UrV6a5uTmXXXZZNm3alMmTJ2fLli1HbX/FFVfk3nvvzXXXXZfKyspjjjthwoTU1NQMX9OmTSu1NAAAAAAAgJKUdJj7wMBAdu/enTVr1gzfKy8vT2NjY3bu3HlChfz617/OjBkzUlVVlUWLFqW1tTXvfOc7j9q2v78//f39w697e3tP6GsDJ67Ug6kBAAAAAM4EJa0oOXjwYA4fPpzq6uoR96urq9PV1TXqIhoaGvLwww+nvb09GzduzPPPP5+rrroqr7766lHbt7a2ZurUqcNXbW3tqL82AAAAAABQXKM6zP1k+9CHPpRrr702c+fOTVNTUx5//PEcOnQo//k//+ejtl+zZk16enqGr/3795/migEAAAAAgPGgpK23pk2bloqKinR3d4+4393d/bYHtZfq/PPPz+///u/n2WefPer7lZWVb3veCQAAAAAAwPEoaUXJxIkTM3/+/HR0dAzfGxwcTEdHRxYtWnTSinrttdfy3HPP5aKLLjppYwIAAAAAAPxTJa0oSZKWlpasWLEiCxYsyMKFC9PW1pa+vr40NzcnSZYvX56ZM2emtbU1yW8OgP/Vr341/O8XX3wxe/bsybnnnptLLrkkSXLrrbfmwx/+cH7v934vL730UtatW5eKiopcf/31J+s5AQAAAAAAjlByULJs2bIcOHAga9euTVdXV+rr69Pe3j58wPu+fftSXv7WQpWXXnopf/RHfzT8+r777st9992X973vfdmxY0eS5IUXXsj111+fV155Jb/zO7+TK6+8Mj/96U/zO7/zOyf4eAAAAAAAAMdWclCSJKtWrcqqVauO+t6b4ceb6urqMjQ09LbjPfroo6MpAwAAAAAA4ISUdEYJAAAAAADAeCIoAQAAAAAACktQAgAAAAAAFNaozigBAAAAADhV6lZvL7lP5/olp6ASoAisKAEAAAAAAApLUAIAAAAAABSWoAQAAAAAACgsQQkAAAAAAFBYghIAAAAAAKCwBCUAAAAAAEBhCUoAAAAAAIDCEpQAAAAAAACFNWGsCwAAAABg7NSt3j7WJQDAmLKiBAAAAAAAKCxBCQAAAAAAUFiCEgAAAAAAoLAEJQAAAAAAQGEJSgAAAAAAgMISlAAAAAAAAIUlKAEAAAAAAApLUAIAAAAAABSWoAQAAAAAACgsQQkAAAAAAFBYghIAAAAAAKCwBCUAAAAAAEBhTRjrAgCA365u9faS+3SuX3IKKgEAAAAYX6woAQAAAAAACktQAgAAAAAAFJagBAAAAAAAKCxBCQAAAAAAUFiCEgAAAAAAoLAEJQAAAAAAQGEJSgAAAAAAgMIaVVCyYcOG1NXVpaqqKg0NDdm1a9cx2z711FP56Ec/mrq6upSVlaWtre2ExwQAAAAAADgZJpTaYdu2bWlpacmmTZvS0NCQtra2NDU1Ze/evZk+ffoR7V9//fXMnj071157bf7tv/23J2VM4NSqW719rEsAAAAAADgtyoaGhoZK6dDQ0JArrrgiDz30UJJkcHAwtbW1ufnmm7N69eq37VtXV5dbbrklt9xyy0kbM0l6e3szderU9PT0ZMqUKaU8DnAUghIYHzrXLxnrEoBTyO/AlMpnBjgWc0CKzLwJxq9Sfv8taeutgYGB7N69O42NjW8NUF6exsbG7Ny5c1TFjmbM/v7+9Pb2jrgAAADOJqdiS2MAAKB0JQUlBw8ezOHDh1NdXT3ifnV1dbq6ukZVwGjGbG1tzdSpU4ev2traUX1tAACAsfDm9sPr1q3Lk08+mXnz5qWpqSkvv/zyUdu/uaXx+vXrU1NTc5qrBQCA8W1Uh7mPtTVr1qSnp2f42r9//1iXBAAAcNzuv//+rFy5Ms3NzbnsssuyadOmTJ48OVu2bDlq+yuuuCL33ntvrrvuulRWVp7magEAYHwr6TD3adOmpaKiIt3d3SPud3d3j/qvmkYzZmVlpckBAABwVnpz++E1a9YM3zvRLY2Ppr+/P/39/cOvbVkMAABHV9KKkokTJ2b+/Pnp6OgYvjc4OJiOjo4sWrRoVAWcijEBAADOVKdiS+OjsWUxAAAcn5K33mppacnmzZuzdevWPP3007nxxhvT19eX5ubmJMny5ctH/GXUwMBA9uzZkz179mRgYCAvvvhi9uzZk2efffa4xwQAAKA0tiwGAIDjU9LWW0mybNmyHDhwIGvXrk1XV1fq6+vT3t4+/NdQ+/btS3n5W/nLSy+9lD/6oz8afn3fffflvvvuy/ve977s2LHjuMYEAAAYL07FlsZHY8tiAAA4PiUHJUmyatWqrFq16qjvvRl+vKmuri5DQ0MnNCYAAMB48f9vP7x06dIkb20/bE4EAACn36iCEgAAAEavpaUlK1asyIIFC7Jw4cK0tbUdsaXxzJkz09ramuQ3Wxr/6le/Gv73m1san3vuubnkkkvG7DkAAGA8EJQAAACcZqdiS2OAJKlbvX2sSwCAs46gBAAAYAycii2NAQCA0pX/9iYAAAAAAADjk6AEAAAAAAAoLEEJAAAAAABQWIISAAAAAACgsAQlAAAAAABAYU0Y6wKAU6tu9faxLgEYI6P5/3/n+iWnoBIAAACAM5cVJQAAAAAAQGEJSgAAAAAAgMISlAAAAAAAAIUlKAEAAAAAAApLUAIAAAAAABSWoAQAAAAAACgsQQkAAAAAAFBYghIAAAAAAKCwBCUAAAAAAEBhTRjrAgAAAAAAxkLd6u0lte9cv+QUVQKMJStKAAAAAACAwhKUAAAAAAAAhSUoAQAAAAAACktQAgAAAAAAFJbD3AEAAADOQKUeMg0AjI4VJQAAAAAAQGEJSgAAAAAAgMISlAAAAAAAAIXljBIAYNho9sHuXL/kFFQCAAAAcHoISuAs4iA/AAAAAICTy9ZbAAAAAABAYQlKAAAAAACAwhKUAAAAAAAAhSUoAQAAAAAACmtUQcmGDRtSV1eXqqqqNDQ0ZNeuXW/b/jvf+U7mzJmTqqqqvOtd78rjjz8+4v0bbrghZWVlI67FixePpjQAAAAAAIDjVnJQsm3btrS0tGTdunV58sknM2/evDQ1NeXll18+avu//du/zfXXX59PfvKT+fu///ssXbo0S5cuzS9/+csR7RYvXpx/+Id/GL7+03/6T6N7IgAAAAAAgONUclBy//33Z+XKlWlubs5ll12WTZs2ZfLkydmyZctR2//H//gfs3jx4nzuc5/LP/tn/yx33nln3v3ud+ehhx4a0a6ysjI1NTXD1wUXXHDMGvr7+9Pb2zviAgAAAAAAKNWEUhoPDAxk9+7dWbNmzfC98vLyNDY2ZufOnUfts3PnzrS0tIy419TUlMcee2zEvR07dmT69Om54IIL8s//+T/PXXfdlXe84x1HHbO1tTV33HFHKaUDAAAAAJyQutXbS+7TuX7JKagEOJlKWlFy8ODBHD58ONXV1SPuV1dXp6ur66h9urq6fmv7xYsX5xvf+EY6Ojpy991358c//nE+9KEP5fDhw0cdc82aNenp6Rm+9u/fX8pjAAAAAAAAJClxRcmpct111w3/+13velfmzp2biy++ODt27MgHPvCBI9pXVlamsrLydJYIAAAAMGqj+St0AOD0KGlFybRp01JRUZHu7u4R97u7u1NTU3PUPjU1NSW1T5LZs2dn2rRpefbZZ0spDwAAAAAAoCQlBSUTJ07M/Pnz09HRMXxvcHAwHR0dWbRo0VH7LFq0aET7JPnRj350zPZJ8sILL+SVV17JRRddVEp5AAAAAAAAJSkpKEmSlpaWbN68OVu3bs3TTz+dG2+8MX19fWlubk6SLF++fMRh75/5zGfS3t6eP//zP88zzzyTL3/5y/m7v/u7rFq1Kkny2muv5XOf+1x++tOfprOzMx0dHfnIRz6SSy65JE1NTSfpMQEAAAAAAI5U8hkly5Yty4EDB7J27dp0dXWlvr4+7e3twwe279u3L+Xlb+Uv73nPe/Ltb387X/rSl/KFL3whl156aR577LFcfvnlSZKKior8/Oc/z9atW3Po0KHMmDEj11xzTe68807nkAAAAAAAAKfUqA5zX7Vq1fCKkH9qx44dR9y79tprc+211x61/aRJk/LDH/5wNGUAAAAAAACckJK33gIAAAAAABgvBCUAAAAAAEBhCUoAAAAAAIDCEpQAAAAAAACFNarD3AEAAAAA+O3qVm8vuU/n+iWnoBLgWAQlMEZG8x9JgDORX/oBAACAs5mttwAAAAAAgMISlAAAAAAAAIUlKAEAAAAAAArLGSUAAAAAJXDmJACML1aUAAAAAAAAhSUoAQAAAAAACsvWWwDAGW8021t0rl9yCioBAAAAxhsrSgAAAAAAgMKyogROEof5ARw/PzMBAACAM4UVJQAAAAAAQGFZUQIAAAAAcAZxTiOcXoISAAAAoLBsCQoACEoY9yTwAAAAAAAcizNKAAAAAACAwrKiBI7C0msAAAAAgGIQlAAAAAAAnOVsPw+jZ+stAAAAAACgsAQlAAAAAABAYdl6CwAYlyw7B4BicuYkAFAqK0oAAAAAAIDCEpQAAAAAAACFZestAAAA4IxkGy2AU8uWxfAbVpQAAAAAAACFJSgBAAAAAAAKy9ZbAAAnwFJ1AACgSEqdA5n/cDYQlDBm7DULwJnGf5sA4NTx31kA4Exl6y0AAAAAAKCwRrWiZMOGDbn33nvT1dWVefPm5cEHH8zChQuP2f473/lObr/99nR2dubSSy/N3XffnT/90z8dfn9oaCjr1q3L5s2bc+jQofzJn/xJNm7cmEsvvXQ05QEAAJzxTva8Ck4nq0MAOF62K+ZsUHJQsm3btrS0tGTTpk1paGhIW1tbmpqasnfv3kyfPv2I9n/7t3+b66+/Pq2trfkX/+Jf5Nvf/naWLl2aJ598MpdffnmS5J577skDDzyQrVu3ZtasWbn99tvT1NSUX/3qV6mqqjrxpwQAADiDnIp5FYyW0AOAM41whdOtbGhoaKiUDg0NDbniiivy0EMPJUkGBwdTW1ubm2++OatXrz6i/bJly9LX15cf/OAHw/f++I//OPX19dm0aVOGhoYyY8aMfPazn82tt96aJOnp6Ul1dXUefvjhXHfddUeM2d/fn/7+/uHXPT09eec735n9+/dnypQppTwOJ9Hl63441iUAACfgl3c0jXUJlKC3tze1tbU5dOhQpk6dOtblUKKTPa86GvOm4jI3A4BTx7zp7FHKnKmkFSUDAwPZvXt31qxZM3yvvLw8jY2N2blz51H77Ny5My0tLSPuNTU15bHHHkuSPP/88+nq6kpjY+Pw+1OnTk1DQ0N27tx51KCktbU1d9xxxxH3a2trS3kcAAD+P1PbxroCRuPVV18VlJxlTsW86mjMmwAATj7zprPP8cyZSgpKDh48mMOHD6e6unrE/erq6jzzzDNH7dPV1XXU9l1dXcPvv3nvWG3+qTVr1oyYJAwODub//t//m3e84x0pKysr5ZHGzJtplr/mYrR8hjgRPj+cCJ8fToTPz8kzNDSUV199NTNmzBjrUijRqZhXHc14mDfh52ZR+D4Xg+9zMfg+F4Pv89mhlDnTqA5zH2uVlZWprKwcce/8888fm2JO0JQpU/yfiRPiM8SJ8PnhRPj8cCJ8fk4OK0l4O+Np3oSfm0Xh+1wMvs/F4PtcDL7PZ77jnTOVlzLotGnTUlFRke7u7hH3u7u7U1NTc9Q+NTU1b9v+zf8tZUwAAICz1amYVwEAAKNXUlAyceLEzJ8/Px0dHcP3BgcH09HRkUWLFh21z6JFi0a0T5If/ehHw+1nzZqVmpqaEW16e3vzxBNPHHNMAACAs9WpmFcBAACjV/LWWy0tLVmxYkUWLFiQhQsXpq2tLX19fWlubk6SLF++PDNnzkxra2uS5DOf+Uze97735c///M+zZMmSPProo/m7v/u7/OVf/mWSpKysLLfcckvuuuuuXHrppZk1a1Zuv/32zJgxI0uXLj15T3qGqayszLp1645YCg/Hy2eIE+Hzw4nw+eFE+PzAb5zseRXjl5+bxeD7XAy+z8Xg+1wMvs/jT9nQ0NBQqZ0eeuih3Hvvvenq6kp9fX0eeOCBNDQ0JEmuvvrq1NXV5eGHHx5u/53vfCdf+tKX0tnZmUsvvTT33HNP/vRP/3T4/aGhoaxbty5/+Zd/mUOHDuXKK6/MV7/61fz+7//+iT8hAADAGehkz6sAAIDRGVVQAgAAAAAAMB6UdEYJAAAAAADAeCIoAQAAAAAACktQAgAAAAAAFJagBAAAAAAAKCxByRmmv78/9fX1KSsry549e8a6HM4CnZ2d+eQnP5lZs2Zl0qRJufjii7Nu3boMDAyMdWmcoTZs2JC6urpUVVWloaEhu3btGuuSOAu0trbmiiuuyHnnnZfp06dn6dKl2bt371iXxVlq/fr1KSsryy233DLWpQCctcwdxyfzu/HNXGx8M2cqHvOa8UVQcoa57bbbMmPGjLEug7PIM888k8HBwfzFX/xFnnrqqfyH//AfsmnTpnzhC18Y69I4A23bti0tLS1Zt25dnnzyycybNy9NTU15+eWXx7o0znA//vGP8+lPfzo//elP86Mf/ShvvPFGrrnmmvT19Y11aZxlfvazn+Uv/uIvMnfu3LEuBeCsZu44PpnfjV/mYuOfOVOxmNeMP2VDQ0NDY10Ev/Hf//t/T0tLS/7Lf/kv+cM//MP8/d//ferr68e6LM5C9957bzZu3Jj//b//91iXwhmmoaEhV1xxRR566KEkyeDgYGpra3PzzTdn9erVY1wdZ5MDBw5k+vTp+fGPf5z3vve9Y10OZ4nXXnst7373u/PVr341d911V+rr69PW1jbWZQGcdcwdi8X8bnwwFysec6bxy7xmfLKi5AzR3d2dlStX5pFHHsnkyZPHuhzOcj09PbnwwgvHugzOMAMDA9m9e3caGxuH75WXl6exsTE7d+4cw8o4G/X09CSJnzWU5NOf/nSWLFky4ucQAKUxdywe87uzn7lYMZkzjV/mNePThLEugGRoaCg33HBD/s2/+TdZsGBBOjs7x7okzmLPPvtsHnzwwdx3331jXQpnmIMHD+bw4cOprq4ecb+6ujrPPPPMGFXF2WhwcDC33HJL/uRP/iSXX375WJfDWeLRRx/Nk08+mZ/97GdjXQrAWcvcsXjM78YHc7HiMWcav8xrxi8rSk6h1atXp6ys7G2vZ555Jg8++GBeffXVrFmzZqxL5gxyvJ+f/9+LL76YxYsX59prr83KlSvHqHJgvPv0pz+dX/7yl3n00UfHuhTOEvv3789nPvOZfOtb30pVVdVYlwNwxjF3HP/M76BYzJnGJ/Oa8c0ZJafQgQMH8sorr7xtm9mzZ+df/+t/nf/23/5bysrKhu8fPnw4FRUV+fjHP56tW7ee6lI5Ax3v52fixIlJkpdeeilXX311/viP/zgPP/xwysvloIw0MDCQyZMn56//+q+zdOnS4fsrVqzIoUOH8v3vf3/siuOssWrVqnz/+9/P//pf/yuzZs0a63I4Szz22GP5l//yX6aiomL43uHDh1NWVpby8vL09/ePeA+gaMwdxz/zu2IzFysWc6bxy7xmfBOUnAH27duX3t7e4dcvvfRSmpqa8td//ddpaGjI7/7u745hdZwNXnzxxbz//e/P/Pnz881vftMPZY6poaEhCxcuzIMPPpjkN8uB3/nOd2bVqlUOEORtDQ0N5eabb873vve97NixI5deeulYl8RZ5NVXX83/+T//Z8S95ubmzJkzJ5///OdtRwBwnMwdi8H8bnwyFxv/zJnGP/Oa8c0ZJWeAd77znSNen3vuuUmSiy++2C+6/FYvvvhirr766vze7/1e7rvvvhw4cGD4vZqamjGsjDNRS0tLVqxYkQULFmThwoVpa2tLX19fmpubx7o0znCf/vSn8+1vfzvf//73c95556WrqytJMnXq1EyaNGmMq+NMd9555x0xaTjnnHPyjne8w2QCoATmjuOf+d34ZS42/pkzjX/mNeOboATOcj/60Y/y7LPP5tlnnz1icmTBGP/UsmXLcuDAgaxduzZdXV2pr69Pe3v7EYcKwj+1cePGJMnVV1894v7Xv/713HDDDae/IACAccj8bvwyFxv/zJng7GbrLQAAAAAAoLCcBgYAAAAAABSWoAQAAAAAACgsQQkAAAAAAFBYghIAAAAAAKCwBCUAAAAAAEBhCUoAAAAAAIDCEpQAAAAAAACFJSgBAAAAAAAKS1ACAAAAAAAUlqAEAAAAAAAoLEEJAAAAAABQWP8PIlRyBH1cW4QAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 2000x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = torch.randn(1000, 10) \n",
    "w = torch.randn(10, 200) * 1/(10 ** 0.5)\n",
    "y = x @ w\n",
    "print(x.mean(), x.std(), y.mean(), y.std())\n",
    "plt.figure(figsize=(20, 5))\n",
    "plt.subplot(121)\n",
    "plt.hist(x.view(-1).tolist(), bins=50, density=True);\n",
    "plt.subplot(122)\n",
    "plt.hist(y.view(-1).tolist(), bins=50, density=True);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The std of y is now roughly 1 and this is what we want while training our neural network. One paper that has delved deep into this topic and is often cited is [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://arxiv.org/abs/1502.01852)\n",
    "\n",
    "The paper discusses Relu and leaky Relu instead of tanh but the conclusion applied to both since both of them are squashing functions\n",
    "![https://pub.mdpi-res.com/sensors/sensors-23-04072/article_deploy/html/images/sensors-23-04072-g001.png?1681810532](https://pub.mdpi-res.com/sensors/sensors-23-04072/article_deploy/html/images/sensors-23-04072-g001.png?1681810532)\n",
    "\n",
    "The paper concludes that since Relu clamps the input which is less than 0 and only allows input > 0 to \"pass through\", we need a gain factor of 2/sqrt(fan_in) to compensate for that. They also discuss that by adding this gain factor the backpropagation is also taken care of and we don't need to perform any additional scaling to control gradients.\n",
    "\n",
    "## Kaiming Initialization\n",
    "\n",
    "Pytorch has an implementation of the methods discussed in above paper in [kaiming.init_normal](https://pytorch.org/docs/stable/nn.init.html#torch.nn.init.kaiming_normal_) and it provides 5/3 * (1/sqrt(fan_in)) as the scale_down factor of w. let's use it with our example\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor(-0.0036) tensor(1.0131) tensor(-0.0065) tensor(1.6528)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABkoAAAGsCAYAAACSD/sZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABJ3ElEQVR4nO39fXRW5Z0v/n+SWBIUidhIQigaHlS0irEgGRytdswYLNNKaz3IcRXMePBUpV+dtCp0FKTaE6SUUpWR1hYfWi3oOtWeUzxYzTR2dYxQQZbWB5awpIiYIMyCKI7EIfv3R3/GSUmQOwRukv16rbVXyd7XvvLZu1vY+37f175ykiRJAgAAAAAAIIVys10AAAAAAABAtghKAAAAAACA1BKUAAAAAAAAqSUoAQAAAAAAUktQAgAAAAAApJagBAAAAAAASC1BCQAAAAAAkFpHZLuA7tDa2hpbtmyJo48+OnJycrJdDgAAHHRJksS7774bpaWlkZvr+098Ms9NAACkSSbPTL0iKNmyZUsMGTIk22UAAMAh9+abb8ZnPvOZbJdBD+C5CQCANNqfZ6ZeEZQcffTREfGXA+7fv3+WqwEAgIOvubk5hgwZ0nYvDJ/EcxMAAGmSyTNTrwhKPho23r9/fzf8AACkilcosb88NwEAkEb788zkZcYAAAAAAEBqCUoAAAAAAIDUEpQAAAAAAACpJSgBAAAAAABSS1ACAAAAAACklqAEAAAAAABILUEJAAAAAACQWoISAAAAAAAgtQQlAAAAAABAaglKAAAAAACA1BKUAAAAAAAAqSUoAQAAAAAAUktQAgAAAAAApJagBAAAAAAASC1BCQAAAAAAkFqCEgAAAAAAILWOyHYBAKRT2YzlGbXfOHfCQaoEAAA4HGX6zNBVnjUAMKIEAAAAAABILUEJAAAAAACQWoISAAAAAAAgtboUlCxatCjKysqioKAgKioqYtWqVfu139KlSyMnJycmTpzYbn2SJDFr1qwYNGhQ9O3bNyorK+P111/vSmkAAAAAAAD7LeOgZNmyZVFTUxOzZ8+ONWvWxBlnnBFVVVWxdevWfe63cePG+Pa3vx3nnnvuXtvmzZsXd955ZyxevDhWrlwZRx11VFRVVcUHH3yQaXkAAAAAAAD7LeOgZMGCBTFt2rSorq6OU089NRYvXhxHHnlkLFmypNN99uzZE5dffnnMmTMnhg0b1m5bkiSxcOHCuPnmm+Piiy+OUaNGxYMPPhhbtmyJxx9/POMDAgAAyLZMRuHfe++9ce6558aAAQNiwIABUVlZuVf7K664InJyctot48ePP9iHAQAAqZBRUNLS0hKrV6+OysrKjzvIzY3KyspoaGjodL/vfve7MXDgwLjyyiv32vbGG29EY2Njuz4LCwujoqKi0z53794dzc3N7RYAAIDDQaaj8Ovr62Py5Mnxu9/9LhoaGmLIkCFx4YUXxltvvdWu3fjx4+Ptt99uW375y18eisMBAIBeL6OgZNu2bbFnz54oLi5ut764uDgaGxs73OcPf/hD/OxnP4t77723w+0f7ZdJn7W1tVFYWNi2DBkyJJPDAAAAOGgyHYX/0EMPxTXXXBPl5eUxcuTI+OlPfxqtra1RV1fXrl1+fn6UlJS0LQMGDDgUhwMAAL1elyZz31/vvvtufP3rX4977703ioqKuq3fmTNnxs6dO9uWN998s9v6BgAA6KqujsL/r95///348MMP49hjj223vr6+PgYOHBgnn3xyXH311bF9+/Z99mMkPgAA7J8jMmlcVFQUeXl50dTU1G59U1NTlJSU7NV+w4YNsXHjxvjSl77Utq61tfUvv/iII2LdunVt+zU1NcWgQYPa9VleXt5hHfn5+ZGfn59J6QD0cGUzlme8z8a5Ew5CJQDQuX2Nwn/ttdf2q4+bbropSktL24Ut48ePj69+9asxdOjQ2LBhQ3znO9+Jiy66KBoaGiIvL6/Dfmpra2POnDldPxgAAEiJjEaU9OnTJ0aPHt1uCPhHQ8LHjRu3V/uRI0fGSy+9FGvXrm1bvvzlL8cXvvCFWLt2bQwZMiSGDh0aJSUl7fpsbm6OlStXdtgnAABAbzV37txYunRpPPbYY1FQUNC2/rLLLosvf/nLcfrpp8fEiRPjN7/5Tfzxj3+M+vr6TvsyEh8AAPZPRiNKIiJqampi6tSpMWbMmBg7dmwsXLgwdu3aFdXV1RERMWXKlBg8eHDU1tZGQUFBnHbaae32P+aYYyIi2q2//vrr4/bbb48TTzwxhg4dGrfcckuUlpbGxIkTu35kAAAAh1imo/D/q/nz58fcuXPj6aefjlGjRu2z7bBhw6KoqCjWr18fF1xwQYdtjMQHAID9k3FQMmnSpHjnnXdi1qxZ0djYGOXl5bFixYq2oeWbNm2K3NzMpj658cYbY9euXXHVVVfFjh074pxzzokVK1a0+wYVAADA4e6/jsL/6ItfH43Cnz59eqf7zZs3L773ve/Fk08+GWPGjPnE37N58+bYvn17u9cXAwAAXZOTJEmS7SIOVHNzcxQWFsbOnTujf//+2S4HgP3QlTlHMmWOEqA3cw98+Fq2bFlMnTo1fvzjH7eNwn/kkUfitddei+Li4naj8CMi7rjjjpg1a1Y8/PDD8bd/+7dt/fTr1y/69esX7733XsyZMycuueSSKCkpiQ0bNsSNN94Y7777brz00kv7PWrENQP0NIfimSHCcwNAb5XJ/W/GI0oAAADoXKaj8O+5555oaWmJr33ta+36mT17dtx6662Rl5cXL774YjzwwAOxY8eOKC0tjQsvvDBuu+02r9YCAIBuICgBAADoZtOnT+/0VVt/PQH7xo0b99lX375948knn+ymygAAgL+W2WQiAAAAAAAAvYgRJQAAAACkVqZzoZjTBKD3MaIEAAAAAABILUEJAAAAAACQWoISAAAAAAAgtQQlAAAAAABAaglKAAAAAACA1BKUAAAAAAAAqSUoAQAAAAAAUuuIbBcAwOGlbMbyjPfZOHfCQagEAAAAAA4+I0oAAAAAAIDUEpQAAAAAAACp5dVbAPRaXXmNWFd49RgAAABAz2VECQAAAAAAkFqCEgAAAAAAILUEJQAAAAAAQGoJSgAAAAAAgNQSlAAAAAAAAKklKAEAAAAAAFJLUAIAAAAAAKSWoAQAAAAAAEitI7JdAAAAAAC9W9mM5dkuAQA6JSgB4IB56AEAAACgp/LqLQAAAAAAILUEJQAAAAAAQGoJSgAAAAAAgNQSlAAAAAAAAKklKAEAAAAAAFJLUAIAAAAAAKSWoAQAAAAAAEgtQQkAAAAAAJBaghIAAAAAACC1uhSULFq0KMrKyqKgoCAqKipi1apVnbb91a9+FWPGjIljjjkmjjrqqCgvL4+f//zn7dpcccUVkZOT024ZP358V0oDAAAAAADYb0dkusOyZcuipqYmFi9eHBUVFbFw4cKoqqqKdevWxcCBA/dqf+yxx8Y///M/x8iRI6NPnz7xm9/8Jqqrq2PgwIFRVVXV1m78+PFx3333tf2cn5/fxUMCAAAAAADYPxmPKFmwYEFMmzYtqqur49RTT43FixfHkUceGUuWLOmw/fnnnx9f+cpX4pRTTonhw4fHddddF6NGjYo//OEP7drl5+dHSUlJ2zJgwICuHREAAAAAAMB+yigoaWlpidWrV0dlZeXHHeTmRmVlZTQ0NHzi/kmSRF1dXaxbty4+//nPt9tWX18fAwcOjJNPPjmuvvrq2L59e6f97N69O5qbm9stAAAAAAAAmcro1Vvbtm2LPXv2RHFxcbv1xcXF8dprr3W6386dO2Pw4MGxe/fuyMvLi3/5l3+Jv//7v2/bPn78+PjqV78aQ4cOjQ0bNsR3vvOduOiii6KhoSHy8vL26q+2tjbmzJmTSekAAAAAAAB7yXiOkq44+uijY+3atfHee+9FXV1d1NTUxLBhw+L888+PiIjLLrusre3pp58eo0aNiuHDh0d9fX1ccMEFe/U3c+bMqKmpafu5ubk5hgwZctCPAwAAAAAA6F0yCkqKiooiLy8vmpqa2q1vamqKkpKSTvfLzc2NESNGREREeXl5vPrqq1FbW9sWlPy1YcOGRVFRUaxfv77DoCQ/P99k7wAAAAAAwAHLaI6SPn36xOjRo6Ourq5tXWtra9TV1cW4ceP2u5/W1tbYvXt3p9s3b94c27dvj0GDBmVSHgAAAAAAQEYyfvVWTU1NTJ06NcaMGRNjx46NhQsXxq5du6K6ujoiIqZMmRKDBw+O2traiPjLfCJjxoyJ4cOHx+7du+OJJ56In//853HPPfdERMR7770Xc+bMiUsuuSRKSkpiw4YNceONN8aIESOiqqqqGw8VAAAAAACgvYyDkkmTJsU777wTs2bNisbGxigvL48VK1a0TfC+adOmyM39eKDKrl274pprronNmzdH3759Y+TIkfGLX/wiJk2aFBEReXl58eKLL8YDDzwQO3bsiNLS0rjwwgvjtttu83otAAAAAA4rZTOWZ7zPxrkTDkIlAHSXnCRJkmwXcaCam5ujsLAwdu7cGf379892OQA9Wldu+tPOQw+QDe6ByZRrBsimtD9neGYAOPQyuf/NaI4SAAAAAACA3iTjV28B0HOk/VtbAAAAAPBJjCgBAAAAAABSS1ACAAAAAACklqAEAAAAAABILUEJAAAAAACQWoISAAAAAAAgtQQlAAAAAABAaglKAAAAAACA1BKUAAAAAAAAqSUoAQAAAAAAUktQAgAAAAAApJagBAAAAAAASC1BCQAAAAAAkFqCEgAAAAAAILUEJQAAAAAAQGoJSgAAAAAAgNQSlAAAAHSzRYsWRVlZWRQUFERFRUWsWrWq07b33ntvnHvuuTFgwIAYMGBAVFZW7tU+SZKYNWtWDBo0KPr27RuVlZXx+uuvH+zDAACAVBCUAAAAdKNly5ZFTU1NzJ49O9asWRNnnHFGVFVVxdatWztsX19fH5MnT47f/e530dDQEEOGDIkLL7ww3nrrrbY28+bNizvvvDMWL14cK1eujKOOOiqqqqrigw8+OFSHBQAAvVZOkiRJtos4UM3NzVFYWBg7d+6M/v37Z7scgIOibMbybJdAJzbOnZDtEoAUcg98+KqoqIizzjor7r777oiIaG1tjSFDhsQ3v/nNmDFjxifuv2fPnhgwYEDcfffdMWXKlEiSJEpLS+Nb3/pWfPvb346IiJ07d0ZxcXHcf//9cdlll+1XXa4ZIJvS/jzjmQHg0Mvk/teIEgAAgG7S0tISq1evjsrKyrZ1ubm5UVlZGQ0NDfvVx/vvvx8ffvhhHHvssRER8cYbb0RjY2O7PgsLC6OiomKffe7evTuam5vbLQAAwN4EJQAAAN1k27ZtsWfPniguLm63vri4OBobG/erj5tuuilKS0vbgpGP9su0z9ra2igsLGxbhgwZksmhAABAahyR7QIAoKfrymsEDL0HoCNz586NpUuXRn19fRQUFBxQXzNnzoyampq2n5ubm4UlAADQAUEJAABANykqKoq8vLxoampqt76pqSlKSkr2ue/8+fNj7ty58fTTT8eoUaPa1n+0X1NTUwwaNKhdn+Xl5Z32l5+fH/n5+V04CgAASBdBCQAAQDfp06dPjB49Ourq6mLixIkR8ZfJ3Ovq6mL69Omd7jdv3rz43ve+F08++WSMGTOm3bahQ4dGSUlJ1NXVtQUjzc3NsXLlyrj66qsP1qEAdCrtE7MD0PsISgAAALpRTU1NTJ06NcaMGRNjx46NhQsXxq5du6K6ujoiIqZMmRKDBw+O2traiIi44447YtasWfHwww9HWVlZ27wj/fr1i379+kVOTk5cf/31cfvtt8eJJ54YQ4cOjVtuuSVKS0vbwhgAAKDrBCUAAADdaNKkSfHOO+/ErFmzorGxMcrLy2PFihVtk7Fv2rQpcnNz29rfc8890dLSEl/72tfa9TN79uy49dZbIyLixhtvjF27dsVVV10VO3bsiHPOOSdWrFhxwPOYAAAAETlJkiTZLuJANTc3R2FhYezcuTP69++f7XIADgrD23sXk7kDB8o9MJlyzQDdxbNJ5tz/Axx6mdz/5u5zKwAAAAAAQC8mKAEAAAAAAFJLUAIAAAAAAKSWoAQAAAAAAEgtQQkAAAAAAJBaghIAAAAAACC1uhSULFq0KMrKyqKgoCAqKipi1apVnbb91a9+FWPGjIljjjkmjjrqqCgvL4+f//zn7dokSRKzZs2KQYMGRd++faOysjJef/31rpQGAAAAAACw3zIOSpYtWxY1NTUxe/bsWLNmTZxxxhlRVVUVW7du7bD9scceG//8z/8cDQ0N8eKLL0Z1dXVUV1fHk08+2dZm3rx5ceedd8bixYtj5cqVcdRRR0VVVVV88MEHXT8yAAAAAACAT5BxULJgwYKYNm1aVFdXx6mnnhqLFy+OI488MpYsWdJh+/PPPz++8pWvxCmnnBLDhw+P6667LkaNGhV/+MMfIuIvo0kWLlwYN998c1x88cUxatSoePDBB2PLli3x+OOPd9jn7t27o7m5ud0CAAAAAACQqYyCkpaWlli9enVUVlZ+3EFublRWVkZDQ8Mn7p8kSdTV1cW6devi85//fEREvPHGG9HY2Niuz8LCwqioqOi0z9ra2igsLGxbhgwZkslhAAAAAAAARESGQcm2bdtiz549UVxc3G59cXFxNDY2drrfzp07o1+/ftGnT5+YMGFC3HXXXfH3f//3ERFt+2XS58yZM2Pnzp1ty5tvvpnJYQAAAAAAAERExBGH4pccffTRsXbt2njvvfeirq4uampqYtiwYXH++ed3qb/8/PzIz8/v3iIBAAAAAIDUySgoKSoqiry8vGhqamq3vqmpKUpKSjrdLzc3N0aMGBEREeXl5fHqq69GbW1tnH/++W37NTU1xaBBg9r1WV5enkl5AAAAAAAAGcno1Vt9+vSJ0aNHR11dXdu61tbWqKuri3Hjxu13P62trbF79+6IiBg6dGiUlJS067O5uTlWrlyZUZ8AAAAAAACZyvjVWzU1NTF16tQYM2ZMjB07NhYuXBi7du2K6urqiIiYMmVKDB48OGprayPiLxOvjxkzJoYPHx67d++OJ554In7+85/HPffcExEROTk5cf3118ftt98eJ554YgwdOjRuueWWKC0tjYkTJ3bfkQIAAAAAAPyVjIOSSZMmxTvvvBOzZs2KxsbGKC8vjxUrVrRNxr5p06bIzf14oMquXbvimmuuic2bN0ffvn1j5MiR8Ytf/CImTZrU1ubGG2+MXbt2xVVXXRU7duyIc845J1asWBEFBQXdcIgAAAAAAAAdy0mSJMl2EQequbk5CgsLY+fOndG/f/9slwNwUJTNWJ7tEuhGG+dOyHYJQA/nHphMuWaA7uLZJHPu/wEOvUzufzOaowQAAAAAAKA3EZQAAAAAAACpJSgBAAAAAABSS1ACAAAAAACklqAEAAAAAABILUEJAAAAAACQWkdkuwAAAAAA6M3KZizPeJ+NcycchEoA6IgRJQAAAAAAQGoJSgAAAAAAgNQSlAAAAAAAAKklKAEAAAAAAFJLUAIAAAAAAKSWoAQAAAAAAEgtQQkAAAAAAJBaghIAAAAAACC1jsh2AQBpVTZjebZLIIsy/f9/49wJB6kSAAAAgHQzogQAAAAAAEgtQQkAAAAAAJBaghIAAAAAACC1BCUAAAAAAEBqCUoAAAAAAIDUEpQAAAAAAACpJSgBAAAAAABSS1ACAAAAAACklqAEAAAAAABILUEJAAAAAACQWoISAAAAAAAgtQQlAAAAAABAaglKAAAAAACA1BKUAAAAAAAAqSUoAQAAAAAAUuuIbBcAAHyyshnLM95n49wJB6ESAAAAgN7FiBIAAAAAACC1jCgBAAAASLGujF4GgN6kSyNKFi1aFGVlZVFQUBAVFRWxatWqTtvee++9ce6558aAAQNiwIABUVlZuVf7K664InJyctot48eP70ppAAAAAAAA+y3jESXLli2LmpqaWLx4cVRUVMTChQujqqoq1q1bFwMHDtyrfX19fUyePDnOPvvsKCgoiDvuuCMuvPDCePnll2Pw4MFt7caPHx/33Xdf28/5+fldPCSAQ883sAAAAACgZ8p4RMmCBQti2rRpUV1dHaeeemosXrw4jjzyyFiyZEmH7R966KG45pprory8PEaOHBk//elPo7W1Nerq6tq1y8/Pj5KSkrZlwIABndawe/fuaG5ubrcAAAAAAABkKqOgpKWlJVavXh2VlZUfd5CbG5WVldHQ0LBffbz//vvx4YcfxrHHHttufX19fQwcODBOPvnkuPrqq2P79u2d9lFbWxuFhYVty5AhQzI5DAAAAAAAgIjIMCjZtm1b7NmzJ4qLi9utLy4ujsbGxv3q46abborS0tJ2Ycv48ePjwQcfjLq6urjjjjvimWeeiYsuuij27NnTYR8zZ86MnTt3ti1vvvlmJocBAAAAAAAQEV2Yo+RAzJ07N5YuXRr19fVRUFDQtv6yyy5r+/Ppp58eo0aNiuHDh0d9fX1ccMEFe/WTn59vDhMAAAAAAOCAZTSipKioKPLy8qKpqand+qampigpKdnnvvPnz4+5c+fGb3/72xg1atQ+2w4bNiyKiopi/fr1mZQHAAAAAACQkYyCkj59+sTo0aPbTcT+0cTs48aN63S/efPmxW233RYrVqyIMWPGfOLv2bx5c2zfvj0GDRqUSXkAAAAAAAAZySgoiYioqamJe++9Nx544IF49dVX4+qrr45du3ZFdXV1RERMmTIlZs6c2db+jjvuiFtuuSWWLFkSZWVl0djYGI2NjfHee+9FRMR7770XN9xwQzz33HOxcePGqKuri4svvjhGjBgRVVVV3XSYAAAAAAAAe8t4jpJJkybFO++8E7NmzYrGxsYoLy+PFStWtE3wvmnTpsjN/Th/ueeee6KlpSW+9rWvtetn9uzZceutt0ZeXl68+OKL8cADD8SOHTuitLQ0LrzwwrjtttvMQwIAAAAAABxUGY8oiYiYPn16/PnPf47du3fHypUro6Kiom1bfX193H///W0/b9y4MZIk2Wu59dZbIyKib9++8eSTT8bWrVujpaUlNm7cGD/5yU/aghcAAICeZtGiRVFWVhYFBQVRUVERq1at6rTtyy+/HJdcckmUlZVFTk5OLFy4cK82t956a+Tk5LRbRo4ceRCPAAAA0qNLQQkAAAAdW7ZsWdTU1MTs2bNjzZo1ccYZZ0RVVVVs3bq1w/bvv/9+DBs2LObOnRslJSWd9vvZz3423n777bblD3/4w8E6BAAASBVBCQAAQDdasGBBTJs2Laqrq+PUU0+NxYsXx5FHHhlLlizpsP1ZZ50V3//+9+Oyyy7b5+uHjzjiiCgpKWlbioqK9lnH7t27o7m5ud0CAADsTVACAADQTVpaWmL16tVRWVnZti43NzcqKyujoaHhgPp+/fXXo7S0NIYNGxaXX355bNq0aZ/ta2tro7CwsG0ZMmTIAf1+AADorQQlAAAA3WTbtm2xZ8+eveZcLC4ujsbGxi73W1FREffff3+sWLEi7rnnnnjjjTfi3HPPjXfffbfTfWbOnBk7d+5sW958880u/34AAOjNjsh2AQAAAOzbRRdd1PbnUaNGRUVFRZxwwgnxyCOPxJVXXtnhPvn5+ft8lRcAAPAXRpQAAAB0k6KiosjLy4umpqZ265uamvY5UXumjjnmmDjppJNi/fr13dYnAACklaAEAACgm/Tp0ydGjx4ddXV1betaW1ujrq4uxo0b122/57333osNGzbEoEGDuq1PAABIK6/eAgAA6EY1NTUxderUGDNmTIwdOzYWLlwYu3btiurq6oiImDJlSgwePDhqa2sj4i8TwL/yyittf37rrbdi7dq10a9fvxgxYkRERHz729+OL33pS3HCCSfEli1bYvbs2ZGXlxeTJ0/OzkECAEAvIigBAADoRpMmTYp33nknZs2aFY2NjVFeXh4rVqxom+B906ZNkZv78eD+LVu2xJlnntn28/z582P+/Plx3nnnRX19fUREbN68OSZPnhzbt2+P4447Ls4555x47rnn4rjjjjukxwYAAL2RoAQAAKCbTZ8+PaZPn97hto/Cj4+UlZVFkiT77G/p0qXdVRoAAPBXzFECAAAAAACklqAEAAAAAABILUEJAAAAAACQWuYoAQAAAIDDTNmM5Rnvs3HuhINQCUDvZ0QJAAAAAACQWoISAAAAAAAgtQQlAAAAAABAaglKAAAAAACA1BKUAAAAAAAAqSUoAQAAAAAAUktQAgAAAAAApJagBAAAAAAASC1BCQAAAAAAkFqCEgAAAAAAILUEJQAAAAAAQGoJSgAAAAAAgNQSlAAAAAAAAKklKAEAAAAAAFJLUAIAAAAAAKSWoAQAAAAAAEgtQQkAAAAAAJBaghIAAAAAACC1BCUAAAAAAEBqdSkoWbRoUZSVlUVBQUFUVFTEqlWrOm177733xrnnnhsDBgyIAQMGRGVl5V7tkySJWbNmxaBBg6Jv375RWVkZr7/+eldKAwAAAAAA2G8ZByXLli2LmpqamD17dqxZsybOOOOMqKqqiq1bt3bYvr6+PiZPnhy/+93voqGhIYYMGRIXXnhhvPXWW21t5s2bF3feeWcsXrw4Vq5cGUcddVRUVVXFBx980PUjAwAAAAAA+AQZByULFiyIadOmRXV1dZx66qmxePHiOPLII2PJkiUdtn/ooYfimmuuifLy8hg5cmT89Kc/jdbW1qirq4uIv4wmWbhwYdx8881x8cUXx6hRo+LBBx+MLVu2xOOPP35ABwcAAAAAALAvGQUlLS0tsXr16qisrPy4g9zcqKysjIaGhv3q4/33348PP/wwjj322IiIeOONN6KxsbFdn4WFhVFRUdFpn7t3747m5uZ2CwAAAAAAQKYyCkq2bdsWe/bsieLi4nbri4uLo7Gxcb/6uOmmm6K0tLQtGPlov0z6rK2tjcLCwrZlyJAhmRwGAAAAAABARHRxMveumjt3bixdujQee+yxKCgo6HI/M2fOjJ07d7Ytb775ZjdWCQAAAAAApMURmTQuKiqKvLy8aGpqare+qakpSkpK9rnv/PnzY+7cufH000/HqFGj2tZ/tF9TU1MMGjSoXZ/l5eUd9pWfnx/5+fmZlA4AAADQ65XNWJ7tEgCgx8loREmfPn1i9OjRbROxR0TbxOzjxo3rdL958+bFbbfdFitWrIgxY8a02zZ06NAoKSlp12dzc3OsXLlyn30CAAAAAAAcqIxGlERE1NTUxNSpU2PMmDExduzYWLhwYezatSuqq6sjImLKlCkxePDgqK2tjYiIO+64I2bNmhUPP/xwlJWVtc070q9fv+jXr1/k5OTE9ddfH7fffnuceOKJMXTo0LjllluitLQ0Jk6c2H1HCrCffAOLNOvK9b9x7oSDUAkAAADAoZFxUDJp0qR45513YtasWdHY2Bjl5eWxYsWKtsnYN23aFLm5Hw9Uueeee6KlpSW+9rWvtetn9uzZceutt0ZExI033hi7du2Kq666Knbs2BHnnHNOrFix4oDmMQGAtBP6AQAAAHyynCRJkmwXcaCam5ujsLAwdu7cGf379892OUAP58NlyIwRJZAd7oHJlGsG0sHzTLq5Nwf4WCb3vxnNUQIAAAAAANCbCEoAAAAAAIDUEpQAAAAAAACpJSgBAAAAAABSS1ACAAAAAACklqAEAAAAAABILUEJAAAAAACQWoISAAAAAAAgtQQlAAAAAABAaglKAAAAAACA1BKUAAAAAAAAqSUoAQAAAAAAUktQAgAAAAAApJagBAAAAAAASC1BCQAAAAAAkFqCEgAAAAAAILUEJQAAAAAAQGoJSgAAAAAAgNQSlAAAAAAAAKklKAEAAAAAAFJLUAIAAAAAAKTWEdkuAAAAAAA4cGUzlme8z8a5Ew5CJQA9ixElAAAAAABAaglKAAAAAACA1PLqLaBX68qwYwAAAAAgPYwoAQAAAAAAUktQAgAA0M0WLVoUZWVlUVBQEBUVFbFq1apO27788stxySWXRFlZWeTk5MTChQsPuE8AAGD/CUoAAAC60bJly6KmpiZmz54da9asiTPOOCOqqqpi69atHbZ///33Y9iwYTF37twoKSnplj4BAID9JygBAADoRgsWLIhp06ZFdXV1nHrqqbF48eI48sgjY8mSJR22P+uss+L73/9+XHbZZZGfn98tfQIAAPtPUAIAANBNWlpaYvXq1VFZWdm2Ljc3NyorK6OhoeGQ9rl79+5obm5utwAAAHsTlAAAAHSTbdu2xZ49e6K4uLjd+uLi4mhsbDykfdbW1kZhYWHbMmTIkC79fgAA6O0EJQAAAL3QzJkzY+fOnW3Lm2++me2SAADgsHREtgsAAADoLYqKiiIvLy+amprarW9qaup0ovaD1Wd+fn6nc54AAAAfM6IEAACgm/Tp0ydGjx4ddXV1betaW1ujrq4uxo0bd9j0CQAAfKxLQcmiRYuirKwsCgoKoqKiIlatWtVp25dffjkuueSSKCsri5ycnFi4cOFebW699dbIyclpt4wcObIrpQEAAGRVTU1N3HvvvfHAAw/Eq6++GldffXXs2rUrqqurIyJiypQpMXPmzLb2LS0tsXbt2li7dm20tLTEW2+9FWvXro3169fvd58AAEDXZfzqrWXLlkVNTU0sXrw4KioqYuHChVFVVRXr1q2LgQMH7tX+/fffj2HDhsWll14a//RP/9Rpv5/97Gfj6aef/riwI7wVDAAA6HkmTZoU77zzTsyaNSsaGxujvLw8VqxY0TYZ+6ZNmyI39+PvrG3ZsiXOPPPMtp/nz58f8+fPj/POOy/q6+v3q08AAKDrMk4jFixYENOmTWv75tLixYtj+fLlsWTJkpgxY8Ze7c8666w466yzIiI63N5WyBFHdPmdvQAAAIeT6dOnx/Tp0zvc9lH48ZGysrJIkuSA+gQAALouo1dvtbS0xOrVq6OysvLjDnJzo7KyMhoaGg6okNdffz1KS0tj2LBhcfnll8emTZs6bbt79+5obm5utwAAAAAAAGQqoxEl27Ztiz179uw1vLu4uDhee+21LhdRUVER999/f5x88snx9ttvx5w5c+Lcc8+NP/3pT3H00Ufv1b62tjbmzJnT5d8HAHSfshnLM95n49wJB6ESAAAAgMx1aTL37nbRRRfFpZdeGqNGjYqqqqp44oknYseOHfHII4902H7mzJmxc+fOtuXNN988xBUDAAAAAAC9QUYjSoqKiiIvLy+amprarW9qaurW+UWOOeaYOOmkk2L9+vUdbs/Pz4/8/Pxu+30AAAAAAEA6ZTSipE+fPjF69Oioq6trW9fa2hp1dXUxbty4bivqvffeiw0bNsSgQYO6rU8AAAAAAIC/ltGIkoiImpqamDp1aowZMybGjh0bCxcujF27dkV1dXVEREyZMiUGDx4ctbW1EfGXCeBfeeWVtj+/9dZbsXbt2ujXr1+MGDEiIiK+/e1vx5e+9KU44YQTYsuWLTF79uzIy8uLyZMnd9dxAgAAAAAA7CXjoGTSpEnxzjvvxKxZs6KxsTHKy8tjxYoVbRO8b9q0KXJzPx6osmXLljjzzDPbfp4/f37Mnz8/zjvvvKivr4+IiM2bN8fkyZNj+/btcdxxx8U555wTzz33XBx33HEHeHgAAAAAPVPZjOXZLgEAUiHjoCQiYvr06TF9+vQOt30UfnykrKwskiTZZ39Lly7tShkAAAAAAAAHJKM5SgAAAAAAAHoTQQkAAAAAAJBaghIAAAAAACC1BCUAAAAAAEBqdWkyd4BsKJuxPNslAAAAAAC9jBElAAAAAABAaglKAAAAAACA1BKUAAAAAAAAqSUoAQAAAAAAUktQAgAAAAAApJagBAAAAAAASK0jsl0AAJA+ZTOWZ9R+49wJB6kSAAAAIO2MKAEAAAAAAFJLUAIAAAAAAKSWoAQAAAAAAEgtQQkAAAAAAJBaghIAAAAAACC1BCUAAAAAAEBqCUoAAAAAAIDUEpQAAAAAAACpdUS2CwAAAAAAsqNsxvKM2m+cO+EgVQKQPUaUAAAAAAAAqSUoAQAAAAAAUktQAgAAAAAApJagBAAAAAAASC1BCQAAAAAAkFqCEgAAAAAAILUEJQAAAAAAQGoJSgAAAAAAgNQSlAAAAAAAAKklKAEAAAAAAFJLUAIAAAAAAKSWoAQAAAAAAEitI7JdAAAAAEBvVzZjebZLAAA60aURJYsWLYqysrIoKCiIioqKWLVqVadtX3755bjkkkuirKwscnJyYuHChQfcJwAAAAAAQHfIOChZtmxZ1NTUxOzZs2PNmjVxxhlnRFVVVWzdurXD9u+//34MGzYs5s6dGyUlJd3SJwAAAAAAQHfI+NVbCxYsiGnTpkV1dXVERCxevDiWL18eS5YsiRkzZuzV/qyzzoqzzjorIqLD7V3pE+j5DDsHAAAAAA4HGY0oaWlpidWrV0dlZeXHHeTmRmVlZTQ0NHSpgK70uXv37mhubm63AAAAAAAAZCqjoGTbtm2xZ8+eKC4ubre+uLg4Ghsbu1RAV/qsra2NwsLCtmXIkCFd+t0AAAAAAEC6dWky92ybOXNm7Ny5s2158803s10SAAAAAADQA2U0R0lRUVHk5eVFU1NTu/VNTU2dTtR+MPrMz8+P/Pz8Lv0+AAAAAACAj2QUlPTp0ydGjx4ddXV1MXHixIiIaG1tjbq6upg+fXqXCjgYfQIAvUvZjOUZ77Nx7oSDUAkAAADQ22QUlERE1NTUxNSpU2PMmDExduzYWLhwYezatSuqq6sjImLKlCkxePDgqK2tjYi/TNb+yiuvtP35rbfeirVr10a/fv1ixIgR+9UnAAAAAADAwZBxUDJp0qR45513YtasWdHY2Bjl5eWxYsWKtsnYN23aFLm5H099smXLljjzzDPbfp4/f37Mnz8/zjvvvKivr9+vPgEAAAAAAA6GjIOSiIjp06d3+lqsj8KPj5SVlUWSJAfUJwAAAAAAwMGQ+8lNAAAAAAAAeidBCQAAAAAAkFqCEgAAAAAAILUEJQAAAAAAQGoJSgAAAAAAgNQSlAAAAAAAAKl1RLYLAAA4GMpmLM94n41zJxyESgAAAIDDmRElAAAA3WzRokVRVlYWBQUFUVFREatWrdpn+0cffTRGjhwZBQUFcfrpp8cTTzzRbvsVV1wROTk57Zbx48cfzEMAAIDUMKIEAACgGy1btixqampi8eLFUVFREQsXLoyqqqpYt25dDBw4cK/2zz77bEyePDlqa2vjH/7hH+Lhhx+OiRMnxpo1a+K0005razd+/Pi477772n7Oz88/JMcDAP+VkdtAb2RECQAAQDdasGBBTJs2Laqrq+PUU0+NxYsXx5FHHhlLlizpsP2PfvSjGD9+fNxwww1xyimnxG233Raf+9zn4u67727XLj8/P0pKStqWAQMGHIrDAQCAXk9QAgAA0E1aWlpi9erVUVlZ2bYuNzc3Kisro6GhocN9Ghoa2rWPiKiqqtqrfX19fQwcODBOPvnkuPrqq2P79u37rGX37t3R3NzcbgEAAPYmKAEAAOgm27Ztiz179kRxcXG79cXFxdHY2NjhPo2NjZ/Yfvz48fHggw9GXV1d3HHHHfHMM8/ERRddFHv27Om0ltra2igsLGxbhgwZcgBHBgAAvZc5SgAAAA5zl112WdufTz/99Bg1alQMHz486uvr44ILLuhwn5kzZ0ZNTU3bz83NzcISAADogBElAAAA3aSoqCjy8vKiqamp3fqmpqYoKSnpcJ+SkpKM2kdEDBs2LIqKimL9+vWdtsnPz4/+/fu3WwAAgL0ZUQJ0i7IZy7NdAgBA1vXp0ydGjx4ddXV1MXHixIiIaG1tjbq6upg+fXqH+4wbNy7q6uri+uuvb1v31FNPxbhx4zr9PZs3b47t27fHoEGDurN8AABIJSNKAAAAulFNTU3ce++98cADD8Srr74aV199dezatSuqq6sjImLKlCkxc+bMtvbXXXddrFixIn7wgx/Ea6+9Frfeems8//zzbcHKe++9FzfccEM899xzsXHjxqirq4uLL744RowYEVVVVVk5RgAA6E2MKAEAAOhGkyZNinfeeSdmzZoVjY2NUV5eHitWrGibsH3Tpk2Rm/vxd9bOPvvsePjhh+Pmm2+O73znO3HiiSfG448/HqeddlpEROTl5cWLL74YDzzwQOzYsSNKS0vjwgsvjNtuuy3y8/OzcowAANCbCEoAAAC62fTp0zt91VZ9ff1e6y699NK49NJLO2zft2/fePLJJ7uzPAAA4L/w6i0AAAAAACC1BCUAAAAAAEBqCUoAAAAAAIDUMkcJAAAAQAbKZizPdgkAQDcyogQAAAAAAEgtQQkAAAAAAJBaghIAAAAAACC1zFEC7MX7dgEAAACAtDCiBAAAAAAASC1BCQAAAAAAkFqCEgAAAAAAILUEJQAAAAAAQGqZzB0A4P+vbMbyjPfZOHfCQagEAAAAOFSMKAEAAAAAAFJLUAIAAAAAAKSWV28BAAAAAAeNV9wCh7sujShZtGhRlJWVRUFBQVRUVMSqVav22f7RRx+NkSNHRkFBQZx++unxxBNPtNt+xRVXRE5OTrtl/PjxXSkNAAAAAABgv2UclCxbtixqampi9uzZsWbNmjjjjDOiqqoqtm7d2mH7Z599NiZPnhxXXnllvPDCCzFx4sSYOHFi/OlPf2rXbvz48fH222+3Lb/85S+7dkQAAAAAAAD7KeOgZMGCBTFt2rSorq6OU089NRYvXhxHHnlkLFmypMP2P/rRj2L8+PFxww03xCmnnBK33XZbfO5zn4u77767Xbv8/PwoKSlpWwYMGNBpDbt3747m5uZ2CwAAAAAAQKYyCkpaWlpi9erVUVlZ+XEHublRWVkZDQ0NHe7T0NDQrn1ERFVV1V7t6+vrY+DAgXHyySfH1VdfHdu3b++0jtra2igsLGxbhgwZkslhAAAAAAAARESGQcm2bdtiz549UVxc3G59cXFxNDY2drhPY2PjJ7YfP358PPjgg1FXVxd33HFHPPPMM3HRRRfFnj17Ouxz5syZsXPnzrblzTffzOQwAAAAAAAAIiLiiGwXEBFx2WWXtf359NNPj1GjRsXw4cOjvr4+Lrjggr3a5+fnR35+/qEsEQAAAAAA6IUyGlFSVFQUeXl50dTU1G59U1NTlJSUdLhPSUlJRu0jIoYNGxZFRUWxfv36TMoDAAAAAADISEYjSvr06ROjR4+Ourq6mDhxYkREtLa2Rl1dXUyfPr3DfcaNGxd1dXVx/fXXt6176qmnYty4cZ3+ns2bN8f27dtj0KBBmZQHdKBsxvJslwAAAAAAcNjK+NVbNTU1MXXq1BgzZkyMHTs2Fi5cGLt27Yrq6uqIiJgyZUoMHjw4amtrIyLiuuuui/POOy9+8IMfxIQJE2Lp0qXx/PPPx09+8pOIiHjvvfdizpw5cckll0RJSUls2LAhbrzxxhgxYkRUVVV146ECAAAAtOfLZQBAxkHJpEmT4p133olZs2ZFY2NjlJeXx4oVK9ombN+0aVPk5n78Rq+zzz47Hn744bj55pvjO9/5Tpx44onx+OOPx2mnnRYREXl5efHiiy/GAw88EDt27IjS0tK48MIL47bbbjMPCQAAAAAAcFB1aTL36dOnd/qqrfr6+r3WXXrppXHppZd22L5v377x5JNPdqUMAAAAAACAA5LRZO4AAAAAAAC9iaAEAAAAAABILUEJAAAAAACQWoISAAAAAAAgtbo0mTsAAAAAwMFSNmN5xvtsnDvhIFQCpIERJQAAAAAAQGoJSgAAAAAAgNTy6i0AgAPglQAAAADQsxlRAgAAAAAApJYRJdCDdOVbywAAAAAAdE5QAgAAAPQavmAGAGRKUAIAcIiZ1wQAAAAOH+YoAQAAAAAAUktQAgAAAAAApJagBAAAAAAASC1BCQAAAAAAkFqCEgAAAAAAILWOyHYBAAAAAAAHqmzG8oz32Th3wkGoBOhpBCWQJV35xxsAAAAAgO7l1VsAAAAAAEBqGVECAAAAHJaMxAcADgUjSgAAAAAAgNQSlAAAAAAAAKklKAEAAAAAAFLLHCUAAAAAQCplOhfSxrkTDlIlQDYZUQIAAAAAAKSWESUAAL1Qpt+Mi/DtOAAAANJJUALdpCsfSAEAAKSFZyYA4HDl1VsAAAAAAEBqGVECHfBNJwAON/5tAgCA7POKW+idjCgBAAAAAABSy4gSAAAiwrfjANh/RjoCAL2JESUAAAAAAEBqGVFCj+JbSwBweDEKBaDn85wFcHC5Z4bDX5dGlCxatCjKysqioKAgKioqYtWqVfts/+ijj8bIkSOjoKAgTj/99HjiiSfabU+SJGbNmhWDBg2Kvn37RmVlZbz++utdKQ0AACDrPDMBAAeibMbyjBeg6zIeUbJs2bKoqamJxYsXR0VFRSxcuDCqqqpi3bp1MXDgwL3aP/vsszF58uSora2Nf/iHf4iHH344Jk6cGGvWrInTTjstIiLmzZsXd955ZzzwwAMxdOjQuOWWW6KqqipeeeWVKCgoOPCjBACgR8v0wc838Mgmz0xkmw/LANLJyBXoupwkSZJMdqioqIizzjor7r777oiIaG1tjSFDhsQ3v/nNmDFjxl7tJ02aFLt27Yrf/OY3bev+5m/+JsrLy2Px4sWRJEmUlpbGt771rfj2t78dERE7d+6M4uLiuP/+++Oyyy7bq8/du3fH7t27237euXNnHH/88fHmm29G//79MzkcOnHa7Ccz3udPc6oOQiXtdaUuACB9DsV9SbY1NzfHkCFDYseOHVFYWJjtcvgvDodnpgjPTYcjzzMA9AZpuNemd8jkmSmjESUtLS2xevXqmDlzZtu63NzcqKysjIaGhg73aWhoiJqamnbrqqqq4vHHH4+IiDfeeCMaGxujsrKybXthYWFUVFREQ0NDhzf9tbW1MWfOnL3WDxkyJJPDoZsVLsx2BQAAf5Gm+5J3331XUHIYOVyemSI8NwEAB0ea7rXpHfbnmSmjoGTbtm2xZ8+eKC4ubre+uLg4XnvttQ73aWxs7LB9Y2Nj2/aP1nXW5q/NnDmz3YNEa2tr/Pu//3t8+tOfjpycnEwOqcf4KP3y7a/MOXcHxvk7MM7fgXH+us65OzDO34Fx/g7M/p6/JEni3XffjdLS0kNYHZ/kcHlmikjnc9P+8HdU93Euu5fz2b2cz+7lfHYf57J7OZ/dq7edz0yemTKeo+RwkJ+fH/n5+e3WHXPMMdkp5hDr379/r7hIs8G5OzDO34Fx/g6M89d1zt2Bcf4OjPN3YPbn/BlJwr6k+blpf/g7qvs4l93L+exezmf3cj67j3PZvZzP7tWbzuf+PjPlZtJpUVFR5OXlRVNTU7v1TU1NUVJS0uE+JSUl+2z/0f9m0icAAMDhyDMTAAD0PBkFJX369InRo0dHXV1d27rW1taoq6uLcePGdbjPuHHj2rWPiHjqqafa2g8dOjRKSkratWlubo6VK1d22icAAMDhyDMTAAD0PBm/equmpiamTp0aY8aMibFjx8bChQtj165dUV1dHRERU6ZMicGDB0dtbW1ERFx33XVx3nnnxQ9+8IOYMGFCLF26NJ5//vn4yU9+EhEROTk5cf3118ftt98eJ554YgwdOjRuueWWKC0tjYkTJ3bfkfZw+fn5MXv27L2GzvPJnLsD4/wdGOfvwDh/XefcHRjn78A4fwfG+ev5PDMd3vw31n2cy+7lfHYv57N7OZ/dx7nsXs5n90rz+cxJkiTJdKe77747vv/970djY2OUl5fHnXfeGRUVFRERcf7550dZWVncf//9be0fffTRuPnmm2Pjxo1x4oknxrx58+KLX/xi2/YkSWL27Nnxk5/8JHbs2BHnnHNO/Mu//EucdNJJB36EAAAAh5hnJgAA6Dm6FJQAAAAAAAD0BhnNUQIAAAAAANCbCEoAAAAAAIDUEpQAAAAAAACpJSgBAAAAAABSS1DSg+3evTvKy8sjJycn1q5dm+1yeowvf/nLcfzxx0dBQUEMGjQovv71r8eWLVuyXdZhb+PGjXHllVfG0KFDo2/fvjF8+PCYPXt2tLS0ZLu0HuN73/tenH322XHkkUfGMccck+1yDnuLFi2KsrKyKCgoiIqKili1alW2S+oxfv/738eXvvSlKC0tjZycnHj88cezXVKPUVtbG2eddVYcffTRMXDgwJg4cWKsW7cu22X1GPfcc0+MGjUq+vfvH/37949x48bF//t//y/bZfVIc+fOjZycnLj++uuzXQr0aPX19ZGTk9Ph8sc//rHT/c4///y92n/jG984hJUfvsrKyvY6N3Pnzt3nPh988EFce+218elPfzr69esXl1xySTQ1NR2iig9fXX3Gcn1+LNNnhkcffTRGjhwZBQUFcfrpp8cTTzxxiCo9vHXlHvj+++/f6zosKCg4RBUf3m699da9zs3IkSP3uY9rs2Md/ZuTk5MT1157bYftXZftfdJnA0mSxKxZs2LQoEHRt2/fqKysjNdff/0T++2tn9cISnqwG2+8MUpLS7NdRo/zhS98IR555JFYt25d/O///b9jw4YN8bWvfS3bZR32XnvttWhtbY0f//jH8fLLL8cPf/jDWLx4cXznO9/Jdmk9RktLS1x66aVx9dVXZ7uUw96yZcuipqYmZs+eHWvWrIkzzjgjqqqqYuvWrdkurUfYtWtXnHHGGbFo0aJsl9LjPPPMM3HttdfGc889F0899VR8+OGHceGFF8auXbuyXVqP8JnPfCbmzp0bq1evjueffz7+7u/+Li6++OJ4+eWXs11aj/LHP/4xfvzjH8eoUaOyXQr0eGeffXa8/fbb7Zb/8T/+RwwdOjTGjBmzz32nTZvWbr958+YdoqoPf9/97nfbnZtvfvOb+2z/T//0T/F//+//jUcffTSeeeaZ2LJlS3z1q189RNUevg7kGcv1mfkzw7PPPhuTJ0+OK6+8Ml544YWYOHFiTJw4Mf70pz8d4soPP129B+7fv3+76/DPf/7zIar48PfZz3623bn5wx/+0Glb12bn/vjHP7Y7j0899VRERFx66aWd7uO6/NgnfTYwb968uPPOO2Px4sWxcuXKOOqoo6Kqqio++OCDTvvs1Z/XJPRITzzxRDJy5Mjk5ZdfTiIieeGFF7JdUo/161//OsnJyUlaWlqyXUqPM2/evGTo0KHZLqPHue+++5LCwsJsl3FYGzt2bHLttde2/bxnz56ktLQ0qa2tzWJVPVNEJI899li2y+ixtm7dmkRE8swzz2S7lB5rwIAByU9/+tNsl9FjvPvuu8mJJ56YPPXUU8l5552XXHfdddkuCXqVlpaW5Ljjjku++93v7rOd//46d8IJJyQ//OEP97v9jh07kk996lPJo48+2rbu1VdfTSIiaWhoOAgV9mz784zl+vyLTJ8Z/tt/+2/JhAkT2q2rqKhI/uf//J8Htc6eaH/ugT3Xdm727NnJGWecsd/tXZv777rrrkuGDx+etLa2drjdddm5v/5soLW1NSkpKUm+//3vt63bsWNHkp+fn/zyl7/stJ/e/HmNESU9UFNTU0ybNi1+/vOfx5FHHpntcnq0f//3f4+HHnoozj777PjUpz6V7XJ6nJ07d8axxx6b7TLoZVpaWmL16tVRWVnZti43NzcqKyujoaEhi5WRRjt37oyI8HddF+zZsyeWLl0au3btinHjxmW7nB7j2muvjQkTJrT7OxDoPv/n//yf2L59e1RXV39i24ceeiiKioritNNOi5kzZ8b7779/CCrsGebOnRuf/vSn48wzz4zvf//78Z//+Z+dtl29enV8+OGH7f5eGzlyZBx//PHu7Tqwv89Yab8+u/LM0NDQsNe/r1VVVa7DDuzvPfB7770XJ5xwQgwZMsQo4r/y+uuvR2lpaQwbNiwuv/zy2LRpU6dtXZv7p6WlJX7xi1/EP/7jP0ZOTk6n7VyX++eNN96IxsbGdtdeYWFhVFRUdHrt9fbPa47IdgFkJkmSuOKKK+Ib3/hGjBkzJjZu3Jjtknqkm266Ke6+++54//3342/+5m/iN7/5TbZL6nHWr18fd911V8yfPz/bpdDLbNu2Lfbs2RPFxcXt1hcXF8drr72WpapIo9bW1rj++uvjb//2b+O0007Ldjk9xksvvRTjxo2LDz74IPr16xePPfZYnHrqqdkuq0dYunRprFmzZp/zJgAH5mc/+1lUVVXFZz7zmX22++///b/HCSecEKWlpfHiiy/GTTfdFOvWrYtf/epXh6jSw9f/9//9f/G5z30ujj322Hj22Wdj5syZ8fbbb8eCBQs6bN/Y2Bh9+vTZa46+4uLiaGxsPAQV9xz7+4zl+uzaM0NjY2OH7V2H7e3vPfDJJ58cS5YsiVGjRsXOnTtj/vz5cfbZZ8fLL7/8iX/H9nYVFRVx//33x8knnxxvv/12zJkzJ84999z405/+FEcfffRe7V2b++fxxx+PHTt2xBVXXNFpG9fl/vvo+srk2uvtn9cYUXKYmDFjRqeTDH60vPbaa3HXXXfFu+++GzNnzsx2yYeV/T1/H7nhhhvihRdeiN/+9reRl5cXU6ZMiSRJsngE2ZPpuYuIeOutt2L8+PFx6aWXxrRp07JU+eGhK+cP6Bmuvfba+NOf/hRLly7Ndik9ysknnxxr166NlStXxtVXXx1Tp06NV155JdtlHfbefPPNuO666+Khhx5K9YSTsL+6cg+2efPmePLJJ+PKK6/8xP6vuuqqqKqqitNPPz0uv/zyePDBB+Oxxx6LDRs2HKxDyqpMzmdNTU2cf/75MWrUqPjGN74RP/jBD+Kuu+6K3bt3Z/koDh8H+xkrbdcnh9b+3gOPGzcupkyZEuXl5XHeeefFr371qzjuuOPixz/+8SGq9PB10UUXxaWXXhqjRo2KqqqqeOKJJ2LHjh3xyCOPZLu0Hu1nP/tZXHTRRfucr9l1yYEwouQw8a1vfWufiWhExLBhw+Jf//Vfo6GhIfLz89ttGzNmTFx++eXxwAMPHMQqD1/7e/4+UlRUFEVFRXHSSSfFKaecEkOGDInnnnsula8GyfTcbdmyJb7whS/E2WefHT/5yU8OcnWHv0zPH5+sqKgo8vLyoqmpqd36pqamKCkpyVJVpM306dPjN7/5Tfz+97/3zaMM9enTJ0aMGBEREaNHj44//vGP8aMf/cjDySdYvXp1bN26NT73uc+1rduzZ0/8/ve/j7vvvjt2794deXl5WawQDi9duQe777774tOf/nR8+ctfzvj3VVRURMRfvvE/fPjwjPc/3B3IPW1FRUX853/+Z2zcuDFOPvnkvbaXlJRES0tL7Nixo92okt58b3eon7F6+/XZka48M5SUlHjG+AQHcg/8qU99Ks4888xYv379Qaqu5zrmmGPipJNO6vTcuDY/2Z///Od4+umnMx4557rs3EfXV1NTUwwaNKhtfVNTU5SXl3e4T2//vEZQcpg47rjj4rjjjvvEdnfeeWfcfvvtbT9v2bIlqqqqYtmyZW03R2m0v+evI62trRERqf0GVCbn7q233oovfOELMXr06LjvvvsiN9egtAO59uhYnz59YvTo0VFXVxcTJ06MiL/8d1pXVxfTp0/PbnH0ekmSxDe/+c147LHHor6+PoYOHZrtknq81tbW1P4bm4kLLrggXnrppXbrqqurY+TIkXHTTTcJSeCvZHoPliRJ3HfffTFlypQuzU24du3aiIh2HyT0JgdyT7t27drIzc2NgQMHdrh99OjR8alPfSrq6urikksuiYiIdevWxaZNm3rtF9UO9TNWb78+O9KVZ4Zx48ZFXV1dXH/99W3rnnrqqV57HWaiO+6B9+zZEy+99FJ88YtfPAgV9mzvvfdebNiwIb7+9a93uN21+cnuu+++GDhwYEyYMCGj/VyXnRs6dGiUlJREXV1dWzDS3Nzc9maAjvT6z2uyOpU8B+yNN95IIiJ54YUXsl1Kj/Dcc88ld911V/LCCy8kGzduTOrq6pKzzz47GT58ePLBBx9ku7zD2ubNm5MRI0YkF1xwQbJ58+bk7bffblvYP3/+85+TF154IZkzZ07Sr1+/5IUXXkheeOGF5N133812aYedpUuXJvn5+cn999+fvPLKK8lVV12VHHPMMUljY2O2S+sR3n333bbrKyKSBQsWJC+88ELy5z//OdulHfauvvrqpLCwMKmvr2/399z777+f7dJ6hBkzZiTPPPNM8sYbbyQvvvhiMmPGjCQnJyf57W9/m+3SeqTzzjsvue6667JdBvQKTz/9dBIRyauvvrrXts2bNycnn3xysnLlyiRJkmT9+vXJd7/73eT5559P3njjjeTXv/51MmzYsOTzn//8oS77sPPss88mP/zhD5O1a9cmGzZsSH7xi18kxx13XDJlypS2Nn99PpMkSb7xjW8kxx9/fPKv//qvyfPPP5+MGzcuGTduXDYO4bCyP89Yrs/OfdIzw9e//vVkxowZbe3/7d/+LTniiCOS+fPnJ6+++moye/bs5FOf+lTy0ksvZesQDhv7cw/81+dzzpw5yZNPPpls2LAhWb16dXLZZZclBQUFycsvv5yNQzisfOtb30rq6+uTN954I/m3f/u3pLKyMikqKkq2bt2aJIlrM1N79uxJjj/++OSmm27aa5vrct8+6bOBuXPnJsccc0zy61//OnnxxReTiy++OBk6dGjyH//xH219/N3f/V1y1113tf3cmz+vEZT0cIKSzLz44ovJF77wheTYY49N8vPzk7KysuQb3/hGsnnz5myXdti77777kojocGH/TJ06tcPz97vf/S7bpR2W7rrrruT4449P+vTpk4wdOzZ57rnnsl1Sj/G73/2uw2tt6tSp2S7tsNfZ33P33XdftkvrEf7xH/8xOeGEE5I+ffokxx13XHLBBRcISQ6AoAS6z+TJk5Ozzz67w20fPVN9dE+2adOm5POf/3zbM8OIESOSG264Idm5c+chrPjwtHr16qSioiIpLCxMCgoKklNOOSX5X//rf7X70tlfn88kSZL/+I//SK655ppkwIAByZFHHpl85Stf8YWrZP+esVyf+7avZ4bzzjtvr/vfRx55JDnppJOSPn36JJ/97GeT5cuXH+KKD0/7cw/81+fz+uuvbzv3xcXFyRe/+MVkzZo1h774w9CkSZOSQYMGJX369EkGDx6cTJo0KVm/fn3bdtdmZp588skkIpJ169bttc11uW+f9NlAa2trcssttyTFxcVJfn5+csEFF+x1nk844YRk9uzZ7db11s9rcpIkpTNYAwAAAAAAqWeCAQAAAAAAILUEJQAAAAAAQGoJSgAAAAAAgNQSlAAAAAAAAKklKAEAAAAAAFJLUAIAAAAAAKSWoAQAAAAAAEgtQQkAAAAAAJBaghIAAAAAACC1BCUAAAAAAEBqCUoAAAAAAIDU+v8BBet/VEka1OMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 2000x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = torch.randn(1000, 10) \n",
    "w = torch.randn(10, 200) * (5/3.0)/((10 ** 0.5))\n",
    "y = x @ w\n",
    "print(x.mean(), x.std(), y.mean(), y.std())\n",
    "plt.figure(figsize=(20, 5))\n",
    "plt.subplot(121)\n",
    "plt.hist(x.view(-1).tolist(), bins=50, density=True);\n",
    "plt.subplot(122)\n",
    "plt.hist(y.view(-1).tolist(), bins=50, density=True);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using this modification in our weight initialization now and running training loop again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "11697\n"
     ]
    }
   ],
   "source": [
    "n_embd = 10 # the dimensionality of the character embedding vectors\n",
    "n_hidden = 200 # the number of neurons in the hidden layer of the MLP\n",
    "\n",
    "g = torch.Generator().manual_seed(2147483647) # for reproducibility\n",
    "C  = torch.randn((vocab_size, n_embd),            generator=g)\n",
    "W1 = torch.randn((n_embd * block_size, n_hidden), generator=g) * (5/3.0)/((n_embd * block_size) ** 0.5)\n",
    "W2 = torch.randn((n_hidden, vocab_size),          generator=g) * 0.01\n",
    "b2 = torch.randn(vocab_size,                      generator=g) \n",
    "\n",
    "parameters = [C, W1, W2, b2]\n",
    "print(sum(p.nelement() for p in parameters)) # number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.8202\n",
      "  10000/ 200000: 2.0395\n",
      "  20000/ 200000: 2.5593\n",
      "  30000/ 200000: 2.0449\n",
      "  40000/ 200000: 2.3927\n",
      "  50000/ 200000: 1.8230\n",
      "  60000/ 200000: 2.0606\n",
      "  70000/ 200000: 2.3983\n",
      "  80000/ 200000: 2.2355\n",
      "  90000/ 200000: 2.0813\n",
      " 100000/ 200000: 2.1729\n",
      " 110000/ 200000: 2.3369\n",
      " 120000/ 200000: 1.6170\n",
      " 130000/ 200000: 1.8658\n",
      " 140000/ 200000: 2.0709\n",
      " 150000/ 200000: 2.0079\n",
      " 160000/ 200000: 1.9552\n",
      " 170000/ 200000: 2.4202\n",
      " 180000/ 200000: 2.0596\n",
      " 190000/ 200000: 2.1027\n"
     ]
    }
   ],
   "source": [
    "run_training_loop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train 2.0395455360412598\n",
      "val 2.1068387031555176\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x12af168d0>]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAGdCAYAAADJ6dNTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABTEklEQVR4nO3deVxUVf8H8M8MO8qmyKYo7rugqIjmUpJLVpYtZpZGZWXaY1E9Zou245Ol/SrTMk2fLJfKrCdNS5RMRVEU9w03UFlEZVPZz+8PZGRghrl31jvM5/168XrpzF3OnYG53znne75HJYQQICIiIlIIta0bQERERFQTgxMiIiJSFAYnREREpCgMToiIiEhRGJwQERGRojA4ISIiIkVhcEJERESKwuCEiIiIFMXZ1g2QorKyEhcvXoSXlxdUKpWtm0NEREQSCCFQWFiIkJAQqNXS+0PsIji5ePEiQkNDbd0MIiIiMkJGRgZatGgheXu7CE68vLwAVF2ct7e3jVtDREREUhQUFCA0NFRzH5fKLoKT6qEcb29vBidERER2Rm5KBhNiiYiISFEYnBAREZGiMDghIiIiRWFwQkRERIrC4ISIiIgUhcEJERERKYpRwcn8+fMRFhYGd3d3REVFITk5ud7t8/LyMGXKFAQHB8PNzQ0dOnTA+vXrjWowERERNWyy65ysWrUKcXFxWLhwIaKiovDpp59i+PDhOH78OAICAupsX1paijvvvBMBAQH46aef0Lx5c5w7dw6+vr7maD8RERE1MCohhJCzQ1RUFPr06YMvvvgCQNW6N6GhoXjhhRfw2muv1dl+4cKFmDNnDo4dOwYXFxejGllQUAAfHx/k5+ezCBsREZGdMPb+LWtYp7S0FCkpKYiJibl1ALUaMTExSEpK0rnPb7/9hujoaEyZMgWBgYHo1q0bPvzwQ1RUVMg5NRERETkIWcM6ubm5qKioQGBgoNbjgYGBOHbsmM59Tp8+jc2bN2P8+PFYv3490tLS8Pzzz6OsrAyzZs3SuU9JSQlKSko0/y8oKJDTTCIiIrJjFp+tU1lZiYCAAHz99deIjIzE2LFj8cYbb2DhwoV694mPj4ePj4/mhysSExEROQ5ZwYm/vz+cnJyQnZ2t9Xh2djaCgoJ07hMcHIwOHTrAyclJ81jnzp2RlZWF0tJSnfvMmDED+fn5mp+MjAw5zZTsaGYBvvnnNMorKi1yfCIiIpJPVnDi6uqKyMhIJCQkaB6rrKxEQkICoqOjde4zYMAApKWlobLyVgBw4sQJBAcHw9XVVec+bm5umhWILbkS8cj/+wfvrzuK73aes8jxiYiISD7ZwzpxcXFYtGgRli1bhqNHj2Ly5Mm4du0aYmNjAQATJkzAjBkzNNtPnjwZV65cwbRp03DixAmsW7cOH374IaZMmWK+qzDR4YvMaSEiIlIK2XVOxo4di0uXLmHmzJnIyspCREQENmzYoEmSTU9Ph1p9K+YJDQ3Fxo0b8dJLL6FHjx5o3rw5pk2bhunTp5vvKoiIiKjBkF3nxBYsVeck7LV1AIAHI1vg44fCzXZcIiIislKdEyIiIiJLY3BCREREisLghIiIiBSFwQkREREpCoMTIiIiUhQGJ0RERKQoDE4AhDX1tHUTiIiI6CaHDk56t/IDALQL8LJxS4iIiKiaQwcnREREpDwMToiIiEhRGJwQERGRojA4ISIiIkVhcEJERESKwuCEiIiIFIXBCRERESkKgxMiIiJSFAYnREREpCgMTgAAwtYNICIiopscOjhRqWzdAiIiIqrNoYMTIiIiUh4GJ0RERKQoDE6IiIhIURicEBERkaIwOCEiIiJFYXBCREREisLghIiIiBSFwQkREREpCoMTAIIFYomIiBTDoYMTFVgiloiISGkcOjghIiIi5WFwQkRERIrC4ISIiIgUhcEJERERKQqDEyIiIlIUBidERESkKAxOiIiISFEYnBAREZGiMDghIiIiRWFwAoDV64mIiJTDsYMTVq8nIiJSHMcOToiIiEhxGJwQERGRojA4ISIiIkVhcEJERESKwuCEiIiIFIXBCRERESkKgxMiIiJSFAYnZnbwfD72Z+TZuhlERER2y9nWDbCl0vJKAEClME+N2JLyCtzzxTYAwJF3h8PT1aFfXiIiIqM4dM9J6s0ejv/uOGeW4xWXVmr+XVRcbpZjEhERORqHDk6qJZ+9YusmEBER0U0MTuqRllOEM7nXbN0MIiIih8KkCD2ul5YjZu7fAICTH4yEixPjOCIiImvgHVePq9fLNP+uTpwlIiIiy2NwQkRERIrC4ISIiIgUxajgZP78+QgLC4O7uzuioqKQnJysd9ulS5dCpVJp/bi7uxvdYCIiImrYZAcnq1atQlxcHGbNmoW9e/ciPDwcw4cPR05Ojt59vL29kZmZqfk5d848dUWIGqr9GXmI/TYZJ7MLbd0UIiKrkx2czJ07F5MmTUJsbCy6dOmChQsXwtPTE0uWLNG7j0qlQlBQkOYnMDDQpEZbyq+pF/Dy6v0oq2ACrCGl5ZV4+7fD2HJMf1BKxhs9fzu2HL+Exxfr75UkImqoZAUnpaWlSElJQUxMzK0DqNWIiYlBUlKS3v2KiorQqlUrhIaGYvTo0Th8+HC95ykpKUFBQYHWj6WVlFdg2spU/Lz3PFbvyYCQUNK+uKwCz/x3D77fVdUTJGCeMvj2YPnOc1i64yxil+62dVMalNozw7IKim3UEiIi25EVnOTm5qKioqJOz0dgYCCysrJ07tOxY0csWbIEv/76K5YvX47Kykr0798f58+f13ue+Ph4+Pj4aH5CQ0PlNNMo3d/+U/PvK0WlWs+pVLr3WbU7A38eycYbvxzCsh1ntZ/Us09DcSHvhq2b0OB88udxdHjzD+xLvypp+7KKSmw4lIWr10oNb0xEZEcsPlsnOjoaEyZMQEREBAYPHow1a9agWbNm+Oqrr/TuM2PGDOTn52t+MjIyLN3MOt9Ytxy/ZHCfghu3aqHM+u0wzl/lDZuM9/nmNADAh+uPStp+/pY0PLc8BQ8s3GHJZhERWZ2sCrH+/v5wcnJCdna21uPZ2dkICgqSdAwXFxf07NkTaWlperdxc3ODm5ubnKaZ1ba0XOw6I3+9nZrBiiPadjIXs347hI8e7IHIVk1s3ZwGb92BTADA6UtcYoGIGhZZPSeurq6IjIxEQkKC5rHKykokJCQgOjpa0jEqKipw8OBBBAcHy2upFaWc0+5W54e/NI8t3oVTl67hka932ropDcrEJckcRiMihyJ7WCcuLg6LFi3CsmXLcPToUUyePBnXrl1DbGwsAGDChAmYMWOGZvt3330Xf/75J06fPo29e/fisccew7lz5/D000+b7yos7O7Pt2G3DVcuLq+oRFpOoaQkXSUoq7BcO7en5WLJtjN281qYw98nLuG1nw/YuhlERFYje+G/sWPH4tKlS5g5cyaysrIQERGBDRs2aJJk09PToVbfinmuXr2KSZMmISsrC35+foiMjMSOHTvQpUsX812FFfxxMAt9wuofqrhsocTEF1el4vcDmXhvdFc8Hh1mkXPYQtKpywj0dkObZo0l7zP+m10AgE5BXujfzh9Ltp2Bl7szHupt+aRpfX7YlQ4PVzXu79nCYufIKSix2LGJrKm4rAIl5ZXw8XCxdVNIwYxalXjq1KmYOnWqzucSExO1/j9v3jzMmzfPmNPYjLHfyV9Ysc+s7aj2+83cggWJpxQTnJjacXE8qxDjFlUN/5ydPUr2/ufzbiDjynW8+/sRALBZcJJTUIzXfzkIALg3vDmc1A18mhaRiaLjE3D1ehn2zxwGH08GKKQb19bRoaLScYYMbOVIZr7JxygsLjdDS0xsQ4nt20BkT6pXfN9/Ps+2DSFFY3BiAkcOYRyp4BwR2ZfrpeXIKWQBQ3vG4KQBulZSjjvn/i25XoapsguKcaO0wirnIiIypM/7m9D3gwRk5nOWm71icGIhKhuWiF29JwMnc4rw9dbTkravrBQmrScU9WEC+s9OMLyhlVVyeI7IIV27+WUp2Yh6VaQMDE7sSM1b7aEL+bjj40RsOFR32QC5OTN3f74N/T5MqFMlV47qcWRjTFySjD8OZhq9vy5vrj2IAf/ZjHwLF8arb0pzcZlle5P0LatARGTvGJxYyM979a8dJEVhcRm2nriE8ho9GpVCaLopn1uegtO51/Dc8hSTzgMARzILcPlaKU5kF0rex9TZOjX3//vEJUz+fq+s/VfvzsDBC3l6n1++Mx2Z+cVYtTvdyBYaNm3lPsTM3arzuRXJ6ej01gas3XdB9nHtvYTLv3/aj8e+2cWeKyIyGoMTIxWVlOOHXfpvfLP/OGZSobDHFidjwpJkzN9ySvNYdkEJouM344+DmXVyPNYdyMSvqfJvhDX9uCfDbiqR7jl3FdN/PmhwO0PDa7vPXkH65etGteHX1It6n5uxpqptL65KNerY9mz1nvPYlpaLwxctv5p4Q2HpHj4ie8PgxEhv/HLQosvZ78/IAwDM23SiznNfJp7SGuIpLqvAlB/2YtrKVORdN74Q3LKkc4j55G+9z5dVVKKk3HqJrzkFxVi+8xyuWWi67onsQjy0MAmD5mxx6Mz+4rIKXC81/2t8zxfbsOVYjtmP29As3nYG4e/8ie92nrN1U4gUg8GJkTYerpvrYS0HL+TjSo1qtKU1hn6u1zNrRkpPzg09eRJCCPSfvRmR720yKXm2PnvTr+K9349ogpEHFybhzbWH8O7/jljkfIcv3qq10veD+nNu1h3IxMHzptdmURohBLq/vRFdZm60SOAZu3S32Y/Z0Lx3s5DgW2sP2bglDY+9D5E6MgYnEpla16OguAxpOdJzOox1uagU76/TnkK85VgOwl5bh9Yz1uO/SWeNOm5JeSUuFZagqKQcWfmW6WUY8+UOLN52BvP+quotSr9SNdySYKVv34XFdbvWS8sr8V3SWUz5YS/u+WKbVdphrNLySvxrxT5ZeTalFZWatZDe/906U8+JiAxhcCKDKV3fA2ZvRszcrZrhGkv5d60F4tJyirS+vc789bDBY+w8fRmv/rgf+SbMwKkpNSMPA2ZvltxtfepSkexzWGohwBdW7MVbEl4zJfgp5Tx+239RUi6OLpYeViguq8DoL7bh/d8t0xNGRA0HgxOJvt1+Fl1mbsSmI9mS96k5RFJdan3Lccv2AhzN1E5C/ClF/qyhR77eiR9TziP+D/N8k3562W5cyLuBt9Yesvj1m9vGw7rfb13TtW//OFFnYuOK5HTc8/k2i+e15N2wzMKT5vLHoUzsP5+Pb7adsXVTGhQhBE5kF1o1H8wc7GkqfHlFpdGJ85Zy9VopDl1oeEPN1RicyPTWr9LHhcsrBcorKnEsy/Cshaz8Yqw7kGn2D5iFf58yvJEe1cMqukjpqTieVYgr10pRUnYrl2NB4qmb+0tvhxI/xP44VLcuS/qV6/hOx7DZjDUHcfBCPj7ZWDe5WaoiA0nB3+08hwVbjH+vraF6+IjM6/cDmRg2byvGfb3T4LbFZRW45/NtVqse3VA8+10KBs3ZgnUHzFuPyRRRHybg7s+3YW/6VVs3xSKMWpWYpLnj40Q4qVXIlrDcfb/4WxVWH+7dwpLNsorjWYUY/mlVDRAvN9v9mh2/WbultLwSsUuTEdmqCeLu7KBz2/wbZWjSyBUqCdFQbqHu97S+0h7XJRZl03UIXVO8a06T1pdMKYSASqXC/206if3n8/D145FwdpL3naSiUtjVass/p5zH/yWcxDcTe6NDoJetm2OyLcdy8OeRLMy6pyvcXZzqPL8iuSrHaG96nsFj/W//RRy8kI+DF/Lx+l2dzd1UxdmWlouvtp7G7DHdER7qa/RxqvPevtl2GqN6BJupdaapngix9cQl9GrpZ+PWmB97Tiwot6hUUmBS2+o9phVwM5cdpy4DqLoxXr4mb8hg5+nLlmiSbNXDWhsOZ2F72mV8lnASm49l65yZc8cnf+OFFfs0/9+Rlmu1dtZ0Iss8idMX825gwOzNWJB4CvM2ncDmYznYfPNDtqJSSMqhyikoRtvX12Payn0Wy+sxt5d/3I/0K9fx8ur9tm6KWcQu3Y0VyRlYbIbhMEdbcf2nlPM4mlmAxxfvMvoYNXPvHO31syUGJ0ay5do51pR/vQwDZm/GgNmbZe13Jvea2dog95W+pKNXo2Yw8uTSPfh00wmU6xhm+L1Gt+2j3xj/gSZXzSRgfdO55fr4z+O4mF+M/2w4pnms5ObrMOqzf9Bl5kasr7VswMxfDyG1RtJ23w+revR+Tb2I+7/cYVdVX01ZjkGJ7HkRu9yiEjy8MAlrTKycbayCYuMnM9TM5WJwYj0MTmTKzC/G+78fMdsNxBaSz1zBwfP5kuqV/GvlPoPb6LJ0x9l6n8+WkRxaPcpS3zf3tJyqm/sfBzPR54NNBo+5Zu8FvPrTAYPbWVpFpUBhcRmG1ih+Vy7hA3Dh36c0Q1b66AskLheV4NjN3pmXVmn3Lvw36Rzum79d536pGXnItGDhQUsqLC7DU0t3G7WcQGpGHvp9mGByBWZHNmfDcSSfvYK4BtKbJVdqRh6W7TiLX/adxyd/HrebXkhbYs6JEex9tsHDXyVJ3vbvE5dMPl+hjmTOjzYcl32c+v6e7/1iO46+NwIf/yntuJas7ivHPZ9vw5FM+WXeZ/9xzPBGepiy7tOJ7EIkn7mM0vJKNHJzxt09Qow6zt70q+gY6IVGVspH+jLxFBKO5SDhWA7u69m8zvMZV67j2+1n8dTA1mju66H13OTlKcgqKMa0lakYHVF3X3tiq+TyAh01hJSsOlfLXD2FtQP+29r5I6pNU7Mcu6FicGIDl4tKcCK7CP3aNIFKpdJa3M8Y1v68OX/1htbwh6VJGUKzVk9W/o0y+Hi4YM5G+cGVLsYEJqbYffaKUXVkqsV+q13xtWZwkn+9DKdy6x67sLgML61KRXGNWVtjvtyBriHeWPevgUa3RQ5DyzqMW7QT56/ewPa0XGx8aZDWc5aqiEzKdP7qddz/5Q5cKixByyae+HpCpOR9C4vL8FPKeYzsFowgH3e925myirujYHBiZZWVApHvVw07PNy7BT56MBy/7de/gJwSjVtkeMqiXjbozbz7839w6IJ5goDwd/5E7IAwXKtnmQBL+f3ARVk9FboCtv8mWa7Q2tC5icgtqhsELEg8hU1H69a3UdLCgOevVuVzGBoqo4bvkz9PaPLW0q9cx596ah3p8ubaQ/g19SIWbzuDbdPvsFQTHQJzTqzss81pmn+v3nMez3+f4lDjsMlnr8jeJ6ugGAlHpX9A1GauwKTat9vP6n1u7l8nDNYkMdbUH/bJSsgrNCEJUKrTl4qw/mAm3lx7UGdgAgBXTViM0hYqKoXeYQhb1pSoFFV1SkxhbCK/EAKbj2Ujo57aR+aWU1iMn1POm3zNclXWGj+u+YoZGharng1XHeyS8Ric2Nj6g7ZbQFCpdNUZeWrZHvzvgH30MH299bTFjj3+GxN6rSzgjk/+xvPf78XyndLX87E1Q8OoY77cjh5v/4nzV+veiMd8ucNSzTLoh13p6PTWBq1FPw0RQmDP2Ss6142SY8vxHDy5dA8GfrTFpOPIcf/8HXj5x/2Y+5fxxQvJfjE4aQAaWt73qUtFiP02uc7j01amGtxXSgE1KUxJhMs30FOQeDwHP+7JMOrYO0/L73kyRk5hMab+sNcsx1qRrP9aX/nR+r2GXWdtrHfxyv03V5/+w0JfHC4VlpiUILpJRi/iiuQMPLgwCaP1zMCSatcZ8//eGerFqS48KGfJEFMxv0g5GJyQ1Rnqpj13+Tq2HDd9lpApSk34kEox0PX/xLe78epPBxRTqE6XN345ZJWk5+oiefP+OoEvE9PqPH829xq2nTRvMbyS8kpJq3PnFBbju6SzZh2my79Rhj4fbEKPt/802zH1+XrrKbz+S9UikKcv3aw7pKDyTPvP59m6CVpWJKej/Rt/4C8JwZApi8BWsc5XyoLiMqzenWG2RVyticEJWZ09LFb1oxELJlbTl+Pyv/0XtQqDPSJhLRR9xzFk64lLkuql6GPN3IJfUy/g/xJO4qMNx+usLTXk40Q8tngXDhhxI7uYdwMPLdyBDTrWQZJi0T9n8Navh7VmGZnK0EypPw8b7q05m3sNU77fi8MX6/87+nC98dPNTVFWUVmn51FXGQBzzXgzlxlrqgK567WS3a/U6glNOnUZXWZuxDv/U/5q5S+tTMW/fz6A539IsXVTZGNw0gCUmPHD0xoeXCi9zopc1cXYTBH22jq9a9WY6ostdXsH5KpZYl+fCUuSkWyBrnhLqDlcp6+WjTFJzYUl5dh99iqeW26e4SlreOa7FIPv25eJp7DuYCZGf2HaUM1JC8xMKi6rQJ8PNuGeL7aZ/di2UjsBfvbN1drrS4wHlFFNtnpNoO1pyu2l1YfBSQPQ90PDFVEdgT2UVpfSZawEx8y0vo+5vP7LQeQWyV+nyh5JLZJoTM9YzVGdO+dtNesyE0BVr2je9TLNNHFzVEI9n3cDhy/m46MNxzRDbGUVlXj7t8MmzeKziBqX+8Vm07+IODIGJw0AKyHbj6IS+xv7VYrqbndjNOS/kX0SViPWZ+85y02NfmlVKobN21pnqE6u0vJKjPpsG75MPIWPbq4TtSI5HUt3nMVTy/bgWkm5rF6KjCvX8dbaQzhr5sCstnmbTph87VIcOK/8YXJjMDihBqOiId+BCKdyirDtZK7Zep82Hs7CB+uOWLT7fW/6VRyxcLE5pa7z9cu+CziZU4R/TpgvoXn1ngzM35KGc5dv5UR1nbURPd7eiBMSh6lil+7GdzvPGZ3zJcfj39SddQhUzaI6fDEf983frun9OXg+H2GvrZO9/lN1bZXa7KEnuT4MTqjB+HGPbVY8lSMzTxlr+ihVffGlAPDY4l2Y9N89Jg/xCADPfpeCRf+ckZRgrMvpS0WImfs3ftmn+/fuyrVSjPlyB+767B8TWlq/uNWpkrfdeDjLbFPt5TDnKYvLKjFn43EsrrW+2bXSCry/7qikY1TnpVWvr/XNP6fxicQ1uQzlPtVeR0xf0cm/T1zCqM+2ITUjD08t24PM/BuaPJ0XV6VKaoshJXa+KjeDE2owLuYpvyqjKTNoHEFKPcMMNfMX8sw4NTJb5iKQ2QXFGD5vK+745G+k5RTVWdm5Wo6OlbdLZd4wcgy0bc1e6d+yn/3O8jM29P12CxlTZ62dSPr+uqP4XEZ+SM3W3TDTMhY/S5gdmHA0G59uOoGN9czoyi0qwaYj2TpfQyEEViSn1/s3piQMTqjB2JdhH390pF9phfwP+6+3nsKsXw8ZnXwZL3OF5//8cUzSGjyJOmr1TFisu5tfH7k1Vqpfgy82n5S1nxxbjuXovfm99/sRk4599VopIt//C3Fm6j2wtM4zN1js2EIIJBytWjIgp7AYTy3bg083ncSz36XorF4MAHfO/RtP/3cPfkiuW7F5e9plzFhzEA8s2IFlO85qPZdx5Xq9hQltgcEJNRj2OF2OpNM3JPHh+mNYlnTO7Gso6SM1x2O2jqBHbnE/uT1tF/Ju4PzV6/j4T90l3+sbYblcVIIdabkGg7zYpbuxfKf2ApKXi0rwwbqjepMzpcaNP6WcR971MqyRmXdhTdZIQN11+jK2nszFU8uqlgyovWTBBT1r91SvdqxrFtPpGiuGz/rtVo2WguIyDPxoC/rFJ5ij6WbD4ISIFKn2cIuhaa95N/QvG2DuKbNyLd52ps5Nf8OhLIPDPMPmbZV1HiGMH2oYPCcRj36zCxsOGS4EV71NSXkFtp3MxaCPtuCbWnkgZLyxX+9ESj2LpI6tlcxbVFJudM+hUofDGZwQkWKUVQgIIfBlYhqiPpT3TW7Wr/ordh6sVZVY3zdPY10vLccH647oHc9/7/cj2JamPWvlueUpmLdJOYvaVQ8hfbfzHHbVXlpBz33vrbWH8NjiXbhmptwLR1Vz9pExus3aiJdrrG5/9Vop8m9o52XNrOfvQ4kYnBCRYjz7XQru+WIbPtogv7T56dxrOG2gPHy134ycoQPoHqL4v00nseifM3hggf5VizOu1A2IjJ0pZKyXJSy0uOPUZYz9eie+qzV0U1N1gutqiTPk9CUdS+mlUboP1x81ee0aU5bLqFZzKGz/+XyDwzQZV64ruoo0gxMiUhRTckfu+ORvM7ZEN131dGoum7AjzfS6HsZOlf5g3VFkykxsLC2vxHwdyyqYcwmH/XryNJ5bbr4ZRNsNvO6/7DuPuRKnDMvx9dbTCH/3T3y99ZRZj/tZjRlERzPNn0818KMtePirJBzLVFY16GoMTojIbs36VdoN1Jz1+QwVgXv0m10ml23v/f4moxZf3HA4CxOWyJsR9Obag3oX4ZtsxuDB0ioqBXaevowyPUnHL63ar3XDBwyvkC6HJRda1Ddd3RyUuhCrs60bQERkrGVJ+ocearJ0FdUEPVU6DTlfT+7Ln1ZYh6mopLzeoZk/9Ay7CGGedXPMTW7VV2sEX+sPGrcqtqNjcEJEZAVy11kxtWaIFDWnlOoz7uud2F1r5siuM1dwp8yZREq0RUctGnNbwllMRmFwQkR2Z8n2M5g8uK2tm6GXriJl7/zP8sGGJSTVnrlzU808G1Ms3X4GlQLo2dIXfxxiLwNVYXBCRHbnh13p+N3KM13ksEavR0Pxtp0GbWRZTIglIrtUUCyvtLs1Sc2FcQQ2WGuQZCiWOdxoLQxOiIjIYhpCLRNT1Jf0rATLd9Zdh0cJGJwQEZHFSC3U1lBlyVz1mqowOCEiIiJFYXBCREREisLghIiIiJAlc+kDS2JwQkRERHoXaLQFBidERESkKAxOiIiISFEYnBAREZGiCuYxOCEiIiJFYXBCREREisLghIiIiKCCcsZ1jApO5s+fj7CwMLi7uyMqKgrJycmS9lu5ciVUKhXuu+8+Y05LREREDkB2cLJq1SrExcVh1qxZ2Lt3L8LDwzF8+HDk5OTUu9/Zs2fxyiuvYODAgUY3loiIiBo+2cHJ3LlzMWnSJMTGxqJLly5YuHAhPD09sWTJEr37VFRUYPz48XjnnXfQpk0bkxpMREREDZus4KS0tBQpKSmIiYm5dQC1GjExMUhKStK737vvvouAgAA89dRTks5TUlKCgoICrR8iIiJyDLKCk9zcXFRUVCAwMFDr8cDAQGRlZencZ9u2bVi8eDEWLVok+Tzx8fHw8fHR/ISGhsppJhEREcnkMHVOCgsL8fjjj2PRokXw9/eXvN+MGTOQn5+v+cnIyLBgK4mIiEhJnOVs7O/vDycnJ2RnZ2s9np2djaCgoDrbnzp1CmfPnsU999yjeayysrLqxM7OOH78ONq2bVtnPzc3N7i5uclpGhERETUQsnpOXF1dERkZiYSEBM1jlZWVSEhIQHR0dJ3tO3XqhIMHDyI1NVXzc++99+L2229Hamoqh2uIiIioDlk9JwAQFxeHiRMnonfv3ujbty8+/fRTXLt2DbGxsQCACRMmoHnz5oiPj4e7uzu6deumtb+vry8A1HmciIiICDAiOBk7diwuXbqEmTNnIisrCxEREdiwYYMmSTY9PR1qNQvPEhERkXFUQghh60YYUlBQAB8fH+Tn58Pb29tsxw17bZ3ZjkVERGTPXr6zA14Y2t6sxzT2/s0uDiIiIsL5qzds3QQNBidERETkOHVOiIiIiORicEJERESKwuCEiIiIOKxDREREpA+DEyIiIlIUBidERESkKAxOiIiISFEYnBAREZGiMDghIiIiAMqZrsPghIiIiBSFwQkREREpCoMTIiIiUhQGJ0RERKQoDE6IiIiI5euJiIhIWYSwdQtuYXBCREREisLghIiIiDisQ0RERKQPgxMiIiJSFAYnREREpCgMToiIiEhRGJwQERGRojA4ISIiIkVhcEJEREQswkZERESkD4MTIiIiYhE2IiIiIn0YnBAREZGiMDghIiIiRWFwQkRERFBQygmDEyIiIlIWBidERESkKAxOiIiICAqqwcbghIiIiJSFwQkRERExIZaIiIhIHwYnREREpCgMToiIiIgJsURERET6MDghIiIiRWFwQkRERBAKGtdhcEJERESKwuCEiIiIFIXBCREREUGloCpsDE6IiIhIURicEBEREcvXExEREenD4ISIiIgUhcEJERERKQqDEyIiIuLaOkRERET6MDghIiIiRWFwQkRERPY/lXj+/PkICwuDu7s7oqKikJycrHfbNWvWoHfv3vD19UWjRo0QERGB7777zugGExERUcMmOzhZtWoV4uLiMGvWLOzduxfh4eEYPnw4cnJydG7fpEkTvPHGG0hKSsKBAwcQGxuL2NhYbNy40eTGExERUcMjOziZO3cuJk2ahNjYWHTp0gULFy6Ep6cnlixZonP7IUOG4P7770fnzp3Rtm1bTJs2DT169MC2bdtMbjwRERE1PLKCk9LSUqSkpCAmJubWAdRqxMTEICkpyeD+QggkJCTg+PHjGDRokN7tSkpKUFBQoPVDREREjkFWcJKbm4uKigoEBgZqPR4YGIisrCy9++Xn56Nx48ZwdXXFqFGj8Pnnn+POO+/Uu318fDx8fHw0P6GhoXKaKdnA9v4WOS4REREZzyqzdby8vJCamordu3fjgw8+QFxcHBITE/VuP2PGDOTn52t+MjIyLNKuLx7tZZHjEhER2RslFWFzlrOxv78/nJyckJ2drfV4dnY2goKC9O6nVqvRrl07AEBERASOHj2K+Ph4DBkyROf2bm5ucHNzk9M0o/h4uFj8HERERCSPrJ4TV1dXREZGIiEhQfNYZWUlEhISEB0dLfk4lZWVKCkpkXNqIiIisiAl1TmR1XMCAHFxcZg4cSJ69+6Nvn374tNPP8W1a9cQGxsLAJgwYQKaN2+O+Ph4AFX5I71790bbtm1RUlKC9evX47vvvsOCBQvMeyVERERkNJWCohPZwcnYsWNx6dIlzJw5E1lZWYiIiMCGDRs0SbLp6elQq291yFy7dg3PP/88zp8/Dw8PD3Tq1AnLly/H2LFjzXcVRERE1GCohBBKyoHRqaCgAD4+PsjPz4e3t7dZjx322jqzHo+IiMgePdavJd6/r7tZj2ns/Ztr6xAREZGiMDghIiIiRWFwQkRERIrC4ISIiIigpAxUBidERESkKAxOiIiISFEYnBAREZGiirAxOCEiIiJFYXBCREREisLghIiIiBSFwQkREREpCoMTIiIiUhSHD07euberrZtARERENTh8cDKxf5itm0BEREQ1OHxwQkRERECAl7utm6DB4ISIiIjQ3NfD1k3QYHBCREREisLghIiIiBSFwQkRERFB2LoBNTA4ISIiIkVhcEJEREQQQjl9JwxOamkX0NjWTSAiInJoDE5qWTtlgK2bQERE5NAYnNTS2M3Z1k0gIiJyaAxOiIiICCqVytZN0GBwQkRERIrC4ISIiIgUhcEJERERQTmDOgxOiIiISGEYnBARERHL1yvVkwNa27oJREREDo/BCRERESkKgxMiIiJSFAYnRERExNk6RERERPowONHh2cFtbN0EIiIih8XgRIfXRnSydROIiIgcFoMTHVQqFQK93WzdDCIiIofE4ISIiIhYhM0efDk+0tZNICIickgMTvSIbOVn6yYQERE5JAYnANo0awQAuDs8WOvxyUPa2qI5REREDs3Z1g1QgvX/GoicghK0bOqp9fijfVtiQeIpG7WKiIjIeliETWHcXZzqBCZERESORKWg6ITBST2U9EYRERE5CgYnREREpCgMToiIiEhRGJwQERERhIKqsDE4qYd/Y5awJyIisjYGJ/Vwd3HCzhlDGaQQERFZEYMTA4J83NHMS3pwMrxrYL3Pz3043NQmERERNWgMTszs1eGd6n3exUn6S97I1cnU5hAREUmipPIZDE7MrF1AY73P9W7lh+FdgyQf653R3Yxqw+AOzYzaj4iISAkYnFjQA71aaP3/p8n94eos/SU3Nohd9mRfI/ckIiKyPQYnMsXd2UHvc61qlcAf1zcUn46NAGC9XJO+YU2sch4iIiJLMSo4mT9/PsLCwuDu7o6oqCgkJyfr3XbRokUYOHAg/Pz84Ofnh5iYmHq3V6LqYRJvd2f8a2h7ndvEj+mOjS8O0nosyMcd9/VsjhPvj8SYWr0oRERESmLXdU5WrVqFuLg4zJo1C3v37kV4eDiGDx+OnJwcndsnJiZi3Lhx2LJlC5KSkhAaGophw4bhwoULJjfeWl6MaY/4Md2xoVbwUVOIrwfcXaoSWNdOGYClsX3Qwq+qJ6X2UE6AjNk/REREjsZZ7g5z587FpEmTEBsbCwBYuHAh1q1bhyVLluC1116rs/3333+v9f9vvvkGP//8MxISEjBhwgQjm21d7i5OGNe3peTtI0J9631+2/Q7cKOsAkUl5UjLKcLEJbp7ksL8uVIyERE5Hlk9J6WlpUhJSUFMTMytA6jViImJQVJSkqRjXL9+HWVlZWjSxHFzI1yd1fDxcEFzX496Z9a4Ojnh2HsjrNiyKn1bO+57Q0TkqJQ0lVhWz0lubi4qKioQGKhdaCwwMBDHjh2TdIzp06cjJCREK8CpraSkBCUlJZr/FxQUyGlmg1I9VEREROQorDpbZ/bs2Vi5ciV++eUXuLu7690uPj4ePj4+mp/Q0FArtpIUFDzjsX7Sh9OIiMh4Suo5kRWc+Pv7w8nJCdnZ2VqPZ2dnIyio/uJiH3/8MWbPno0///wTPXr0qHfbGTNmID8/X/OTkZEhp5mkAGunDDD5GK7Oarx7r3GF6IiIyH7JCk5cXV0RGRmJhIQEzWOVlZVISEhAdHS03v0++ugjvPfee9iwYQN69+5t8Dxubm7w9vbW+qFbpt7eztZNMMhQUrAUYU09oVYrKJS/aVzfUIzp2dzWzSAiarBkD+vExcVh0aJFWLZsGY4ePYrJkyfj2rVrmtk7EyZMwIwZMzTb/+c//8Fbb72FJUuWICwsDFlZWcjKykJRUZH5rkIBGrtZLzfkleEdsefNmDo3SCe1CjPv6QIAeP8+2/U4hDbxsPo5W/s3AgAsGN/L4ueKH9MDYTfPR0RE5id7KvHYsWNx6dIlzJw5E1lZWYiIiMCGDRs0SbLp6elQq2/FPAsWLEBpaSkefPBBrePMmjULb7/9tmmtV5BeLf2sej7/xnVrpRx5dzjcnKuCpPt6Nsebaw8ZdWxT6/A83q+ViUeQZ8Wkfohs5YeLeTcYNBARGUlJRdhkBycAMHXqVEydOlXnc4mJiVr/P3v2rDGnsCtPDmgNlQUziZ4d3AZf/X3a4HbONYLCxm5139oWfh44f/WGWdumS3WAZC3BPu5wdVYzMDGDidGtsCzpnK2bQUQOjmvr2IGXYvSv5yNHHyutu/Nwb86uMkb7ela0lsvNWY3JQ9rK3u/BSMu9d22aMXgkImkYnBhhQLumVj2fu4sThnYKMLidUlJHPVyl9ZzcEx5i4ZbYl1XP6k8qN0YzHUN/hlh7KuGrwzta94REZBcYnBjh68d7479P9tX8v2dLX9s1xo51CzHPLKzaq0Eb8trITmY5r9KN79cSD0Uqe8HJ0REMUImoLgYnRmjk5oxBHZphyytD8H+PRODuHsG2bpJO3z8dhbfv6YKHIlvgvdFdJe83oK2/BVt1S8sm9QcVKgl9QW2aNZKd7/PcYPnDHbVNiLZO0u/iiYan3uvj5uyEOQ+Fm7E1RNSQ2W0RNtLW2r8RRkc0t2gyrCkGtPPHEwNaY85D4Xg8OkzyfmN6WaeGx4huQXiif5js/b5+PFLzb1u98r6erlY5z9DOgYY3UpB29eTNKPOvhIiUiMEJ1WGtWEulUuH5203vxWgoPExYRylpxh2IUsCCjc4KLJpHRPaHwQkZrWbejb0ZH6W8NXs8XJ3ww9NRkrbd/Yb2wpnBPh54d7T9lfq3RK9j/7bWTVg3lzskJL3bEx8PF1s3gewYgxM71taMU0+NEd7CFx0C9bdhXF/DAUDTRvJnlEidDaRL9UyrD+7vbvQxLKl/O2n5Ps286r5uATUec7JiD8bhd4Zb7VxS/DCpn62bYJSG1ufUo4WPrZtAMhnKA7QmBid24vGbCZg1vxU+dVtrDOtiXE7CwPb+Zln/ZvlTUXhzVGdMG9q+znOzbpbSr4+cm2hoEw/cFxFidMLujJGd8Pm4W+Xtk98Yih+fM+/03Wrmqk1Tn9rDQH6NbuXBGHpdW/jpXmLA1/PWt12p3+Qb6Sj4Z29q5jGRecx50LzJ2I1M+FJC0kSEWrfSeX0YnNiJIR0DsG367VpDKe4uTrLqRMTd2QFe7s6Yens7fPdUFPwbm57UGeDtjqcHttHZhetuQg6FLs8NbotPH+lp9GKAzw5uiyY1buABXu6yC9NJnbY8LaZusKYkgd7uOh9v4eeJjx7sgYWPRcLLXX7QUd8wjVITx8kymprh86Umfx29hdRwMThRsNoVNVv4ecLZSf9bZuizP7SJJ/bPHIZXDAQ0NcvgS6WgJRks6o9pA23dBIt7uHcoRnQLsto6G1teGQIXp4YTuHjyG75FBPvoDqgbqshWfg6dYM7gRMEs0V1uqNchqnUTBPm4w93Fsr8aj/TRXSb9sX7WT1Rd+FgvyT0inq6WH8II9Lb8N8SavwU1h3JsobV/I/RrY54kVn3DVZaw8cVBOh+faMT0eKqfp6sTuoU4Vg6Ll7uzwS+S5qakWIjBCWmpThTd/PIQwxsb8Ys89fZ2+Om5aLx3n+6ZJe/fdytR1a+R9k1TSlE2AJg+Ql4F2BHdgvH3q7fL2seSYge01vm4l7szvN2dJS1lIKcrq7mvtBv6Q5EtsNrMJfbNrauZqg5L0VjPsJex33bdLPyFwJ456npd1o4VlDT0yr8G0inE1wN9wqQnR0n9lXZ2UqF3WBO41DM8tXhib/Rt3cTohLrJQ9qaVDPEkqTk+eh7bRq7OWPvW3fiGxOqxhrr+PsjMOehcPRt3QRdgq0XANTHnAsl2lqHwMZ4Y5ThBHJHNrhjM1s3weJqDwkqKFawOgYnpEXXFFVrG9o5EKufjUaoCdPalPpHPbC94Q9YUU+yh7OTWvPtxprX6OYsPdj7dGyEzscf76e/5L/c/JbH+rXEX3GD5e1khJqLU9Ye/jLHy//OvV0x/9Fe+POlwZJ7sByVlL8de1fflzZHw1fCgdW+Iaz/10CtWTfWSogk63h3dFcM6tAMn43radHztPZvpPNxXesRGXuDlzrEZ6qavYd/v2Lc0N/zQ/RXQZ7YPwyjLLw2V31BYUMnp/e3mi0Tmmv2rI7sFmSzdigBgxPS6GLF8XqSz9TbsY+HC/77ZF/cG669EvCYXlUrF3cK8jLxDPq19pe/QKMSPNq3JV6MaY+fnouGj6cLdr0+VNb+rZp64t8yc6DM7V0Zi36+Msx89Xls/W4HeLlh5TPRCJE5y8eUdsvNd6vtq8cj0THQC1Nvb4eHIh0zz6aa/VdPaoDev68bujeXlplec2qxEModzjCHhnxttvRE/zB0DvJCdwtW9DR3zRtjBHm7I6ugWO/zf0wbiLs++0erx9DZSY0XaxTUs6eCcysm9UO0zFL+XRvYjBgntQrxD/TAxCXJVjmfq3P93/fbBTRGWk5RPc97YeNLumeBGaO5rwcu5N0w2/GsiT0nCrApTvuX8bF+rRAusXprWFNPPNy7BZ4Z1Mbo4mRSNGnkih8maa/7ouRg4cHIFrZugkZYjWnKUkr6W5uTWoX+7fzh5W6ZKcVtmzXC/z0SYZFj16ZvSvLLd3bA5lfqz1HpHOyNM/GjTG4Dh0OlWfKE9RO7ba1XS1+rns/ZjusHMThRgHYBXuhs5AwIlUqFjx4Mx+t3dTZzq7Rt/fftaNvs1uyI/m2bwlvCzcxWK+W+fldnLHysl+ENLWjtlAH4cnwvrHwmGhGhvnhzVGf0VcDKwYBpgeVYPTVqqtW+Nye8PAQdAi0zZFS9BINaBXzyULje/IqeLf2sUqPGERm7iGZjN/MGw8ue7AtXMySU9mjha3pjzMTU3CopS4goFYMThahvhoYSNK7Vnb3gMWlrkdiqPoG7ixOGd7VtQllEqC/u6h6MIB93rJ0yAE8PbKP1fM1ESWPyPWIHhAEAYjpLWwPHXAXKHu/XCl43fx9svd7J149HYurt7bD137fjgcgW9VZQrk+vVvITJ639F1vft25DM32evk137Zz63NVd2t+PlMqto7oH49cpA7QeM3fPq39jVxx4e1idx+XWnXnvvq6YXE8Sc7XaibPzH7XtlyFd7ugUiLftNEBhcEIWxVLe+pn64fxSTAeseqYfvqjxoVhdhKzmDJAfno7C+/d1Q2Qr8/TaqNUqJL46BM8Nbov/vXCbWY5p7GsR4O2OV4Z3RAs/46edxw4Ig39jaVPoldpJ/tzgNvU+/8aozvWuIK7LPD1TwmuTmugsdajaWCqozJLb5O3hopXYqq/dta9aqXVYPO0oT6omBiek1xujqoaKpt7ezuC2+r493dklUFpFUwnk3hhUKhWeG2z4G5AufcOaoFVTT606F9YitRPN2UmNqDZNtT6Qlz8VhXljwzHrnlszNPq388djZp5O2rSxG14b2QltmpmnEFrNa17+VJT+DY1UvU7Vs4OqbuJP1qjC27SReReok+qTh8y7am99VCoVfD3lXadapTKqx6XmOet93ugjm+7lO/XPSqrdSyynnUoNXu2RfYZUZBU9W/rhxPsjDWagA8DwrkGYens79KzV9ezspMbiJ/og7LV1Zm3b4/1a4bud5wxO0wwwoqhcG/9GWP3crTLt/9t/Ued2EaG+SM3Iw2sjO+GzhJO4Xloh+1y6CBMGDPwaueL+nspJBjZEVyXf29r7m/08ITeHPaaP6IT7ezVHhwAvLNl+RvZxpLwzhjoSxvVtiVn3dLH4DKbaN1kpav6tW+JGu+b5/hjz5Q6zHc9ZrUJ5pfy/l946ViNf83x/CGGd9bN0YS+zNgYnDkzKN6maH1Y1k81qj+Oq1SqrLlL17uiuiLuzA/xs9K0XAFY92w/nr95A22aNMWlgG7R9fb3N2qIkUvKnGrk64c27uxj9/j1l5Dd6tVqFTkHmqeej7+b95IDW+GFXOi5fK8Vt7bQDraGdAvDh/d0kDYU80icUf5+4hMx8/dOfq9Vc52fm3V2QU1hi1NTwQG/Lrvzbq6X83B5Af8CnVqsAI4ITXfS1TeqQo77NRnYLwh+HsuqsMi/1/I6KwYkDe21kJ/y89zyAqumkhvg1csWrwzvCWa2yeb0HlUol6cbWw5jaHRI/jNycnTQzmKS8fkpirQqr+jwY2cLoadWvDu+IMD1VaG2tfUBj+DVyxf9euA1/HcnGQ721e7EauTlLztF4YWh7DOnYDM8t3wugqnz73vQ8ndt6ubngmwm94eykwpCOeoZRDdzDx/RqLqld5mKLUgSrn41GiK870q9cl7xP52Bv7NPzuktxX8/meHpgG3QIbIyEozlYvee85H2VXK7B0hicODBj1tGZIiH/REl6hzXBkid6I6yp5W9mD/dugdV7zuNfd1jmNTJrhVUJhzLm+2iTRpaplVKTOaaLWkJEqC8+fqgHgKphpIn9w0w+ZscavTzP394WIb7umP7zwTrbCVTld0nVr00TDGjrjzD/RnhhxT6920npfZH0a6ljG3P8PncK8sKB8/k6n2uuY3Za9VR+KcHJun/dhv/tz8SU29vih13pRrdRBSDy5myw0REh8HB1QvfmPug/e7PRx9Tn1eEdcfVaKb7ZZnjI0t1FjWFdgvCbnmFrW2NwQgDsI5HL2M+yOzpJ/9A2RfyYHpjYPwydzTRsUJuSp5svmtAbi/45jdljeiC3qMRsx/3PA9113oyVaG2tqbKmEkKgtX8j/PRcNJo2doObsxPG9mlpltdjxaR+muCgvuDknh4huFFagdfWyD+nvj/XB3q1QMbV6wg3sZ7II31C8cLQ9hhQ6ya/+tlofL31lCYp3Ng/m64hPmavmKtSqSxa4qBDoBeCfdz1BievDOuASgHkFpVg0sA2mLPxuMXaYioGJwTA8t2HoU08kHHlhs1rj1iSk1pllg8zBccget3ZJVDzzb1mcPLW3eavsWBKwrA90pW8aSqpvRZqtQqP9G2JzzenyS6Drq9i9ScPmz5L6bnBbfHaSN3J8H1bNzF7scNvn+iDV37cj+i2TfH7gUwAwPBuQViz94LZzmGOz2BDX2Cm3tFee3vTT2kxyuwfpQbnr5cGY/trdxhdCZfsk7GJq/VxVpv3Y8uU4YUXY6o+7O210JUl/PJ8/zpLckhR3+rMtWcMGkouNUTu0ODtnQKw580YvFRjCnLN6fpkfuw5Iatwd3EyWMVSqYJ93JGZX2zxIlINhZRvY6Z8Y1PSeiEvxnTA+KhWRuVvKZPp36V7GjnrxKmeIHFyrXpFpv4G9Grph5jOAdh0NEfyPrWDWGtM/VWbpTvF9EPYAntOiAxY/Ww0nh3UBl8/Lq1kP8mnhFWLjdVwAhNtUnuULD3z6737uskuIGeIWq3CNxP7aBJVlerhPqFo499Iq2igo2DPCdncPeEh+N/+i+gTVv8HhaU/BD1dnXC9tKLOyrahTTwxw8ILK9Zkp190TDLrni7IuHJdMwxU3Vs1sL0yS4LTLY483bUmFyc1Isy86nBjN2dsfmUIAGDXmcs4fLEAya8PRdLpyziVU4TPNqeZ9XxKwuCEbG72mO4Y0qEZhkpcwM5SNkwbhI2Hs/Cokausmos9JsSaKrSJJza+dCtPIfHVISgqLkdTiWveWEvN4oNSKiebQk5ehJJncpmqdiE7JalZodbVWY1eLf2w/KkotGziiUFztpj1XL9NvQ3FZRVo5OaM0RFVNWl0BScNJVjksI5CvHNvVXLVSzH613ywJFsW5Wrk5owHIluYvetWrpZNPTFpUBubF5izBqV/frk5OykuMAGqhp/euKszXh3eUfJigcCtG8aAdk3r37CGADNXa9U3y6m6+Nozg+pfPFCKFZP6GbWfrrVuxvUNxbbpt6O1iQX3mli5ivRt7f3RsqnxC1Hq46SA4pfW5DhXqnBRbZri+Psj4OZsv2PvJE31mjz392yO+VtO2bo5ZmfpImm366uAaiWTjLiJ73jtDqSm55llKv226bfj19SLZqtR8clD4fjgvu7wcHVCWk6hSceKbis9+KqpR6gvTuYUaT3m4eKsd7VpOTOsuoR44/W7OmnWVyL7wOBEQRiYOIafJ/dHwY0ym64LZEldQ7wxqkew2Wdnje0dildHyOuxUIpgHw8EdzfP69HCzxPDuwaZLThRqVTwMGHmibl64aTEG+OjWmLn6csY1b3utGO/eqoTPzPIuNXJ7U2/tk2RIaM0v5IxOCGyMid1/esC6et+N2v5egtSqVSY/2gvsx/X1Vltl4FJQxZhpun1UnNmPri/O4QQWn8Liyf2RmFxOYJ92DPi7W755SOshTknZD/s495M9QjxqcqjGNFNuZWC37+vGwDorUDakLX2b4w2zRohPNQXUtayXDyxt+UbVUvtIH1o50Dc19O6ixZWG9KxajaZqXkxxhrYvm6ysJzaKN1ClFsUkz0nRGQ1f8UNxoW8G+gQ6GXrpuj1WL9WuLtHsE0StJ3UKlRUGu5FCPC2TA+Sk1qFv14aDLVKf0/dh/d3x9dbT2FpbF9FJi1b0ycPRWD1ngzcEx4ie9/PxvVEanoelmw3vEifPsti++KftFxM/WEvpo+oCqY7Bnqhdys/TS9jfcs9xA5ojfg/jgEAbu+orGn7DE6oCnslHErPln7YdeaK1c/byM1Z0YFJNVvNHFv5TD+8+uN+vH1v/aXRvd1d8NdLgzBjzUEUFJdhsBlvLE4GukwejWqpNd1eyhf16h6zhsbH08WoBGkAuDc8BPeGh2iCE2OGbdVqFQZ3aIb9M4dp1jJSq1X4aXJ/SfvXnA5vlmq0ZsTghEhhrPEhMW1oe3h7OOPOzvpXbHaW0q9PZtUnrAkSX71d0rbtA73w0+T+dXIwLKG+nJD6yhAsje2D3w9k4l9D2+vdRu85HawcoSnvoL5FFu0ZgxNSPD9PF1y9XoYBCi7GZE6PRrXEiuT0OtNOzVloy8PVCc8PaVfvNq8M74jkM1cwIbqV2c5rClNmlDRkUgOTzsHe2H32qoVbo21IxwAMkTj1u6HdXn09XZB3vUzxJfKVisEJKV7SjKEoKC5DgFfD7BquzdvdBYmvDLH57Jzmvh7Y/todNm0DUFWg8Jd9F/D8EMeYDmop/x7RCZ6uzri7ntV/baVriA/W7rtg9fNaskDbzhlDcaO0osGWDLA0BiekeO4uTna9MJwUtWuC2DowUZKJ/cMwsX+YxY5vKMeioWjs5qy4GUj7Zw3DtZJymy2e+N7obrhRWmGR3y9H+NyyJE4ldnDV0ya/tEBdCjLsh0lRGNktCPFjutu6KQ7nhTvaoUNgY4y38VpKjszHw8WmlVuDfNyx/Oko3NlFf+4V2QZ7ThzcY/1aYWyfULhYuOQ46da/rT/6t3WMXBqleXlYR7w8rKOtm2H3HKmTr2aRM6XNbmloGJwQAxM7waEeIttq5uWGz8b1hLuz2mGGA22FwQkRESnOmJ4tbN0Ene41ouCaFJb67mGveS8MToiIyOZq9gzuen0oAr0dY3aepd3VPRhr9l5A39ZNbN0UWRicEBFRvaxdDo2Bifm4OKmx7Mm+ep9v5uWGS4Uldeoq2RqDEyIiMhpzoezbxhcH4fDFfAxQWGI+gxMiIiIH1aSRKwa2V9aifwDrnBAREZHCMDghIiKb68U1aKgGDusQEZHNPdq3JVydVOgdZl+zSsyFmTvajOo5mT9/PsLCwuDu7o6oqCgkJyfr3fbw4cN44IEHEBYWBpVKhU8//dTYthI5tMZu/C4h1YORyqyR0RCN6FY1y6NTkJdJx3FSqzC2T0u0bdbYHM0iOyc7OFm1ahXi4uIwa9Ys7N27F+Hh4Rg+fDhycnJ0bn/9+nW0adMGs2fPRlCQsqYqEdmDrx+PRJdgb3zxaE9bN8VufPxQOD5+KNzWzXAIzX09sH/mMPz+wm22bgo1ILKDk7lz52LSpEmIjY1Fly5dsHDhQnh6emLJkiU6t+/Tpw/mzJmDRx55BG5utll5ksieDesahPXTBqJ9oGnfTIksxcfTBc5cBoPMSNZvU2lpKVJSUhATE3PrAGo1YmJikJSUZLZGlZSUoKCgQOuHiEiOIR2rpkeaOtxAwGfjesLNWY33bq5iTmRpsgaxc3NzUVFRgcBA7eWlAwMDcezYMbM1Kj4+Hu+8847ZjkdEjse/sRsOvTMcHna6toiS9AlrgiPvjuBid2Q1iuyHmzFjBvLz8zU/GRkZtm4SEdmhxm7OvKGaCV9Hy2KlXW2yek78/f3h5OSE7Oxsrcezs7PNmuzq5ubG/BQiIiIHJavnxNXVFZGRkUhISNA8VllZiYSEBERHR5u9cUREROR4ZBdOiIuLw8SJE9G7d2/07dsXn376Ka5du4bY2FgAwIQJE9C8eXPEx8cDqEqiPXLkiObfFy5cQGpqKho3box27dqZ8VKIiIioIZAdnIwdOxaXLl3CzJkzkZWVhYiICGzYsEGTJJueng61+laHzMWLF9Gz5636DB9//DE+/vhjDB48GImJiaZfARERETUoRpWcnDp1KqZOnarzudoBR1hYGIQQxpyGiIiIHJAiZ+sQERE5kj4OuqaQPlysg4iIyEYSXxmC7ady8VBkqK2boigMToiIiGwkzL8Rwvwb2boZisNhHSIiIlIUBidERESkKAxOiIiISFEYnBAREZGiMDghIiIiRWFwQkRERIrC4ISIiIgUhcEJERERKQqDEyIiIlIUBidERESkKAxOiIiISFEYnBAREZGiMDghIiIiRbGLVYmFEACAgoICG7eEiIiIpKq+b1ffx6Wyi+CksLAQABAaGmrjlhAREZFchYWF8PHxkby9SsgNZ2ygsrISFy9ehJeXF1QqldmOW1BQgNDQUGRkZMDb29tsx1WShn6NvD7719Cvkddn/xr6NVry+oQQKCwsREhICNRq6ZkkdtFzolar0aJFC4sd39vbu0H+wtXU0K+R12f/Gvo18vrsX0O/Rktdn5wek2pMiCUiIiJFYXBCREREiuLQwYmbmxtmzZoFNzc3WzfFYhr6NfL67F9Dv0Zen/1r6NeoxOuzi4RYIiIichwO3XNCREREysPghIiIiBSFwQkREREpCoMTIiIiUhSHDk7mz5+PsLAwuLu7IyoqCsnJybZuEuLj49GnTx94eXkhICAA9913H44fP661zZAhQ6BSqbR+nnvuOa1t0tPTMWrUKHh6eiIgIACvvvoqysvLtbZJTExEr1694Obmhnbt2mHp0qV12mPu1+jtt9+u0/ZOnTppni8uLsaUKVPQtGlTNG7cGA888ACys7Pt4tqqhYWF1blGlUqFKVOmALC/92/r1q245557EBISApVKhbVr12o9L4TAzJkzERwcDA8PD8TExODkyZNa21y5cgXjx4+Ht7c3fH198dRTT6GoqEhrmwMHDmDgwIFwd3dHaGgoPvroozpt+fHHH9GpUye4u7uje/fuWL9+vey2yLm+srIyTJ8+Hd27d0ejRo0QEhKCCRMm4OLFi1rH0PWez549WxHXZ+gaAeCJJ56o0/4RI0ZobWOv7yEAnX+PKpUKc+bM0Wyj5PdQyn1BSZ+dUtpikHBQK1euFK6urmLJkiXi8OHDYtKkScLX11dkZ2fbtF3Dhw8X3377rTh06JBITU0Vd911l2jZsqUoKirSbDN48GAxadIkkZmZqfnJz8/XPF9eXi66desmYmJixL59+8T69euFv7+/mDFjhmab06dPC09PTxEXFyeOHDkiPv/8c+Hk5CQ2bNig2cYSr9GsWbNE165dtdp+6dIlzfPPPfecCA0NFQkJCWLPnj2iX79+on///nZxbdVycnK0ru+vv/4SAMSWLVuEEPb3/q1fv1688cYbYs2aNQKA+OWXX7Senz17tvDx8RFr164V+/fvF/fee69o3bq1uHHjhmabESNGiPDwcLFz507xzz//iHbt2olx48Zpns/PzxeBgYFi/Pjx4tChQ2LFihXCw8NDfPXVV5pttm/fLpycnMRHH30kjhw5It58803h4uIiDh48KKstcq4vLy9PxMTEiFWrVoljx46JpKQk0bdvXxEZGal1jFatWol3331X6z2t+Tdry+szdI1CCDFx4kQxYsQIrfZfuXJFaxt7fQ+FEFrXlZmZKZYsWSJUKpU4deqUZhslv4dS7gtK+uw01BYpHDY46du3r5gyZYrm/xUVFSIkJETEx8fbsFV15eTkCADi77//1jw2ePBgMW3aNL37rF+/XqjVapGVlaV5bMGCBcLb21uUlJQIIYT497//Lbp27aq139ixY8Xw4cM1/7fEazRr1iwRHh6u87m8vDzh4uIifvzxR81jR48eFQBEUlKS4q9Nn2nTpom2bduKyspKIYR9v3+1P/grKytFUFCQmDNnjuaxvLw84ebmJlasWCGEEOLIkSMCgNi9e7dmmz/++EOoVCpx4cIFIYQQX375pfDz89NcnxBCTJ8+XXTs2FHz/4cffliMGjVKqz1RUVHi2WefldwWudenS3JysgAgzp07p3msVatWYt68eXr3Ucr1CaH7GidOnChGjx6td5+G9h6OHj1a3HHHHVqP2dN7WPu+oKTPTiltkcIhh3VKS0uRkpKCmJgYzWNqtRoxMTFISkqyYcvqys/PBwA0adJE6/Hvv/8e/v7+6NatG2bMmIHr169rnktKSkL37t0RGBioeWz48OEoKCjA4cOHNdvUvP7qbaqv35Kv0cmTJxESEoI2bdpg/PjxSE9PBwCkpKSgrKxM65ydOnVCy5YtNedU+rXVVlpaiuXLl+PJJ5/UWrTSnt+/ms6cOYOsrCyt8/j4+CAqKkrrPfP19UXv3r0128TExECtVmPXrl2abQYNGgRXV1et6zl+/DiuXr0q6ZqltMUc8vPzoVKp4Ovrq/X47Nmz0bRpU/Ts2RNz5szR6i63h+tLTExEQEAAOnbsiMmTJ+Py5cta7W8o72F2djbWrVuHp556qs5z9vIe1r4vKOmzU0pbpLCLhf/MLTc3FxUVFVpvEgAEBgbi2LFjNmpVXZWVlXjxxRcxYMAAdOvWTfP4o48+ilatWiEkJAQHDhzA9OnTcfz4caxZswYAkJWVpfPaqp+rb5uCggLcuHEDV69etchrFBUVhaVLl6Jjx47IzMzEO++8g4EDB+LQoUPIysqCq6trnQ/9wMBAg+1WwrXpsnbtWuTl5eGJJ57QPGbP719t1e3RdZ6abQ0ICNB63tnZGU2aNNHapnXr1nWOUf2cn5+f3muueQxDbTFVcXExpk+fjnHjxmktkPavf/0LvXr1QpMmTbBjxw7MmDEDmZmZmDt3rl1c34gRIzBmzBi0bt0ap06dwuuvv46RI0ciKSkJTk5ODeo9XLZsGby8vDBmzBitx+3lPdR1X1DSZ6eUtkjhkMGJvZgyZQoOHTqEbdu2aT3+zDPPaP7dvXt3BAcHY+jQoTh16hTatm1r7WbKMnLkSM2/e/TogaioKLRq1QqrV6+Gh4eHDVtmGYsXL8bIkSMREhKiecye3z9HVlZWhocffhhCCCxYsEDrubi4OM2/e/ToAVdXVzz77LOIj49XVElwfR555BHNv7t3744ePXqgbdu2SExMxNChQ23YMvNbsmQJxo8fD3d3d63H7eU91HdfaGgccljH398fTk5OdbKHs7OzERQUZKNWaZs6dSp+//13bNmyBS1atKh326ioKABAWloaACAoKEjntVU/V9823t7e8PDwsNpr5Ovriw4dOiAtLQ1BQUEoLS1FXl6e3nPa07WdO3cOmzZtwtNPP13vdvb8/lUfq77zBAUFIScnR+v58vJyXLlyxSzva83nDbXFWNWByblz5/DXX38ZXFY+KioK5eXlOHv2bL1tr9luW15fbW3atIG/v7/W76S9v4cA8M8//+D48eMG/yYBZb6H+u4LSvrslNIWKRwyOHF1dUVkZCQSEhI0j1VWViIhIQHR0dE2bFnVNLOpU6fil19+webNm+t0I+qSmpoKAAgODgYAREdH4+DBg1ofJtUfqF26dNFsU/P6q7epvn5rvUZFRUU4deoUgoODERkZCRcXF61zHj9+HOnp6Zpz2tO1ffvttwgICMCoUaPq3c6e37/WrVsjKChI6zwFBQXYtWuX1nuWl5eHlJQUzTabN29GZWWlJjCLjo7G1q1bUVZWpnU9HTt2hJ+fn6RrltIWY1QHJidPnsSmTZvQtGlTg/ukpqZCrVZrhkKUfH26nD9/HpcvX9b6nbTn97Da4sWLERkZifDwcIPbKuk9NHRfUNJnp5S2SCI5dbaBWblypXBzcxNLly4VR44cEc8884zw9fXVymS2hcmTJwsfHx+RmJioNaXt+vXrQggh0tLSxLvvviv27Nkjzpw5I3799VfRpk0bMWjQIM0xqqeMDRs2TKSmpooNGzaIZs2a6Zwy9uqrr4qjR4+K+fPn65wyZu7X6OWXXxaJiYnizJkzYvv27SImJkb4+/uLnJwcIUTVFLSWLVuKzZs3iz179ojo6GgRHR1tF9dWU0VFhWjZsqWYPn261uP2+P4VFhaKffv2iX379gkAYu7cuWLfvn2a2SqzZ88Wvr6+4tdffxUHDhwQo0eP1jmVuGfPnmLXrl1i27Zton379lrTUPPy8kRgYKB4/PHHxaFDh8TKlSuFp6dnnWmazs7O4uOPPxZHjx4Vs2bN0jlN01Bb5FxfaWmpuPfee0WLFi1Eamqq1t9k9QyHHTt2iHnz5onU1FRx6tQpsXz5ctGsWTMxYcIERVyfoWssLCwUr7zyikhKShJnzpwRmzZtEr169RLt27cXxcXFdv8eVsvPzxeenp5iwYIFdfZX+nto6L4ghLI+Ow21RQqHDU6EEOLzzz8XLVu2FK6urqJv375i586dtm6SAKDz59tvvxVCCJGeni4GDRokmjRpItzc3ES7du3Eq6++qlUnQwghzp49K0aOHCk8PDyEv7+/ePnll0VZWZnWNlu2bBERERHC1dVVtGnTRnOOmsz9Go0dO1YEBwcLV1dX0bx5czF27FiRlpamef7GjRvi+eefF35+fsLT01Pcf//9IjMz0y6uraaNGzcKAOL48eNaj9vj+7dlyxadv5MTJ04UQlRNj3zrrbdEYGCgcHNzE0OHDq1z3ZcvXxbjxo0TjRs3Ft7e3iI2NlYUFhZqbbN//35x2223CTc3N9G8eXMxe/bsOm1ZvXq16NChg3B1dRVdu3YV69at03peSlvkXN+ZM2f0/k1W161JSUkRUVFRwsfHR7i7u4vOnTuLDz/8UOvGbsvrM3SN169fF8OGDRPNmjUTLi4uolWrVmLSpEl1glh7fQ+rffXVV8LDw0Pk5eXV2V/p76Gh+4IQyvrslNIWQ1Q3L5yIiIhIERwy54SIiIiUi8EJERERKQqDEyIiIlIUBidERESkKAxOiIiISFEYnBAREZGiMDghIiIiRWFwQkRERIrC4ISIiIgUhcEJERERKQqDEyIiIlIUBidERESkKP8PYc22ambjVgAAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "split_loss('train')\n",
    "split_loss('val')\n",
    "plt.plot(lossi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Summarizing the results so far\n",
    "\n",
    "- Initial results\n",
    "```\n",
    "train 2.135850667953491\n",
    "val 2.1770708560943604\n",
    "```\n",
    "\n",
    "- Results after softmax fix\n",
    "```\n",
    "train 2.0693342685699463\n",
    "val 2.1324031352996826\n",
    "```\n",
    "\n",
    "- Results after tanh fix\n",
    "```\n",
    "train 2.0361340045928955\n",
    "val 2.102936267852783\n",
    "```\n",
    "\n",
    "- Results with kaiming init\n",
    "```\n",
    "train 2.0395455360412598\n",
    "val 2.1068387031555176\n",
    "```\n",
    "\n",
    "We can see that we arrive at roughly the same position as after tanh fix, but without using magic number for scaling down W1.\n",
    "\n",
    "These optimization we very crucial in early years of neural networks, but not so much today. Today with techniques like Batch normalization, it's not as crucial to initialize neural network perfectly as it was earlier. Let's take a look into why that's the case\n",
    "\n",
    "## Batch Normalization\n",
    "\n",
    "Batch normalization was introduced by [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://arxiv.org/abs/1502.03167). This was a very significant paper as it allows us to train large neural networks\n",
    "The core insight from the paper was to normalize the hidden state to be gaussian. Since we already know that we want the preactivations to lie within [-1,1] for tanh function to be active, we can directly attempt to normalize the preactivations to be roughly guassian. Mathematically this looks like\n",
    "(hpreact - h.mean())/h.std()\n",
    "\n",
    "All the operations involved in the above expressions are differentiable, we can just use this expressions for forward pass and calculate its derivative during backward pass.\n",
    "\n",
    "One thing to note here, is that we only want to force preactivations to be guassian during first ietrations and allow enough flexibility to be updated during backpropagation. For this we'll make use of scale and shift as mentioned in the paper above. Let's re-initialize the training and testing loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "12097\n"
     ]
    }
   ],
   "source": [
    "n_embd = 10 # the dimensionality of the character embedding vectors\n",
    "n_hidden = 200 # the number of neurons in the hidden layer of the MLP\n",
    "\n",
    "g = torch.Generator().manual_seed(2147483647) # for reproducibility\n",
    "C  = torch.randn((vocab_size, n_embd),            generator=g)\n",
    "W1 = torch.randn((n_embd * block_size, n_hidden), generator=g)\n",
    "W2 = torch.randn((n_hidden, vocab_size),          generator=g) * 0.01\n",
    "b2 = torch.randn(vocab_size,                      generator=g) \n",
    "\n",
    "# BatchNorm parameters\n",
    "bngain = torch.ones((1, n_hidden)) # scale\n",
    "bnbias = torch.zeros((1, n_hidden)) # shift\n",
    "\n",
    "parameters = [C, W1, W2, b2, bngain, bnbias]\n",
    "print(sum(p.nelement() for p in parameters)) # number of parameters in total\n",
    "for p in parameters:\n",
    "  p.requires_grad = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "# same optimization as last time\n",
    "max_steps = 200000\n",
    "batch_size = 32\n",
    "lossi = []\n",
    "\n",
    "def run_training_loop(break_on_first=False):\n",
    "\n",
    "  for i in range(max_steps):\n",
    "    \n",
    "    # minibatch construct\n",
    "    ix = torch.randint(0, Xtr.shape[0], (batch_size,), generator=g)\n",
    "    Xb, Yb = Xtr[ix], Ytr[ix] # batch X,Y\n",
    "    \n",
    "    # forward pass\n",
    "    emb = C[Xb] # embed the characters into vectors\n",
    "    embcat = emb.view(emb.shape[0], -1) # concatenate the vectors\n",
    "    # Linear layer\n",
    "    hpreact = embcat @ W1 #+ b1 # hidden layer pre-activation\n",
    "\n",
    "    hpreact = (hpreact - hpreact.mean(dim=0, keepdim=True))/(hpreact.std(dim=0, keepdim=True)) * bngain + bnbias\n",
    "\n",
    "    # Non-linearity\n",
    "    h = torch.tanh(hpreact) # hidden layer\n",
    "    logits = h @ W2 + b2 # output layer\n",
    "    loss = F.cross_entropy(logits, Yb) # loss function\n",
    "    \n",
    "    # backward pass\n",
    "    for p in parameters:\n",
    "      p.grad = None\n",
    "    loss.backward()\n",
    "    \n",
    "    # update\n",
    "    lr = 0.1 if i < 100000 else 0.01 # step learning rate decay\n",
    "    for p in parameters:\n",
    "      p.data += -lr * p.grad\n",
    "\n",
    "    # track stats\n",
    "    if i % 10000 == 0: # print every once in a while\n",
    "      print(f'{i:7d}/{max_steps:7d}: {loss.item():.4f}')\n",
    "    lossi.append(loss.log10().item())\n",
    "\n",
    "    if break_on_first:\n",
    "      return logits, h, hpreact\n",
    "      break"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "@torch.no_grad() # this decorator disables gradient tracking\n",
    "def split_loss(split):\n",
    "  x,y = {\n",
    "    'train': (Xtr, Ytr),\n",
    "    'val': (Xdev, Ydev),\n",
    "    'test': (Xte, Yte),\n",
    "  }[split]\n",
    "  emb = C[x] # (N, block_size, n_embd)\n",
    "  embcat = emb.view(emb.shape[0], -1) # concat into (N, block_size * n_embd)\n",
    "  hpreact = embcat @ W1 # + b1\n",
    "  hpreact = (hpreact - hpreact.mean(dim=0, keepdim=True))/(hpreact.std(dim=0, keepdim=True)) * bngain + bnbias\n",
    "  h = torch.tanh(hpreact) # (N, n_hidden)\n",
    "  logits = h @ W2 + b2 # (N, vocab_size)\n",
    "  loss = F.cross_entropy(logits, y)\n",
    "  print(split, loss.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's train the model again"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      0/ 200000: 3.8443\n",
      "  10000/ 200000: 2.0775\n",
      "  20000/ 200000: 2.7031\n",
      "  30000/ 200000: 2.1059\n",
      "  40000/ 200000: 2.2516\n",
      "  50000/ 200000: 1.9634\n",
      "  60000/ 200000: 2.1270\n",
      "  70000/ 200000: 2.3852\n",
      "  80000/ 200000: 2.5304\n",
      "  90000/ 200000: 2.1849\n",
      " 100000/ 200000: 2.2979\n",
      " 110000/ 200000: 2.3865\n",
      " 120000/ 200000: 1.6278\n",
      " 130000/ 200000: 2.0458\n",
      " 140000/ 200000: 2.4157\n",
      " 150000/ 200000: 2.0811\n",
      " 160000/ 200000: 2.0468\n",
      " 170000/ 200000: 2.4009\n",
      " 180000/ 200000: 2.1332\n",
      " 190000/ 200000: 2.1841\n"
     ]
    }
   ],
   "source": [
    "run_training_loop()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train 2.1137709617614746\n",
      "val 2.1398322582244873\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x12bc3b5d0>]"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAicAAAGdCAYAAADJ6dNTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABSTklEQVR4nO3dd3gU1foH8O+mJ5AChDQIhA6hJNQQkaIEKRbsiFzB/BQV8YrGghEpYglXEfUqgqIRFRUsFK8gKIEoYCAQCB2kJpQUAqRQ0s/vD8iym2yZ2Z3dnU2+n+fJ8yS7szNndjc7757znvdohBACRERERCrh4ugGEBEREelicEJERESqwuCEiIiIVIXBCREREakKgxMiIiJSFQYnREREpCoMToiIiEhVGJwQERGRqrg5ugFSVFdX4+zZs/D19YVGo3F0c4iIiEgCIQRKSkoQFhYGFxfp/SFOEZycPXsW4eHhjm4GERERWeDUqVNo2bKl5O2dIjjx9fUFcO3k/Pz8HNwaIiIikqK4uBjh4eHa67hUThGc1Azl+Pn5MTghIiJyMnJTMpgQS0RERKrC4ISIiIhUhcEJERERqQqDEyIiIlIVBidERESkKgxOiIiISFUYnBAREZGqMDghIiIiVWFwQkRERKrC4ISIiIhUhcEJERERqQqDEyIiIlKVBh2cHMwpxuebjqOyqtrRTSEiIqLrnGJVYlsZ+eEmAICriwbxA9o4uDVEREQENPCekxr7zxY7uglERER0HYMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOAEghKNbQERERDUYnBAREZGqMDghIiIiVWFwQkRERKrC4ISIiIhUhcEJgKpqrq1DRESkFgxOAKzMPOvoJhAREdF1DE6IiIhIVSwKTubPn4+IiAh4eXkhJiYG6enpJrcvLCzE5MmTERoaCk9PT3Ts2BFr1qyxqMFERERUv7nJfcCyZcuQkJCAhQsXIiYmBh988AGGDx+Ow4cPIygoqM725eXlGDZsGIKCgvDTTz+hRYsWyMrKQkBAgBLtJyIionpGdnAyb948TJw4EfHx8QCAhQsXYvXq1UhOTsYrr7xSZ/vk5GRcuHABf//9N9zd3QEAERER1rWaiIiI6i1Zwzrl5eXIyMhAXFzcjR24uCAuLg5paWkGH/PLL78gNjYWkydPRnBwMLp164a3334bVVVVRo9TVlaG4uJivR8iIiJqGGQFJwUFBaiqqkJwcLDe7cHBwcjNzTX4mOPHj+Onn35CVVUV1qxZg+nTp+O9997Dm2++afQ4SUlJ8Pf31/6Eh4fLaSYRERE5MZvP1qmurkZQUBA+++wz9O7dG2PGjMG0adOwcOFCo49JTExEUVGR9ufUqVO2biYRERGphKyck8DAQLi6uiIvL0/v9ry8PISEhBh8TGhoKNzd3eHq6qq9rUuXLsjNzUV5eTk8PDzqPMbT0xOenp5ymkZERET1hKyeEw8PD/Tu3RspKSna26qrq5GSkoLY2FiDjxkwYACOHj2Kap0qrP/88w9CQ0MNBiZERETUsMke1klISMCiRYvw1Vdf4eDBg5g0aRIuX76snb0zfvx4JCYmarefNGkSLly4gClTpuCff/7B6tWr8fbbb2Py5MnKnQURERHVG7KnEo8ZMwbnzp3DjBkzkJubi+joaKxdu1abJJudnQ0XlxsxT3h4ONatW4fnn38ePXr0QIsWLTBlyhRMnTpVubMgIiKiekMjhBCOboQ5xcXF8Pf3R1FREfz8/BTbb8Qrq7W/n5xzu2L7JSIiIsuv31xbh4iIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIgUJ4RwdBOIyIkxOCEiRT35zQ7c9fEWVFUzQCEiyzA4ISJFrdufh71nirD3TJGjm0JETorBCREREakKgxMiIiJSFQYnREREpCoMToiIiEhVGJyoUG5RKaat2It/8koc3RQiIiK7Y3Cio+BSGXafKnR0M/DMdzvx7bZsjPxwk6ObQkREZHcMTnT0eXM9Rs/fgoysixbvY8nWLHyddtKqdhzIKQYA1okgIqIGicGJAWnHCix63OWySry2ch9mrNqPoisVCreKiIioYWBwoqCKqmrt72VVVQ5sCUm1KvMMMlUwlEdERDe4OboBRI6SeaoQU5ZmAgBOzrndsY0hIiIt9pxQg3Wi4JKjm0BERAYwODFCCIEJyemY+PUORzcFqzLPYMCcDdjHtUqIiKgBYHBiRF5xGf785xz+OJCHklLDya1V1QKz/3cA6/bn1r3Tiok2mlp/T1maiTOFVzH5u52W75TIzoTgbDMisoxFwcn8+fMREREBLy8vxMTEID093ei2ixcvhkaj0fvx8vKyuMH2IiREF8t3nkbylhN48puMa4/ReUj84u2Kt6mistr8RmRTZZU3Ep1/2HEKM1btQzWnfBMRKUp2cLJs2TIkJCRg5syZ2LlzJ6KiojB8+HDk5+cbfYyfnx9ycnK0P1lZWVY1Wi3yikuN3rf/bDEvWvXMqswz6PTaWvyw4xQA4OWf9uDrtCyk/mP8vU9ERPLJDk7mzZuHiRMnIj4+HpGRkVi4cCF8fHyQnJxs9DEajQYhISHan+DgYKsa7Sw0tcdn6iEhBL7ZmoVtx8/b/dglpRV48cfd2HTknF2OVzOz5+Wf9ujdXnTVMTVtjp27hJEfbsLqPTkOOT4Rka3ICk7Ky8uRkZGBuLi4GztwcUFcXBzS0tKMPu7SpUto3bo1wsPDMXr0aOzfv9/kccrKylBcXKz340jO3v+x53QhJn+3E6cuXFF832nHzmP6yn0Y89lWxfdtzvt/HMFPGafxyBfGhxVN0dTJ7nEuL/64GwdzipmLRET1jqzgpKCgAFVVVXV6PoKDg5GbayApFECnTp2QnJyMVatWYcmSJaiursZNN92E06dPGz1OUlIS/P39tT/h4eFymqkISy5cag1i7vp4C1bvycHT3yp/ETt5XvmAR6rTFx13bFs7cPZa0HGi4LLRbS6VVtqxRURE9mPz2TqxsbEYP348oqOjMXjwYCxfvhzNmzfHp59+avQxiYmJKCoq0v6cOnXK1s202tp9ynWta2w0HnTSxIWO1GX0/M1YvScHj35pWa8QEZEzkxWcBAYGwtXVFXl5eXq35+XlISQkRNI+3N3d0bNnTxw9etToNp6envDz89P7cSRjMyJ1b39qyU7kFhlPkCV95ZXVuH/B33h7zUGHtUHNOUEVVdfeXFkO7JkiInIUWcGJh4cHevfujZSUFO1t1dXVSElJQWxsrKR9VFVVYe/evQgNDZXXUjvTu3BJHK+5cLncJm1xJCEEcoquKr7fPw7kYUfWRXz213HF912j4FKZzWptTFm6yyb7Jarvtp+8wCRuMkv2sE5CQgIWLVqEr776CgcPHsSkSZNw+fJlxMfHAwDGjx+PxMRE7fazZ8/G77//juPHj2Pnzp3417/+haysLDz++OPKnYUCftyhP3SkG5vM+GWfXn2L+qa0ospooblXft6L2KQN+DnDcI6QlHowhlRW27Zmy/fp2ejz5nq8u+6wTfa/KvOs9ndnT6y1tfLKasxddxjbT15wdFNIBR5YmIbJ3+3E0fwSRzeFVEx2cDJmzBjMnTsXM2bMQHR0NDIzM7F27Vptkmx2djZycm5ExRcvXsTEiRPRpUsXjBo1CsXFxfj7778RGRmp3Fko4KVa00N1rco8i2/SnL82i7EwInr27+g+63dcKa+bYLnsetA2749/ZB3rbOFVPLgwDWv2Gv6GZOsL1cxV12aEfZJ6zKbHIfO+TjuJjzcexQMLjc/oI+N+zjiNCcnpKDbyBcJZnSnkMDgZZ9GqxM888wyeeeYZg/elpqbq/f3+++/j/ffft+QwDlVRq4DamUL5QxtCqDuvoUZpxbVejKP5l9CjZYAi+5z5y36kn7yA9JMXDK74u2RrttXHcIbnFgBSDuZhZeZZvHVPN/h5uTu6OXZ3nInYVnnhx90AgIWpx/DyiM4Obk3DVF5ZDY0GcHflii/2wmfagLm//4MBczbY9Zh7Thfivd8P42p5lexBAiEE/m/xdjzyxTarcyxKK6oUqWxbdEX+t7ys85dxuUx902OtfU4f+2oH/rf7LP67/oik7atYWZgMqG89J86ioqoafd78Azf/ZwPXi7Iji3pOGiJD78naN1mafwFcq0Vi6DiGemxKK6qw8M9jiOsSjG4t/HGprBIbDl0roX62qBQtArwtakPhlQp0nr4WvVoFYPnTAyzah6UO5RZjxAebEODjjswZt9n12KasyjyDN349iM/G97Z6X3klZWa3OX3xCoa//5fVx1IDfoyTvVRXC7i42KYrNaewFMWllSgurURZZTW83F1tchzSx54TBSkRVB/O008SM9SDM3/jUXyw/gju+GizgTYYb4S5qL8mwNmZXVjnvoqqaoz6cBNe/mm30cf/svssss5b1oVfc+xCC3pcbGnK0kwUXCrDE1/vsMvx5m88isvl0pKvT9mhCN1327Ix+budqKjiopOkTvM3HkXU7N+tSrA9fu4SS0GoDIMTBdnjm+K83w/jow36NWJsVbRNV35JGQ7kFOOHHfqzdnTjnWe/34XB76bavC22MOsX00sqmBpqcVTuS02ukBLKjax4/eqKvVi9Jwcrdp5R7FhkPUet56RG7647jJLSSsz+1bKaSRcul+PW9/5E/6QU8xtbac5vh/DIF9tQyWDfLAYnCrLHeOR/NxgvXnetDZbve0eWcjNotiu4L0OqqwXW7c8zv6ERQgi9oG7x3ydReMWyOjXbTlzAgtRjSD2srtWJi65W4NM/j+GsmWTu9BMX0PG13/DxBuM5MSVllSi4VIb1B/KMBmocj7ePpDUHEfX67/jNyEw4kuekhb29llj45zFsOlKAv+y0WKkzY3CioEe/3O6Q49b+4l5dLTDvj3+w8ZDxi+Xlskos33laL3F13xl5CyxmZF3AsXOXDN6ne52ytEbMlqMFeHBhWp3u2o9SjiDq9d8t2icA/LDjFPq+tR77zxTp3V77mrsz+6Kk/X23LRv/WXsIj365XS/AySsu1UtitHcHyys/70HSb4dw/4K/TW43feU+ANcSwU0Z8cFfePzrHfh2W91p9SWlFbj5PxsxbcXeOvc5yaQq1aupp/Pp9cKFb66+0VOgm8R+MKcYH6UcQWlF/a3N5OxqKkCTcUyIlWjx3ycR6u+FJwe3s/3BZH6a1x5WWLc/F/9NufYtWHcab1llNeb8dghxXYLwdVoWftl9Fn0jmljUxE1Hzlm8GrBU4z7fBgB44usMbHhxiPb292TWXKnt5es1bT41U5323k9uXNQvSsyFKbpagQAfD1y8XI6Yt23fTWzKX/9c+3Z21sBYenFpBb7YdAJ3RYdJ3l/BpWuB1x8H8jA+NkLvvp8zTuNM4VV8u82yKeKFV8qh0Wjg7+2YqdZnC6/iu23ZeCS2NYL9vBzSBksdzS/B/QvT8PSQdnhiUDuM/HATgGv/7y8O7+Tg1hFZhj0nMiT9dgj5JdYlTV24XI5fdp81+q3GVFxiKNau3cVeWlGldzHS/UZVWS2w8M9juH9hGn7Zfa3C6faThnsHdp8qxIs/Gk9+TT1sv27Jc5fMz3JRmrXTqQ/lWpacZ3BWmBCoqhb4cssJ7KvV2yPXpbJKnLpwBa//cgAfphzB0Pf+tGp/2jZK3K60oqpOfktpRRWiZ/+BqNd/V2QauyX+9cU2fLzxKB7/yj6Jz0qasWo/Cq9U4O01h/Ru33/22nulPgy3VVZVY+/pono9zf5cSRkW/XW8Xi6DYgn2nMhUppOEaMn//MOLtuJQbgkevSkCs+7qanV7XvpxN96+t7v276/TshAR2Ej797YTluV+jJ6/xeI2Ld9puNS9KUqVgL9cVonvtmWj3EjC2QkJBcH+OJCH55dlKtIeJfxvTw6ulFXi9f8dAACDRe0M0Z31c7msEo083dD/7RRcKquElFmXugH0G78ekNdoI6Jn/47SimqM6h6Csf1aYWCH5sgrvhFMl1dVw8vFPlM1dfOOjp+79r7Ya2XwJ0XSbwfh5eaK54d1tPmxjuZfwgML/8ZTg9vZpdf3h+2nEBHYCP3aNFV0v9NX7cP36afwxKC2eHVUF0X3rRaPf7Udu08XYf3BPCx7UtpadfUZe05k2nK0QHL0bmirmm/Ui/8+KfvYlQaOu3yX/iyK2lG3rdewMSThB+M9LrbWdeY6vGVipeP0E+fN7mPi1ztwyQbF4Cyd1fPs97u0w3SWqpkKXHNeum+l2tPXa9iiF6FmhtGavbk2HxY0RgiBUxeuoO9bKZi/0XSCudLyikvx6Z/H8aGdckJm/3oAF69UIOm3Q+Y3tlJG1gW8/PMePPip8ssUfJ9+bRkNWywUaurf8tM/j+HW91IVP6Yhu09fC4wt/UJZ3zA4kemV5Xtx18d164tYQu50wHMSingJXPvgrU9KSm8ECt+ZyGmwtstXiel99y9Mw9Sf9lgciBjrjTOUN2ILM1ftQ+apQgDA5qMFiuxTTcsMvLP2EAa+sxGvrtiLgktlNlsY0hhjU7alqP08qul5BYCTBfI+d5QebpK7v4uXy80+Jum3Qwa/FFpLZS+dKjE4scD+s9JmtZRVVuGkiWEEQ7NYNBr5b9zaH1KW9MrYS15xKd773fILwqsGZoMopZ8CCaznSsq0iyXakrU9KcZ8lZaFu80M6el+ngshMPHrHdohJ0OsHbJTsgDcJ6nHcPriVWw6okzg1ZCcKbxab2YArd6Tg55v/IG3Vh+sUyfK2rxCUgaDExu6/b+bMWRuKrYeNzyU8P4fR7D1+Hm9b1MaaGQXVdP98LdX7pulx5n49Y46ReSUIPVbU7mdpvBZmtgpdQkEuatE2+p9cfFKBf44YHm9mcqqavy6x3i9jp8yTqPDtN9Y08PB9p0pwoA5GzDq+kwgQ5wpVfXN1deC6c83n9C7/fNNx9HvrRQkLt9r1VpGFVXVDkvuri+YEGuhb7Zm4aKZol01yZerMs+if9tmde7/Pj0b36dn44HeLRVrV+2LkC3+PzYcykPylhPmN9Sx/eQFLN95BntOG044/D7d+lWKM7LM1yWpqelha09/t7PObfbsyl2xS35Ssq5B72w0u03hlXKrn88vt5w0ObRSM2Ns0rc7ERnqhyaN3LHksRi7VEW2p9pFAY353/VZdpYqrajCmM+2IrZtM7wyUvoKxzUBZH1cYVr3Wa+pHVPz2WyMEAKZpwrRMdgXjTz1L6PlldUY8J8NaOrjgXXPD7JFkxsE9pxYaPrKfYoNn/yYYd2FRNfa/bl6f6+wYOaMOf+3WH6i5AML04z+s1+8XI5sBfJk/vXFNqP37T1dhGHzlJk2K4Uj1wg6ffEKnl9mXVKyuddj4+F83P7fzVhtokfj2e934bCZKdWp/0ivqnsgpxhbjp43msBrSlllFd42kShtzFk7DGVsOJSHvm+laOvSGLP1+Hn8+/td2r9PX7yK11bKG+b83+6z2H2qEAv/PGZRW6Wy5Ll2Jit2ncE9n/yNez7Zgi1HC9D/7RRt0cuj+ZdwrqTMovcp3cDgRGWsWdnYkCP5hiu4qknPN/6QtJ25Gh+mkg0fSd7mFM+FEu75pG5FWCXfVQWXyhD/5XaDK2brOn3xKu79ZIvFdRuMVR+uqJR/Nou3nJQ90+NgTjFumrNBW9SstqIrFZj9vwNW1575v8U7UHCpDOOT685eWrb9RkD/T17d52PJ1mz8fczwsHHtZ2mOQsmdd3602eBrrzu0+tlfxx02rHE0vwRvrzmI8zasj7Ti+izJf/IuYdzn25BbXIr4xXUrhButZ6XTS1Z4pRwfrJdfWHLJ1iw8tnh7vckDqo3BiR0dlXBxXLc/r94t6nW20PoEs5yiqwZXYa5hKqdi/sajqlvt2Bord9VdhO+yztRnQ7O6lJwZISfYuFxeZbJ3xZR7rKi1U9vJ89J75hakHsO+M0VYc73dxmrjvLH6AJK3nDD5vjRFyksy9WflEsAX/nlM0oy23/bm4M1fDxgNLvaeKcKsX/bjwuVybDlaYPS9NVFnJe8r5ZWYuWqf0fw7Jd32/l/47K/jmPBluqRkaiVHCE9fvIJnl97o3eo8fS1KDOSu6B7y5Z/24IP18hPcX1u5DymH8rFkq/5yEmWVVXVyGc1ZlXkG663IHbMFBid28H16Nub98Q/Gmxh2sEbicuMfYFJnFtnSMwbyL8y5WOsCKCWwM8be00WN0Wg0KC6twNjPtmqHuIquVuCbrVk4f6kMmacK66z6bMhzBgrEdZ25zuTCfc6ouFT5WjNS/GftIUkBhxr+t+R6TUKO0KRvd+LzzSdMBpVXyisxbN6fGPf5Nm216dpSdNb2+njDUXyVloWHPtsqv9Ey1cRU+84U4z4z60oB8maTmQson/luV53Pqu0nTdctSTdxvxACS7ZmmeydK6n1f/Lq8n146LOtmGlmpfUaecWlmLI0E49/ra7qyAxO7OS/KUdsVqviZxvklSjJksBi2Pt/ydq+yglKdO/IuoBP/zyGtOPntQFlrzf+wPSV+zDonY1mp/CaY27hvtUmZsU4ilKVgQ3Ze7rI4pWmzckpuoqDOdKCk9V7cpC05mCdHgbdyri2cP6Sdedurq7S+etfIH6//o3b1H+g0iv/CiHwy+6zRof+auy5nmuWfuKCpOrQ1jpupj1ylJRW4JPUY3ht5T5ZvXM11wOpkwzU2qvM2TqkSgUyx4uVSna15Tj5qQtXkX1Bf6y+pptdt9S8LQgAky3owTIkX0IxQKlsdcFIO3YeYxdtha+XG/bOGq7ovo+duyRrTaKa57136yboEuqnvX3I3FTJSxFYwlwp/lMXriC8qY/Njm+piqpqbD5agD6tm8DXy/BCkL8fyMOzOsnBALDpSAGSDCTiHsm/pK1a+2V8X9zSKcjituUVl+otDyLFuZIy/N/i7YgOD5D1uH5vpeCqTj7J1uPn4e3uiqha+5HziXWprBLe7q5wlbJ+hYOx54RsrsyKqpg1zHVHy8kpMMXWXZvWTgU1J3mzvCnejlY7sfKqhCBNStL4qsxreTm1u7zlMNYZN89AD9Vinan1m46cu5Z8WivfocDKngylDXxno+Rq0pLyMhSK6/+bcgTxX243mCBco6aKcW3mVhr/0coCiTMkDpXomvXLAWw4lK9Xm0gA+HjDEaQZSWYGoBeYAMBDn23F6PlbMPS9VPxnrfzlCPKKS9Ft5jrc84lyuVy2xOCEnEKWQsGHORsOSZ/aqkazjSzQ9+1W6+vI2MO6WlPhLfH6//Zj6Xbrq/R+rLPuzuItJ0wmFc/RuVg88kU6Fv55zKJKwUm/HURGlv3WVkk3sY7LvrPGe15ydIaoV+/JwZzfDuHln/fIOvac3w5h9Z6cOsXOfrpeWmFXdqGs/Vlqzlrp056lLCFSW+0gAwB+3XMWc3//B2MXyc/BOXbuMhakyp8KXvO/ZazWlNowOCHV2aLQmi50w/sWTFV0BCnf0HPM5G59ueWkMo3RMet/B/DGr8YvYqUVdXsHT180PdUaAHZm6xcO/PTP47hvgfIL55libAbP8p1ntPdV1Or9rFnFuYYldVMO5ZZg8nc78dQ3GUa3Ka2oMpjHYWmK2Zq9uZi0JENvFs+Wo7afQVSbUj29AEw+Gb/tzcHnm47LHiZXAwYnpDrjPrfNrCZSv8O55hMKn/wmwyarRptjriqyJQtP3mugJo015OYVHcgpRuSMtUZXZx6ffO1/MdeGybvG6rQA16bi3lorv2fJ1ixUWbHa+m/7cm02xGbJDDNbJqRO+nYn3lx90OQK42ottszghMhBZlkwfl3fJW85gS+3nDA7syjHTAE4RzAXnAgIyReCm5JSkGXBDBe5M7K+2HwCZZXVeHfdYQgh6gxdbTl6Hn+rrCfztZX78HValvkNbUCN1/FTF69i2oq9dXrhdGWeKsSqzBv5buM+t/2Ubmtxtg6Rg6h59WhHMrXCcY1yBVcqPisj0DliZUlyqcMRZ4tK8YaR/CFbmfh1Bg4YyDN5+PNtuKNHqFX7Vrp3QIkke0vlF5davaq1uSBHzheXmmq1327LNjn7S3ftMUcMZcnFnhMicjq3/3ezYkM7cmZQ3fmxvGqwy2vVIPpmq/Rv/PauFL3+YJ7RWkxrrFwV2lyekLMQAPq9nWLz4zjqi4uSlaStxeCEiJxSzTTMv48W2HyKdo3SimqjX3t3ZV/UK1W+bl8uEn64sQBjVbWQtb7P9pPmV9m2F0vL/8y9Xp1ZjcMhlshXKPfG2FRoR1Dra8PghIic0srrtUwe/nwb/v39Lpw0UtCtsqraZpVidY35bCu6z/pd+/fxWu2Zsarh5RjVTMeWEtvkl1y78Kv1YgncqIqrVrVr6zgzBidE5JRW78nBil03hk2MTd196ac92GxljgBZTghhthLwF5tPoN9bKRatzks3GFoZ2VkxOCEip/X8shvDJm+uPmCwnsMKA6s461q+0/T9ZJ02iWvMblOT/PvBetutQdYQyEnU/T49W/L6UI7A4ISI6oVDuSW4X8IqtLUdtnIGDpEzSly+FyM/3KR3m4ryYRmcEFH9YazyZomCRdvUuLozkaVYhI2IiIhUq9KGq7LLxeCEiIiogbpcdmNhwv0mFnu0NwYnREREDdTo+Vsc3QSDGJwQERGRpHo09sLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERMQibERERKQ26olOGJwQERGRqjA4ISIiIlVhcEJERESqwuCEiIiImBBLREREZAyDEyIiIlIVBidERESkKgxOiIiISEVVThicEBEREZgQS0RERGQUgxMiIiJSFQYnREREpCoMToiIiAhCRUknDE6IiIhIVSwKTubPn4+IiAh4eXkhJiYG6enpkh63dOlSaDQa3H333ZYcloiIiGxEPf0mFgQny5YtQ0JCAmbOnImdO3ciKioKw4cPR35+vsnHnTx5Ei+++CIGDhxocWOJiIio/pMdnMybNw8TJ05EfHw8IiMjsXDhQvj4+CA5OdnoY6qqqjBu3Di8/vrraNu2rVUNJiIiovpNVnBSXl6OjIwMxMXF3diBiwvi4uKQlpZm9HGzZ89GUFAQHnvsMUnHKSsrQ3Fxsd4PERER2Y6K8mHlBScFBQWoqqpCcHCw3u3BwcHIzc01+JjNmzfjiy++wKJFiyQfJykpCf7+/tqf8PBwOc0kIiIiJ2bT2TolJSV45JFHsGjRIgQGBkp+XGJiIoqKirQ/p06dsmEriYiISE3c5GwcGBgIV1dX5OXl6d2el5eHkJCQOtsfO3YMJ0+exJ133qm9rbq6+tqB3dxw+PBhtGvXrs7jPD094enpKadpREREVE/I6jnx8PBA7969kZKSor2turoaKSkpiI2NrbN9586dsXfvXmRmZmp/7rrrLtxyyy3IzMzkcA0RERHVIavnBAASEhIwYcIE9OnTB/369cMHH3yAy5cvIz4+HgAwfvx4tGjRAklJSfDy8kK3bt30Hh8QEAAAdW4nIiIix7laUenoJmjJDk7GjBmDc+fOYcaMGcjNzUV0dDTWrl2rTZLNzs6GiwsLzxIRETmTnzJO49bOweY3tAONUFMxfSOKi4vh7++PoqIi+Pn5KbbfiFdWK7YvIiIiZzYsMhiLxvdRdJ+WXr/ZxUFERESqwuCEiIiIVIXBCREREakKgxMiIiJSFQYnRERE5Lxr6xARERHZGoMTIiIiAqCerhMGJ0RERKQqDE6IiIgIxwsuO7oJWgxOiIiICMfPMTghIiIiMojBCREREakKgxMiIiJSFQYnREREpCoMToiIiEhVGJwQERGRqjA4ISIiIlVhcEJERESqwuCEiIiIVIXBCREREakKgxMiIiJSFQYnREREpCoMToiIiEhVGJwQERGRqjA4ISIiIlVhcEJERESqwuCEiIiIVIXBCREREalKgw5Ognw9Hd0EIiIiqqVBByfJj/Z1dBOIiIiolgYdnAT5seeEiIhIbRp0cEJERETqw+CEiIiIVKVBBycaaBzdBCIiIqqlQQcnREREpD4MToiIiEhVGJwQERGRqjA4ISIiIlVp0MGJhvmwREREqtOggxMiIiJSHwYnREREpCoNOjjx9XJzdBOIiIiolgYdnHi6uTq6CURERFRLgw5OiIiISH0YnBAREZGqMDghIiIiVWFwQkRERKrC4ISIiIhUhcEJERERwU9F5TUYnBAREZGqMDghIiIiCEc3QAeDEyIiIlIVBidERESkKgxOiIiISFXjOgxOiIiISFUYnBAREZGqWBSczJ8/HxEREfDy8kJMTAzS09ONbrt8+XL06dMHAQEBaNSoEaKjo/HNN99Y3GAiIiKyAY2jG3CD7OBk2bJlSEhIwMyZM7Fz505ERUVh+PDhyM/PN7h906ZNMW3aNKSlpWHPnj2Ij49HfHw81q1bZ3XjiYiISCHOnHMyb948TJw4EfHx8YiMjMTChQvh4+OD5ORkg9sPGTIE99xzD7p06YJ27dphypQp6NGjBzZv3mx144mIiEgZKopN5AUn5eXlyMjIQFxc3I0duLggLi4OaWlpZh8vhEBKSgoOHz6MQYMGGd2urKwMxcXFej9ERERkO0KoJzyRFZwUFBSgqqoKwcHBercHBwcjNzfX6OOKiorQuHFjeHh44Pbbb8dHH32EYcOGGd0+KSkJ/v7+2p/w8HA5zSQiIiInZpfZOr6+vsjMzMT27dvx1ltvISEhAampqUa3T0xMRFFRkfbn1KlT9mgmERERqYCsJQgDAwPh6uqKvLw8vdvz8vIQEhJi9HEuLi5o3749ACA6OhoHDx5EUlIShgwZYnB7T09PeHp6ymkaERERWUE9gzoye048PDzQu3dvpKSkaG+rrq5GSkoKYmNjJe+nuroaZWVlcg5NREREDYSsnhMASEhIwIQJE9CnTx/069cPH3zwAS5fvoz4+HgAwPjx49GiRQskJSUBuJY/0qdPH7Rr1w5lZWVYs2YNvvnmGyxYsEDZMyEiIqJ6QXZwMmbMGJw7dw4zZsxAbm4uoqOjsXbtWm2SbHZ2NlxcbnTIXL58GU8//TROnz4Nb29vdO7cGUuWLMGYMWOUOwsiIiKyioom60Aj1DR3yIji4mL4+/ujqKgIfn5+iu474pXViu6PiIjIGXm7u+LgGyMU3ael12+urUNEREQI8HF3dBO0GJwQERERwgK8Hd0ELQYnREREpCoMToiIiEhVGJwQERGRqjA4ISIiIudd+I+IiIjI1hicEBERkaowOKnl+4n9Hd0EIiKiBo3BSS2x7Zo5uglERER218hT9oo2NsPghIiIiHB3dAtHN0GLwQkRERHBzVXj6CZoNfjg5JWRnR3dBCIiItLR4IOTpwa3c3QTiIiISEeDD06IiIhIXRicEBERkaowODFg3oNRjm4CERFRg8XgxIB7e7XEwzGtHN0MIiKiBonBiY6uYX6ObgIREVGDx+BER782TR3dBCIiogaPwYkRTXzcHd0EIiKiBonBiRFPDW6HuC5Bjm4GERFRg8PgBICPhysA4NbON4IRXy93fD6hr6OaRERE1GCpZwlCB9oy9VacOH8ZvVo1cXRTiIiIGjwGJwCaNPJAk0Yejm4GERERgcM6krUI8HZ0E4iIiBoEBicSRYX7w9vd1dHNICIiqvcYnMjw6SO9Hd0EIiKieo/BiURCAF5Gek5uiwy2c2uIiIjqLwYnCghv6qP9Pczfy4EtISIicn4MThQ2/Y5IRzeBiIjIqXEqsUIWjOuFXacKMbxriKObQkREJJsQjm7BDew5kah/22bQaG78/duUgXr3j+weildHdYGLiwZSNfLg7B8iIqLaGJyYsXnqLfjwoWj8q39rvdu7hPpZve/WzRqZvD+Aiw8SEVEDxODEjJZNfDA6ugVcZfSISNUhuLHJ+9+8u5vixyQiIlI7BicKkDNOd/jNEYoeW3exQiIiIkvZ4ku4pRic2NBdUWF1bvN0UzbPxMudLyEREVlvmIpqdvHKZgOTb2mH0dFh6NHSX/F9t2rqg0Xj+yi+XyIiatg8XNUTEnAqsQxhEhf/e2l4ZwDA55uOW3U8Q8NFf718i97fvVo1wZq9uVYdh4iISE0YnMjQIsAb3zzWD/7e9plFYyqVZdPLt+BIfglaBPgAOGiX9hAREdmDevpwnMTADs3Ro2WApG1tmVwU3tQHt3YO1qu9Ys68B6PQN6KJzdpERESkBAYnNvRAn3C7Hi992lCkvzoUEwe2MXi/n5e7qrKxiYiIDGFwYkONPfVHzd65r4dNj9e8sSeC/Lww7fZIpL861KbHIiIishUGJ1Z4pH9reLm74HEjPRW63Fw0eLCv/XpSvOxYGv/nSbEWPe6RWlV3iYiIAAYnVnnj7m7YN2u45Fk8tbVsYtnj1MbL3bJAqL6cvykrJw9wdBOIiJwOgxMruUmcF657Af/28RiMi2mFybe0N/mYVk196gwNWUNK8uyTg9uidTMfWfv183KuNYBW2TFgiA4PsNuxiIjqCwYnNvb5+D5o1dQHX/1fP+1tA9oH4q17usPHw3Tg4e6qwc7pw2zdRD2JI7sg9cUhsh4T3tQHbQP1FzF8Pq6jgq1Slrk1jZR2b68Wdj0eEZElXFQ0YYLBiY3FRQbjr5dvQe/WhqfwvnVPNwT7eRq8TwjAw82yl8jbwqEWANDImZ983cMxrbS/j+3XClPiOlh8fF3/vtV075IlzAWFSpv3YLRdj6dWXz7a19FNICInweDEwcbFtMbWxKHoFOxr0eONDfu4u7pgyyu3YlOtirL2ECFzWMgUZxsy0hXbtpmjmyDL5zZeFuEWBy5SeVM753otqK5bOjV3dBPqtRA/L0c3QQ8rxKqAJT0VNcICvDHrzkg09nKvs58WFibqkvXG9gvHm3d3d3QzZImqx/kxz8V1xN/H0hzdDLJCvzbNsPHwOUc3g+yEwUk98OgA81OZyX7G9gvH7NHdbFbwbsrQDjiYU4xNRwpwtaJKsf029zU8vFgfCEMLVZFTaexpv/II5Hgc1iFF6PbadA2TvxrzsbdHKdkch3rhtk5wt+Hqns8P64jPxvdhtV8ZGJo4P3tX3CbHYnDiRF67vYujmyDJzR0CZT/G1UWDsf1amd/QBpRcb2j7tDgENq6/PRDOTO4sNFIXS+spkXNicOJEWjW1LtG0c6gffCUmmDriS7kSCW8+Hq7YPFVeErCStWTq89CIs4uoNd3dWpZWRiZltG2u7OtJ6sLgpAFIS7wVv00ZiBYB3ph1V1dEhQfgw4eiTT5m76zhCG9q34TauC7BeOzmNmbbZsr2aXFo2US52UJke17uzvkxZGllaLKem4sGr9/V1dHNIBtyzk8FkiXU3xtdQv0AXJvBs2ryAIyONl0YrJGnm92HJ1xcNJh+R6TZtgHAqO4hBm9vdL0XRM4Fz5rZUpZY8fRNVtWhUYK9K9caW6rgu8djkDnjNru2hZxf0r3qmwnn7e6K6XdEOroZFhMqy8yyKDiZP38+IiIi4OXlhZiYGKSnpxvddtGiRRg4cCCaNGmCJk2aIC4uzuT2pB5qS7fs3uJGou1HY3vhz5eGGN3WVUbA4WbnMayerZpg4SO9rd6PqRko5s7J0uJ+ljL0ciTd2x03tQ+El7srbm4vP0/JmLfvceyF67GbOXvOmIdjWsHTzu89e2F+urJkv0uWLVuGhIQEzJw5Ezt37kRUVBSGDx+O/Px8g9unpqZi7Nix2LhxI9LS0hAeHo7bbrsNZ86csbrx9YnaolZHqykIdGuXG4W7dJ8jVxcNWjdTZsy5fZB9y9kDQIC3tNyfuC6OK1wmxw9PSsu/6Bxyo9igpQnQgzuazk3SrVbsCOa+PY+PbbircUt939dXAy2YLNBQyQ5O5s2bh4kTJyI+Ph6RkZFYuHAhfHx8kJycbHD7b7/9Fk8//TSio6PRuXNnfP7556iurkZKSorVjW9o1BS+hPlfCx5qLja3RQYDgN4aO9Ykmqa+NARpibeiXXPbBw4aDbD0if42P46u+lbwLDLMD20kJJw6qjCgpWVOTF1MWDpFPiVHUKW836whZXZk7eHjAe2NVyJ+8baOqkmYN9VOtZAVnJSXlyMjIwNxcXE3duDigri4OKSlSau+eOXKFVRUVKBp06ZGtykrK0NxcbHeT32nccAgys+TbjJ5f4j/jXLGrZv5IFTn75WTB+C9B6Lw7eMxAK4t/rdz+jCse36Qdhtz36ZHdgsFAHQ0sBCfl7srQv31L2RyLgbPDzO98OCiWqXa+6uk1PwEB3+r/usl+csd3NerpdlAtOb9fVd0GADUWShSDlvWkKnN1hdAW7m3p2WLTdp6SMrS3k5DvU0tm/jgp6disT5hsLXNqmPqiM5ml1twddHU+dzuHOKHdc8Nwi4DC7Y2UnBWoFR39AiFj4d+flvb5o2cIjdG1n95QUEBqqqqEBwcrHd7cHAwcnNzJe1j6tSpCAsL0wtwaktKSoK/v7/2JzycxXdswdhihDVm3dUVcV2CsTi+Lza8MATLn74RzHh5uOK+3i3RTCdptmkjD70LR2SYHw69McLoP0KrZtcCmtXPDpTUXkPBibEVf819yA6LDNYuuDi8q+HkWkfo1boJXh7Ryer9LJrQB24uGrxzfw9J29ckyLayYF2k9x6MAgB0DfMzu+1dUWFY/vRNWPXMANnHeXZoB/RsFYD7LFjlWU3Dpsa+hsS0Mf6FTY6fJ92ERwdEaP9eNVn6c903Qpk2GHNLJ8uGKY0VHOwT0dQmw7Ih/vo9HMY+Kw29rzqF+KJJI486t0eGmv//qPHsUGUWTv344V51ZjUtGGd9vps92DUzac6cOVi6dClWrFgBLy/jiwwlJiaiqKhI+3Pq1Ck7trLhqp0BH+Trhc8n9MGQTkFwddEg2NcLHYIao3OIL3wlfgvwcnc1GSjUDmjkmvdgNE4kjcKHD0Xjp6du9NRImYGz8cUhSH1xCHq0DLD4+LoM9QDJVTsASxhmWaByS6cgHHpjBB6UUFXz7ugwfDbe+g+sN+/uhicGtTW5jUajQa9WTSTX29GVMKwjVjw9AN1ayK9A7AyWSczbMaf2RVBNQ4hKDevYcoZd1zA/ba9uje8mxtTZ7nGZvUwxBnpnnzTy/2JspXoldAqxbJFZe5PVzxQYGAhXV1fk5eXp3Z6Xl4eQENPfPufOnYs5c+Zg/fr16NHD9Lc5T09PeHqqY2zOkYJs+AatzdPNxWyCoouLBuueuzZsY+/pt6ZoNBpJ049r8/FwQ0SgvK5WX083lJRVwsPVBeVV1QCAoZ2D8PywjiaLQnUMbox/8i7JbmOkhN4IY9xMBH26r94HD/W0+Bi6Anw88OqoLvjsr+N1j6fg28XYtGQpWjX1QfaFK8o1RqUszYex5rmVQj2fGsZJ7cl9aXgnfJV20rqDOegJ8bDj0KilZLXQw8MDvXv31ktmrUlujY01HvW/8847eOONN7B27Vr06WPbZdnrg1//fTOWPtEfQb7qWsIauBaguDTgOXNLn+yPgR0C9fJ1NBqgWwt/+HgYD3RiVZLTooQxfcIl95zZgkajwfyHe+Gd+6QNWVnD0qrMastVkZLcGeLnZdP1mto1b4QmPnWHO+ytj5nh7BrmAjw3Vxe9bd59IMrsPuvkFtpotPHpIe1MvuZtAhupfuaQ7PApISEBixYtwldffYWDBw9i0qRJuHz5MuLj4wEA48ePR2Jionb7//znP5g+fTqSk5MRERGB3Nxc5Obm4tIl+d8iG4puLfxVk6CpJnHXZwQ5asYHcG1Rw28ei0H3lrYbWhjU4dpUWUd9u5l+RyRaN/MxOs4+dWRnm7fhl2cG6NW1qe32HqF4sK/8XDS5PTjjYyMwcaD8JNGfnorFhw9Fo5GHvGJ7SkwdN5QH8fjAtogy8561ZWAyqnsIfn9+sMO/2Pz+/CDJ097lGtU91PxGOpr7euLxgYaHdQxNkPhkXC/8+9b2kvb98ojO2n0bin80Gg2+eazuUJWayP70GzNmDObOnYsZM2YgOjoamZmZWLt2rTZJNjs7Gzk5OdrtFyxYgPLyctx///0IDQ3V/sydO1e5s6AGIaKZD9KnDcWGFy3Pzn9qcDsFW2Qb3Vr4Y+1zA5E+bahDjv/YzW3w50u3oH9by5IjDX1jk3tJUioPqDa57fBwc8G02+XPbGjW2BOjo1vA00AlYI1GY/R9+N6D0Xjz7m51bnd3rdvyl4Zbnjg9xs4r/LpoNFYFP1Jyp4C6iau1n+eOwb51AqQOFiTUys03MST91aFo7uuJXyQkhw+LDMao7qF44TbrkuW/eayfVY+3J4u+mj3zzDPIyspCWVkZtm3bhpiYGxFYamoqFi9erP375MmTEELU+Zk1a5a1bacGKMjXC55ulpV+//XfN+NlKz7QjQmwQVd15xA/i/Z7X6+WZqeISzX5lvb4V/9W6NemKfxlFM/q2SpAkePrur93S4O3//rvm+tMCzfIDpN1fL3c8JnEyr+vjOyM42+Pwou36U959/d2x7/6S5tOPvkWad+iDanphXSEYD/5w9VdZMx0qdHc1xOvSOjl+yNhsOzeWCn7Nacmb09KIC71fWXOwA7WL65qL+rPiiG6ztqiV91a+Cvarfzxwz0xoH0zSR9U3ibyUZT03oNRRodjdAtGdZaQse/j4YY37+6OH56MxR8JN+rXOKJj/l0jU6K7tfDHMAddaJvWmi768ojOuK3WtHRTSwy4uGgwLuZaIFK7V2GJzC73WXc6rm6FlHwWXZ9P6IP+bZvavPChku9Tdxf9S6WpZHNbMDQBQYlV3NXMcVltRE7ujh5huKNHmKRtJw1uh20nzuNuC2YVKeWDMdFYkHoMD8e0wqjuofBwc5Hcfl3m8zbqbiBndpehQnS2nh3m4+GKK+VVkrdPnzYUXrWGbCxpYZNGHtjxWlydhSC7tbB8lhYAw5VIrXgOQ/y8kFtcakWLbugY7IulT5jP+/DzckNxaaVivQbWuisqDL/sPotBOssnyP2+pKJJjqrH4MQJNPZ0w6WySvRqJS3LnNTH38cdK56WX3isNkvKX3/3eAyOF1zGiG6hGKFTv8GSXApbmTioLTYfLcDwrsF4fXTdnAtbej6uI77ZelJycBLXJUjRmXSGVv+uPaQnp9dQCCAswBsLxvXSG47T6G1jeIfGLp5uBnJeTBnYIRDR4QH4aMPRa8eT9ehrHugTjldGdrZLRWBTPVw15tzXHcMigzFEt8dCoeHCxJGdkfTbIe3fDXhCpBaHdZzAjtfikPFanGrWZXAWtbvdnYG5WVq//vtm7e/eEmeC3NQ+UHIeg1Rzr1eFfXWUtLH3O3uYnskwuGNzpE8bqle98u7rJdgtyTcwxlAPjJurRlZFTnv9H258cYjkbQ2d18juobhJgdWePxprug6OoWP/q39r2cmbhvI+LAlM3hjdFW4uGnwoo36PlBjDx8MNd0aFWVRA0FAAquvJwe2w47U4jOkTjn5tmuoNzdau8lyzNMHTVuQcOQMGJ07Ay91Vr0y8kmqqt06TOW7sDNZOkVZMSQ3aBjbC9mlxeusZ1dYiwFsvmfCLCX0R0cwHjygceEgxvGsIDr0xAk8Mkjb76d8SLv5Bvl56OUGP3hSB7x6PwQ9PWp+bYO7i80j/1phTq0KyLt0Cey8PNxyQdQmtm8djzcKVurVSdK//9/ZsgbkSamooIcjXE3dGhdl8kcMv4/tifcJg/Ku/dStKCyHwSGwEDr0xArHt1FGOYeqIzvjr5SFmtwts7In/3N8DPzwZq5fTUnudpPcejMLumbfZfKkBR+Owjko4au2P127vgsduboMwB9YOsRVn6jnxdHeV/Y08KjwAqS/dgr/+OYdvtmbZqGXG1c65MMWSb8CuLhrJ3/xn3RmJWf87YPH0Wo1GgwgThdN+fDIWW46dx/CuwUZni/VuXfdi8d+xPTF33WEs33XGonYZMm9MtGL70iWE0KuvkT5tKPws6CWwlLeHK968uzuWbM2+3h7L92UuYTXI1xP5JWWWH0CGVk19TBZoNCSimQ+Gdg6Cn7d7nXPRaDSyZs85K/acNHAajaZeBiaWeOuebk75T6+2JDtHFI97dEAbHJg93KrptaY0a+yJu6LCZE9jDwvwtlkwYYxSX3OCfL1kBaCmqOwtilm1FsOzNBCy1ZdKjUaDLx7ti/dt/N4Z1f3G7DKlXmulMDhRCUMVAeszc9UqHWFcTGvsmj4MSx6LwVAzy6UrbaYDp4IqzdqZJpYy9e205uLzwvW6IlKKkNlqwTw3G2Q7WrKatBJsuUCdLQX4ON+XEEMkzWQyET+56kyR/mRcLwVapBwO65BdPdK/NVo08cZ9vQwX1XI0FxcNbu4QiJs7BCLildV2Oeb7Y6JMJsIO7RyElEP5iB8QYZf2GKLbvW+ui1pNi0LWdkePMPRr0xTNG3ti2Q791c51W31w9gh4upn/7tandRPsyLqIEV1NL3yqa9IQ+VWKe7ZqgvQTF+rc/sszA5BTVIpOwcqvNCuEMPtaLn0iFhsO5Vt9LHu/Y/pGNEVkqJ/JxTqdwW1dQ9AmsBFOFFy2el9dw9T1hZHBCdlVs8YeFpeQl19TQL0XSV0erqa7Uxf8qzf+yStBVytWKLaWl7srNr18CzSaayXdnZmUacBSZ0J9PqEPft+fh5HdTQcnn4/vg1W7z+Kte7rJyuPY8MJg7Dh5EW6uGoPBSY+WAejREjhbeNXsvnT/HZQYjOjftqnqFjiUyt3VBaufvdkunxG6h7DF0aRMg3ZGzv0pQySRM60pUZuHmwu6tfB3eLAV3tQHLZtIGz4wVEitPgrw8cCDfcPNTi+NiwzGR2N7yk4wbdu8MR7sG27TRflq2GpGIGA+GDJ0v60nCej+P9WsZ/RcnPQp5WRbDE6oQWDSr33NvLMrfD3ZMWsvuoncchOSlzwWg9i2zfBfM/VM6rO4yGAcmD0cz8V1NL8x2QWDE2oQ6mnPp2q5uGhUP54/sMO1acr39HTckgJKaeTphl//fTPWPDtQ9rDbzR0C8f0T/dEmsJHRmV/1YejAXBKs3Om+ALSzaeSuL0Tm8asN2ZW5CqhKco6ME8fVuGnovv6/frhaUaW9KIU3dcyMF6V0a+GYhMa2Ks87WTV5AN5ddxivjlI+gLijRxiGdg42maOk5s8hb3f19k8wOCG72PbqUJwsuIwYOwYnSnv21vpdLrqh0Wg0et+WwwK8seyJ/vBzwlo3jjSkU3NEhvrhQE6xovtVqrxCVHgAljwub5VnOaQmT6vRi8M7Yf/ZYoztZ11lXltgcEJ2EeznpVd63ZmM7dcKj94UgY7BlpciJ8fqEyFt0UxnDp7l8FHwgqrRaPDrv2/GlmMFeOSLdADWzeh6dmgH/JxxWtaU65vbB2Lz0QJMuCnC4uM2REG+Xlj9rDqX+WBwQmSGiwboFKJ8HYn6LqZtM+w+XaSKCrZqq37pCLovw4B2gbivV0t0CrE84Nbt2XBx0WBgh+Z4Lq4D9p8txqAOzU08si4XnTdJwrCOeD6ug6zZaV/G90X2hStWrWVUt02K7cqhnHXYmMEJ1Vu6n21SCmqRsp6P64gQPy8M7WLfartkmO7F3sVFg/celL544Jz7emB8cjpeHtEJXm6u+CT1GN64Pv1Wl9zZLmP7heNwbgkG1FqkT+60eXdXF0UDE+DawovDIoPh7+2utyAl2QeDE3IacmcMaDQaPBfXAcVXK50+2dEZeXu44v+ur3pNzm1Qx+Y4/OYI7dpC8QMiFKm7k3RvD6v3YSsajQaLxvdRYD8KNEYiXy83m1QLdgQGJ1SvsW4BkTJ0Fz2UGph0b+GPvWeK8EBvdS5XUd/snD7MJms3OQKDEyIHs/abjtSqrUT29tOkWJwtLHXaMvfOxt0BK4LbSv05Eyf32PXu71vtvBouOc7GF4dg6RP90cHK4KRNYCMsGt8HK56+SaGWESnD082VgYmOmjWYgnyVWyqgvla/Zs+JSjzQpyV6tQ5A62b8R24o2gQ2UuyDe1hksCL7IXWKDLX/oo9C2DdfoiG4pVMQVk0egDYKVk+e+0AUZv2yH/ED6ld+F4MTldBoNGgfVD8SmYhIWR2CfbH0if6KfuM2xFlikQ5BzlRz6MazqtFoEBUeoOjewwK88ZkCibtqw+CEyIjAxp4ouFSG27qGOLopRFYv/eAsgYcpa54diOwLVxS/wJP6MDghMmLDi4ORVXAF3Vs6Zs0SItIXGeaHyDD7D3E5Mz8v51yOgcEJOQ17r2Hh5+XOwISInNptXUPwQO+W6NlK2hIOasHghFTv1VGdkZF1ESM4vEJETsbRScWuLhq8+4D0asBqweCEVO+JQdIXACMiIufHOidERESkKgxOiIgaAHvnbBFZg8EJEVED8Obd3dAhqDHevV+9i+0R1WDOCRFRA9C6WSP8kTDY5DaNvXhJUFp9qC/jCHwnEhERACDU3xvT74hEIw9XuNST1W3JOTE4ISIirZpFSIkciTknREREpCoMToiIiBQW6u8FAOgb0dTBLXFOHNYhIiJS2J8v3YLSyiqnXdvG0RicEBERKczDzQUebhycsBSfOSIiIlIVBidERESkKgxOiIiISFWYc0JE9VbTRh54Pq6Do5tBRDIxOCGiemvpE/3RMdjX0c0gIpkYnBBRvfPTU7HILS5lYELkpBicEFG904eFr4icGhNiiYiISFUYnBAREZGqMDghIiIiVWFwQkRERKrC4ISIiIhUhcEJERERqQqDEyIiIlIVBidERESkKgxOiIiISFUYnBAREZGqWBSczJ8/HxEREfDy8kJMTAzS09ONbrt//37cd999iIiIgEajwQcffGBpW4mIiKgBkB2cLFu2DAkJCZg5cyZ27tyJqKgoDB8+HPn5+Qa3v3LlCtq2bYs5c+YgJCTE6gYTERFR/SY7OJk3bx4mTpyI+Ph4REZGYuHChfDx8UFycrLB7fv27Yt3330XDz30EDw9Pa1uMBEREdVvslYlLi8vR0ZGBhITE7W3ubi4IC4uDmlpaYo1qqysDGVlZdq/i4qKAADFxcWKHYOIiIhsq+a6LYSQ9ThZwUlBQQGqqqoQHBysd3twcDAOHTok68CmJCUl4fXXX69ze3h4uGLHICIiIvsoKSmBv7+/5O1lBSf2kpiYiISEBO3f1dXVuHDhApo1awaNRqPYcYqLixEeHo5Tp07Bz89Psf2qSX0/R56f86vv58jzc371/RxteX5CCJSUlCAsLEzW42QFJ4GBgXB1dUVeXp7e7Xl5eYomu3p6etbJTwkICFBs/7X5+fnVyzecrvp+jjw/51ffz5Hn5/zq+zna6vzk9JjUkJUQ6+Hhgd69eyMlJUV7W3V1NVJSUhAbGyv74ERERES1yR7WSUhIwIQJE9CnTx/069cPH3zwAS5fvoz4+HgAwPjx49GiRQskJSUBuJZEe+DAAe3vZ86cQWZmJho3boz27dsreCpERERUH8gOTsaMGYNz585hxowZyM3NRXR0NNauXatNks3OzoaLy40OmbNnz6Jnz57av+fOnYu5c+di8ODBSE1Ntf4MrODp6YmZM2fW6ynO9f0ceX7Or76fI8/P+dX3c1Tj+WmE3Pk9RERERDbEtXWIiIhIVRicEBERkaowOCEiIiJVYXBCREREqtKgg5P58+cjIiICXl5eiImJQXp6uqObhKSkJPTt2xe+vr4ICgrC3XffjcOHD+ttM2TIEGg0Gr2fp556Sm+b7Oxs3H777fDx8UFQUBBeeuklVFZW6m2TmpqKXr16wdPTE+3bt8fixYvrtEfp52jWrFl12t65c2ft/aWlpZg8eTKaNWuGxo0b47777qtT9E+t51YjIiKizjlqNBpMnjwZgPO9fn/99RfuvPNOhIWFQaPRYOXKlXr3CyEwY8YMhIaGwtvbG3FxcThy5IjeNhcuXMC4cePg5+eHgIAAPPbYY7h06ZLeNnv27MHAgQPh5eWF8PBwvPPOO3Xa8uOPP6Jz587w8vJC9+7dsWbNGtltkXN+FRUVmDp1Krp3745GjRohLCwM48ePx9mzZ/X2Yeg1nzNnjirOz9w5AsCjjz5ap/0jRozQ28ZZX0MABv8fNRoN3n33Xe02an4NpVwX1PTZKaUtZokGaunSpcLDw0MkJyeL/fv3i4kTJ4qAgACRl5fn0HYNHz5cfPnll2Lfvn0iMzNTjBo1SrRq1UpcunRJu83gwYPFxIkTRU5OjvanqKhIe39lZaXo1q2biIuLE7t27RJr1qwRgYGBIjExUbvN8ePHhY+Pj0hISBAHDhwQH330kXB1dRVr167VbmOL52jmzJmia9euem0/d+6c9v6nnnpKhIeHi5SUFLFjxw7Rv39/cdNNNznFudXIz8/XO78//vhDABAbN24UQjjf67dmzRoxbdo0sXz5cgFArFixQu/+OXPmCH9/f7Fy5Uqxe/ducdddd4k2bdqIq1evarcZMWKEiIqKElu3bhWbNm0S7du3F2PHjtXeX1RUJIKDg8W4cePEvn37xPfffy+8vb3Fp59+qt1my5YtwtXVVbzzzjviwIED4rXXXhPu7u5i7969stoi5/wKCwtFXFycWLZsmTh06JBIS0sT/fr1E71799bbR+vWrcXs2bP1XlPd/1lHnp+5cxRCiAkTJogRI0botf/ChQt62zjrayiE0DuvnJwckZycLDQajTh27Jh2GzW/hlKuC2r67DTXFikabHDSr18/MXnyZO3fVVVVIiwsTCQlJTmwVXXl5+cLAOLPP//U3jZ48GAxZcoUo49Zs2aNcHFxEbm5udrbFixYIPz8/ERZWZkQQoiXX35ZdO3aVe9xY8aMEcOHD9f+bYvnaObMmSIqKsrgfYWFhcLd3V38+OOP2tsOHjwoAIi0tDTVn5sxU6ZMEe3atRPV1dVCCOd+/Wp/8FdXV4uQkBDx7rvvam8rLCwUnp6e4vvvvxdCCHHgwAEBQGzfvl27zW+//SY0Go04c+aMEEKITz75RDRp0kR7fkIIMXXqVNGpUyft3w8++KC4/fbb9doTExMjnnzyScltkXt+hqSnpwsAIisrS3tb69atxfvvv2/0MWo5PyEMn+OECRPE6NGjjT6mvr2Go0ePFrfeeqvebc70Gta+Lqjps1NKW6RokMM65eXlyMjIQFxcnPY2FxcXxMXFIS0tzYEtq6uoqAgA0LRpU73bv/32WwQGBqJbt25ITEzElStXtPelpaWhe/fueqtHDx8+HMXFxdi/f792G93zr9mm5vxt+RwdOXIEYWFhaNu2LcaNG4fs7GwAQEZGBioqKvSO2blzZ7Rq1Up7TLWfW23l5eVYsmQJ/u///k9v0Upnfv10nThxArm5uXrH8ff3R0xMjN5rFhAQgD59+mi3iYuLg4uLC7Zt26bdZtCgQfDw8NA7n8OHD+PixYuSzllKW5RQVFQEjUZTZ72vOXPmoFmzZujZsyfeffddve5yZzi/1NRUBAUFoVOnTpg0aRLOnz+v1/768hrm5eVh9erVeOyxx+rc5yyvYe3rgpo+O6W0RQpVrkpsawUFBaiqqtJ7kQAgODgYhw4dclCr6qqursZzzz2HAQMGoFu3btrbH374YbRu3RphYWHYs2cPpk6disOHD2P58uUAgNzcXIPnVnOfqW2Ki4tx9epVXLx40SbPUUxMDBYvXoxOnTohJycHr7/+OgYOHIh9+/YhNzcXHh4edT70g4ODzbZbDedmyMqVK1FYWIhHH31Ue5szv3611bTH0HF02xoUFKR3v5ubG5o2baq3TZs2berso+a+Jk2aGD1n3X2Ya4u1SktLMXXqVIwdO1ZvgbRnn30WvXr1QtOmTfH3338jMTEROTk5mDdvnlOc34gRI3DvvfeiTZs2OHbsGF599VWMHDkSaWlpcHV1rVev4VdffQVfX1/ce++9erc7y2to6Lqgps9OKW2RokEGJ85i8uTJ2LdvHzZv3qx3+xNPPKH9vXv37ggNDcXQoUNx7NgxtGvXzt7NlGXkyJHa33v06IGYmBi0bt0aP/zwA7y9vR3YMtv44osvMHLkSL3lwp359WvIKioq8OCDD0IIgQULFujdl5CQoP29R48e8PDwwJNPPomkpCRVlQQ35qGHHtL+3r17d/To0QPt2rVDamoqhg4d6sCWKS85ORnjxo2Dl5eX3u3O8hoauy7UNw1yWCcwMBCurq51sofz8vIQEhLioFbpe+aZZ/Drr79i48aNaNmypcltY2JiAABHjx4FAISEhBg8t5r7TG3j5+cHb29vuz1HAQEB6NixI44ePYqQkBCUl5ejsLDQ6DGd6dyysrKwfv16PP744ya3c+bXr2Zfpo4TEhKC/Px8vfsrKytx4cIFRV5X3fvNtcVSNYFJVlYW/vjjD7PLysfExKCyshInT5402Xbddjvy/Gpr27YtAgMD9d6Tzv4aAsCmTZtw+PBhs/+TgDpfQ2PXBTV9dkppixQNMjjx8PBA7969kZKSor2turoaKSkpiI2NdWDLrk0ze+aZZ7BixQps2LChTjeiIZmZmQCA0NBQAEBsbCz27t2r92FS84EaGRmp3Ub3/Gu2qTl/ez1Hly5dwrFjxxAaGorevXvD3d1d75iHDx9Gdna29pjOdG5ffvklgoKCcPvtt5vczplfvzZt2iAkJETvOMXFxdi2bZvea1ZYWIiMjAztNhs2bEB1dbU2MIuNjcVff/2FiooKvfPp1KkTmjRpIumcpbTFEjWByZEjR7B+/Xo0a9bM7GMyMzPh4uKiHQpR8/kZcvr0aZw/f17vPenMr2GNL774Ar1790ZUVJTZbdX0Gpq7Lqjps1NKWySRnDpbzyxdulR4enqKxYsXiwMHDognnnhCBAQE6GUyO8KkSZOEv7+/SE1N1ZvSduXKFSGEEEePHhWzZ88WO3bsECdOnBCrVq0Sbdu2FYMGDdLuo2bK2G233SYyMzPF2rVrRfPmzQ1OGXvppZfEwYMHxfz58w1OGVP6OXrhhRdEamqqOHHihNiyZYuIi4sTgYGBIj8/XwhxbQpaq1atxIYNG8SOHTtEbGysiI2NdYpz01VVVSVatWolpk6dqne7M75+JSUlYteuXWLXrl0CgJg3b57YtWuXdrbKnDlzREBAgFi1apXYs2ePGD16tMGpxD179hTbtm0TmzdvFh06dNCbhlpYWCiCg4PFI488Ivbt2yeWLl0qfHx86kzTdHNzE3PnzhUHDx4UM2fONDhN01xb5JxfeXm5uOuuu0TLli1FZmam3v9kzQyHv//+W7z//vsiMzNTHDt2TCxZskQ0b95cjB8/XhXnZ+4cS0pKxIsvvijS0tLEiRMnxPr160WvXr1Ehw4dRGlpqdO/hjWKioqEj4+PWLBgQZ3Hq/01NHddEEJdn53m2iJFgw1OhBDio48+Eq1atRIeHh6iX79+YuvWrY5ukgBg8OfLL78UQgiRnZ0tBg0aJJo2bSo8PT1F+/btxUsvvaRXJ0MIIU6ePClGjhwpvL29RWBgoHjhhRdERUWF3jYbN24U0dHRwsPDQ7Rt21Z7DF1KP0djxowRoaGhwsPDQ7Ro0UKMGTNGHD16VHv/1atXxdNPPy2aNGkifHx8xD333CNycnKc4tx0rVu3TgAQhw8f1rvdGV+/jRs3GnxPTpgwQQhxbXrk9OnTRXBwsPD09BRDhw6tc97nz58XY8eOFY0bNxZ+fn4iPj5elJSU6G2ze/ducfPNNwtPT0/RokULMWfOnDpt+eGHH0THjh2Fh4eH6Nq1q1i9erXe/VLaIuf8Tpw4YfR/sqZuTUZGhoiJiRH+/v7Cy8tLdOnSRbz99tt6F3ZHnp+5c7xy5Yq47bbbRPPmzYW7u7to3bq1mDhxYp0g1llfwxqffvqp8Pb2FoWFhXUer/bX0Nx1QQh1fXZKaYs5musnTkRERKQKDTLnhIiIiNSLwQkRERGpCoMTIiIiUhUGJ0RERKQqDE6IiIhIVRicEBERkaowOCEiIiJVYXBCREREqsLghIiIiFSFwQkRERGpCoMTIiIiUhUGJ0RERKQq/w+uLIl1v8alxQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "split_loss('train')\n",
    "split_loss('val')\n",
    "plt.plot(lossi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "Let's look at the results so far\n",
    "\n",
    "- Initial results\n",
    "```\n",
    "train 2.135850667953491\n",
    "val 2.1770708560943604\n",
    "```\n",
    "\n",
    "- Results after softmax fix\n",
    "```\n",
    "train 2.0693342685699463\n",
    "val 2.1324031352996826\n",
    "```\n",
    "\n",
    "- Results after tanh fix\n",
    "```\n",
    "train 2.0361340045928955\n",
    "val 2.102936267852783\n",
    "```\n",
    "\n",
    "- Results with kaiming init\n",
    "```\n",
    "train 2.0395455360412598\n",
    "val 2.1068387031555176\n",
    "```\n",
    "\n",
    "- Results with batch norm\n",
    "```\n",
    "train 2.1137709617614746\n",
    "val 2.1398322582244873\n",
    "```\n",
    "\n",
    "The results obtained using batch norm are comparable to the other results, but we don't see any improvements. This is to be expected for such a simple neural network with essentially a single hidden layer. We were able to manually calculate the scale down factor of W1 which performs better than batch norm. but that would be tedious and difficult to do in a neural network with 100s of hidden layers and that's where the batch norm really shines.\n",
    "\n",
    "For larger neural networks it is customary to take a hidden layer and append a batch normalization layer on top of that to control the preactivations at every point of neural network.\n",
    "\n",
    "Another interesting thing about batch norm is that now our logits and hidden state activations are not just the function of input example but also function of other examples of in that batch. What this does is that it adds a bit of entropy to the example and acts as a form of \"data augmentation\" which makes it harder for neural network to overfit to the example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}