2024-04-02_Ufert_Homework_Session_5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1fd0e269-9883-4469-bb22-84d6df3c0ba0",
   "metadata": {},
   "source": [
    "## Session 5: Large Language Models Part 2"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28238f2d-4d14-4b83-99e9-7290b0115600",
   "metadata": {},
   "source": [
    "My homework can be found at the bottom of this notebook, under a dedicated section called \"Homework\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "6ef512c6-d86f-4b55-84b5-cf566f995444",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-04-07 00:32:55.285503: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/h5py/__init__.py:36: UserWarning: h5py is running against HDF5 1.12.2 when it was built against 1.12.1, this may cause problems\n",
      "  _warn((\"h5py is running against HDF5 {0} when it was built against {1}, \"\n",
      "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/transformers/generation/utils.py:1186: UserWarning: You have modified the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/main_classes/text_generation)\n",
      "  warnings.warn(\n",
      "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[{'generated_text': 'My dog really wanted to see if I\\'m going to die,\" said his wife, Karen, 44'},\n",
       " {'generated_text': 'My dog really wanted to know what my dog was doing, so she started playing a game of ball'},\n",
       " {'generated_text': 'My dog really wanted to play on the train. As we started our journey, his owner had given'},\n",
       " {'generated_text': \"My dog really wanted to sit on the bed and do a little leg press but he wasn't ready\"},\n",
       " {'generated_text': 'My dog really wanted to be with me,\" I said. \"But I couldn\\'t because my dog'}]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from transformers import AutoTokenizer,AutoModelForCausalLM, AutoConfig\n",
    "input_text = \"My dog really wanted to\"\n",
    "from transformers import pipeline\n",
    "generator = pipeline(\"text-generation\", model=\"openai-community/gpt2\")\n",
    "generator(input_text, max_length=20, num_return_sequences=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9fc95610-b277-46bc-b76b-4faa1a691781",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n",
      "Defaulting to user installation because normal site-packages is not writeable\n",
      "Requirement already satisfied: accelerate in ./.local/lib/python3.10/site-packages (0.29.1)\n",
      "Requirement already satisfied: numpy>=1.17 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from accelerate) (1.23.5)\n",
      "Requirement already satisfied: packaging>=20.0 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from accelerate) (21.3)\n",
      "Requirement already satisfied: psutil in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from accelerate) (5.9.4)\n",
      "Requirement already satisfied: pyyaml in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from accelerate) (6.0)\n",
      "Requirement already satisfied: torch>=1.10.0 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from accelerate) (1.13.0a0+git49444c3)\n",
      "Requirement already satisfied: huggingface-hub in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from accelerate) (0.12.0)\n",
      "Requirement already satisfied: safetensors>=0.3.1 in ./.local/lib/python3.10/site-packages (from accelerate) (0.4.2)\n",
      "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from packaging>=20.0->accelerate) (3.0.9)\n",
      "Requirement already satisfied: typing-extensions in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from torch>=1.10.0->accelerate) (4.4.0)\n",
      "Requirement already satisfied: filelock in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from huggingface-hub->accelerate) (3.9.0)\n",
      "Requirement already satisfied: requests in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from huggingface-hub->accelerate) (2.28.1)\n",
      "Requirement already satisfied: tqdm>=4.42.1 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from huggingface-hub->accelerate) (4.64.1)\n",
      "Requirement already satisfied: charset-normalizer<3,>=2 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (2.0.4)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (3.4)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (1.26.13)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from requests->huggingface-hub->accelerate) (2022.12.7)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install accelerate -U"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "0af5b8ed-74cc-4603-89d0-7c2a21371d12",
   "metadata": {},
   "outputs": [],
   "source": [
    "from transformers import TextDataset,DataCollatorForLanguageModeling\n",
    "\n",
    "# Data collators are objects that will form a batch by using a list of dataset elements as input. These elements are of the same type as the elements of train_dataset or eval_dataset.\n",
    "\n",
    "def load_dataset(train_path,test_path,tokenizer):\n",
    "    train_dataset = TextDataset(\n",
    "          tokenizer=tokenizer,\n",
    "          file_path=train_path,\n",
    "          block_size=128) \n",
    "    test_dataset = TextDataset(\n",
    "          tokenizer=tokenizer,\n",
    "          file_path=test_path,\n",
    "          block_size=128)   \n",
    "    data_collator = DataCollatorForLanguageModeling(\n",
    "        tokenizer=tokenizer, mlm=False,\n",
    "    )\n",
    "    return train_dataset,test_dataset,data_collator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "e3b298c6-2fea-469a-9586-932990231a9a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages/transformers/data/datasets/language_modeling.py:54: FutureWarning: This dataset will be removed from the library soon, preprocessing should be handled with the 🤗 Datasets library. You can have a look at this example script for pointers: https://github.com/huggingface/transformers/blob/main/examples/pytorch/language-modeling/run_mlm.py\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "from transformers import AutoTokenizer,AutoModelForCausalLM\n",
    "\n",
    "tokenizer = AutoTokenizer.from_pretrained(\"openai-community/gpt2\")\n",
    "model = AutoModelForCausalLM.from_pretrained(\"openai-community/gpt2\")\n",
    "\n",
    "train_dataset,test_dataset,data_collator = load_dataset('ai-science-training-series/05_llm_part2/dataset/train_input.txt','ai-science-training-series/05_llm_part2/dataset/test_input.txt', tokenizer)\n",
    "\n",
    "from transformers import Trainer, TrainingArguments\n",
    "\n",
    "training_args = TrainingArguments(\n",
    "    output_dir=\"./gpt2\", #The output directory\n",
    "    overwrite_output_dir=True, #overwrite the content of the output directory\n",
    "    num_train_epochs=3, # number of training epochs\n",
    "    per_device_train_batch_size=32, # batch size for training\n",
    "    per_device_eval_batch_size=64,  # batch size for evaluation\n",
    "    eval_steps = 40, # Number of update steps between two evaluations.\n",
    "    save_steps=80, # after # steps model is saved \n",
    "    warmup_steps=50,# number of warmup steps for learning rate scheduler\n",
    "    prediction_loss_only=True,\n",
    "    )\n",
    "\n",
    "trainer = Trainer(\n",
    "    model=model,\n",
    "    args=training_args,\n",
    "    data_collator=data_collator,\n",
    "    train_dataset=train_dataset,\n",
    "    eval_dataset=test_dataset,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "9e058cbc-df41-45f1-bcfb-ec72d7a5a6b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "loading configuration file config.json from cache at /home/julian_ufert/.cache/huggingface/hub/models--openai-community--gpt2/snapshots/607a30d783dfa663caf39e06633721c8d4cfcd7e/config.json\n",
      "Model config GPT2Config {\n",
      "  \"activation_function\": \"gelu_new\",\n",
      "  \"architectures\": [\n",
      "    \"GPT2LMHeadModel\"\n",
      "  ],\n",
      "  \"attn_pdrop\": 0.1,\n",
      "  \"bos_token_id\": 50256,\n",
      "  \"embd_pdrop\": 0.1,\n",
      "  \"eos_token_id\": 50256,\n",
      "  \"initializer_range\": 0.02,\n",
      "  \"layer_norm_epsilon\": 1e-05,\n",
      "  \"model_type\": \"gpt2\",\n",
      "  \"n_ctx\": 1024,\n",
      "  \"n_embd\": 768,\n",
      "  \"n_head\": 12,\n",
      "  \"n_inner\": null,\n",
      "  \"n_layer\": 12,\n",
      "  \"n_positions\": 1024,\n",
      "  \"reorder_and_upcast_attn\": false,\n",
      "  \"resid_pdrop\": 0.1,\n",
      "  \"scale_attn_by_inverse_layer_idx\": false,\n",
      "  \"scale_attn_weights\": true,\n",
      "  \"summary_activation\": null,\n",
      "  \"summary_first_dropout\": 0.1,\n",
      "  \"summary_proj_to_labels\": true,\n",
      "  \"summary_type\": \"cls_index\",\n",
      "  \"summary_use_proj\": true,\n",
      "  \"task_specific_params\": {\n",
      "    \"text-generation\": {\n",
      "      \"do_sample\": true,\n",
      "      \"max_length\": 50\n",
      "    }\n",
      "  },\n",
      "  \"transformers_version\": \"4.26.0\",\n",
      "  \"use_cache\": true,\n",
      "  \"vocab_size\": 50257\n",
      "}\n",
      "\n",
      "loading weights file model.safetensors from cache at /home/julian_ufert/.cache/huggingface/hub/models--openai-community--gpt2/snapshots/607a30d783dfa663caf39e06633721c8d4cfcd7e/model.safetensors\n",
      "Generate config GenerationConfig {\n",
      "  \"bos_token_id\": 50256,\n",
      "  \"eos_token_id\": 50256,\n",
      "  \"transformers_version\": \"4.26.0\"\n",
      "}\n",
      "\n",
      "All model checkpoint weights were used when initializing GPT2LMHeadModel.\n",
      "\n",
      "All the weights of GPT2LMHeadModel were initialized from the model checkpoint at openai-community/gpt2.\n",
      "If your task is similar to the task the model of the checkpoint was trained on, you can already use GPT2LMHeadModel for predictions without further training.\n",
      "loading configuration file generation_config.json from cache at /home/julian_ufert/.cache/huggingface/hub/models--openai-community--gpt2/snapshots/607a30d783dfa663caf39e06633721c8d4cfcd7e/generation_config.json\n",
      "Generate config GenerationConfig {\n",
      "  \"_from_model_config\": true,\n",
      "  \"bos_token_id\": 50256,\n",
      "  \"eos_token_id\": 50256,\n",
      "  \"transformers_version\": \"4.26.0\"\n",
      "}\n",
      "\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GPT2LMHeadModel(\n",
      "  (transformer): GPT2Model(\n",
      "    (wte): Embedding(50257, 768)\n",
      "    (wpe): Embedding(1024, 768)\n",
      "    (drop): Dropout(p=0.1, inplace=False)\n",
      "    (h): ModuleList(\n",
      "      (0): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (1): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (2): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (3): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (4): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (5): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (6): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (7): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (8): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (9): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (10): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "      (11): GPT2Block(\n",
      "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (attn): GPT2Attention(\n",
      "          (c_attn): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
      "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "        (mlp): GPT2MLP(\n",
      "          (c_fc): Conv1D()\n",
      "          (c_proj): Conv1D()\n",
      "          (act): NewGELUActivation()\n",
      "          (dropout): Dropout(p=0.1, inplace=False)\n",
      "        )\n",
      "      )\n",
      "    )\n",
      "    (ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
      "  )\n",
      "  (lm_head): Linear(in_features=768, out_features=50257, bias=False)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "from transformers import GPT2Tokenizer, GPT2LMHeadModel\n",
    "model = GPT2LMHeadModel.from_pretrained('openai-community/gpt2')\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "fb3211df-57da-4925-9002-ab58577698d0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<torch._C.Generator at 0x1552aa230bf0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## IMPORTS\n",
    "\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "from torch.nn import functional as F\n",
    "torch.manual_seed(1337)\n",
    "# hyperparameters\n",
    "batch_size = 16 # how many independent sequences will we process in parallel?\n",
    "block_size = 32 # what is the maximum context length for predictions?\n",
    "max_iters = 5000\n",
    "eval_interval = 100\n",
    "learning_rate = 1e-3\n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "eval_iters = 200\n",
    "n_embd = 64\n",
    "n_head = 4 ## so head_size = 16\n",
    "n_layer = 4\n",
    "dropout = 0.0\n",
    "# ------------\n",
    "\n",
    "torch.manual_seed(1337)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7bcd1c40-b6a1-4ad1-a1f2-d3f4a5d42d29",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "from torch.nn import functional as F\n",
    "torch.manual_seed(1337)\n",
    "B,T,C = 4,8,32 # batch, time, channels\n",
    "x = torch.randn(B,T,C)\n",
    "\n",
    "# Here we want the wei to be data dependent - ie gather info from the past but in a data dependant way\n",
    "\n",
    "head_size = 16\n",
    "key = nn.Linear(C, head_size, bias=False)\n",
    "query = nn.Linear(C, head_size, bias=False)\n",
    "value = nn.Linear(C, head_size, bias=False)\n",
    "k = key(x)   # (B, T, 16) # each token here (totally B*T) produce a key and query in parallel and independently\n",
    "q = query(x) # (B, T, 16)\n",
    "v = value(x)\n",
    "\n",
    "wei =  q @ k.transpose(-2, -1) * head_size**-0.5 # (B, T, 16) @ (B, 16, T) ---> (B, T, T). #\n",
    "wei = F.softmax(wei, dim=-1) # exponentiate and normalize giving a nice distibution that sums to 1 and\n",
    "                             # now it tells us that in a data dependent manner how much of info to aggregate from\n",
    "\n",
    "out = wei @ v # aggregate the attention scores and value vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "a4f3f3bd-3b8c-4559-b41f-dcdbbf7374e1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[ 0.0618, -0.0091, -0.3488,  0.3208,  0.2971, -0.1573, -0.0561,  0.1068,\n",
      "          0.0368,  0.0139, -0.0017,  0.3110,  0.1404, -0.0158,  0.1853,  0.4290],\n",
      "        [ 0.1578, -0.0971, -0.4256,  0.3538,  0.3621, -0.2392, -0.0536,  0.1759,\n",
      "          0.1115,  0.0282, -0.0649,  0.3641,  0.1928,  0.0261,  0.2162,  0.3758],\n",
      "        [ 0.1293,  0.0759, -0.2946,  0.2292,  0.2215, -0.0710, -0.0107,  0.1616,\n",
      "         -0.0930, -0.0877,  0.0567,  0.1899,  0.0311, -0.0894,  0.0309,  0.5471],\n",
      "        [ 0.1247,  0.1400, -0.2436,  0.1819,  0.1976,  0.0338, -0.0028,  0.1124,\n",
      "         -0.1477, -0.0748,  0.0650,  0.1392, -0.0314, -0.0989,  0.0613,  0.5433],\n",
      "        [ 0.0667,  0.1845, -0.2135,  0.2813,  0.2064,  0.0873,  0.0084,  0.2055,\n",
      "         -0.1130, -0.1466,  0.0459,  0.1923, -0.0275, -0.1107,  0.0065,  0.4674],\n",
      "        [ 0.1924,  0.1693, -0.1568,  0.2284,  0.1620,  0.0737,  0.0443,  0.2519,\n",
      "         -0.1912, -0.1979,  0.0832,  0.0713, -0.0826, -0.0848, -0.1047,  0.6089],\n",
      "        [ 0.1184,  0.0884, -0.2652,  0.2560,  0.1840,  0.0284, -0.0621,  0.1181,\n",
      "         -0.0880,  0.0104,  0.1123,  0.1850,  0.0369, -0.0730,  0.0663,  0.5242],\n",
      "        [ 0.1243,  0.0453, -0.3412,  0.2709,  0.2335, -0.0948, -0.0421,  0.2143,\n",
      "         -0.0330, -0.0313,  0.0520,  0.2378,  0.1084, -0.0959,  0.0300,  0.4707]],\n",
      "       grad_fn=<SelectBackward0>)\n"
     ]
    }
   ],
   "source": [
    "print(out[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "ba97aae5-589b-47f6-b534-0f5f27d76bd7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...\n",
      "To disable this warning, you can either:\n",
      "\t- Avoid using `tokenizers` before the fork if possible\n",
      "\t- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)\n",
      "Defaulting to user installation because normal site-packages is not writeable\n",
      "Requirement already satisfied: bertviz in ./.local/lib/python3.10/site-packages (1.4.0)\n",
      "Requirement already satisfied: transformers>=2.0 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from bertviz) (4.26.0)\n",
      "Requirement already satisfied: torch>=1.0 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from bertviz) (1.13.0a0+git49444c3)\n",
      "Requirement already satisfied: tqdm in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from bertviz) (4.64.1)\n",
      "Requirement already satisfied: boto3 in ./.local/lib/python3.10/site-packages (from bertviz) (1.34.75)\n",
      "Requirement already satisfied: requests in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from bertviz) (2.28.1)\n",
      "Requirement already satisfied: regex in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from bertviz) (2022.10.31)\n",
      "Requirement already satisfied: sentencepiece in ./.local/lib/python3.10/site-packages (from bertviz) (0.2.0)\n",
      "Requirement already satisfied: typing-extensions in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from torch>=1.0->bertviz) (4.4.0)\n",
      "Requirement already satisfied: filelock in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from transformers>=2.0->bertviz) (3.9.0)\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.11.0 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from transformers>=2.0->bertviz) (0.12.0)\n",
      "Requirement already satisfied: numpy>=1.17 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from transformers>=2.0->bertviz) (1.23.5)\n",
      "Requirement already satisfied: packaging>=20.0 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from transformers>=2.0->bertviz) (21.3)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from transformers>=2.0->bertviz) (6.0)\n",
      "Requirement already satisfied: tokenizers!=0.11.3,<0.14,>=0.11.1 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from transformers>=2.0->bertviz) (0.13.2)\n",
      "Requirement already satisfied: botocore<1.35.0,>=1.34.75 in ./.local/lib/python3.10/site-packages (from boto3->bertviz) (1.34.75)\n",
      "Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in ./.local/lib/python3.10/site-packages (from boto3->bertviz) (1.0.1)\n",
      "Requirement already satisfied: s3transfer<0.11.0,>=0.10.0 in ./.local/lib/python3.10/site-packages (from boto3->bertviz) (0.10.1)\n",
      "Requirement already satisfied: charset-normalizer<3,>=2 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from requests->bertviz) (2.0.4)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from requests->bertviz) (3.4)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from requests->bertviz) (1.26.13)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from requests->bertviz) (2022.12.7)\n",
      "Requirement already satisfied: python-dateutil<3.0.0,>=2.1 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from botocore<1.35.0,>=1.34.75->boto3->bertviz) (2.8.2)\n",
      "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from packaging>=20.0->transformers>=2.0->bertviz) (3.0.9)\n",
      "Requirement already satisfied: six>=1.5 in /soft/datascience/conda/2023-01-10/mconda3/lib/python3.10/site-packages (from python-dateutil<3.0.0,>=2.1->botocore<1.35.0,>=1.34.75->boto3->bertviz) (1.16.0)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.2.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.0\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install bertviz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6c54483f-afd3-4ef5-a9c7-0305061f9cf5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<script src=\"https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js\"></script>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "      \n",
       "        <div id=\"bertviz-5a257cb0119e4b7ea970ea74266027b3\" style=\"font-family:'Helvetica Neue', Helvetica, Arial, sans-serif;\">\n",
       "            <span style=\"user-select:none\">\n",
       "                \n",
       "            </span>\n",
       "            <div id='vis'></div>\n",
       "        </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/javascript": [
       "/**\n",
       " * @fileoverview Transformer Visualization D3 javascript code.\n",
       " *\n",
       " * Based on: https://github.com/tensorflow/tensor2tensor/blob/master/tensor2tensor/visualization/attention.js\n",
       " *\n",
       " * Change log:\n",
       " *\n",
       " * 02/01/19  Jesse Vig   Initial implementation\n",
       " * 12/31/20  Jesse Vig   Support multiple visualizations in single notebook.\n",
       " * 01/19/21  Jesse Vig   Support light/dark modes\n",
       " * 02/06/21  Jesse Vig   Move require config from separate jupyter notebook step\n",
       " * 05/03/21  Jesse Vig   Adjust visualization height dynamically\n",
       " * 03/23/22  Daniel SC   Update requirement URLs for d3 and jQuery (source of bug not allowing end result to be displayed on browsers)\n",
       " **/\n",
       "\n",
       "require.config({\n",
       "  paths: {\n",
       "      d3: 'https://cdnjs.cloudflare.com/ajax/libs/d3/5.7.0/d3.min',\n",
       "    jquery: 'https://cdnjs.cloudflare.com/ajax/libs/jquery/2.0.0/jquery.min',\n",
       "  }\n",
       "});\n",
       "\n",
       "requirejs(['jquery', 'd3'], function($, d3) {\n",
       "\n",
       "        const params = {\"attention\": [{\"name\": null, \"attn\": [[[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.961219847202301, 0.038780149072408676, 0.0, 0.0, 0.0, 0.0], [0.7466979026794434, 0.11987314373254776, 0.1334289014339447, 0.0, 0.0, 0.0], [0.5885030031204224, 0.13792067766189575, 0.212137371301651, 0.06143897399306297, 0.0, 0.0], [0.6570857763290405, 0.08996301889419556, 0.12751281261444092, 0.08361563086509705, 0.041822850704193115, 0.0], [0.2728874385356903, 0.11203353852033615, 0.1663985401391983, 0.08467111736536026, 0.16952736675739288, 0.19448210299015045]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.010616563260555267, 0.9893833994865417, 0.0, 0.0, 0.0, 0.0], [0.0024677535984665155, 0.008448007516562939, 0.9890841841697693, 0.0, 0.0, 0.0], [0.0001232847134815529, 0.0018733182223513722, 0.013126976788043976, 0.9848763942718506, 0.0, 0.0], [0.0010669564362615347, 0.001136627048254013, 0.003034998197108507, 0.0015735096530988812, 0.9931879043579102, 0.0], [0.00019791982776951045, 0.0010528112761676311, 0.0015437351539731026, 0.0009642760851420462, 3.4924432839034125e-05, 0.9962062835693359]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.47578439116477966, 0.524215579032898, 0.0, 0.0, 0.0, 0.0], [0.5906045436859131, 0.2486611008644104, 0.16073434054851532, 0.0, 0.0, 0.0], [0.5529289841651917, 0.18856702744960785, 0.14457571506500244, 0.11392831057310104, 0.0, 0.0], [0.45094072818756104, 0.16486799716949463, 0.17318038642406464, 0.11748014390468597, 0.09353074431419373, 0.0], [0.4257245659828186, 0.1732865273952484, 0.15651953220367432, 0.07022649794816971, 0.0808701142668724, 0.09337282180786133]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.6133623123168945, 0.38663768768310547, 0.0, 0.0, 0.0, 0.0], [0.06098509579896927, 0.03253461793065071, 0.9064802527427673, 0.0, 0.0, 0.0], [0.006717085838317871, 0.0004012881254311651, 0.7572958469390869, 0.23558568954467773, 0.0, 0.0], [0.03722766041755676, 0.002948855282738805, 0.10081092268228531, 0.04142269119620323, 0.8175898790359497, 0.0], [0.04989781975746155, 0.00030758307548239827, 0.0024198265746235847, 0.0034334994852542877, 0.0006823898293077946, 0.9432588815689087]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9489555954933167, 0.051044441759586334, 0.0, 0.0, 0.0, 0.0], [0.6821408867835999, 0.1395241767168045, 0.17833495140075684, 0.0, 0.0, 0.0], [0.20366324484348297, 0.05641487240791321, 0.06399301439523697, 0.6759288311004639, 0.0, 0.0], [0.3419547975063324, 0.06725440919399261, 0.07926183938980103, 0.1783619523048401, 0.3331669867038727, 0.0], [0.09464015811681747, 0.0074282134883105755, 0.006983973551541567, 0.0071843694895505905, 0.018724264577031136, 0.865039050579071]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.33834606409072876, 0.6616539359092712, 0.0, 0.0, 0.0, 0.0], [0.07855993509292603, 0.006165449041873217, 0.9152746200561523, 0.0, 0.0, 0.0], [0.01677597686648369, 0.0004037705948576331, 0.003340460592880845, 0.9794798493385315, 0.0, 0.0], [0.027600426226854324, 0.00044415233423933387, 0.0006541680195368826, 0.0002266185765620321, 0.971074640750885, 0.0], [0.010248198173940182, 3.701553578139283e-05, 0.00016064041119534522, 2.7341819077264518e-05, 1.0187304724240676e-05, 0.98951655626297]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.982503354549408, 0.017496665939688683, 0.0, 0.0, 0.0, 0.0], [0.8874197006225586, 0.05467939004302025, 0.05790085718035698, 0.0, 0.0, 0.0], [0.6849910616874695, 0.1228068619966507, 0.04972026124596596, 0.14248186349868774, 0.0, 0.0], [0.6015856862068176, 0.09881888329982758, 0.07070108503103256, 0.16652540862560272, 0.06236903741955757, 0.0], [0.3232504427433014, 0.12567411363124847, 0.04432179778814316, 0.07076980918645859, 0.06606649607419968, 0.36991727352142334]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9191647171974182, 0.0808352455496788, 0.0, 0.0, 0.0, 0.0], [0.45986413955688477, 0.39703112840652466, 0.14310479164123535, 0.0, 0.0, 0.0], [0.3003872334957123, 0.22181738913059235, 0.38161516189575195, 0.09618020057678223, 0.0, 0.0], [0.18963925540447235, 0.1376371532678604, 0.20173484086990356, 0.23632164299488068, 0.23466713726520538, 0.0], [0.15410441160202026, 0.09489496797323227, 0.11902562528848648, 0.10277965664863586, 0.4317220449447632, 0.09747327119112015]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.364999920129776, 0.6350001096725464, 0.0, 0.0, 0.0, 0.0], [0.24595215916633606, 0.5519201755523682, 0.20212766528129578, 0.0, 0.0, 0.0], [0.2721358835697174, 0.40738627314567566, 0.25186213850975037, 0.06861574947834015, 0.0, 0.0], [0.10242555290460587, 0.16683615744113922, 0.524804949760437, 0.05445462837815285, 0.15147870779037476, 0.0], [0.25029507279396057, 0.22198128700256348, 0.18899968266487122, 0.10677118599414825, 0.1303267478942871, 0.10162602365016937]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.6990506649017334, 0.3009493350982666, 0.0, 0.0, 0.0, 0.0], [0.5107942819595337, 0.2948642075061798, 0.1943415403366089, 0.0, 0.0, 0.0], [0.4604707360267639, 0.2805190980434418, 0.19174803793430328, 0.0672621801495552, 0.0, 0.0], [0.37648412585258484, 0.21120662987232208, 0.20214538276195526, 0.10207021236419678, 0.10809355974197388, 0.0], [0.30138441920280457, 0.20456179976463318, 0.18250338733196259, 0.11019382625818253, 0.1629127413034439, 0.03844383731484413]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.7131582498550415, 0.2868417799472809, 0.0, 0.0, 0.0, 0.0], [0.4058799147605896, 0.18063297867774963, 0.41348710656166077, 0.0, 0.0, 0.0], [0.265546053647995, 0.1698586493730545, 0.3358593285083771, 0.228736013174057, 0.0, 0.0], [0.31385406851768494, 0.1831669807434082, 0.14928358793258667, 0.05377671495079994, 0.29991865158081055, 0.0], [0.20466560125350952, 0.18731118738651276, 0.15959151089191437, 0.06381776183843613, 0.03642302006483078, 0.34819093346595764]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.6586242914199829, 0.3413757383823395, 0.0, 0.0, 0.0, 0.0], [0.5917776226997375, 0.3160035014152527, 0.0922188088297844, 0.0, 0.0, 0.0], [0.5477152466773987, 0.23586955666542053, 0.061456020921468735, 0.1549593061208725, 0.0, 0.0], [0.4587061107158661, 0.22439992427825928, 0.07887422293424606, 0.0992034301161766, 0.13881628215312958, 0.0], [0.32743722200393677, 0.19600819051265717, 0.068057119846344, 0.0892510637640953, 0.11618079245090485, 0.20306548476219177]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9961552023887634, 0.0038448425475507975, 0.0, 0.0, 0.0, 0.0], [0.8594854474067688, 0.06906110048294067, 0.07145342975854874, 0.0, 0.0, 0.0], [0.3800053000450134, 0.04127567633986473, 0.5496612787246704, 0.029057776555418968, 0.0, 0.0], [0.21445226669311523, 0.05088742449879646, 0.4317440092563629, 0.25869303941726685, 0.044223275035619736, 0.0], [0.11175256222486496, 0.017593080177903175, 0.027507441118359566, 0.04086771607398987, 0.7754669785499573, 0.026812179014086723]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9285967946052551, 0.07140326499938965, 0.0, 0.0, 0.0, 0.0], [0.6077286005020142, 0.3121427297592163, 0.08012867718935013, 0.0, 0.0, 0.0], [0.4942909777164459, 0.28503698110580444, 0.11849315464496613, 0.10217894613742828, 0.0, 0.0], [0.4183879494667053, 0.23117904365062714, 0.0834062322974205, 0.11365949362516403, 0.1533672958612442, 0.0], [0.42215850949287415, 0.12917140126228333, 0.08740927278995514, 0.1016375944018364, 0.21230268478393555, 0.04732053726911545]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9786475896835327, 0.02135237120091915, 0.0, 0.0, 0.0, 0.0], [0.7749121785163879, 0.06510371714830399, 0.15998409688472748, 0.0, 0.0, 0.0], [0.6484923362731934, 0.07483134418725967, 0.14751605689525604, 0.12916021049022675, 0.0, 0.0], [0.5224639773368835, 0.06921815127134323, 0.13823404908180237, 0.1110658198595047, 0.15901805460453033, 0.0], [0.3964517116546631, 0.07325823605060577, 0.12938153743743896, 0.1064242571592331, 0.14864002168178558, 0.1458442211151123]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.5525906085968018, 0.44740936160087585, 0.0, 0.0, 0.0, 0.0], [0.5585009455680847, 0.2176259458065033, 0.22387312352657318, 0.0, 0.0, 0.0], [0.5143128633499146, 0.15964674949645996, 0.15491968393325806, 0.1711207628250122, 0.0, 0.0], [0.5039961338043213, 0.11401888728141785, 0.11974027007818222, 0.12552587687969208, 0.13671889901161194, 0.0], [0.5061842799186707, 0.08567393571138382, 0.08903021365404129, 0.09759818762540817, 0.1027572825551033, 0.11875619739294052]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9242545366287231, 0.07574543356895447, 0.0, 0.0, 0.0, 0.0], [0.8257425427436829, 0.07932533323764801, 0.09493216127157211, 0.0, 0.0, 0.0], [0.7306380271911621, 0.0857183039188385, 0.08043931424617767, 0.10320431739091873, 0.0, 0.0], [0.6383238434791565, 0.07886394113302231, 0.07815027981996536, 0.08758097141981125, 0.1170809343457222, 0.0], [0.5552157163619995, 0.07409121096134186, 0.06834889203310013, 0.07778600603342056, 0.09999319165945053, 0.12456497550010681]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8578913807868958, 0.14210854470729828, 0.0, 0.0, 0.0, 0.0], [0.6423038244247437, 0.166290283203125, 0.19140593707561493, 0.0, 0.0, 0.0], [0.5530979633331299, 0.10609274357557297, 0.07821257412433624, 0.26259663701057434, 0.0, 0.0], [0.40121692419052124, 0.12223611027002335, 0.1934729963541031, 0.14164622128009796, 0.14142780005931854, 0.0], [0.40212565660476685, 0.18450751900672913, 0.07516805827617645, 0.05849048122763634, 0.1444634348154068, 0.13524490594863892]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9791558980941772, 0.020844051614403725, 0.0, 0.0, 0.0, 0.0], [0.8829841613769531, 0.06233249977231026, 0.05468335747718811, 0.0, 0.0, 0.0], [0.8105455040931702, 0.08617085963487625, 0.07321777194738388, 0.03006584383547306, 0.0, 0.0], [0.6819812059402466, 0.04990820586681366, 0.08296552300453186, 0.08369525521993637, 0.10144983977079391, 0.0], [0.4056689441204071, 0.07337666302919388, 0.08601408451795578, 0.061709366738796234, 0.13226434588432312, 0.2409665435552597]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9670190811157227, 0.03298088163137436, 0.0, 0.0, 0.0, 0.0], [0.8449064493179321, 0.0851450264453888, 0.06994850933551788, 0.0, 0.0, 0.0], [0.7123572826385498, 0.07896047830581665, 0.055410757660865784, 0.15327158570289612, 0.0, 0.0], [0.6402613520622253, 0.0739755630493164, 0.044393062591552734, 0.14322125911712646, 0.09814881533384323, 0.0], [0.5073903799057007, 0.07523059099912643, 0.07754647731781006, 0.11362491548061371, 0.13947951793670654, 0.08672808855772018]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8487569093704224, 0.1512431502342224, 0.0, 0.0, 0.0, 0.0], [0.8415648937225342, 0.12107233703136444, 0.03736274689435959, 0.0, 0.0, 0.0], [0.7505517601966858, 0.11348944902420044, 0.06179959326982498, 0.07415912300348282, 0.0, 0.0], [0.6614719033241272, 0.10242646187543869, 0.052934251725673676, 0.07529708743095398, 0.10787025839090347, 0.0], [0.6014202237129211, 0.11340376734733582, 0.05631929263472557, 0.07096721231937408, 0.10906282067298889, 0.04882663115859032]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9445484280586243, 0.05545158311724663, 0.0, 0.0, 0.0, 0.0], [0.8874568939208984, 0.05474215745925903, 0.0578010231256485, 0.0, 0.0, 0.0], [0.8281888961791992, 0.06895001977682114, 0.059034693986177444, 0.0438263975083828, 0.0, 0.0], [0.6429892778396606, 0.0674755647778511, 0.11629703640937805, 0.05417950078845024, 0.11905858665704727, 0.0], [0.7367823719978333, 0.056119054555892944, 0.06857288628816605, 0.034219540655612946, 0.0787537544965744, 0.02555238828063011]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0002913394710049033, 0.9997085928916931, 0.0, 0.0, 0.0, 0.0], [0.0007981209782883525, 0.5288336873054504, 0.4703682065010071, 0.0, 0.0, 0.0], [0.0007648481405340135, 0.34519824385643005, 0.3085267245769501, 0.34551018476486206, 0.0, 0.0], [0.0010283143492415547, 0.241359144449234, 0.23320138454437256, 0.2555713355541229, 0.2688397467136383, 0.0], [0.0009746829164214432, 0.17789699137210846, 0.16743157804012299, 0.1858760118484497, 0.18734444677829742, 0.28047630190849304]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.824492871761322, 0.17550717294216156, 0.0, 0.0, 0.0, 0.0], [0.12386877834796906, 0.044499922543764114, 0.8316312432289124, 0.0, 0.0, 0.0], [0.07924355566501617, 0.01296587660908699, 0.0015277155907824636, 0.9062628149986267, 0.0, 0.0], [0.08806384354829788, 0.0213409923017025, 0.0028886159416288137, 0.002845379989594221, 0.884861171245575, 0.0], [0.09983218461275101, 0.03363388776779175, 0.0054999832063913345, 0.002433052286505699, 0.0015082412865012884, 0.8570926189422607]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9646892547607422, 0.03531072288751602, 0.0, 0.0, 0.0, 0.0], [0.7529157400131226, 0.08733473718166351, 0.15974950790405273, 0.0, 0.0, 0.0], [0.4202282726764679, 0.09195102006196976, 0.23549850285053253, 0.25232216715812683, 0.0, 0.0], [0.30848920345306396, 0.05908140912652016, 0.38391315937042236, 0.15659146010875702, 0.09192468225955963, 0.0], [0.44790443778038025, 0.04329312965273857, 0.0796918049454689, 0.11081931740045547, 0.22124572098255157, 0.09704558551311493]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.991096019744873, 0.008904009126126766, 0.0, 0.0, 0.0, 0.0], [0.9697675704956055, 0.026084503158926964, 0.004147922620177269, 0.0, 0.0, 0.0], [0.9082901477813721, 0.033206019550561905, 0.00942116230726242, 0.049082688987255096, 0.0, 0.0], [0.8949133157730103, 0.05544555187225342, 0.005577624775469303, 0.03150692582130432, 0.012556522153317928, 0.0], [0.8497740030288696, 0.028890123590826988, 0.0036647915840148926, 0.03751987963914871, 0.038427725434303284, 0.04172350466251373]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9984525442123413, 0.0015474462416023016, 0.0, 0.0, 0.0, 0.0], [0.48947831988334656, 0.4812193810939789, 0.029302269220352173, 0.0, 0.0, 0.0], [0.11772153526544571, 0.13121186196804047, 0.6702314615249634, 0.08083520829677582, 0.0, 0.0], [0.13043689727783203, 0.04068669304251671, 0.2652038037776947, 0.4114362895488739, 0.15223638713359833, 0.0], [0.12661904096603394, 0.03275119513273239, 0.03567872568964958, 0.06039190664887428, 0.6021825075149536, 0.1423766165971756]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9805176854133606, 0.019482342526316643, 0.0, 0.0, 0.0, 0.0], [0.7948849201202393, 0.12061909586191177, 0.08449601382017136, 0.0, 0.0, 0.0], [0.5612356066703796, 0.15743127465248108, 0.20339730381965637, 0.0779358446598053, 0.0, 0.0], [0.42583736777305603, 0.10742014646530151, 0.15123659372329712, 0.08755031228065491, 0.22795552015304565, 0.0], [0.24752654135227203, 0.024188270792365074, 0.03039524517953396, 0.08586956560611725, 0.5714336633682251, 0.040586672723293304]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9887767434120178, 0.011223225854337215, 0.0, 0.0, 0.0, 0.0], [0.7572693228721619, 0.22317346930503845, 0.019557112827897072, 0.0, 0.0, 0.0], [0.5341880321502686, 0.22107566893100739, 0.1762184202671051, 0.06851787120103836, 0.0, 0.0], [0.17095312476158142, 0.0822940468788147, 0.576022207736969, 0.11097585409879684, 0.059754710644483566, 0.0], [0.2487109899520874, 0.08880793303251266, 0.08980197459459305, 0.09729334712028503, 0.4413093626499176, 0.03407646715641022]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8422133326530457, 0.15778663754463196, 0.0, 0.0, 0.0, 0.0], [0.468412846326828, 0.46105360984802246, 0.07053359597921371, 0.0, 0.0, 0.0], [0.2588140666484833, 0.4635888636112213, 0.18503506481647491, 0.09256205707788467, 0.0, 0.0], [0.18399578332901, 0.29154160618782043, 0.17031098902225494, 0.27173006534576416, 0.08242159336805344, 0.0], [0.1646990180015564, 0.2472696155309677, 0.08770562708377838, 0.22575001418590546, 0.1774536371231079, 0.09712201356887817]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9919946193695068, 0.008005390875041485, 0.0, 0.0, 0.0, 0.0], [0.9068724513053894, 0.044065121561288834, 0.04906242713332176, 0.0, 0.0, 0.0], [0.8582221865653992, 0.055348269641399384, 0.040419407188892365, 0.046010036021471024, 0.0, 0.0], [0.7855252623558044, 0.041242364794015884, 0.08369296044111252, 0.04887620359659195, 0.040663279592990875, 0.0], [0.7856317162513733, 0.05014643445611, 0.04751267284154892, 0.027365952730178833, 0.05614755302667618, 0.03319567069411278]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9041035175323486, 0.09589648246765137, 0.0, 0.0, 0.0, 0.0], [0.5862312912940979, 0.07199832051992416, 0.34177035093307495, 0.0, 0.0, 0.0], [0.3878960907459259, 0.04660807177424431, 0.20278996229171753, 0.36270591616630554, 0.0, 0.0], [0.2665242552757263, 0.024533024057745934, 0.12211935967206955, 0.20041218400001526, 0.386411190032959, 0.0], [0.23357485234737396, 0.02053728699684143, 0.09610321372747421, 0.13062246143817902, 0.22990450263023376, 0.289257675409317]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9639912247657776, 0.036008793860673904, 0.0, 0.0, 0.0, 0.0], [0.7075552344322205, 0.2542775869369507, 0.038167137652635574, 0.0, 0.0, 0.0], [0.2566526234149933, 0.20589298009872437, 0.01665665954351425, 0.5207977294921875, 0.0, 0.0], [0.1037939190864563, 0.04639088362455368, 0.008698614314198494, 0.7866851687431335, 0.05443140119314194, 0.0], [0.2214341163635254, 0.03379744663834572, 0.029023902490735054, 0.541292130947113, 0.15286092460155487, 0.021591555327177048]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9891703724861145, 0.010829661041498184, 0.0, 0.0, 0.0, 0.0], [0.7913155555725098, 0.12309625744819641, 0.08558809012174606, 0.0, 0.0, 0.0], [0.2954600155353546, 0.15808308124542236, 0.4217240810394287, 0.1247328370809555, 0.0, 0.0], [0.23440983891487122, 0.09886523336172104, 0.33160170912742615, 0.1971396654844284, 0.1379835456609726, 0.0], [0.19728390872478485, 0.05741839483380318, 0.06909029185771942, 0.16469819843769073, 0.2797277867794037, 0.23178131878376007]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9359127879142761, 0.0640871673822403, 0.0, 0.0, 0.0, 0.0], [0.7888627648353577, 0.08673475682735443, 0.12440246343612671, 0.0, 0.0, 0.0], [0.6535118818283081, 0.07573551684617996, 0.09732568264007568, 0.17342689633369446, 0.0, 0.0], [0.522276759147644, 0.058278825134038925, 0.09920477122068405, 0.17020836472511292, 0.15003129839897156, 0.0], [0.4108840823173523, 0.047306034713983536, 0.07265672832727432, 0.10560744255781174, 0.10550004243850708, 0.25804558396339417]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9683833122253418, 0.03161672502756119, 0.0, 0.0, 0.0, 0.0], [0.8965396881103516, 0.038870569318532944, 0.06458976864814758, 0.0, 0.0, 0.0], [0.8264952898025513, 0.03213464096188545, 0.05196719989180565, 0.0894029513001442, 0.0, 0.0], [0.7718173265457153, 0.030402837321162224, 0.045827414840459824, 0.07118473201990128, 0.08076759427785873, 0.0], [0.7292331457138062, 0.021699821576476097, 0.033074747771024704, 0.04720093309879303, 0.06474557518959045, 0.10404567420482635]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9979567527770996, 0.0020432830788195133, 0.0, 0.0, 0.0, 0.0], [0.955294132232666, 0.00802531372755766, 0.03668047487735748, 0.0, 0.0, 0.0], [0.9254710078239441, 0.002755576279014349, 0.0020629852078855038, 0.06971040368080139, 0.0, 0.0], [0.8660576939582825, 0.0038883681409060955, 0.0006785982404835522, 0.0006981453043408692, 0.1286771297454834, 0.0], [0.8455929160118103, 0.0037804055027663708, 0.000253423087997362, 6.0270751419011503e-05, 0.00011820747749879956, 0.15019479393959045]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9262455105781555, 0.07375453412532806, 0.0, 0.0, 0.0, 0.0], [0.7717157006263733, 0.16241952776908875, 0.06586471945047379, 0.0, 0.0, 0.0], [0.8167637586593628, 0.07807160913944244, 0.06324034929275513, 0.041924238204956055, 0.0, 0.0], [0.6867184638977051, 0.07755157351493835, 0.10056912153959274, 0.05955080687999725, 0.07561002671718597, 0.0], [0.6421161890029907, 0.11014898866415024, 0.07688194513320923, 0.054033469408750534, 0.10333634912967682, 0.013483096845448017]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9395954608917236, 0.060404520481824875, 0.0, 0.0, 0.0, 0.0], [0.23004619777202606, 0.6617380380630493, 0.1082158014178276, 0.0, 0.0, 0.0], [0.2670227289199829, 0.3607950508594513, 0.3249626159667969, 0.047219593077898026, 0.0, 0.0], [0.595201313495636, 0.12269274890422821, 0.06302059441804886, 0.08916817605495453, 0.12991715967655182, 0.0], [0.10284596681594849, 0.02938011661171913, 0.013739082030951977, 0.045860596001148224, 0.7698501348495483, 0.03832406550645828]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9040980935096741, 0.09590194374322891, 0.0, 0.0, 0.0, 0.0], [0.357237845659256, 0.6274612545967102, 0.015300876460969448, 0.0, 0.0, 0.0], [0.5917996764183044, 0.2764042019844055, 0.10476048290729523, 0.027035649865865707, 0.0, 0.0], [0.7254403829574585, 0.04983152449131012, 0.014982940629124641, 0.1778142899274826, 0.031930916011333466, 0.0], [0.7612743973731995, 0.06158972904086113, 0.005942251533269882, 0.01642685756087303, 0.1267806589603424, 0.0279861893504858]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9947587847709656, 0.005241230130195618, 0.0, 0.0, 0.0, 0.0], [0.9632415771484375, 0.017816413193941116, 0.018942030146718025, 0.0, 0.0, 0.0], [0.9671078324317932, 0.008509586565196514, 0.00856222677975893, 0.015820473432540894, 0.0, 0.0], [0.9340996146202087, 0.011952387169003487, 0.02018021047115326, 0.02675083465874195, 0.0070168930105865, 0.0], [0.9587237238883972, 0.004657115787267685, 0.003326789475977421, 0.006545313633978367, 0.010182461701333523, 0.016564540565013885]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9769991040229797, 0.023000910878181458, 0.0, 0.0, 0.0, 0.0], [0.7917609214782715, 0.1753319948911667, 0.032907065004110336, 0.0, 0.0, 0.0], [0.7949192523956299, 0.10531841963529587, 0.040218502283096313, 0.05954383686184883, 0.0, 0.0], [0.7097718715667725, 0.10552527755498886, 0.06597573310136795, 0.05765606462955475, 0.061070989817380905, 0.0], [0.7506601214408875, 0.026514461264014244, 0.021576043218374252, 0.034296683967113495, 0.08494450151920319, 0.08200812339782715]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.983751654624939, 0.016248304396867752, 0.0, 0.0, 0.0, 0.0], [0.5615494847297668, 0.08956841379404068, 0.3488820493221283, 0.0, 0.0, 0.0], [0.32929039001464844, 0.024114903062582016, 0.5428059697151184, 0.10378880053758621, 0.0, 0.0], [0.34330207109451294, 0.01308644749224186, 0.5121983289718628, 0.11146228760480881, 0.019950881600379944, 0.0], [0.4792812764644623, 0.01733359508216381, 0.1180536150932312, 0.06130281835794449, 0.20071913301944733, 0.12330964207649231]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9908847212791443, 0.009115329943597317, 0.0, 0.0, 0.0, 0.0], [0.5282707214355469, 0.3292262554168701, 0.1425030380487442, 0.0, 0.0, 0.0], [0.48788541555404663, 0.23368670046329498, 0.17578084766864777, 0.10264702141284943, 0.0, 0.0], [0.31444698572158813, 0.18065163493156433, 0.168714240193367, 0.09506598114967346, 0.24112118780612946, 0.0], [0.5168765187263489, 0.035897161811590195, 0.026188155636191368, 0.04039734974503517, 0.18791745603084564, 0.1927233189344406]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8750308156013489, 0.12496919929981232, 0.0, 0.0, 0.0, 0.0], [0.4550614655017853, 0.4900427758693695, 0.05489582195878029, 0.0, 0.0, 0.0], [0.2933720052242279, 0.5449907183647156, 0.09444297850131989, 0.06719419360160828, 0.0, 0.0], [0.489708811044693, 0.2720997631549835, 0.06861965358257294, 0.14694802463054657, 0.022623788565397263, 0.0], [0.4729066491127014, 0.08103099465370178, 0.016052134335041046, 0.30672287940979004, 0.10120721161365509, 0.022080255672335625]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9630220532417297, 0.03697792813181877, 0.0, 0.0, 0.0, 0.0], [0.7557195425033569, 0.16436372697353363, 0.07991670072078705, 0.0, 0.0, 0.0], [0.6947705745697021, 0.08409853279590607, 0.0638260766863823, 0.15730486810207367, 0.0, 0.0], [0.5821147561073303, 0.03297805413603783, 0.07936596870422363, 0.19441406428813934, 0.11112712323665619, 0.0], [0.5974540710449219, 0.04261096194386482, 0.06919723749160767, 0.14563441276550293, 0.12481734901666641, 0.020285936072468758]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9957822561264038, 0.004217816516757011, 0.0, 0.0, 0.0, 0.0], [0.9312832951545715, 0.010560247115790844, 0.05815650522708893, 0.0, 0.0, 0.0], [0.8435326814651489, 0.015695005655288696, 0.045751139521598816, 0.09502115100622177, 0.0, 0.0], [0.772409975528717, 0.011981245130300522, 0.03504609689116478, 0.03876771405339241, 0.14179500937461853, 0.0], [0.7642908692359924, 0.009868789464235306, 0.00812275055795908, 0.013314393348991871, 0.04824395477771759, 0.15615922212600708]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9701177477836609, 0.02988232672214508, 0.0, 0.0, 0.0, 0.0], [0.6564007997512817, 0.22506150603294373, 0.11853761970996857, 0.0, 0.0, 0.0], [0.6958062648773193, 0.14701850712299347, 0.07145983725786209, 0.08571550250053406, 0.0, 0.0], [0.6353274583816528, 0.1346064656972885, 0.030994214117527008, 0.056916315108537674, 0.1421555131673813, 0.0], [0.6779401898384094, 0.053654152899980545, 0.01800631172955036, 0.06284520775079727, 0.1103820651769638, 0.07717210054397583]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9822334051132202, 0.017766647040843964, 0.0, 0.0, 0.0, 0.0], [0.9037663340568542, 0.06541544198989868, 0.03081829659640789, 0.0, 0.0, 0.0], [0.8119193911552429, 0.03679030388593674, 0.060560714453458786, 0.09072960168123245, 0.0, 0.0], [0.40546438097953796, 0.10383912175893784, 0.10211236774921417, 0.35434210300445557, 0.03424208238720894, 0.0], [0.22824221849441528, 0.017278727144002914, 0.05055465176701546, 0.6015752553939819, 0.09411764144897461, 0.008231506682932377]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9873148202896118, 0.012685136869549751, 0.0, 0.0, 0.0, 0.0], [0.35445743799209595, 0.5317603349685669, 0.11378221958875656, 0.0, 0.0, 0.0], [0.07823363691568375, 0.7221359014511108, 0.10936623811721802, 0.090264230966568, 0.0, 0.0], [0.21967869997024536, 0.4048435091972351, 0.12358088046312332, 0.20018866658210754, 0.051708199083805084, 0.0], [0.36089760065078735, 0.10459021478891373, 0.06983799487352371, 0.2976483404636383, 0.13869903981685638, 0.02832675166428089]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9732162356376648, 0.0267837755382061, 0.0, 0.0, 0.0, 0.0], [0.9167553782463074, 0.061452705413103104, 0.02179192565381527, 0.0, 0.0, 0.0], [0.8543081283569336, 0.08049600571393967, 0.030334919691085815, 0.03486092761158943, 0.0, 0.0], [0.8919214606285095, 0.04280779883265495, 0.022045055404305458, 0.023470671847462654, 0.01975487545132637, 0.0], [0.8116763234138489, 0.03413533419370651, 0.03567665070295334, 0.04748587682843208, 0.0253971628844738, 0.04562860727310181]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9502761960029602, 0.04972382262349129, 0.0, 0.0, 0.0, 0.0], [0.7637454271316528, 0.2007361352443695, 0.03551840782165527, 0.0, 0.0, 0.0], [0.6279097199440002, 0.03768139332532883, 0.1994536966085434, 0.13495522737503052, 0.0, 0.0], [0.6397060751914978, 0.027007432654500008, 0.09082036465406418, 0.20653828978538513, 0.03592785820364952, 0.0], [0.4559425115585327, 0.021641194820404053, 0.12939567863941193, 0.21800927817821503, 0.10379841923713684, 0.07121295481920242]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9498406648635864, 0.050159383565187454, 0.0, 0.0, 0.0, 0.0], [0.8688724637031555, 0.0872218981385231, 0.043905653059482574, 0.0, 0.0, 0.0], [0.6937950253486633, 0.06359200924634933, 0.091790571808815, 0.15082231163978577, 0.0, 0.0], [0.7266597151756287, 0.04389883577823639, 0.04683985933661461, 0.09851823002099991, 0.08408336341381073, 0.0], [0.7848998308181763, 0.037147827446460724, 0.012907838448882103, 0.01053939200937748, 0.12079165875911713, 0.03371351957321167]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9891054034233093, 0.01089458167552948, 0.0, 0.0, 0.0, 0.0], [0.8929519653320312, 0.08700055629014969, 0.02004752680659294, 0.0, 0.0, 0.0], [0.7891124486923218, 0.09797251224517822, 0.08633202314376831, 0.026582980528473854, 0.0, 0.0], [0.8850635886192322, 0.03645012155175209, 0.05395457148551941, 0.01237727515399456, 0.012154522351920605, 0.0], [0.6861329674720764, 0.05720378831028938, 0.011636304669082165, 0.021660611033439636, 0.1748800277709961, 0.048486363142728806]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9396191835403442, 0.06038080155849457, 0.0, 0.0, 0.0, 0.0], [0.7851794958114624, 0.19751444458961487, 0.017306052148342133, 0.0, 0.0, 0.0], [0.7660509943962097, 0.15444670617580414, 0.03188290074467659, 0.04761936888098717, 0.0, 0.0], [0.703522801399231, 0.05171430483460426, 0.07760990411043167, 0.1533905267715454, 0.013762423768639565, 0.0], [0.7121888399124146, 0.04994234815239906, 0.03772548958659172, 0.08649132400751114, 0.06541401147842407, 0.04823806509375572]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.974072277545929, 0.025927715003490448, 0.0, 0.0, 0.0, 0.0], [0.792539656162262, 0.01171559002250433, 0.19574476778507233, 0.0, 0.0, 0.0], [0.5106770992279053, 0.007296787109225988, 0.039619915187358856, 0.4424062669277191, 0.0, 0.0], [0.5862472057342529, 0.012099712155759335, 0.024585209786891937, 0.06737840175628662, 0.30968940258026123, 0.0], [0.30196306109428406, 0.007724012713879347, 0.011518122628331184, 0.046947259455919266, 0.22146707773208618, 0.41038045287132263]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9744554162025452, 0.02554464340209961, 0.0, 0.0, 0.0, 0.0], [0.9769195318222046, 0.015048524364829063, 0.008031901903450489, 0.0, 0.0, 0.0], [0.9060619473457336, 0.025875424966216087, 0.025954782962799072, 0.04210779070854187, 0.0, 0.0], [0.9400081038475037, 0.00555665697902441, 0.005828304681926966, 0.031757812947034836, 0.016849134117364883, 0.0], [0.9105738401412964, 0.0019752182997763157, 0.008646721951663494, 0.013360846787691116, 0.03543964773416519, 0.030003678053617477]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9791666865348816, 0.020833350718021393, 0.0, 0.0, 0.0, 0.0], [0.8444858193397522, 0.13507869839668274, 0.020435383543372154, 0.0, 0.0, 0.0], [0.7903086543083191, 0.14559169113636017, 0.037529975175857544, 0.026569725945591927, 0.0, 0.0], [0.7298924326896667, 0.056496407836675644, 0.032735615968704224, 0.10400459170341492, 0.07687094807624817, 0.0], [0.5684185028076172, 0.04388832300901413, 0.026293467730283737, 0.0811714455485344, 0.24314835667610168, 0.037079911679029465]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9499868154525757, 0.05001320689916611, 0.0, 0.0, 0.0, 0.0], [0.9336170554161072, 0.05848868936300278, 0.007894262671470642, 0.0, 0.0, 0.0], [0.7897834181785583, 0.11071821302175522, 0.05360178276896477, 0.04589657858014107, 0.0, 0.0], [0.885930061340332, 0.05752986669540405, 0.01374326553195715, 0.0033877466339617968, 0.03940902277827263, 0.0], [0.9337607622146606, 0.02647063508629799, 0.004523396957665682, 0.0061904797330498695, 0.014132906682789326, 0.014921708963811398]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [1.0, 1.8521224554035598e-09, 0.0, 0.0, 0.0, 0.0], [6.6758907451003324e-06, 0.9999804496765137, 1.2841281204600818e-05, 0.0, 0.0, 0.0], [2.2194194926328237e-08, 2.6684581211355862e-09, 0.9999971389770508, 2.8136880700913025e-06, 0.0, 0.0], [1.0145409987671883e-06, 4.464065739284706e-08, 0.00035356366424821317, 0.9993677735328674, 0.0002776293840724975, 0.0], [9.436550429953172e-10, 1.382057315812979e-11, 5.017835036369434e-10, 2.965183876213473e-09, 0.9999971389770508, 2.8644042231462663e-06]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9948632121086121, 0.005136783700436354, 0.0, 0.0, 0.0, 0.0], [0.9274215698242188, 0.01832387037575245, 0.05425456911325455, 0.0, 0.0, 0.0], [0.9678993225097656, 0.004143435508012772, 0.004314453341066837, 0.023642776533961296, 0.0, 0.0], [0.8999068737030029, 0.001467161695472896, 0.00029133574571460485, 0.002585014794021845, 0.09574954956769943, 0.0], [0.9386115670204163, 0.00022248300956562161, 0.0006146665546111763, 0.0015495637198910117, 0.030689461156725883, 0.028312424197793007]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9999959468841553, 4.042720775032649e-06, 0.0, 0.0, 0.0, 0.0], [0.9982761144638062, 3.2613831990602193e-06, 0.001720669330097735, 0.0, 0.0, 0.0], [0.9998809099197388, 5.328835683826583e-08, 6.376215537784446e-07, 0.00011847059795400128, 0.0, 0.0], [0.9996154308319092, 3.473169556400535e-07, 3.8920820344401363e-08, 4.468433303372876e-07, 0.00038369710091501474, 0.0], [0.9994840621948242, 1.655020476221125e-08, 2.8715557931491276e-08, 1.0638284493325045e-06, 0.0002126671897713095, 0.00030212008277885616]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9514135718345642, 0.048586405813694, 0.0, 0.0, 0.0, 0.0], [0.5749948024749756, 0.39028096199035645, 0.03472418338060379, 0.0, 0.0, 0.0], [0.7442318201065063, 0.1752411425113678, 0.0756477490067482, 0.004879283253103495, 0.0, 0.0], [0.5232070684432983, 0.09429339319467545, 0.1138191670179367, 0.19979268312454224, 0.06888769567012787, 0.0], [0.47472575306892395, 0.05636607110500336, 0.04530389606952667, 0.06967321783304214, 0.3098014295101166, 0.0441296212375164]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8734648823738098, 0.12653514742851257, 0.0, 0.0, 0.0, 0.0], [0.6097912788391113, 0.3541727066040039, 0.036036062985658646, 0.0, 0.0, 0.0], [0.45984190702438354, 0.38697871565818787, 0.0996011346578598, 0.05357823893427849, 0.0, 0.0], [0.572220504283905, 0.23636263608932495, 0.08344558626413345, 0.06921917200088501, 0.03875211998820305, 0.0], [0.5143564343452454, 0.16723087430000305, 0.09019406139850616, 0.0765448659658432, 0.10578085482120514, 0.04589281603693962]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.981228768825531, 0.018771231174468994, 0.0, 0.0, 0.0, 0.0], [0.6142941117286682, 0.3503977954387665, 0.0353081189095974, 0.0, 0.0, 0.0], [0.5770686268806458, 0.32858458161354065, 0.05508256331086159, 0.03926428034901619, 0.0, 0.0], [0.17188192903995514, 0.011042501777410507, 0.054578714072704315, 0.7326585650444031, 0.029838265851140022, 0.0], [0.3783015012741089, 0.017070062458515167, 0.021754134446382523, 0.4409688115119934, 0.06093813106417656, 0.08096737414598465]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9923112392425537, 0.007688735146075487, 0.0, 0.0, 0.0, 0.0], [0.9498787522315979, 0.016709784045815468, 0.03341152146458626, 0.0, 0.0, 0.0], [0.9961295127868652, 0.0008787295082584023, 0.0006868162308819592, 0.0023048371076583862, 0.0, 0.0], [0.9935757517814636, 0.0032634998206049204, 0.0009993825806304812, 0.00027932299417443573, 0.0018820574041455984, 0.0], [0.9907532930374146, 0.00021344318520277739, 0.0004595233185682446, 0.0007905619568191469, 0.004424723796546459, 0.003358350833877921]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9647740125656128, 0.03522596135735512, 0.0, 0.0, 0.0, 0.0], [0.8194130063056946, 0.1365436613559723, 0.04404333233833313, 0.0, 0.0, 0.0], [0.7584245800971985, 0.006878929678350687, 0.20653395354747772, 0.028162529692053795, 0.0, 0.0], [0.5298128128051758, 0.002678812015801668, 0.07857988774776459, 0.3598373234272003, 0.02909109927713871, 0.0], [0.7544413208961487, 0.00036782227107323706, 0.0019713479559868574, 0.00324004958383739, 0.1942344754934311, 0.04574500769376755]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9749131202697754, 0.02508680149912834, 0.0, 0.0, 0.0, 0.0], [0.9306471943855286, 0.05705660209059715, 0.012296222150325775, 0.0, 0.0, 0.0], [0.9305251836776733, 0.052770983427762985, 0.01111945416778326, 0.005584415514022112, 0.0, 0.0], [0.8863320350646973, 0.01292418036609888, 0.017724711447954178, 0.06150198355317116, 0.021517015993595123, 0.0], [0.791684627532959, 0.015036096796393394, 0.0317479707300663, 0.03392200171947479, 0.03707978501915932, 0.09052948653697968]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9608501195907593, 0.039149850606918335, 0.0, 0.0, 0.0, 0.0], [0.9121272563934326, 0.02257651649415493, 0.06529619544744492, 0.0, 0.0, 0.0], [0.9364108443260193, 0.015584447421133518, 0.024544963613152504, 0.02345985174179077, 0.0, 0.0], [0.9454620480537415, 0.006762288510799408, 0.022026237100362778, 0.009137796238064766, 0.016611700877547264, 0.0], [0.8346164226531982, 0.001881699077785015, 0.00560904573649168, 0.01887359470129013, 0.12449200451374054, 0.014527074061334133]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9964227080345154, 0.0035772807896137238, 0.0, 0.0, 0.0, 0.0], [0.9713928699493408, 0.024453025311231613, 0.004154058638960123, 0.0, 0.0, 0.0], [0.9735792279243469, 0.019003381952643394, 0.003664410673081875, 0.0037529165856540203, 0.0, 0.0], [0.9586312174797058, 0.007116180844604969, 0.009218388237059116, 0.022725583985447884, 0.0023084774147719145, 0.0], [0.973607063293457, 0.008490582928061485, 0.0032512471079826355, 0.003606445388868451, 0.004877461586147547, 0.006167212035506964]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.97598797082901, 0.024011990055441856, 0.0, 0.0, 0.0, 0.0], [0.9460638165473938, 0.04211375489830971, 0.011822436936199665, 0.0, 0.0, 0.0], [0.8446813225746155, 0.04293116182088852, 0.05218198522925377, 0.06020559370517731, 0.0, 0.0], [0.9378372430801392, 0.03354858607053757, 0.008826455101370811, 0.0028792242519557476, 0.016908427700400352, 0.0], [0.8124931454658508, 0.02696753479540348, 0.05999218672513962, 0.03445731848478317, 0.011011860333383083, 0.05507794767618179]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9001203775405884, 0.09987961500883102, 0.0, 0.0, 0.0, 0.0], [0.627193033695221, 0.07988718152046204, 0.29291975498199463, 0.0, 0.0, 0.0], [0.7624077796936035, 0.02734432928264141, 0.038679543882608414, 0.17156831920146942, 0.0, 0.0], [0.7995968461036682, 0.014336260966956615, 0.01437566988170147, 0.025438452139496803, 0.14625284075737, 0.0], [0.7851970791816711, 0.04204057529568672, 0.025253651663661003, 0.02908395044505596, 0.029306314885616302, 0.08911846578121185]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9954467415809631, 0.0045532057993113995, 0.0, 0.0, 0.0, 0.0], [0.9356001615524292, 0.04476744681596756, 0.019632352516055107, 0.0, 0.0, 0.0], [0.5605552792549133, 0.09861977398395538, 0.29983264207839966, 0.040992289781570435, 0.0, 0.0], [0.5893709659576416, 0.11000988632440567, 0.08033622056245804, 0.16754034161567688, 0.05274256691336632, 0.0], [0.22305884957313538, 0.05680817365646362, 0.05467984080314636, 0.24733951687812805, 0.3111244738101959, 0.1069890558719635]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9301451444625854, 0.06985488533973694, 0.0, 0.0, 0.0, 0.0], [0.8936478495597839, 0.08535721153020859, 0.020994966849684715, 0.0, 0.0, 0.0], [0.8404538035392761, 0.10619214922189713, 0.02363673783838749, 0.029717326164245605, 0.0, 0.0], [0.8927386403083801, 0.024784674867987633, 0.008319000713527203, 0.05165454372763634, 0.022503145039081573, 0.0], [0.8646610975265503, 0.009503193199634552, 0.0024329854641109705, 0.04796753078699112, 0.04273205250501633, 0.03270319849252701]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9859625697135925, 0.014037408865988255, 0.0, 0.0, 0.0, 0.0], [0.9702037572860718, 0.0168070700019598, 0.012989125214517117, 0.0, 0.0, 0.0], [0.9524770379066467, 0.016064459457993507, 0.013456220738589764, 0.018002323806285858, 0.0, 0.0], [0.9332928657531738, 0.01897200010716915, 0.02014683373272419, 0.017023753374814987, 0.010564540512859821, 0.0], [0.9113592505455017, 0.012528638355433941, 0.02209620550274849, 0.01751861348748207, 0.018517911434173584, 0.01797938533127308]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9681769013404846, 0.03182310611009598, 0.0, 0.0, 0.0, 0.0], [0.9096417427062988, 0.07916690409183502, 0.011191264726221561, 0.0, 0.0, 0.0], [0.8379932045936584, 0.13078266382217407, 0.012140989303588867, 0.019083037972450256, 0.0, 0.0], [0.9116525053977966, 0.05451957508921623, 0.009499342180788517, 0.00746585289016366, 0.01686275750398636, 0.0], [0.8510289192199707, 0.07338211685419083, 0.008022507652640343, 0.009083161130547523, 0.04261006414890289, 0.015873271971940994]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9799023866653442, 0.020097682252526283, 0.0, 0.0, 0.0, 0.0], [0.9558742642402649, 0.029063312336802483, 0.015062497928738594, 0.0, 0.0, 0.0], [0.7943133115768433, 0.06074100360274315, 0.06907659024000168, 0.07586916536092758, 0.0, 0.0], [0.5494324564933777, 0.03154711425304413, 0.05482015758752823, 0.05788077041506767, 0.3063195049762726, 0.0], [0.6453980803489685, 0.010770943015813828, 0.017528092488646507, 0.02157985046505928, 0.24958276748657227, 0.05514020845293999]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9506809115409851, 0.0493190623819828, 0.0, 0.0, 0.0, 0.0], [0.8553215265274048, 0.09256264567375183, 0.05211575701832771, 0.0, 0.0, 0.0], [0.850852370262146, 0.04734604433178902, 0.044177331030368805, 0.057624250650405884, 0.0, 0.0], [0.7697131633758545, 0.02788589708507061, 0.031017286702990532, 0.06842502951622009, 0.1029587835073471, 0.0], [0.7931903004646301, 0.04052198305726051, 0.029242033138871193, 0.04478124529123306, 0.04894689470529556, 0.04331749677658081]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9770310521125793, 0.02296893112361431, 0.0, 0.0, 0.0, 0.0], [0.9429817199707031, 0.017321482300758362, 0.03969680890440941, 0.0, 0.0, 0.0], [0.9144344925880432, 0.008583576418459415, 0.013035810552537441, 0.06394599378108978, 0.0, 0.0], [0.9222429990768433, 0.0036440351977944374, 0.003740275977179408, 0.010410364717245102, 0.05996239185333252, 0.0], [0.9198879599571228, 0.0030822583939880133, 0.0034827394410967827, 0.004206796642392874, 0.02125428058207035, 0.048085976392030716]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.977458119392395, 0.022541873157024384, 0.0, 0.0, 0.0, 0.0], [0.8929325342178345, 0.07475466281175613, 0.032312843948602676, 0.0, 0.0, 0.0], [0.8423511385917664, 0.05980278551578522, 0.03740081936120987, 0.06044524535536766, 0.0, 0.0], [0.7674624919891357, 0.03536349534988403, 0.042155250906944275, 0.06658654659986496, 0.08843226730823517, 0.0], [0.6182611584663391, 0.01611059531569481, 0.020167622715234756, 0.03868892416357994, 0.23147016763687134, 0.07530155777931213]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9634856581687927, 0.036514393985271454, 0.0, 0.0, 0.0, 0.0], [0.4363938570022583, 0.522637128829956, 0.04096902906894684, 0.0, 0.0, 0.0], [0.3608614206314087, 0.35129693150520325, 0.2655103802680969, 0.022331148386001587, 0.0, 0.0], [0.3942921757698059, 0.021704670041799545, 0.07794328778982162, 0.37168896198272705, 0.1343708038330078, 0.0], [0.6310713887214661, 0.01698400266468525, 0.025942081585526466, 0.08615949749946594, 0.2183200567960739, 0.021522950381040573]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9988250136375427, 0.0011750265257433057, 0.0, 0.0, 0.0, 0.0], [0.9944871068000793, 0.0004826401418540627, 0.0050302306190133095, 0.0, 0.0, 0.0], [0.9981209635734558, 2.705173392314464e-05, 0.0001130745149566792, 0.0017389442073181272, 0.0, 0.0], [0.9982239603996277, 6.83655816828832e-05, 0.00010199935059063137, 6.028370262356475e-05, 0.0015453165397047997, 0.0], [0.9982888102531433, 1.055222810464329e-06, 3.2781026675365865e-05, 0.00013038977340329438, 0.0006605894886888564, 0.0008863684488460422]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9936710596084595, 0.006328921765089035, 0.0, 0.0, 0.0, 0.0], [0.9727688431739807, 0.0018561368342489004, 0.025375060737133026, 0.0, 0.0, 0.0], [0.9724299907684326, 0.0019586149137467146, 0.011192461475729942, 0.014418890699744225, 0.0, 0.0], [0.9782041311264038, 0.0009589138207957149, 0.0018706483533605933, 0.006326568778604269, 0.012639678083360195, 0.0], [0.9592596888542175, 0.0024555064737796783, 0.00161241355817765, 0.005019655916839838, 0.006687097251415253, 0.024965662509202957]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9629000425338745, 0.03709998354315758, 0.0, 0.0, 0.0, 0.0], [0.36801934242248535, 0.6152258515357971, 0.016754813492298126, 0.0, 0.0, 0.0], [0.3173511326313019, 0.6140013337135315, 0.05375149846076965, 0.014896026812493801, 0.0, 0.0], [0.48987284302711487, 0.21071474254131317, 0.04693019017577171, 0.20700432360172272, 0.04547784850001335, 0.0], [0.48774227499961853, 0.1769528090953827, 0.06915216147899628, 0.09849268198013306, 0.12091436982154846, 0.046745721250772476]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9794419407844543, 0.020558049902319908, 0.0, 0.0, 0.0, 0.0], [0.6677903532981873, 0.31032365560531616, 0.021886007860302925, 0.0, 0.0, 0.0], [0.7118757367134094, 0.11108540743589401, 0.14187385141849518, 0.03516504913568497, 0.0, 0.0], [0.4501457214355469, 0.04036055505275726, 0.040458209812641144, 0.388570100069046, 0.08046531677246094, 0.0], [0.49346262216567993, 0.013696977868676186, 0.008126799948513508, 0.13074499368667603, 0.3086138069629669, 0.04535480588674545]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9846054315567017, 0.015394587069749832, 0.0, 0.0, 0.0, 0.0], [0.9806739091873169, 0.007713791914284229, 0.011612347327172756, 0.0, 0.0, 0.0], [0.932663083076477, 0.01957838423550129, 0.02410353161394596, 0.023654978722333908, 0.0, 0.0], [0.9422016739845276, 0.0009538981830701232, 0.0010898025939241052, 0.00319337984547019, 0.05256118252873421, 0.0], [0.9352930784225464, 0.0010279357666149735, 0.004444425459951162, 0.001637140172533691, 0.010590963996946812, 0.04700646549463272]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9985783100128174, 0.0014216724084690213, 0.0, 0.0, 0.0, 0.0], [0.9893348813056946, 0.0011178902350366116, 0.00954714696854353, 0.0, 0.0, 0.0], [0.9979978203773499, 7.997050124686211e-05, 0.00013218850654084235, 0.0017900333041325212, 0.0, 0.0], [0.9986976385116577, 4.1044117097044364e-05, 3.8683547245454974e-06, 2.3676282580709085e-05, 0.0012337174266576767, 0.0], [0.9971563816070557, 1.852225250331685e-05, 1.8826559653462027e-06, 2.7900125132873654e-05, 0.0006533482228405774, 0.0021419788245111704]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9768233299255371, 0.023176640272140503, 0.0, 0.0, 0.0, 0.0], [0.9194678068161011, 0.05088186264038086, 0.029650341719388962, 0.0, 0.0, 0.0], [0.8474554419517517, 0.06100169196724892, 0.04372376948595047, 0.04781914874911308, 0.0, 0.0], [0.8011623620986938, 0.041866958141326904, 0.04375807195901871, 0.041894737631082535, 0.07131782174110413, 0.0], [0.8031871914863586, 0.02450493723154068, 0.017323585227131844, 0.04744395986199379, 0.06109930947422981, 0.046441152691841125]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9829428195953369, 0.01705716922879219, 0.0, 0.0, 0.0, 0.0], [0.8863736987113953, 0.09492647647857666, 0.018699750304222107, 0.0, 0.0, 0.0], [0.9231085777282715, 0.03696346655488014, 0.032198335975408554, 0.007729663979262114, 0.0, 0.0], [0.9068527221679688, 0.016046639531850815, 0.014310522936284542, 0.04543786868453026, 0.017352323979139328, 0.0], [0.6555973887443542, 0.05091019719839096, 0.028384855017066002, 0.1256549060344696, 0.10546853393316269, 0.03398407623171806]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9502318501472473, 0.049768079072237015, 0.0, 0.0, 0.0, 0.0], [0.8829865455627441, 0.1000962108373642, 0.01691717840731144, 0.0, 0.0, 0.0], [0.8057457804679871, 0.14463546872138977, 0.03018922731280327, 0.019429458305239677, 0.0, 0.0], [0.8706230521202087, 0.032440632581710815, 0.026951627805829048, 0.04410304129123688, 0.025881657376885414, 0.0], [0.688364565372467, 0.009681451134383678, 0.016449343413114548, 0.0987110361456871, 0.08971209079027176, 0.09708156436681747]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9792683124542236, 0.02073168195784092, 0.0, 0.0, 0.0, 0.0], [0.9523284435272217, 0.025933818891644478, 0.021737735718488693, 0.0, 0.0, 0.0], [0.9144353270530701, 0.017671240493655205, 0.022358495742082596, 0.04553484544157982, 0.0, 0.0], [0.9448292851448059, 0.006467597559094429, 0.006386063527315855, 0.03263096138834953, 0.00968620739877224, 0.0], [0.9347906112670898, 0.007862505502998829, 0.007788175716996193, 0.021432818844914436, 0.008491144515573978, 0.01963483914732933]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.983370304107666, 0.016629677265882492, 0.0, 0.0, 0.0, 0.0], [0.963111400604248, 0.009229931980371475, 0.027658598497509956, 0.0, 0.0, 0.0], [0.9706628322601318, 0.0041494048200547695, 0.0068131014704704285, 0.018374638631939888, 0.0, 0.0], [0.987951934337616, 0.002165885642170906, 0.00034901127219200134, 0.001583816367201507, 0.00794942770153284, 0.0], [0.9457950592041016, 0.014583553187549114, 0.0003652951563708484, 0.0009569536778144538, 0.013621564954519272, 0.02467755414545536]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9878059029579163, 0.01219407469034195, 0.0, 0.0, 0.0, 0.0], [0.87103670835495, 0.09448163211345673, 0.03448161482810974, 0.0, 0.0, 0.0], [0.6309783458709717, 0.11090382188558578, 0.1923021823167801, 0.06581564992666245, 0.0, 0.0], [0.5360490083694458, 0.04618944972753525, 0.13605308532714844, 0.26455509662628174, 0.017153292894363403, 0.0], [0.8287520408630371, 0.023732755333185196, 0.02008037269115448, 0.07245264202356339, 0.030431220307946205, 0.024550989270210266]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8995685577392578, 0.10043150931596756, 0.0, 0.0, 0.0, 0.0], [0.270343542098999, 0.6504329442977905, 0.07922357320785522, 0.0, 0.0, 0.0], [0.20541730523109436, 0.5892508625984192, 0.18085837364196777, 0.024473490193486214, 0.0, 0.0], [0.5573861002922058, 0.1774134784936905, 0.08806808292865753, 0.09881848096847534, 0.07831384986639023, 0.0], [0.5922912359237671, 0.08700639009475708, 0.05643285810947418, 0.05685883015394211, 0.12181518226861954, 0.08559554070234299]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9316380620002747, 0.06836195290088654, 0.0, 0.0, 0.0, 0.0], [0.9572945833206177, 0.026243582367897034, 0.0164618119597435, 0.0, 0.0, 0.0], [0.9880544543266296, 0.00427332753315568, 0.002954584313556552, 0.004717645235359669, 0.0, 0.0], [0.99403977394104, 0.0009413420339114964, 0.0004739820142276585, 0.00011646930943243206, 0.004428447224199772, 0.0], [0.9806035161018372, 2.5468933017691597e-05, 0.00016239412070717663, 0.0001476418401580304, 0.0013442443450912833, 0.017716845497488976]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.993178129196167, 0.006821857299655676, 0.0, 0.0, 0.0, 0.0], [0.9756524562835693, 0.01318411435931921, 0.011163423769176006, 0.0, 0.0, 0.0], [0.9418966770172119, 0.004721744451671839, 0.0023818055633455515, 0.050999753177165985, 0.0, 0.0], [0.9905040860176086, 0.0022848136723041534, 6.198462506290525e-05, 0.0005984465242363513, 0.006550676189363003, 0.0], [0.9697660207748413, 0.0008878845837898552, 0.00023466735729016364, 0.0017040816601365805, 0.004128355998545885, 0.02327893301844597]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9716231822967529, 0.02837684564292431, 0.0, 0.0, 0.0, 0.0], [0.9223619699478149, 0.028907248750329018, 0.048730745911598206, 0.0, 0.0, 0.0], [0.8426317572593689, 0.023872116580605507, 0.04748132824897766, 0.08601479232311249, 0.0, 0.0], [0.8521121740341187, 0.020744236186146736, 0.04494619369506836, 0.05765002593398094, 0.02454746514558792, 0.0], [0.8800725936889648, 0.022448532283306122, 0.018235722556710243, 0.01925482600927353, 0.015854258090257645, 0.044134121388196945]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9412723779678345, 0.058727629482746124, 0.0, 0.0, 0.0, 0.0], [0.916313886642456, 0.05759201943874359, 0.02609400637447834, 0.0, 0.0, 0.0], [0.8392423391342163, 0.057690516114234924, 0.01382902916520834, 0.08923812955617905, 0.0, 0.0], [0.8987162113189697, 0.0134778693318367, 0.0003456450067460537, 0.003298751311376691, 0.08416149020195007, 0.0], [0.8701692223548889, 0.002700856188312173, 0.00143499206751585, 0.0056661744602024555, 0.08874300867319107, 0.031285665929317474]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9656725525856018, 0.03432750701904297, 0.0, 0.0, 0.0, 0.0], [0.9178615808486938, 0.062257930636405945, 0.019880469888448715, 0.0, 0.0, 0.0], [0.823314905166626, 0.06282395124435425, 0.03670429438352585, 0.07715693861246109, 0.0, 0.0], [0.8501748442649841, 0.03816927224397659, 0.03196492791175842, 0.0516013503074646, 0.02808968350291252, 0.0], [0.6572404503822327, 0.05877397954463959, 0.04336007311940193, 0.09013211727142334, 0.08146599680185318, 0.06902744621038437]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9162061810493469, 0.0837937667965889, 0.0, 0.0, 0.0, 0.0], [0.9451773762702942, 0.04099284112453461, 0.013829832896590233, 0.0, 0.0, 0.0], [0.8928355574607849, 0.05368670076131821, 0.017596954479813576, 0.03588071092963219, 0.0, 0.0], [0.8337052464485168, 0.04799601063132286, 0.033513229340314865, 0.04680858924984932, 0.03797686845064163, 0.0], [0.8167192339897156, 0.06337132304906845, 0.013286277651786804, 0.020469767972826958, 0.025292355567216873, 0.06086111441254616]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9525133371353149, 0.04748663306236267, 0.0, 0.0, 0.0, 0.0], [0.3019869327545166, 0.6520938873291016, 0.04591925069689751, 0.0, 0.0, 0.0], [0.285582959651947, 0.556952178478241, 0.1444743126630783, 0.012990524061024189, 0.0, 0.0], [0.843804121017456, 0.032251205295324326, 0.03954290598630905, 0.06848159432411194, 0.015920041128993034, 0.0], [0.6664940714836121, 0.06095913052558899, 0.04064354673027992, 0.06804485619068146, 0.09186329692602158, 0.07199501991271973]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9682655334472656, 0.031734466552734375, 0.0, 0.0, 0.0, 0.0], [0.738521933555603, 0.22856839001178741, 0.032909639179706573, 0.0, 0.0, 0.0], [0.5946676135063171, 0.2303314357995987, 0.14867636561393738, 0.02632458508014679, 0.0, 0.0], [0.6339254975318909, 0.05813034623861313, 0.09654320776462555, 0.14291946589946747, 0.06848153471946716, 0.0], [0.40375572443008423, 0.08945391327142715, 0.07635112851858139, 0.25587135553359985, 0.1433039754629135, 0.03126389905810356]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9869793653488159, 0.013020593672990799, 0.0, 0.0, 0.0, 0.0], [0.8631385564804077, 0.1105666309595108, 0.02629482001066208, 0.0, 0.0, 0.0], [0.9488080143928528, 0.028614996001124382, 0.006535546388477087, 0.016041526570916176, 0.0, 0.0], [0.9672170877456665, 0.006604980677366257, 0.00045171406236477196, 0.004844417329877615, 0.020881708711385727, 0.0], [0.9354621171951294, 0.02047806605696678, 0.0011700231116265059, 0.007056943140923977, 0.0163181871175766, 0.019514625892043114]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9846673011779785, 0.015332723967730999, 0.0, 0.0, 0.0, 0.0], [0.9052747488021851, 0.08373606950044632, 0.010989243164658546, 0.0, 0.0, 0.0], [0.8145939111709595, 0.04283742979168892, 0.10568301379680634, 0.03688570484519005, 0.0, 0.0], [0.23519809544086456, 0.012018457986414433, 0.05280117318034172, 0.6516180038452148, 0.04836418479681015, 0.0], [0.31818512082099915, 0.018632443621754646, 0.03948190063238144, 0.3755541741847992, 0.20787373185157776, 0.04027257487177849]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9811733365058899, 0.018826685845851898, 0.0, 0.0, 0.0, 0.0], [0.8618939518928528, 0.06479164958000183, 0.07331438362598419, 0.0, 0.0, 0.0], [0.7664540410041809, 0.07330425828695297, 0.10353513062000275, 0.056706514209508896, 0.0, 0.0], [0.8128499984741211, 0.03215480223298073, 0.059005625545978546, 0.05416511744260788, 0.04182446748018265, 0.0], [0.8687856197357178, 0.026987861841917038, 0.02047000452876091, 0.01629738137125969, 0.03218390792608261, 0.03527523949742317]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9264583587646484, 0.07354167848825455, 0.0, 0.0, 0.0, 0.0], [0.8403540849685669, 0.06373751163482666, 0.09590838104486465, 0.0, 0.0, 0.0], [0.7330995798110962, 0.06451118737459183, 0.10380073636770248, 0.09858842939138412, 0.0, 0.0], [0.9143612384796143, 0.008257776498794556, 0.007320381235331297, 0.017966248095035553, 0.05209439620375633, 0.0], [0.8971915245056152, 0.008555498905479908, 0.007019453682005405, 0.014860544353723526, 0.03399762138724327, 0.03837529569864273]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9180347919464111, 0.08196526020765305, 0.0, 0.0, 0.0, 0.0], [0.8328666687011719, 0.1219901517033577, 0.04514322429895401, 0.0, 0.0, 0.0], [0.7994157075881958, 0.0874413549900055, 0.03605784848332405, 0.07708510011434555, 0.0, 0.0], [0.880984902381897, 0.020749641582369804, 0.020554615184664726, 0.017120830714702606, 0.06058995798230171, 0.0], [0.745303213596344, 0.044334057718515396, 0.022549288347363472, 0.0331527441740036, 0.03357058763504028, 0.12109009176492691]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9867060780525208, 0.013293893076479435, 0.0, 0.0, 0.0, 0.0], [0.982177734375, 0.012414131313562393, 0.005408108700066805, 0.0, 0.0, 0.0], [0.9630486369132996, 0.015290752984583378, 0.010345698334276676, 0.0113149369135499, 0.0, 0.0], [0.9213568568229675, 0.014132463373243809, 0.017639216035604477, 0.016567690297961235, 0.030303770676255226, 0.0], [0.9373326301574707, 0.009064299054443836, 0.007548365276306868, 0.006576443091034889, 0.011827622540295124, 0.027650514617562294]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9951004385948181, 0.00489962799474597, 0.0, 0.0, 0.0, 0.0], [0.9476007223129272, 0.041407931596040726, 0.010991275310516357, 0.0, 0.0, 0.0], [0.9142175316810608, 0.023523783311247826, 0.039145033806562424, 0.023113621398806572, 0.0, 0.0], [0.9534738659858704, 0.008932933211326599, 0.015272765420377254, 0.007908251136541367, 0.014412266202270985, 0.0], [0.9427101016044617, 0.00823307130485773, 0.004650997929275036, 0.004178107250481844, 0.005463531706482172, 0.03476419299840927]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9543376564979553, 0.045662373304367065, 0.0, 0.0, 0.0, 0.0], [0.9696040749549866, 0.01954760029911995, 0.01084828469902277, 0.0, 0.0, 0.0], [0.9710449576377869, 0.012425386346876621, 0.008068876340985298, 0.008460716344416142, 0.0, 0.0], [0.9726192951202393, 0.002697656163945794, 0.00044831327977590263, 0.0013814778067171574, 0.022853154689073563, 0.0], [0.9675466418266296, 0.009613442234694958, 0.003203035332262516, 0.00424883933737874, 0.007442260626703501, 0.00794589426368475]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9887008666992188, 0.011299116536974907, 0.0, 0.0, 0.0, 0.0], [0.9382632374763489, 0.04204244911670685, 0.019694412127137184, 0.0, 0.0, 0.0], [0.8351995944976807, 0.03487853705883026, 0.05134471505880356, 0.07857715338468552, 0.0, 0.0], [0.9042676687240601, 0.010541575029492378, 0.016426723450422287, 0.025921987369656563, 0.04284200444817543, 0.0], [0.8913140892982483, 0.00891267228871584, 0.005010711494833231, 0.008175632916390896, 0.013514749705791473, 0.07307209819555283]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8693912029266357, 0.13060881197452545, 0.0, 0.0, 0.0, 0.0], [0.3507988452911377, 0.606351912021637, 0.04284917935729027, 0.0, 0.0, 0.0], [0.35475659370422363, 0.3502019941806793, 0.24722407758235931, 0.04781729355454445, 0.0, 0.0], [0.35370609164237976, 0.03527737781405449, 0.09567111730575562, 0.449796199798584, 0.06554921716451645, 0.0], [0.4132595360279083, 0.09055527299642563, 0.05286579951643944, 0.174679696559906, 0.173848956823349, 0.09479076415300369]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9629756212234497, 0.037024300545454025, 0.0, 0.0, 0.0, 0.0], [0.9756426811218262, 0.01965854875743389, 0.004698706325143576, 0.0, 0.0, 0.0], [0.9775736927986145, 0.013286248780786991, 0.0025590297300368547, 0.006581062916666269, 0.0, 0.0], [0.9870142936706543, 0.007388236932456493, 0.0009579154429957271, 0.0018318220973014832, 0.0028077505994588137, 0.0], [0.9409245848655701, 0.016633737832307816, 0.0022979143541306257, 0.0058906711637973785, 0.0055129327811300755, 0.02874022163450718]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.962827205657959, 0.037172831594944, 0.0, 0.0, 0.0, 0.0], [0.9582237601280212, 0.024641817435622215, 0.017134377732872963, 0.0, 0.0, 0.0], [0.9351300001144409, 0.015331573784351349, 0.014810982160270214, 0.034727465361356735, 0.0, 0.0], [0.9225171208381653, 0.010528750717639923, 0.011010154150426388, 0.01944003626704216, 0.036503832787275314, 0.0], [0.8420165777206421, 0.04357199743390083, 0.007488282397389412, 0.01496153138577938, 0.02385285682976246, 0.06810864061117172]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9926387071609497, 0.00736132962629199, 0.0, 0.0, 0.0, 0.0], [0.9957393407821655, 0.0033469819463789463, 0.000913690309971571, 0.0, 0.0, 0.0], [0.9869900345802307, 0.001974786864593625, 0.001524551771581173, 0.009510699659585953, 0.0, 0.0], [0.9933527708053589, 0.001020324882119894, 0.00034337223041802645, 0.0010291127255186439, 0.004254369530826807, 0.0], [0.9749016761779785, 0.00043480272870510817, 0.0004306558985263109, 0.0012364407302811742, 0.0015347707085311413, 0.021461669355630875]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9897475242614746, 0.010252462700009346, 0.0, 0.0, 0.0, 0.0], [0.9790639281272888, 0.01650906540453434, 0.0044270907528698444, 0.0, 0.0, 0.0], [0.9521436095237732, 0.029432358220219612, 0.008943161927163601, 0.009480923414230347, 0.0, 0.0], [0.939594030380249, 0.021510960534214973, 0.010278552770614624, 0.004555229097604752, 0.024061163887381554, 0.0], [0.9205074906349182, 0.016153652220964432, 0.010818594135344028, 0.01664440892636776, 0.014566398225724697, 0.021309375762939453]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9898501634597778, 0.010149780660867691, 0.0, 0.0, 0.0, 0.0], [0.9820910096168518, 0.006907520350068808, 0.011001535691320896, 0.0, 0.0, 0.0], [0.9684997200965881, 0.008987602777779102, 0.015342563390731812, 0.007170087192207575, 0.0, 0.0], [0.9274120330810547, 0.009485266171395779, 0.022066107019782066, 0.03222890570759773, 0.008807653561234474, 0.0], [0.900665819644928, 0.021623756736516953, 0.013808279298245907, 0.009843860752880573, 0.008521373383700848, 0.04553695768117905]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9954444169998169, 0.004555588588118553, 0.0, 0.0, 0.0, 0.0], [0.995254397392273, 0.002460238989442587, 0.002285485854372382, 0.0, 0.0, 0.0], [0.9862446188926697, 0.0015168144600465894, 0.004072288051247597, 0.008166354149580002, 0.0, 0.0], [0.9889963865280151, 0.001226040069013834, 0.0007996349013410509, 0.0006774227367714047, 0.008300574496388435, 0.0], [0.9865202903747559, 0.00039427157025784254, 0.0009571771952323616, 0.0004954367759637535, 0.0009604979422874749, 0.010672281496226788]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9821295142173767, 0.017870500683784485, 0.0, 0.0, 0.0, 0.0], [0.7489436268806458, 0.22002726793289185, 0.031029189005494118, 0.0, 0.0, 0.0], [0.28547799587249756, 0.21125678718090057, 0.47871601581573486, 0.024549242109060287, 0.0, 0.0], [0.8056644201278687, 0.026974644511938095, 0.04302806034684181, 0.06993705034255981, 0.05439583212137222, 0.0], [0.3307209014892578, 0.022326624020934105, 0.016627125442028046, 0.08019453287124634, 0.41574832797050476, 0.13438253104686737]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9697746634483337, 0.030225319787859917, 0.0, 0.0, 0.0, 0.0], [0.9800565838813782, 0.015018894337117672, 0.004924521781504154, 0.0, 0.0, 0.0], [0.9237861037254333, 0.052764780819416046, 0.00630240747705102, 0.017146753147244453, 0.0, 0.0], [0.9451844096183777, 0.03618047758936882, 0.001989208161830902, 0.003958724904805422, 0.012687299400568008, 0.0], [0.9633325934410095, 0.018662991002202034, 0.0030418417882174253, 0.007070912979543209, 0.0050094155594706535, 0.002882065251469612]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9873244762420654, 0.012675459496676922, 0.0, 0.0, 0.0, 0.0], [0.9904569983482361, 0.0055419523268938065, 0.004001122899353504, 0.0, 0.0, 0.0], [0.9814971685409546, 0.004653455223888159, 0.003725277027115226, 0.010124054737389088, 0.0, 0.0], [0.9744365811347961, 0.004632251337170601, 0.002379992976784706, 0.006518087349832058, 0.012033028528094292, 0.0], [0.9624497294425964, 0.0033743639942258596, 0.0013198587112128735, 0.0017275003483518958, 0.002944675739854574, 0.028183799237012863]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9807674288749695, 0.01923258602619171, 0.0, 0.0, 0.0, 0.0], [0.9664245843887329, 0.015413926914334297, 0.018161438405513763, 0.0, 0.0, 0.0], [0.9632682204246521, 0.004538117907941341, 0.002925391308963299, 0.029268190264701843, 0.0, 0.0], [0.9562349319458008, 0.0012223608791828156, 0.0005304080550558865, 0.00867149606347084, 0.03334089741110802, 0.0], [0.9657101035118103, 0.0009808284230530262, 0.0016686266753822565, 0.002634831238538027, 0.005866361316293478, 0.023139292374253273]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9639716148376465, 0.036028459668159485, 0.0, 0.0, 0.0, 0.0], [0.9562800526618958, 0.03373315557837486, 0.009986846707761288, 0.0, 0.0, 0.0], [0.8539998531341553, 0.08073022216558456, 0.03334445133805275, 0.031925540417432785, 0.0, 0.0], [0.9547491073608398, 0.009605025872588158, 0.004146162886172533, 0.0020133228972554207, 0.029486361891031265, 0.0], [0.9331137537956238, 0.028699662536382675, 0.005477475933730602, 0.006368075497448444, 0.012613046914339066, 0.013728085905313492]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9392993450164795, 0.06070063263177872, 0.0, 0.0, 0.0, 0.0], [0.9298391342163086, 0.061895377933979034, 0.008265496231615543, 0.0, 0.0, 0.0], [0.8471823334693909, 0.09035038203001022, 0.01763608679175377, 0.044831156730651855, 0.0, 0.0], [0.8857703804969788, 0.03918175399303436, 0.007867704145610332, 0.02276589721441269, 0.04441439360380173, 0.0], [0.8563280701637268, 0.10088995099067688, 0.006531452294439077, 0.008485927246510983, 0.007368441205471754, 0.020396249368786812]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8353264331817627, 0.1646735519170761, 0.0, 0.0, 0.0, 0.0], [0.6160858869552612, 0.3137648403644562, 0.07014927268028259, 0.0, 0.0, 0.0], [0.34316325187683105, 0.2758493721485138, 0.1196604073047638, 0.26132699847221375, 0.0, 0.0], [0.5908172130584717, 0.050290752202272415, 0.041665926575660706, 0.2199493646621704, 0.0972767099738121, 0.0], [0.8481413125991821, 0.06318090111017227, 0.014733693562448025, 0.055267371237277985, 0.00901501253247261, 0.009661628864705563]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9627319574356079, 0.03726799786090851, 0.0, 0.0, 0.0, 0.0], [0.7757522463798523, 0.1799626499414444, 0.044285036623477936, 0.0, 0.0, 0.0], [0.6317060589790344, 0.24380716681480408, 0.10925652086734772, 0.015230235643684864, 0.0, 0.0], [0.9539909958839417, 0.018182311207056046, 0.011601822450757027, 0.012299076654016972, 0.003925766795873642, 0.0], [0.40356943011283875, 0.14237558841705322, 0.05661217123270035, 0.1975736767053604, 0.0929921343922615, 0.10687707364559174]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9802619218826294, 0.019738124683499336, 0.0, 0.0, 0.0, 0.0], [0.9873908162117004, 0.007800452411174774, 0.004808681085705757, 0.0, 0.0, 0.0], [0.9283918738365173, 0.008301235735416412, 0.01330565195530653, 0.05000120773911476, 0.0, 0.0], [0.8981055021286011, 0.015591299161314964, 0.010177576914429665, 0.039987027645111084, 0.0361386202275753, 0.0], [0.9753499031066895, 0.00035433052107691765, 0.0005866039427928627, 0.0011877501383423805, 0.0010750899091362953, 0.021446440368890762]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9295330047607422, 0.07046692818403244, 0.0, 0.0, 0.0, 0.0], [0.9361506104469299, 0.04116682708263397, 0.022682538256049156, 0.0, 0.0, 0.0], [0.8486821055412292, 0.05802798643708229, 0.024856165051460266, 0.0684337466955185, 0.0, 0.0], [0.8661180734634399, 0.02232467755675316, 0.010369130410254002, 0.02600197121500969, 0.07518619298934937, 0.0], [0.8074421882629395, 0.044382549822330475, 0.01849711686372757, 0.03357789292931557, 0.018561245873570442, 0.07753907144069672]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9680535197257996, 0.03194643557071686, 0.0, 0.0, 0.0, 0.0], [0.9693689942359924, 0.02568492479622364, 0.004946070723235607, 0.0, 0.0, 0.0], [0.9620568156242371, 0.022552406415343285, 0.005471326876431704, 0.009919456206262112, 0.0, 0.0], [0.9727528095245361, 0.010137127712368965, 0.000757327419705689, 0.0028828983195126057, 0.013469807803630829, 0.0], [0.9624635577201843, 0.0031109037809073925, 0.0010007602395489812, 0.0019475930603221059, 0.008266227319836617, 0.02321087196469307]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8542501330375671, 0.14574992656707764, 0.0, 0.0, 0.0, 0.0], [0.9725967645645142, 0.014116315171122551, 0.01328685600310564, 0.0, 0.0, 0.0], [0.9257621765136719, 0.03257262706756592, 0.01461210660636425, 0.027053095400333405, 0.0, 0.0], [0.7923423051834106, 0.027305101975798607, 0.01880674995481968, 0.13854165375232697, 0.023004096001386642, 0.0], [0.6152060627937317, 0.02665526419878006, 0.029352931305766106, 0.05590886250138283, 0.11611279845237732, 0.15676409006118774]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9804654121398926, 0.019534552469849586, 0.0, 0.0, 0.0, 0.0], [0.9882452487945557, 0.007509466726332903, 0.004245325922966003, 0.0, 0.0, 0.0], [0.9584206938743591, 0.0109635591506958, 0.010456060990691185, 0.020159708335995674, 0.0, 0.0], [0.9604811668395996, 0.007182627450674772, 0.003072339342907071, 0.006898913532495499, 0.02236509881913662, 0.0], [0.966888964176178, 0.0032812939025461674, 0.00550054432824254, 0.004234083462506533, 0.005038043484091759, 0.015057181939482689]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9498194456100464, 0.05018055811524391, 0.0, 0.0, 0.0, 0.0], [0.9781363606452942, 0.016430046409368515, 0.0054335566237568855, 0.0, 0.0, 0.0], [0.8618696331977844, 0.036093585193157196, 0.07555554062128067, 0.026481209322810173, 0.0, 0.0], [0.5449837446212769, 0.015411133877933025, 0.023516526445746422, 0.25743600726127625, 0.15865260362625122, 0.0], [0.9571874737739563, 0.0030803855042904615, 0.0014446862041950226, 0.006861559115350246, 0.014818714000284672, 0.01660723052918911]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.6156560778617859, 0.3843439519405365, 0.0, 0.0, 0.0, 0.0], [0.36760634183883667, 0.42816370725631714, 0.20423001050949097, 0.0, 0.0, 0.0], [0.16471554338932037, 0.4136792719364166, 0.2509237229824066, 0.17068152129650116, 0.0, 0.0], [0.4184456169605255, 0.1524762362241745, 0.10305401682853699, 0.11071498692035675, 0.21530911326408386, 0.0], [0.19686934351921082, 0.2014620453119278, 0.12827259302139282, 0.09203246980905533, 0.09167550504207611, 0.2896881103515625]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9027364253997803, 0.09726352989673615, 0.0, 0.0, 0.0, 0.0], [0.9736634492874146, 0.014004302211105824, 0.01233230996876955, 0.0, 0.0, 0.0], [0.8504456281661987, 0.05690572410821915, 0.032060906291007996, 0.06058764085173607, 0.0, 0.0], [0.7661210298538208, 0.03530392050743103, 0.03433045372366905, 0.09675204753875732, 0.06749245524406433, 0.0], [0.8650374412536621, 0.020085260272026062, 0.01149806659668684, 0.01855834573507309, 0.018430285155773163, 0.06639053672552109]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9653082489967346, 0.03469168767333031, 0.0, 0.0, 0.0, 0.0], [0.9816323518753052, 0.014176066033542156, 0.004191514104604721, 0.0, 0.0, 0.0], [0.9275256395339966, 0.04737218841910362, 0.01152826938778162, 0.013573966920375824, 0.0, 0.0], [0.9293117523193359, 0.025833239778876305, 0.007227106485515833, 0.014300585724413395, 0.02332727052271366, 0.0], [0.8895062804222107, 0.04689619690179825, 0.0047171092592179775, 0.006286581978201866, 0.00609014043584466, 0.04650374501943588]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8938026428222656, 0.10619727522134781, 0.0, 0.0, 0.0, 0.0], [0.8221707940101624, 0.06304481625556946, 0.11478441953659058, 0.0, 0.0, 0.0], [0.5047380924224854, 0.15375731885433197, 0.2277037501335144, 0.11380083113908768, 0.0, 0.0], [0.4082071781158447, 0.09066355973482132, 0.11696872115135193, 0.24553199112415314, 0.13862857222557068, 0.0], [0.7291035652160645, 0.06638889014720917, 0.023112818598747253, 0.031103096902370453, 0.057143256068229675, 0.09314827620983124]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9247531890869141, 0.07524678111076355, 0.0, 0.0, 0.0, 0.0], [0.8957376480102539, 0.06989553570747375, 0.03436679765582085, 0.0, 0.0, 0.0], [0.7924937605857849, 0.0960114598274231, 0.05509118735790253, 0.056403566151857376, 0.0, 0.0], [0.7891505360603333, 0.07880303263664246, 0.03840155899524689, 0.05396979674696922, 0.03967496380209923, 0.0], [0.7807856798171997, 0.0799354612827301, 0.042531758546829224, 0.03234211727976799, 0.0178169384598732, 0.046588052064180374]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9480886459350586, 0.05191127583384514, 0.0, 0.0, 0.0, 0.0], [0.863694965839386, 0.04756204038858414, 0.08874296396970749, 0.0, 0.0, 0.0], [0.9341371059417725, 0.022224076092243195, 0.022624483332037926, 0.021014342084527016, 0.0, 0.0], [0.9588143229484558, 0.008020909503102303, 0.004490078426897526, 0.005862293299287558, 0.022812429815530777, 0.0], [0.9385918378829956, 0.021227721124887466, 0.0048724692314863205, 0.010940189473330975, 0.009524582885205746, 0.014843451790511608]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9763734340667725, 0.023626558482646942, 0.0, 0.0, 0.0, 0.0], [0.9884802103042603, 0.005189393647015095, 0.0063303736969828606, 0.0, 0.0, 0.0], [0.9477092027664185, 0.0179851483553648, 0.010156610049307346, 0.024149026721715927, 0.0, 0.0], [0.967192530632019, 0.006552813574671745, 0.0033227826934307814, 0.00556332478299737, 0.017368387430906296, 0.0], [0.9584562182426453, 0.007502961438149214, 0.0051363310776650906, 0.008071648888289928, 0.005997124593704939, 0.014835843816399574]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.884070873260498, 0.11592914164066315, 0.0, 0.0, 0.0, 0.0], [0.9931254386901855, 0.005070806015282869, 0.0018038019770756364, 0.0, 0.0, 0.0], [0.9534159302711487, 0.02382904477417469, 0.007748977281153202, 0.015006075613200665, 0.0, 0.0], [0.9151289463043213, 0.010873105376958847, 0.013190957717597485, 0.011050421744585037, 0.04975655674934387, 0.0], [0.8769673109054565, 0.03385210782289505, 0.00848648976534605, 0.009969149716198444, 0.03468578681349754, 0.036039214581251144]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0003709519514814019, 0.999629020690918, 0.0, 0.0, 0.0, 0.0], [6.525027856696397e-05, 0.3737829029560089, 0.6261518597602844, 0.0, 0.0, 0.0], [4.606018774211407e-05, 0.210508793592453, 0.4115968942642212, 0.3778482675552368, 0.0, 0.0], [4.753069515572861e-05, 0.11616954207420349, 0.23264272511005402, 0.3985331058502197, 0.2526070475578308, 0.0], [1.247641534973809e-06, 0.14819711446762085, 0.15813173353672028, 0.30074331164360046, 0.11939018964767456, 0.27353641390800476]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.971555769443512, 0.028444187715649605, 0.0, 0.0, 0.0, 0.0], [0.9529065489768982, 0.03233075141906738, 0.014762768521904945, 0.0, 0.0, 0.0], [0.9343128204345703, 0.02351292595267296, 0.02049802988767624, 0.021676240488886833, 0.0, 0.0], [0.9529678225517273, 0.00855141133069992, 0.004359325394034386, 0.008064556866884232, 0.026056913658976555, 0.0], [0.9653593897819519, 0.008487647399306297, 0.003499280195683241, 0.002721576252952218, 0.0032828773837536573, 0.016649367287755013]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8630780577659607, 0.13692188262939453, 0.0, 0.0, 0.0, 0.0], [0.7696157097816467, 0.0851333811879158, 0.14525099098682404, 0.0, 0.0, 0.0], [0.7133337259292603, 0.10170899331569672, 0.11931268870830536, 0.06564456224441528, 0.0, 0.0], [0.7186222076416016, 0.05444284901022911, 0.01386815495789051, 0.07808027416467667, 0.13498654961585999, 0.0], [0.7990148663520813, 0.05805593729019165, 0.009447019547224045, 0.017770467326045036, 0.02113853208720684, 0.09457314014434814]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9518988728523254, 0.048101115971803665, 0.0, 0.0, 0.0, 0.0], [0.8580653071403503, 0.02944577857851982, 0.11248888075351715, 0.0, 0.0, 0.0], [0.6577738523483276, 0.08513449877500534, 0.1261308640241623, 0.1309608370065689, 0.0, 0.0], [0.8087368607521057, 0.0323016420006752, 0.01841817982494831, 0.06856140494346619, 0.07198194414377213, 0.0], [0.6683295965194702, 0.13281384110450745, 0.021880635991692543, 0.02787741646170616, 0.04923408478498459, 0.0998644009232521]]]], \"left_text\": [\"No\", \",\", \" I\", \" am\", \" your\", \" father\"], \"right_text\": [\"No\", \",\", \" I\", \" am\", \" your\", \" father\"]}], \"default_filter\": \"0\", \"display_mode\": \"dark\", \"root_div_id\": \"bertviz-5a257cb0119e4b7ea970ea74266027b3\", \"include_layers\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], \"include_heads\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], \"total_heads\": 12}; // HACK: {\"attention\": [{\"name\": null, \"attn\": [[[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.961219847202301, 0.038780149072408676, 0.0, 0.0, 0.0, 0.0], [0.7466979026794434, 0.11987314373254776, 0.1334289014339447, 0.0, 0.0, 0.0], [0.5885030031204224, 0.13792067766189575, 0.212137371301651, 0.06143897399306297, 0.0, 0.0], [0.6570857763290405, 0.08996301889419556, 0.12751281261444092, 0.08361563086509705, 0.041822850704193115, 0.0], [0.2728874385356903, 0.11203353852033615, 0.1663985401391983, 0.08467111736536026, 0.16952736675739288, 0.19448210299015045]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.010616563260555267, 0.9893833994865417, 0.0, 0.0, 0.0, 0.0], [0.0024677535984665155, 0.008448007516562939, 0.9890841841697693, 0.0, 0.0, 0.0], [0.0001232847134815529, 0.0018733182223513722, 0.013126976788043976, 0.9848763942718506, 0.0, 0.0], [0.0010669564362615347, 0.001136627048254013, 0.003034998197108507, 0.0015735096530988812, 0.9931879043579102, 0.0], [0.00019791982776951045, 0.0010528112761676311, 0.0015437351539731026, 0.0009642760851420462, 3.4924432839034125e-05, 0.9962062835693359]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.47578439116477966, 0.524215579032898, 0.0, 0.0, 0.0, 0.0], [0.5906045436859131, 0.2486611008644104, 0.16073434054851532, 0.0, 0.0, 0.0], [0.5529289841651917, 0.18856702744960785, 0.14457571506500244, 0.11392831057310104, 0.0, 0.0], [0.45094072818756104, 0.16486799716949463, 0.17318038642406464, 0.11748014390468597, 0.09353074431419373, 0.0], [0.4257245659828186, 0.1732865273952484, 0.15651953220367432, 0.07022649794816971, 0.0808701142668724, 0.09337282180786133]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.6133623123168945, 0.38663768768310547, 0.0, 0.0, 0.0, 0.0], [0.06098509579896927, 0.03253461793065071, 0.9064802527427673, 0.0, 0.0, 0.0], [0.006717085838317871, 0.0004012881254311651, 0.7572958469390869, 0.23558568954467773, 0.0, 0.0], [0.03722766041755676, 0.002948855282738805, 0.10081092268228531, 0.04142269119620323, 0.8175898790359497, 0.0], [0.04989781975746155, 0.00030758307548239827, 0.0024198265746235847, 0.0034334994852542877, 0.0006823898293077946, 0.9432588815689087]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9489555954933167, 0.051044441759586334, 0.0, 0.0, 0.0, 0.0], [0.6821408867835999, 0.1395241767168045, 0.17833495140075684, 0.0, 0.0, 0.0], [0.20366324484348297, 0.05641487240791321, 0.06399301439523697, 0.6759288311004639, 0.0, 0.0], [0.3419547975063324, 0.06725440919399261, 0.07926183938980103, 0.1783619523048401, 0.3331669867038727, 0.0], [0.09464015811681747, 0.0074282134883105755, 0.006983973551541567, 0.0071843694895505905, 0.018724264577031136, 0.865039050579071]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.33834606409072876, 0.6616539359092712, 0.0, 0.0, 0.0, 0.0], [0.07855993509292603, 0.006165449041873217, 0.9152746200561523, 0.0, 0.0, 0.0], [0.01677597686648369, 0.0004037705948576331, 0.003340460592880845, 0.9794798493385315, 0.0, 0.0], [0.027600426226854324, 0.00044415233423933387, 0.0006541680195368826, 0.0002266185765620321, 0.971074640750885, 0.0], [0.010248198173940182, 3.701553578139283e-05, 0.00016064041119534522, 2.7341819077264518e-05, 1.0187304724240676e-05, 0.98951655626297]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.982503354549408, 0.017496665939688683, 0.0, 0.0, 0.0, 0.0], [0.8874197006225586, 0.05467939004302025, 0.05790085718035698, 0.0, 0.0, 0.0], [0.6849910616874695, 0.1228068619966507, 0.04972026124596596, 0.14248186349868774, 0.0, 0.0], [0.6015856862068176, 0.09881888329982758, 0.07070108503103256, 0.16652540862560272, 0.06236903741955757, 0.0], [0.3232504427433014, 0.12567411363124847, 0.04432179778814316, 0.07076980918645859, 0.06606649607419968, 0.36991727352142334]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9191647171974182, 0.0808352455496788, 0.0, 0.0, 0.0, 0.0], [0.45986413955688477, 0.39703112840652466, 0.14310479164123535, 0.0, 0.0, 0.0], [0.3003872334957123, 0.22181738913059235, 0.38161516189575195, 0.09618020057678223, 0.0, 0.0], [0.18963925540447235, 0.1376371532678604, 0.20173484086990356, 0.23632164299488068, 0.23466713726520538, 0.0], [0.15410441160202026, 0.09489496797323227, 0.11902562528848648, 0.10277965664863586, 0.4317220449447632, 0.09747327119112015]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.364999920129776, 0.6350001096725464, 0.0, 0.0, 0.0, 0.0], [0.24595215916633606, 0.5519201755523682, 0.20212766528129578, 0.0, 0.0, 0.0], [0.2721358835697174, 0.40738627314567566, 0.25186213850975037, 0.06861574947834015, 0.0, 0.0], [0.10242555290460587, 0.16683615744113922, 0.524804949760437, 0.05445462837815285, 0.15147870779037476, 0.0], [0.25029507279396057, 0.22198128700256348, 0.18899968266487122, 0.10677118599414825, 0.1303267478942871, 0.10162602365016937]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.6990506649017334, 0.3009493350982666, 0.0, 0.0, 0.0, 0.0], [0.5107942819595337, 0.2948642075061798, 0.1943415403366089, 0.0, 0.0, 0.0], [0.4604707360267639, 0.2805190980434418, 0.19174803793430328, 0.0672621801495552, 0.0, 0.0], [0.37648412585258484, 0.21120662987232208, 0.20214538276195526, 0.10207021236419678, 0.10809355974197388, 0.0], [0.30138441920280457, 0.20456179976463318, 0.18250338733196259, 0.11019382625818253, 0.1629127413034439, 0.03844383731484413]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.7131582498550415, 0.2868417799472809, 0.0, 0.0, 0.0, 0.0], [0.4058799147605896, 0.18063297867774963, 0.41348710656166077, 0.0, 0.0, 0.0], [0.265546053647995, 0.1698586493730545, 0.3358593285083771, 0.228736013174057, 0.0, 0.0], [0.31385406851768494, 0.1831669807434082, 0.14928358793258667, 0.05377671495079994, 0.29991865158081055, 0.0], [0.20466560125350952, 0.18731118738651276, 0.15959151089191437, 0.06381776183843613, 0.03642302006483078, 0.34819093346595764]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.6586242914199829, 0.3413757383823395, 0.0, 0.0, 0.0, 0.0], [0.5917776226997375, 0.3160035014152527, 0.0922188088297844, 0.0, 0.0, 0.0], [0.5477152466773987, 0.23586955666542053, 0.061456020921468735, 0.1549593061208725, 0.0, 0.0], [0.4587061107158661, 0.22439992427825928, 0.07887422293424606, 0.0992034301161766, 0.13881628215312958, 0.0], [0.32743722200393677, 0.19600819051265717, 0.068057119846344, 0.0892510637640953, 0.11618079245090485, 0.20306548476219177]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9961552023887634, 0.0038448425475507975, 0.0, 0.0, 0.0, 0.0], [0.8594854474067688, 0.06906110048294067, 0.07145342975854874, 0.0, 0.0, 0.0], [0.3800053000450134, 0.04127567633986473, 0.5496612787246704, 0.029057776555418968, 0.0, 0.0], [0.21445226669311523, 0.05088742449879646, 0.4317440092563629, 0.25869303941726685, 0.044223275035619736, 0.0], [0.11175256222486496, 0.017593080177903175, 0.027507441118359566, 0.04086771607398987, 0.7754669785499573, 0.026812179014086723]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9285967946052551, 0.07140326499938965, 0.0, 0.0, 0.0, 0.0], [0.6077286005020142, 0.3121427297592163, 0.08012867718935013, 0.0, 0.0, 0.0], [0.4942909777164459, 0.28503698110580444, 0.11849315464496613, 0.10217894613742828, 0.0, 0.0], [0.4183879494667053, 0.23117904365062714, 0.0834062322974205, 0.11365949362516403, 0.1533672958612442, 0.0], [0.42215850949287415, 0.12917140126228333, 0.08740927278995514, 0.1016375944018364, 0.21230268478393555, 0.04732053726911545]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9786475896835327, 0.02135237120091915, 0.0, 0.0, 0.0, 0.0], [0.7749121785163879, 0.06510371714830399, 0.15998409688472748, 0.0, 0.0, 0.0], [0.6484923362731934, 0.07483134418725967, 0.14751605689525604, 0.12916021049022675, 0.0, 0.0], [0.5224639773368835, 0.06921815127134323, 0.13823404908180237, 0.1110658198595047, 0.15901805460453033, 0.0], [0.3964517116546631, 0.07325823605060577, 0.12938153743743896, 0.1064242571592331, 0.14864002168178558, 0.1458442211151123]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.5525906085968018, 0.44740936160087585, 0.0, 0.0, 0.0, 0.0], [0.5585009455680847, 0.2176259458065033, 0.22387312352657318, 0.0, 0.0, 0.0], [0.5143128633499146, 0.15964674949645996, 0.15491968393325806, 0.1711207628250122, 0.0, 0.0], [0.5039961338043213, 0.11401888728141785, 0.11974027007818222, 0.12552587687969208, 0.13671889901161194, 0.0], [0.5061842799186707, 0.08567393571138382, 0.08903021365404129, 0.09759818762540817, 0.1027572825551033, 0.11875619739294052]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9242545366287231, 0.07574543356895447, 0.0, 0.0, 0.0, 0.0], [0.8257425427436829, 0.07932533323764801, 0.09493216127157211, 0.0, 0.0, 0.0], [0.7306380271911621, 0.0857183039188385, 0.08043931424617767, 0.10320431739091873, 0.0, 0.0], [0.6383238434791565, 0.07886394113302231, 0.07815027981996536, 0.08758097141981125, 0.1170809343457222, 0.0], [0.5552157163619995, 0.07409121096134186, 0.06834889203310013, 0.07778600603342056, 0.09999319165945053, 0.12456497550010681]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8578913807868958, 0.14210854470729828, 0.0, 0.0, 0.0, 0.0], [0.6423038244247437, 0.166290283203125, 0.19140593707561493, 0.0, 0.0, 0.0], [0.5530979633331299, 0.10609274357557297, 0.07821257412433624, 0.26259663701057434, 0.0, 0.0], [0.40121692419052124, 0.12223611027002335, 0.1934729963541031, 0.14164622128009796, 0.14142780005931854, 0.0], [0.40212565660476685, 0.18450751900672913, 0.07516805827617645, 0.05849048122763634, 0.1444634348154068, 0.13524490594863892]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9791558980941772, 0.020844051614403725, 0.0, 0.0, 0.0, 0.0], [0.8829841613769531, 0.06233249977231026, 0.05468335747718811, 0.0, 0.0, 0.0], [0.8105455040931702, 0.08617085963487625, 0.07321777194738388, 0.03006584383547306, 0.0, 0.0], [0.6819812059402466, 0.04990820586681366, 0.08296552300453186, 0.08369525521993637, 0.10144983977079391, 0.0], [0.4056689441204071, 0.07337666302919388, 0.08601408451795578, 0.061709366738796234, 0.13226434588432312, 0.2409665435552597]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9670190811157227, 0.03298088163137436, 0.0, 0.0, 0.0, 0.0], [0.8449064493179321, 0.0851450264453888, 0.06994850933551788, 0.0, 0.0, 0.0], [0.7123572826385498, 0.07896047830581665, 0.055410757660865784, 0.15327158570289612, 0.0, 0.0], [0.6402613520622253, 0.0739755630493164, 0.044393062591552734, 0.14322125911712646, 0.09814881533384323, 0.0], [0.5073903799057007, 0.07523059099912643, 0.07754647731781006, 0.11362491548061371, 0.13947951793670654, 0.08672808855772018]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8487569093704224, 0.1512431502342224, 0.0, 0.0, 0.0, 0.0], [0.8415648937225342, 0.12107233703136444, 0.03736274689435959, 0.0, 0.0, 0.0], [0.7505517601966858, 0.11348944902420044, 0.06179959326982498, 0.07415912300348282, 0.0, 0.0], [0.6614719033241272, 0.10242646187543869, 0.052934251725673676, 0.07529708743095398, 0.10787025839090347, 0.0], [0.6014202237129211, 0.11340376734733582, 0.05631929263472557, 0.07096721231937408, 0.10906282067298889, 0.04882663115859032]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9445484280586243, 0.05545158311724663, 0.0, 0.0, 0.0, 0.0], [0.8874568939208984, 0.05474215745925903, 0.0578010231256485, 0.0, 0.0, 0.0], [0.8281888961791992, 0.06895001977682114, 0.059034693986177444, 0.0438263975083828, 0.0, 0.0], [0.6429892778396606, 0.0674755647778511, 0.11629703640937805, 0.05417950078845024, 0.11905858665704727, 0.0], [0.7367823719978333, 0.056119054555892944, 0.06857288628816605, 0.034219540655612946, 0.0787537544965744, 0.02555238828063011]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0002913394710049033, 0.9997085928916931, 0.0, 0.0, 0.0, 0.0], [0.0007981209782883525, 0.5288336873054504, 0.4703682065010071, 0.0, 0.0, 0.0], [0.0007648481405340135, 0.34519824385643005, 0.3085267245769501, 0.34551018476486206, 0.0, 0.0], [0.0010283143492415547, 0.241359144449234, 0.23320138454437256, 0.2555713355541229, 0.2688397467136383, 0.0], [0.0009746829164214432, 0.17789699137210846, 0.16743157804012299, 0.1858760118484497, 0.18734444677829742, 0.28047630190849304]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.824492871761322, 0.17550717294216156, 0.0, 0.0, 0.0, 0.0], [0.12386877834796906, 0.044499922543764114, 0.8316312432289124, 0.0, 0.0, 0.0], [0.07924355566501617, 0.01296587660908699, 0.0015277155907824636, 0.9062628149986267, 0.0, 0.0], [0.08806384354829788, 0.0213409923017025, 0.0028886159416288137, 0.002845379989594221, 0.884861171245575, 0.0], [0.09983218461275101, 0.03363388776779175, 0.0054999832063913345, 0.002433052286505699, 0.0015082412865012884, 0.8570926189422607]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9646892547607422, 0.03531072288751602, 0.0, 0.0, 0.0, 0.0], [0.7529157400131226, 0.08733473718166351, 0.15974950790405273, 0.0, 0.0, 0.0], [0.4202282726764679, 0.09195102006196976, 0.23549850285053253, 0.25232216715812683, 0.0, 0.0], [0.30848920345306396, 0.05908140912652016, 0.38391315937042236, 0.15659146010875702, 0.09192468225955963, 0.0], [0.44790443778038025, 0.04329312965273857, 0.0796918049454689, 0.11081931740045547, 0.22124572098255157, 0.09704558551311493]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.991096019744873, 0.008904009126126766, 0.0, 0.0, 0.0, 0.0], [0.9697675704956055, 0.026084503158926964, 0.004147922620177269, 0.0, 0.0, 0.0], [0.9082901477813721, 0.033206019550561905, 0.00942116230726242, 0.049082688987255096, 0.0, 0.0], [0.8949133157730103, 0.05544555187225342, 0.005577624775469303, 0.03150692582130432, 0.012556522153317928, 0.0], [0.8497740030288696, 0.028890123590826988, 0.0036647915840148926, 0.03751987963914871, 0.038427725434303284, 0.04172350466251373]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9984525442123413, 0.0015474462416023016, 0.0, 0.0, 0.0, 0.0], [0.48947831988334656, 0.4812193810939789, 0.029302269220352173, 0.0, 0.0, 0.0], [0.11772153526544571, 0.13121186196804047, 0.6702314615249634, 0.08083520829677582, 0.0, 0.0], [0.13043689727783203, 0.04068669304251671, 0.2652038037776947, 0.4114362895488739, 0.15223638713359833, 0.0], [0.12661904096603394, 0.03275119513273239, 0.03567872568964958, 0.06039190664887428, 0.6021825075149536, 0.1423766165971756]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9805176854133606, 0.019482342526316643, 0.0, 0.0, 0.0, 0.0], [0.7948849201202393, 0.12061909586191177, 0.08449601382017136, 0.0, 0.0, 0.0], [0.5612356066703796, 0.15743127465248108, 0.20339730381965637, 0.0779358446598053, 0.0, 0.0], [0.42583736777305603, 0.10742014646530151, 0.15123659372329712, 0.08755031228065491, 0.22795552015304565, 0.0], [0.24752654135227203, 0.024188270792365074, 0.03039524517953396, 0.08586956560611725, 0.5714336633682251, 0.040586672723293304]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9887767434120178, 0.011223225854337215, 0.0, 0.0, 0.0, 0.0], [0.7572693228721619, 0.22317346930503845, 0.019557112827897072, 0.0, 0.0, 0.0], [0.5341880321502686, 0.22107566893100739, 0.1762184202671051, 0.06851787120103836, 0.0, 0.0], [0.17095312476158142, 0.0822940468788147, 0.576022207736969, 0.11097585409879684, 0.059754710644483566, 0.0], [0.2487109899520874, 0.08880793303251266, 0.08980197459459305, 0.09729334712028503, 0.4413093626499176, 0.03407646715641022]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8422133326530457, 0.15778663754463196, 0.0, 0.0, 0.0, 0.0], [0.468412846326828, 0.46105360984802246, 0.07053359597921371, 0.0, 0.0, 0.0], [0.2588140666484833, 0.4635888636112213, 0.18503506481647491, 0.09256205707788467, 0.0, 0.0], [0.18399578332901, 0.29154160618782043, 0.17031098902225494, 0.27173006534576416, 0.08242159336805344, 0.0], [0.1646990180015564, 0.2472696155309677, 0.08770562708377838, 0.22575001418590546, 0.1774536371231079, 0.09712201356887817]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9919946193695068, 0.008005390875041485, 0.0, 0.0, 0.0, 0.0], [0.9068724513053894, 0.044065121561288834, 0.04906242713332176, 0.0, 0.0, 0.0], [0.8582221865653992, 0.055348269641399384, 0.040419407188892365, 0.046010036021471024, 0.0, 0.0], [0.7855252623558044, 0.041242364794015884, 0.08369296044111252, 0.04887620359659195, 0.040663279592990875, 0.0], [0.7856317162513733, 0.05014643445611, 0.04751267284154892, 0.027365952730178833, 0.05614755302667618, 0.03319567069411278]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9041035175323486, 0.09589648246765137, 0.0, 0.0, 0.0, 0.0], [0.5862312912940979, 0.07199832051992416, 0.34177035093307495, 0.0, 0.0, 0.0], [0.3878960907459259, 0.04660807177424431, 0.20278996229171753, 0.36270591616630554, 0.0, 0.0], [0.2665242552757263, 0.024533024057745934, 0.12211935967206955, 0.20041218400001526, 0.386411190032959, 0.0], [0.23357485234737396, 0.02053728699684143, 0.09610321372747421, 0.13062246143817902, 0.22990450263023376, 0.289257675409317]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9639912247657776, 0.036008793860673904, 0.0, 0.0, 0.0, 0.0], [0.7075552344322205, 0.2542775869369507, 0.038167137652635574, 0.0, 0.0, 0.0], [0.2566526234149933, 0.20589298009872437, 0.01665665954351425, 0.5207977294921875, 0.0, 0.0], [0.1037939190864563, 0.04639088362455368, 0.008698614314198494, 0.7866851687431335, 0.05443140119314194, 0.0], [0.2214341163635254, 0.03379744663834572, 0.029023902490735054, 0.541292130947113, 0.15286092460155487, 0.021591555327177048]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9891703724861145, 0.010829661041498184, 0.0, 0.0, 0.0, 0.0], [0.7913155555725098, 0.12309625744819641, 0.08558809012174606, 0.0, 0.0, 0.0], [0.2954600155353546, 0.15808308124542236, 0.4217240810394287, 0.1247328370809555, 0.0, 0.0], [0.23440983891487122, 0.09886523336172104, 0.33160170912742615, 0.1971396654844284, 0.1379835456609726, 0.0], [0.19728390872478485, 0.05741839483380318, 0.06909029185771942, 0.16469819843769073, 0.2797277867794037, 0.23178131878376007]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9359127879142761, 0.0640871673822403, 0.0, 0.0, 0.0, 0.0], [0.7888627648353577, 0.08673475682735443, 0.12440246343612671, 0.0, 0.0, 0.0], [0.6535118818283081, 0.07573551684617996, 0.09732568264007568, 0.17342689633369446, 0.0, 0.0], [0.522276759147644, 0.058278825134038925, 0.09920477122068405, 0.17020836472511292, 0.15003129839897156, 0.0], [0.4108840823173523, 0.047306034713983536, 0.07265672832727432, 0.10560744255781174, 0.10550004243850708, 0.25804558396339417]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9683833122253418, 0.03161672502756119, 0.0, 0.0, 0.0, 0.0], [0.8965396881103516, 0.038870569318532944, 0.06458976864814758, 0.0, 0.0, 0.0], [0.8264952898025513, 0.03213464096188545, 0.05196719989180565, 0.0894029513001442, 0.0, 0.0], [0.7718173265457153, 0.030402837321162224, 0.045827414840459824, 0.07118473201990128, 0.08076759427785873, 0.0], [0.7292331457138062, 0.021699821576476097, 0.033074747771024704, 0.04720093309879303, 0.06474557518959045, 0.10404567420482635]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9979567527770996, 0.0020432830788195133, 0.0, 0.0, 0.0, 0.0], [0.955294132232666, 0.00802531372755766, 0.03668047487735748, 0.0, 0.0, 0.0], [0.9254710078239441, 0.002755576279014349, 0.0020629852078855038, 0.06971040368080139, 0.0, 0.0], [0.8660576939582825, 0.0038883681409060955, 0.0006785982404835522, 0.0006981453043408692, 0.1286771297454834, 0.0], [0.8455929160118103, 0.0037804055027663708, 0.000253423087997362, 6.0270751419011503e-05, 0.00011820747749879956, 0.15019479393959045]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9262455105781555, 0.07375453412532806, 0.0, 0.0, 0.0, 0.0], [0.7717157006263733, 0.16241952776908875, 0.06586471945047379, 0.0, 0.0, 0.0], [0.8167637586593628, 0.07807160913944244, 0.06324034929275513, 0.041924238204956055, 0.0, 0.0], [0.6867184638977051, 0.07755157351493835, 0.10056912153959274, 0.05955080687999725, 0.07561002671718597, 0.0], [0.6421161890029907, 0.11014898866415024, 0.07688194513320923, 0.054033469408750534, 0.10333634912967682, 0.013483096845448017]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9395954608917236, 0.060404520481824875, 0.0, 0.0, 0.0, 0.0], [0.23004619777202606, 0.6617380380630493, 0.1082158014178276, 0.0, 0.0, 0.0], [0.2670227289199829, 0.3607950508594513, 0.3249626159667969, 0.047219593077898026, 0.0, 0.0], [0.595201313495636, 0.12269274890422821, 0.06302059441804886, 0.08916817605495453, 0.12991715967655182, 0.0], [0.10284596681594849, 0.02938011661171913, 0.013739082030951977, 0.045860596001148224, 0.7698501348495483, 0.03832406550645828]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9040980935096741, 0.09590194374322891, 0.0, 0.0, 0.0, 0.0], [0.357237845659256, 0.6274612545967102, 0.015300876460969448, 0.0, 0.0, 0.0], [0.5917996764183044, 0.2764042019844055, 0.10476048290729523, 0.027035649865865707, 0.0, 0.0], [0.7254403829574585, 0.04983152449131012, 0.014982940629124641, 0.1778142899274826, 0.031930916011333466, 0.0], [0.7612743973731995, 0.06158972904086113, 0.005942251533269882, 0.01642685756087303, 0.1267806589603424, 0.0279861893504858]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9947587847709656, 0.005241230130195618, 0.0, 0.0, 0.0, 0.0], [0.9632415771484375, 0.017816413193941116, 0.018942030146718025, 0.0, 0.0, 0.0], [0.9671078324317932, 0.008509586565196514, 0.00856222677975893, 0.015820473432540894, 0.0, 0.0], [0.9340996146202087, 0.011952387169003487, 0.02018021047115326, 0.02675083465874195, 0.0070168930105865, 0.0], [0.9587237238883972, 0.004657115787267685, 0.003326789475977421, 0.006545313633978367, 0.010182461701333523, 0.016564540565013885]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9769991040229797, 0.023000910878181458, 0.0, 0.0, 0.0, 0.0], [0.7917609214782715, 0.1753319948911667, 0.032907065004110336, 0.0, 0.0, 0.0], [0.7949192523956299, 0.10531841963529587, 0.040218502283096313, 0.05954383686184883, 0.0, 0.0], [0.7097718715667725, 0.10552527755498886, 0.06597573310136795, 0.05765606462955475, 0.061070989817380905, 0.0], [0.7506601214408875, 0.026514461264014244, 0.021576043218374252, 0.034296683967113495, 0.08494450151920319, 0.08200812339782715]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.983751654624939, 0.016248304396867752, 0.0, 0.0, 0.0, 0.0], [0.5615494847297668, 0.08956841379404068, 0.3488820493221283, 0.0, 0.0, 0.0], [0.32929039001464844, 0.024114903062582016, 0.5428059697151184, 0.10378880053758621, 0.0, 0.0], [0.34330207109451294, 0.01308644749224186, 0.5121983289718628, 0.11146228760480881, 0.019950881600379944, 0.0], [0.4792812764644623, 0.01733359508216381, 0.1180536150932312, 0.06130281835794449, 0.20071913301944733, 0.12330964207649231]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9908847212791443, 0.009115329943597317, 0.0, 0.0, 0.0, 0.0], [0.5282707214355469, 0.3292262554168701, 0.1425030380487442, 0.0, 0.0, 0.0], [0.48788541555404663, 0.23368670046329498, 0.17578084766864777, 0.10264702141284943, 0.0, 0.0], [0.31444698572158813, 0.18065163493156433, 0.168714240193367, 0.09506598114967346, 0.24112118780612946, 0.0], [0.5168765187263489, 0.035897161811590195, 0.026188155636191368, 0.04039734974503517, 0.18791745603084564, 0.1927233189344406]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8750308156013489, 0.12496919929981232, 0.0, 0.0, 0.0, 0.0], [0.4550614655017853, 0.4900427758693695, 0.05489582195878029, 0.0, 0.0, 0.0], [0.2933720052242279, 0.5449907183647156, 0.09444297850131989, 0.06719419360160828, 0.0, 0.0], [0.489708811044693, 0.2720997631549835, 0.06861965358257294, 0.14694802463054657, 0.022623788565397263, 0.0], [0.4729066491127014, 0.08103099465370178, 0.016052134335041046, 0.30672287940979004, 0.10120721161365509, 0.022080255672335625]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9630220532417297, 0.03697792813181877, 0.0, 0.0, 0.0, 0.0], [0.7557195425033569, 0.16436372697353363, 0.07991670072078705, 0.0, 0.0, 0.0], [0.6947705745697021, 0.08409853279590607, 0.0638260766863823, 0.15730486810207367, 0.0, 0.0], [0.5821147561073303, 0.03297805413603783, 0.07936596870422363, 0.19441406428813934, 0.11112712323665619, 0.0], [0.5974540710449219, 0.04261096194386482, 0.06919723749160767, 0.14563441276550293, 0.12481734901666641, 0.020285936072468758]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9957822561264038, 0.004217816516757011, 0.0, 0.0, 0.0, 0.0], [0.9312832951545715, 0.010560247115790844, 0.05815650522708893, 0.0, 0.0, 0.0], [0.8435326814651489, 0.015695005655288696, 0.045751139521598816, 0.09502115100622177, 0.0, 0.0], [0.772409975528717, 0.011981245130300522, 0.03504609689116478, 0.03876771405339241, 0.14179500937461853, 0.0], [0.7642908692359924, 0.009868789464235306, 0.00812275055795908, 0.013314393348991871, 0.04824395477771759, 0.15615922212600708]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9701177477836609, 0.02988232672214508, 0.0, 0.0, 0.0, 0.0], [0.6564007997512817, 0.22506150603294373, 0.11853761970996857, 0.0, 0.0, 0.0], [0.6958062648773193, 0.14701850712299347, 0.07145983725786209, 0.08571550250053406, 0.0, 0.0], [0.6353274583816528, 0.1346064656972885, 0.030994214117527008, 0.056916315108537674, 0.1421555131673813, 0.0], [0.6779401898384094, 0.053654152899980545, 0.01800631172955036, 0.06284520775079727, 0.1103820651769638, 0.07717210054397583]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9822334051132202, 0.017766647040843964, 0.0, 0.0, 0.0, 0.0], [0.9037663340568542, 0.06541544198989868, 0.03081829659640789, 0.0, 0.0, 0.0], [0.8119193911552429, 0.03679030388593674, 0.060560714453458786, 0.09072960168123245, 0.0, 0.0], [0.40546438097953796, 0.10383912175893784, 0.10211236774921417, 0.35434210300445557, 0.03424208238720894, 0.0], [0.22824221849441528, 0.017278727144002914, 0.05055465176701546, 0.6015752553939819, 0.09411764144897461, 0.008231506682932377]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9873148202896118, 0.012685136869549751, 0.0, 0.0, 0.0, 0.0], [0.35445743799209595, 0.5317603349685669, 0.11378221958875656, 0.0, 0.0, 0.0], [0.07823363691568375, 0.7221359014511108, 0.10936623811721802, 0.090264230966568, 0.0, 0.0], [0.21967869997024536, 0.4048435091972351, 0.12358088046312332, 0.20018866658210754, 0.051708199083805084, 0.0], [0.36089760065078735, 0.10459021478891373, 0.06983799487352371, 0.2976483404636383, 0.13869903981685638, 0.02832675166428089]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9732162356376648, 0.0267837755382061, 0.0, 0.0, 0.0, 0.0], [0.9167553782463074, 0.061452705413103104, 0.02179192565381527, 0.0, 0.0, 0.0], [0.8543081283569336, 0.08049600571393967, 0.030334919691085815, 0.03486092761158943, 0.0, 0.0], [0.8919214606285095, 0.04280779883265495, 0.022045055404305458, 0.023470671847462654, 0.01975487545132637, 0.0], [0.8116763234138489, 0.03413533419370651, 0.03567665070295334, 0.04748587682843208, 0.0253971628844738, 0.04562860727310181]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9502761960029602, 0.04972382262349129, 0.0, 0.0, 0.0, 0.0], [0.7637454271316528, 0.2007361352443695, 0.03551840782165527, 0.0, 0.0, 0.0], [0.6279097199440002, 0.03768139332532883, 0.1994536966085434, 0.13495522737503052, 0.0, 0.0], [0.6397060751914978, 0.027007432654500008, 0.09082036465406418, 0.20653828978538513, 0.03592785820364952, 0.0], [0.4559425115585327, 0.021641194820404053, 0.12939567863941193, 0.21800927817821503, 0.10379841923713684, 0.07121295481920242]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9498406648635864, 0.050159383565187454, 0.0, 0.0, 0.0, 0.0], [0.8688724637031555, 0.0872218981385231, 0.043905653059482574, 0.0, 0.0, 0.0], [0.6937950253486633, 0.06359200924634933, 0.091790571808815, 0.15082231163978577, 0.0, 0.0], [0.7266597151756287, 0.04389883577823639, 0.04683985933661461, 0.09851823002099991, 0.08408336341381073, 0.0], [0.7848998308181763, 0.037147827446460724, 0.012907838448882103, 0.01053939200937748, 0.12079165875911713, 0.03371351957321167]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9891054034233093, 0.01089458167552948, 0.0, 0.0, 0.0, 0.0], [0.8929519653320312, 0.08700055629014969, 0.02004752680659294, 0.0, 0.0, 0.0], [0.7891124486923218, 0.09797251224517822, 0.08633202314376831, 0.026582980528473854, 0.0, 0.0], [0.8850635886192322, 0.03645012155175209, 0.05395457148551941, 0.01237727515399456, 0.012154522351920605, 0.0], [0.6861329674720764, 0.05720378831028938, 0.011636304669082165, 0.021660611033439636, 0.1748800277709961, 0.048486363142728806]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9396191835403442, 0.06038080155849457, 0.0, 0.0, 0.0, 0.0], [0.7851794958114624, 0.19751444458961487, 0.017306052148342133, 0.0, 0.0, 0.0], [0.7660509943962097, 0.15444670617580414, 0.03188290074467659, 0.04761936888098717, 0.0, 0.0], [0.703522801399231, 0.05171430483460426, 0.07760990411043167, 0.1533905267715454, 0.013762423768639565, 0.0], [0.7121888399124146, 0.04994234815239906, 0.03772548958659172, 0.08649132400751114, 0.06541401147842407, 0.04823806509375572]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.974072277545929, 0.025927715003490448, 0.0, 0.0, 0.0, 0.0], [0.792539656162262, 0.01171559002250433, 0.19574476778507233, 0.0, 0.0, 0.0], [0.5106770992279053, 0.007296787109225988, 0.039619915187358856, 0.4424062669277191, 0.0, 0.0], [0.5862472057342529, 0.012099712155759335, 0.024585209786891937, 0.06737840175628662, 0.30968940258026123, 0.0], [0.30196306109428406, 0.007724012713879347, 0.011518122628331184, 0.046947259455919266, 0.22146707773208618, 0.41038045287132263]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9744554162025452, 0.02554464340209961, 0.0, 0.0, 0.0, 0.0], [0.9769195318222046, 0.015048524364829063, 0.008031901903450489, 0.0, 0.0, 0.0], [0.9060619473457336, 0.025875424966216087, 0.025954782962799072, 0.04210779070854187, 0.0, 0.0], [0.9400081038475037, 0.00555665697902441, 0.005828304681926966, 0.031757812947034836, 0.016849134117364883, 0.0], [0.9105738401412964, 0.0019752182997763157, 0.008646721951663494, 0.013360846787691116, 0.03543964773416519, 0.030003678053617477]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9791666865348816, 0.020833350718021393, 0.0, 0.0, 0.0, 0.0], [0.8444858193397522, 0.13507869839668274, 0.020435383543372154, 0.0, 0.0, 0.0], [0.7903086543083191, 0.14559169113636017, 0.037529975175857544, 0.026569725945591927, 0.0, 0.0], [0.7298924326896667, 0.056496407836675644, 0.032735615968704224, 0.10400459170341492, 0.07687094807624817, 0.0], [0.5684185028076172, 0.04388832300901413, 0.026293467730283737, 0.0811714455485344, 0.24314835667610168, 0.037079911679029465]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9499868154525757, 0.05001320689916611, 0.0, 0.0, 0.0, 0.0], [0.9336170554161072, 0.05848868936300278, 0.007894262671470642, 0.0, 0.0, 0.0], [0.7897834181785583, 0.11071821302175522, 0.05360178276896477, 0.04589657858014107, 0.0, 0.0], [0.885930061340332, 0.05752986669540405, 0.01374326553195715, 0.0033877466339617968, 0.03940902277827263, 0.0], [0.9337607622146606, 0.02647063508629799, 0.004523396957665682, 0.0061904797330498695, 0.014132906682789326, 0.014921708963811398]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [1.0, 1.8521224554035598e-09, 0.0, 0.0, 0.0, 0.0], [6.6758907451003324e-06, 0.9999804496765137, 1.2841281204600818e-05, 0.0, 0.0, 0.0], [2.2194194926328237e-08, 2.6684581211355862e-09, 0.9999971389770508, 2.8136880700913025e-06, 0.0, 0.0], [1.0145409987671883e-06, 4.464065739284706e-08, 0.00035356366424821317, 0.9993677735328674, 0.0002776293840724975, 0.0], [9.436550429953172e-10, 1.382057315812979e-11, 5.017835036369434e-10, 2.965183876213473e-09, 0.9999971389770508, 2.8644042231462663e-06]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9948632121086121, 0.005136783700436354, 0.0, 0.0, 0.0, 0.0], [0.9274215698242188, 0.01832387037575245, 0.05425456911325455, 0.0, 0.0, 0.0], [0.9678993225097656, 0.004143435508012772, 0.004314453341066837, 0.023642776533961296, 0.0, 0.0], [0.8999068737030029, 0.001467161695472896, 0.00029133574571460485, 0.002585014794021845, 0.09574954956769943, 0.0], [0.9386115670204163, 0.00022248300956562161, 0.0006146665546111763, 0.0015495637198910117, 0.030689461156725883, 0.028312424197793007]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9999959468841553, 4.042720775032649e-06, 0.0, 0.0, 0.0, 0.0], [0.9982761144638062, 3.2613831990602193e-06, 0.001720669330097735, 0.0, 0.0, 0.0], [0.9998809099197388, 5.328835683826583e-08, 6.376215537784446e-07, 0.00011847059795400128, 0.0, 0.0], [0.9996154308319092, 3.473169556400535e-07, 3.8920820344401363e-08, 4.468433303372876e-07, 0.00038369710091501474, 0.0], [0.9994840621948242, 1.655020476221125e-08, 2.8715557931491276e-08, 1.0638284493325045e-06, 0.0002126671897713095, 0.00030212008277885616]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9514135718345642, 0.048586405813694, 0.0, 0.0, 0.0, 0.0], [0.5749948024749756, 0.39028096199035645, 0.03472418338060379, 0.0, 0.0, 0.0], [0.7442318201065063, 0.1752411425113678, 0.0756477490067482, 0.004879283253103495, 0.0, 0.0], [0.5232070684432983, 0.09429339319467545, 0.1138191670179367, 0.19979268312454224, 0.06888769567012787, 0.0], [0.47472575306892395, 0.05636607110500336, 0.04530389606952667, 0.06967321783304214, 0.3098014295101166, 0.0441296212375164]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8734648823738098, 0.12653514742851257, 0.0, 0.0, 0.0, 0.0], [0.6097912788391113, 0.3541727066040039, 0.036036062985658646, 0.0, 0.0, 0.0], [0.45984190702438354, 0.38697871565818787, 0.0996011346578598, 0.05357823893427849, 0.0, 0.0], [0.572220504283905, 0.23636263608932495, 0.08344558626413345, 0.06921917200088501, 0.03875211998820305, 0.0], [0.5143564343452454, 0.16723087430000305, 0.09019406139850616, 0.0765448659658432, 0.10578085482120514, 0.04589281603693962]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.981228768825531, 0.018771231174468994, 0.0, 0.0, 0.0, 0.0], [0.6142941117286682, 0.3503977954387665, 0.0353081189095974, 0.0, 0.0, 0.0], [0.5770686268806458, 0.32858458161354065, 0.05508256331086159, 0.03926428034901619, 0.0, 0.0], [0.17188192903995514, 0.011042501777410507, 0.054578714072704315, 0.7326585650444031, 0.029838265851140022, 0.0], [0.3783015012741089, 0.017070062458515167, 0.021754134446382523, 0.4409688115119934, 0.06093813106417656, 0.08096737414598465]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9923112392425537, 0.007688735146075487, 0.0, 0.0, 0.0, 0.0], [0.9498787522315979, 0.016709784045815468, 0.03341152146458626, 0.0, 0.0, 0.0], [0.9961295127868652, 0.0008787295082584023, 0.0006868162308819592, 0.0023048371076583862, 0.0, 0.0], [0.9935757517814636, 0.0032634998206049204, 0.0009993825806304812, 0.00027932299417443573, 0.0018820574041455984, 0.0], [0.9907532930374146, 0.00021344318520277739, 0.0004595233185682446, 0.0007905619568191469, 0.004424723796546459, 0.003358350833877921]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9647740125656128, 0.03522596135735512, 0.0, 0.0, 0.0, 0.0], [0.8194130063056946, 0.1365436613559723, 0.04404333233833313, 0.0, 0.0, 0.0], [0.7584245800971985, 0.006878929678350687, 0.20653395354747772, 0.028162529692053795, 0.0, 0.0], [0.5298128128051758, 0.002678812015801668, 0.07857988774776459, 0.3598373234272003, 0.02909109927713871, 0.0], [0.7544413208961487, 0.00036782227107323706, 0.0019713479559868574, 0.00324004958383739, 0.1942344754934311, 0.04574500769376755]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9749131202697754, 0.02508680149912834, 0.0, 0.0, 0.0, 0.0], [0.9306471943855286, 0.05705660209059715, 0.012296222150325775, 0.0, 0.0, 0.0], [0.9305251836776733, 0.052770983427762985, 0.01111945416778326, 0.005584415514022112, 0.0, 0.0], [0.8863320350646973, 0.01292418036609888, 0.017724711447954178, 0.06150198355317116, 0.021517015993595123, 0.0], [0.791684627532959, 0.015036096796393394, 0.0317479707300663, 0.03392200171947479, 0.03707978501915932, 0.09052948653697968]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9608501195907593, 0.039149850606918335, 0.0, 0.0, 0.0, 0.0], [0.9121272563934326, 0.02257651649415493, 0.06529619544744492, 0.0, 0.0, 0.0], [0.9364108443260193, 0.015584447421133518, 0.024544963613152504, 0.02345985174179077, 0.0, 0.0], [0.9454620480537415, 0.006762288510799408, 0.022026237100362778, 0.009137796238064766, 0.016611700877547264, 0.0], [0.8346164226531982, 0.001881699077785015, 0.00560904573649168, 0.01887359470129013, 0.12449200451374054, 0.014527074061334133]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9964227080345154, 0.0035772807896137238, 0.0, 0.0, 0.0, 0.0], [0.9713928699493408, 0.024453025311231613, 0.004154058638960123, 0.0, 0.0, 0.0], [0.9735792279243469, 0.019003381952643394, 0.003664410673081875, 0.0037529165856540203, 0.0, 0.0], [0.9586312174797058, 0.007116180844604969, 0.009218388237059116, 0.022725583985447884, 0.0023084774147719145, 0.0], [0.973607063293457, 0.008490582928061485, 0.0032512471079826355, 0.003606445388868451, 0.004877461586147547, 0.006167212035506964]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.97598797082901, 0.024011990055441856, 0.0, 0.0, 0.0, 0.0], [0.9460638165473938, 0.04211375489830971, 0.011822436936199665, 0.0, 0.0, 0.0], [0.8446813225746155, 0.04293116182088852, 0.05218198522925377, 0.06020559370517731, 0.0, 0.0], [0.9378372430801392, 0.03354858607053757, 0.008826455101370811, 0.0028792242519557476, 0.016908427700400352, 0.0], [0.8124931454658508, 0.02696753479540348, 0.05999218672513962, 0.03445731848478317, 0.011011860333383083, 0.05507794767618179]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9001203775405884, 0.09987961500883102, 0.0, 0.0, 0.0, 0.0], [0.627193033695221, 0.07988718152046204, 0.29291975498199463, 0.0, 0.0, 0.0], [0.7624077796936035, 0.02734432928264141, 0.038679543882608414, 0.17156831920146942, 0.0, 0.0], [0.7995968461036682, 0.014336260966956615, 0.01437566988170147, 0.025438452139496803, 0.14625284075737, 0.0], [0.7851970791816711, 0.04204057529568672, 0.025253651663661003, 0.02908395044505596, 0.029306314885616302, 0.08911846578121185]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9954467415809631, 0.0045532057993113995, 0.0, 0.0, 0.0, 0.0], [0.9356001615524292, 0.04476744681596756, 0.019632352516055107, 0.0, 0.0, 0.0], [0.5605552792549133, 0.09861977398395538, 0.29983264207839966, 0.040992289781570435, 0.0, 0.0], [0.5893709659576416, 0.11000988632440567, 0.08033622056245804, 0.16754034161567688, 0.05274256691336632, 0.0], [0.22305884957313538, 0.05680817365646362, 0.05467984080314636, 0.24733951687812805, 0.3111244738101959, 0.1069890558719635]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9301451444625854, 0.06985488533973694, 0.0, 0.0, 0.0, 0.0], [0.8936478495597839, 0.08535721153020859, 0.020994966849684715, 0.0, 0.0, 0.0], [0.8404538035392761, 0.10619214922189713, 0.02363673783838749, 0.029717326164245605, 0.0, 0.0], [0.8927386403083801, 0.024784674867987633, 0.008319000713527203, 0.05165454372763634, 0.022503145039081573, 0.0], [0.8646610975265503, 0.009503193199634552, 0.0024329854641109705, 0.04796753078699112, 0.04273205250501633, 0.03270319849252701]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9859625697135925, 0.014037408865988255, 0.0, 0.0, 0.0, 0.0], [0.9702037572860718, 0.0168070700019598, 0.012989125214517117, 0.0, 0.0, 0.0], [0.9524770379066467, 0.016064459457993507, 0.013456220738589764, 0.018002323806285858, 0.0, 0.0], [0.9332928657531738, 0.01897200010716915, 0.02014683373272419, 0.017023753374814987, 0.010564540512859821, 0.0], [0.9113592505455017, 0.012528638355433941, 0.02209620550274849, 0.01751861348748207, 0.018517911434173584, 0.01797938533127308]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9681769013404846, 0.03182310611009598, 0.0, 0.0, 0.0, 0.0], [0.9096417427062988, 0.07916690409183502, 0.011191264726221561, 0.0, 0.0, 0.0], [0.8379932045936584, 0.13078266382217407, 0.012140989303588867, 0.019083037972450256, 0.0, 0.0], [0.9116525053977966, 0.05451957508921623, 0.009499342180788517, 0.00746585289016366, 0.01686275750398636, 0.0], [0.8510289192199707, 0.07338211685419083, 0.008022507652640343, 0.009083161130547523, 0.04261006414890289, 0.015873271971940994]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9799023866653442, 0.020097682252526283, 0.0, 0.0, 0.0, 0.0], [0.9558742642402649, 0.029063312336802483, 0.015062497928738594, 0.0, 0.0, 0.0], [0.7943133115768433, 0.06074100360274315, 0.06907659024000168, 0.07586916536092758, 0.0, 0.0], [0.5494324564933777, 0.03154711425304413, 0.05482015758752823, 0.05788077041506767, 0.3063195049762726, 0.0], [0.6453980803489685, 0.010770943015813828, 0.017528092488646507, 0.02157985046505928, 0.24958276748657227, 0.05514020845293999]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9506809115409851, 0.0493190623819828, 0.0, 0.0, 0.0, 0.0], [0.8553215265274048, 0.09256264567375183, 0.05211575701832771, 0.0, 0.0, 0.0], [0.850852370262146, 0.04734604433178902, 0.044177331030368805, 0.057624250650405884, 0.0, 0.0], [0.7697131633758545, 0.02788589708507061, 0.031017286702990532, 0.06842502951622009, 0.1029587835073471, 0.0], [0.7931903004646301, 0.04052198305726051, 0.029242033138871193, 0.04478124529123306, 0.04894689470529556, 0.04331749677658081]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9770310521125793, 0.02296893112361431, 0.0, 0.0, 0.0, 0.0], [0.9429817199707031, 0.017321482300758362, 0.03969680890440941, 0.0, 0.0, 0.0], [0.9144344925880432, 0.008583576418459415, 0.013035810552537441, 0.06394599378108978, 0.0, 0.0], [0.9222429990768433, 0.0036440351977944374, 0.003740275977179408, 0.010410364717245102, 0.05996239185333252, 0.0], [0.9198879599571228, 0.0030822583939880133, 0.0034827394410967827, 0.004206796642392874, 0.02125428058207035, 0.048085976392030716]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.977458119392395, 0.022541873157024384, 0.0, 0.0, 0.0, 0.0], [0.8929325342178345, 0.07475466281175613, 0.032312843948602676, 0.0, 0.0, 0.0], [0.8423511385917664, 0.05980278551578522, 0.03740081936120987, 0.06044524535536766, 0.0, 0.0], [0.7674624919891357, 0.03536349534988403, 0.042155250906944275, 0.06658654659986496, 0.08843226730823517, 0.0], [0.6182611584663391, 0.01611059531569481, 0.020167622715234756, 0.03868892416357994, 0.23147016763687134, 0.07530155777931213]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9634856581687927, 0.036514393985271454, 0.0, 0.0, 0.0, 0.0], [0.4363938570022583, 0.522637128829956, 0.04096902906894684, 0.0, 0.0, 0.0], [0.3608614206314087, 0.35129693150520325, 0.2655103802680969, 0.022331148386001587, 0.0, 0.0], [0.3942921757698059, 0.021704670041799545, 0.07794328778982162, 0.37168896198272705, 0.1343708038330078, 0.0], [0.6310713887214661, 0.01698400266468525, 0.025942081585526466, 0.08615949749946594, 0.2183200567960739, 0.021522950381040573]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9988250136375427, 0.0011750265257433057, 0.0, 0.0, 0.0, 0.0], [0.9944871068000793, 0.0004826401418540627, 0.0050302306190133095, 0.0, 0.0, 0.0], [0.9981209635734558, 2.705173392314464e-05, 0.0001130745149566792, 0.0017389442073181272, 0.0, 0.0], [0.9982239603996277, 6.83655816828832e-05, 0.00010199935059063137, 6.028370262356475e-05, 0.0015453165397047997, 0.0], [0.9982888102531433, 1.055222810464329e-06, 3.2781026675365865e-05, 0.00013038977340329438, 0.0006605894886888564, 0.0008863684488460422]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9936710596084595, 0.006328921765089035, 0.0, 0.0, 0.0, 0.0], [0.9727688431739807, 0.0018561368342489004, 0.025375060737133026, 0.0, 0.0, 0.0], [0.9724299907684326, 0.0019586149137467146, 0.011192461475729942, 0.014418890699744225, 0.0, 0.0], [0.9782041311264038, 0.0009589138207957149, 0.0018706483533605933, 0.006326568778604269, 0.012639678083360195, 0.0], [0.9592596888542175, 0.0024555064737796783, 0.00161241355817765, 0.005019655916839838, 0.006687097251415253, 0.024965662509202957]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9629000425338745, 0.03709998354315758, 0.0, 0.0, 0.0, 0.0], [0.36801934242248535, 0.6152258515357971, 0.016754813492298126, 0.0, 0.0, 0.0], [0.3173511326313019, 0.6140013337135315, 0.05375149846076965, 0.014896026812493801, 0.0, 0.0], [0.48987284302711487, 0.21071474254131317, 0.04693019017577171, 0.20700432360172272, 0.04547784850001335, 0.0], [0.48774227499961853, 0.1769528090953827, 0.06915216147899628, 0.09849268198013306, 0.12091436982154846, 0.046745721250772476]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9794419407844543, 0.020558049902319908, 0.0, 0.0, 0.0, 0.0], [0.6677903532981873, 0.31032365560531616, 0.021886007860302925, 0.0, 0.0, 0.0], [0.7118757367134094, 0.11108540743589401, 0.14187385141849518, 0.03516504913568497, 0.0, 0.0], [0.4501457214355469, 0.04036055505275726, 0.040458209812641144, 0.388570100069046, 0.08046531677246094, 0.0], [0.49346262216567993, 0.013696977868676186, 0.008126799948513508, 0.13074499368667603, 0.3086138069629669, 0.04535480588674545]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9846054315567017, 0.015394587069749832, 0.0, 0.0, 0.0, 0.0], [0.9806739091873169, 0.007713791914284229, 0.011612347327172756, 0.0, 0.0, 0.0], [0.932663083076477, 0.01957838423550129, 0.02410353161394596, 0.023654978722333908, 0.0, 0.0], [0.9422016739845276, 0.0009538981830701232, 0.0010898025939241052, 0.00319337984547019, 0.05256118252873421, 0.0], [0.9352930784225464, 0.0010279357666149735, 0.004444425459951162, 0.001637140172533691, 0.010590963996946812, 0.04700646549463272]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9985783100128174, 0.0014216724084690213, 0.0, 0.0, 0.0, 0.0], [0.9893348813056946, 0.0011178902350366116, 0.00954714696854353, 0.0, 0.0, 0.0], [0.9979978203773499, 7.997050124686211e-05, 0.00013218850654084235, 0.0017900333041325212, 0.0, 0.0], [0.9986976385116577, 4.1044117097044364e-05, 3.8683547245454974e-06, 2.3676282580709085e-05, 0.0012337174266576767, 0.0], [0.9971563816070557, 1.852225250331685e-05, 1.8826559653462027e-06, 2.7900125132873654e-05, 0.0006533482228405774, 0.0021419788245111704]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9768233299255371, 0.023176640272140503, 0.0, 0.0, 0.0, 0.0], [0.9194678068161011, 0.05088186264038086, 0.029650341719388962, 0.0, 0.0, 0.0], [0.8474554419517517, 0.06100169196724892, 0.04372376948595047, 0.04781914874911308, 0.0, 0.0], [0.8011623620986938, 0.041866958141326904, 0.04375807195901871, 0.041894737631082535, 0.07131782174110413, 0.0], [0.8031871914863586, 0.02450493723154068, 0.017323585227131844, 0.04744395986199379, 0.06109930947422981, 0.046441152691841125]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9829428195953369, 0.01705716922879219, 0.0, 0.0, 0.0, 0.0], [0.8863736987113953, 0.09492647647857666, 0.018699750304222107, 0.0, 0.0, 0.0], [0.9231085777282715, 0.03696346655488014, 0.032198335975408554, 0.007729663979262114, 0.0, 0.0], [0.9068527221679688, 0.016046639531850815, 0.014310522936284542, 0.04543786868453026, 0.017352323979139328, 0.0], [0.6555973887443542, 0.05091019719839096, 0.028384855017066002, 0.1256549060344696, 0.10546853393316269, 0.03398407623171806]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9502318501472473, 0.049768079072237015, 0.0, 0.0, 0.0, 0.0], [0.8829865455627441, 0.1000962108373642, 0.01691717840731144, 0.0, 0.0, 0.0], [0.8057457804679871, 0.14463546872138977, 0.03018922731280327, 0.019429458305239677, 0.0, 0.0], [0.8706230521202087, 0.032440632581710815, 0.026951627805829048, 0.04410304129123688, 0.025881657376885414, 0.0], [0.688364565372467, 0.009681451134383678, 0.016449343413114548, 0.0987110361456871, 0.08971209079027176, 0.09708156436681747]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9792683124542236, 0.02073168195784092, 0.0, 0.0, 0.0, 0.0], [0.9523284435272217, 0.025933818891644478, 0.021737735718488693, 0.0, 0.0, 0.0], [0.9144353270530701, 0.017671240493655205, 0.022358495742082596, 0.04553484544157982, 0.0, 0.0], [0.9448292851448059, 0.006467597559094429, 0.006386063527315855, 0.03263096138834953, 0.00968620739877224, 0.0], [0.9347906112670898, 0.007862505502998829, 0.007788175716996193, 0.021432818844914436, 0.008491144515573978, 0.01963483914732933]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.983370304107666, 0.016629677265882492, 0.0, 0.0, 0.0, 0.0], [0.963111400604248, 0.009229931980371475, 0.027658598497509956, 0.0, 0.0, 0.0], [0.9706628322601318, 0.0041494048200547695, 0.0068131014704704285, 0.018374638631939888, 0.0, 0.0], [0.987951934337616, 0.002165885642170906, 0.00034901127219200134, 0.001583816367201507, 0.00794942770153284, 0.0], [0.9457950592041016, 0.014583553187549114, 0.0003652951563708484, 0.0009569536778144538, 0.013621564954519272, 0.02467755414545536]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9878059029579163, 0.01219407469034195, 0.0, 0.0, 0.0, 0.0], [0.87103670835495, 0.09448163211345673, 0.03448161482810974, 0.0, 0.0, 0.0], [0.6309783458709717, 0.11090382188558578, 0.1923021823167801, 0.06581564992666245, 0.0, 0.0], [0.5360490083694458, 0.04618944972753525, 0.13605308532714844, 0.26455509662628174, 0.017153292894363403, 0.0], [0.8287520408630371, 0.023732755333185196, 0.02008037269115448, 0.07245264202356339, 0.030431220307946205, 0.024550989270210266]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8995685577392578, 0.10043150931596756, 0.0, 0.0, 0.0, 0.0], [0.270343542098999, 0.6504329442977905, 0.07922357320785522, 0.0, 0.0, 0.0], [0.20541730523109436, 0.5892508625984192, 0.18085837364196777, 0.024473490193486214, 0.0, 0.0], [0.5573861002922058, 0.1774134784936905, 0.08806808292865753, 0.09881848096847534, 0.07831384986639023, 0.0], [0.5922912359237671, 0.08700639009475708, 0.05643285810947418, 0.05685883015394211, 0.12181518226861954, 0.08559554070234299]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9316380620002747, 0.06836195290088654, 0.0, 0.0, 0.0, 0.0], [0.9572945833206177, 0.026243582367897034, 0.0164618119597435, 0.0, 0.0, 0.0], [0.9880544543266296, 0.00427332753315568, 0.002954584313556552, 0.004717645235359669, 0.0, 0.0], [0.99403977394104, 0.0009413420339114964, 0.0004739820142276585, 0.00011646930943243206, 0.004428447224199772, 0.0], [0.9806035161018372, 2.5468933017691597e-05, 0.00016239412070717663, 0.0001476418401580304, 0.0013442443450912833, 0.017716845497488976]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.993178129196167, 0.006821857299655676, 0.0, 0.0, 0.0, 0.0], [0.9756524562835693, 0.01318411435931921, 0.011163423769176006, 0.0, 0.0, 0.0], [0.9418966770172119, 0.004721744451671839, 0.0023818055633455515, 0.050999753177165985, 0.0, 0.0], [0.9905040860176086, 0.0022848136723041534, 6.198462506290525e-05, 0.0005984465242363513, 0.006550676189363003, 0.0], [0.9697660207748413, 0.0008878845837898552, 0.00023466735729016364, 0.0017040816601365805, 0.004128355998545885, 0.02327893301844597]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9716231822967529, 0.02837684564292431, 0.0, 0.0, 0.0, 0.0], [0.9223619699478149, 0.028907248750329018, 0.048730745911598206, 0.0, 0.0, 0.0], [0.8426317572593689, 0.023872116580605507, 0.04748132824897766, 0.08601479232311249, 0.0, 0.0], [0.8521121740341187, 0.020744236186146736, 0.04494619369506836, 0.05765002593398094, 0.02454746514558792, 0.0], [0.8800725936889648, 0.022448532283306122, 0.018235722556710243, 0.01925482600927353, 0.015854258090257645, 0.044134121388196945]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9412723779678345, 0.058727629482746124, 0.0, 0.0, 0.0, 0.0], [0.916313886642456, 0.05759201943874359, 0.02609400637447834, 0.0, 0.0, 0.0], [0.8392423391342163, 0.057690516114234924, 0.01382902916520834, 0.08923812955617905, 0.0, 0.0], [0.8987162113189697, 0.0134778693318367, 0.0003456450067460537, 0.003298751311376691, 0.08416149020195007, 0.0], [0.8701692223548889, 0.002700856188312173, 0.00143499206751585, 0.0056661744602024555, 0.08874300867319107, 0.031285665929317474]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9656725525856018, 0.03432750701904297, 0.0, 0.0, 0.0, 0.0], [0.9178615808486938, 0.062257930636405945, 0.019880469888448715, 0.0, 0.0, 0.0], [0.823314905166626, 0.06282395124435425, 0.03670429438352585, 0.07715693861246109, 0.0, 0.0], [0.8501748442649841, 0.03816927224397659, 0.03196492791175842, 0.0516013503074646, 0.02808968350291252, 0.0], [0.6572404503822327, 0.05877397954463959, 0.04336007311940193, 0.09013211727142334, 0.08146599680185318, 0.06902744621038437]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9162061810493469, 0.0837937667965889, 0.0, 0.0, 0.0, 0.0], [0.9451773762702942, 0.04099284112453461, 0.013829832896590233, 0.0, 0.0, 0.0], [0.8928355574607849, 0.05368670076131821, 0.017596954479813576, 0.03588071092963219, 0.0, 0.0], [0.8337052464485168, 0.04799601063132286, 0.033513229340314865, 0.04680858924984932, 0.03797686845064163, 0.0], [0.8167192339897156, 0.06337132304906845, 0.013286277651786804, 0.020469767972826958, 0.025292355567216873, 0.06086111441254616]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9525133371353149, 0.04748663306236267, 0.0, 0.0, 0.0, 0.0], [0.3019869327545166, 0.6520938873291016, 0.04591925069689751, 0.0, 0.0, 0.0], [0.285582959651947, 0.556952178478241, 0.1444743126630783, 0.012990524061024189, 0.0, 0.0], [0.843804121017456, 0.032251205295324326, 0.03954290598630905, 0.06848159432411194, 0.015920041128993034, 0.0], [0.6664940714836121, 0.06095913052558899, 0.04064354673027992, 0.06804485619068146, 0.09186329692602158, 0.07199501991271973]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9682655334472656, 0.031734466552734375, 0.0, 0.0, 0.0, 0.0], [0.738521933555603, 0.22856839001178741, 0.032909639179706573, 0.0, 0.0, 0.0], [0.5946676135063171, 0.2303314357995987, 0.14867636561393738, 0.02632458508014679, 0.0, 0.0], [0.6339254975318909, 0.05813034623861313, 0.09654320776462555, 0.14291946589946747, 0.06848153471946716, 0.0], [0.40375572443008423, 0.08945391327142715, 0.07635112851858139, 0.25587135553359985, 0.1433039754629135, 0.03126389905810356]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9869793653488159, 0.013020593672990799, 0.0, 0.0, 0.0, 0.0], [0.8631385564804077, 0.1105666309595108, 0.02629482001066208, 0.0, 0.0, 0.0], [0.9488080143928528, 0.028614996001124382, 0.006535546388477087, 0.016041526570916176, 0.0, 0.0], [0.9672170877456665, 0.006604980677366257, 0.00045171406236477196, 0.004844417329877615, 0.020881708711385727, 0.0], [0.9354621171951294, 0.02047806605696678, 0.0011700231116265059, 0.007056943140923977, 0.0163181871175766, 0.019514625892043114]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9846673011779785, 0.015332723967730999, 0.0, 0.0, 0.0, 0.0], [0.9052747488021851, 0.08373606950044632, 0.010989243164658546, 0.0, 0.0, 0.0], [0.8145939111709595, 0.04283742979168892, 0.10568301379680634, 0.03688570484519005, 0.0, 0.0], [0.23519809544086456, 0.012018457986414433, 0.05280117318034172, 0.6516180038452148, 0.04836418479681015, 0.0], [0.31818512082099915, 0.018632443621754646, 0.03948190063238144, 0.3755541741847992, 0.20787373185157776, 0.04027257487177849]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9811733365058899, 0.018826685845851898, 0.0, 0.0, 0.0, 0.0], [0.8618939518928528, 0.06479164958000183, 0.07331438362598419, 0.0, 0.0, 0.0], [0.7664540410041809, 0.07330425828695297, 0.10353513062000275, 0.056706514209508896, 0.0, 0.0], [0.8128499984741211, 0.03215480223298073, 0.059005625545978546, 0.05416511744260788, 0.04182446748018265, 0.0], [0.8687856197357178, 0.026987861841917038, 0.02047000452876091, 0.01629738137125969, 0.03218390792608261, 0.03527523949742317]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9264583587646484, 0.07354167848825455, 0.0, 0.0, 0.0, 0.0], [0.8403540849685669, 0.06373751163482666, 0.09590838104486465, 0.0, 0.0, 0.0], [0.7330995798110962, 0.06451118737459183, 0.10380073636770248, 0.09858842939138412, 0.0, 0.0], [0.9143612384796143, 0.008257776498794556, 0.007320381235331297, 0.017966248095035553, 0.05209439620375633, 0.0], [0.8971915245056152, 0.008555498905479908, 0.007019453682005405, 0.014860544353723526, 0.03399762138724327, 0.03837529569864273]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9180347919464111, 0.08196526020765305, 0.0, 0.0, 0.0, 0.0], [0.8328666687011719, 0.1219901517033577, 0.04514322429895401, 0.0, 0.0, 0.0], [0.7994157075881958, 0.0874413549900055, 0.03605784848332405, 0.07708510011434555, 0.0, 0.0], [0.880984902381897, 0.020749641582369804, 0.020554615184664726, 0.017120830714702606, 0.06058995798230171, 0.0], [0.745303213596344, 0.044334057718515396, 0.022549288347363472, 0.0331527441740036, 0.03357058763504028, 0.12109009176492691]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9867060780525208, 0.013293893076479435, 0.0, 0.0, 0.0, 0.0], [0.982177734375, 0.012414131313562393, 0.005408108700066805, 0.0, 0.0, 0.0], [0.9630486369132996, 0.015290752984583378, 0.010345698334276676, 0.0113149369135499, 0.0, 0.0], [0.9213568568229675, 0.014132463373243809, 0.017639216035604477, 0.016567690297961235, 0.030303770676255226, 0.0], [0.9373326301574707, 0.009064299054443836, 0.007548365276306868, 0.006576443091034889, 0.011827622540295124, 0.027650514617562294]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9951004385948181, 0.00489962799474597, 0.0, 0.0, 0.0, 0.0], [0.9476007223129272, 0.041407931596040726, 0.010991275310516357, 0.0, 0.0, 0.0], [0.9142175316810608, 0.023523783311247826, 0.039145033806562424, 0.023113621398806572, 0.0, 0.0], [0.9534738659858704, 0.008932933211326599, 0.015272765420377254, 0.007908251136541367, 0.014412266202270985, 0.0], [0.9427101016044617, 0.00823307130485773, 0.004650997929275036, 0.004178107250481844, 0.005463531706482172, 0.03476419299840927]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9543376564979553, 0.045662373304367065, 0.0, 0.0, 0.0, 0.0], [0.9696040749549866, 0.01954760029911995, 0.01084828469902277, 0.0, 0.0, 0.0], [0.9710449576377869, 0.012425386346876621, 0.008068876340985298, 0.008460716344416142, 0.0, 0.0], [0.9726192951202393, 0.002697656163945794, 0.00044831327977590263, 0.0013814778067171574, 0.022853154689073563, 0.0], [0.9675466418266296, 0.009613442234694958, 0.003203035332262516, 0.00424883933737874, 0.007442260626703501, 0.00794589426368475]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9887008666992188, 0.011299116536974907, 0.0, 0.0, 0.0, 0.0], [0.9382632374763489, 0.04204244911670685, 0.019694412127137184, 0.0, 0.0, 0.0], [0.8351995944976807, 0.03487853705883026, 0.05134471505880356, 0.07857715338468552, 0.0, 0.0], [0.9042676687240601, 0.010541575029492378, 0.016426723450422287, 0.025921987369656563, 0.04284200444817543, 0.0], [0.8913140892982483, 0.00891267228871584, 0.005010711494833231, 0.008175632916390896, 0.013514749705791473, 0.07307209819555283]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8693912029266357, 0.13060881197452545, 0.0, 0.0, 0.0, 0.0], [0.3507988452911377, 0.606351912021637, 0.04284917935729027, 0.0, 0.0, 0.0], [0.35475659370422363, 0.3502019941806793, 0.24722407758235931, 0.04781729355454445, 0.0, 0.0], [0.35370609164237976, 0.03527737781405449, 0.09567111730575562, 0.449796199798584, 0.06554921716451645, 0.0], [0.4132595360279083, 0.09055527299642563, 0.05286579951643944, 0.174679696559906, 0.173848956823349, 0.09479076415300369]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9629756212234497, 0.037024300545454025, 0.0, 0.0, 0.0, 0.0], [0.9756426811218262, 0.01965854875743389, 0.004698706325143576, 0.0, 0.0, 0.0], [0.9775736927986145, 0.013286248780786991, 0.0025590297300368547, 0.006581062916666269, 0.0, 0.0], [0.9870142936706543, 0.007388236932456493, 0.0009579154429957271, 0.0018318220973014832, 0.0028077505994588137, 0.0], [0.9409245848655701, 0.016633737832307816, 0.0022979143541306257, 0.0058906711637973785, 0.0055129327811300755, 0.02874022163450718]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.962827205657959, 0.037172831594944, 0.0, 0.0, 0.0, 0.0], [0.9582237601280212, 0.024641817435622215, 0.017134377732872963, 0.0, 0.0, 0.0], [0.9351300001144409, 0.015331573784351349, 0.014810982160270214, 0.034727465361356735, 0.0, 0.0], [0.9225171208381653, 0.010528750717639923, 0.011010154150426388, 0.01944003626704216, 0.036503832787275314, 0.0], [0.8420165777206421, 0.04357199743390083, 0.007488282397389412, 0.01496153138577938, 0.02385285682976246, 0.06810864061117172]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9926387071609497, 0.00736132962629199, 0.0, 0.0, 0.0, 0.0], [0.9957393407821655, 0.0033469819463789463, 0.000913690309971571, 0.0, 0.0, 0.0], [0.9869900345802307, 0.001974786864593625, 0.001524551771581173, 0.009510699659585953, 0.0, 0.0], [0.9933527708053589, 0.001020324882119894, 0.00034337223041802645, 0.0010291127255186439, 0.004254369530826807, 0.0], [0.9749016761779785, 0.00043480272870510817, 0.0004306558985263109, 0.0012364407302811742, 0.0015347707085311413, 0.021461669355630875]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9897475242614746, 0.010252462700009346, 0.0, 0.0, 0.0, 0.0], [0.9790639281272888, 0.01650906540453434, 0.0044270907528698444, 0.0, 0.0, 0.0], [0.9521436095237732, 0.029432358220219612, 0.008943161927163601, 0.009480923414230347, 0.0, 0.0], [0.939594030380249, 0.021510960534214973, 0.010278552770614624, 0.004555229097604752, 0.024061163887381554, 0.0], [0.9205074906349182, 0.016153652220964432, 0.010818594135344028, 0.01664440892636776, 0.014566398225724697, 0.021309375762939453]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9898501634597778, 0.010149780660867691, 0.0, 0.0, 0.0, 0.0], [0.9820910096168518, 0.006907520350068808, 0.011001535691320896, 0.0, 0.0, 0.0], [0.9684997200965881, 0.008987602777779102, 0.015342563390731812, 0.007170087192207575, 0.0, 0.0], [0.9274120330810547, 0.009485266171395779, 0.022066107019782066, 0.03222890570759773, 0.008807653561234474, 0.0], [0.900665819644928, 0.021623756736516953, 0.013808279298245907, 0.009843860752880573, 0.008521373383700848, 0.04553695768117905]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9954444169998169, 0.004555588588118553, 0.0, 0.0, 0.0, 0.0], [0.995254397392273, 0.002460238989442587, 0.002285485854372382, 0.0, 0.0, 0.0], [0.9862446188926697, 0.0015168144600465894, 0.004072288051247597, 0.008166354149580002, 0.0, 0.0], [0.9889963865280151, 0.001226040069013834, 0.0007996349013410509, 0.0006774227367714047, 0.008300574496388435, 0.0], [0.9865202903747559, 0.00039427157025784254, 0.0009571771952323616, 0.0004954367759637535, 0.0009604979422874749, 0.010672281496226788]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9821295142173767, 0.017870500683784485, 0.0, 0.0, 0.0, 0.0], [0.7489436268806458, 0.22002726793289185, 0.031029189005494118, 0.0, 0.0, 0.0], [0.28547799587249756, 0.21125678718090057, 0.47871601581573486, 0.024549242109060287, 0.0, 0.0], [0.8056644201278687, 0.026974644511938095, 0.04302806034684181, 0.06993705034255981, 0.05439583212137222, 0.0], [0.3307209014892578, 0.022326624020934105, 0.016627125442028046, 0.08019453287124634, 0.41574832797050476, 0.13438253104686737]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9697746634483337, 0.030225319787859917, 0.0, 0.0, 0.0, 0.0], [0.9800565838813782, 0.015018894337117672, 0.004924521781504154, 0.0, 0.0, 0.0], [0.9237861037254333, 0.052764780819416046, 0.00630240747705102, 0.017146753147244453, 0.0, 0.0], [0.9451844096183777, 0.03618047758936882, 0.001989208161830902, 0.003958724904805422, 0.012687299400568008, 0.0], [0.9633325934410095, 0.018662991002202034, 0.0030418417882174253, 0.007070912979543209, 0.0050094155594706535, 0.002882065251469612]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9873244762420654, 0.012675459496676922, 0.0, 0.0, 0.0, 0.0], [0.9904569983482361, 0.0055419523268938065, 0.004001122899353504, 0.0, 0.0, 0.0], [0.9814971685409546, 0.004653455223888159, 0.003725277027115226, 0.010124054737389088, 0.0, 0.0], [0.9744365811347961, 0.004632251337170601, 0.002379992976784706, 0.006518087349832058, 0.012033028528094292, 0.0], [0.9624497294425964, 0.0033743639942258596, 0.0013198587112128735, 0.0017275003483518958, 0.002944675739854574, 0.028183799237012863]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9807674288749695, 0.01923258602619171, 0.0, 0.0, 0.0, 0.0], [0.9664245843887329, 0.015413926914334297, 0.018161438405513763, 0.0, 0.0, 0.0], [0.9632682204246521, 0.004538117907941341, 0.002925391308963299, 0.029268190264701843, 0.0, 0.0], [0.9562349319458008, 0.0012223608791828156, 0.0005304080550558865, 0.00867149606347084, 0.03334089741110802, 0.0], [0.9657101035118103, 0.0009808284230530262, 0.0016686266753822565, 0.002634831238538027, 0.005866361316293478, 0.023139292374253273]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9639716148376465, 0.036028459668159485, 0.0, 0.0, 0.0, 0.0], [0.9562800526618958, 0.03373315557837486, 0.009986846707761288, 0.0, 0.0, 0.0], [0.8539998531341553, 0.08073022216558456, 0.03334445133805275, 0.031925540417432785, 0.0, 0.0], [0.9547491073608398, 0.009605025872588158, 0.004146162886172533, 0.0020133228972554207, 0.029486361891031265, 0.0], [0.9331137537956238, 0.028699662536382675, 0.005477475933730602, 0.006368075497448444, 0.012613046914339066, 0.013728085905313492]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9392993450164795, 0.06070063263177872, 0.0, 0.0, 0.0, 0.0], [0.9298391342163086, 0.061895377933979034, 0.008265496231615543, 0.0, 0.0, 0.0], [0.8471823334693909, 0.09035038203001022, 0.01763608679175377, 0.044831156730651855, 0.0, 0.0], [0.8857703804969788, 0.03918175399303436, 0.007867704145610332, 0.02276589721441269, 0.04441439360380173, 0.0], [0.8563280701637268, 0.10088995099067688, 0.006531452294439077, 0.008485927246510983, 0.007368441205471754, 0.020396249368786812]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8353264331817627, 0.1646735519170761, 0.0, 0.0, 0.0, 0.0], [0.6160858869552612, 0.3137648403644562, 0.07014927268028259, 0.0, 0.0, 0.0], [0.34316325187683105, 0.2758493721485138, 0.1196604073047638, 0.26132699847221375, 0.0, 0.0], [0.5908172130584717, 0.050290752202272415, 0.041665926575660706, 0.2199493646621704, 0.0972767099738121, 0.0], [0.8481413125991821, 0.06318090111017227, 0.014733693562448025, 0.055267371237277985, 0.00901501253247261, 0.009661628864705563]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9627319574356079, 0.03726799786090851, 0.0, 0.0, 0.0, 0.0], [0.7757522463798523, 0.1799626499414444, 0.044285036623477936, 0.0, 0.0, 0.0], [0.6317060589790344, 0.24380716681480408, 0.10925652086734772, 0.015230235643684864, 0.0, 0.0], [0.9539909958839417, 0.018182311207056046, 0.011601822450757027, 0.012299076654016972, 0.003925766795873642, 0.0], [0.40356943011283875, 0.14237558841705322, 0.05661217123270035, 0.1975736767053604, 0.0929921343922615, 0.10687707364559174]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9802619218826294, 0.019738124683499336, 0.0, 0.0, 0.0, 0.0], [0.9873908162117004, 0.007800452411174774, 0.004808681085705757, 0.0, 0.0, 0.0], [0.9283918738365173, 0.008301235735416412, 0.01330565195530653, 0.05000120773911476, 0.0, 0.0], [0.8981055021286011, 0.015591299161314964, 0.010177576914429665, 0.039987027645111084, 0.0361386202275753, 0.0], [0.9753499031066895, 0.00035433052107691765, 0.0005866039427928627, 0.0011877501383423805, 0.0010750899091362953, 0.021446440368890762]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9295330047607422, 0.07046692818403244, 0.0, 0.0, 0.0, 0.0], [0.9361506104469299, 0.04116682708263397, 0.022682538256049156, 0.0, 0.0, 0.0], [0.8486821055412292, 0.05802798643708229, 0.024856165051460266, 0.0684337466955185, 0.0, 0.0], [0.8661180734634399, 0.02232467755675316, 0.010369130410254002, 0.02600197121500969, 0.07518619298934937, 0.0], [0.8074421882629395, 0.044382549822330475, 0.01849711686372757, 0.03357789292931557, 0.018561245873570442, 0.07753907144069672]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9680535197257996, 0.03194643557071686, 0.0, 0.0, 0.0, 0.0], [0.9693689942359924, 0.02568492479622364, 0.004946070723235607, 0.0, 0.0, 0.0], [0.9620568156242371, 0.022552406415343285, 0.005471326876431704, 0.009919456206262112, 0.0, 0.0], [0.9727528095245361, 0.010137127712368965, 0.000757327419705689, 0.0028828983195126057, 0.013469807803630829, 0.0], [0.9624635577201843, 0.0031109037809073925, 0.0010007602395489812, 0.0019475930603221059, 0.008266227319836617, 0.02321087196469307]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8542501330375671, 0.14574992656707764, 0.0, 0.0, 0.0, 0.0], [0.9725967645645142, 0.014116315171122551, 0.01328685600310564, 0.0, 0.0, 0.0], [0.9257621765136719, 0.03257262706756592, 0.01461210660636425, 0.027053095400333405, 0.0, 0.0], [0.7923423051834106, 0.027305101975798607, 0.01880674995481968, 0.13854165375232697, 0.023004096001386642, 0.0], [0.6152060627937317, 0.02665526419878006, 0.029352931305766106, 0.05590886250138283, 0.11611279845237732, 0.15676409006118774]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9804654121398926, 0.019534552469849586, 0.0, 0.0, 0.0, 0.0], [0.9882452487945557, 0.007509466726332903, 0.004245325922966003, 0.0, 0.0, 0.0], [0.9584206938743591, 0.0109635591506958, 0.010456060990691185, 0.020159708335995674, 0.0, 0.0], [0.9604811668395996, 0.007182627450674772, 0.003072339342907071, 0.006898913532495499, 0.02236509881913662, 0.0], [0.966888964176178, 0.0032812939025461674, 0.00550054432824254, 0.004234083462506533, 0.005038043484091759, 0.015057181939482689]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9498194456100464, 0.05018055811524391, 0.0, 0.0, 0.0, 0.0], [0.9781363606452942, 0.016430046409368515, 0.0054335566237568855, 0.0, 0.0, 0.0], [0.8618696331977844, 0.036093585193157196, 0.07555554062128067, 0.026481209322810173, 0.0, 0.0], [0.5449837446212769, 0.015411133877933025, 0.023516526445746422, 0.25743600726127625, 0.15865260362625122, 0.0], [0.9571874737739563, 0.0030803855042904615, 0.0014446862041950226, 0.006861559115350246, 0.014818714000284672, 0.01660723052918911]]], [[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.6156560778617859, 0.3843439519405365, 0.0, 0.0, 0.0, 0.0], [0.36760634183883667, 0.42816370725631714, 0.20423001050949097, 0.0, 0.0, 0.0], [0.16471554338932037, 0.4136792719364166, 0.2509237229824066, 0.17068152129650116, 0.0, 0.0], [0.4184456169605255, 0.1524762362241745, 0.10305401682853699, 0.11071498692035675, 0.21530911326408386, 0.0], [0.19686934351921082, 0.2014620453119278, 0.12827259302139282, 0.09203246980905533, 0.09167550504207611, 0.2896881103515625]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9027364253997803, 0.09726352989673615, 0.0, 0.0, 0.0, 0.0], [0.9736634492874146, 0.014004302211105824, 0.01233230996876955, 0.0, 0.0, 0.0], [0.8504456281661987, 0.05690572410821915, 0.032060906291007996, 0.06058764085173607, 0.0, 0.0], [0.7661210298538208, 0.03530392050743103, 0.03433045372366905, 0.09675204753875732, 0.06749245524406433, 0.0], [0.8650374412536621, 0.020085260272026062, 0.01149806659668684, 0.01855834573507309, 0.018430285155773163, 0.06639053672552109]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9653082489967346, 0.03469168767333031, 0.0, 0.0, 0.0, 0.0], [0.9816323518753052, 0.014176066033542156, 0.004191514104604721, 0.0, 0.0, 0.0], [0.9275256395339966, 0.04737218841910362, 0.01152826938778162, 0.013573966920375824, 0.0, 0.0], [0.9293117523193359, 0.025833239778876305, 0.007227106485515833, 0.014300585724413395, 0.02332727052271366, 0.0], [0.8895062804222107, 0.04689619690179825, 0.0047171092592179775, 0.006286581978201866, 0.00609014043584466, 0.04650374501943588]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8938026428222656, 0.10619727522134781, 0.0, 0.0, 0.0, 0.0], [0.8221707940101624, 0.06304481625556946, 0.11478441953659058, 0.0, 0.0, 0.0], [0.5047380924224854, 0.15375731885433197, 0.2277037501335144, 0.11380083113908768, 0.0, 0.0], [0.4082071781158447, 0.09066355973482132, 0.11696872115135193, 0.24553199112415314, 0.13862857222557068, 0.0], [0.7291035652160645, 0.06638889014720917, 0.023112818598747253, 0.031103096902370453, 0.057143256068229675, 0.09314827620983124]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9247531890869141, 0.07524678111076355, 0.0, 0.0, 0.0, 0.0], [0.8957376480102539, 0.06989553570747375, 0.03436679765582085, 0.0, 0.0, 0.0], [0.7924937605857849, 0.0960114598274231, 0.05509118735790253, 0.056403566151857376, 0.0, 0.0], [0.7891505360603333, 0.07880303263664246, 0.03840155899524689, 0.05396979674696922, 0.03967496380209923, 0.0], [0.7807856798171997, 0.0799354612827301, 0.042531758546829224, 0.03234211727976799, 0.0178169384598732, 0.046588052064180374]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9480886459350586, 0.05191127583384514, 0.0, 0.0, 0.0, 0.0], [0.863694965839386, 0.04756204038858414, 0.08874296396970749, 0.0, 0.0, 0.0], [0.9341371059417725, 0.022224076092243195, 0.022624483332037926, 0.021014342084527016, 0.0, 0.0], [0.9588143229484558, 0.008020909503102303, 0.004490078426897526, 0.005862293299287558, 0.022812429815530777, 0.0], [0.9385918378829956, 0.021227721124887466, 0.0048724692314863205, 0.010940189473330975, 0.009524582885205746, 0.014843451790511608]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9763734340667725, 0.023626558482646942, 0.0, 0.0, 0.0, 0.0], [0.9884802103042603, 0.005189393647015095, 0.0063303736969828606, 0.0, 0.0, 0.0], [0.9477092027664185, 0.0179851483553648, 0.010156610049307346, 0.024149026721715927, 0.0, 0.0], [0.967192530632019, 0.006552813574671745, 0.0033227826934307814, 0.00556332478299737, 0.017368387430906296, 0.0], [0.9584562182426453, 0.007502961438149214, 0.0051363310776650906, 0.008071648888289928, 0.005997124593704939, 0.014835843816399574]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.884070873260498, 0.11592914164066315, 0.0, 0.0, 0.0, 0.0], [0.9931254386901855, 0.005070806015282869, 0.0018038019770756364, 0.0, 0.0, 0.0], [0.9534159302711487, 0.02382904477417469, 0.007748977281153202, 0.015006075613200665, 0.0, 0.0], [0.9151289463043213, 0.010873105376958847, 0.013190957717597485, 0.011050421744585037, 0.04975655674934387, 0.0], [0.8769673109054565, 0.03385210782289505, 0.00848648976534605, 0.009969149716198444, 0.03468578681349754, 0.036039214581251144]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0003709519514814019, 0.999629020690918, 0.0, 0.0, 0.0, 0.0], [6.525027856696397e-05, 0.3737829029560089, 0.6261518597602844, 0.0, 0.0, 0.0], [4.606018774211407e-05, 0.210508793592453, 0.4115968942642212, 0.3778482675552368, 0.0, 0.0], [4.753069515572861e-05, 0.11616954207420349, 0.23264272511005402, 0.3985331058502197, 0.2526070475578308, 0.0], [1.247641534973809e-06, 0.14819711446762085, 0.15813173353672028, 0.30074331164360046, 0.11939018964767456, 0.27353641390800476]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.971555769443512, 0.028444187715649605, 0.0, 0.0, 0.0, 0.0], [0.9529065489768982, 0.03233075141906738, 0.014762768521904945, 0.0, 0.0, 0.0], [0.9343128204345703, 0.02351292595267296, 0.02049802988767624, 0.021676240488886833, 0.0, 0.0], [0.9529678225517273, 0.00855141133069992, 0.004359325394034386, 0.008064556866884232, 0.026056913658976555, 0.0], [0.9653593897819519, 0.008487647399306297, 0.003499280195683241, 0.002721576252952218, 0.0032828773837536573, 0.016649367287755013]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.8630780577659607, 0.13692188262939453, 0.0, 0.0, 0.0, 0.0], [0.7696157097816467, 0.0851333811879158, 0.14525099098682404, 0.0, 0.0, 0.0], [0.7133337259292603, 0.10170899331569672, 0.11931268870830536, 0.06564456224441528, 0.0, 0.0], [0.7186222076416016, 0.05444284901022911, 0.01386815495789051, 0.07808027416467667, 0.13498654961585999, 0.0], [0.7990148663520813, 0.05805593729019165, 0.009447019547224045, 0.017770467326045036, 0.02113853208720684, 0.09457314014434814]], [[1.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.9518988728523254, 0.048101115971803665, 0.0, 0.0, 0.0, 0.0], [0.8580653071403503, 0.02944577857851982, 0.11248888075351715, 0.0, 0.0, 0.0], [0.6577738523483276, 0.08513449877500534, 0.1261308640241623, 0.1309608370065689, 0.0, 0.0], [0.8087368607521057, 0.0323016420006752, 0.01841817982494831, 0.06856140494346619, 0.07198194414377213, 0.0], [0.6683295965194702, 0.13281384110450745, 0.021880635991692543, 0.02787741646170616, 0.04923408478498459, 0.0998644009232521]]]], \"left_text\": [\"No\", \",\", \" I\", \" am\", \" your\", \" father\"], \"right_text\": [\"No\", \",\", \" I\", \" am\", \" your\", \" father\"]}], \"default_filter\": \"0\", \"display_mode\": \"dark\", \"root_div_id\": \"bertviz-5a257cb0119e4b7ea970ea74266027b3\", \"include_layers\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], \"include_heads\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], \"total_heads\": 12} is a template marker that is replaced by actual params.\n",
       "        const config = {};\n",
       "\n",
       "        const MIN_X = 0;\n",
       "        const MIN_Y = 0;\n",
       "        const DIV_WIDTH = 970;\n",
       "        const THUMBNAIL_PADDING = 5;\n",
       "        const DETAIL_WIDTH = 300;\n",
       "        const DETAIL_ATTENTION_WIDTH = 140;\n",
       "        const DETAIL_BOX_WIDTH = 80;\n",
       "        const DETAIL_BOX_HEIGHT = 18;\n",
       "        const DETAIL_PADDING = 15;\n",
       "        const ATTN_PADDING = 0;\n",
       "        const DETAIL_HEADING_HEIGHT = 25;\n",
       "        const HEADING_TEXT_SIZE = 15;\n",
       "        const HEADING_PADDING = 5;\n",
       "        const TEXT_SIZE = 13;\n",
       "        const TEXT_PADDING = 5;\n",
       "        const LAYER_COLORS = d3.schemeCategory10;\n",
       "        const PALETTE = {\n",
       "            'light': {\n",
       "                'text': 'black',\n",
       "                'background': 'white',\n",
       "                'highlight': '#F5F5F5'\n",
       "            },\n",
       "            'dark': {\n",
       "                'text': '#ccc',\n",
       "                'background': 'black',\n",
       "                'highlight': '#222'\n",
       "            }\n",
       "        }\n",
       "\n",
       "        function render() {\n",
       "\n",
       "            // Set global state variables\n",
       "\n",
       "            var attData = config.attention[config.filter];\n",
       "            config.leftText = attData.left_text;\n",
       "            config.rightText = attData.right_text;\n",
       "            config.attn = attData.attn;\n",
       "            config.numLayers = config.attn.length;\n",
       "            config.numHeads = config.attn[0].length;\n",
       "            config.thumbnailBoxHeight = 7 * (12 / config.totalHeads);\n",
       "            const axisSize = HEADING_TEXT_SIZE + HEADING_PADDING + TEXT_SIZE + TEXT_PADDING;\n",
       "            config.thumbnailHeight = Math.max(config.leftText.length, config.rightText.length) * config.thumbnailBoxHeight + 2 * THUMBNAIL_PADDING;\n",
       "            config.thumbnailWidth = (DIV_WIDTH - axisSize) / config.totalHeads;\n",
       "            config.detailHeight = Math.max(config.leftText.length, config.rightText.length) * DETAIL_BOX_HEIGHT + 2 * DETAIL_PADDING + DETAIL_HEADING_HEIGHT;\n",
       "            config.divHeight = Math.max(config.numLayers * config.thumbnailHeight + axisSize, config.detailHeight);\n",
       "\n",
       "            const vis = $(`#${config.rootDivId} #vis`)\n",
       "            vis.empty();\n",
       "            vis.attr(\"height\", config.divHeight);\n",
       "            config.svg = d3.select(`#${config.rootDivId} #vis`)\n",
       "                .append('svg')\n",
       "                .attr(\"width\", DIV_WIDTH)\n",
       "                .attr(\"height\", config.divHeight)\n",
       "                .attr(\"fill\", getBackgroundColor());\n",
       "\n",
       "            renderAxisLabels();\n",
       "\n",
       "            var i;\n",
       "            var j;\n",
       "            for (i = 0; i < config.numLayers; i++) {\n",
       "                for (j = 0; j < config.numHeads; j++) {\n",
       "                    renderThumbnail(i, j);\n",
       "                }\n",
       "            }\n",
       "        }\n",
       "\n",
       "        function renderAxisLabels() {\n",
       "            const axisSize = HEADING_TEXT_SIZE + HEADING_PADDING + TEXT_SIZE + TEXT_PADDING;\n",
       "            const tableWidth = config.thumbnailWidth * config.heads.length;\n",
       "            config.svg.append(\"text\")\n",
       "                .text(\"Heads\")\n",
       "                .attr(\"fill\", \"black\")\n",
       "                .attr(\"font-weight\", \"bold\")\n",
       "                .attr(\"font-size\", HEADING_TEXT_SIZE + \"px\")\n",
       "                .attr(\"x\", axisSize + tableWidth / 2)\n",
       "                .attr(\"text-anchor\", \"middle\")\n",
       "                .attr(\"y\", 0)\n",
       "                .attr(\"dy\", HEADING_TEXT_SIZE);\n",
       "            for (let i = 0; i < config.numHeads; i++) {\n",
       "                config.svg.append(\"text\")\n",
       "                    .text(config.heads[i])\n",
       "                    .attr(\"fill\", \"black\")\n",
       "                    .attr(\"font-size\", TEXT_SIZE + \"px\")\n",
       "                    .attr(\"x\", axisSize + (i + .5) * config.thumbnailWidth)\n",
       "                    .attr(\"text-anchor\", \"middle\")\n",
       "                    .attr(\"y\", HEADING_TEXT_SIZE + HEADING_PADDING)\n",
       "                    .attr(\"dy\", TEXT_SIZE);\n",
       "            }\n",
       "            let x = 0;\n",
       "            let y = axisSize + config.thumbnailHeight * config.layers.length / 2;\n",
       "            console.log(\"x\", x, y)\n",
       "            config.svg.append(\"text\")\n",
       "                .text(\"Layers\")\n",
       "                .attr(\"fill\", \"black\")\n",
       "                .attr(\"font-weight\", \"bold\")\n",
       "                .attr(\"transform\", \"rotate(270, \" + x  + \", \" + y + \")\")\n",
       "                .attr(\"font-size\", HEADING_TEXT_SIZE + \"px\")\n",
       "                .attr(\"x\", x)\n",
       "                .attr(\"text-anchor\", \"middle\")\n",
       "                .attr(\"y\", y)\n",
       "                .attr(\"dy\", HEADING_TEXT_SIZE);\n",
       "            for (let i = 0; i < config.numLayers; i++) {\n",
       "                x = HEADING_TEXT_SIZE + HEADING_PADDING + TEXT_SIZE; // HACK\n",
       "                y = axisSize + (i + .5) * config.thumbnailHeight;\n",
       "                config.svg.append(\"text\")\n",
       "                    .text(config.layers[i])\n",
       "                    .attr(\"fill\", \"black\")\n",
       "                    .attr(\"font-size\", TEXT_SIZE + \"px\")\n",
       "                    .attr(\"x\", x)\n",
       "                    .attr(\"text-anchor\", \"end\")\n",
       "                    .attr(\"y\", y)\n",
       "                    .attr(\"dy\", TEXT_SIZE / 2);\n",
       "            }\n",
       "        }\n",
       "\n",
       "\n",
       "        function renderThumbnail(layerIndex, headIndex) {\n",
       "            const axisSize = HEADING_TEXT_SIZE + HEADING_PADDING + TEXT_SIZE + TEXT_PADDING\n",
       "            const x = headIndex * config.thumbnailWidth + axisSize;\n",
       "            const y = layerIndex * config.thumbnailHeight + axisSize;\n",
       "            renderThumbnailAttn(x, y, config.attn[layerIndex][headIndex], layerIndex, headIndex);\n",
       "        }\n",
       "\n",
       "        function renderDetail(att, layerIndex, headIndex) {\n",
       "            const axisSize = TEXT_SIZE + HEADING_PADDING + TEXT_SIZE + TEXT_PADDING;\n",
       "            var xOffset = .8 * config.thumbnailWidth;\n",
       "            var maxX = DIV_WIDTH;\n",
       "            var maxY = config.divHeight - 3;\n",
       "            var leftPos = axisSize + headIndex * config.thumbnailWidth;\n",
       "            var x = leftPos + THUMBNAIL_PADDING + xOffset;\n",
       "            if (x < MIN_X) {\n",
       "                x = MIN_X;\n",
       "            } else if (x + DETAIL_WIDTH > maxX) {\n",
       "                x = leftPos + THUMBNAIL_PADDING - DETAIL_WIDTH + 8;\n",
       "            }\n",
       "            var posLeftText = x;\n",
       "            var posAttention = posLeftText + DETAIL_BOX_WIDTH;\n",
       "            var posRightText = posAttention + DETAIL_ATTENTION_WIDTH;\n",
       "            var thumbnailHeight = Math.max(config.leftText.length, config.rightText.length) * config.thumbnailBoxHeight + 2 * THUMBNAIL_PADDING;\n",
       "            var yOffset = 20;\n",
       "            var y = layerIndex * thumbnailHeight + THUMBNAIL_PADDING + yOffset;\n",
       "            if (y < MIN_Y) {\n",
       "                y = MIN_Y;\n",
       "            } else if (y + config.detailHeight > maxY) {\n",
       "                y = maxY - config.detailHeight;\n",
       "            }\n",
       "            renderDetailFrame(x, y, layerIndex);\n",
       "            y = y + DETAIL_PADDING;\n",
       "            renderDetailHeading(x, y, layerIndex, headIndex);\n",
       "            y = y + DETAIL_HEADING_HEIGHT;\n",
       "            renderDetailText(config.leftText, \"leftText\", posLeftText, y , layerIndex);\n",
       "            renderDetailAttn(posAttention, y, att, layerIndex, headIndex);\n",
       "            renderDetailText(config.rightText, \"rightText\", posRightText, y, layerIndex);\n",
       "        }\n",
       "\n",
       "        function renderDetailHeading(x, y, layerIndex, headIndex) {\n",
       "            var fillColor = getTextColor();\n",
       "            config.svg.append(\"text\")\n",
       "                .classed(\"detail\", true)\n",
       "                .text('Layer ' + config.layers[layerIndex] + \", Head \" + config.heads[headIndex])\n",
       "                .attr(\"font-size\", TEXT_SIZE + \"px\")\n",
       "                .attr(\"font-weight\", \"bold\")\n",
       "                .style(\"cursor\", \"default\")\n",
       "                .style(\"-webkit-user-select\", \"none\")\n",
       "                .attr(\"fill\", fillColor)\n",
       "                .attr(\"x\", x + DETAIL_WIDTH / 2)\n",
       "                .attr(\"text-anchor\", \"middle\")\n",
       "                .attr(\"y\", y)\n",
       "                .attr(\"height\", DETAIL_HEADING_HEIGHT)\n",
       "                .attr(\"width\", DETAIL_WIDTH)\n",
       "                .attr(\"dy\", HEADING_TEXT_SIZE);\n",
       "        }\n",
       "\n",
       "        function renderDetailText(text, id, x, y, layerIndex) {\n",
       "            var tokenContainer = config.svg.append(\"svg:g\")\n",
       "                .classed(\"detail\", true)\n",
       "                .selectAll(\"g\")\n",
       "                .data(text)\n",
       "                .enter()\n",
       "                .append(\"g\");\n",
       "\n",
       "            var fillColor = getTextColor();\n",
       "\n",
       "            tokenContainer.append(\"rect\")\n",
       "                .classed(\"highlight\", true)\n",
       "                .attr(\"fill\", fillColor)\n",
       "                .style(\"opacity\", 0.0)\n",
       "                .attr(\"height\", DETAIL_BOX_HEIGHT)\n",
       "                .attr(\"width\", DETAIL_BOX_WIDTH)\n",
       "                .attr(\"x\", x)\n",
       "                .attr(\"y\", function (d, i) {\n",
       "                    return y + i * DETAIL_BOX_HEIGHT;\n",
       "                });\n",
       "\n",
       "            var textContainer = tokenContainer.append(\"text\")\n",
       "                .classed(\"token\", true)\n",
       "                .text(function (d) {\n",
       "                    return d;\n",
       "                })\n",
       "                .attr(\"font-size\", TEXT_SIZE + \"px\")\n",
       "                .style(\"cursor\", \"default\")\n",
       "                .style(\"-webkit-user-select\", \"none\")\n",
       "                .attr(\"fill\", fillColor)\n",
       "                .attr(\"x\", x)\n",
       "                .attr(\"y\", function (d, i) {\n",
       "                    return i * DETAIL_BOX_HEIGHT + y;\n",
       "                })\n",
       "                .attr(\"height\", DETAIL_BOX_HEIGHT)\n",
       "                .attr(\"width\", DETAIL_BOX_WIDTH)\n",
       "                .attr(\"dy\", TEXT_SIZE);\n",
       "\n",
       "            if (id == \"leftText\") {\n",
       "                textContainer.style(\"text-anchor\", \"end\")\n",
       "                    .attr(\"dx\", DETAIL_BOX_WIDTH - 2);\n",
       "                tokenContainer.on(\"mouseover\", function (d, index) {\n",
       "                    highlightSelection(index);\n",
       "                });\n",
       "                tokenContainer.on(\"mouseleave\", function () {\n",
       "                    unhighlightSelection();\n",
       "                });\n",
       "            }\n",
       "        }\n",
       "\n",
       "        function highlightSelection(index) {\n",
       "            config.svg.select(\"#leftText\")\n",
       "                .selectAll(\".highlight\")\n",
       "                .style(\"opacity\", function (d, i) {\n",
       "                    return i == index ? 1.0 : 0.0;\n",
       "                });\n",
       "            config.svg.selectAll(\".attn-line-group\")\n",
       "                .style(\"opacity\", function (d, i) {\n",
       "                    return i == index ? 1.0 : 0.0;\n",
       "                });\n",
       "        }\n",
       "\n",
       "        function unhighlightSelection() {\n",
       "            config.svg.select(\"#leftText\")\n",
       "                .selectAll(\".highlight\")\n",
       "                .style(\"opacity\", 0.0);\n",
       "            config.svg.selectAll(\".attn-line-group\")\n",
       "                .style(\"opacity\", 1);\n",
       "        }\n",
       "\n",
       "        function renderThumbnailAttn(x, y, att, layerIndex, headIndex) {\n",
       "\n",
       "            var attnContainer = config.svg.append(\"svg:g\");\n",
       "\n",
       "            var attnBackground = attnContainer.append(\"rect\")\n",
       "                .attr(\"id\", 'attn_background_' + layerIndex + \"_\" + headIndex)\n",
       "                .classed(\"attn_background\", true)\n",
       "                .attr(\"x\", x)\n",
       "                .attr(\"y\", y)\n",
       "                .attr(\"height\", config.thumbnailHeight)\n",
       "                .attr(\"width\", config.thumbnailWidth)\n",
       "                .attr(\"stroke-width\", 2)\n",
       "                .attr(\"stroke\", getLayerColor(layerIndex))\n",
       "                .attr(\"stroke-opacity\", 0)\n",
       "                .attr(\"fill\", getBackgroundColor());\n",
       "            var x1 = x + THUMBNAIL_PADDING;\n",
       "            var x2 = x1 + config.thumbnailWidth - 14;\n",
       "            var y1 = y + THUMBNAIL_PADDING;\n",
       "\n",
       "            attnContainer.selectAll(\"g\")\n",
       "                .data(att)\n",
       "                .enter()\n",
       "                .append(\"g\") // Add group for each source token\n",
       "                .attr(\"source-index\", function (d, i) { // Save index of source token\n",
       "                    return i;\n",
       "                })\n",
       "                .selectAll(\"line\")\n",
       "                .data(function (d) { // Loop over all target tokens\n",
       "                    return d;\n",
       "                })\n",
       "                .enter() // When entering\n",
       "                .append(\"line\")\n",
       "                .attr(\"x1\", x1)\n",
       "                .attr(\"y1\", function (d) {\n",
       "                    var sourceIndex = +this.parentNode.getAttribute(\"source-index\");\n",
       "                    return y1 + (sourceIndex + .5) * config.thumbnailBoxHeight;\n",
       "                })\n",
       "                .attr(\"x2\", x2)\n",
       "                .attr(\"y2\", function (d, targetIndex) {\n",
       "                    return y1 + (targetIndex + .5) * config.thumbnailBoxHeight;\n",
       "                })\n",
       "                .attr(\"stroke-width\", 2.2)\n",
       "                .attr(\"stroke\", getLayerColor(layerIndex))\n",
       "                .attr(\"stroke-opacity\", function (d) {\n",
       "                    return d;\n",
       "                });\n",
       "\n",
       "            var clickRegion = attnContainer.append(\"rect\")\n",
       "                .attr(\"x\", x)\n",
       "                .attr(\"y\", y)\n",
       "                .attr(\"height\", config.thumbnailHeight)\n",
       "                .attr(\"width\", config.thumbnailWidth)\n",
       "                .style(\"opacity\", 0);\n",
       "\n",
       "            clickRegion.on(\"click\", function (d, index) {\n",
       "                var attnBackgroundOther = config.svg.selectAll(\".attn_background\");\n",
       "                attnBackgroundOther.attr(\"fill\", getBackgroundColor());\n",
       "                attnBackgroundOther.attr(\"stroke-opacity\", 0);\n",
       "\n",
       "                config.svg.selectAll(\".detail\").remove();\n",
       "                if (config.detail_layer != layerIndex || config.detail_head != headIndex) {\n",
       "                    renderDetail(att, layerIndex, headIndex);\n",
       "                    config.detail_layer = layerIndex;\n",
       "                    config.detail_head = headIndex;\n",
       "                    attnBackground.attr(\"fill\", getHighlightColor());\n",
       "                    attnBackground.attr(\"stroke-opacity\", .8);\n",
       "                } else {\n",
       "                    config.detail_layer = null;\n",
       "                    config.detail_head = null;\n",
       "                    attnBackground.attr(\"fill\", getBackgroundColor());\n",
       "                    attnBackground.attr(\"stroke-opacity\", 0);\n",
       "                }\n",
       "            });\n",
       "\n",
       "            clickRegion.on(\"mouseover\", function (d) {\n",
       "                d3.select(this).style(\"cursor\", \"pointer\");\n",
       "            });\n",
       "        }\n",
       "\n",
       "        function renderDetailFrame(x, y, layerIndex) {\n",
       "            var detailFrame = config.svg.append(\"rect\")\n",
       "                .classed(\"detail\", true)\n",
       "                .attr(\"x\", x)\n",
       "                .attr(\"y\", y)\n",
       "                .attr(\"height\", config.detailHeight)\n",
       "                .attr(\"width\", DETAIL_WIDTH)\n",
       "                .style(\"opacity\", 1)\n",
       "                .attr(\"stroke-width\", 1.5)\n",
       "                .attr(\"stroke-opacity\", 0.7)\n",
       "                .attr(\"stroke\", getLayerColor(layerIndex));\n",
       "        }\n",
       "\n",
       "        function renderDetailAttn(x, y, att, layerIndex) {\n",
       "            var attnContainer = config.svg.append(\"svg:g\")\n",
       "                .classed(\"detail\", true)\n",
       "                .attr(\"pointer-events\", \"none\");\n",
       "            attnContainer.selectAll(\"g\")\n",
       "                .data(att)\n",
       "                .enter()\n",
       "                .append(\"g\") // Add group for each source token\n",
       "                .classed('attn-line-group', true)\n",
       "                .attr(\"source-index\", function (d, i) { // Save index of source token\n",
       "                    return i;\n",
       "                })\n",
       "                .selectAll(\"line\")\n",
       "                .data(function (d) { // Loop over all target tokens\n",
       "                    return d;\n",
       "                })\n",
       "                .enter()\n",
       "                .append(\"line\")\n",
       "                .attr(\"x1\", x + ATTN_PADDING)\n",
       "                .attr(\"y1\", function (d) {\n",
       "                    var sourceIndex = +this.parentNode.getAttribute(\"source-index\");\n",
       "                    return y + (sourceIndex + .5) * DETAIL_BOX_HEIGHT;\n",
       "                })\n",
       "                .attr(\"x2\", x + DETAIL_ATTENTION_WIDTH - ATTN_PADDING)\n",
       "                .attr(\"y2\", function (d, targetIndex) {\n",
       "                    return y + (targetIndex + .5) * DETAIL_BOX_HEIGHT;\n",
       "                })\n",
       "                .attr(\"stroke-width\", 2.2)\n",
       "                .attr(\"stroke\", getLayerColor(layerIndex))\n",
       "                .attr(\"stroke-opacity\", function (d) {\n",
       "                    return d;\n",
       "                });\n",
       "        }\n",
       "\n",
       "        function getLayerColor(layer) {\n",
       "          return LAYER_COLORS[config.layers[layer] % 10];\n",
       "        }\n",
       "\n",
       "        function getTextColor() {\n",
       "            return PALETTE[config.mode]['text']\n",
       "        }\n",
       "\n",
       "        function getBackgroundColor() {\n",
       "           return PALETTE[config.mode]['background']\n",
       "        }\n",
       "\n",
       "        function getHighlightColor() {\n",
       "           return PALETTE[config.mode]['highlight']\n",
       "        }\n",
       "\n",
       "        function initialize() {\n",
       "            config.attention = params['attention'];\n",
       "            config.filter = params['default_filter'];\n",
       "            config.mode = params['display_mode'];\n",
       "            config.layers = params['include_layers']\n",
       "            config.heads = params['include_heads']\n",
       "            config.totalHeads = params['total_heads']\n",
       "            config.rootDivId = params['root_div_id'];\n",
       "            $(`#${config.rootDivId} #filter`).on('change', function (e) {\n",
       "                config.filter = e.currentTarget.value;\n",
       "                render();\n",
       "            });\n",
       "        }\n",
       "\n",
       "        initialize();\n",
       "        render();\n",
       "\n",
       "    });"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from transformers import AutoTokenizer, AutoModel, utils, AutoModelForCausalLM\n",
    "\n",
    "from bertviz import model_view\n",
    "utils.logging.set_verbosity_error()  # Suppress standard warnings\n",
    "\n",
    "model_name = 'openai-community/gpt2'\n",
    "input_text = \"No, I am your father\"  \n",
    "model = AutoModelForCausalLM.from_pretrained(model_name, output_attentions=True)\n",
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n",
    "inputs = tokenizer.encode(input_text, return_tensors='pt')  # Tokenize input text\n",
    "outputs = model(inputs)  # Run model\n",
    "attention = outputs[-1]  # Retrieve attention from model outputs\n",
    "tokens = tokenizer.convert_ids_to_tokens(inputs[0])  # Convert input ids to token strings\n",
    "model_view(attention, tokens)  # Display model view"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "007b57fa-711d-4d1c-8603-b8a8d44747b2",
   "metadata": {},
   "source": [
    "## Let's train a mini-LLM from scratch"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd1fb5f1-3b1e-46a0-8bc5-d47d2dcc3222",
   "metadata": {},
   "source": [
    "Set up hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "fa15fa8e-035d-4a42-b666-cb01d92ffd11",
   "metadata": {},
   "outputs": [],
   "source": [
    "# hyperparameters\n",
    "batch_size = 16 # how many independent sequences will we process in parallel?\n",
    "block_size = 32 # what is the maximum context length for predictions?\n",
    "max_iters = 5000\n",
    "eval_interval = 10\n",
    "learning_rate = 1e-3\n",
    "device = 'cuda' if torch.cuda.is_available() else 'cpu'\n",
    "eval_iters = 200\n",
    "n_embd = 64\n",
    "n_head = 4 ## so head_size = 16\n",
    "n_layer = 4\n",
    "dropout = 0.0\n",
    "# ------------"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14907519-cd93-45aa-b24a-e1e95f99cb54",
   "metadata": {},
   "source": [
    "Load in data and create train and test datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "ad38be1a-5f85-4bc3-894d-38d12cc8cbc3",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('ai-science-training-series/05_llm_part2/dataset/input.txt', 'r', encoding='utf-8') as f:\n",
    "    text = f.read()\n",
    "\n",
    "# here are all the unique characters that occur in this text\n",
    "chars = sorted(list(set(text)))\n",
    "vocab_size = len(chars)\n",
    "# create a mapping from characters to integers\n",
    "stoi = { ch:i for i,ch in enumerate(chars) }\n",
    "itos = { i:ch for i,ch in enumerate(chars) }\n",
    "encode = lambda s: [stoi[c] for c in s] # encoder: take a string, output a list of integers\n",
    "decode = lambda l: ''.join([itos[i] for i in l]) # decoder: take a list of integers, output a string\n",
    "\n",
    "# Train and test splits\n",
    "data = torch.tensor(encode(text), dtype=torch.long)\n",
    "n = int(0.9*len(data)) # first 90% will be train, rest val\n",
    "train_data = data[:n]\n",
    "val_data = data[n:]\n",
    "\n",
    "# data loading\n",
    "def get_batch(split):\n",
    "    # generate a small batch of data of inputs x and targets y\n",
    "    data = train_data if split == 'train' else val_data\n",
    "    ix = torch.randint(len(data) - block_size, (batch_size,))\n",
    "    x = torch.stack([data[i:i+block_size] for i in ix])\n",
    "    y = torch.stack([data[i+1:i+block_size+1] for i in ix]) # The character sequence right after the character sequence at the corresponding index in x\n",
    "    x, y = x.to(device), y.to(device)\n",
    "    return x, y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24d121e6-4e7c-4557-a3a8-b279059d8ade",
   "metadata": {},
   "source": [
    "Set up the components of the decoder block"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "e3d6cf21-6849-4b62-b7df-0b395d98104f",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Head(nn.Module):\n",
    "    \"\"\" one head of self-attention \"\"\"\n",
    "\n",
    "    def __init__(self, head_size):\n",
    "        super().__init__()\n",
    "        self.key = nn.Linear(n_embd, head_size, bias=False)\n",
    "        self.query = nn.Linear(n_embd, head_size, bias=False)\n",
    "        self.value = nn.Linear(n_embd, head_size, bias=False)\n",
    "        self.register_buffer('tril', torch.tril(torch.ones(block_size, block_size)))\n",
    "\n",
    "        self.dropout = nn.Dropout(dropout)\n",
    "\n",
    "    def forward(self, x):\n",
    "        B,T,C = x.shape\n",
    "        k = self.key(x)   # (B,T,C) 16,32,16\n",
    "        q = self.query(x) # (B,T,C)\n",
    "        # compute attention scores (\"affinities\")\n",
    "        wei = q @ k.transpose(-2,-1) * C**-0.5 # (B, T, C) @ (B, C, T) -> (B, T, T)\n",
    "        wei = wei.masked_fill(self.tril[:T, :T] == 0, float('-inf')) # (B, T, T)\n",
    "        wei = F.softmax(wei, dim=-1) # (B, T, T)\n",
    "        wei = self.dropout(wei)\n",
    "        # perform the weighted aggregation of the values\n",
    "        v = self.value(x) # (B,T,C)\n",
    "        out = wei @ v # (B, T, T) @ (B, T, C) -> (B, T, C)\n",
    "        return out\n",
    "\n",
    "class MultiHeadAttention(nn.Module):\n",
    "    \"\"\" multiple heads of self-attention in parallel \"\"\"\n",
    "\n",
    "    def __init__(self, num_heads, head_size):\n",
    "        super().__init__()\n",
    "        self.heads = nn.ModuleList([Head(head_size) for _ in range(num_heads)])\n",
    "        self.proj = nn.Linear(n_embd, n_embd)\n",
    "        self.dropout = nn.Dropout(dropout)\n",
    "\n",
    "    def forward(self, x):\n",
    "        out = torch.cat([h(x) for h in self.heads], dim=-1)\n",
    "        out = self.dropout(self.proj(out))\n",
    "        return out\n",
    "\n",
    "class FeedFoward(nn.Module):\n",
    "    \"\"\" a simple linear layer followed by a non-linearity \"\"\"\n",
    "\n",
    "    def __init__(self, n_embd):\n",
    "        super().__init__()\n",
    "        self.net = nn.Sequential(\n",
    "            nn.Linear(n_embd, 4 * n_embd),\n",
    "            nn.ReLU(),\n",
    "            nn.Linear(4 * n_embd, n_embd), # Projection layer going back into the residual pathway\n",
    "            nn.Dropout(dropout),\n",
    "        )\n",
    "\n",
    "    def forward(self, x):\n",
    "        return self.net(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf583492-0efb-48b7-99c7-4bbf40078ca8",
   "metadata": {},
   "source": [
    "Combine components into the Decoder block"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "e3430cd8-02be-45e9-ac8f-4d7df9e28b33",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Block(nn.Module):\n",
    "    \"\"\" Transformer block: communication followed by computation \"\"\"\n",
    "\n",
    "    def __init__(self, n_embd, n_head):\n",
    "        # n_embd: embedding dimension, n_head: the number of heads we'd like\n",
    "        super().__init__()\n",
    "        head_size = n_embd // n_head\n",
    "        self.sa = MultiHeadAttention(n_head, head_size)\n",
    "        self.ffwd = FeedFoward(n_embd)\n",
    "        self.ln1 = nn.LayerNorm(n_embd)\n",
    "        self.ln2 = nn.LayerNorm(n_embd)\n",
    "\n",
    "    def forward(self, x):\n",
    "        x = x + self.sa(self.ln1(x))    # Communication\n",
    "        x = x + self.ffwd(self.ln2(x))  # Computation\n",
    "        return x"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9971e73-13ea-4088-9a88-189280459a01",
   "metadata": {},
   "source": [
    "Set up the full transformer model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "21e8bd1d-a088-4e1f-8b88-dd0a60210d6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# super simple bigram model\n",
    "class BigramLanguageModel(nn.Module):\n",
    "\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        # each token directly reads off the logits for the next token from a lookup table\n",
    "        self.token_embedding_table = nn.Embedding(vocab_size, n_embd)\n",
    "        self.position_embedding_table = nn.Embedding(block_size, n_embd)\n",
    "        self.blocks = nn.Sequential(*[Block(n_embd, n_head=n_head) for _ in range(n_layer)])\n",
    "        self.ln_f = nn.LayerNorm(n_embd) # final layer norm\n",
    "        self.lm_head = nn.Linear(n_embd, vocab_size)\n",
    "\n",
    "    def forward(self, idx, targets=None):\n",
    "        B, T = idx.shape\n",
    "\n",
    "        # idx and targets are both (B,T) tensor of integers\n",
    "        tok_emb = self.token_embedding_table(idx) # (B,T,C)\n",
    "        pos_emb = self.position_embedding_table(torch.arange(T, device=device)) # (T,C)\n",
    "        x = tok_emb + pos_emb # (B,T,C)\n",
    "        x = self.blocks(x) # (B,T,C)\n",
    "        x = self.ln_f(x) # (B,T,C)\n",
    "        logits = self.lm_head(x) # (B,T,vocab_size)\n",
    "\n",
    "        if targets is None:\n",
    "            loss = None\n",
    "        else:\n",
    "            B, T, C = logits.shape\n",
    "            logits = logits.view(B*T, C)\n",
    "            targets = targets.view(B*T)\n",
    "            loss = F.cross_entropy(logits, targets)\n",
    "\n",
    "        return logits, loss\n",
    "\n",
    "    def generate(self, idx, max_new_tokens):\n",
    "        # idx is (B, T) array of indices in the current context\n",
    "        for _ in range(max_new_tokens):\n",
    "            # crop idx to the last block_size tokens\n",
    "            idx_cond = idx[:, -block_size:]\n",
    "            # get the predictions\n",
    "            logits, loss = self(idx_cond)\n",
    "            # focus only on the last time step\n",
    "            logits = logits[:, -1, :] # becomes (B, C)\n",
    "            # apply softmax to get probabilities\n",
    "            probs = F.softmax(logits, dim=-1) # (B, C)\n",
    "            # sample from the distribution\n",
    "            idx_next = torch.multinomial(probs, num_samples=1) # (B, 1)\n",
    "            # append sampled index to the running sequence\n",
    "            idx = torch.cat((idx, idx_next), dim=1) # (B, T+1)\n",
    "        return idx"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "539c4c8a-794d-4c5f-8468-93b6894e62cc",
   "metadata": {},
   "source": [
    "## Homework"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3959374-de3b-4932-85aa-67b1d37fffb6",
   "metadata": {},
   "source": [
    "Import additional packages and instantiate model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "dca76c96-a490-4c03-9336-efef567aacac",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "model = BigramLanguageModel()\n",
    "model.to('cuda')\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4b431ca-d78c-4a7a-be1c-8a3c629c30dd",
   "metadata": {},
   "source": [
    "Define useful functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "afe83cb5-173e-49a1-a388-7c6ecd8dfb99",
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_one_batch(model, optimizer):\n",
    "    model.train()\n",
    "    X, Y = get_batch('train')\n",
    "        \n",
    "    # forward pass\n",
    "    logits, loss = model(X, Y)\n",
    "        \n",
    "    # backward pass calculates gradients\n",
    "    loss.backward()\n",
    "\n",
    "    # take one step with these gradients\n",
    "    optimizer.step()\n",
    "\n",
    "    # resets the gradients \n",
    "    optimizer.zero_grad()      \n",
    "\n",
    "@torch.no_grad()\n",
    "def estimate_loss():\n",
    "    out = {}\n",
    "    model.eval()\n",
    "    for split in ['train', 'val']:\n",
    "        losses = torch.zeros(eval_iters)\n",
    "        for k in range(eval_iters):\n",
    "            X, Y = get_batch(split)\n",
    "            logits, loss = model(X, Y)\n",
    "            losses[k] = loss.item()\n",
    "        out[split] = losses.mean()\n",
    "    model.train()\n",
    "    return out"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b042b229-e302-4a9f-b9f3-91d1765b8a52",
   "metadata": {},
   "source": [
    "Train the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "bfea4ad3-0d04-4c6d-a43d-e8e26fb37ef6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 0 batches: training perplexity: 59.290, validation perplexity: 59.377\n",
      "First 200 batches: training perplexity: 12.461, validation perplexity: 12.521\n",
      "First 400 batches: training perplexity: 10.650, validation perplexity: 10.813\n",
      "First 600 batches: training perplexity: 9.665, validation perplexity: 9.665\n",
      "First 800 batches: training perplexity: 8.935, validation perplexity: 9.069\n",
      "First 1000 batches: training perplexity: 8.237, validation perplexity: 8.497\n",
      "First 1200 batches: training perplexity: 7.834, validation perplexity: 8.180\n",
      "First 1400 batches: training perplexity: 7.422, validation perplexity: 7.879\n",
      "First 1600 batches: training perplexity: 7.115, validation perplexity: 7.658\n",
      "First 1800 batches: training perplexity: 6.808, validation perplexity: 7.445\n",
      "First 2000 batches: training perplexity: 6.615, validation perplexity: 7.260\n",
      "First 2200 batches: training perplexity: 6.462, validation perplexity: 7.108\n",
      "First 2400 batches: training perplexity: 6.226, validation perplexity: 7.067\n",
      "First 2600 batches: training perplexity: 6.133, validation perplexity: 7.068\n",
      "First 2800 batches: training perplexity: 6.033, validation perplexity: 6.918\n",
      "First 3000 batches: training perplexity: 5.915, validation perplexity: 6.764\n",
      "First 3200 batches: training perplexity: 5.698, validation perplexity: 6.755\n",
      "First 3400 batches: training perplexity: 5.711, validation perplexity: 6.730\n",
      "First 3600 batches: training perplexity: 5.667, validation perplexity: 6.559\n",
      "First 3800 batches: training perplexity: 5.543, validation perplexity: 6.439\n",
      "First 4000 batches: training perplexity: 5.545, validation perplexity: 6.402\n",
      "First 4200 batches: training perplexity: 5.423, validation perplexity: 6.425\n",
      "First 4400 batches: training perplexity: 5.356, validation perplexity: 6.306\n",
      "First 4600 batches: training perplexity: 5.270, validation perplexity: 6.167\n",
      "First 4800 batches: training perplexity: 5.321, validation perplexity: 6.199\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1UAAAHWCAYAAACfRKOZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB5NUlEQVR4nO3dd3wUdf7H8fduspveCCEFErp0EAEhooKCICiKchbkpIj60wMVUQ89UYoK9o7l1AMb6nkiVkRAQEVARECaoQWpCSWdtE12fn+sWbMkgZRNdhNez8djzE7Zmc8s38S88535jskwDEMAAAAAgGoxe7oAAAAAAKjPCFUAAAAAUAOEKgAAAACoAUIVAAAAANQAoQoAAAAAaoBQBQAAAAA1QKgCAAAAgBogVAEAAABADRCqAAAAAKAGCFUAUEljx45VixYtqvXe6dOny2QyubcgL7N3716ZTCbNmzevzo9tMpk0ffp05/y8efNkMpm0d+/e0763RYsWGjt2rFvrqUlbaYhO/vdxtzPh+wuAdyNUAaj3TCZTpaYVK1Z4utQz3p133imTyaRdu3ZVuM2DDz4ok8mk3377rQ4rq7pDhw5p+vTp2rhxo6dLcSoJtiWTj4+PEhISdNVVV3lVnXVh1qxZWrhwoafLAHCGIFQBqPfeffddl+mSSy4pd3mHDh1qdJw33nhDSUlJ1Xrv1KlTlZeXV6PjNwSjRo2SJM2fP7/CbT744AN16dJFXbt2rfZxbrzxRuXl5al58+bV3sfpHDp0SDNmzCg3rNSkrbjDyJEj9e677+o///mPbrjhBn333Xfq06dPgw1W5X1/EaoA1CVfTxcAADX197//3WV+zZo1WrJkSZnlJ8vNzVVgYGClj2OxWKpVnyT5+vrK15cfub1791abNm30wQcf6OGHHy6zfvXq1UpOTtbjjz9eo+P4+PjIx8enRvuoiZq0FXc455xzXNp/3759dcUVV+jVV1/V66+/XqN9nzhxQkFBQTUt0a34/gLgafRUATgj9O/fX507d9b69et14YUXKjAwUP/6178kSZ999pkuu+wyxcXFyc/PT61bt9Yjjzyi4uJil32cfJ9MyaVWTz/9tP7973+rdevW8vPzU69evbRu3TqX95Z3z4fJZNLEiRO1cOFCde7cWX5+furUqZO++eabMvWvWLFCPXv2lL+/v1q3bq3XX3+90veR/PDDD7rmmmuUkJAgPz8/xcfH6+677y7zl/2xY8cqODhYBw8e1PDhwxUcHKyoqCjde++9ZT6LjIwMjR07VmFhYQoPD9eYMWOUkZFx2lokR2/V77//rl9//bXMuvnz58tkMmnkyJEqLCzUww8/rB49eigsLExBQUG64IILtHz58tMeo7x7qgzD0KOPPqpmzZopMDBQF110kbZu3VrmvWlpabr33nvVpUsXBQcHKzQ0VEOGDNGmTZuc26xYsUK9evWSJI0bN855uV3J/WTl3VN14sQJ3XPPPYqPj5efn5/atWunp59+WoZhuGxXlXZRWRdffLEkKTk52bls7dq1uvTSSxUWFqbAwED169dPq1atcnlfSRvbtm2bbrjhBkVEROj88893nmNwcLD27NmjwYMHKygoSHFxcZo5c2aZcyrPwYMHddNNNyk6Otp5jv/5z3+c6/Py8tS+fXu1b9/epa2mpaUpNjZW5513nrNdnvy9YDKZdOLECb399tvOf5uxY8dq+fLlMplM+vTTT8vUU9L2Vq9eXZmPFABc8GcdAGeM48ePa8iQIbr++uv197//XdHR0ZIcv4AHBwdr8uTJCg4O1nfffaeHH35YWVlZeuqpp0673/nz5ys7O1v/93//J5PJpCeffFJXX3219uzZc9oeix9//FELFizQP/7xD4WEhOjFF1/UiBEjtG/fPkVGRkqSNmzYoEsvvVSxsbGaMWOGiouLNXPmTEVFRVXqvD/++GPl5ubq9ttvV2RkpH7++We99NJLOnDggD7++GOXbYuLizV48GD17t1bTz/9tJYuXapnnnlGrVu31u233y7JEU6uvPJK/fjjj7rtttvUoUMHffrppxozZkyl6hk1apRmzJih+fPn65xzznE59n//+19dcMEFSkhI0LFjx/Tmm29q5MiRuuWWW5Sdna233npLgwcP1s8//6yzzz67Uscr8fDDD+vRRx/V0KFDNXToUP36668aNGiQCgsLXbbbs2ePFi5cqGuuuUYtW7ZUamqqXn/9dfXr10/btm1TXFycOnTooJkzZ+rhhx/WrbfeqgsuuECSdN5555V7bMMwdMUVV2j58uUaP368zj77bC1evFj33XefDh48qOeee85l+8q0i6rYvXu3JDnf+91332nIkCHq0aOHpk2bJrPZrLlz5+riiy/WDz/8oHPPPdfl/ddcc43atm2rWbNmuQSm4uJiXXrpperTp4+efPJJffPNN5o2bZqKioo0c+bMCutJTU1Vnz59nAEyKipKixYt0vjx45WVlaVJkyYpICBAb7/9tvr27asHH3xQzz77rCRpwoQJyszM1Lx58yrsjXz33Xd1880369xzz9Wtt94qSWrdurX69Omj+Ph4vf/++7rqqqtc3vP++++rdevWSkxMrOKnCwCSDABoYCZMmGCc/OOtX79+hiTjtddeK7N9bm5umWX/93//ZwQGBhr5+fnOZWPGjDGaN2/unE9OTjYkGZGRkUZaWppz+WeffWZIMr744gvnsmnTppWpSZJhtVqNXbt2OZdt2rTJkGS89NJLzmXDhg0zAgMDjYMHDzqX7dy50/D19S2zz/KUd36zZ882TCaT8ccff7icnyRj5syZLtt2797d6NGjh3N+4cKFhiTjySefdC4rKioyLrjgAkOSMXfu3NPW1KtXL6NZs2ZGcXGxc9k333xjSDJef/115z4LCgpc3peenm5ER0cbN910k8tySca0adOc83PnzjUkGcnJyYZhGMaRI0cMq9VqXHbZZYbdbndu969//cuQZIwZM8a5LD8/36Uuw3D8W/v5+bl8NuvWravwfE9uKyWf2aOPPuqy3d/+9jfDZDK5tIHKtovylLTJGTNmGEePHjVSUlKMFStWGN27dzckGZ988olht9uNtm3bGoMHD3b5LHJzc42WLVsal1xyiXNZSbsdOXJkuecoybjjjjucy+x2u3HZZZcZVqvVOHr0qMs5lf73GT9+vBEbG2scO3bMZZ/XX3+9ERYW5tJmH3jgAcNsNhvff/+98fHHHxuSjOeff97lfeV9fwUFBbn8u5ben5+fn5GRkeFcduTIEcPX19elRgCoCi7/A3DG8PPz07hx48osDwgIcL7Ozs7WsWPHdMEFFyg3N1e///77afd73XXXKSIiwjlf0muxZ8+e07534MCBat26tXO+a9euCg0Ndb63uLhYS5cu1fDhwxUXF+fcrk2bNhoyZMhp9y+5nt+JEyd07NgxnXfeeTIMQxs2bCiz/W233eYyf8EFF7icy9dffy1fX19nz5XkuIfpjjvuqFQ9kuM+uAMHDuj77793Lps/f76sVquuueYa5z6tVqskyW63Ky0tTUVFRerZs2e5lw6eytKlS1VYWKg77rjD5TKxSZMmldnWz89PZrPjf4/FxcU6fvy4goOD1a5duyoft8TXX38tHx8f3XnnnS7L77nnHhmGoUWLFrksP127OJ1p06YpKipKMTEx6t+/v3bv3q0nnnhCV199tTZu3KidO3fqhhtu0PHjx3Xs2DEdO3ZMJ06c0IABA/T999/Lbre77O/kNlHaxIkTna9Lep4KCwu1dOnScrc3DEOffPKJhg0bJsMwnMc/duyYBg8erMzMTJfPefr06erUqZPGjBmjf/zjH+rXr1+Zz7EqRo8erYKCAv3vf/9zLvvoo49UVFR02vswAaAiXP4H4IzRtGlT5y/ppW3dulVTp07Vd999p6ysLJd1mZmZp91vQkKCy3xJwEpPT6/ye0veX/LeI0eOKC8vT23atCmzXXnLyrNv3z49/PDD+vzzz8vUdPL5+fv7l7mssHQ9kvTHH38oNjZWwcHBLtu1a9euUvVI0vXXX6/Jkydr/vz56t+/v/Lz8/Xpp59qyJAhLgH17bff1jPPPKPff/9dNpvNubxly5aVPlZJzZLUtm1bl+VRUVEux5McAe6FF17QK6+8ouTkZJf7yapz6V3J8ePi4hQSEuKyvGREypL6SpyuXZzOrbfeqmuuuUZms1nh4eHq1KmT/Pz8JEk7d+6UpFNerpmZmenyuVT0eZvNZrVq1cpl2VlnnSVJFT4j7OjRo8rIyNC///1v/fvf/y53myNHjjhfW61W/ec//1GvXr3k7++vuXPn1uiZVO3bt1evXr30/vvva/z48ZIcl/716dOn0t9TAHAyQhWAM0bpHpsSGRkZ6tevn0JDQzVz5ky1bt1a/v7++vXXXzVlypQyf7EvT0X3dRiVuFm/Ju+tjOLiYl1yySVKS0vTlClT1L59ewUFBengwYMaO3ZsmfOrqxHzmjRpoksuuUSffPKJ5syZoy+++ELZ2dnOIdcl6b333tPYsWM1fPhw3XfffWrSpIl8fHw0e/Zs5z1CtWHWrFl66KGHdNNNN+mRRx5Ro0aNZDabNWnSpEq1B3eoabto27atBg4cWO66knN46qmnKrwv7eTAXN73TnWVHP/vf/97hcHu5OH0Fy9eLEnKz8/Xzp07qxyqTzZ69GjdddddOnDggAoKCrRmzRq9/PLLNdongDMboQrAGW3FihU6fvy4FixYoAsvvNC5vPQoaZ7UpEkT+fv7l/uw3FM9QLfE5s2btWPHDr399tsaPXq0c/mSJUuqXVPz5s21bNky5eTkuPzyXdXnMo0aNUrffPONFi1apPnz5ys0NFTDhg1zrv/f//6nVq1aacGCBS49E9OmTatWzZKjl6Z0z8rRo0fL9P7873//00UXXaS33nrLZXlGRoYaN27snK9Kb0nz5s21dOlSZWdnu/RWlVxeWpvP0zpZyWWFoaGhFQavyrLb7dqzZ4+zd0qSduzYIUllRj8sERUVpZCQEBUXF1fq+L/99ptmzpypcePGaePGjbr55pu1efNmhYWFnfJ9p/r3Kekp/eCDD5SXlyeLxaLrrrvutLUAQEW4pwrAGa2kR6B0D0BhYaFeeeUVT5XkwsfHRwMHDtTChQt16NAh5/Jdu3aVuQ+novdLrudnGIZeeOGFatc0dOhQFRUV6dVXX3UuKy4u1ksvvVSl/QwfPlyBgYF65ZVXtGjRIl199dXy9/c/Ze1r166t1pDXAwcOlMVi0UsvveSyv+eff77Mtj4+PmV6hD7++GMdPHjQZVnJs5oqM5T80KFDVVxcXKY35LnnnpPJZKr0/XHu0KNHD7Vu3VpPP/20cnJyyqw/evRolfZX+pwMw9DLL78si8WiAQMGlLu9j4+PRowYoU8++URbtmw55fFtNpvGjh2ruLg4vfDCC5o3b55SU1N19913n7auoKCgCv9tGjdurCFDhui9997T+++/r0svvdQlMANAVdFTBeCMdt555ykiIkJjxozRnXfeKZPJpHfffddtl9+5w/Tp0/Xtt9+qb9++uv32252/nHfu3FkbN2485Xvbt2+v1q1b695779XBgwcVGhqqTz75pNL35pRn2LBh6tu3r+6//37t3btXHTt21IIFCyp1/1lpwcHBGj58uObPny9JLpf+SdLll1+uBQsW6KqrrtJll12m5ORkvfbaa+rYsWO5YeBUSp63NXv2bF1++eUaOnSoNmzYoEWLFpX5Zfryyy939oycd9552rx5s95///0y9w61bt1a4eHheu211xQSEqKgoCD17t273EvThg0bposuukgPPvig9u7dq27duunbb7/VZ599pkmTJrkMSlHbzGaz3nzzTQ0ZMkSdOnXSuHHj1LRpUx08eFDLly9XaGiovvjii0rty9/fX998843GjBmj3r17a9GiRfrqq6/0r3/965RD/j/++ONavny5evfurVtuuUUdO3ZUWlqafv31Vy1dulRpaWmSpEcffVQbN27UsmXLFBISoq5du+rhhx/W1KlT9be//U1Dhw6t8Bg9evTQ0qVL9eyzzyouLk4tW7ZU7969netHjx6tv/3tb5KkRx55pFLnCwAVoacKwBktMjJSX375pWJjYzV16lQ9/fTTuuSSS/Tkk096ujSnHj16aNGiRYqIiNBDDz2kt956SzNnztSAAQNcenbKY7FY9MUXX+jss8/W7NmzNWPGDLVt21bvvPNOtesxm836/PPPNWrUKL333nt68MEH1bRpU7399ttV3ldJkIqNjXU+oLbE2LFjNWvWLG3atEl33nmnFi9erPfee089e/asVt2PPvqoZsyYoQ0bNui+++7T7t279e233zp7nEr861//0j333KPFixfrrrvu0q+//qqvvvpK8fHxLttZLBa9/fbb8vHx0W233aaRI0dq5cqV5R675DObNGmSvvzyS02aNEnbtm3TU0895Xz+Ul3q37+/Vq9erZ49e+rll1/WHXfcoXnz5ikmJqZSvUAlfHx89M033yglJUX33Xef1q1bp2nTpp02pERHR+vnn3/WuHHjtGDBAk2cOFEvvPCC0tLS9MQTT0iSfv31V82aNUsTJ07URRdd5Hzv/fffr169eumWW245ZS/hs88+qx49emjq1KkaOXKkS8+q5Ai6ERERCgsL0xVXXFHpcwaA8pgMb/pzLACg0oYPH66tW7c6R3MD6tLYsWP1v//9r8q9ht6iqKhIcXFxGjZsWJn75wCgquipAoB6IC8vz2V+586d+vrrr9W/f3/PFATUcwsXLtTRo0ddBnABgOrinioAqAdatWqlsWPHqlWrVvrjjz/06quvymq16p///KenSwPqlbVr1+q3337TI488ou7du6tfv36eLglAA0CoAoB64NJLL9UHH3yglJQU+fn5KTExUbNmzSrzMFsAp/bqq6/qvffe09lnn6158+Z5uhwADQT3VAEAAABADXBPFQAAAADUAKEKAAAAAGqgwd9TZbfbdejQIYWEhMhkMnm6HAAAAAAeYhiGsrOzFRcXJ7PZff1LDT5UHTp0qMwDGwEAAACcufbv369mzZq5bX8NPlSFhIRIcnxwoaGhHq3FZrPp22+/1aBBg2SxWDxaC+o32hLchbYEd6AdwV1oS3CXitpSVlaW4uPjnRnBXRp8qCq55C80NNQrQlVgYKBCQ0P5QYEaoS3BXWhLcAfaEdyFtgR3OV1bcvdtQQxUAQAAAAA1QKgCAAAAgBogVAEAAABADTT4e6oAAADOVMXFxbLZbJ4uo9JsNpt8fX2Vn5+v4uJiT5eDesjHx0e+vnUfcQhVAAAADVBOTo4OHDggwzA8XUqlGYahmJgY7d+/n+eLotoCAwMVFRVVp8ckVAEAADQwxcXFOnDggPOXy/oSUOx2u3JychQcHOzWB7PizGAYhgoLC3X06FHt27evTo/t8VB18OBBTZkyRYsWLVJubq7atGmjuXPnqmfPnpIcH860adP0xhtvKCMjQ3379tWrr76qtm3berhyAAAA72Sz2WQYhqKiohQQEODpcirNbrersLBQ/v7+hCpUS0BAgCwWi/bu3SsfH586O65HW2t6err69u0ri8WiRYsWadu2bXrmmWcUERHh3ObJJ5/Uiy++qNdee01r165VUFCQBg8erPz8fA9WDgAA4P3qSw8V4E4lgbwu279He6qeeOIJxcfHa+7cuc5lLVu2dL42DEPPP/+8pk6dqiuvvFKS9M477yg6OloLFy7U9ddfX+c1AwAAAEBpHg1Vn3/+uQYPHqxrrrlGK1euVNOmTfWPf/xDt9xyiyQpOTlZKSkpGjhwoPM9YWFh6t27t1avXl1uqCooKFBBQYFzPisrS5KjG9zTo9+UHN/TdaD+oy3BXWhLcAfakfcpufzPbrfLbrd7upxKKxlUo6R2oDrsdruzLZ38c6m2fk55NFTt2bNHr776qiZPnqx//etfWrdune68805ZrVaNGTNGKSkpkqTo6GiX90VHRzvXnWz27NmaMWNGmeXffvutAgMD3X8S1bBkyRJPl4AGgrYEd6EtwR1oR97D19dXMTExysnJUWFhoafLqbLs7Gy37atr1666/fbbdfvtt1dq+x9//FHDhg3T3r17FRYW5rY6vMn8+fP1wAMP6I8//nDbPiMiIvTee+/psssuc9s+q6uwsNB5q9DJP5dyc3Nr5ZgeDVV2u109e/bUrFmzJEndu3fXli1b9Nprr2nMmDHV2ucDDzygyZMnO+ezsrIUHx+vQYMGKTQ01C11V5fNZtOSJUt0ySWXyGKxeLQW1G+0JbgLbQnuQDvyPvn5+dq/f7+Cg4Pl7+/v6XIq5XSDCjz88MOaNm1alfe7bt06BQUFVfqP6wMHDtTBgwcVHR1dq/fkrFixQgMGDHDON2nSRH379tWTTz6pVq1a1dpxJcnf318mk8mtvxsfPHhQERER8vPz0969e9W6dWutX79eZ599ttuOUVn5+fnOdn/yz6WSq9jczaOhKjY2Vh07dnRZ1qFDB33yySeSpJiYGElSamqqYmNjndukpqZW+A/k5+cnPz+/MsstFovX/KD3plpQv9GW4C60JbgD7ch7FBcXy2QyyWw215tR9A4fPiy73a7s7GwtWrRI06ZNU1JSknN96WHWDcNQcXFxpR7yevIVT6fj7++vuLi4qhVfDSXnkpSUpJCQEO3cuVO33nqrrrzySv3222/VGrnOZrNV6nuw5NjubBulP7PS+/dE+zObzc5AfPLPpdr6GeXR77K+ffu6fLNI0o4dO9S8eXNJjkErYmJitGzZMuf6rKwsrV27VomJiXVaa00ZdkN5vQfq/LETVJB8yNPlAACAM4hhSCdOeGaq7LOHY2JiFBMTo+joaIWGhspkMjmX/f777woJCdGiRYvUo0cP+fn56ccff9Tu3bt15ZVXKjo6WsHBwerVq5eWLl3qst8WLVro+eefd86bTCa9+eabuuqqqxQYGKi2bdvq888/d65fsWKFTCaTMjIyJEnz5s1TeHi4Fi9erA4dOig4OFiXXnqpDh8+7HxPUVGR7rzzToWHhysyMlJTpkzRmDFjNHz48NOed5MmTRQbG6sLL7xQDz/8sLZt26Zdu3ZJkj777DOdc8458vf3V6tWrTRjxgwVFRW5nMurr76qK664QkFBQXrsscec9X/11Vfq2rWr/P391adPH23ZsuWUdZzqWDNnzlRcXJyOHz/u3P6yyy7TRRdd5Lz3zWQyaeHChZL+Gniue/fuMplM6t+/v77//ntZLJYyt/BMmjRJF1xwwWk/J2/n0VB19913a82aNZo1a5Z27dql+fPn69///rcmTJggyfGPM2nSJD366KP6/PPPtXnzZo0ePVpxcXGVaqTexGQ2KWfjLkVmHFTqhsOnfwMAAICb5OZKwcGemdx5C8v999+vxx9/XNu3b1fXrl2Vk5OjoUOHatmyZdqwYYMuvfRSDRs27LQPfp0xY4auvfZa/fbbbxo6dKhGjRqltLS0U3x+uXr66af17rvv6vvvv9e+fft07733Otc/8cQTev/99zV37lytWrVKWVlZzoBRFSXPFCssLNQPP/yg0aNH66677tK2bdv0+uuva968eXrsscdc3jN9+nRdddVV2rx5s2666Sbn8vvuu0/PPPOM1q1bp6ioKA0bNqzCQRpOd6wHH3xQLVq00M033yxJmjNnjn766Se9/fbb5fZE/fzzz5KkpUuX6vDhw1qwYIEuvPBCtWrVSu+++65zO5vNpvfff9+l7nrL8LAvvvjC6Ny5s+Hn52e0b9/e+Pe//+2y3m63Gw899JARHR1t+Pn5GQMGDDCSkpIqvf/MzExDkpGZmenu0qvsN78ehiEZGx/51NOloJ4rLCw0Fi5caBQWFnq6FNRztCW4A+3I++Tl5Rnbtm0z8vLyDMMwjJwcw3D0GdX9lJNT+bqLi4uN9PR046233jLCwsKcy5cvX25IMhYuXHjafXTq1Ml46aWXnPPNmzc3nnvuOee8JGPq1KnO+ZycHEOSsWjRIpdjpaenG4ZhGHPnzjUkGbt27XK+Z86cOUZ0dLRzPjo62njqqaec80VFRUZCQoJx5ZVXVljnycc5dOiQcd555xlNmzY1CgoKjAEDBhizZs1yec+7775rxMbGupzLpEmTyt3vhx9+6Fx2/PhxIyAgwPjoo4+c51T6863MsXbv3m2EhIQYU6ZMMQICAoz333/fZXtJxqeffmoYhmEkJycbkowNGza4bPPEE08YHTp0cM5/8sknRnBwsJFTlUZSCXl5ecbWrVuNL7/8sszPpdrKBh69p0qSLr/8cl1++eUVrjeZTJo5c6ZmzpxZh1XVjqygGKlAyttDTxUAAKg7gYFSTo7nju0uPXv2dJnPycnR9OnT9dVXX+nw4cMqKipSXl7eaXuqunbt6nwdFBSk0NBQHTlypMLtAwMD1bp1a+d8bGysc/vMzEylpqbq3HPPda738fFRjx49KjUsfLNmzWQYhnJzc9WtWzd98sknslqt2rRpk1atWuXSM1VcXKz8/Hzl5uY6B944+TMpUfpWmUaNGqldu3bavn17udtW5litWrXS008/rf/7v//TddddpxtuuOG053aysWPHaurUqVqzZo369OmjefPm6dprr1VQUFCV9+VtPB6qziR5YbFSmlR8MNXTpQAAgDOIySQ1gN9by/zyfe+992rJkiV6+umn1aZNGwUEBOhvf/vbaYeRP3mwApPJdMoAVN72RmVvFjuNH374QaGhoWrSpIlCQkKcy3NycjRjxgxdffXVZd5TekRHdwSSyh7r+++/l4+Pj/bu3auioqJKDRRSWpMmTTRs2DDNnTtXLVu21KJFi7RixYqalu8VCFV1qKhxtJQsmVLLf8YWAAAAKm/VqlUaO3asrrrqKkmOcLB37946rSEsLEzR0dFat26dLrzwQkmOXp5ff/21UsOJt2zZUuHh4WWWn3POOUpKSlKbNm2qVdeaNWuUkJAgSUpPT9eOHTvUoUOHcretzLE++ugjLViwQCtWrNC1116rRx55pNxnw0qS1WqV5PgcTnbzzTdr5MiRatasmVq3bq2+fftW9dS8EqGqLv05LLz1OJf/AQAA1FTbtm21YMECDRs2TCaTSQ899FClLrlztzvuuEOzZ89WmzZt1L59e7300ktKT0+v0XOuHn74YV1++eVKSEjQ3/72N5nNZm3atElbtmzRo48+etr3z5w5U5GRkYqOjtaDDz6oxo0bVzjQ2+mOdeDAAd1+++164okndP7552vu3Lm6/PLLNWTIEPXp06fM/po0aaKAgAB98803atasmfz9/Z0PUh48eLBCQ0P16KOPNojbe0rUjwcXNBC+8Y7nbgVl0VMFAABQU88++6wiIiJ03nnnadiwYRo8eLDOOeecOq9jypQpGjlypEaPHq3ExEQFBwdr8ODBNXrw8uDBg/Xll1/q22+/Va9evdSnTx8999xzzkcPnc7jjz+uu+66Sz169FBKSoq++OILZw9SVY5lGIbGjh2rc889VxMnTnRuf/vtt+vvf/+7csq5Wc/X11cvvviiXn/9dcXFxenKK690rjObzRo7dqyKi4s1evToanwy3slkuOuCUC+VlZWlsLAwZWZmuvWp0dXx03OrdN7k83XYN16xtlPfQAmcis1m09dff62hQ4fyoE3UCG0J7kA78j75+flKTk5Wy5Yta/SLfV2z2+3KyspSaGhovXlocXnsdrs6dOjgvEyuLq1YsUIXXXSR0tPTy72s0BuMHz9eR48edXk+mDvl5+drz549Sk5O1qBBg1x+LtVWNuDyvzoU2s7RUxVZlOIYZbQGXcIAAADwDn/88Ye+/fZb9evXTwUFBXr55ZeVnJxcrRHyGrLMzExt3rxZ8+fPr7VA5Sn1908A9VBkx2hJklU2FaZU/IA5AAAA1B9ms1nz5s1Tr1691LdvX23evFlLly6tcGCIM9WVV16pQYMG6bbbbtMll1zi6XLcip6qOhQZ56fjaqRIpen4lsOKjY30dEkAAACoofj4eK1atcrTZUiS+vfv77bh3t2toQyfXh56quqQySQd8XH0VmVsZwRAAAAAoCEgVNWxNGsTSVLubkIVAAAA0BAQqupYZmCUJKlwH8OqAwAAAA0BoaqOnQht5HhxmJ4qAAAAoCEgVNWxvPAISZLvUUIVAAAA0BAQquqYLTJckhSQQagCAAAAGgJCVR2zRzue3BxygnuqAAAA3K1///6aNGmSc75FixZ6/vnnT/kek8mkhQsX1vjY7tqPt5o+fbrOPvtst+1v7969MplM2rhxo9v26SmEqjrm08wRqhoV0lMFAABQYtiwYRoyZEi563744QeZTCb99ttvVd7vunXrdOutt9a0PBcVhYvDhw9XeA7uMm/ePJlMJplMJpnNZjVr1kzjxo3TkSNHavW4tSE+Pl6HDx9W586dJTmeY2UymZSRkeHZwqqBh//WMUt8oCQpxMiWPfuEzCFBHq4IAADA88aPH68RI0bo4MGDCg0NdVk3d+5c9ezZU127dq3yfqOiotxV4mnFxMTUyXFCQ0OVlJQku92uTZs2ady4cTp06JAWL15crf3ZbDZZLBY3V3l6Pj4+dfaZ1TZ6qupYYLSPTsgRrNK30VsFAADqgGFIJ054ZjKMSpV4+eWXKyoqSh988IHL8pycHH388ccaP368jh8/rpEjR6pp06YKDAxUly5dymx/spMv/9u5c6cuvPBC+fv7q2PHjlqyZEmZ90yZMkVnnXWWAgMD1apVKz300EOy2WySHD1FM2bM0KZNm5w9RvPmzZNU9vK/zZs36+KLL1ZAQIAiIyN16623Kicnx7l+7NixGj58uJ5++mnFxsYqMjJSEyZMcB6rIiaTSTExMYqLi9OQIUN05513aunSpcrLy5Mkvfnmm+rQoYP8/f3Vvn17vfLKK873llxy99FHH6lfv37y9/fX+++/r3nz5ik8PFwLFy5U27Zt5e/vr8GDB2v//v2nrOVUx7rpppvUtWtXFRQUSJIKCwvVvXt3jR492qWWjRs3au/evbroooskSRERETKZTBo7dqzeeecdRUZGOvdRYvjw4brxxhtPWVtdoqeqjvlapCPmWLW071b6tsOK7N3G0yUBAICGLjdXCg72zLFzcqSg01+Z4+vrqxtvvFHz58/XjBkznMs//vhjFRcXa+TIkcrJyVGPHj00ZcoUhYaG6quvvtKNN96o1q1b69xzzz3tMex2u66++mpFR0dr7dq1yszMdLn/qkRISIjmzZunuLg4bd68WbfccotCQkL0z3/+U9ddd522bNmib775RkuXLpUkhYWFldnHiRMnNHjwYCUmJmrdunU6cuSIbr75Zk2cONEZwiRp+fLlio2N1fLly7Vr1y5dd911Ovvss3XLLbec9nxKBAQEyG63q6ioSO+//74efvhhvfzyy+revbs2bNigW265RUFBQRozZozzPffff7+eeeYZde/eXf7+/lq8eLFyc3P12GOP6Z133pHVatU//vEPXX/99Vq1alW5xz3dsV588UV169ZN999/v5577jk9+OCDysjI0Msvv1xmX/Hx8frkk080YsQIJSUlKTQ0VAEBAbJarbrzzjv1+eef65prrpEkHTlyRF999ZW+/fbbSn9GtY1Q5QHp/rFqmbtb2bsYrAIAAKDEuHHj9PTTT2vlypW6+OKLJTku/RsxYoTCwsIUFhame++917n9HXfcocWLF+u///1vpULV0qVL9fvvv2vx4sWKi4uTJM2aNavMfVBTp051vm7RooXuvfdeffjhh/rnP/+pgIAABQcHy9fX95SXrs2fP1/5+fl65513FPRnqHz55Zc1bNgwPfHEE4qOjpbk6JV5+eWX5ePjo/bt2+uyyy7TsmXLKh2qdu7cqddee009e/ZUSEiIpk2bpmeeeUZXX321JKlly5batm2bXn/9dZdQNWnSJOc2JWw2m15++WX17t1bkvT222+rQ4cO+vnnn8v9fE93rODgYL333nvq16+fQkJC9Pzzz2v58uVlLu+UHJcCNmrkeJ5rkyZNFB4e7lx3ww03aO7cuc5Q9d577ykhIUH9+/ev1GdUFwhVHpATEiPlSgXJXP4HAADqQGCgo8fIU8eupPbt2+vcc8/V3LlzdfHFF2vXrl364YcfNHPmTElScXGxZs2apf/+9786ePCgCgsLVVBQoMBKHmP79u2Kj493BipJSkxMLLPdRx99pBdffFG7d+9WTk6OioqKyg0CpztWt27dnIFKkvr27Su73a6kpCRnqOrUqZN8fHyc28TGxmrz5s2n3HdmZqaCg4Nlt9uVn5+v888/X2+++aZOnDih3bt3a/z48S6hrKioqExvWs+ePcvs19fXV7169XLOt2/fXuHh4dq+fXuZUFXZYyUmJuree+/VI488oilTpuj8888/5bmV55ZbblGvXr108OBBNW3aVPPmzdPYsWNlMpmqvK/aQqjygPyIGClVsh8iVAEAgDpgMlXqEjxvcOONN2rKlCl65ZVXNHfuXLVu3Vr9+vWTJD311FN64YUX9Pzzz6tLly4KCgrSpEmTVFhY6Lbjr169WqNGjdKMGTM0ePBghYWF6cMPP9QzzzzjtmOUdvIAESaTSXa7/ZTvCQkJ0a+//iqz2azY2FgFBARIklJTUyVJb7zxhrO3qUTp4CbJJexVR8m9Yac7lt1u16pVq+Tj46Ndu3ZV61jdu3dXt27d9M4772jQoEHaunWrvvrqq+oXXwsIVR5QHBUj/S75pBKqAAAAShs+fLgeeOABzZ8/X++8845uv/12Z4/EqlWrdOWVV+rvf/+7JMcv7Dt27FDHjh0rte8OHTpo//79Onz4sGJjYyVJa9ascdnmp59+UvPmzfXggw86l/3xxx8u21itVhUXF5/2WPPmzdOJEyecAWbVqlUym81q165dpeqtiNlsVps2Ze/Lj46OVlxcnPbs2aNRo0ZVeb9FRUX65ZdfnL1SSUlJysjIUIcOHap9rKeeekq///67Vq5cqcGDB2vu3LkaN25cudtarVZJKvezvfnmm/X888/r4MGDGjhwoOLj46t8frWJ0f88wNzU8U3sl06oAgAAKC04OFjXXnutHnjgAR0+fFhjx451rmvbtq2WLFmin376Sdu3b9f//d//OXtnKmPgwIE666yzNGbMGG3atEk//PCDS3gqOca+ffv04Ycfavfu3XrxxRf16aefumzTokULJScna+PGjTp27FiZkekkadSoUfL399eYMWO0ZcsWLV++XHfccYduvPFG56V/tWHGjBmaPXu2XnzxRe3YsUObN2/W3Llz9eyzz572vRaLRXfccYfWrl2r9evXa+zYserTp0+F96ud7lgbNmzQww8/rDfffFN9+/bVs88+q7vuukt79uwpd3/NmzeXyWTSl19+qaNHj7qMlHjDDTfowIEDeuONN3TTTTdV45OpXYQqD7A2d9zUGJzNQBUAAAAnu+mmm5Senq7Bgwe73P80depUnXPOORo8eLD69++vmJgYDR8+vNL7NZvN+vTTT5WXl6dzzz1XN998sx577DGXba644grdfffdmjhxos4++2z99NNPeuihh1y2GTFihC699FJddNFF5Q4DL0mBgYFavHix0tLS1KtXL/3tb3/TgAEDyh35zp1uvvlmvfnmm5o7d666dOmifv36ad68eWrZsuVp3xsYGKgpU6bohhtuUN++fRUcHKyPPvqoWsfKz8/X3//+d40dO1bDhg2TJN1666266KKLdOONN5bbG9W0aVPNmDFD999/v6KjozVx4kTnurCwMI0YMULBwcFV+jevKybDqOTDA+qprKwshYWFKTMzs8o3GLqbzWbT119/rbijzdTrlp46bo5SZHH9e/o1PK+kLQ0dOtQjD+tDw0FbgjvQjrxPfn6+kpOT1bJlS/n7+3u6nEqz2+3KyspSaGiozGb+9l+X5s2bp0mTJikjI8PTpVRowIAB6tSpk1588cVTbpefn689e/YoOTlZgwYNcvm5VFvZgHuqPCC8g6OnKtJ+VEahTSYr/wMCAAAAypOenq4VK1ZoxYoVLg8X9iaEKg+I6tBYNvnKoiLl7E5VSIdmni4JAAAA8Erdu3dXenq6nnjiiRoP8lFbCFUeEBRi1hFTtJoaB5W29TChCgAAAB41duxYl0FBvMnevXs9XcJpcbGqh6RZHSMAZu1gsAoAAACgPiNUeUh2kOO+qrw9DKsOAABqRwMfjwwolyfaPaHKQ/LCHT1VRfsJVQAAwL18fHwkSYWFhR6uBKh7ubm5ksp/iHBt4Z4qDymKipX2SOZUQhUAAHAvX19fBQYG6ujRo7JYLPVmeHK73a7CwkLl5+fXm5rhPQzDUG5uro4cOaLQ0NA67bEiVHmIKc7RU2U5TqgCAADuZTKZFBsbq+TkZP3xxx+eLqfSDMNQXl6eAgICZDKZPF0O6qnw8HBFRkbW6TEJVR5iSXCEqsAsBqoAAADuZ7Va1bZt23p1CaDNZtP333+vCy+8kAdJo1osFot8fHxks9nq9LiEKg8Jau0YqCI8l54qAABQO8xms/z9/T1dRqX5+PioqKhI/v7+hCrUK1ys6iHhHRw9VZFFKRIj8wAAAAD1FqHKQxp3dvRUWWVTwaHjHq4GAAAAQHURqjwkItqqY3LcQHd8C5cAAgAAAPUVocpDTCbpmMVxCWBmEoNVAAAAAPUVocqDsgIclwCe2EVPFQAAAFBfEao86ESYo6fKto9QBQAAANRXhCoPskU6QpUOE6oAAACA+opQ5UFGjCNU+R4lVAEAAAD1FaHKg3zjHaHKP5OBKgAAAID6ilDlQQEtHQNVhJ6gpwoAAACorwhVHhTaztFTFVlIqAIAAADqK0KVB0V2doSqYCNHxZk5Hq4GAAAAQHUQqjwoqlWIchQkSUrfRm8VAAAAUB8RqjzI11c64uPorUrfzmAVAAAAQH1EqPKwDH/HYBU5O+mpAgAAAOojQpWH5YQ4eqoK9hKqAAAAgPqIUOVhBRGOUGU/RKgCAAAA6iNClYfZox2hyucI91QBAAAA9ZFHQ9X06dNlMplcpvbt2zvX5+fna8KECYqMjFRwcLBGjBih1NRUD1bsfuamjlDll0ZPFQAAAFAfebynqlOnTjp8+LBz+vHHH53r7r77bn3xxRf6+OOPtXLlSh06dEhXX321B6t1P/8WjoEqgnMIVQAAAEB95OvxAnx9FRMTU2Z5Zmam3nrrLc2fP18XX3yxJGnu3Lnq0KGD1qxZoz59+tR1qbUiuK2jpyoin1AFAAAA1EceD1U7d+5UXFyc/P39lZiYqNmzZyshIUHr16+XzWbTwIEDndu2b99eCQkJWr16dYWhqqCgQAUFBc75rKwsSZLNZpPNZqvdkzmNkuOXriPkrMaSpEj7MRXmnJDJz+qR2lC/lNeWgOqgLcEdaEdwF9oS3KWitlRbbcujoap3796aN2+e2rVrp8OHD2vGjBm64IILtGXLFqWkpMhqtSo8PNzlPdHR0UpJqXhQh9mzZ2vGjBllln/77bcKDAx09ylUy5IlS5yvC/JMai5fWVSkr+b+T+aEcM8VhnqndFsCaoK2BHegHcFdaEtwl5PbUm5ubq0cx6OhasiQIc7XXbt2Ve/evdW8eXP997//VUBAQLX2+cADD2jy5MnO+aysLMXHx2vQoEEKDQ2tcc01YbPZtGTJEl1yySWyWCzO5ammGDUzDqhL47PUfGhPD1aI+qKitgRUFW0J7kA7grvQluAuFbWlkqvY3M3jl/+VFh4errPOOku7du3SJZdcosLCQmVkZLj0VqWmppZ7D1YJPz8/+fn5lVlusVi85pvz5FrS/WLULP+Acncf9ZoaUT94U7tG/UZbgjvQjuAutCW4y8ltqbbalcdH/ystJydHu3fvVmxsrHr06CGLxaJly5Y51yclJWnfvn1KTEz0YJXulxXkGKwiP5nBKgAAAID6xqM9Vffee6+GDRum5s2b69ChQ5o2bZp8fHw0cuRIhYWFafz48Zo8ebIaNWqk0NBQ3XHHHUpMTGwwI/+VyA+PlY5LRfsJVQAAAEB949FQdeDAAY0cOVLHjx9XVFSUzj//fK1Zs0ZRUVGSpOeee05ms1kjRoxQQUGBBg8erFdeecWTJdeKoqhYabdkOlLxABwAAAAAvJNHQ9WHH354yvX+/v6aM2eO5syZU0cVeYYp1nGPmPU4PVUAAABAfeNV91SdqazNHfdUBWUSqgAAAID6hlDlBYLaOEJVWB6hCgAAAKhvCFVeIKy9I1Q1LkqR7HYPVwMAAACgKghVXiCqc7QkyaIi5R9K83A1AAAAAKqCUOUFwptYdUyRkqTjW7gEEAAAAKhPCFVewGSSjlkclwBm/k6oAgAAAOoTQpWXyAx0hKrc3YQqAAAAoD4hVHmJ3FBHqLLtI1QBAAAA9QmhykvYGjtClZGS4uFKAAAAAFQFocpbxMRIkixH6akCAAAA6hNClZfwjXf0VAVkEKoAAACA+oRQ5SUCWjlCVWguoQoAAACoTwhVXiK0nSNURRYSqgAAAID6hFDlJSI7O0JVkHFCxZk5Hq4GAAAAQGURqrxEVMtg5ShIkpS2ld4qAAAAoL4gVHkJHx/piI+jtyp9G6EKAAAAqC8IVV4kw98Rqk7sIlQBAAAA9QWhyovkhDhCVcFeQhUAAABQXxCqvEhBI0eoKj6U4uFKAAAAAFQWocqLGNExkiTfI/RUAQAAAPUFocqLmJs6eqr80glVAAAAQH1BqPIifi0coSokm1AFAAAA1BeEKi8ScpYjVEUUEKoAAACA+oJQ5UUiOjpCVSP7cRkFhR6uBgAAAEBlEKq8SJP2jWSTryQpa2eqh6sBAAAAUBmEKi8SEGRWqskxAmDaVi4BBAAAAOoDQpWXSfNzXAKYvYNQBQAAANQHhCovkx3sCFX5yYQqAAAAoD4gVHmZ/HBHqCo6kOLhSgAAAABUBqHKyxRHOe6pMqXSUwUAAADUB4QqL2OKc/RUWY8TqgAAAID6gFDlZSwJjlAVlEWoAgAAAOoDQpWXCWrjCFVhedxTBQAAANQHhCovE97BEaoaF6VIdruHqwEAAABwOoQqLxPVqYkkyaIi5R047uFqAAAAAJwOocrLhEVZdVSNJUnHt3BfFQAAAODtCFVexmSSjlsclwBmJRGqAAAAAG9HqPJCmYGOUHViD4NVAAAAAN6OUOWFcsMcocr2Bz1VAAAAgLcjVHkhW+MYx4sUQhUAAADg7QhV3ijG0VNlOUqoAgAAALwdocoL+cY7QlVAJqEKAAAA8HaEKi8U0MoRqkJPMFAFAAAA4O0IVV4orJ3jnqpIGz1VAAAAgLcjVHmhyM6Onqog44SK0rM9XA0AAACAUyFUeaHGLYKVrWBJUtpWeqsAAAAAb0ao8kI+PtIRH0dvVcZ2QhUAAADgzQhVXiojwBGqcnYxWAUAAADgzQhVXupEiGOwioI/6KkCAAAAvBmhyksVNHL0VBkHCVUAAACANyNUeSkj2hGqfI4QqgAAAABvRqjyUj7NHKHKP51QBQAAAHgzQpWX8mvhCFXBOQxUAQAAAHgzQpWXCmnrGKgiooCeKgAAAMCbeU2oevzxx2UymTRp0iTnsvz8fE2YMEGRkZEKDg7WiBEjlJqa6rki61BER0dPVSP7cRkFhR6uBgAAAEBFvCJUrVu3Tq+//rq6du3qsvzuu+/WF198oY8//lgrV67UoUOHdPXVV3uoyrrVpEOkCmWRJGXu4BJAAAAAwFt5PFTl5ORo1KhReuONNxQREeFcnpmZqbfeekvPPvusLr74YvXo0UNz587VTz/9pDVr1niw4rrhH2DSEZPjEsD0rVwCCAAAAHgrX08XMGHCBF122WUaOHCgHn30Uefy9evXy2azaeDAgc5l7du3V0JCglavXq0+ffqUu7+CggIVFBQ457OysiRJNptNNputls6ickqOX9k6jvvFqln+fqX/flDNPFw7vEtV2xJQEdoS3IF2BHehLcFdKmpLtdW2PBqqPvzwQ/36669at25dmXUpKSmyWq0KDw93WR4dHa2UlIovh5s9e7ZmzJhRZvm3336rwMDAGtfsDkuWLKnUdqGWRlK+tPuHddr/tU8tV4X6qLJtCTgd2hLcgXYEd6EtwV1Obku5ubm1chyPhar9+/frrrvu0pIlS+Tv7++2/T7wwAOaPHmycz4rK0vx8fEaNGiQQkND3Xac6rDZbFqyZIkuueQSWSyW026/MvprKVuKla96Dx1aBxWivqhqWwIqQluCO9CO4C60JbhLRW2p5Co2d/NYqFq/fr2OHDmic845x7msuLhY33//vV5++WUtXrxYhYWFysjIcOmtSk1NVUxMTIX79fPzk5+fX5nlFovFa745K1uLvUmctEvyOZrqNbXDu3hTu0b9RluCO9CO4C60JbjLyW2pttqVx0LVgAEDtHnzZpdl48aNU/v27TVlyhTFx8fLYrFo2bJlGjFihCQpKSlJ+/btU2JioidKrnOmOMew6n7HGagCAAAA8FYeC1UhISHq3Lmzy7KgoCBFRkY6l48fP16TJ09Wo0aNFBoaqjvuuEOJiYkVDlLR0FibO0JVYDZDqgMAAADeyuOj/53Kc889J7PZrBEjRqigoECDBw/WK6+84umy6kxQa8dljuF59FQBAAAA3sqrQtWKFStc5v39/TVnzhzNmTPHMwV5WHgHR09VZFGqZLdLZo8/VgwAAADASfgt3YtFdY6WXSZZVKTcfcc8XQ4AAACAchCqvFhopEXH1FiSlLaVSwABAAAAb0So8mImk3Tc6rgEMDOJwSoAAAAAb0So8nJZgY7BKnJ301MFAAAAeCNClZfLDXP0VNn2EaoAAAAAb0So8nK2SEeo0mFCFQAAAOCNCFXeLtYRqizHCVUAAACANyJUeTlLgiNUBWQyUAUAAADgjQhVXi6gpWOgitAT9FQBAAAA3ohQ5eVC2zl6qhoXHpYMw8PVAAAAADgZocrLRXZ2hKpA5aooPdvD1QAAAAA4WbVC1dy5c5Wbm+vuWlCOqBZBylKIJOn4Vu6rAgAAALxNtULV/fffr5iYGI0fP14//fSTu2tCKWazdNTH0VuVsZ37qgAAAABvU61QdfDgQb399ts6duyY+vfvr/bt2+uJJ55QSgo9KbUhI8AxWMWJXYQqAAAAwNtUK1T5+vrqqquu0meffab9+/frlltu0fvvv6+EhARdccUV+uyzz2S3291d6xnrRIijp6pgL6EKAAAA8DY1HqgiOjpa559/vhITE2U2m7V582aNGTNGrVu31ooVK9xQIgoiHaHKOESoAgAAALxNtUNVamqqnn76aXXq1En9+/dXVlaWvvzySyUnJ+vgwYO69tprNWbMGHfWesYyoh2hyucol1cCAAAA3qZaoWrYsGGKj4/XvHnzdMstt+jgwYP64IMPNHDgQElSUFCQ7rnnHu3fv9+txZ6pfJo67qnyS6enCgAAAPA2vtV5U5MmTbRy5UolJiZWuE1UVJSSk5OrXRj+4tfC0VMVkkOoAgAAALxNtXqq+vXrp3POOafM8sLCQr3zzjuSJJPJpObNm9esOkiSQs5yhKpGBYQqAAAAwNtUK1SNGzdOmZmZZZZnZ2dr3LhxNS4Krhp1coSqCHuajPwCD1cDAAAAoLRqhSrDMGQymcosP3DggMLCwmpcFFw1ad9IhbJIkjJ3pHq4GgAAAAClVemequ7du8tkMslkMmnAgAHy9f3r7cXFxUpOTtall17q9iLPdH7+Jh0wx6iZfb/Sth5WeNcET5cEAAAA4E9VClXDhw+XJG3cuFGDBw9WcHCwc53ValWLFi00YsQItxYIhzRrrJrl71f2Du6rAgAAALxJlULVtGnTJEktWrTQddddJ39//1opCmVlB8dK+VJ+MqEKAAAA8CbVGlKdh/rWvfyIWOmYVHyAUAUAAAB4k0qHqkaNGmnHjh1q3LixIiIiyh2ookRaWppbisNfipvESjslU2qKp0sBAAAAUEqlQ9Vzzz2nkJAQ5+tThSq4nzkuRpJkTaOnCgAAAPAmlQ5VpS/5Gzt2bG3UglOwNnc8qyo4i1AFAAAAeJNqPadq3rx55S4vKirSAw88UJN6UIGgNo5QFZ5HqAIAAAC8SbVC1Z133qlrrrlG6enpzmVJSUnq3bu3PvjgA7cVh7+Ed3CEqsjiVKm42MPVAAAAAChRrVC1YcMGHThwQF26dNGSJUs0Z84cnXPOOWrfvr02bdrk7hohKapTE9llkq+KdWLfcU+XAwAAAOBP1RpSvXXr1lq1apUmTZqkSy+9VD4+Pnr77bc1cuRId9eHP4U0suiYGitKR5W29bCCWjbxdEkAAAAAVM2eKkn66quv9OGHHyoxMVHh4eF66623dOjQIXfWhlJMJumY1XEJYGYS91UBAAAA3qJaoer//u//dM0112jKlCn64Ycf9Ntvv8lqtapLly7673//6+4a8aesQEeoyttNqAIAAAC8RbVC1apVq7R27Vrdc889MplMiomJ0ddff62ZM2fqpptucneN+FNumCNU2fYRqgAAAABvUa17qtavXy8/P78yyydMmKCBAwfWuCiUz9Y4VvpDUkqKp0sBAAAA8Kdq9VT5+flp9+7dmjp1qkaOHKkjR45IkhYtWqSioiK3Foi/mGJjJEmWY/RUAQAAAN6iWqFq5cqV6tKli9auXasFCxYoJydHkrRp0yZNmzbNrQXiL74Jjsv/AjMJVQAAAIC3qFaouv/++/Xoo49qyZIlslqtzuUXX3yx1qxZ47bi4CqgpSNUheYSqgAAAABvUa1QtXnzZl111VVlljdp0kTHjh2rcVEoX1h7R6iKLDwsGYaHqwEAAAAgVTNUhYeH6/Dhsr0lGzZsUNOmTWtcFMoX2clxT1Wg8mRLy/ZwNQAAAACkaoaq66+/XlOmTFFKSopMJpPsdrtWrVqle++9V6NHj3Z3jfhT4+ZBylKIJCltK5cAAgAAAN6gWqFq1qxZat++veLj45WTk6OOHTvqwgsv1HnnnaepU6e6u0b8yWyWjvo6LgHM2E6oAgAAALxBtZ5TZbVa9cYbb+ihhx7Sli1blJOTo+7du6tt27burg8nyfCPlXJ2KGcnoQoAAADwBtUKVSUSEhKUkJDgrlpQCTmhsVKOVPgHoQoAAADwBpUOVZMnT670Tp999tlqFYPTK2wUKx2S7IdSPF0KAAAAAFUhVG3YsKFS25lMpmoXg9MzomOkLZLvUXqqAAAAAG9Q6VC1fPny2qwDleQT7xiowj+dUAUAAAB4g2qN/lfa/v37tX//fnfUgkrwb+EIVSEnCFUAAACAN6hWqCoqKtJDDz2ksLAwtWjRQi1atFBYWJimTp0qm83m7hpRSshZjlDVqIBQBQAAAHiDao3+d8cdd2jBggV68sknlZiYKElavXq1pk+fruPHj+vVV191a5H4S6NOjlAVbk+XkV8gk7+fhysCAAAAzmzVClXz58/Xhx9+qCFDhjiXde3aVfHx8Ro5ciShqhY1aRehAlnlp0Klb09Ro+7NPV0SAAAAcEar1uV/fn5+atGiRZnlLVu2lNVqrfR+Xn31VXXt2lWhoaEKDQ1VYmKiFi1a5Fyfn5+vCRMmKDIyUsHBwRoxYoRSU1OrU3KDYfUz6Yg5RpKUvo1LAAEAAABPq1aomjhxoh555BEVFBQ4lxUUFOixxx7TxIkTK72fZs2a6fHHH9f69ev1yy+/6OKLL9aVV16prVu3SpLuvvtuffHFF/r444+1cuVKHTp0SFdffXV1Sm5Q0vwclwBm7yBUAQAAAJ5Wrcv/NmzYoGXLlqlZs2bq1q2bJGnTpk0qLCzUgAEDXILPggULKtzPsGHDXOYfe+wxvfrqq1qzZo2aNWumt956S/Pnz9fFF18sSZo7d646dOigNWvWqE+fPtUpvUHIDo6V8qT8vTwAGAAAAPC0aoWq8PBwjRgxwmVZfHx8jQopLi7Wxx9/rBMnTigxMVHr16+XzWbTwIEDndu0b99eCQkJWr16dYWhqqCgwKUHLSsrS5Jks9k8PjJhyfFrWkd+eLR0VCraf9Dj5wTPcFdbAmhLcAfaEdyFtgR3qagt1VbbqnKoMgxDM2bMUFRUlAICAmpcwObNm5WYmKj8/HwFBwfr008/VceOHbVx40ZZrVaFh4e7bB8dHa2UlIp7aGbPnq0ZM2aUWf7tt98qMDCwxvW6w5IlS2r0/lxfx4h/J3bt1Ndff+2OklBP1bQtASVoS3AH2hHchbYEdzm5LeXm5tbKcaoVqtq0aaOtW7eqbdu2NS6gXbt22rhxozIzM/W///1PY8aM0cqVK6u9vwceeECTJ092zmdlZSk+Pl6DBg1SaGhojeutCZvNpiVLluiSSy6RxWKp9n6Wv5cibZcaF+Wo69ChbqwQ9YW72hJAW4I70I7gLrQluEtFbankKjZ3q3KoMpvNatu2rY4fP+6WUGW1WtWmTRtJUo8ePbRu3Tq98MILuu6661RYWKiMjAyX3qrU1FTFxMRUuD8/Pz/5+ZV9dpPFYvGab86a1uLfsqkkKTg7xWvOCZ7hTe0a9RttCe5AO4K70JbgLie3pdpqV9Ua/e/xxx/Xfffdpy1btri7HtntdhUUFKhHjx6yWCxatmyZc11SUpL27dvnfODwmSqozZ8PAM5joAoAAADA06o1UMXo0aOVm5urbt26yWq1lrm3Ki0trVL7eeCBBzRkyBAlJCQoOztb8+fP14oVK7R48WKFhYVp/Pjxmjx5sho1aqTQ0FDdcccdSkxMPKNH/pOkiA6OnrrI4lSpuFjy8fFwRQAAAMCZq1qh6vnnn3fLwY8cOaLRo0fr8OHDCgsLU9euXbV48WJdcsklkqTnnntOZrNZI0aMUEFBgQYPHqxXXnnFLceuz6I6R8suk3xVrBN/HFNQq2hPlwQAAACcsaoVqsaMGeOWg7/11lunXO/v7685c+Zozpw5bjleQxES4asjilITHdHxLYcJVQAAAIAHVeueKknavXu3pk6dqpEjR+rIkSOSpEWLFmnr1q1uKw4VO2513FeVlXTYw5UAAAAAZ7ZqhaqVK1eqS5cuWrt2rRYsWKCcnBxJ0qZNmzRt2jS3FojyZQY5QlXuHgarAAAAADypWqHq/vvv16OPPqolS5bIarU6l1988cVas2aN24pDxfLCHINVFO2npwoAAADwpGqFqs2bN+uqq64qs7xJkyY6duxYjYvC6RU1dvRU6TChCgAAAPCkaoWq8PBwHS7nl/kNGzaoadOmNS4KlRDrCFWW44QqAAAAwJOqFaquv/56TZkyRSkpKTKZTLLb7Vq1apXuvfdejR492t01ohyWBEeoCswkVAEAAACeVK1QNWvWLHXo0EEJCQnKyclRx44ddeGFF+q8887T1KlT3V0jyhHQyhGqQnMZqAIAAADwpCo9p8put+upp57S559/rsLCQt14440aMWKEcnJy1L17d7Vt27a26sRJwto5BqqILDwsGYZkMnm4IgAAAODMVKVQ9dhjj2n69OkaOHCgAgICNH/+fBmGof/85z+1VR8qENn5z8v/lKfCY1myRoV5uCIAAADgzFSly//eeecdvfLKK1q8eLEWLlyoL774Qu+//77sdntt1YcKRMYHKlOhkqTjW7ivCgAAAPCUKoWqffv2aejQoc75gQMHymQy6dChQ24vDKdmNktHfR29VZm/E6oAAAAAT6lSqCoqKpK/v7/LMovFIpvN5taiUDkZAY5QdWI3g1UAAAAAnlKle6oMw9DYsWPl5+fnXJafn6/bbrtNQUFBzmULFixwX4WoUG5IjJQtFfxBTxUAAADgKVUKVWPGjCmz7O9//7vbikHVFEbGSock4yChCgAAAPCUKoWquXPn1lYdqAZ7dKy0WfI9SqgCAAAAPKVaD/+Fd/CNd9xT5Z9BqAIAAAA8hVBVj/m3dISqkBwGqgAAAAA8hVBVj4W0jZEkNSqgpwoAAADwFEJVPdaok6OnKtxIlz0338PVAAAAAGcmQlU9FnVWhPLlGN4+fTuXAAIAAACeQKiqx6x+Jh0xOy4BzNjOJYAAAACAJxCq6rl0P8clgNk76akCAAAAPIFQVc/lBDt6qvKT6akCAAAAPIFQVc/lRzh6qooPEKoAAAAATyBU1XPFTRyhypxKqAIAAAA8gVBVz5mbOkKVXzqhCgAAAPAEQlU959fCEaqCshmoAgAAAPAEQlU9F9TaMVBFeB49VQAAAIAnEKrqufAOjp6qxsWpUnGxh6sBAAAAzjyEqnquSecmssskH9mVk3zU0+UAAAAAZxxCVT0XHO6ro6YmkqS0rVwCCAAAANQ1QlUDcNzquAQwM4nBKgAAAIC6RqhqALIDHYNV5O2hpwoAAACoa4SqBiA3zNFTZdtPqAIAAADqGqGqAbBFOUKV6TChCgAAAKhrhKoGwBTrCFW+x7mnCgAAAKhrhKoGwJLguKcqKJOeKgAAAKCuEaoagMDWjp6qsFxCFQAAAFDXCFUNQGg7R6iKtB2WDMPD1QAAAABnFkJVA9C4syNUBShfhUczPVwNAAAAcGYhVDUAkc0ClKEwSdKxLQxWAQAAANQlQlUDYDJJx3wdg1Vk/s59VQAAAEBdIlQ1EJkBjksAT+wiVAEAAAB1iVDVQOSEOkJV4T5CFQAAAFCXCFUNRGGkI1QZBwlVAAAAQF0iVDUQRrQjVPkcY6AKAAAAoC4RqhoI32aOgSoCMuipAgAAAOoSoaqBCGjl6KkKySFUAQAAAHWJUNVABLd1hKpGBYQqAAAAoC4RqhqIRp0coSrcyJD9RJ6HqwEAAADOHISqBqLJWeHKl58kKW17qoerAQAAAM4chKoGwmI16YjZMVhFxnYuAQQAAADqCqGqAUn3d1wCmL2DUAUAAADUFUJVA5IT7AhVBXsJVQAAAEBd8Wiomj17tnr16qWQkBA1adJEw4cPV1JSkss2+fn5mjBhgiIjIxUcHKwRI0YoNZV7hsqTH+EIVcUHCFUAAABAXfFoqFq5cqUmTJigNWvWaMmSJbLZbBo0aJBOnDjh3Obuu+/WF198oY8//lgrV67UoUOHdPXVV3uwau9V3MQRqsxHUjxcCQAAAHDm8PXkwb/55huX+Xnz5qlJkyZav369LrzwQmVmZuqtt97S/PnzdfHFF0uS5s6dqw4dOmjNmjXq06ePJ8r2Wj5NHQNV+KXRUwUAAADUFY+GqpNlZmZKkho1aiRJWr9+vWw2mwYOHOjcpn379kpISNDq1avLDVUFBQUqKChwzmdlZUmSbDabbDZbbZZ/WiXHr606fOObSJKCsg97/FxRu2q7LeHMQVuCO9CO4C60JbhLRW2p1n4Pr5W9VoPdbtekSZPUt29fde7cWZKUkpIiq9Wq8PBwl22jo6OVklL+JW6zZ8/WjBkzyiz/9ttvFRgY6Pa6q2PJkiW1st/juUfVT1J43iF9/fXXtXIMeJfaaks489CW4A60I7gLbQnucnJbys3NrZXjeE2omjBhgrZs2aIff/yxRvt54IEHNHnyZOd8VlaW4uPjNWjQIIWGhta0zBqx2WxasmSJLrnkElksFrfvf2/EYWmO1Nh+VEMGDZbJ18ftx4B3qO22hDMHbQnuQDuCu9CW4C4VtaWSq9jczStC1cSJE/Xll1/q+++/V7NmzZzLY2JiVFhYqIyMDJfeqtTUVMXExJS7Lz8/P/n5+ZVZbrFYvOabs7ZqiTu7qYpllo/syj2QoZC25X9GaDi8qV2jfqMtwR1oR3AX2hLc5eS2VFvtyqOj/xmGoYkTJ+rTTz/Vd999p5YtW7qs79GjhywWi5YtW+ZclpSUpH379ikxMbGuy/V6QaE+OmaKkiQd38JgFQAAAEBd8GhP1YQJEzR//nx99tlnCgkJcd4nFRYWpoCAAIWFhWn8+PGaPHmyGjVqpNDQUN1xxx1KTExk5L8KHLfGKrogVdk7Dkvq7ulyAAAAgAbPo6Hq1VdflST179/fZfncuXM1duxYSdJzzz0ns9msESNGqKCgQIMHD9Yrr7xSx5XWH1lBsVLBRuXtoacKAAAAqAseDVWGYZx2G39/f82ZM0dz5sypg4rqv7ywWClNsu0nVAEAAAB1waP3VMH9bFGxjhcVDDkPAAAAwL0IVQ2MKdYx4p/1GD1VAAAAQF0gVDUw1gRHT1VgFqEKAAAAqAuEqgYmsLUjVIXlEqoAAACAukCoamBC2zlCVWPbYakSA4EAAAAAqBlCVQPTuLPjnip/FajgSKaHqwEAAAAaPkJVA9OoaYAyFCZJOraFSwABAACA2kaoamBMJumYr+MSwKzfCVUAAABAbSNUNUAZgY5QdWIXoQoAAACobYSqBuhEqCNUFe7jAcAAAABAbSNUNUC2Ro7BKoxD9FQBAAAAtY1Q1QAZMY6eKt+jhCoAAACgthGqGiCfeEeo8s8gVAEAAAC1jVDVAAW0dISq0BOEKgAAAKC2EaoaoNB2jlAVUcBAFQAAAEBtI1Q1QBEdHANVhBsZKs7J83A1AAAAQMNGqGqAmpwVrnz5SZLSttFbBQAAANQmQlUD5Gsx6YjZcQlgxnbuqwIAAABqE6GqgUr3d4SqnJ2EKgAAAKA2EaoaqOwQR6jK38vlfwAAAEBtIlQ1UAURjsEq7AfpqQIAAABqE6GqgbI3cfRUmVMJVQAAAEBtIlQ1UOamjlDll0aoAgAAAGoToaqB8mvhCFXBOYQqAAAAoDYRqhqo4LaOUBWez0AVAAAAQG0iVDVQ4e0dA1VEFh+RUVTs4WoAAACAhotQ1UA16dxExTLLR3Zl7z7i6XIAAACABotQ1UAFhvjoqKmJJOn4Fu6rAgAAAGoLoaoBO2513FeVvYNQBQAAANQWQlUDlh3kCFV5yQxWAQAAANQWQlUDlhfuGKyiaD89VQAAAEBtIVQ1YEWNHT1VphRCFQAAAFBbCFUNmCnOEaosxwlVAAAAQG0hVDVglgRHqArKJFQBAAAAtYVQ1YAFtnGEqtA8BqoAAAAAaguhqgELb+cYqKKx7bBkGB6uBgAAAGiYCFUNWGRnR0+VvwqUn5Lh2WIAAACABopQ1YBFxPorXeGSpONbuK8KAAAAqA2EqgbMZJKOWRy9VZm/E6oAAACA2kCoauAyAxyh6sRuBqsAAAAAagOhqoHLDXUMVhGyZomUlubhagAAAICGh1DVwGXFtZMktV/7tmxRsTo+8FoZX30tFRV5uDIAAACgYSBUNXDxL/5Ts6Of10Z1k8VeqMhlH8t0+WU60ThBBZOmSNu3e7pEAAAAoF4jVDVw3Xr76/7Ddynnh42aetkGveRzl46qsYIyD8vvhSeljh2V06m3jFdfk9LTPV0uAAAAUO8Qqs4AJpN0/vnSo1+erVFHntdHzxzUnc0W6DNdoSL5KHjbzzL943YVNYlV4YiR0uLFUnGxp8sGAAAA6gVC1RmmUSNp4mSrXth3laJWfabJ1x7UFN9ntFmd5VtUIOuCD6VLL1VBbHMZD/xLSkrydMkAAACAVyNUnaFMJum886QXP4rW/Ucma8ULv+m61r/oJU3UcTWS39GDMj0+W2rfXkXnnif9+99SZqanywYAAAC8DqEKioiQ7rjTpA939lDPn17S/Tce0g3Wj/WlLlOxzPJdt1r6v/9TcZMYGaNGSUuWcHkgAAAA8CdCFZxMJikxUXrjHT+9kvo3/fHyl7qk/QHdpye1TR3kU5gv0/z50qBBKm7eUpo6Vdq509NlAwAAAB5FqEK5wsOlCROkZdti9bc19+mZcVt1od9avaLbla5w+RzcLz32mHTWWTIuuEB66y0pK8vTZQMAAAB1jlCFUzKZpN69pbf+Y9IXqedKc17RoM6Hda0+0iJdqmKZZfrxR+nmm2XExkqjR0vffSfZ7Z4uHQAAAKgThCpUWliY9I9/SD//5q97116rT8YvUjv/fZqix/W72smUmyu9+640YICMli2le+6RvvySAS4AAADQoBGqUGUmk3TuudKbb0q/pjZVy1enaGS37eqj1XpdtypToTLt2yc9+6w0bJiMRo0cb5gyRfrmGyk729OnAAAAALgNoQo1Ehoq3Xab9OsGk176uY9+ufl1tQ5M0bX6SG/oZu1Sa5nsdmndOunJJ6UhQ2RERMjeJ1H6178cIwnm5nr6NAAAAIBq82io+v777zVs2DDFxcXJZDJp4cKFLusNw9DDDz+s2NhYBQQEaODAgdrJaHNeyWSSevWS3nhD2nM4QIPfvFY/jn5Dl7fbpXjt02i9rf9onJLVQqbiYpnXrpFmz3aMJBgarvxe58uY+pDjfqy8PE+fDgAAAFBpHg1VJ06cULdu3TRnzpxy1z/55JN68cUX9dprr2nt2rUKCgrS4MGDlZ+fX8eVoipCQ6Xx46W335Z+/13adDxeNywarf3T/6N/XJqsbqHJGqf/6B3dqP1qJp9im/x/WSXTY49KAwaoKCRcGWf3V+G/pksrV0oFBZ4+JQAAAKBCvp48+JAhQzRkyJBy1xmGoeeff15Tp07VlVdeKUl65513FB0drYULF+r666+vy1JRA40aSZde6pgkyTBaaOfOcVqzZpweX2PowPd7FL1tufoZy3WRliuu+LDCN62UNq2UZs9QoY+/jrU9T76XXKTG11wkc+9ektXq2ZMCAAAA/uTRUHUqycnJSklJ0cCBA53LwsLC1Lt3b61evbrCUFVQUKCCUj0bWX8+O8lms8lms9Vu0adRcnxP1+ENWrZ0TCNHSlKCcnPHaMOGsXp3jbT/u10KXrdCZ2esUH+tUExxquJ+/076/TvpJSnPHKgDCeep8Pz+anJtP4UPOEeyWDx9SnWKtgR3oS3BHWhHcBfaEtylorZUW23LZBiGUSt7riKTyaRPP/1Uw4cPlyT99NNP6tu3rw4dOqTY2Fjndtdee61MJpM++uijcvczffp0zZgxo8zy+fPnKzAwsFZqR+04ftxfO5LCdWJ9mpps26yOKT/rQmOlonTMZbscU7C2hvfU/jZnq6BnGwV0DZdPTJDjRi8AAADgT7m5ubrhhhuUmZmp0NBQt+3Xa3uqquuBBx7Q5MmTnfNZWVmKj4/XoEGD3PrBVYfNZtOSJUt0ySWXyHKG9azU3EAVFUmbN9m19PPtKlr6vaK3r1CPnJWKNNLUO32Feq9bIa1zbJ1rClRKYCvlRLWQvXkrBXRqocieLRTW/c8usnoesGlLcBfaEtyBdgR3oS3BXSpqSyVXsbmb14aqmJgYSVJqaqpLT1VqaqrOPvvsCt/n5+cnPz+/MsstFovXfHN6Uy31icUindtHOrfP2dKssyXdqfTjdq3+72/K+my5wjYsV/yxDYq1H1SgkatWJ7ZIJ7ZIeyWtdN1XZkC0TjRpJbVsqcDOrRR6diuZW7eUWrWSmjaVfHzq/Pyqg7YEd6EtwR1oR3AX2hLc5eS2VFvtymtDVcuWLRUTE6Nly5Y5Q1RWVpbWrl2r22+/3bPFwWtERJqVePvZ0u1nS7pbkpSRWqDkFX/oyNpkndi8R8aePQpMTVb0iT1qpT0KV6bC8lIV9keq9MdqaYXrPot9LMqLai61bqWAji3l06aVI2y1/DN0RUTU8VkCAADAm3k0VOXk5GjXrl3O+eTkZG3cuFGNGjVSQkKCJk2apEcffVRt27ZVy5Yt9dBDDykuLs553xVQnvBoP3W/7izpurNcluflSTt2SEt/TtfRn/cod0uylLxHIUf3qLk9Wa20R831h6zFNgWn7JJSdkmryu6/KCRcplZ/hq2SoNWypdSihdS8uRQQUDcnCgAAAK/g0VD1yy+/6KKLLnLOl9wLNWbMGM2bN0///Oc/deLECd16663KyMjQ+eefr2+++Ub+/v6eKhn1WECA1K2b1K1bhHRLD0k9JElFRdKePdK27dInW4qV+utB5W7ZI/MfyYor2KOWcgSuVtqjGKXKNztD2rTBMZXDiI6WqSRklZ5atpQSEiTaLwAAQIPi0VDVv39/nWrwQZPJpJkzZ2rmzJl1WBXONL6+0llnOaYrr/SRlCApQYbRXwcOSNu3S+u2S29vk5K3nFDutr0Kz/graJX0cLVUskKVLVNqqpSaKq1ZU/4BY2P/ClknB6+EBKmcewIBAADgvbz2nirA00wmKT7eMQ0aVLI0SFInHTvWSdu3OwLX8u3S779LSb8bytybrubaqxbaq5ZKVos/X5fMB+uEdPiwY1q9uvyDxsW59m6VDl3x8Tz4GAAAwMsQqoBqaNxYuuACx/QXk/LyGmnXrkZKSjpHSUnSz79L7yY5Qld2tqFIHS8TtFpor1qZHfMB9lzp4EHHtKqcG7rMZqlpU/k0b66ehiHz1187imnUqOwUEeH4yuWGAAAAtYpQBbhRQIDUpYtjKs0wpJQUk5KSGuv33xsrKamntiVJC36X9u6VDLskGYrS0ZNC1151DEhWK/NexeTvlaU4X9q/X+b9+9VUKj94lVdUeaHr5PB18hQczAOUAQAAKoFQBdQBk8lxK1VsrNS/v+u6/Hxp1y7p999NSkpqoqSkJvr993O1OEnKypKUV7KloSY64uzhijWnKCE4XbH+6Yr2TVOkOU3h9nQF29IUmJ8mvxNpMtntjmEPS3q/qsLXt/zwFRFx+okREAEAwBmEUAV4mL+/1LmzYyrN0bslJSU5JkfoilZSUrTWJZ8rw26SsuSYymGSXSHKViOlKcqcppZhaWoekqb4oDRF+6WriU+aIk1pCrOnKbgwTQF5abLmpMmcflymwkLHsIhHjjimqvLzq1z4qiiQ0UMGAADqEUIV4KVO1buVnV2kDz5Yrk6dLtbx4746csQx4GDp6cgRs1JTw7Q3LUx77S21Ll1SemWObKhZozy1aZSmluHpah6SpmaBaYqxpqmxT5oamdIVVpyuIFu6AvLSZc5Ml9L/nDIypOJiqaDAkQhTUqp+4lZr2aAVEuKYgoOr9rWWnpoOAABQGqEKqIf8/aXo6Dyde65x2txQWCgdPaoywau8IHb0qGS3m3QgLVAH0gK1Qs1OW0t4uBQdLUV3kqKbGGoRma344HQ1DUxXjF+6mljSFaF0hdnT5ZtdKoCVNxUXOwouKaimrNayYasqwezkKShI8vGpeV0AAKBBIVQBDZzVKjVt6phOx26Xjh8/dfAqPdlsjs6pjAzHJYqSSVLon1PzMvsPC5NiYv4MYdFSdFvH15gYRyCLDclRjF+6onzT5ZdbKmzl5EjZ2ZX/WlDgOGBhoeOEjh9318fpuDyxvMBVk4lh8gEAqNcIVQCczGYpKsoxnXyP18kMw5F3SoeslJSK5202KTPTMTkC2MlMkkL+nBIUFvZX+IqMLDVORmz5Y2Y0auToNfPxkeNgOTlVD2MlX7OzpRMnXPdhtzvKzMtzTEePuu+Dt1j+6gkLCpICAyv+WtV1gYGOe9y4Tw0AgFpDqAJQLSbTXwMDduhw6m0Nw9GbdarQVXq+dADbsaNqdYWGSo0aWRQREaGIiAiX8OV8HSk1ausaykJDHaGywhMoKPgrZLlrKulRs9n+6pWrDWZzhYHLJzBQPTMy5PPf/zqWBwQ4ri8t/bW8ZadaR4gDAJxhCFUAap3J9Fd4qWwAKx2y0tOltLS/ckd5r3NyHO/PynJMe/dWrUaz2dHTdXIvmOO1SRER/mrUyF8REY3/Wt7UsU1QUDUzhM3m2iOWkyPl5jqWlf5a3rLKrLPZHMex2//a/8nnLTmeefbTT9U4gVM4XfiyWh3dimbzX1Pp+dpYV2FqrmUWS8XBtKKwSjAFgHqFUAXAq5QOYO3bV/59Jfd3nS58lTefl+fIHWlpjmn37qrVbLGUvRSxcq8t8g8Pd6S52mCzlQ1eJ4WvoqwsbVu/Xp1atZKPzeb4MPLz/7rMseT1yV/LW1ZyiaTkWJafXzvndaYoCV0VBa/KhLPKvi6ZJ8wBQLUQqgA0CBbLX/eDVVV+fsWB63SvbTbHVN1Hevn7u94TVnJb1clTyVV7lVkXEPBnp4zF4hgdJCyswuMbNpuSo6PVYehQ+dRkCHrDcHwQlQ1hhYWOEGa3O0Z9rMrr6ryn5HVNAkN13lvyuZx8/hV9Tp4OpiaTa0irTBALCJDZatVZ+/bJvHGjo3ew5LNy59dTrTOb/9qm5PXJX925ztfX8Y0XEPDXvYslr61WgilwBiJUATjj+fv/9UywqjAMR4dPVUJYyeuMDMfvz/n50uHDjsmdSgetUwUyf3+zDhxop61bzQoOPv3tUuVdxWcyyfEfq9UxhYa692TOFIbheOh2ZQPYqbY51evy5g3jrxpKllXhHj8fSae5svfMYTaXH7bKW1bR64qW+fm5TlYrvYuAlyBUAUA1mUx/hZP4+Kq91253DCpYOmxlZDiuzCs9lVytV5nleXl/7b/kSr/TD1LoI6m9PvywavWXMJmqHsQCAhyfWckznUNDK34dGHgG/b5oMjl6Fy2Wug2mpXvTqhLGSr0uPnFC+3fsUEJCgswl/2Clg1p1v1Z2G8NwfFOd/Lomy061vvTnVfLNVlzsqMlu/+ubsq5YLK4h6+TQVZN5i8X1vsSK7les6bqS5cXFCjp8WNq507HsVP8u7nxtGOV/jhW99tQ9mvBahCoA8ACz+a8r81q2dM8+7XbH73hVCWbZ2cX6/fd9atKkuQoLzaf9vbrkdenfaUt+p6wNZrNjtPnSYet0Qayi18HB/B5UrtK9jKe4VPRU7DabNn39tZoOHSpzTS4jrc9K7mEsHbSq8roy2xYUOC6dLRmEpvSxT15WT1kkDfR0EZXh43P64FXZgFb6/smTp8qs9/U9g/765L0IVQDQQJjNf/WcVZbNZtfXX/+moUObyWKpXOIwDMfvdZW50qyiW6pKPxIsO9sxYuPJr0v+kFwyoqM7+Pj81RlU8kf48ubdva4ks5xq2enW+/i45zNALanEPYxuY7c7vglLQlZBwV9T6flTravMfEFB2fsZy7vHsTLrKvkew25XUWGhfK1WmU6+p+3k19VdV952kiOYVvT5nRxai4tr9y9KVWE2Vy2UWSxle2JLTzVZdqpt+/aVXn/d059WrSFUAQCqxGT66w+stfX7Y0kPWHlh61SvK1pXVOTYb3GxY6qPAxOazZUPaiVTZf9QXt7rqmxrNv/Ve4k6UPqX6AamyGbT119/raFDh8riTb2eJX9NOlVwrez6k1+X3BeZn192/uSpZH1h4V+12e3eE/BOpVkzT1dQqwhVAACvU/p+taoOIHKykmc3Z2f/dZVUyR+ky3t9uvnqbFve14pel3wtuUWnhN3+1+9g3scik+kKBQfLZQoJqfx8eev8/bmqCV6i9F+TvEHJD4TKBrLS6woLXUe0LJlON1/TbSIjPf2p1SpCFQCgQSs9Snh9UjIeQkWh61ShrCZ/NK/KH95LjwBvGCZnD6G7lNxTd6oA5uvrOtZAeWNYnDxf2WUVbSP9Na7Cqb66cxsfn/J7ESszcS9hA1QyymRAgKcrwZ8IVQAAeCGz2bv+MF6e4mJHuMrJsemrr5apT58BKiiwKCfnr/vmynt9uvmSgfPcfU/dmcrX99Sh61QhrTL38lX20s/KbGe3m7V/f1ctXWp23hJU2el0YZNHiKE2EaoAAEC1+Pg4hr23WKRGjQrUpo3jdU2VjEpemRBWcplk6XEHanu+5D78kvEV3PH1dNsUFZU/jkRF40uUVlTkmOpylPfq85HUUosX187eTzVQn8XiaNO+vq5TZZZVd5uSqfQANxZL9ZfVRa9keW32dJPd7vhZUdXHj9QnhCoAAOBVzOa/hsNH1ZU8eqy8AHa66eSAVvoSz1OpbA/Q6bYrLi5WUtIOJSScpaIinwpDY2XP4eRB+0oul3XnZarepGRAm1MFMF/fygeh8pZX16WXSosWue9cvQ2hCgAAoAEp/eix+hZMHY952KGhQ9vIYqn5cwRKjz5/uhBW0qNXVOQID6Xny1tW021str+WlR7spmQ63fLyAo43DGhjMrneC1hyf2BVHvdRHxGqAAAA0CA14NHnnZeFVjaElUzFxWUHQaloqsx2J29zpt63RqgCAAAA6hmz+a8eSXgeg2wCAAAAQA0QqgAAAACgBghVAAAAAFADhCoAAAAAqAFCFQAAAADUAKEKAAAAAGqAUAUAAAAANUCoAgAAAIAaIFQBAAAAQA0QqgAAAACgBghVAAAAAFADhCoAAAAAqAFCFQAAAADUAKEKAAAAAGrA19MF1DbDMCRJWVlZHq5Estlsys3NVVZWliwWi6fLQT1GW4K70JbgDrQjuAttCe5SUVsqyQQlGcFdGnyoys7OliTFx8d7uBIAAAAA3iA7O1thYWFu25/JcHdM8zJ2u12HDh1SSEiITCaTR2vJyspSfHy89u/fr9DQUI/WgvqNtgR3oS3BHWhHcBfaEtylorZkGIays7MVFxcns9l9d0I1+J4qs9msZs2aeboMF6GhofyggFvQluAutCW4A+0I7kJbgruU15bc2UNVgoEqAAAAAKAGCFUAAAAAUAOEqjrk5+enadOmyc/Pz9OloJ6jLcFdaEtwB9oR3IW2BHep67bU4AeqAAAAAIDaRE8VAAAAANQAoQoAAAAAaoBQBQAAAAA1QKgCAAAAgBogVNWhOXPmqEWLFvL391fv3r31888/e7okeND333+vYcOGKS4uTiaTSQsXLnRZbxiGHn74YcXGxiogIEADBw7Uzp07XbZJS0vTqFGjFBoaqvDwcI0fP145OTku2/z222+64IIL5O/vr/j4eD355JO1fWqoQ7Nnz1avXr0UEhKiJk2aaPjw4UpKSnLZJj8/XxMmTFBkZKSCg4M1YsQIpaamumyzb98+XXbZZQoMDFSTJk103333qaioyGWbFStW6JxzzpGfn5/atGmjefPm1fbpoQ69+uqr6tq1q/NBmYmJiVq0aJFzPe0I1fH444/LZDJp0qRJzmW0JVTW9OnTZTKZXKb27ds713tVWzJQJz788EPDarUa//nPf4ytW7cat9xyixEeHm6kpqZ6ujR4yNdff208+OCDxoIFCwxJxqeffuqy/vHHHzfCwsKMhQsXGps2bTKuuOIKo2XLlkZeXp5zm0svvdTo1q2bsWbNGuOHH34w2rRpY4wcOdK5PjMz04iOjjZGjRplbNmyxfjggw+MgIAA4/XXX6+r00QtGzx4sDF37lxjy5YtxsaNG42hQ4caCQkJRk5OjnOb2267zYiPjzeWLVtm/PLLL0afPn2M8847z7m+qKjI6Ny5szFw4EBjw4YNxtdff200btzYeOCBB5zb7NmzxwgMDDQmT55sbNu2zXjppZcMHx8f45tvvqnT80Xt+fzzz42vvvrK2LFjh5GUlGT861//MiwWi7FlyxbDMGhHqLqff/7ZaNGihdG1a1fjrrvuci6nLaGypk2bZnTq1Mk4fPiwczp69KhzvTe1JUJVHTn33HONCRMmOOeLi4uNuLg4Y/bs2R6sCt7i5FBlt9uNmJgY46mnnnIuy8jIMPz8/IwPPvjAMAzD2LZtmyHJWLdunXObRYsWGSaTyTh48KBhGIbxyiuvGBEREUZBQYFzmylTphjt2rWr5TOCpxw5csSQZKxcudIwDEe7sVgsxscff+zcZvv27YYkY/Xq1YZhOAK+2Ww2UlJSnNu8+uqrRmhoqLPt/POf/zQ6derkcqzrrrvOGDx4cG2fEjwoIiLCePPNN2lHqLLs7Gyjbdu2xpIlS4x+/fo5QxVtCVUxbdo0o1u3buWu87a2xOV/daCwsFDr16/XwIEDncvMZrMGDhyo1atXe7AyeKvk5GSlpKS4tJmwsDD17t3b2WZWr16t8PBw9ezZ07nNwIEDZTabtXbtWuc2F154oaxWq3ObwYMHKykpSenp6XV0NqhLmZmZkqRGjRpJktavXy+bzebSltq3b6+EhASXttSlSxdFR0c7txk8eLCysrK0detW5zal91GyDT/DGqbi4mJ9+OGHOnHihBITE2lHqLIJEybosssuK/PvTVtCVe3cuVNxcXFq1aqVRo0apX379knyvrZEqKoDx44dU3Fxscs/qCRFR0crJSXFQ1XBm5W0i1O1mZSUFDVp0sRlva+vrxo1auSyTXn7KH0MNBx2u12TJk1S37591blzZ0mOf2er1arw8HCXbU9uS6drJxVtk5WVpby8vNo4HXjA5s2bFRwcLD8/P91222369NNP1bFjR9oRquTDDz/Ur7/+qtmzZ5dZR1tCVfTu3Vvz5s3TN998o1dffVXJycm64IILlJ2d7XVtybeqJwcA8E4TJkzQli1b9OOPP3q6FNRT7dq108aNG5WZman//e9/GjNmjFauXOnpslCP7N+/X3fddZeWLFkif39/T5eDem7IkCHO1127dlXv3r3VvHlz/fe//1VAQIAHKyuLnqo60LhxY/n4+JQZjSQ1NVUxMTEeqgrerKRdnKrNxMTE6MiRIy7ri4qKlJaW5rJNefsofQw0DBMnTtSXX36p5cuXq1mzZs7lMTExKiwsVEZGhsv2J7el07WTirYJDQ31uv+xofqsVqvatGmjHj16aPbs2erWrZteeOEF2hEqbf369Tpy5IjOOecc+fr6ytfXVytXrtSLL74oX19fRUdH05ZQbeHh4TrrrLO0a9cur/u5RKiqA1arVT169NCyZcucy+x2u5YtW6bExEQPVgZv1bJlS8XExLi0maysLK1du9bZZhITE5WRkaH169c7t/nuu+9kt9vVu3dv5zbff/+9bDabc5slS5aoXbt2ioiIqKOzQW0yDEMTJ07Up59+qu+++04tW7Z0Wd+jRw9ZLBaXtpSUlKR9+/a5tKXNmze7hPQlS5YoNDRUHTt2dG5Teh8l2/AzrGGz2+0qKCigHaHSBgwYoM2bN2vjxo3OqWfPnho1apTzNW0J1ZWTk6Pdu3crNjbW+34uVWlYC1Tbhx9+aPj5+Rnz5s0ztm3bZtx6661GeHi4y2gkOLNkZ2cbGzZsMDZs2GBIMp599lljw4YNxh9//GEYhmNI9fDwcOOzzz4zfvvtN+PKK68sd0j17t27G2vXrjV+/PFHo23bti5DqmdkZBjR0dHGjTfeaGzZssX48MMPjcDAQIZUb0Buv/12IywszFixYoXLkLO5ubnObW677TYjISHB+O6774xffvnFSExMNBITE53rS4acHTRokLFx40bjm2++MaKiosodcva+++4ztm/fbsyZM4fhixuY+++/31i5cqWRnJxs/Pbbb8b9999vmEwm49tvvzUMg3aE6is9+p9h0JZQeffcc4+xYsUKIzk52Vi1apUxcOBAo3HjxsaRI0cMw/CutkSoqkMvvfSSkZCQYFitVuPcc8811qxZ4+mS4EHLly83JJWZxowZYxiGY1j1hx56yIiOjjb8/PyMAQMGGElJSS77OH78uDFy5EgjODjYCA0NNcaNG2dkZ2e7bLNp0ybj/PPPN/z8/IymTZsajz/+eF2dIupAeW1IkjF37lznNnl5ecY//vEPIyIiwggMDDSuuuoq4/Dhwy772bt3rzFkyBAjICDAaNy4sXHPPfcYNpvNZZvly5cbZ599tmG1Wo1WrVq5HAP130033WQ0b97csFqtRlRUlDFgwABnoDIM2hGq7+RQRVtCZV133XVGbGysYbVajaZNmxrXXXedsWvXLud6b2pLJsMwjKr1bQEAAAAASnBPFQAAAADUAKEKAAAAAGqAUAUAAAAANUCoAgAAAIAaIFQBAAAAQA0QqgAAAACgBghVAAAAAFADhCoAAAAAqAFCFQDgjNaiRQs9//zzni4DAFCPEaoAAHVm7NixGj58uCSpf//+mjRpUp0de968eQoPDy+zfN26dbr11lvrrA4AQMPj6+kCAACoicLCQlmt1mq/Pyoqyo3VAADORPRUAQDq3NixY7Vy5Uq98MILMplMMplM2rt3ryRpy5YtGjJkiIKDgxUdHa0bb7xRx44dc763f//+mjhxoiZNmqTGjRtr8ODBkqRnn31WXbp0UVBQkOLj4/WPf/xDOTk5kqQVK1Zo3LhxyszMdB5v+vTpkspe/rdv3z5deeWVCg4OVmhoqK699lqlpqY610+fPl1nn3223n33XbVo0UJhYWG6/vrrlZ2d7dzmf//7n7p06aKAgABFRkZq4MCBOnHiRC19mgAATyNUAQDq3AsvvKDExETdcsstOnz4sA4fPqz4+HhlZGTo4osvVvfu3fXLL7/om2++UWpqqq699lqX97/99tuyWq1atWqVXnvtNUmS2WzWiy++qK1bt+rtt9/Wd999p3/+85+SpPPOO0/PP/+8QkNDnce79957y9Rlt9t15ZVXKi0tTStXrtSSJUu0Z88eXXfddS7b7d69WwsXLtSXX36pL7/8UitXrtTjjz8uSTp8+LBGjhypm266Sdu3b9eKFSt09dVXyzCM2vgoAQBegMv/AAB1LiwsTFarVYGBgYqJiXEuf/nll9W9e3fNmjXLuew///mP4uPjtWPHDp111lmSpLZt2+rJJ5902Wfp+7NatGihRx99VLfddpteeeUVWa1WhYWFyWQyuRzvZMuWLdPmzZuVnJys+Ph4SdI777yjTp06ad26derVq5ckR/iaN2+eQkJCJEk33nijli1bpscee0yHDx9WUVGRrr76ajVv3lyS1KVLlxp8WgAAb0dPFQDAa2zatEnLly9XcHCwc2rfvr0kR+9QiR49epR579KlSzVgwAA1bdpUISEhuvHGG3X8+HHl5uZW+vjbt29XfHy8M1BJUseOHRUeHq7t27c7l7Vo0cIZqCQpNjZWR44ckSR169ZNAwYMUJcuXXTNNdfojTfeUHp6euU/BABAvUOoAgB4jZycHA0bNkwbN250mXbu3KkLL7zQuV1QUJDL+/bu3avLL79cXbt21SeffKL169drzpw5khwDWbibxWJxmTeZTLLb7ZIkHx8fLVmyRIsWLVLHjh310ksvqV27dkpOTnZ7HQAA70CoAgB4hNVqVXFxscuyc845R1u3blWLFi3Upk0bl+nkIFXa+vXrZbfb9cwzz6hPnz4666yzdOjQodMe72QdOnTQ/v37tX//fueybdu2KSMjQx07dqz0uZlMJvXt21czZszQhg0bZLVa9emnn1b6/QCA+oVQBQDwiBYtWmjt2rXau3evjh07JrvdrgkTJigtLU0jR47UunXrtHv3bi1evFjjxo07ZSBq06aNbDabXnrpJe3Zs0fvvvuucwCL0sfLycnRsmXLdOzYsXIvCxw4cKC6dOmiUaNG6ddff9XPP/+s0aNHq1+/furZs2elzmvt2rWaNWuWfvnlF+3bt08LFizQ0aNH1aFDh6p9QACAeoNQBQDwiHvvvVc+Pj7q2LGjoqKitG/fPsXFxWnVqlUqLi7WoEGD1KVLF02aNEnh4eEymyv+X1a3bt307LPP6oknnlDnzp31/vvva/bs2S7bnHfeebrtttt03XXXKSoqqsxAF5Kjh+mzzz5TRESELrzwQg0cOFCtWrXSRx99VOnzCg0N1ffff6+hQ4fqrLPO0tSpU/XMM89oyJAhlf9wAAD1islgjFcAAAAAqDZ6qgAAAACgBghVAAAAAFADhCoAAAAAqAFCFQAAAADUAKEKAAAAAGqAUAUAAAAANUCoAgAAAIAaIFQBAAAAQA0QqgAAAACgBghVAAAAAFADhCoAAAAAqIH/B6fcQWqnIht4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1000x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Train model\n",
    "\n",
    "iterations = np.empty(int(max_iters/eval_iters))\n",
    "iterations[:] = np.nan\n",
    "train_loss = np.empty(int(max_iters/eval_iters))\n",
    "train_loss[:] = np.nan\n",
    "val_loss = np.empty(int(max_iters/eval_iters))\n",
    "val_loss[:] = np.nan\n",
    "\n",
    "for i in range(max_iters):\n",
    "    train_one_batch(model, optimizer)\n",
    "    if i % eval_iters == 0:\n",
    "        out = estimate_loss()\n",
    "        iterations[int(i/eval_iters)] = i\n",
    "        train_loss[int(i/eval_iters)] = torch.exp(out['train'])\n",
    "        val_loss[int(i/eval_iters)] = torch.exp(out['val'])\n",
    "        print(f\"First {i} batches: training perplexity: {torch.exp(out['train']):.3f}, validation perplexity: {torch.exp(out['val']):.3f}\")\n",
    "        \n",
    "# Visualize results\n",
    "plt.figure(figsize=(10, 5))\n",
    "plt.plot(iterations, train_loss, label='Training Perplexity', color='blue')\n",
    "plt.plot(iterations, val_loss, label='Validation Perplexity', color='red')\n",
    "plt.title('Training and Validation Perplexity')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Perplexity')\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "f948049a-30d4-4f18-9724-16b53df02fa4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Output some text\n",
    "\n",
    "model.eval()\n",
    "prompt = \"Hello, my name is\"\n",
    "max_new_tokens = 100\n",
    "ix = torch.tensor([encode(prompt)], dtype=torch.long)\n",
    "X =  ix.to('cuda')\n",
    "output = model.generate(X, max_new_tokens)\n",
    "[output] = output.tolist()\n",
    "response = decode(output)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "342c954e-cc6a-4931-8939-8d2cb38dbf06",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello, my name is lip you sucwured no craw out speakless king not have wretch.\n",
      "\n",
      "BENVINCLIV:\n",
      "Wout if fair; you confore\n"
     ]
    }
   ],
   "source": [
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "744ddb69-4c7a-4009-b54b-dfa35fc1b8a4",
   "metadata": {},
   "source": [
    "#### Change hyperparameters and re-train model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1ea276a-56c0-491e-9599-96687c7a60d7",
   "metadata": {},
   "source": [
    "Change size of embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "7202d7da-6a66-4a77-b1b3-386252745eb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_embd = 128\n",
    "\n",
    "model = BigramLanguageModel()\n",
    "model.to('cuda')\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "4925d9eb-9814-4b6f-8f6f-e613380a6f31",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 0 batches: training perplexity: 48.109, validation perplexity: 48.645\n",
      "First 200 batches: training perplexity: 10.769, validation perplexity: 10.682\n",
      "First 400 batches: training perplexity: 8.697, validation perplexity: 8.949\n",
      "First 600 batches: training perplexity: 7.773, validation perplexity: 8.210\n",
      "First 800 batches: training perplexity: 7.123, validation perplexity: 7.757\n",
      "First 1000 batches: training perplexity: 6.525, validation perplexity: 7.212\n",
      "First 1200 batches: training perplexity: 6.214, validation perplexity: 6.988\n",
      "First 1400 batches: training perplexity: 5.924, validation perplexity: 6.751\n",
      "First 1600 batches: training perplexity: 5.699, validation perplexity: 6.485\n",
      "First 1800 batches: training perplexity: 5.536, validation perplexity: 6.386\n",
      "First 2000 batches: training perplexity: 5.402, validation perplexity: 6.246\n",
      "First 2200 batches: training perplexity: 5.354, validation perplexity: 6.176\n",
      "First 2400 batches: training perplexity: 5.249, validation perplexity: 6.176\n",
      "First 2600 batches: training perplexity: 5.164, validation perplexity: 6.098\n",
      "First 2800 batches: training perplexity: 5.081, validation perplexity: 6.032\n",
      "First 3000 batches: training perplexity: 5.012, validation perplexity: 5.871\n",
      "First 3200 batches: training perplexity: 4.961, validation perplexity: 5.790\n",
      "First 3400 batches: training perplexity: 4.949, validation perplexity: 5.935\n",
      "First 3600 batches: training perplexity: 4.847, validation perplexity: 5.755\n",
      "First 3800 batches: training perplexity: 4.863, validation perplexity: 5.715\n",
      "First 4000 batches: training perplexity: 4.795, validation perplexity: 5.733\n",
      "First 4200 batches: training perplexity: 4.775, validation perplexity: 5.770\n",
      "First 4400 batches: training perplexity: 4.738, validation perplexity: 5.688\n",
      "First 4600 batches: training perplexity: 4.672, validation perplexity: 5.569\n",
      "First 4800 batches: training perplexity: 4.640, validation perplexity: 5.728\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1UAAAHWCAYAAACfRKOZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB11ElEQVR4nO3dd3gU5d7G8XtTNr1QUyD0roCICAEFFBRBUZRjQY4UUY8eUFFR9Ig0C/aO5VUP2BCPitgoAlIUAREBadI7CS2kt0123j/GXbIpkLLJbsL3c11zZXdmdua3y5OQO88zz1gMwzAEAAAAACgXH08XAAAAAADVGaEKAAAAACqAUAUAAAAAFUCoAgAAAIAKIFQBAAAAQAUQqgAAAACgAghVAAAAAFABhCoAAAAAqABCFQAAAABUAKEKAEppxIgRatKkSbleO3nyZFksFvcW5GX27dsni8WimTNnVvm5LRaLJk+e7Hw+c+ZMWSwW7du376yvbdKkiUaMGOHWeirSVmqiwv8+7nYufH8B8G6EKgDVnsViKdWybNkyT5d6zrvvvvtksVi0a9euEvd5/PHHZbFY9Oeff1ZhZWV35MgRTZ48WRs2bPB0KU6OYOtYfH191ahRI11//fVeVWdVeOaZZzR37lxPlwHgHEGoAlDtffzxxy7LFVdcUez6tm3bVug87733nrZv316u106YMEFZWVkVOn9NMHToUEnSrFmzStzns88+U/v27dWhQ4dyn+e2225TVlaWGjduXO5jnM2RI0c0ZcqUYsNKRdqKOwwZMkQff/yx/vvf/+rWW2/VTz/9pG7dutXYYFXc9xehCkBV8vN0AQBQUf/85z9dnq9evVqLFi0qsr6wzMxMBQcHl/o8/v7+5apPkvz8/OTnx4/crl27qkWLFvrss880ceLEIttXrVqlvXv36tlnn63QeXx9feXr61uhY1RERdqKO1x44YUu7b9Hjx669tpr9fbbb+vdd9+t0LEzMjIUEhJS0RLdiu8vAJ5GTxWAc0Lv3r11/vnna926derZs6eCg4P1n//8R5L0zTff6Oqrr1ZsbKwCAgLUvHlzPfnkk8rPz3c5RuHrZBxDrV588UX93//9n5o3b66AgAB16dJFa9eudXltcdd8WCwWjRkzRnPnztX555+vgIAAnXfeeVqwYEGR+pctW6aLLrpIgYGBat68ud59991SX0fy888/68Ybb1SjRo0UEBCguLg4PfDAA0X+sj9ixAiFhobq8OHDGjRokEJDQ1WvXj2NGzeuyGeRnJysESNGKCIiQpGRkRo+fLiSk5PPWotk9lb99ddf+uOPP4psmzVrliwWi4YMGaLc3FxNnDhRnTt3VkREhEJCQnTppZdq6dKlZz1HcddUGYahp556Sg0bNlRwcLAuu+wybdmypchrk5KSNG7cOLVv316hoaEKDw9X//79tXHjRuc+y5YtU5cuXSRJI0eOdA63c1xPVtw1VRkZGXrooYcUFxengIAAtW7dWi+++KIMw3DZryztorQuv/xySdLevXud69asWaOrrrpKERERCg4OVq9evbRy5UqX1zna2NatW3XrrbeqVq1auuSSS5zvMTQ0VHv27FG/fv0UEhKi2NhYTZ06tch7Ks7hw4d1++23Kyoqyvke//vf/zq3Z2VlqU2bNmrTpo1LW01KSlJMTIy6d+/ubJeFvxcsFosyMjL04YcfOv9tRowYoaVLl8pisejrr78uUo+j7a1atao0HykAuODPOgDOGSdPnlT//v11yy236J///KeioqIkmb+Ah4aG6sEHH1RoaKh++uknTZw4UampqXrhhRfOetxZs2YpLS1N//rXv2SxWPT888/rhhtu0J49e87aY/HLL79ozpw5+ve//62wsDC9/vrrGjx4sA4cOKA6depIktavX6+rrrpKMTExmjJlivLz8zV16lTVq1evVO/7iy++UGZmpu655x7VqVNHv/32m9544w0dOnRIX3zxhcu++fn56tevn7p27aoXX3xRixcv1ksvvaTmzZvrnnvukWSGk+uuu06//PKL7r77brVt21Zff/21hg8fXqp6hg4dqilTpmjWrFm68MILXc79v//9T5deeqkaNWqkEydO6P3339eQIUN05513Ki0tTR988IH69eun3377TRdccEGpzucwceJEPfXUUxowYIAGDBigP/74Q1deeaVyc3Nd9tuzZ4/mzp2rG2+8UU2bNtXRo0f17rvvqlevXtq6datiY2PVtm1bTZ06VRMnTtRdd92lSy+9VJLUvXv3Ys9tGIauvfZaLV26VKNGjdIFF1yghQsX6uGHH9bhw4f1yiuvuOxfmnZRFrt375Yk52t/+ukn9e/fX507d9akSZPk4+OjGTNm6PLLL9fPP/+siy++2OX1N954o1q2bKlnnnnGJTDl5+frqquuUrdu3fT8889rwYIFmjRpkvLy8jR16tQS6zl69Ki6devmDJD16tXT/PnzNWrUKKWmpmrs2LEKCgrShx9+qB49eujxxx/Xyy+/LEkaPXq0UlJSNHPmzBJ7Iz/++GPdcccduvjii3XXXXdJkpo3b65u3bopLi5On376qa6//nqX13z66adq3ry54uPjy/jpAoAkAwBqmNGjRxuFf7z16tXLkGS88847RfbPzMwssu5f//qXERwcbGRnZzvXDR8+3GjcuLHz+d69ew1JRp06dYykpCTn+m+++caQZHz33XfOdZMmTSpSkyTDarUau3btcq7buHGjIcl44403nOsGDhxoBAcHG4cPH3au27lzp+Hn51fkmMUp7v1NmzbNsFgsxv79+13enyRj6tSpLvt26tTJ6Ny5s/P53LlzDUnG888/71yXl5dnXHrppYYkY8aMGWetqUuXLkbDhg2N/Px857oFCxYYkox3333XecycnByX1506dcqIiooybr/9dpf1koxJkyY5n8+YMcOQZOzdu9cwDMM4duyYYbVajauvvtqw2+3O/f7zn/8Ykozhw4c712VnZ7vUZRjmv3VAQIDLZ7N27doS32/htuL4zJ566imX/f7xj38YFovFpQ2Utl0Ux9Emp0yZYhw/ftxITEw0li1bZnTq1MmQZHz11VeG3W43WrZsafTr18/ls8jMzDSaNm1qXHHFFc51jnY7ZMiQYt+jJOPee+91rrPb7cbVV19tWK1W4/jx4y7vqeC/z6hRo4yYmBjjxIkTLse85ZZbjIiICJc2+9hjjxk+Pj7GihUrjC+++MKQZLz66qsuryvu+yskJMTl37Xg8QICAozk5GTnumPHjhl+fn4uNQJAWTD8D8A5IyAgQCNHjiyyPigoyPk4LS1NJ06c0KWXXqrMzEz99ddfZz3uzTffrFq1ajmfO3ot9uzZc9bX9u3bV82bN3c+79Chg8LDw52vzc/P1+LFizVo0CDFxsY692vRooX69+9/1uNLru8vIyNDJ06cUPfu3WUYhtavX19k/7vvvtvl+aWXXuryXubNmyc/Pz9nz5VkXsN07733lqoeybwO7tChQ1qxYoVz3axZs2S1WnXjjTc6j2m1WiVJdrtdSUlJysvL00UXXVTs0MEzWbx4sXJzc3Xvvfe6DBMbO3ZskX0DAgLk42P+95ifn6+TJ08qNDRUrVu3LvN5HebNmydfX1/dd999LusfeughGYah+fPnu6w/W7s4m0mTJqlevXqKjo5W7969tXv3bj333HO64YYbtGHDBu3cuVO33nqrTp48qRMnTujEiRPKyMhQnz59tGLFCtntdpfjFW4TBY0ZM8b52NHzlJubq8WLFxe7v2EY+uqrrzRw4EAZhuE8/4kTJ9SvXz+lpKS4fM6TJ0/Weeedp+HDh+vf//63evXqVeRzLIthw4YpJydHX375pXPd559/rry8vLNehwkAJWH4H4BzRoMGDZy/pBe0ZcsWTZgwQT/99JNSU1NdtqWkpJz1uI0aNXJ57ghYp06dKvNrHa93vPbYsWPKyspSixYtiuxX3LriHDhwQBMnTtS3335bpKbC7y8wMLDIsMKC9UjS/v37FRMTo9DQUJf9WrduXap6JOmWW27Rgw8+qFmzZql3797Kzs7W119/rf79+7sE1A8//FAvvfSS/vrrL9lsNuf6pk2blvpcjpolqWXLli7r69Wr53I+yQxwr732mt566y3t3bvX5Xqy8gy9c5w/NjZWYWFhLusdM1I66nM4W7s4m7vuuks33nijfHx8FBkZqfPOO08BAQGSpJ07d0rSGYdrpqSkuHwuJX3ePj4+atasmcu6Vq1aSVKJ9wg7fvy4kpOT9X//93/6v//7v2L3OXbsmPOx1WrVf//7X3Xp0kWBgYGaMWNGhe5J1aZNG3Xp0kWffvqpRo0aJckc+tetW7dSf08BQGGEKgDnjII9Ng7Jycnq1auXwsPDNXXqVDVv3lyBgYH6448/NH78+CJ/sS9OSdd1GKW4WL8iry2N/Px8XXHFFUpKStL48ePVpk0bhYSE6PDhwxoxYkSR91dVM+bVr19fV1xxhb766itNnz5d3333ndLS0pxTrkvSJ598ohEjRmjQoEF6+OGHVb9+ffn6+mratGnOa4QqwzPPPKMnnnhCt99+u5588knVrl1bPj4+Gjt2bKnagztUtF20bNlSffv2LXab4z288MILJV6XVjgwF/e9U16O8//zn/8sMdgVnk5/4cKFkqTs7Gzt3LmzzKG6sGHDhun+++/XoUOHlJOTo9WrV+vNN9+s0DEBnNsIVQDOacuWLdPJkyc1Z84c9ezZ07m+4CxpnlS/fn0FBgYWe7PcM91A12HTpk3asWOHPvzwQw0bNsy5ftGiReWuqXHjxlqyZInS09Ndfvku632Zhg4dqgULFmj+/PmaNWuWwsPDNXDgQOf2L7/8Us2aNdOcOXNceiYmTZpUrpols5emYM/K8ePHi/T+fPnll7rsssv0wQcfuKxPTk5W3bp1nc/L0lvSuHFjLV68WGlpaS69VY7hpZV5P63CHMMKw8PDSwxepWW327Vnzx5n75Qk7dixQ5KKzH7oUK9ePYWFhSk/P79U5//zzz81depUjRw5Uhs2bNAdd9yhTZs2KSIi4oyvO9O/j6On9LPPPlNWVpb8/f118803n7UWACgJ11QBOKc5egQK9gDk5ubqrbfe8lRJLnx9fdW3b1/NnTtXR44cca7ftWtXketwSnq95Pr+DMPQa6+9Vu6aBgwYoLy8PL399tvOdfn5+XrjjTfKdJxBgwYpODhYb731lubPn68bbrhBgYGBZ6x9zZo15Zryum/fvvL399cbb7zhcrxXX321yL6+vr5FeoS++OILHT582GWd415NpZlKfsCAAcrPzy/SG/LKK6/IYrGU+vo4d+jcubOaN2+uF198Uenp6UW2Hz9+vEzHK/ieDMPQm2++KX9/f/Xp06fY/X19fTV48GB99dVX2rx58xnPb7PZNGLECMXGxuq1117TzJkzdfToUT3wwANnrSskJKTEf5u6deuqf//++uSTT/Tpp5/qqquucgnMAFBW9FQBOKd1795dtWrV0vDhw3XffffJYrHo448/dtvwO3eYPHmyfvzxR/Xo0UP33HOP85fz888/Xxs2bDjja9u0aaPmzZtr3LhxOnz4sMLDw/XVV1+V+tqc4gwcOFA9evTQo48+qn379qldu3aaM2dOqa4/Kyg0NFSDBg3SrFmzJMll6J8kXXPNNZozZ46uv/56XX311dq7d6/eeecdtWvXrtgwcCaO+21NmzZN11xzjQYMGKD169dr/vz5RX6Zvuaaa5w9I927d9emTZv06aefFrl2qHnz5oqMjNQ777yjsLAwhYSEqGvXrsUOTRs4cKAuu+wyPf7449q3b586duyoH3/8Ud98843Gjh3rMilFZfPx8dH777+v/v3767zzztPIkSPVoEEDHT58WEuXLlV4eLi+++67Uh0rMDBQCxYs0PDhw9W1a1fNnz9fP/zwg/7zn/+cccr/Z599VkuXLlXXrl115513ql27dkpKStIff/yhxYsXKykpSZL01FNPacOGDVqyZInCwsLUoUMHTZw4URMmTNA//vEPDRgwoMRzdO7cWYsXL9bLL7+s2NhYNW3aVF27dnVuHzZsmP7xj39Ikp588slSvV8AKAk9VQDOaXXq1NH333+vmJgYTZgwQS+++KKuuOIKPf/8854uzalz586aP3++atWqpSeeeEIffPCBpk6dqj59+rj07BTH399f3333nS644AJNmzZNU6ZMUcuWLfXRRx+Vux4fHx99++23Gjp0qD755BM9/vjjatCggT788MMyH8sRpGJiYpw3qHUYMWKEnnnmGW3cuFH33XefFi5cqE8++UQXXXRRuep+6qmnNGXKFK1fv14PP/ywdu/erR9//NHZ4+Twn//8Rw899JAWLlyo+++/X3/88Yd++OEHxcXFuezn7++vDz/8UL6+vrr77rs1ZMgQLV++vNhzOz6zsWPH6vvvv9fYsWO1detWvfDCC877L1Wl3r17a9WqVbrooov05ptv6t5779XMmTMVHR1dql4gB19fXy1YsECJiYl6+OGHtXbtWk2aNOmsISUqKkq//fabRo4cqTlz5mjMmDF67bXXlJSUpOeee06S9Mcff+iZZ57RmDFjdNlllzlf++ijj6pLly668847z9hL+PLLL6tz586aMGGChgwZ4tKzKplBt1atWoqIiNC1115b6vcMAMWxGN7051gAQKkNGjRIW7Zscc7mBlSlESNG6Msvvyxzr6G3yMvLU2xsrAYOHFjk+jkAKCt6qgCgGsjKynJ5vnPnTs2bN0+9e/f2TEFANTd37lwdP37cZQIXACgvrqkCgGqgWbNmGjFihJo1a6b9+/fr7bffltVq1SOPPOLp0oBqZc2aNfrzzz/15JNPqlOnTurVq5enSwJQAxCqAKAauOqqq/TZZ58pMTFRAQEBio+P1zPPPFPkZrYAzuztt9/WJ598ogsuuEAzZ870dDkAagiuqQIAAACACuCaKgAAAACoAEIVAAAAAFRAjb+mym6368iRIwoLC5PFYvF0OQAAAAA8xDAMpaWlKTY2Vj4+7utfqvGh6siRI0Vu2AgAAADg3HXw4EE1bNjQbcer8aEqLCxMkvnBhYeHe7QWm82mH3/8UVdeeaX8/f09WguqN9oS3IW2BHegHcFdaEtwl5LaUmpqquLi4pwZwV1qfKhyDPkLDw/3ilAVHBys8PBwflCgQmhLcBfaEtyBdgR3oS3BXc7Wltx9WZBHJ6qYPHmyLBaLy9KmTRvn9uzsbI0ePVp16tRRaGioBg8erKNHj3qwYgAAAABw5fHZ/8477zwlJCQ4l19++cW57YEHHtB3332nL774QsuXL9eRI0d0ww03eLBaAAAAAHDl8eF/fn5+io6OLrI+JSVFH3zwgWbNmqXLL79ckjRjxgy1bdtWq1evVrdu3Yo9Xk5OjnJycpzPU1NTJZldgDabrRLeQek5zu/pOlD90ZbgLrQluAPtCO5CW4K7lNSWKqtteTxU7dy5U7GxsQoMDFR8fLymTZumRo0aad26dbLZbOrbt69z3zZt2qhRo0ZatWpViaFq2rRpmjJlSpH1P/74o4KDgyvtfZTFokWLPF0CagjaEtyFtgR3oB15H4vFIl9fX0+XUSZ+fn5aunSpp8tANWW322W3253PC/9cyszMrJTzejRUde3aVTNnzlTr1q2VkJCgKVOm6NJLL9XmzZuVmJgoq9WqyMhIl9dERUUpMTGxxGM+9thjevDBB53PHTN8XHnllV4xUcWiRYt0xRVXcPElKoS2BHehLcEdaEfeKSMjQwkJCTIMw9OllJphGMrOzlZgYCD3F0W5BQUFqXbt2lq2bFmRn0uOUWzu5tFQ1b9/f+fjDh06qGvXrmrcuLH+97//KSgoqFzHDAgIUEBAQJH1/v7+XvOD3ptqQfVGW4K70JbgDrQj75Gfn6/ExESFhISoXr161Sag2O12paenKzQ01K03ZsW5wTAM5ebm6vjx40pISJBU9OdSZf2M8vjwv4IiIyPVqlUr7dq1S1dccYVyc3OVnJzs0lt19OjRYq/BAgAAgMlms8kwDNWrV6/cf6j2BLvdrtzcXAUGBhKqUC5BQUHy9/fXvn37qnToq1e11vT0dO3evVsxMTHq3Lmz/P39tWTJEuf27du368CBA4qPj/dglQAAANVDdemhAtzJEcirsv17tKdq3LhxGjhwoBo3bqwjR45o0qRJ8vX11ZAhQxQREaFRo0bpwQcfVO3atRUeHq57771X8fHxJU5SAQAAAABVzaOh6tChQxoyZIhOnjypevXq6ZJLLtHq1atVr149SdIrr7wiHx8fDR48WDk5OerXr5/eeustT5YMAAAAAC48Ovxv9uzZOnLkiHJycnTo0CHNnj1bzZs3d24PDAzU9OnTlZSUpIyMDM2ZM4frqQAAAFBqTZo00auvvlrq/ZctWyaLxaLk5ORKq8nTZs6cWWSG7YqyWCyaO3euW49ZnXjVNVUAAAA4NznuqVWrVi35+vrKYrG4LJMnTy7XcdeuXau77rqr1Pt3795dCQkJioiIKNf5SssR3hxLVFSUBg8erD179lTqeStLQkKCc2bvffv2yWKxaMOGDZ4tqgp51ex/AAAAODclJCTIbrcrLS1N8+fP16RJk7R9+3bn9tDQUOdjwzCUn58vP7+z/yrruKyktKxWa5WOjNq+fbvCwsK0c+dO3XXXXRo4cKD+/PPPcs1cZ7PZPHZbg3N9NBk9VVXFMOTbt6/6/utf0pEjnq4GAACcQwxDysjwzFLaew9HR0crOjpaUVFRCg8Pl8Vica7766+/FBYWpvnz56tz584KCAjQL7/8ot27d+u6665TVFSUQkND1aVLFy1evNjluIWH/1ksFr3//vu6/vrrFRwcrJYtW+rbb791bi88/M8xVG7hwoVq27atQkNDddVVVznvgyRJeXl5uu+++xQZGak6depo/PjxGj58uAYNGnTW912/fn3FxMSoZ8+emjhxorZu3apdu3ZJkr755htdeOGFCgwMVLNmzTRlyhTl5eW5vJe3335b1157rUJCQvT000876//hhx/UoUMHBQYGqlu3btq8efMZ6zjTuaZOnarY2FidPHnSuf/VV1+tyy67THa73VmLY/hf06ZNJUmdOnWSxWJR7969tWLFCvn7+ysxMdHlvGPHjtWll1561s/J2xGqqorFIsuuXQo5elSWAt+EAAAAlS0zUwoN9cySmem+9/Hoo4/q2Wef1bZt29ShQwelp6drwIABWrJkidavX6+rrrpKAwcO1IEDB854nClTpuimm27Sn3/+qQEDBmjo0KFKSko6w+eXqRdffFEff/yxVqxYoQMHDmjcuHHO7c8995w+/fRTzZgxQytXrlRqamq5ri9y3FMsNzdXP//8s4YNG6b7779fW7du1bvvvquZM2fq6aefdnnN5MmTdf3112vTpk26/fbbnesffvhhvfTSS1q7dq3q1aungQMHymazFXves53r8ccfV5MmTXTHHXdIkqZPn65ff/1VH374YbH3E/vtt98kSYsXL1ZCQoLmzJmjnj17qlmzZvr444+d+9lsNn366acudVdbRg2XkpJiSDJSUlI8XYqRf+GFhiEZtjlzPF0Kqrnc3Fxj7ty5Rm5urqdLQTVHW4I70I68T1ZWlrF161YjKyvLMAzDSE83DLPPqOqX9PTS152fn2+cOnXK+OCDD4yIiAjn+qVLlxqSjLlz5571GOedd57xxhtvOJ83btzYeOWVV5zPJRkTJkxwPk9PTzckGfPnz3c516lTpwzDMIwZM2YYkoxdu3Y5XzN9+nQjKirK+TwqKsp44YUXnM/z8vKMRo0aGdddd12JdRY+z5EjR4zu3bsbDRo0MHJycow+ffoYzzzzjMtrPv74YyMmJsblvYwdO7bY486ePdu57uTJk0ZQUJDx+eefO99Twc+3NOfavXu3ERYWZowfP94ICgoyPv30U5f9JRlff/21YRiGsXfvXkOSsX79epd9nnvuOaNt27bO51999ZURGhpqpJelkZRCVlaWsWXLFuP7778v8nOpsrIB11RVJcdY00LdngAAAJUpOFhKT/fcud3loosucnmenp6uyZMn64cfflBCQoLy8vKUlZV11p6qDh06OB+HhIQoPDxcx44dK3H/4OBglxmqY2JinPunpKTo6NGjuvjii53bfX191blzZ+fQuDNp2LChDMNQZmamOnbsqK+++kpWq1UbN27UypUrXXqm8vPzlZ2drczMTAX//cEW/kwc4uPjnY9r166t1q1ba9u2bcXuW5pzNWvWTC+++KL+9a9/6eabb9att9561vdW2IgRIzRhwgStXr1a3bp108yZM3XTTTcpJCSkzMfyNoSqKmIY0rqEWHWRlLM3gQ8eAABUGYtFqgG/txb55XvcuHFatGiRXnzxRbVo0UJBQUH6xz/+odzc3DMep/BkDhaL5YwBqLj9jdJeLHYWP//8s8LDw1W/fn2FhYU516enp2vKlCm64YYbirwmMDDQ+dgdgaS051qxYoV8fX21b98+5eXllWqikILq16+vgQMHasaMGWratKnmz5+vZcuWVbR8r8Dv9lXEYpGW/RWjLpIy9ySqBvxcAwAA8KiVK1dqxIgRuv766yWZ4WDfvn1VWkNERISioqK0du1a9ezZU5LZy/PHH3/oggsuOOvrmzZtWuw9oy688EJt375dLVq0KFddq1evVqNGjSRJp06d0o4dO9S2bdti9y3NuT7//HPNmTNHy5Yt00033aQnn3xSU6ZMKXZfq9UqyfwcCrvjjjs0ZMgQNWzYUM2bN1ePHj3K+ta8EqGqCmXXipGyJPshhv8BAABUVMuWLTVnzhwNHDhQFotFTzzxRKmG3Lnbvffeq2nTpqlFixZq06aN3njjDZ06dUoWi6Xcx5w4caKuueYaNWrUSP/4xz/k4+OjjRs3avPmzXrqqafO+vqpU6eqTp06ioqK0uOPP666deuWOBvh2c516NAh3XPPPXruued0ySWXaMaMGbrmmmvUv39/devWrcjx6tevr6CgIC1YsEANGzZUYGCg875f/fr1U3h4uJ566ilNnTq13J+Pt2H2vyqUX9+8pspyjFAFAABQUS+//LJq1aql7t27a+DAgerXr58uvPDCKq9j/PjxGjJkiIYNG6b4+HiFhoaqX79+LkPnyqpfv376/vvv9eOPP6pLly7q1q2bXnnlFTVu3LhUr3/22Wd1//33q3PnzkpMTNR3333n7EEqy7kMw9CIESN08cUXa8yYMc7977nnHv3zn/9UejEX6/n5+en111/Xu+++q9jYWF133XXObT4+PhoxYoTy8/M1bNiwcnwy3sliuGtAqJdKTU1VRESEUlJSFB4e7tFaply9WpPmxSslIk4RyWe+gBI4E5vNpnnz5mnAgAEeu8kfagbaEtyBduR9srOztXfvXjVt2rRCv9hXNbvdrtTUVIWHhxc7VXd1Ybfb1bZtW+cwuaq0bNkyXXbZZTp16lSxwwq9wahRo3T8+HGX+4O5U3Z2tvbs2aO9e/fqyiuvdPm5VFnZgOF/VcjaKEqSFJKWaM5cUYEuYQAAAHiH/fv368cff1SvXr2Uk5OjN998U3v37i3XDHk1WUpKijZt2qRZs2ZVWqDylOr7J4BqKLSFOfzPz26TCtyRGgAAANWXj4+PZs6cqS5duqhHjx7atGmTFi9eXOLEEOeq6667TldeeaXuvvtuXXHFFZ4ux63oqapC9Rv664TqqK5OSgkJUt26ni4JAAAAFRQXF6eVK1d6ugxJUu/evd023bu71ZTp04tDT1UViomREhRjPklI8GwxAAAAANyCUFWFYmIMZ6gyjhCqAAAAgJqAUFWFoqNP91Rl7SVUAQAAADUBoaoKBQZKJ/3rS5Ky9hCqAAAAgJqAUFXFUkPMySnyDnIDYAAAAKAmIFRVscyI2uaDRHqqAAAAgJqAUFXFcmrVkiRZTxCqAAAA3K13794aO3as83mTJk306quvnvE1FotFc+fOrfC53XUcbzV58mRdcMEFbjvevn37ZLFYtGHDBrcd01MIVVUsv36EJCk4lVAFAADgMHDgQPXv37/YbT///LMsFov+/PPPMh937dq1uuuuuypanouSwkVCQkKJ78FdZs6cKYvFIovFIh8fHzVs2FAjR47UsWPHKvW8lSEuLk4JCQk6//zzJZn3sbJYLEpOTvZsYeXAzX+rmBEdLkkKsGVIaWlSWJiHKwIAAPC8UaNGafDgwTp8+LDCw8Ndts2YMUMXXXSROnToUObj1qtXz10lnlV0dHSVnCc8PFzbt2+X3W7Xxo0bNXLkSB05ckQLFy4s1/FsNpv8/f3dXOXZ+fr6VtlnVtnoqapiwfV9lKZQ8wk3AAYAAFXBMKSMDM8shlGqEq+55hrVq1dPn332mcv69PR0ffHFFxo1apROnjypIUOGqEGDBgoODlb79u2L7F9Y4eF/O3fuVM+ePRUYGKh27dpp0aJFRV4zfvx4tWrVSsHBwWrWrJmeeOIJ2Ww2SWZP0ZQpU7Rx40Znj9HMmTMlFR3+t2nTJl1++eUKCgpSnTp1dNdddyk9Pd25fcSIERo0aJBefPFFxcTEqE6dOho9erTzXCWxWCyKjo5WbGys+vfvr/vuu0+LFy9WVlaWJOn9999X27ZtFRgYqDZt2uitt95yvtYx5O7zzz9Xr169FBgYqE8//VQzZ85UZGSk5s6dq5YtWyowMFD9+vXTwYMHz1jLmc51++23q0OHDsrJyZEk5ebmqlOnTho2bJhLLRs2bNC+fft02WWXSZJq1aoli8WiESNG6KOPPlKdOnWcx3AYNGiQbrvttjPWVpXoqapitWplK0ExCtNOM1S1auXpkgAAQE2XmSmFhnrm3OnpUkjIWXfz8/PTbbfdplmzZmnKlCnO9V988YXy8/M1ZMgQpaenq3Pnzho/frzCw8P1ww8/6LbbblPz5s118cUXn/UcdrtdN9xwg6KiorRmzRqlpKS4XH/lEBYWppkzZyo2NlabNm3SnXfeqbCwMD3yyCO6+eabtXnzZi1YsECLFy+WJEVERBQ5RkZGhvr166f4+HitXbtWx44d0x133KExY8Y4Q5gkLV26VDExMVq6dKl27dqlm2++WRdccIHuvPPOs74fh6CgINntduXl5enTTz/VxIkT9eabb6pTp05av3697rzzToWEhGj48OHO1zz66KN66aWX1KlTJwUGBmrhwoXKzMzU008/rY8++khWq1X//ve/dcstt2jlypXFnvds53r99dfVsWNHPfroo3rllVf0+OOPKzk5WW+++WaRY8XFxemrr77S4MGDtX37doWHhysoKEhWq1X33Xefvv32W914442SpGPHjumHH37Qjz/+WOrPqLIRqqpY7dpmqGqlnTKOJMji6YIAAAC8xMiRI/Xiiy9q+fLluvzyyyWZQ/8GDx6siIgIRUREaNy4cc797733Xi1cuFD/+9//ShWqFi9erL/++ksLFy5UbGysJOmZZ54pch3UhAkTnI+bNGmicePGafbs2XrkkUcUFBSk0NBQ+fn5nXHo2qxZs5Sdna2PPvpIIX+HyjfffFMDBw7Uc889p6ioKElmr8ybb74pX19ftWnTRldffbWWLFlS6lC1c+dOvfPOO7rooosUFhamSZMm6aWXXtINN9wgSWratKm2bt2qd9991yVUjR071rmPg81m05tvvqmuXbtKkj788EO1bdtWv/32W7Gf79nOFRoaqk8++US9evVSWFiYXn31VS1durTI8E7JHApYu7Y5S3b9+vUVGRnp3HbrrbdqxowZzlD1ySefqFGjRurdu3epPqOqQKiqYrVr5yhBMZKknH0JCvRwPQAA4BwQHGz2GHnq3KXUpk0bXXzxxZoxY4Yuv/xy7dq1Sz///LOmTp0qScrPz9czzzyj//3vfzp8+LByc3OVk5Oj4FKeY9u2bYqLi3MGKkmKj48vst/nn3+u119/Xbt371Z6erry8vKKDQJnO1fHjh2dgUqSevToIbvdru3btztD1XnnnSdfX1/nPjExMdq0adMZj52SkqLQ0FDZ7XZlZ2frkksu0fvvv6+MjAzt3r1bo0aNcglleXl5RXrTLrrooiLH9fPzU5cuXZzP27Rpo8jISG3btq1IqCrtueLj4zVu3Dg9+eSTGj9+vC655JIzvrfi3HnnnerSpYsOHz6sBg0aaObMmRoxYoQsFu/pniBUVbGAgHwdskZLuVLGnkRCFQAAqHwWS6mG4HmD2267TePHj9dbb72lGTNmqHnz5urVq5ck6YUXXtBrr72mV199Ve3bt1dISIjGjh2r3Nxct51/1apVGjp0qKZMmaJ+/fopIiJCs2fP1ksvveS2cxRUeIIIi8Uiu91+xteEhYXpjz/+kI+Pj2JiYhQUFCRJOnr0qCTpvffec/Y2ORQMbpJcwl55OK4NO9u57Ha7Vq5cKV9fX+3atatc5+rUqZM6duyojz76SFdeeaW2bNmiH374ofzFVwJClQdkhsdIJyTbfiaqAAAAKGjQoEF67LHHNGvWLH300Ue65557nD0SK1eu1HXXXad//vOfksxf2Hfs2KF27dqV6tht27bVwYMHlZCQoJgYc+TQ6tWrXfb59ddf1bhxYz3++OPOdfv373fZx2q1Kj8//6znmjlzpjIyMpwBZuXKlfLx8VHr1q1LVW9JfHx81KJFiyLro6KiFBsbqz179mjo0KFlPm5eXp5+//13Z6/U9u3blZycrLZt25b7XC+88IL++usvLV++XP369dOMGTM0cuTIYve1Wq2SVOxne8cdd+jVV1/V4cOH1bdvX8XFxZX5/VUmZv/zgLy6Znevwex/AAAALkJDQ3XTTTfpscceU0JCgkaMGOHc1rJlSy1atEi//vqrtm3bpn/961/O3pnS6Nu3r1q1aqXhw4dr48aN+vnnn13Ck+McBw4c0OzZs7V79269/vrr+vrrr132adKkifbu3asNGzboxIkTRWamk6ShQ4cqMDBQw4cP1+bNm7V06VLde++9uu2225xD/yrDlClTNG3aNL3++uvasWOHNm3apBkzZujll18+62v9/f117733as2aNVq3bp1GjBihbt26lXi92tnOtX79ek2cOFHvv/++evTooZdffln333+/9uzZU+zxGjduLIvFou+//17Hjx93mSnx1ltv1aFDh/Tee+/p9ttvL8cnU7kIVR5gRJt/GfE/TqgCAAAo7Pbbb9epU6fUr18/l+ufJkyYoAsvvFD9+vVT7969FR0drUGDBpX6uD4+Pvr666+VlZWliy++WHfccYeefvppl32uvfZaPfDAAxozZowuuOAC/frrr3riiSdc9hk8eLCuuuoqXXbZZcVOAy9JwcHBWrhwoZKSktSlSxf94x//UJ8+fYqd+c6d7rjjDr3//vuaMWOG2rdvr169emnmzJlq2rTpWV8bHBys8ePH69Zbb1WPHj0UGhqqzz//vFznys7O1j//+U+NGDFCAwcOlCTddddduuyyy3TbbbcV2xvVoEEDTZkyRY8++qiioqI0ZswY57aIiAgNHjxYoaGhZfo3ryoWwyjlzQOqqdTUVEVERCglJaXMFxi6m81m07x587RjThM9/NEFygisrZCskx6tCdWToy0NGDDAIzfrQ81BW4I70I68T3Z2tvbu3aumTZsqMLD6XMFtt9uVmpqq8PBw+fjwt/+qNHPmTI0dO1bJycmeLqVEffr00XnnnafXX3/9jPtlZ2drz5492rt3r6688kqXn0uVlQ24psoDApuaPVUh2UlSTo4UEODhigAAAADvdOrUKS1btkzLli1zubmwNyFUeUBks1rKkVUBypUSE6XGjT1dEgAAAOCVOnXqpFOnTum5556r8CQflYVQ5QExsRYlKlqNdUBKSCBUAQAAwKNGjBjhMimIN9m3b5+nSzgrBqt6QEyM4bwBsJgBEAAAAKjWCFUeEBMjZ6jK2Z/o4WoAAEBNVcPnIwOK5Yl2T6jygLAw6YSfGaoydtFTBQAA3MvX11eSlJub6+FKgKqXmZkpqfibCFcWrqnykMzwaClJytlHqAIAAO7l5+en4OBgHT9+XP7+/tVmenK73a7c3FxlZ2dXm5rhPQzDUGZmpo4dO6bw8PAq7bEiVHlIbp0YKUkyjhCqAACAe1ksFsXExGjv3r3av3+/p8spNcMwlJWVpaCgIFksFk+Xg2oqMjJSderUqdJzEqo8xIiOkXZKvscJVQAAwP2sVqtatmxZrYYA2mw2rVixQj179uRG0igXf39/+fr6ymazVel5CVUe4hdnXlMVdIpQBQAAKoePj48CAwM9XUap+fr6Ki8vT4GBgYQqVCsMVvWQwKZmqArNPCpV4UV0AAAAANyLUOUhES3ryy6LfAy7dPy4p8sBAAAAUE6EKg+JbuinY6pvPuEGwAAAAEC1RajykNjY0zcAViI3AAYAAACqK0KVh8TEnA5VufvpqQIAAACqK0KVh0RESMd9oyVJ6bsIVQAAAEB1RajyEItFyggze6py9hKqAAAAgOqKUOVB2bXNUJV/mFAFAAAAVFeEKg+y1zdDle9RQhUAAABQXRGqPMg3zgxVAacIVQAAAEB1RajyoMAmZqgKS0uQDMPD1QAAAAAoD0KVB4W2NEOVvz1HSk72bDEAAAAAyoVQ5UHRTQJ1SpHmE24ADAAAAFRLhCoPKngDYCVwXRUAAABQHRGqPCgmRkqUeQPgvIOEKgAAAKA6IlR5UO3a0lEfs6cqbQehCgAAAKiOCFUeZLFIaaFmqMraS6gCAAAAqiNClYdlR5qhKv8QoQoAAACojghVHpZf3wxVPomEKgAAAKA6IlR5mE8DM1RZkwhVAAAAQHVEqPKwgCZmqApJJVQBAAAA1RGhysNCWpihKtiWKmVmergaAAAAAGVFqPKwes3Dlakg80liomeLAQAAAFBmhCoPi4m1OG8ArASGAAIAAADVDaHKw2JipAQxrToAAABQXRGqPKxuXSnRYoaqtB2EKgAAAKC68ZpQ9eyzz8pisWjs2LHOddnZ2Ro9erTq1Kmj0NBQDR48WEePHvVckZXAx0dKDTZDVeYeQhUAAABQ3XhFqFq7dq3effdddejQwWX9Aw88oO+++05ffPGFli9friNHjuiGG27wUJWVJyvSDFV5BwhVAAAAQHXj8VCVnp6uoUOH6r333lOtWrWc61NSUvTBBx/o5Zdf1uWXX67OnTtrxowZ+vXXX7V69WoPVux+efXMUGVJJFQBAAAA1Y2fpwsYPXq0rr76avXt21dPPfWUc/26detks9nUt29f57o2bdqoUaNGWrVqlbp161bs8XJycpSTk+N8npqaKkmy2Wyy2WyV9C5Kx3H+InXE1Jc2SH4nEjxeI6qHEtsSUEa0JbgD7QjuQluCu5TUliqrbXk0VM2ePVt//PGH1q5dW2RbYmKirFarIiMjXdZHRUUp8Qz3c5o2bZqmTJlSZP2PP/6o4ODgCtfsDosWLXJ5fsSwS5ICkw9r3rx5nigJ1VThtgSUF20J7kA7grvQluAuhdtSZmZmpZzHY6Hq4MGDuv/++7Vo0SIFBga67biPPfaYHnzwQefz1NRUxcXF6corr1R4eLjbzlMeNptNixYt0hVXXCF/f3/n+lPbj0sLpIjcJA248krJz+MdiPByJbUloKxoS3AH2hHchbYEdympLTlGsbmbx357X7dunY4dO6YLL7zQuS4/P18rVqzQm2++qYULFyo3N1fJyckuvVVHjx5VdHR0iccNCAhQQEBAkfX+/v5e881ZuJZ6baOVJ1/5KV8+SUlSgwYerA7ViTe1a1RvtCW4A+0I7kJbgrsUbkuV1a48NlFFnz59tGnTJm3YsMG5XHTRRRo6dKjzsb+/v5YsWeJ8zfbt23XgwAHFx8d7quxKEdPAR0cVZT5JYLIKAAAAoDrxWE9VWFiYzj//fJd1ISEhqlOnjnP9qFGj9OCDD6p27doKDw/Xvffeq/j4+BInqaiuYmKkw4pRAx1R/qEE+V7k6YoAAAAAlJZXX7zzyiuvyMfHR4MHD1ZOTo769eunt956y9NluV39+tI6mdOqp+9MUISH6wEAAABQel4VqpYtW+byPDAwUNOnT9f06dM9U1AV8fWVkoNipCwpYzehCgAAAKhOPH7zX5gyI8yeKtt+rqkCAAAAqhNClZew1TVDFRNVAAAAANULocpbxJihyu9EyTc2BgAAAOB9CFVewr+RGaqCUuipAgAAAKoTQpWXCG5m3tA4LCNRMgwPVwMAAACgtAhVXiKitRmq/A2bdPKkh6sBAAAAUFqEKi8R3ciqE6pjPmGyCgAAAKDaIFR5iZgYKeHvGwDbDxOqAAAAgOqCUOUloqJOh6r0nYQqAAAAoLogVHkJf3/pVKAZqjJ2EaoAAACA6oJQ5UUywsxQlbOfUAUAAABUF4QqL5JbxwxVxhFuAAwAAABUF4QqL2JEm6HK9xg9VQAAAEB1QajyIv5x5r2qApMJVQAAAEB1QajyIoFNzZ6qsHRCFQAAAFBdEKq8SHhrM1QF5WdIaWkergYAAABAaRCqvEj9ZqFKU6j5JIHeKgAAAKA6IFR5kZiY0zcANo4QqgAAAIDqgFDlRaKjT4eq9J2EKgAAAKA6IFR5kYAAKclKqAIAAACqE0KVl0kPM0NV9n5uAAwAAABUB4QqL5Nd2wxV9kP0VAEAAADVAaHKyxhR5g2AfY4RqgAAAIDqgFDlZXwbmD1VgUmEKgAAAKA6IFR5mYAmZqgKTSdUAQAAANUBocrLhLc2Q1VYbpKUk+PhagAAAACcDaHKy9RpWVs5sppPEpkBEAAAAPB2hCovExNrUaLMySqMIwwBBAAAALwdocrLxMRICTKHAGbuJlQBAAAA3o5Q5WWCgqQT/maoSt3J8D8AAADA2xGqvFBaiBmqsvfQUwUAAAB4O0KVF8qpZV5TlXeIUAUAAAB4O0KVF8qvb/ZU+SQSqgAAAABvR6jyQj4NzFBlTSJUAQAAAN6OUOWFrI3NUBWaSqgCAAAAvB2hyguFtjRDVXj2USk/38PVAAAAADgTQpUXqtW6vuyyyFd26fhxT5cDAAAA4AwIVV4oJs5Px1TffJLAEEAAAADAmxGqvFBMjJQgcwhg5h5uAAwAAAB4M0KVFwoNlY77mqEqdTs9VQAAAIA3I1R5qbQQ8wbAWXsIVQAAAIA3I1R5qaxIs6fKdoBQBQAAAHgzQpWXyqtnhipLIqEKAAAA8GaEKm8VY4Yq/5OEKgAAAMCbEaq8lLWxGaqCUwhVAAAAgDcjVHmpkBZmqIrMSpAMw8PVAAAAACgJocpLRbY1Q5XVniMlJ3u2GAAAAAAlIlR5qajGgTqlSPNJIjcABgAAALwVocpLxcZKCTJ7q7L3cl0VAAAA4K0IVV4qLEw67mPeADjlL0IVAAAA4K0IVV7KYpGSg82eqszdhCoAAADAWxGqvFhWhBmqcvcTqgAAAABvRajyYra6ZqhSAqEKAAAA8FaEKi9mRJuhyu8EoQoAAADwVoQqL+bfyAxVQcmEKgAAAMBbEaq8WHBzM1RFZBCqAAAAAG9FqPJi4W3MUBWSnyplZnq4GgAAAADFIVR5sfrNw5WpIPNJYqJniwEAAABQLEKVF4ttYFGizBsA5+xjCCAAAADgjQhVXiwyUjpqMYcApvxFqAIAAAC8EaHKi1ksUnKQGaoydhGqAAAAAG9EqPJyGeFmqGL4HwAAAOCdCFVeLreOGaqMI4QqAAAAwBsRqrycPcoMVb7HCVUAAACANyJUeTm/ODNUBZ4iVAEAAADeiFDl5YKamaEqLIP7VAEAAADeiFDl5cJbm6EqIve4lJfn4WoAAAAAFEao8nJ1WtdVnnzlI0M6etTT5QAAAAAopFyhasaMGcrMzHR3LShGbEMfHVWUJMl2gOuqAAAAAG9TrlD16KOPKjo6WqNGjdKvv/5a7pO//fbb6tChg8LDwxUeHq74+HjNnz/fuT07O1ujR49WnTp1FBoaqsGDB+voOdZbU6eOlGgxhwAmbyNUAQAAAN6mXKHq8OHD+vDDD3XixAn17t1bbdq00XPPPafExLJNptCwYUM9++yzWrdunX7//Xddfvnluu6667RlyxZJ0gMPPKDvvvtOX3zxhZYvX64jR47ohhtuKE/J1ZbFIp0KMENV+k5CFQAAAOBtyhWq/Pz8dP311+ubb77RwYMHdeedd+rTTz9Vo0aNdO211+qbb76R3W4/63EGDhyoAQMGqGXLlmrVqpWefvpphYaGavXq1UpJSdEHH3ygl19+WZdffrk6d+6sGTNm6Ndff9Xq1avLU3a1lR5mhqrsfYQqAAAAwNv4VfQAUVFRuuSSS7Rjxw7t2LFDmzZt0vDhw1WrVi3NmDFDvXv3LtVx8vPz9cUXXygjI0Px8fFat26dbDab+vbt69ynTZs2atSokVatWqVu3boVe5ycnBzl5OQ4n6empkqSbDabbDZb+d+oGzjOX9Y6smtFS8el/INHPP4e4B3K25aAwmhLcAfaEdyFtgR3KaktVVbbKneoOnr0qD7++GPNmDFDe/bs0aBBg/T999+rb9++ysjI0NSpUzV8+HDt37//jMfZtGmT4uPjlZ2drdDQUH399ddq166dNmzYIKvVqsjISJf9o6KizjjMcNq0aZoyZUqR9T/++KOCg4PL9V7dbdGiRWXa/7ifVZKUtXeX5s2bVxkloZoqa1sCSkJbgjvQjuAutCW4S+G2VFmT7ZUrVA0cOFALFy5Uq1atdOedd2rYsGGqXbu2c3tISIgeeughvfDCC2c9VuvWrbVhwwalpKToyy+/1PDhw7V8+fLylCVJeuyxx/Tggw86n6empiouLk5XXnmlwsPDy31cd7DZbFq0aJGuuOIK+fv7l/p1X822S1ulOrnpumDAgEqsENVFedsSUBhtCe5AO4K70JbgLiW1JccoNncrV6iqX7++li9frvj4+BL3qVevnvbu3XvWY1mtVrVo0UKS1LlzZ61du1avvfaabr75ZuXm5io5Odmlt+ro0aOKjo4u8XgBAQEKCAgost7f399rvjnLWktQswaSpND0BK95D/AO3tSuUb3RluAOtCO4C20J7lK4LVVWuyrXRBW9evXShRdeWGR9bm6uPvroI0mSxWJR48aNy3xsu92unJwcde7cWf7+/lqyZIlz2/bt23XgwIEzhrmaKKylGSJr5SRKhuHhagAAAAAUVK5QNXLkSKWkpBRZn5aWppEjR5b6OI899phWrFihffv2adOmTXrssce0bNkyDR06VBERERo1apQefPBBLV26VOvWrdPIkSMVHx9f4iQVNVXtdmao8jds0smTHq4GAAAAQEHlGv5nGIYsFkuR9YcOHVJERESpj3Ps2DENGzZMCQkJioiIUIcOHbRw4UJdccUVkqRXXnlFPj4+Gjx4sHJyctSvXz+99dZb5Sm5WotpbNUJ1VFdnVTewQT51a3r6ZIAAAAA/K1MoapTp06yWCyyWCzq06eP/PxOvzw/P1979+7VVVddVerjffDBB2fcHhgYqOnTp2v69OllKbPGqVdP2qIY1dVJpfyVoDqd2nu6JAAAAAB/K1OoGjRokCRpw4YN6tevn0JDQ53brFarmjRposGDB7u1QEg+PlJSQIyUs1mpOxJUx9MFAQAAAHAqU6iaNGmSJKlJkya6+eabFRgYWClFoai00BgpR8rek+DpUgAAAAAUUK5rqoYPH+7uOnAW2bVipJNS3qGSb3wMAAAAoOqVOlTVrl1bO3bsUN26dVWrVq1iJ6pwSEpKcktxOC2/Xoy0S7IcpacKAAAA8CalDlWvvPKKwsLCnI/PFKrgfj4NYiRJAScJVQAAAIA3KXWoKjjkb8SIEZVRC84goLF5r6qQNEIVAAAA4E3KdfPfmTNnFrs+Ly9Pjz32WEXqQQlCW5o9VZFZhCoAAADAm5QrVN1333268cYbderUKee67du3q2vXrvrss8/cVhxOi2xrhqpge4aUlubhagAAAAA4lCtUrV+/XocOHVL79u21aNEiTZ8+XRdeeKHatGmjjRs3urtGSIpuEao0mfcFyz9EbxUAAADgLco1pXrz5s21cuVKjR07VldddZV8fX314YcfasiQIe6uD3+rX1/aoxiFaadS/kpQ7batPF0SAAAAAJWzp0qSfvjhB82ePVvx8fGKjIzUBx98oCNHjrizNhTg5yed9DeHAKZsp6cKAAAA8BblClX/+te/dOONN2r8+PH6+eef9eeff8pqtap9+/b63//+5+4a8bfUEDNUZe3hBsAAAACAtyhXqFq5cqXWrFmjhx56SBaLRdHR0Zo3b56mTp2q22+/3d014m9ZkWaoyjtITxUAAADgLcp1TdW6desUEBBQZP3o0aPVt2/fCheF4uXVi5H2SUogVAEAAADeolw9VQEBAdq9e7cmTJigIUOG6NixY5Kk+fPnKy8vz60F4jRLjHkDYP+ThCoAAADAW5QrVC1fvlzt27fXmjVrNGfOHKWnp0uSNm7cqEmTJrm1QJxmbWwO/wtJIVQBAAAA3qJcoerRRx/VU089pUWLFslqtTrXX3755Vq9erXbioOr4OZmqIrIIlQBAAAA3qJcoWrTpk26/vrri6yvX7++Tpw4UeGiULzItn+HqrwkKSfHw9UAAAAAkMoZqiIjI5VQzGQJ69evV4MGDSpcFIpXv01t5cjsGTQSmFYdAAAA8AblClW33HKLxo8fr8TERFksFtntdq1cuVLjxo3TsGHD3F0j/hYVbVGizMkqUv5iCCAAAADgDcoVqp555hm1adNGcXFxSk9PV7t27dSzZ091795dEyZMcHeN+JvVKh33M4cAJv9FTxUAAADgDcp1nyqr1ar33ntPTzzxhDZv3qz09HR16tRJLVu2dHd9KCQ1OEZKlTJ301MFAAAAeINyhSqHRo0aqVGjRu6qBaWQEWGGqtwDhCoAAADAG5Q6VD344IOlPujLL79crmJwdnl1oqWDknGEUAUAAAB4g1KHqvXr15dqP4vFUu5iUAoxMdIGyf84oQoAAADwBqUOVUuXLq3MOlBK/o3MiSqCUghVAAAAgDco1+x/BR08eFAHDx50Ry0ohaBmf98AOINQBQAAAHiDcoWqvLw8PfHEE4qIiFCTJk3UpEkTRUREaMKECbLZbO6uEQVEtDFDVS3bUSk/38PVAAAAACjX7H/33nuv5syZo+eff17x8fGSpFWrVmny5Mk6efKk3n77bbcWidPqtqsvuyzylV3GseOyxER7uiQAAADgnFauUDVr1izNnj1b/fv3d67r0KGD4uLiNGTIEEJVJYpu6Kdjqq9oHVXazkSFE6oAAAAAjyrX8L+AgAA1adKkyPqmTZvKarVWtCacQWCgdMzXHAJ4aivXVQEAAACeVq5QNWbMGD355JPKyclxrsvJydHTTz+tMWPGuK04FC85yAxVGbsIVQAAAICnlWv43/r167VkyRI1bNhQHTt2lCRt3LhRubm56tOnj2644QbnvnPmzHFPpXDKDIuW0qWc/YQqAAAAwNPKFaoiIyM1ePBgl3VxcXFuKQhnl1snRkqQ7IcJVQAAAICnlTlUGYahKVOmqF69egoKCqqMmnAWRnSMtFnyO0aoAgAAADytzNdUGYahFi1a6NChQ5VRD0rBL868piowmVAFAAAAeFqZQ5WPj49atmypkydPVkY9KIXApmaoCk8nVAEAAACeVq7Z/5599lk9/PDD2rx5s7vrQSmEtzZDVe3cBMkwPFwNAAAAcG4r10QVw4YNU2Zmpjp27Cir1Vrk2qqkpCS3FIfi1TnfDFUBRo6M5BRZakV6tiAAAADgHFauUPXqq6+6uQyURXSTQJ1SpGopWRm7EhTaJdLTJQEAAADnrHKFquHDh7u7DpRBcLB0wCdGtezJStqSoNAubT1dEgAAAHDOKtc1VZK0e/duTZgwQUOGDNGxY8ckSfPnz9eWLVvcVhxKlhwYLUlK38lkFQAAAIAnlStULV++XO3bt9eaNWs0Z84cpaenS5I2btyoSZMmubVAFC89zLyuKnsfoQoAAADwpHKFqkcffVRPPfWUFi1aJKvV6lx/+eWXa/Xq1W4rDiXLqWWGKvshQhUAAADgSeUKVZs2bdL1119fZH39+vV14sSJCheFs7NHmaHK5yihCgAAAPCkcoWqyMhIJSQU/WV+/fr1atCgQYWLwtn5Nvx7WvVThCoAAADAk8oVqm655RaNHz9eiYmJslgsstvtWrlypcaNG6dhw4a5u0YUI6CJGarC0ghVAAAAgCeVK1Q988wzatu2rRo1aqT09HS1a9dOPXv2VPfu3TVhwgR314hihLUyQ1VkTqKHKwEAAADObWW6T5XdbtcLL7ygb7/9Vrm5ubrttts0ePBgpaenq1OnTmrZsmVl1YlCap9nhqpwe4qUlSUFBXm4IgAAAODcVKZQ9fTTT2vy5Mnq27evgoKCNGvWLBmGof/+97+VVR9KENUyXJkKUrCylLErQSHtm3m6JAAAAOCcVKbhfx999JHeeustLVy4UHPnztV3332nTz/9VHa7vbLqQwnCwi06ajFvAJy0heuqAAAAAE8pU6g6cOCABgwY4Hzet29fWSwWHTlyxO2F4eySAswhgGk7CFUAAACAp5QpVOXl5SkwMNBlnb+/v2w2m1uLQumkhZqhKnsvoQoAAADwlDJdU2UYhkaMGKGAgADnuuzsbN19990KCQlxrpszZ477KkSJsiNjpBNS3kFCFQAAAOApZQpVw4cPL7Lun//8p9uKQdnkR8VIuyTLUUIVAAAA4CllClUzZsyorDpQDj6x5vC/gJOEKgAAAMBTynXzX3gHaxMzVAWncQNgAAAAwFMIVdVYWEszVEVm0VMFAAAAeAqhqhqLbGuGqtr5x6W8PA9XAwAAAJybCFXVWP12dZUnX/nIUNa+o54uBwAAADgnEaqqsYhaPjqmKEnSyc0MAQQAAAA8gVBVjVks0kmrOQQwdTuhCgAAAPAEQlU1lxpihqqsPYQqAAAAwBMIVdVcVqQZqmwHCFUAAACAJxCqqrm8emaosiQSqgAAAABPIFRVd7FmqPI/yQ2AAQAAAE8gVFVz1kZmqApOoacKAAAA8ARCVTUX0sIMVRFZhCoAAADAEzwaqqZNm6YuXbooLCxM9evX16BBg7R9+3aXfbKzszV69GjVqVNHoaGhGjx4sI4e5Ua3DpFtoiVJdWyJkmF4uBoAAADg3OPRULV8+XKNHj1aq1ev1qJFi2Sz2XTllVcqIyPDuc8DDzyg7777Tl988YWWL1+uI0eO6IYbbvBg1d6l7vlmqLLKppwjJz1cDQAAAHDu8fPkyRcsWODyfObMmapfv77WrVunnj17KiUlRR988IFmzZqlyy+/XJI0Y8YMtW3bVqtXr1a3bt08UbZXqR1t1QnVUV2d1MnNCYptUNfTJQEAAADnFI+GqsJSUlIkSbVr15YkrVu3TjabTX379nXu06ZNGzVq1EirVq0qNlTl5OQoJyfH+Tw1NVWSZLPZZLPZKrP8s3Kc3911nPCPUV3bSSVtOaR6l7dx67HhnSqrLeHcQ1uCO9CO4C60JbhLSW2pstqW14Qqu92usWPHqkePHjr//PMlSYmJibJarYqMjHTZNyoqSomJxU8hPm3aNE2ZMqXI+h9//FHBwcFur7s8Fi1a5Nbj1fWvK9mkbT+t1u7muW49Nrybu9sSzl20JbgD7QjuQluCuxRuS5mZmZVyHq8JVaNHj9bmzZv1yy+/VOg4jz32mB588EHn89TUVMXFxenKK69UeHh4RcusEJvNpkWLFumKK66Qv7+/2477c52vpEwp1hKgiwcMcNtx4b0qqy3h3ENbgjvQjuAutCW4S0ltyTGKzd28IlSNGTNG33//vVasWKGGDRs610dHRys3N1fJyckuvVVHjx5VdHR0sccKCAhQQEBAkfX+/v5e883p7lry6sVKByXL0WNe8x5RNbypXaN6oy3BHWhHcBfaEtylcFuqrHbl0dn/DMPQmDFj9PXXX+unn35S06ZNXbZ37txZ/v7+WrJkiXPd9u3bdeDAAcXHx1d1ud4r2rxXld8J7lUFAAAAVDWP9lSNHj1as2bN0jfffKOwsDDndVIREREKCgpSRESERo0apQcffFC1a9dWeHi47r33XsXHxzPzXwH+jcxQFZRMqAIAAACqmkdD1dtvvy1J6t27t8v6GTNmaMSIEZKkV155RT4+Pho8eLBycnLUr18/vfXWW1VcqXcLbmYOhYzIIFQBAAAAVc2jocowjLPuExgYqOnTp2v69OlVUFH1FN7a7KmqYyNUAQAAAFXNo9dUwT3qtjdDVYiRIVtSmoerAQAAAM4thKoaoE7jUKUpVJJ0cjO9VQAAAEBVIlTVAD4+0nE/s7cqeRuhCgAAAKhKhKoaIjnIDFXpOwlVAAAAQFUiVNUQGWFmqMrZn+jhSgAAAIBzC6GqhsitY4Yq4wg9VQAAAEBVIlTVEEa0Gap8jxGqAAAAgKpEqKoh/BqaNwAOTCZUAQAAAFWJUFVDBDUze6rC0glVAAAAQFUiVNUQYa3MUFU7h1AFAAAAVCVCVQ1R+7y/Q5WRpPzMHA9XAwAAAJw7CFU1RL3WtZUjqyTp5BamVQcAAACqCqGqhvD1s+i4rzlZRdJWhgACAAAAVYVQVYOcCjCHAKbtoKcKAAAAqCqEqhokPcwMVTn76KkCAAAAqgqhqgbJqW2GKvthQhUAAABQVQhVNYi9vnlNlc9RQhUAAABQVQhVNYhvQ7OnKuAUoQoAAACoKoSqGiSwqRmqQtMIVQAAAEBVIVTVIKEt/74BcDahCgAAAKgqhKoapFY7M1TVtR+V3Zbv4WoAAACAcwOhqgapd1592WWRr+xK2n7c0+UAAAAA5wRCVQ3iH+SnEz71JUknt3ADYAAAAKAqEKpqmCSrOQQwdTvXVQEAAABVgVBVw6SFmaEqey+hCgAAAKgKhKoaJifSvAFw/iFCFQAAAFAVCFU1TF49s6fKkkioAgAAAKoCoaqG8WlghiprEqEKAAAAqAqEqhomoIkZqkJTCVUAAABAVSBU1TAhLcxQFZlFqAIAAACqAqGqhqnVzgxV9fITZNgND1cDAAAA1HyEqhqmbnszVAUqR6f2pXi4GgAAAKDmI1TVMAERgUq2REqSTm5mCCAAAABQ2QhVNdBJq9lblfIXoQoAAACobISqGigt2LwBcOZuQhUAAABQ2QhVNVBWpNlTlXeQUAUAAABUNkJVDWSrZ4YqJRCqAAAAgMpGqKqBLDFmqLKeJFQBAAAAlY1QVQNZG5uhKjiFUAUAAABUNkJVDRTSwgxVEVmEKgAAAKCyEapqoIg2ZqiqY0uUYXi4GAAAAKCGI1TVQHXbm6EqUilKSczycDUAAABAzUaoqoGCosKVqSBJ0olNDAEEAAAAKhOhqiayWHTS37wBcPI2QhUAAABQmQhVNVRKsDkEMHM3oQoAAACoTISqGiozwgxVuQcIVQAAAEBlIlTVULY6ZqgyjhCqAAAAgMpEqKqpYsxQ5X+cUAUAAABUJkJVDeXfyAxVQSmEKgAAAKAyEapqqKBmZqgKz0j0cCUAAABAzUaoqqHCW5uhqk4uPVUAAABAZSJU1VB125uhqq6OK+1UnoerAQAAAGouQlUNFdK4rvLkKx8ZOr75qKfLAQAAAGosQlVN5eOjk75RkqRTWxkCCAAAAFQWQlUNlhxkDgFM30moAgAAACoLoaoGywg3Q1XufkIVAAAAUFkIVTVYTh0zVNkPE6oAAACAykKoqsmizVDld5xQBQAAAFQWQlUN5tvQDFUBydwAGAAAAKgshKoaLKiZGarC0umpAgAAACoLoaoGC2tlhqraOYQqAAAAoLIQqmqwOudFS5KijERlpBsergYAAAComQhVNVhoCzNUWWXT0a0nPVwNAAAAUDMRqmowS4BVST51JEmntjIEEAAAAKgMhKoaLjnQvK4qfSehCgAAAKgMhKoaLi3s71C1i1AFAAAAVAZCVQ1nq2uGqqz/fa+HRxzXsWMeLggAAACoYQhVNVyrWy6UJP1DX2rKh401p+F9emv8fmVne7gwAAAAoIYgVNVw4Y/fJ335pdJad1awsnS37Q3d+XwL/VB3uOa9sEUGM60DAAAAFeLRULVixQoNHDhQsbGxslgsmjt3rst2wzA0ceJExcTEKCgoSH379tXOnTs9U2x1ZbFIgwcrbNta2X9crITz+shfeRqc8ZEGPHK+fq5znTb93ypPVwkAAABUWx4NVRkZGerYsaOmT59e7Pbnn39er7/+ut555x2tWbNGISEh6tevn7IZu1Z2Fot8ruijmM2Llb3iN21rN1h2WdTz1Ldq/6/u2lK/txJnLhBdVwAAAEDZeDRU9e/fX0899ZSuv/76ItsMw9Crr76qCRMm6LrrrlOHDh300Ucf6ciRI0V6tFA2gZd2UdstX+rEim36pdXtypW/zju+XNEj++tIVCdl/ne2lJfn6TIBAACAasHP0wWUZO/evUpMTFTfvn2d6yIiItS1a1etWrVKt9xyS7Gvy8nJUU5OjvN5amqqJMlms8lms1Vu0WfhOL+n63Co1a2Zum5+R1t/nKAd97yh/gf/T7HHN0qjhijlkccVNOkhWUbcJgUGerpUFOJtbQnVF20J7kA7grvQluAuJbWlympbFsPwjvFeFotFX3/9tQYNGiRJ+vXXX9WjRw8dOXJEMTExzv1uuukmWSwWff7558UeZ/LkyZoyZUqR9bNmzVJwcHCl1F4TGIa09ZdAWd/7WSNS31FdnZQkpYXU1oEbrtG+q/opLyTEw1UCAAAA5ZeZmalbb71VKSkpCg8Pd9txa1yoKq6nKi4uTidOnHDrB1ceNptNixYt0hVXXCF/f3+P1lISm0368K0sHZw8Q3dnvKRGOihJyg8Jl0bfLfu990pRUR6uEtWhLaF6oC3BHWhHcBfaEtylpLaUmpqqunXruj1Uee3wv+joaEnS0aNHXULV0aNHdcEFF5T4uoCAAAUEBBRZ7+/v7zXfnN5US2H+/tI94/yVfMf9evbJf+vYa5/pofzndF7GVun55+Xz2muy3H67NG6c1KyZp8s953lzW0L1QluCO9CO4C60JbhL4bZUWe3Ka+9T1bRpU0VHR2vJkiXOdampqVqzZo3i4+M9WNm5ITJSevYlfz2xc5ievHGTrtNcrVZXWXJypLffltGypTR0qPTnn54uFQAAAPAoj4aq9PR0bdiwQRs2bJBkTk6xYcMGHThwQBaLRWPHjtVTTz2lb7/9Vps2bdKwYcMUGxvrHCKIyte0qTT7fz4av/I6jb14lXppmebrKlnsdmnWLKljR+nqq6Wff2Y6dgAAAJyTPBqqfv/9d3Xq1EmdOnWSJD344IPq1KmTJk6cKEl65JFHdO+99+quu+5Sly5dlJ6ergULFiiQ2eiqXPfu0qrVFt3zWS/d3Wi+OukPfaZblC8fad48qWdP6ZJLpO++k+x2T5cLAAAAVBmPhqrevXvLMIwiy8yZMyWZk1dMnTpViYmJys7O1uLFi9WqVStPlnxOs1ikW26R/vpLunlaJ/0r7DO11na9o3/J5mOVfv1VuvZas/fqk0/MWS8AAACAGs5rr6mC9woKkh59VNq1S7ri7hYa7fOOGtn36QWf8cq2hkmbN0u33SaFh0vdukljxkgzZpjXX3FTYQAAANQwhCqUW/360ttvSxs3ShdcFaNH7M8qOveAngx6RhlhUVJ2trRmjTR9unT77WYPVlgYQQsAAAA1itdOqY7q4/zzpfnzpQULpHHjIjVxy2OapPHqGLJbQ1r+rsvC16l1xjqF7fxDltRUM2itWXP6AIGBZuC66CKpc2dzaddO8qN5AgAAwPvxWyvc5qqrpL59pf/+V5o0yUcbEltqw4aWkoZIkiLC7Lqp925dG/u7LrKsU9ShdbKs/0MqKWhdcMHpkEXQAgAAgJfiN1S4lZ+fdNdd0qhR5rDAZcvMZcUKKSXFR+8ta6n3ZAat8HCp16V2DTp/l3qHrVOTk+vks36d9MffQWv1anNxIGgBAADAC/HbKCqFr6904YXm8uCDUn5+cSFL+u4HH333QytJrRQePkQ9e0q9J9jVr/kutcv6O2St+3tJSzt70OrY0by5VmSkOV0hAAAAUMkIVagSxYWsDRtcQ1ZqqvT999L33/tonFopPLyVevYcot7XSL2ft+uC0F3y3fB3wPr9d7NHq7igJZkTYjRpIjVuXPRr48ZSvXqELgAAALgFoQoe4et7unPpoYfOFrIkyUcREa3Us2cr9e49RL1fljq2t8t3767TIWvdOmnrVun4cTNsbdpkLsUJCio5cDVpIkVHSz5MjgkAAICzI1TBK5QmZKWkSN99Zy6SFBHh83fIaqXeQ4eo4/PmcZSZKR04IO3bJ+3ff/qr43FCgpSVZd7F+K+/ii/IapUaNSo5eDVowLVcAAAAkESogpcqX8iSevSQOnQIVvv2bdS+fRu1vtzMRy5ycqSDB4sPXPv3S4cOSbm55t2Nd+0qucCGDc2g1aSJ1KKF1KqVubRoIYWGVs4HAwAAAK9DqEK1UDhk5eW5hqyffzZD1rx55uLg7y+1aSO1b19wCVBc8xaytGhR/Mny8qTDh12DVsHwtX+/ZLOdfrx8edFjxMaeDlkFl6ZNi0l5AAAAqM4IVaiW/PzMewVfdJE0bpyZg9avl9aulf7807yUavNm87qs4i6tiogwb1rsCFodOpjPIyP/PrhjmF/PnkVPbrdLiYmng9bevdLOndKOHeZy4oR05Ii5LFvm+lpfX7Nnq7jA1bAh13EBAABUQ4Qq1Ah+flKXLubiYBjmpVWOUOVY/vrL7NVaudJcCoqLK9yrZfZ0uXQu+fiYPVGxsVL37kWLOXXKNWQVXDIypN27zWX+fNfXBQa6DiMsuNSty2yFAAAAXopQhRrLYjnd4XTNNafX5+aawapw2Dp48PRScAihn5/UurVr0OrQwZzHoticU6uWdPHF5lKQYZg9XMWFrd27pexss3tt8+aix4yIcAlZlmbNVPvAAbN3q149c3tYGD1dAAAAHkCowjnHajVDUYcOruuTk8084whZjmGEqanSli3mMnv26f3Dw08PIWzXzlzatjU7sIoNWxaLFBNjLr16uW7LyzOHEu7YUbSX68ABs2tt7VpzkfmNe6kkPf646/HDw80xjBERpf9a8HFgYAU+WQAAgHMToQr4W2SkdMkl5uJgGGbPVXFDCFNTpV9/NZeCwsPNcOUIWY6vTZqcoSPJz09q3txc+vd33ZaVZfZkFQhc9u3blbl3r0Ly8mRJTja73wzDDF8pKeX/EAICzh7EIiPN3rjivgYFlf/cAAAA1RShCjgDi8Uc5teokXT11afX5+aaGccRsrZtM+87vHu3GbbWrDGXgoKCzGGEhcNWixbmLIUlCgoyu8TOP9+5Kt9m05J58zRgwAD5+/ubQweTk81AVZ6vqanmgXNypGPHzKU8AgJcQ9aZAljhrxERf99oDAAAoHohVAHlYLWezjlDhpxen5NjdiY5Qpbj6/btZofThg3mUpCfn9SyZdGw1bp1GTp+AgOl6GhzKY/8fCkt7ezhq+By6pTrV8MwP4CjR82lPBzDFwsHrtq1pTp1Sv4aHMxEHgAAwGMIVYAbBQQU6VSSZF4ytXevGbIKBq5t26T09NOPC7JYzNtaOUJWwcDl9lF2vr6ne5YaNy776+12M5SVFLgcX0valplpHic11VwOHCjb+a3W0yHrbAGs4FeGKwIAADcgVAFVwNEb1bKldO21p9cbhnTokGuvluNrUpK0Z4+5fP+96/EaNPBTePglmjHDV1FRUv365iSAhb/WrWueu9L5+Jye9KI8oSw31+wNKy6EJSWZz0+eNB8X/HrypHkj5txcKSHBXMoiKKj4sFW7tjmbYkBA0cVqLX59Sdv8/elFAwCghiNUAR5ksZj3xoqLk/r1O73eMKTjx4sGrW3bzHsKHz5s0eHDdYr0bhWnTp3iA1dxX2vX9tBlTVarWUS9emV7nWGY9/4qHLZK8zU/3xyTefiwuVSm0gaxwEAz6AUHm0vBx4Wfn23bGS/UAwAA7kSoAryQxWKGnPr1i86+bk79nqfvvvtDcXGdlZTk65xb4vjx019PnDAzh6ND56+/zn5eH5+zhzBHD1jdumYI8+jv7haLFBpqLo0alf51hmEOM0xKKjl4paeb14jl5Jg9YY7HBZeS1ufnu54vN9dc0tLc+/7PxM+vVGHMNyBA7RMS5PPTT2aw8/Mz/1ELL2Vdf7bX+Pp6pgfP399z5wYA1FiEKqCaiYyUunY1dPJkggYMsMvfv/iupfx8Mx8UDFuFg1fBr0lJ5qVRx4+by9atpa/HEbIcS8HgVXiJjPSCexRbLKeHKzZt6v7j5+eXPYjl5pqzOGZlmdeYZWa6Pi78vKTHhmHWkJd3+hq1M/CR1Mz9n4D3s1rNgOX4WtLj0q4723ZHwCwYNAuvO9O2ktb5+BAQAcALEKqAGsrXt2wj6mw2s4PmTMHr2DFznxMnzBBmGKcvfdq1q3TncfSGnS18FVxCQ6vZ742+vmaPUFVPhOGYgbEMwSw/LU27tm5Vi8aN5Wu3mw0hL8/8WtJS3u2Fe/A8ydF7mJHh6UoqrrjwFRAghYSYS2jo6cfFPS/NuuBgL/hriIfl5Z3+3rHbT38+3AoCgAhVAP7m71+2Wdnz8835I06cOD3c8GxLaqprb1hprgmTzN9ZCt57uLjHZ1oXHl5FE3Z4msViXpcVGGhOR18KdptNf82bp2YDBsi3ssdyGsbpwJWfX/VJ2TBOB7zc3DM/Lu260rymYMh0PC7PupJCqWN7Vlblfn5BQSWGL9+gIF1w8qR85s8/fa1g4Z47d6/z8zPbkOOPCZmZZkgu+EcEdy65ucV/LsHB5ucSFlaxr47HAQHV7K9IACRCFYBy8vU93ZPUpk3pXpOba/Z0lTaEHT9ujohzDGVMSip/vaGhJYev4oKY4/ecgn+w54/1FWSxnO5NQdk5QmlpAll2thkwCi7p6eVb55CVZS7HjxcpzUdSY0lasqSqPg2Tv7/5fh3DXquCxWIudrv53BG6ynvT9ML8/FxDluNrSMjpIZ++vqeXws+LW1eR55L5+ToWu730z8uy79/PffLy1GbXLvmsXGme2243/xMo6euZtpVlH7vd/OwdAb6kQF/WbWfb1zFJUcGluvxHYxjmz4Qz/TwpuDRp4npzzxqGUAWgylitUkyMuZRWRkbJ9yEu7nHhdY5bYKWnm8uhQxV7D8HBxY+OKsvj4tZZrfxxGmdRMJRW1dBSu931l6YSfnnKT03V9j/+UOumTc1hpAV76kp6XNp1jsd5eUXrs9lcn/v7u86K6fhriDsWx7ECAsxz5eSYk8+kp5f+65m2OX5Y5eWd/iF2DvKV1NrTRXia1eoasoKCigavwutKs49jna/v6Z7d0vyhpaSl4HW8pXHVVYQqAPAUR/ho0KB8r3fcAqssoezUqdO//xT+Y73jj9LF/LG+Qnx9T//O5rgcyzFRX0nPy7tvYCABDqXk43P6m/AM7Dabds6bp5aVOYzUbjcDR+HQ5QhSVX0rAccvqWW9FURJ8vPNHzZnCmR5ea69L4V7Ycq6rjT7SOYPDMekKI6l4PPSbivFfvmGof3796tx8+ZmW3L0mhXsPSu8zh1ffXxO9/g62lhp/jhQ+HFZXuOYpCgry3V4r2Ofs0w05FWCgooODS68dOzo6SorFaEKQI1W3ltgFVT4j/UFw1ZFHzsu08jPL9VkfW5zOnz5yWbrq7AwP/n6np7tvPDjM20rz2v8/c3fRwveoqssjx1LdRklAzfw8Tk9ZKom8vU1LwAND/d0JR5lt9m0ad48xVXFdZ7exDFs17FkZZXteVlek5fnOuyiNBPXnGkJDmbCFhGqAOCsSvnH+nKx2VxHUzguW3FM1Ff48Zm2nW2/gn8IdayXLJJC3HY5SFXz9y97ECvusoYzXQ5R2m3FbXfMpQAAZ1TwWjpUS4QqAPAgf//Tk2VUNscEcQUDV1qaTcuXr1LXrt1lsfgpP9/8I2bB0T8Fn59pW1n2LXy7ruzs0j8u/J5sNrPXz1uVFMbOdm17RdaVdP/l4u7HfKZ9CIQAUDqEKgA4Rzh+US44ushmkxITT6l7d6NaTMrnmBW9pOB1tlBW+NKH4i6HqOi2wqrzLbEcwzbPFsR8ff2UkXGpnn/eV35+ci4Fh32WtK682xyPHUNAC16q41iKW++OfX18ivaAFtc7Sk8lcO4gVAEAqg2Lxbsvqyl4Ky5Hb9yZrmcv7bryvK6k+zCXdG/m4ibxcvQsFu4hLMoiqbZ27KiED7Uas1hKDl+lWQoGNEebLzgDeUkzk59pKcu+hmEGyOLuL11SyK7ofpKUleWn9HTzPTvmsihpvgvAWxCqAABwk4KzngcHe7qasrHbSxfAiluXnZ2n1avX6YILOkvycw71LMvX8u5bUhiQSreuPPs6gmZxPaIFr100jNNzBKC0/CVdXeq9zzTJYFnXVfYEg8VNYujodS0paFbWOseEh4UnQHQ8J7CWHaEKAAC4DGkrK5vNkGEkasCA6jGMtDIVDlzlWQoPaXXMvC0VPxSxuCGKJQ1bLOu+jvBaUrAu6XF59ivuVmRnU7AHDu51ptB1pucl7dOjh/T2255+V5WHUAUAAOAmvr6n7xeMsjF79mz67rsF6tfvKvn5+bsMWSw4dLGs60rzGsctukr6eqZt5d0nP7/k0Fn465m2lfb1he+XfSaOz6Q8Ybc45b3fZHVBqAIAAIDHWSzm8LSAALuCg6v2Xs7nkvx8M0g6wl3BpfC6sj4/0z61a3v6nVcuQhUAAABwjnDcp9ePFOBW3IseAAAAACqAUAUAAAAAFUCoAgAAAIAKIFQBAAAAQAUQqgAAAACgAghVAAAAAFABhCoAAAAAqABCFQAAAABUAKEKAAAAACqAUAUAAAAAFUCoAgAAAIAKIFQBAAAAQAUQqgAAAACgAghVAAAAAFABfp4uoLIZhiFJSk1N9XAlks1mU2ZmplJTU+Xv7+/pclCN0ZbgLrQluAPtCO5CW4K7lNSWHJnAkRHcpcaHqrS0NElSXFychysBAAAA4A3S0tIUERHhtuNZDHfHNC9jt9t15MgRhYWFyWKxeLSW1NRUxcXF6eDBgwoPD/doLajeaEtwF9oS3IF2BHehLcFdSmpLhmEoLS1NsbGx8vFx35VQNb6nysfHRw0bNvR0GS7Cw8P5QQG3oC3BXWhLcAfaEdyFtgR3Ka4tubOHyoGJKgAAAACgAghVAAAAAFABhKoqFBAQoEmTJikgIMDTpaCaoy3BXWhLcAfaEdyFtgR3qeq2VOMnqgAAAACAykRPFQAAAABUAKEKAAAAACqAUAUAAAAAFUCoAgAAAIAKIFRVoenTp6tJkyYKDAxU165d9dtvv3m6JHjQihUrNHDgQMXGxspisWju3Lku2w3D0MSJExUTE6OgoCD17dtXO3fudNknKSlJQ4cOVXh4uCIjIzVq1Cilp6e77PPnn3/q0ksvVWBgoOLi4vT8889X9ltDFZo2bZq6dOmisLAw1a9fX4MGDdL27dtd9snOztbo0aNVp04dhYaGavDgwTp69KjLPgcOHNDVV1+t4OBg1a9fXw8//LDy8vJc9lm2bJkuvPBCBQQEqEWLFpo5c2Zlvz1UobffflsdOnRw3igzPj5e8+fPd26nHaE8nn32WVksFo0dO9a5jraE0po8ebIsFovL0qZNG+d2r2pLBqrE7NmzDavVavz3v/81tmzZYtx5551GZGSkcfToUU+XBg+ZN2+e8fjjjxtz5swxJBlff/21y/Znn33WiIiIMObOnWts3LjRuPbaa42mTZsaWVlZzn2uuuoqo2PHjsbq1auNn3/+2WjRooUxZMgQ5/aUlBQjKirKGDp0qLF582bjs88+M4KCgox33323qt4mKlm/fv2MGTNmGJs3bzY2bNhgDBgwwGjUqJGRnp7u3Ofuu+824uLijCVLlhi///670a1bN6N79+7O7Xl5ecb5559v9O3b11i/fr0xb948o27dusZjjz3m3GfPnj1GcHCw8eCDDxpbt2413njjDcPX19dYsGBBlb5fVJ5vv/3W+OGHH4wdO3YY27dvN/7zn/8Y/v7+xubNmw3DoB2h7H777TejSZMmRocOHYz777/fuZ62hNKaNGmScd555xkJCQnO5fjx487t3tSWCFVV5OKLLzZGjx7tfJ6fn2/ExsYa06ZN82BV8BaFQ5Xdbjeio6ONF154wbkuOTnZCAgIMD777DPDMAxj69athiRj7dq1zn3mz59vWCwW4/Dhw4ZhGMZbb71l1KpVy8jJyXHuM378eKN169aV/I7gKceOHTMkGcuXLzcMw2w3/v7+xhdffOHcZ9u2bYYkY9WqVYZhmAHfx8fHSExMdO7z9ttvG+Hh4c6288gjjxjnnXeey7luvvlmo1+/fpX9luBBtWrVMt5//33aEcosLS3NaNmypbFo0SKjV69ezlBFW0JZTJo0yejYsWOx27ytLTH8rwrk5uZq3bp16tu3r3Odj4+P+vbtq1WrVnmwMnirvXv3KjEx0aXNREREqGvXrs42s2rVKkVGRuqiiy5y7tO3b1/5+PhozZo1zn169uwpq9Xq3Kdfv37avn27Tp06VUXvBlUpJSVFklS7dm1J0rp162Sz2VzaUps2bdSoUSOXttS+fXtFRUU59+nXr59SU1O1ZcsW5z4Fj+HYh59hNVN+fr5mz56tjIwMxcfH045QZqNHj9bVV19d5N+btoSy2rlzp2JjY9WsWTMNHTpUBw4ckOR9bYlQVQVOnDih/Px8l39QSYqKilJiYqKHqoI3c7SLM7WZxMRE1a9f32W7n5+fateu7bJPcccoeA7UHHa7XWPHjlWPHj10/vnnSzL/na1WqyIjI132LdyWztZOStonNTVVWVlZlfF24AGbNm1SaGioAgICdPfdd+vrr79Wu3btaEcok9mzZ+uPP/7QtGnTimyjLaEsunbtqpkzZ2rBggV6++23tXfvXl166aVKS0vzurbkV9Y3BwDwTqNHj9bmzZv1yy+/eLoUVFOtW7fWhg0blJKSoi+//FLDhw/X8uXLPV0WqpGDBw/q/vvv16JFixQYGOjpclDN9e/f3/m4Q4cO6tq1qxo3bqz//e9/CgoK8mBlRdFTVQXq1q0rX1/fIrORHD16VNHR0R6qCt7M0S7O1Gaio6N17Ngxl+15eXlKSkpy2ae4YxQ8B2qGMWPG6Pvvv9fSpUvVsGFD5/ro6Gjl5uYqOTnZZf/Cbels7aSkfcLDw73uPzaUn9VqVYsWLdS5c2dNmzZNHTt21GuvvUY7QqmtW7dOx44d04UXXig/Pz/5+flp+fLlev311+Xn56eoqCjaEsotMjJSrVq10q5du7zu5xKhqgpYrVZ17txZS5Ysca6z2+1asmSJ4uPjPVgZvFXTpk0VHR3t0mZSU1O1Zs0aZ5uJj49XcnKy1q1b59znp59+kt1uV9euXZ37rFixQjabzbnPokWL1Lp1a9WqVauK3g0qk2EYGjNmjL7++mv99NNPatq0qcv2zp07y9/f36Utbd++XQcOHHBpS5s2bXIJ6YsWLVJ4eLjatWvn3KfgMRz78DOsZrPb7crJyaEdodT69OmjTZs2acOGDc7loosu0tChQ52PaUsor/T0dO3evVsxMTHe93OpTNNaoNxmz55tBAQEGDNnzjS2bt1q3HXXXUZkZKTLbCQ4t6SlpRnr16831q9fb0gyXn75ZWP9+vXG/v37DcMwp1SPjIw0vvnmG+PPP/80rrvuumKnVO/UqZOxZs0a45dffjFatmzpMqV6cnKyERUVZdx2223G5s2bjdmzZxvBwcFMqV6D3HPPPUZERISxbNkylylnMzMznfvcfffdRqNGjYyffvrJ+P333434+HgjPj7eud0x5eyVV15pbNiwwViwYIFRr169Yqecffjhh41t27YZ06dPZ/riGubRRx81li9fbuzdu9f4888/jUcffdSwWCzGjz/+aBgG7QjlV3D2P8OgLaH0HnroIWPZsmXG3r17jZUrVxp9+/Y16tataxw7dswwDO9qS4SqKvTGG28YjRo1MqxWq3HxxRcbq1ev9nRJ8KClS5cakoosw4cPNwzDnFb9iSeeMKKiooyAgACjT58+xvbt212OcfLkSWPIkCFGaGioER4ebowcOdJIS0tz2Wfjxo3GJZdcYgQEBBgNGjQwnn322ap6i6gCxbUhScaMGTOc+2RlZRn//ve/jVq1ahnBwcHG9ddfbyQkJLgcZ9++fUb//v2NoKAgo27dusZDDz1k2Gw2l32WLl1qXHDBBYbVajWaNWvmcg5Uf7fffrvRuHFjw2q1GvXq1TP69OnjDFSGQTtC+RUOVbQllNbNN99sxMTEGFar1WjQoIFx8803G7t27XJu96a2ZDEMwyhb3xYAAAAAwIFrqgAAAACgAghVAAAAAFABhCoAAAAAqABCFQAAAABUAKEKAAAAACqAUAUAAAAAFUCoAgAAAIAKIFQBAAAAQAUQqgAA57QmTZro1Vdf9XQZAIBqjFAFAKgyI0aM0KBBgyRJvXv31tixY6vs3DNnzlRkZGSR9WvXrtVdd91VZXUAAGoeP08XAABAReTm5spqtZb79fXq1XNjNQCAcxE9VQCAKjdixAgtX75cr732miwWiywWi/bt2ydJ2rx5s/r376/Q0FBFRUXptttu04kTJ5yv7d27t8aMGaOxY8eqbt266tevnyTp5ZdfVvv27RUSEqK4uDj9+9//Vnp6uiRp2bJlGjlypFJSUpznmzx5sqSiw/8OHDig6667TqGhoQoPD9dNN92ko0ePOrdPnjxZF1xwgT7++GM1adJEERERuuWWW5SWlubc58svv1T79u0VFBSkOnXqqG/fvsrIyKikTxMA4GmEKgBAlXvttdcUHx+vO++8UwkJCUpISFBcXJySk5N1+eWXq1OnTvr999+1YMECHT16VDfddJPL6z/88ENZrVatXLlS77zzjiTJx8dHr7/+urZs2aIPP/xQP/30kx555BFJUvfu3fXqq68qPDzceb5x48YVqctut+u6665TUlKSli9frkWLFmnPnj26+eabXfbbvXu35s6dq++//17ff/+9li9frmeffVaSlJCQoCFDhuj222/Xtm3btGzZMt1www0yDKMyPkoAgBdg+B8AoMpFRETIarUqODhY0dHRzvVvvvmmOnXqpGeeeca57r///a/i4uK0Y8cOtWrVSpLUsmVLPf/88y7HLHh9VpMmTfTUU0/p7rvv1ltvvSWr1aqIiAhZLBaX8xW2ZMkSbdq0SXv37lVcXJwk6aOPPtJ5552ntWvXqkuXLpLM8DVz5kyFhYVJkm677TYtWbJETz/9tBISEpSXl6cbbrhBjRs3liS1b9++Ap8WAMDb0VMFAPAaGzdu1NKlSxUaGupc2rRpI8nsHXLo3LlzkdcuXrxYffr0UYMGDRQWFqbbbrtNJ0+eVGZmZqnPv23bNsXFxTkDlSS1a9dOkZGR2rZtm3NdkyZNnIFKkmJiYnTs2DFJUseOHdWnTx+1b99eN954o9577z2dOnWq9B8CAKDaIVQBALxGenq6Bg4cqA0bNrgsO3fuVM+ePZ37hYSEuLxu3759uuaaa9ShQwd99dVXWrdunaZPny7JnMjC3fz9/V2eWywW2e12SZKvr68WLVqk+fPnq127dnrjjTfUunVr7d271+11AAC8A6EKAOARVqtV+fn5LusuvPBCbdmyRU2aNFGLFi1clsJBqqB169bJbrfrpZdeUrdu3dSqVSsdOXLkrOcrrG3btjp48KAOHjzoXLd161YlJyerXbt2pX5vFotFPXr00JQpU7R+/XpZrVZ9/fXXpX49AKB6IVQBADyiSZMmWrNmjfbt26cTJ07Ibrdr9OjRSkpK0pAhQ7R27Vrt3r1bCxcu1MiRI88YiFq0aCGbzaY33nhDe/bs0ccff+ycwKLg+dLT07VkyRKdOHGi2GGBffv2Vfv27TV06FD98ccf+u233zRs2DD16tVLF110Uane15o1a/TMM8/o999/14EDBzRnzhwdP35cbdu2LdsHBACoNghVAACPGDdunHx9fdWuXTvVq1dPBw4cUGxsrFauXKn8/HxdeeWVat++vcaOHavIyEj5+JT8X1bHjh318ssv67nnntP555+vTz/9VNOmTXPZp3v37rr77rt18803q169ekUmupDMHqZvvvlGtWrVUs+ePdW3b181a9ZMn3/+eanfV3h4uFasWKEBAwaoVatWmjBhgl566SX179+/9B8OAKBasRjM8QoAAAAA5UZPFQAAAABUAKEKAAAAACqAUAUAAAAAFUCoAgAAAIAKIFQBAAAAQAUQqgAAAACgAghVAAAAAFABhCoAAAAAqABCFQAAAABUAKEKAAAAACqAUAUAAAAAFfD/VgdRCOXiVO4AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Train model\n",
    "\n",
    "iterations = np.empty(int(max_iters/eval_iters))\n",
    "iterations[:] = np.nan\n",
    "train_loss = np.empty(int(max_iters/eval_iters))\n",
    "train_loss[:] = np.nan\n",
    "val_loss = np.empty(int(max_iters/eval_iters))\n",
    "val_loss[:] = np.nan\n",
    "\n",
    "for i in range(max_iters):\n",
    "    train_one_batch(model, optimizer)\n",
    "    if i % eval_iters == 0:\n",
    "        out = estimate_loss()\n",
    "        iterations[int(i/eval_iters)] = i\n",
    "        train_loss[int(i/eval_iters)] = torch.exp(out['train'])\n",
    "        val_loss[int(i/eval_iters)] = torch.exp(out['val'])\n",
    "        print(f\"First {i} batches: training perplexity: {torch.exp(out['train']):.3f}, validation perplexity: {torch.exp(out['val']):.3f}\")\n",
    "        \n",
    "# Visualize results\n",
    "plt.figure(figsize=(10, 5))\n",
    "plt.plot(iterations, train_loss, label='Training Perplexity', color='blue')\n",
    "plt.plot(iterations, val_loss, label='Validation Perplexity', color='red')\n",
    "plt.title('Training and Validation Perplexity')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Perplexity')\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "6c5ba73d-e403-4f41-8f31-8b4a04cec5b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello, my name is Trutal, it we seals, I hullnno;\n",
      "Could not take, do this fland, and that have warwiced, master injur\n"
     ]
    }
   ],
   "source": [
    "# Output some text\n",
    "\n",
    "model.eval()\n",
    "prompt = \"Hello, my name is\"\n",
    "max_new_tokens = 100\n",
    "ix = torch.tensor([encode(prompt)], dtype=torch.long)\n",
    "X =  ix.to('cuda')\n",
    "output = model.generate(X, max_new_tokens)\n",
    "[output] = output.tolist()\n",
    "response = decode(output)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ec730fe-e11d-4966-89a5-0f6c24cbf46f",
   "metadata": {},
   "source": [
    "Change number of heads"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "89e3647b-0979-4c18-a160-a61071f27916",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_head = 8\n",
    "\n",
    "model = BigramLanguageModel()\n",
    "model.to('cuda')\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "53fbc51c-bf67-45a5-80d1-cc3919ef49f3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 0 batches: training perplexity: 49.166, validation perplexity: 49.458\n",
      "First 200 batches: training perplexity: 11.052, validation perplexity: 11.052\n",
      "First 400 batches: training perplexity: 9.281, validation perplexity: 9.557\n",
      "First 600 batches: training perplexity: 8.125, validation perplexity: 8.451\n",
      "First 800 batches: training perplexity: 7.296, validation perplexity: 7.916\n",
      "First 1000 batches: training perplexity: 6.760, validation perplexity: 7.361\n",
      "First 1200 batches: training perplexity: 6.452, validation perplexity: 7.132\n",
      "First 1400 batches: training perplexity: 6.200, validation perplexity: 6.904\n",
      "First 1600 batches: training perplexity: 5.991, validation perplexity: 6.768\n",
      "First 1800 batches: training perplexity: 5.771, validation perplexity: 6.610\n",
      "First 2000 batches: training perplexity: 5.551, validation perplexity: 6.442\n",
      "First 2200 batches: training perplexity: 5.495, validation perplexity: 6.440\n",
      "First 2400 batches: training perplexity: 5.382, validation perplexity: 6.218\n",
      "First 2600 batches: training perplexity: 5.327, validation perplexity: 6.124\n",
      "First 2800 batches: training perplexity: 5.202, validation perplexity: 6.051\n",
      "First 3000 batches: training perplexity: 5.140, validation perplexity: 6.023\n",
      "First 3200 batches: training perplexity: 5.082, validation perplexity: 5.955\n",
      "First 3400 batches: training perplexity: 5.024, validation perplexity: 5.906\n",
      "First 3600 batches: training perplexity: 4.956, validation perplexity: 5.886\n",
      "First 3800 batches: training perplexity: 4.858, validation perplexity: 5.777\n",
      "First 4000 batches: training perplexity: 4.903, validation perplexity: 5.795\n",
      "First 4200 batches: training perplexity: 4.854, validation perplexity: 5.794\n",
      "First 4400 batches: training perplexity: 4.826, validation perplexity: 5.665\n",
      "First 4600 batches: training perplexity: 4.806, validation perplexity: 5.713\n",
      "First 4800 batches: training perplexity: 4.708, validation perplexity: 5.662\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1UAAAHWCAYAAACfRKOZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB2FElEQVR4nO3dd3gU5d7G8XtTNr0RSIPQOwIiIkQUkCKCopSjiChF1KOCioqiR6RZsHf0+B6V2BCPilgBEQEVAREBadIRhBAIIY20TXbeP+ZkYUkCKZvsJnw/1zVXdqf+ZvMk5OaZecZiGIYhAAAAAECFeLm7AAAAAACoyQhVAAAAAFAJhCoAAAAAqARCFQAAAABUAqEKAAAAACqBUAUAAAAAlUCoAgAAAIBKIFQBAAAAQCUQqgAAAACgEghVAFBGY8aMUePGjSu07fTp02WxWFxbkIfZt2+fLBaLEhMTq/3YFotF06dPd7xPTEyUxWLRvn37zrpt48aNNWbMGJfWU5m2Uhud/v1xtXPh5wuAZyNUAajxLBZLmably5e7u9Rz3t133y2LxaJdu3aVus4jjzwii8WiP/74oxorK79Dhw5p+vTp2rBhg7tLcSgKtkWTt7e3GjZsqCFDhnhUndXhySef1IIFC9xdBoBzBKEKQI33/vvvO039+vUrcX6bNm0qdZz//Oc/2r59e4W2nTJlinJycip1/Npg5MiRkqS5c+eWus5HH32k9u3bq0OHDhU+zk033aScnBw1atSowvs4m0OHDmnGjBklhpXKtBVXGDFihN5//3298847uuGGG/TDDz+oW7dutTZYlfTzRagCUJ183F0AAFTWjTfe6PR+9erVWrJkSbH5p8vOzlZgYGCZj+Pr61uh+iTJx8dHPj78yu3atauaN2+ujz76SFOnTi22fNWqVdq7d6+eeuqpSh3H29tb3t7eldpHZVSmrbjCBRdc4NT+u3fvrquvvlpvvPGG3nzzzUrt+8SJEwoKCqpsiS7FzxcAd6OnCsA5oVevXjrvvPO0bt069ejRQ4GBgfrXv/4lSfriiy905ZVXKi4uTn5+fmrWrJkee+wxFRYWOu3j9Ptkii61eu655/R///d/atasmfz8/NSlSxetXbvWaduS7vmwWCyaMGGCFixYoPPOO09+fn5q166dFi1aVKz+5cuX68ILL5S/v7+aNWumN998s8z3kfz000+69tpr1bBhQ/n5+Sk+Pl733ntvsf/ZHzNmjIKDg3Xw4EENHjxYwcHBqlevniZNmlTss0hLS9OYMWMUFham8PBwjR49WmlpaWetRTJ7q/7880/9/vvvxZbNnTtXFotFI0aMUH5+vqZOnarOnTsrLCxMQUFBuvTSS7Vs2bKzHqOke6oMw9Djjz+uBg0aKDAwUJdddpm2bNlSbNvU1FRNmjRJ7du3V3BwsEJDQzVgwABt3LjRsc7y5cvVpUsXSdLYsWMdl9sV3U9W0j1VJ06c0P3336/4+Hj5+fmpVatWeu6552QYhtN65WkXZdW7d29J0t69ex3z1qxZoyuuuEJhYWEKDAxUz549tXLlSqftitrY1q1bdcMNNygiIkKXXHKJ4xyDg4O1Z88e9e/fX0FBQYqLi9PMmTOLnVNJDh48qJtvvlnR0dGOc3znnXccy3NyctS6dWu1bt3aqa2mpqYqNjZWF198saNdnv6zYLFYdOLECb377ruO782YMWO0bNkyWSwWff7558XqKWp7q1atKstHCgBO+G8dAOeMY8eOacCAAbr++ut14403Kjo6WpL5B3hwcLDuu+8+BQcH64cfftDUqVOVkZGhZ5999qz7nTt3rjIzM/XPf/5TFotFzzzzjIYOHao9e/actcfi559/1vz583XnnXcqJCREr7zyioYNG6b9+/crMjJSkrR+/XpdccUVio2N1YwZM1RYWKiZM2eqXr16ZTrvTz75RNnZ2brjjjsUGRmpX3/9Va+++qr+/vtvffLJJ07rFhYWqn///uratauee+45ff/993r++efVrFkz3XHHHZLMcHLNNdfo559/1u233642bdro888/1+jRo8tUz8iRIzVjxgzNnTtXF1xwgdOx//vf/+rSSy9Vw4YNlZKSorfeeksjRozQrbfeqszMTL399tvq37+/fv31V51//vllOl6RqVOn6vHHH9fAgQM1cOBA/f7777r88suVn5/vtN6ePXu0YMECXXvttWrSpImSk5P15ptvqmfPntq6davi4uLUpk0bzZw5U1OnTtVtt92mSy+9VJJ08cUXl3hswzB09dVXa9myZRo3bpzOP/98LV68WA888IAOHjyoF1980Wn9srSL8ti9e7ckObb94YcfNGDAAHXu3FnTpk2Tl5eX5syZo969e+unn37SRRdd5LT9tddeqxYtWujJJ590CkyFhYW64oor1K1bNz3zzDNatGiRpk2bpoKCAs2cObPUepKTk9WtWzdHgKxXr54WLlyocePGKSMjQxMnTlRAQIDeffddde/eXY888oheeOEFSdL48eOVnp6uxMTEUnsj33//fd1yyy266KKLdNttt0mSmjVrpm7duik+Pl4ffvihhgwZ4rTNhx9+qGbNmikhIaGcny4ASDIAoJYZP368cfqvt549exqSjH//+9/F1s/Ozi4275///KcRGBho5ObmOuaNHj3aaNSokeP93r17DUlGZGSkkZqa6pj/xRdfGJKMr776yjFv2rRpxWqSZFitVmPXrl2OeRs3bjQkGa+++qpj3qBBg4zAwEDj4MGDjnk7d+40fHx8iu2zJCWd36xZswyLxWL89ddfTucnyZg5c6bTup06dTI6d+7seL9gwQJDkvHMM8845hUUFBiXXnqpIcmYM2fOWWvq0qWL0aBBA6OwsNAxb9GiRYYk480333TsMy8vz2m748ePG9HR0cbNN9/sNF+SMW3aNMf7OXPmGJKMvXv3GoZhGEeOHDGsVqtx5ZVXGna73bHev/71L0OSMXr0aMe83Nxcp7oMw/xe+/n5OX02a9euLfV8T28rRZ/Z448/7rTeP/7xD8NisTi1gbK2i5IUtckZM2YYR48eNQ4fPmwsX77c6NSpkyHJ+Oyzzwy73W60aNHC6N+/v9NnkZ2dbTRp0sTo16+fY15Rux0xYkSJ5yjJuOuuuxzz7Ha7ceWVVxpWq9U4evSo0zmd+v0ZN26cERsba6SkpDjt8/rrrzfCwsKc2uzDDz9seHl5GT/++KPxySefGJKMl156yWm7kn6+goKCnL6vp+7Pz8/PSEtLc8w7cuSI4ePj41QjAJQHl/8BOGf4+flp7NixxeYHBAQ4XmdmZiolJUWXXnqpsrOz9eeff551v8OHD1dERITjfVGvxZ49e866bd++fdWsWTPH+w4dOig0NNSxbWFhob7//nsNHjxYcXFxjvWaN2+uAQMGnHX/kvP5nThxQikpKbr44otlGIbWr19fbP3bb7/d6f2ll17qdC7ffvutfHx8HD1XknkP01133VWmeiTzPri///5bP/74o2Pe3LlzZbVade211zr2abVaJUl2u12pqakqKCjQhRdeWOKlg2fy/fffKz8/X3fddZfTZWITJ04stq6fn5+8vMx/HgsLC3Xs2DEFBwerVatW5T5ukW+//Vbe3t66++67nebff//9MgxDCxcudJp/tnZxNtOmTVO9evUUExOjXr16affu3Xr66ac1dOhQbdiwQTt37tQNN9ygY8eOKSUlRSkpKTpx4oT69OmjH3/8UXa73Wl/p7eJU02YMMHxuqjnKT8/X99//32J6xuGoc8++0yDBg2SYRiO46ekpKh///5KT093+pynT5+udu3aafTo0brzzjvVs2fPYp9jeYwaNUp5eXn69NNPHfM+/vhjFRQUnPU+TAAoDZf/AThn1K9f3/FH+qm2bNmiKVOm6IcfflBGRobTsvT09LPut2HDhk7viwLW8ePHy71t0fZF2x45ckQ5OTlq3rx5sfVKmleS/fv3a+rUqfryyy+L1XT6+fn7+xe7rPDUeiTpr7/+UmxsrIKDg53Wa9WqVZnqkaTrr79e9913n+bOnatevXopNzdXn3/+uQYMGOAUUN999109//zz+vPPP2Wz2RzzmzRpUuZjFdUsSS1atHCaX69ePafjSWaAe/nll/X6669r7969TveTVeTSu6Ljx8XFKSQkxGl+0YiURfUVOVu7OJvbbrtN1157rby8vBQeHq527drJz89PkrRz505JOuPlmunp6U6fS2mft5eXl5o2beo0r2XLlpJU6jPCjh49qrS0NP3f//2f/u///q/EdY4cOeJ4bbVa9c4776hLly7y9/fXnDlzKvVMqtatW6tLly768MMPNW7cOEnmpX/dunUr888UAJyOUAXgnHFqj02RtLQ09ezZU6GhoZo5c6aaNWsmf39//f7775o8eXKx/7EvSWn3dRhluFm/MtuWRWFhofr166fU1FRNnjxZrVu3VlBQkA4ePKgxY8YUO7/qGjEvKipK/fr102effabZs2frq6++UmZmpmPIdUn64IMPNGbMGA0ePFgPPPCAoqKi5O3trVmzZjnuEaoKTz75pB599FHdfPPNeuyxx1SnTh15eXlp4sSJZWoPrlDZdtGiRQv17du3xGVF5/Dss8+Wel/a6YG5pJ+diio6/o033lhqsDt9OP3FixdLknJzc7Vz585yh+rTjRo1Svfcc4/+/vtv5eXlafXq1XrttdcqtU8A5zZCFYBz2vLly3Xs2DHNnz9fPXr0cMw/dZQ0d4qKipK/v3+JD8s90wN0i2zatEk7duzQu+++q1GjRjnmL1mypMI1NWrUSEuXLlVWVpbTH9/lfS7TyJEjtWjRIi1cuFBz585VaGioBg0a5Fj+6aefqmnTppo/f75Tz8S0adMqVLNk9tKc2rNy9OjRYr0/n376qS677DK9/fbbTvPT0tJUt25dx/vy9JY0atRI33//vTIzM516q4ouL63K52mdruiywtDQ0FKDV1nZ7Xbt2bPH0TslSTt27JCkYqMfFqlXr55CQkJUWFhYpuP/8ccfmjlzpsaOHasNGzbolltu0aZNmxQWFnbG7c70/SnqKf3oo4+Uk5MjX19fDR8+/Ky1AEBpuKcKwDmtqEfg1B6A/Px8vf766+4qyYm3t7f69u2rBQsW6NChQ475u3btKnYfTmnbS87nZxiGXn755QrXNHDgQBUUFOiNN95wzCssLNSrr75arv0MHjxYgYGBev3117Vw4UINHTpU/v7+Z6x9zZo1FRryum/fvvL19dWrr77qtL+XXnqp2Lre3t7FeoQ++eQTHTx40Gle0bOayjKU/MCBA1VYWFisN+TFF1+UxWIp8/1xrtC5c2c1a9ZMzz33nLKysootP3r0aLn2d+o5GYah1157Tb6+vurTp0+J63t7e2vYsGH67LPPtHnz5jMe32azacyYMYqLi9PLL7+sxMREJScn69577z1rXUFBQaV+b+rWrasBAwbogw8+0IcffqgrrrjCKTADQHnRUwXgnHbxxRcrIiJCo0eP1t133y2LxaL333/fZZffucL06dP13XffqXv37rrjjjscf5yfd9552rBhwxm3bd26tZo1a6ZJkybp4MGDCg0N1WeffVbme3NKMmjQIHXv3l0PPfSQ9u3bp7Zt22r+/Plluv/sVMHBwRo8eLDmzp0rSU6X/knSVVddpfnz52vIkCG68sortXfvXv373/9W27ZtSwwDZ1L0vK1Zs2bpqquu0sCBA7V+/XotXLiw2B/TV111laNn5OKLL9amTZv04YcfFrt3qFmzZgoPD9e///1vhYSEKCgoSF27di3x0rRBgwbpsssu0yOPPKJ9+/apY8eO+u677/TFF19o4sSJToNSVDUvLy+99dZbGjBggNq1a6exY8eqfv36OnjwoJYtW6bQ0FB99dVXZdqXv7+/Fi1apNGjR6tr165auHChvvnmG/3rX/8645D/Tz31lJYtW6auXbvq1ltvVdu2bZWamqrff/9d33//vVJTUyVJjz/+uDZs2KClS5cqJCREHTp00NSpUzVlyhT94x//0MCBA0s9RufOnfX999/rhRdeUFxcnJo0aaKuXbs6lo8aNUr/+Mc/JEmPPfZYmc4XAEpDTxWAc1pkZKS+/vprxcbGasqUKXruuefUr18/PfPMM+4uzaFz585auHChIiIi9Oijj+rtt9/WzJkz1adPH6eenZL4+vrqq6++0vnnn69Zs2ZpxowZatGihd57770K1+Pl5aUvv/xSI0eO1AcffKBHHnlE9evX17vvvlvufRUFqdjYWMcDaouMGTNGTz75pDZu3Ki7775bixcv1gcffKALL7ywQnU//vjjmjFjhtavX68HHnhAu3fv1nfffefocSryr3/9S/fff78WL16se+65R7///ru++eYbxcfHO63n6+urd999V97e3rr99ts1YsQIrVixosRjF31mEydO1Ndff62JEydq69atevbZZx3PX6pOvXr10qpVq3ThhRfqtdde01133aXExETFxMSUqReoiLe3txYtWqTDhw/rgQce0Nq1azVt2rSzhpTo6Gj9+uuvGjt2rObPn68JEybo5ZdfVmpqqp5++mlJ0u+//64nn3xSEyZM0GWXXebY9qGHHlKXLl106623nrGX8IUXXlDnzp01ZcoUjRgxwqlnVTKDbkREhMLCwnT11VeX+ZwBoCQWw5P+OxYAUGaDBw/Wli1bHKO5AdVpzJgx+vTTT8vda+gpCgoKFBcXp0GDBhW7fw4AyoueKgCoAXJycpze79y5U99++6169erlnoKAGm7BggU6evSo0wAuAFBR3FMFADVA06ZNNWbMGDVt2lR//fWX3njjDVmtVj344IPuLg2oUdasWaM//vhDjz32mDp16qSePXu6uyQAtQChCgBqgCuuuEIfffSRDh8+LD8/PyUkJOjJJ58s9jBbAGf2xhtv6IMPPtD555+vxMREd5cDoJbgnioAAAAAqATuqQIAAACASiBUAQAAAEAl1Pp7qux2uw4dOqSQkBBZLBZ3lwMAAADATQzDUGZmpuLi4uTl5br+pVofqg4dOlTsgY0AAAAAzl0HDhxQgwYNXLa/Wh+qQkJCJJkfXGhoqFtrsdls+u6773T55ZfL19fXrbWgZqMtwVVoS3AF2hFchbYEVymtLWVkZCg+Pt6REVyl1oeqokv+QkNDPSJUBQYGKjQ0lF8UqBTaElyFtgRXoB3BVWhLcJWztSVX3xbEQBUAAAAAUAmEKgAAAACoBEIVAAAAAFRCrb+nCgAA4FxVWFgom83m7jLKzGazycfHR7m5uSosLHR3OaiBvL295eNT/RGHUAUAAFALZWVl6e+//5ZhGO4upcwMw1BMTIwOHDjA80VRYYGBgapXr161HpNQBQAAUMsUFhbq77//dvxxWVMCit1uV1ZWloKDg136YFacGwzDUH5+vo4ePar9+/dX67HdGqqmT5+uGTNmOM1r1aqV/vzzT0lSbm6u7r//fs2bN095eXnq37+/Xn/9dUVHR7ujXAAAgBrBZrPJMAzVq1dPAQEB7i6nzOx2u/Lz8+Xv70+oQoUEBATI19dX+/btk7e3d7Ud1+2ttV27dkpKSnJMP//8s2PZvffeq6+++kqffPKJVqxYoUOHDmno0KFurBYAAKDmqCk9VIArFQXy6mz/br/8z8fHRzExMcXmp6en6+2339bcuXPVu3dvSdKcOXPUpk0brV69Wt26davuUgEAAACgGLeHqp07dyouLk7+/v5KSEjQrFmz1LBhQ61bt042m019+/Z1rNu6dWs1bNhQq1atKjVU5eXlKS8vz/E+IyNDktkN7u7Rb4qO7+46UPPRluAqtCW4Au3I8xRd/me322W3291dTpkVDapRVDtQEXa73dGWTv+9VFW/p9waqrp27arExES1atVKSUlJmjFjhi699FJt3rxZhw8fltVqVXh4uNM20dHROnz4cKn7nDVrVrH7tCTpu+++U2BgoKtPoUKWLFni7hJQS9CW4Cq0JbgC7chzFF0JlJWVpfz8fHeXU26ZmZku21eHDh10xx136I477ijT+j///LMGDRqkffv2KSwszGV1eJK5c+fq4Ycf1l9//eWyfUZEROiDDz7QlVde6bJ9VlR+fr5yc3MlFf+9lJ2dXSXHdGuoGjBggON1hw4d1LVrVzVq1Ej//e9/K3xT5cMPP6z77rvP8T4jI0Px8fG6/PLLFRoaWumaK8Nms2nJkiXq16+ffH193VoLajbaElyFtgRXoB15ntzcXB04cEDBwcHy9/d3dzllcrZBBaZOnapp06aVe79r165VUFBQmf9zvW/fvjp48KCio6Or9J6c5cuXq0+fPo73UVFR6t69u5555hk1bdq0yo4rSf7+/rJYLC792/jgwYOKiIiQn5+f9u3bp2bNmmndunU6//zzXXaMssrNzXW0+9N/LxVdxeZqbr/871Th4eFq2bKldu3apX79+ik/P19paWlOvVXJyckl3oNVxM/PT35+fsXm+/r6eswvek+qBTUbbQmuQluCK9COPEdhYaEsFou8vLxqzCh6SUlJstvtyszM1MKFCzVt2jRt377dsfzUYdYNw1BhYWGZHvJa3lGj/f39FRcXV77iK6DoXLZv366QkBDt3LlTt912m6655hr98ccfFRq5zmazlelnsOjYrmwbp35mp+7fHe3Py8vLEYhP/71UVb+jPOqnLCsrS7t371ZsbKw6d+4sX19fLV261LF8+/bt2r9/vxISEtxYZQUZhrz79lXff/5TOnTI3dUAAIBziGFIJ064Zyrrs4djYmIUExOj6OhohYaGymKxOOb9+eefCgkJ0cKFC9W5c2f5+fnp559/1u7du3XNNdcoOjpawcHB6tKli77//nun/TZu3FgvvfSS473FYtFbb72lIUOGKDAwUC1atNCXX37pWL58+XJZLBalpaVJkhITExUeHq7FixerTZs2Cg4O1hVXXKGkpCTHNgUFBbr77rsVHh6uyMhITZ48WaNHj9bgwYPPet5RUVGKjY1Vjx49NHXqVG3dulW7du2SJH3xxRe64IIL5O/vr6ZNm2rGjBkqKChwOpc33nhDV199tYKCgvTEE0846v/mm2/UoUMH+fv7q1u3btq8efMZ6zjTsWbOnKm4uDgdO3bMsf6VV16pyy67zHHvm8Vi0YIFCyRJTZo0kSR16tRJFotFvXr10o8//ihfX99it/FMnDhRl1566Vk/J0/n1lA1adIkrVixQvv27dMvv/yiIUOGyNvbWyNGjFBYWJjGjRun++67T8uWLdO6des0duxYJSQk1MyR/ywWWXbtUlBysiyn/BACAABUtexsKTjYPZMrb2F56KGH9NRTT2nbtm3q0KGDsrKyNHDgQC1dulTr16/XFVdcoUGDBp31wa8zZszQddddpz/++EMDBw7UyJEjlZqaeobPL1vPPfec3n//ff3444/av3+/Jk2a5Fj+9NNP68MPP9ScOXO0cuVKZWRkOAJGeRTd/pKfn6+ffvpJo0aN0j333KOtW7fqzTffVGJiop544gmnbaZPn64hQ4Zo06ZNuvnmmx3zH3jgAT3//PNau3at6tWrp0GDBpU6SMPZjvXII4+ocePGuuWWWyRJs2fP1i+//KJ33323xJ6oX3/9VZL0/fffKykpSfPnz1ePHj3UtGlTvf/++471bDabPvzwQ6e6ayzDjYYPH27ExsYaVqvVqF+/vjF8+HBj165djuU5OTnGnXfeaURERBiBgYHGkCFDjKSkpHIdIz093ZBkpKenu7r8crHbDeNgXGfDkIy09+e7tRbUfPn5+caCBQuM/Px8d5eCGo62BFegHXmenJwcY+vWrUZOTo5hGIaRlWUYZp9R9U9ZWWWvu7Cw0Dh+/Ljx9ttvG2FhYY75y5YtMyQZCxYsOOs+2rVrZ7z66quO940aNTJefPFFx3tJxpQpUxzvs7KyDEnGwoULnY51/PhxwzAMY86cOYYkp79RZ8+ebURHRzveR0dHG88++6zjfUFBgdGwYUPjmmuuKbXO049z6NAh4+KLLzbq169v5OXlGX369DGefPJJp23ef/99IzY21ulcJk6cWOJ+582b55h37NgxIyAgwPj4448d53Tq51uWY+3evdsICQkxJk+ebAQEBBgffvih0/qSjM8//9wwDMPYu3evIclYv3690zpPP/200aZNG8f7zz77zAgODjayytNIyiAnJ8fYsmWL8fXXXxf7vVRV2cCt91TNmzfvjMv9/f01e/ZszZ49u5oqqjoWi7TpaKziJKX9eVi1cywZAADgiQIDpaws9x3bVS688EKn91lZWZo+fbq++eYbJSUlqaCgQDk5OWftqerQoYPjdVBQkEJDQ3XkyJFS1w8MDFSzZs0c72NjYx3rp6enKzk5WRdddJFjube3tzp37lymYeEbNGggwzCUnZ2tjh076rPPPpPVatXGjRu1cuVKp56pwsJC5ebmKjs72zHwxumfSZFTb5epU6eOWrVqpW3btpW4blmO1bRpUz333HP65z//qeHDh+uGG24467mdbsyYMZoyZYrjmbOJiYm67rrrFBQUVO59eRqPGqiitssMjpGOS3l7Sx8SHgAAwNUsFqkW/N1a7I/vSZMmacmSJXruuefUvHlzBQQE6B//+MdZh5E/fbACi8VyxgBU0vpGWW8WO4uffvpJoaGhioqKUkhIiGN+VlaWZsyYoaFDhxbb5tQRHV0RSMp6rB9//FHe3t7at2+fCgoKyjRQyKmioqI0aNAgzZkzR02aNNHChQu1fPnyypbvEQhV1SgnIlY6LhUe5J4qAACAylq5cqXGjBmjIUOGSDLDwb59+6q1hrCwMEVHR2vt2rXq0aOHJLOX5/fffy/TcOJNmjQp9lxWSbrgggu0fft2NW/evEJ1rV69Wg0bNpQkHT9+XDt27FCbNm1KXLcsx/r44481f/58LV++XNddd50ee+yxEp8NK0lWq1WS+Tmc7pZbbtGIESPUoEEDNWvWTN27dy/vqXkkQlU1KqgXI+2RvJLpqQIAAKisFi1aaP78+Ro0aJAsFoseffTRMl1y52p33XWXZs2apebNm6t169Z69dVXdfz48Uo952rq1Km66qqr1LBhQ/3jH/+Ql5eXNm7cqM2bN+vxxx8/6/YzZ85UZGSkoqOj9cgjj6hu3bqljkZ4tmP9/fffuuOOO/T000/rkksu0Zw5c3TVVVdpwIABJQ4gFxUVpYCAAC1atEgNGjSQv7+/40HK/fv3V2hoqB5//HHNnDmzwp+Pp/GoIdVrO8v/xu/3S6WnCgAAoLJeeOEFRURE6OKLL9agQYPUv39/XXDBBdVex+TJkzVixAiNGjVKCQkJCg4OVv/+/Sv14OX+/fvr66+/1nfffacuXbqoW7duevHFF9WoUaMybf/UU0/pnnvuUefOnXX48GF99dVXjh6k8hzLMAyNGTNGF110kSZMmOBY/4477tCNN96orBJu1vPx8dErr7yiN998U3Fxcbrmmmscy7y8vDRmzBgVFhZq1KhRFfhkPJPFcNUFoR4qIyNDYWFhSk9Pd+lToyvio/tWa8SLCUoJaKC62QfcWgtqNpvNpm+//VYDBw7kQZuoFNoSXIF25Hlyc3O1d+9eNWnSpFJ/2Fc3u92ujIwMhYaG1piHFpfEbrerTZs2jsvkqtPy5ct12WWX6fjx4yVeVugJxo0bp6NHjzo9H8yVcnNztWfPHu3du1eXX3650++lqsoGXP5XjQKbxUiSwnMPS3a7VIN/WQAAAMD0119/6bvvvlPPnj2Vl5en1157TXv37q3QCHm1WXp6ujZt2qS5c+dWWaByF/6qr0YRraNll0U+RoGUkuLucgAAAOACXl5eSkxMVJcuXdS9e3dt2rRJ33//fakDQ5yrrrnmGl1++eW6/fbb1a9fP3eX41L0VFWj6AY+SlFdRemojENJskRFubskAAAAVFJ8fLxWrlzp7jIkSb169XLZcO+uVluGTy8JPVXVKDZWSlKsJOnELgarAAAAAGoDQlU1CgiQjnhHS5LStxOqAAAAgNqAUFXNjvuZl/zl7CFUAQAAALUBoaqaZQTVlSQVHCBUAQAAALUBoaqanQirI0myHCZUAQAAALUBoaqa5UVESJKsKYQqAAAAoDYgVFWzwqgwSVJgBqEKAADA1Xr16qWJEyc63jdu3FgvvfTSGbexWCxasGBBpY/tqv14qunTp+v888932f727dsni8WiDRs2uGyf7kKoqmZGTKgkKTwnSfLQZwgAAABUt0GDBmnAgAElLvvpp59ksVj0xx9/lHu/a9eu1W233VbZ8pyUFi6SkpJKPQdXSUxMlMVikcVikZeXlxo0aKCxY8fqyJEjVXrcqhAfH6+kpCSdd955ksznWFksFqWlpbm3sArg4b/VzLtBiCTJz54rpadL4eHuLQgAAMADjBs3TsOGDdPBgwcVGhrqtGzOnDm68MIL1aFDh3Lvt169eq4q8axiYmKq5TihoaHavn277Ha7Nm7cqLFjx+rQoUNavHhxhfZns9nk6+vr4irPztvbu9o+s6pGT1U1C4kylCbzEkAlcQkgAACoBoYhnTjhnqmMV+ZcddVVqlevnj766COn+VlZWfrkk080btw4HTt2TCNGjFD9+vUVGBio9u3bF1v/dKdf/rdz50716NFD/v7+atu2rZYsWVJsm8mTJ6tly5YKDAxU06ZN9eijj8pms0kye4pmzJihjRs3OnqMEhMTJRW//G/Tpk3q3bu3AgICFBkZqdtuu01ZWVmO5WPGjNHgwYP13HPPKTY2VpGRkRo/frzjWKWxWCyKiYlRXFycBgwYoLvvvlvff/+9cnJyJElvvfWW2rRpI39/f7Vu3Vqvv/66Y9uiS+4+/vhj9ezZU/7+/vrwww+VmJio8PBwLViwQC1atJC/v7/69++vAwcOnLGWMx3r5ptvVocOHZSXlydJys/PV6dOnTRq1CinWjZs2KB9+/bpsssukyRFRETIYrFozJgxeu+99xQZGenYR5HBgwfrpptuOmNt1YmeqmpWp06ekhSrcKUre3eSAtu0cXdJAACgtsvOloKD3XPsrCwpKOisq/n4+Oimm27S3LlzNWPGDMf8Tz75RIWFhRoxYoSysrLUuXNnTZ48WaGhofrmm2900003qVmzZrrooovOegy73a6hQ4cqOjpaa9asUXp6utP9V0VCQkKUmJiouLg4bdq0SbfeeqtCQkL04IMPavjw4dq8ebMWLVqk77//XpIUFhZWbB8nTpxQ//79lZCQoLVr1+rIkSO65ZZbNGHCBEcIk6Rly5YpNjZWy5Yt065duzR8+HCdf/75uvXWW896PkUCAgJkt9tVUFCgDz/8UFOnTtVrr72mTp06af369br11lsVFBSk0aNHO7Z56KGH9Pzzz6tTp07y9/fX4sWLlZ2drSeeeELvvfeerFar7rzzTl1//fVauXJlicc927FeeeUVdezYUQ899JBefPFFPfLII0pLS9Nrr71WbF/x8fH67LPPNGzYMG3fvl2hoaEKCAiQ1WrV3XffrS+//FLXXnutJOnIkSP65ptv9N1335X5M6pqhKpqFhBQoCPesWpT+KfS/0xS4FXurggAAMAzjB07Vs8995xWrFih3r17SzIv/Rs2bJjCwsIUFhamSZMmOda/6667tHjxYv33v/8tU6j6/vvv9eeff2rx4sWKi4uTJD355JPF7oOaMmWK43Xjxo01adIkzZs3Tw8++KACAgIUHBwsHx+fM166NnfuXOXm5uq9995T0P9C5WuvvaZBgwbp6aefVnR0tCSzV+a1116Tt7e3WrdurSuvvFJLly4tc6jauXOn/v3vf+vCCy9USEiIpk2bpueff15Dhw6VJDVp0kRbt27Vm2++6RSqJk6c6FiniM1m02uvvaauXbtKkt599121adNGv/76a4mf79mOFRwcrA8++EA9e/ZUSEiIXnrpJS1btqzY5Z2SeSlgnTrmo4eioqIUfsotMjfccIPmzJnjCFUffPCBGjZsqF69epXpM6oOhCo3SA+IlbKk7N1c/gcAAKpBYKDZY+SuY5dR69atddFFF2nOnDnq3bu3du3apZ9++kkzZ86UJBUWFurJJ5/Uf//7Xx08eFD5+fnKy8tTYBmPsW3bNsXHxzsClSQlJCQUW+/jjz/WK6+8ot27dysrK0sFBQUlBoGzHatjx46OQCVJ3bt3l91u1/bt2x2hql27dvL29nasExsbq02bNp1x3+np6QoODpbdbldubq4uueQSvfXWWzpx4oR2796tcePGOYWygoKCYr1pF154YbH9+vj4qEuXLo73rVu3Vnh4uLZt21YsVJX1WAkJCZo0aZIee+wxTZ48WZdccskZz60kt956q7p06aKDBw+qfv36SkxM1JgxY2SxWMq9r6pCqHKD7LAYKUuy7SdUAQCAamCxlOkSPE9w0003afLkyXr99dc1Z84cNWvWTD179pQkPfvss3r55Zf10ksvqX379goKCtLEiROVn5/vsuOvWrVKI0eO1IwZM9S/f3+FhYVp3rx5ev755112jFOdPkCExWKR3W4/4zYhISH6/fff5eXlpdjYWAUEBEiSkpOTJUn/+c9/HL1NRU4NbpKcwl5FFN0bdrZj2e12rVy5Ut7e3tq1a1eFjtWpUyd17NhR7733ni6//HJt2bJF33zzTcWLrwKEKjfIj4yVDoqBKgAAAE4zePBgPfzww5o7d67ee+893XHHHY4eiZUrV+qaa67RjTfeKMn8g33Hjh1q27Ztmfbdpk0bHThwQElJSYqNjZUkrV692mmdX375RY0aNdIjjzzimPfXX385rWO1WlVYWHjWYyUmJurEiROOALNy5Up5eXmpVatWZaq3NF5eXmrevHmx+dHR0YqLi9OePXs0cuTIcu+3oKBAv/32m6NXavv27UpLS1ObEsYAKOuxnn32Wf35559asWKF+vfvrzlz5mjs2LElrmu1WiWpxM/2lltu0UsvvaSDBw+qb9++io+PL/f5VSVG/3OHWPP6W98UQhUAAMCpgoODdd111+nhhx9WUlKSxowZ41jWokULLVmyRL/88ou2bdumf/7zn47embLo27evWrZsqdGjR2vjxo366aefnMJT0TH279+vefPmaffu3XrllVf0+eefO63TuHFj7d27Vxs2bFBKSkqxkekkaeTIkfL399fo0aO1efNmLVu2THfddZduuukmx6V/VWHGjBmaNWuWXnnlFe3YsUObNm3SnDlz9MILL5x1W19fX911111as2aN1q1bpzFjxqhbt26l3q92tmOtX79eU6dO1VtvvaXu3bvrhRde0D333KM9e/aUuL9GjRrJYrHo66+/1tGjR51GSrzhhhv0999/6z//+Y9uvvnmCnwyVYtQ5QY+8WaoCkgnVAEAAJzu5ptv1vHjx9W/f3+n+5+mTJmiCy64QP3791evXr0UExOjwYMHl3m/Xl5e+vzzz5WTk6OLLrpIt9xyi5544gmnda6++mrde++9mjBhgs4//3z98ssvevTRR53WGTZsmK644gpddtllJQ4DL0mBgYFavHixUlNT1aVLF/3jH/9Qnz59Shz5zpVuueUWvfXWW5ozZ47at2+vnj17KjExUU2aNDnrtoGBgZo8ebJuuOEGde/eXcHBwfr4448rdKzc3FzdeOONGjNmjAYNGiRJuu2223TZZZfppptuKrE3qn79+poxY4YeeughRUdHa8KECY5lYWFhGjZsmIKDg8v1Pa8uFsMo48MDaqiMjAyFhYUpPT293DcYuprNZtO3334r+9ZmGvKv9sryDlVwQbpba0LNVNSWBg4c6JaH9aH2oC3BFWhHnic3N1d79+5VkyZN5O/v7+5yysxutysjI0OhoaHy8uL//qtTYmKiJk6cqLS0NHeXUqo+ffqoXbt2euWVV864Xm5urvbs2aO9e/fq8ssvd/q9VFXZgHuq3CC0lXkNb3BhhvnciHKMigMAAACcS44fP67ly5dr+fLlTg8X9iSEKjeo2zRE2QpQoHLMwSqaNXN3SQAAAIBH6tSpk44fP66nn3660oN8VBVClRvE1bcoSbFqpj3K25ckP0IVAAAA3GjMmDFOg4J4kn379rm7hLPiYlU3CAuTki3mJYBp2xisAgAAAKjJCFVuYLFIaQFmqDqxi1AFAACqRi0fjwwokTvaPaHKTU6EmqEq/y9CFQAAcC1vb29JUn5+vpsrAapfdna2pJIfIlxVuKfKTfIjY6XDknGIUAUAAFzLx8dHgYGBOnr0qHx9fWvM8OR2u135+fnKzc2tMTXDcxiGoezsbB05ckShoaHV2mNFqHITIzZW2iJ5HyVUAQAA17JYLIqNjdXevXv1119/ubucMjMMQzk5OQoICJDFYnF3OaihwsPDFRkZWa3HJFS5iU8D8/K/gDRCFQAAcD2r1aoWLVrUqEsAbTabfvzxR/Xo0YMHSaNCfH195e3tLZvNVq3HJVS5SUBTM1SFZBGqAABA1fDy8pK/v7+7yygzb29vFRQUyN/fn1CFGoWLVd0kpKUZqsJtKVIN+h8kAAAAAM4IVW5St1WkbEUdhcnJ7i0GAAAAQIURqtwkroGXDitGkmTbzyWAAAAAQE1FqHKTyEjpsMW8BDBtG6EKAAAAqKkIVW5isUjH/cxQlbWTUAUAAADUVIQqN8oKMUNV3j5CFQAAAFBTEarcKC/SDFX2g4QqAAAAoKYiVLmRPdoMVV5HCFUAAABATUWociPv+mao8j9OqAIAAABqKkKVG/k3MUNVcCahCgAAAKipCFVuFNzCDFXheclSYaGbqwEAAABQEYQqN4psGy27LPJRoZSS4u5yAAAAAFQAocqNYuN9dFT1JEkFB7gEEAAAAKiJCFVuVK+elCTzEsD0PwlVAAAAQE1EqHIjb28p1c8MVZk7CFUAAABATUSocrPMYDNU5e4lVAEAAAA1EaHKzfIizFBV+DehCgAAAKiJCFVuVhhlhipLMqEKAAAAqIkIVW5mqW+GKr9UQhUAAABQExGq3MyvkRmqgjMJVQAAAEBNRKhys+AWZqgKz0mSDMPN1QAAAAAoL0KVm0W0/d/lf0aelJbm3mIAAAAAlBuhys1iGvsrVRGSJPtBLgEEAAAAahpClZtFR0tJMnur0v8kVAEAAAA1DaHKzXx9pWO+ZqjK2EGoAgAAAGoaQpUHyAg2Q1Xu7kNurgQAAABAeRGqPEBOuBmqbAfoqQIAAABqGkKVByioZ4Yqy2FCFQAAAFDTEKo8gCXWDFXWY4QqAAAAoKYhVHkAayMzVAVlEKoAAACAmoZQ5QGCW5ihKiybUAUAAADUNB4Tqp566ilZLBZNnDjRMS83N1fjx49XZGSkgoODNWzYMCUnJ7uvyCoS1vp/PVX2LCkry83VAAAAACgPjwhVa9eu1ZtvvqkOHTo4zb/33nv11Vdf6ZNPPtGKFSt06NAhDR061E1VVp3o5iHKUpAkyThEbxUAAABQk7g9VGVlZWnkyJH6z3/+o4iICMf89PR0vf3223rhhRfUu3dvde7cWXPmzNEvv/yi1atXu7Fi14uJkZJk9lZl8gBgAAAAoEbxcXcB48eP15VXXqm+ffvq8ccfd8xft26dbDab+vbt65jXunVrNWzYUKtWrVK3bt1K3F9eXp7y8vIc7zMyMiRJNptNNputis6ibIqOf3od3t7SUZ9YtSjYpdQtBxTQ3711wvOV1paA8qItwRVoR3AV2hJcpbS2VFVty62hat68efr999+1du3aYssOHz4sq9Wq8PBwp/nR0dE6fPhwqfucNWuWZsyYUWz+d999p8DAwErX7ApLliwpNs/LWlcqkLYv/1UbWwe7oSrURCW1JaAiaEtwBdoRXIW2BFc5vS1lZ2dXyXHcFqoOHDige+65R0uWLJG/v7/L9vvwww/rvvvuc7zPyMhQfHy8Lr/8coWGhrrsOBVhs9m0ZMkS9evXT76+vk7Lvoz8QcqWYmVVm4ED3VQhaooztSWgPGhLcAXaEVyFtgRXKa0tFV3F5mpuC1Xr1q3TkSNHdMEFFzjmFRYW6scff9Rrr72mxYsXKz8/X2lpaU69VcnJyYqJiSl1v35+fvLz8ys239fX12N+OEuqpaBunHRA8kpO9pg64fk8qV2jZqMtwRVoR3AV2hJc5fS2VFXtym2hqk+fPtq0aZPTvLFjx6p169aaPHmy4uPj5evrq6VLl2rYsGGSpO3bt2v//v1KSEhwR8lVKzZWWi/5pjBQBQAAAFCTuC1UhYSE6LzzznOaFxQUpMjISMf8cePG6b777lOdOnUUGhqqu+66SwkJCaUOUlGT+TY0R/8LTCdUAQAAADWJ20f/O5MXX3xRXl5eGjZsmPLy8tS/f3+9/vrr7i6rSgQ2M0NVaDahCgAAAKhJPCpULV++3Om9v7+/Zs+erdmzZ7unoGoU1toMVWEFqVJenlTCfWEAAAAAPI/bH/4LU71WdZQnqyTJSCp9yHgAAAAAnoVQ5SFi4yw6LHNUw+zdXAIIAAAA1BSEKg8RFCQd8TIvAUzbRqgCAAAAagpClQdJCzRDVdYuQhUAAABQUxCqPEh2qBmqbH8RqgAAAICaglDlQfIjzVBlHCJUAQAAADUFocqDGDFmqPI5SqgCAAAAagpClQfxiTdDlX8aoQoAAACoKQhVHiSgqRmqQk8QqgAAAICaglDlQUJbmaEqPP+IVFjo5moAAAAAlAWhyoNEtolSobzkLbt05Ii7ywEAAABQBoQqDxLbwFtHFCVJytnDJYAAAABATUCo8iChoVKyxbwEMG0boQoAAACoCQhVHsRikY4HmKEqcyehCgAAAKgJCFUe5kSIGary9xGqAAAAgJqAUOVh8uqYocp+kFAFAAAA1ASEKg9jjzZDlfcRQhUAAABQExCqPIx3AzNU+R0nVAEAAAA1AaHKw/g3MUNVSBahCgAAAKgJCFUeJqSlGaoi8g5LhuHmagAAAACcDaHKw9RpGyNJshr5Umqqm6sBAAAAcDaEKg8T08hPx1RHkpT/F5cAAgAAAJ6OUOVh6tSRDlvMSwBTtxCqAAAAAE9HqPIwFouU6meGqsydhCoAAADA0xGqPFBWsBmq8vYSqgAAAABPR6jyQLkRZqgq/JtQBQAAAHg6QpUHKow2Q5VXMqEKAAAA8HSEKg/kVd8MVdZUQhUAAADg6QhVHsivkRmqgjMJVQAAAICnI1R5oOAWZqiKyCVUAQAAAJ6OUOWBItqaoSrQfkLKzHRzNQAAAADOhFDlgWKaBytTwZIk2356qwAAAABPRqjyQHXrSkkye6uObyVUAQAAAJ6MUOWBvLykVKsZqjJ3EKoAAAAAT0ao8lAZQWaoytlDqAIAAAA8GaHKQ2VHxEmSCg4QqgAAAABPRqjyUIX1zJ4qSzKhCgAAAPBkhCoPZYkzQ5U1hVAFAAAAeDJClYeyNjJDVVAGoQoAAADwZIQqDxXU3AxV4TmH3FwJAAAAgDMhVHmo8DZmqAotTJNyctxbDAAAAIBSEao8VHSrcOXKT5JUePCwm6sBAAAAUBpClYeKirYoSWZvVdo27qsCAAAAPBWhykP5+EgpvmaoSt9OqAIAAAA8FaHKg2UEmqEqZzehCgAAAPBUhCoPlh1mhirbfkIVAAAA4KkIVR6soJ4ZqpREqAIAAAA8FaHKk8Waoco3hVAFAAAAeCpClQfzbWiGqsB0QhUAAADgqQhVHiywmRmqwrIJVQAAAICnIlR5sLDWZqgKLzgqFRS4uRoAAAAAJSFUebB6beupQN7ykiHjcLK7ywEAAABQAkKVB4uJ81KyoiVJ6X9yCSAAAADgiQhVHsxqlY56m5cApm0jVAEAAACeiFDl4dICzVB1YjehCgAAAPBEhCoPlx1qhirbX4QqAAAAwBMRqjxcfl0zVNkPEaoAAAAAT0So8nBGjBmqfI8SqgAAAABPRKjycL7xZqgKSCNUAQAAAJ6IUOXh/JuaoSrkBKEKAAAA8ESEKg8X1soMVXXyD0t2u5urAQAAAHA6QpWHi2xrPvzXVwUyUo65uRoAAAAApyNUebjYRlYdVV1JUuYOLgEEAAAAPE2FQtWcOXOUnZ3t6lpQgoAA6YiXeQlg2jZCFQAAAOBpKhSqHnroIcXExGjcuHH65ZdfXF0TTnM8wAxVWTsJVQAAAICnqVCoOnjwoN59912lpKSoV69eat26tZ5++mkdPnzY1fVB0okQM1Tl/0WoAgAAADxNhUKVj4+PhgwZoi+++EIHDhzQrbfeqg8//FANGzbU1VdfrS+++EJ2Rqpzmbw6ZqgqPEioAgAAADxNpQeqiI6O1iWXXKKEhAR5eXlp06ZNGj16tJo1a6bly5e7oETYY8xQ5X2EUAUAAAB4mgqHquTkZD333HNq166devXqpYyMDH399dfau3evDh48qOuuu06jR48+4z7eeOMNdejQQaGhoQoNDVVCQoIWLlzoWJ6bm6vx48crMjJSwcHBGjZsmJKTkytaco3l3cAMVf7HCVUAAACAp6lQqBo0aJDi4+OVmJioW2+9VQcPHtRHH32kvn37SpKCgoJ0//3368CBA2fcT4MGDfTUU09p3bp1+u2339S7d29dc8012rJliyTp3nvv1VdffaVPPvlEK1as0KFDhzR06NCKlFyjBTQxQ1VIFqEKAAAA8DQ+FdkoKipKK1asUEJCQqnr1KtXT3v37j3jfgYNGuT0/oknntAbb7yh1atXq0GDBnr77bc1d+5c9e7dW5I5lHubNm20evVqdevWrcR95uXlKS8vz/E+IyNDkmSz2WSz2cp0flWl6PjlrSOwWT1JUp28JNny8yWLxeW1oWapaFsCTkdbgivQjuAqtCW4SmltqaraVoVCVc+ePXXBBRcUm5+fn6958+Zp1KhRslgsatSoUZn3WVhYqE8++UQnTpxQQkKC1q1bJ5vN5uj9kqTWrVurYcOGWrVqVamhatasWZoxY0ax+d99950CAwPLXE9VWrJkSbnWTz7mrYslBRg5+ubTT1UQFFQ1haHGKW9bAkpDW4Ir0I7gKrQluMrpbamqnrVrMQzDKO9G3t7eSkpKUlRUlNP8Y8eOKSoqSoWFhWXe16ZNm5SQkKDc3FwFBwdr7ty5GjhwoObOnauxY8c69TpJ0kUXXaTLLrtMTz/9dIn7K6mnKj4+XikpKQoNDS3HWbqezWbTkiVL1K9fP/n6+pZ5u8xMyTuyrsKUofTVfyjwgtZVWCVqgoq2JeB0tCW4Au0IrkJbgquU1pYyMjJUt25dpaenuzQbVKinyjAMWUq4BO3vv/9WWFhYufbVqlUrbdiwQenp6fr00081evRorVixoiJlSZL8/Pzk5+dXbL6vr6/H/HCWt5Y6daQdlliFGRnK3JGisK6ecR5wP09q16jZaEtwBdoRXIW2BFc5vS1VVbsqV6jq1KmTLBaLLBaL+vTpIx+fk5sXFhZq7969uuKKK8pVgNVqVfPmzSVJnTt31tq1a/Xyyy9r+PDhys/PV1pamsLDwx3rJycnKyYmplzHqA1S/WOlnO3K3MFgFQAAAIAnKVeoGjx4sCRpw4YN6t+/v4KDgx3LrFarGjdurGHDhlWqILvdrry8PHXu3Fm+vr5aunSpY5/bt2/X/v37zzhARm2VFRwr5Ui5ewlVAAAAgCcpV6iaNm2aJKlx48YaPny4/P39K3Xwhx9+WAMGDFDDhg2VmZmpuXPnavny5Vq8eLHCwsI0btw43XfffapTp45CQ0N11113KSEhodRBKmqz3IhY6ahUeJBQBQAAAHiSCt1TdbaH+pbVkSNHNGrUKCUlJSksLEwdOnTQ4sWL1a9fP0nSiy++KC8vLw0bNkx5eXnq37+/Xn/9dZccu6YpjI6VdkjeyYQqAAAAwJOUOVTVqVNHO3bsUN26dRUREVHiQBVFUlNTy7TPt99++4zL/f39NXv2bM2ePbusZdZa3vXNBwD7pRKqAAAAAE9S5lD14osvKiQkxPH6TKEKrufX2AxVQZmEKgAAAMCTlDlUnXrJ35gxY6qiFpxBcAszVEXkEqoAAAAAT+JVkY0SExNLnF9QUKCHH364MvWgFBFtzVAVak+XcnLcXA0AAACAIhUKVXfffbeuvfZaHT9+3DFv+/bt6tq1qz766COXFYeToluGKUfmaIsMqw4AAAB4jgqFqvXr1+vvv/9W+/bttWTJEs2ePVsXXHCBWrdurY0bN7q6RkgKj7DosMzeqtQthCoAAADAU1RoSPVmzZpp5cqVmjhxoq644gp5e3vr3Xff1YgRI1xdH/7HYpGO+cWqSd5eZe4gVAEAAACeokI9VZL0zTffaN68eUpISFB4eLjefvttHTp0yJW14TSZQWZPVfYeQhUAAADgKSoUqv75z3/q2muv1eTJk/XTTz/pjz/+kNVqVfv27fXf//7X1TXif3IizFBVeIBQBQAAAHiKCoWqlStXas2aNbr//vtlsVgUExOjb7/9VjNnztTNN9/s6hrxP4X1zFBlOUyoAgAAADxFhe6pWrdunfz8/IrNHz9+vPr27VvpolAyS5wZqqzHCFUAAACAp6hQT5Wfn592796tKVOmaMSIETpy5IgkaeHChSooKHBpgTjJr7EZqoIyCFUAAACAp6hQqFqxYoXat2+vNWvWaP78+crKypIkbdy4UdOmTXNpgTgpqLkZqsJzCFUAAACAp6hQqHrooYf0+OOPa8mSJbJarY75vXv31urVq11WHJyFtYmTJNUpPCrZbG6uBgAAAIBUwVC1adMmDRkypNj8qKgopaSkVLoolCyqTaRs/7sNLv9AspurAQAAACBVMFSFh4crKan4JWjr169X/fr1K10UShZZz0uHFSNJOr6VSwABAAAAT1ChUHX99ddr8uTJOnz4sCwWi+x2u1auXKlJkyZp1KhRrq4R/+PlJR2zmvdVpW0jVAEAAACeoEKh6sknn1Tr1q0VHx+vrKwstW3bVj169NDFF1+sKVOmuLpGnCIj0AxVObsPubkSAAAAAFIFn1NltVr1n//8R48++qg2b96srKwsderUSS1atHB1fThNdnislCYVHKCnCgAAAPAEFQpVRRo2bKiGDRu6qhaUQUHdWGmfpBLuaQMAAABQ/cocqu67774y7/SFF16oUDEogzjz8j/fFEIVAAAA4AnKHKrWr19fpvUsFkuFi8HZWRuaoSownVAFAAAAeIIyh6ply5ZVZR0oo8BmZqgKyyFUAQAAAJ6gQqP/nerAgQM6cOCAK2pBGYS1NkNVHVuyVFjo5moAAAAAVChUFRQU6NFHH1VYWJgaN26sxo0bKywsTFOmTJHNZnN1jThF3XbRsssiHxWqMDnF3eUAAAAA57wKjf531113af78+XrmmWeUkJAgSVq1apWmT5+uY8eO6Y033nBpkTgpKs5HR1VP0Tqi1C1JqhcX7e6SAAAAgHNahULV3LlzNW/ePA0YMMAxr0OHDoqPj9eIESMIVVXI21tK8YlVdMERpW1LUr1+57u7JAAAAOCcVqHL//z8/NS4ceNi85s0aSKr1VrZmnAWaYHmfVXZuxmsAgAAAHC3CoWqCRMm6LHHHlNeXp5jXl5enp544glNmDDBZcWhZNmhZqjK/4tQBQAAALhbhS7/W79+vZYuXaoGDRqoY8eOkqSNGzcqPz9fffr00dChQx3rzp8/3zWVwiE/Mlb6WzKSCFUAAACAu1UoVIWHh2vYsGFO8+Lj411SEMogNlbaKPkeJVQBAAAA7lbuUGUYhmbMmKF69eopICCgKmrCWfjEm5f/BaQRqgAAAAB3K/c9VYZhqHnz5vr777+roh6UQUAzM1SFniBUAQAAAO5W7lDl5eWlFi1a6NixY1VRD8ogtJUZqurkJ0mG4eZqAAAAgHNbhUb/e+qpp/TAAw9o8+bNrq4HZVD3PDNU+StP9tQ09xYDAAAAnOMqNFDFqFGjlJ2drY4dO8pqtRa7tyo1NdUlxaFk0Y38dVzhilCa0rYlqc4lEe4uCQAAADhnVShUvfTSSy4uA+Xh6ysd8Y5VRGGajm9NUp1L2rq7JAAAAOCcVaFQNXr0aFfXgXJKC4iVsrYpaxeDVQAAAADuVKF7qiRp9+7dmjJlikaMGKEjR45IkhYuXKgtW7a4rDiU7kSIeV9V/j5CFQAAAOBOFQpVK1asUPv27bVmzRrNnz9fWVlZkqSNGzdq2rRpLi0QJcuLNEOV/RChCgAAAHCnCoWqhx56SI8//riWLFkiq9XqmN+7d2+tXr3aZcWhdEa0Gaq8jxCqAAAAAHeqUKjatGmThgwZUmx+VFSUUlJSKl0Uzs67wf+GVT9OqAIAAADcqUKhKjw8XElJxf+YX79+verXr1/ponB2/k3MUBWSRagCAAAA3KlCoer666/X5MmTdfjwYVksFtntdq1cuVKTJk3SqFGjXF0jShDS0gxVdfIIVQAAAIA7VShUPfnkk2rTpo0aNmyorKwstW3bVj169NDFF1+sKVOmuLpGlCDyvP/1VBmZMrJOuLkaAAAA4NxVrudU2e12Pfvss/ryyy+Vn5+vm266ScOGDVNWVpY6deqkFi1aVFWdOE108xCdUKCClK30P5MUfmFzd5cEAAAAnJPKFaqeeOIJTZ8+XX379lVAQIDmzp0rwzD0zjvvVFV9KIV/gEV7vGLV1L5bx7cSqgAAAAB3Kdflf++9955ef/11LV68WAsWLNBXX32lDz/8UHa7varqwxkc9zMvAczcyX1VAAAAgLuUK1Tt379fAwcOdLzv27evLBaLDh065PLCcHZZIWaoyttLqAIAAADcpVyhqqCgQP7+/k7zfH19ZbPZXFoUyia3jhmqCg8SqgAAAAB3Kdc9VYZhaMyYMfLz83PMy83N1e23366goCDHvPnz57uuQpTKHhUr/Sl5JROqAAAAAHcpV6gaPXp0sXk33nijy4pB+XjVN3uq/FIJVQAAAIC7lCtUzZkzp6rqQAX4Nf7fs6oyCVUAAACAu1To4b/wDMEtzFAVnkuoAgAAANyFUFWD1Wlnhqo69mMy8vLdXA0AAABwbiJU1WDRbSOVL19JUtauw26uBgAAADg3EapqsKBgi5ItMZKk1C1cAggAAAC4A6Gqhkv1My8BzNxBqAIAAADcgVBVw2UGmaEqZw+hCgAAAHAHQlUNlxthhqrCvwlVAAAAgDsQqmq4gnpmqNJhQhUAAADgDoSqGs5S3wxVfscIVQAAAIA7EKpqOL9GZqgKyiBUAQAAAO5AqKrhglqYoSo8l1AFAAAAuAOhqoaLaGOGqsiCZKmw0M3VAAAAAOceQlUNV69dlOyyyFt2Zf911N3lAAAAAOccQlUNF1rHR0cVJUk6tplLAAEAAIDqRqiq4SwWKcVqXgKYsZ1QBQAAAFQ3t4aqWbNmqUuXLgoJCVFUVJQGDx6s7du3O62Tm5ur8ePHKzIyUsHBwRo2bJiSk5PdVLFnygiKkyRl7yFUAQAAANXNraFqxYoVGj9+vFavXq0lS5bIZrPp8ssv14kTJxzr3Hvvvfrqq6/0ySefaMWKFTp06JCGDh3qxqo9T3a42VNl20+oAgAAAKqbjzsPvmjRIqf3iYmJioqK0rp169SjRw+lp6fr7bff1ty5c9W7d29J0pw5c9SmTRutXr1a3bp1c0fZHqegbqy0V7IkEaoAAACA6ubWUHW69PR0SVKdOnUkSevWrZPNZlPfvn0d67Ru3VoNGzbUqlWrSgxVeXl5ysvLc7zPyMiQJNlsNtlstqos/6yKju/qOoyYaEmST8oht58jqkdVtSWce2hLcAXaEVyFtgRXKa0tVVXb8phQZbfbNXHiRHXv3l3nnXeeJOnw4cOyWq0KDw93Wjc6OlqHDx8ucT+zZs3SjBkzis3/7rvvFBgY6PK6K2LJkiUu3V+KPVeSZD12QN9++61L9w3P5uq2hHMXbQmuQDuCq9CW4Cqnt6Xs7OwqOY7HhKrx48dr8+bN+vnnnyu1n4cfflj33Xef431GRobi4+N1+eWXKzQ0tLJlVorNZtOSJUvUr18/+fr6umy/v+1ZI30j1bMdVduBA122X3iuqmpLOPfQluAKtCO4Cm0JrlJaWyq6is3VPCJUTZgwQV9//bV+/PFHNWjQwDE/JiZG+fn5SktLc+qtSk5OVkxMTIn78vPzk5+fX7H5vr6+HvPD6epa6rSLlyRF2g7L18fHHGcd5wRPateo2WhLcAXaEVyFtgRXOb0tVVW7cuvof4ZhaMKECfr888/1ww8/qEmTJk7LO3fuLF9fXy1dutQxb/v27dq/f78SEhKqu1yPVfc8M2D6KV95SalurgYAAAA4t7i1p2r8+PGaO3euvvjiC4WEhDjukwoLC1NAQIDCwsI0btw43XfffapTp45CQ0N11113KSEhgZH/TlEn1k/HVEeRStWxzUmKi4t0d0kAAADAOcOtPVVvvPGG0tPT1atXL8XGxjqmjz/+2LHOiy++qKuuukrDhg1Tjx49FBMTo/nz57uxas9jsUgpvuazqtL/ZFh1AAAAoDq5tafKMIyzruPv76/Zs2dr9uzZ1VBRzZUeGCulb9GJXYQqAAAAoDq5tacKrpMdavZU5e8nVAEAAADViVBVS+TXNUOVDhGqAAAAgOpEqKotYsxQ5ZNCqAIAAACqE6GqlvBtaIaqwDRCFQAAAFCdCFW1REBTM1SFniBUAQAAANWJUFVLhLYyQ1WkjVAFAAAAVCdCVS1Rt70ZqoKMEyo4nunmagAAAIBzB6GqlqjbOFiZCpYkHdtMbxUAAABQXQhVtYSXl3TU2+ytOr6VUAUAAABUF0JVLZIWYIaqrJ2EKgAAAKC6EKpqkaxQM1Tl/0WoAgAAAKoLoaoWya9jhirjEKEKAAAAqC6EqlrEiDFDlfcRQhUAAABQXQhVtYhPvBmq/NMIVQAAAEB1IVTVIv5NzFAVkkWoAgAAAKoLoaoWCWlphqo6eYQqAAAAoLoQqmqROu3MUBVhHFfhiVw3VwMAAACcGwhVtUhUqwjlyk+SlLr1sJurAQAAAM4NhKpaxMfXoqNeMZKk41u5BBAAAACoDoSqWua4v3kJYOYOQhUAAABQHQhVtUxmiBmq8vYRqgAAAIDqQKiqZfIizFBlP0ioAgAAAKoDoaqWsUebocormVAFAAAAVAdCVS3j3cAMVX7HCVUAAABAdSBU1TJ+jc1QFZxJqAIAAACqA6Gqlglu8b8HAOcSqgAAAIDqQKiqZSLamqGqrv2IDFuBm6sBAAAAaj9CVS0T1a6eCuUlLxk6vv2Iu8sBAAAAaj1CVS3jF+ito5ZoSVLqFi4BBAAAAKoaoaoWSvUzLwHM2E6oAgAAAKoaoaoWygg2Q1XuXkIVAAAAUNUIVbVQbrgZqgr/JlQBAAAAVY1QVQsVRpmhypJMqAIAAACqGqGqFvKqb4Yq6zFCFQAAAFDVCFW1kLWRGaqCMwhVAAAAQFUjVNVCQc3NUBWeQ6gCAAAAqhqhqhaKaGuGqrqFh2XYDTdXAwAAANRuhKpaqF77GEmSVTZl7jvm5moAAACA2o1QVQsFhlt1zBIpSUrZxCWAAAAAQFUiVNVSx6zmJYAZ2wlVAAAAQFUiVNVSGYFmqMreTagCAAAAqhKhqpbKDjdDVcEBQhUAAABQlQhVtVRBPTNU6TChCgAAAKhKhKpayhJrhiprCqEKAAAAqEqEqlrKt1GcJCkwnVAFAAAAVCVCVS0V1NzsqQrLIVQBAAAAVYlQVUuFtzFDVV1bkmQYbq4GAAAAqL0IVbVU3fZmqApStrKSMt1cDQAAAFB7EapqqZDoQKUrVJKUsolLAAEAAICqQqiqxVJ8zd6q9D8JVQAAAEBVIVTVYumBZqjK3nXIzZUAAAAAtRehqhbLCTNDlW0/PVUAAABAVSFU1WL5dc1QpSRCFQAAAFBVCFW1WYwZqnyOEqoAAACAqkKoqsV8G5qhKiCdUAUAAABUFUJVLRbQ1AxVoScIVQAAAEBVIVTVYmGtzVBV10aoAgAAAKoKoaoWq9veDFVhRrpyUnPcXA0AAABQOxGqarGwhmHKkb8kKWUTvVUAAABAVSBU1WIWL4uO+pi9VWnbCFUAAABAVSBU1XJpAWaoOrGLUAUAAABUBUJVLXci1AxVtt373VwJAAAAUDsRqmq5/HoNJEndF0zS5oYDdPiNzyWbzc1VAQAAALUHoaqWa/TcXVob1ldeMnTegUWKuXOojoc2VPK4f0l79ri7PAAAAKDGI1TVco37NFOXtCX6bd4u/bfpQ0pWlCJyDyv6nVlSs2Y63uVy6bPP6L0CAAAAKohQdY64cHgzXbd7lpLXHtCL3T/Vd7pckhTx2xLpH/9QblS8jMkPSbt2ublSAAAAoGYhVJ1jOlxo1b0/D1OzXYs1ZcRuPeX1LyUpRv5pybI887TUooXsffpK//2vlJ/v7nIBAAAAj+fWUPXjjz9q0KBBiouLk8Vi0YIFC5yWG4ahqVOnKjY2VgEBAerbt6927tzpnmJrmWbNpMfnNtXov5/Qy/ft1w3+87VQV8gui7x+WCoNHy6jQQPpwQclPnMAAACgVG4NVSdOnFDHjh01e/bsEpc/88wzeuWVV/Tvf/9ba9asUVBQkPr376/c3NxqrrT2io2VnnreV7MPDdHvjy/UhRF79Jim6JBiZTl6VHr2WallS6l3b2nePCkvz90lAwAAAB7FraFqwIABevzxxzVkyJBiywzD0EsvvaQpU6bommuuUYcOHfTee+/p0KFDxXq0UHkREdIjj0g//91Y4a88pkvi9+saLdA3Gii7LNKyZdKIEVL9+tKkSdL27e4uGQAAAPAIPu4uoDR79+7V4cOH1bdvX8e8sLAwde3aVatWrdL1119f4nZ5eXnKO6U3JSMjQ5Jks9lkc/MId0XHd3cdZ+LrK91+uzRunDRv3pWa9OzVuuPPAxqnt3WL3lb9Ywel55+Xnn9e9h49ZB83TsaQIZK/v7tLP6fUhLaEmoG2BFegHcFVaEtwldLaUlW1LYthGEaV7LmcLBaLPv/8cw0ePFiS9Msvv6h79+46dOiQYmNjHetdd911slgs+vjjj0vcz/Tp0zVjxoxi8+fOnavAwMAqqb02s9ulX3+N0WeftdSenSEaoIX6p97UAC2Ut+ySpPyQEO2/7DL91a+fsuLj3VwxAAAAULLs7GzdcMMNSk9PV2hoqMv2W+tCVUk9VfHx8UpJSXHpB1cRNptNS5YsUb9+/eTr6+vWWsrLMKTlyy165hkvLV3qpQY6oJv1jib4v616uQcc69kvucTsvRo6VAoIcGPFtVtNbkvwLLQluALtCK5CW4KrlNaWMjIyVLduXZeHKo+9/C8mJkaSlJyc7BSqkpOTdf7555e6nZ+fn/z8/IrN9/X19ZgfTk+qpTwuv9ycfvtNeuqpeD02f5oez52i/lqsf0X+ny4+/rW8fv5ZXj//LN13n3TttdJVV5mDXAQFubv8WqmmtiV4HtoSXIF2BFehLcFVTm9LVdWuPPY5VU2aNFFMTIyWLl3qmJeRkaE1a9YoISHBjZXhwgulTz+Vtm6VRo/11hKfgbr02ALF2//SG3GP6UTdhtLx49L//Z909dVSZKQ0cKA0e7a0b5+7ywcAAABcyq2hKisrSxs2bNCGDRskmYNTbNiwQfv375fFYtHEiRP1+OOP68svv9SmTZs0atQoxcXFOS4RhHu1bi298460Z480caKUFlhfdx6aotCUPRpb/zttuGS8bA0am8OwL1woTZggNWkinXeeNHmy9NNPUkGBu08DAAAAqBS3hqrffvtNnTp1UqdOnSRJ9913nzp16qSpU6dKkh588EHddddduu2229SlSxdlZWVp0aJF8mekOY8SHy+9+KL011/S1KlSWIS3Eg/2U6efX5P17z0a2nKzvuvztDI7XSrD21vaskV65hmpRw8pKkq64Qbpww+lY8fcfSoAAABAubn1nqpevXrpTONkWCwWzZw5UzNnzqzGqlBRdetKM2aYj7FKTJQ++0z66SeLPt/RTp/vaCfpQZ0Xl6p72y3WFQXfKHbjQllSU6WPPjInLy8pIcG8D+vKK80eLYvF3acFAAAAnJHH3lOFmiskRLrrLmn5cunIEendd6WhQ82xKjYfqqNxS0ao/rIPFJF/RFN7/6wtVz+swrbtzfHbV66UHn5Y6tBBatxYuvNO6ZtvpJwcd58WAAAAUCJCFapUZKQ0apTZa5WSIn39tXTrrVJMjJSe5a3Hfuiu8758Uv47/tCIi//SsmtfV3bvK82HCe/fL73xhtlzFRlpfv33v6UDB85+YAAAAKCaeOyQ6qh9/P3Nq/quvNLslFq7VvriC3PaulWa90tDzdMdku5Q1/bZmtBumfrbvlbdX7+R5cABs8fqm2/MnXXocHJn3bpJ3t5uPTcAAACcuwhVcAsvL6lrV3N68klp166TAWvlSmnNpkCt2XSlpCsV38DQ7ddv1rUBX6vZ9m/ktXqV9Mcf5jRrltmLdcUVZk9Wv37mewAAAKCaEKrgEZo3l+6/35xSUswOqS++kBYvlg78bdEj89rrEbVXaOjDuvbqY7o5bpEuPPy1rD8sMkcN/PBDc5LMETOaN5datCj+NTzcrecJAACA2odQBY9Tt640erQ55eRIS5eaAeurr6TkZOntBZF6WyPl4zNSvXsU6J/tf1HfvG8U+uPX5nWEKSnmtHp18Z1HRpYctpo3lyIiqv9kAQAAUOMRquDRAgLMq/quusq8D2vNGunLL82QtW2b9N0PPvruhx6Seuj885/W0Aez1LvhLnUM2qXgpJ3mdYU7//c1Kcns1Tp2rPTAVVoPF4ELAAAApSBUocYoeoxVQoJ5K9XOnc73YW3YIG3YEKypOl/S+Wrd2hzDIuFG82u7Rlny3rf7ZMg69eupgWvNmuIHr1On9B6uOnWq+ZMAAACAJyFUocZq0cJ80PCkSdLRo+Zw7T/8YHZC7dol/fmnOSUmmusHBwfroos6KiGhoxISpK63mJcaSpKysqTdZwhcqalm2CotcLVsaRbUsuXJqUUL8+FcAAAAqNUIVagV6tWTxo41J8kMWWvWSKtWmSHr11/N3PTDD+ZUpEWL//VmJQSrW7eOaj+4o3xO/6k4ccIMWKeHrV27pEOHzMC1enXJlxTWr188bLVsKTVpIlmtVfZ5AAAAoPoQqlAr1at38l4sSSoslLZsORmyVq2Stm83s9HOndL775vrBQZKXbqYlxiaYUuKigqSOnY0p9MVBa6dO6UdO8yp6HVKinTwoDktX+68nbe31Lhx8bDVsqXUoIF5rSMAAABqBEIVzgne3ubzgjt0kP75T3Ne0RV9RSFrzRopI0NascKcijRp4hyyOnaUfH3/tzDoDIErNdU5bJ0auE6cMC833L1bWrjQeTt/f/NerdMvJWzZ0kyLFkuVfEYAAACoGEIVzll16kgDBpiTZI4uuG3byZC1erU5QvveveY0d665nr+/dOGFJ4NWt25SbGwJWadOnZNPOD6VYZj3aZ0etHbsMENWbq60ebM5nS4sTGrZUt7Nm6ttbq68/vjDHLUwPNxcdurX8HAz9BHCAAAAqhShCvgfLy+pXTtzGjfOnJeebt6PVRSyVq+Wjh+Xfv7ZnIqEhZ0cEPD0wQHr1j0t11gsUlycOfXq5VxEQYH011/Fw9aOHdL+/WZBa9fKa+1atZCkzz8/80l5e5vFnRq0Tg9fZ1tW7CYzAAAAnIq/loAzCAuT+vUzJ8nszdq50/nerM2bzayzbp05lbSP0h5/VSxw+fhIzZqZU1EXWpGcHMcIhYXbtmnvmjVqWqeOvDIyzALS0syp6HVBgXkzWWqqOVVUUFDZwldpywIC6C0DAAC1GqEKKAcvL6lVK3MaM8acl5Mj7dlT8mjsBw64MHAFBEjnnSedd57sV12lLd9+q0YDB8rLcYPXKQzDLOzUkHXq69O/ljTvxAlzXydOmNPBgxX70Hx8Kt5LFh4uhYaaPW4AAAAeilAFVFJAwMnLBk9XmcAVGlpy2GrRwswbZ2SxmEMZBgaalxlWhM1mjtxRUvgqa1Cz280es5QUc6oof3/zgw4IMM+p6PXp78+0rKzb+vrSswYAAMqFUAVUobIErpIef3XggJlnSg9cPgoPv0wvvuitqChzUMDSpsjICt4W5etrbhwZWYGNZfaWnThR/h6yU5fl5Jj7ys01p+PHK1ZLeXh5nQxZQUEnp9PfV3Se1UpoAwCgliFUAW5ytsC1d69z2HIOXBZlZIRq//6zH8dikSIidNbwVTTVreui5xJbLFJwsDk1aFCxfeTnmyErJ0fKzja/nul1ZdYzDPOYdvvJSx6PHnXBB3Eab+/Sw1fRFBxcvq9FrxlUBAAAt+BfYMADBQRIbdua0+lyc6Xt22364ou1atLkIh0/7qOjR1XilJpqZoWisSr+/LNsxw8LKz10RUScnOrUOfk6MLAKOmCsVvOgVc0wzAB3asjKzj4Zrk59XdF5Npt5rMJCsxsyI8P152G1ljuQWfz9Fb9liywpKWe/d62s3+CyrGe1nrx37tSJxwAAAGogQhVQw/j7m2Fr376jGjjQUEnjVBQpKJCOHSs5cJU0paSYHTVFt03t2lX2unx9Sw5bpYWwU+cFBFT+c6kUi0Xy8zOn8PCqOYbNdvbwlZV18n3R6zPNy8oyp8JC8xj5+eUe7dFH0gVVc8YVc+pjAEqaSgpip89nxEkAQDUjVAG1mI+PFB1tTmVht5u3LZUUuI4cMQPa8ePm3+zHj5+cCgrMzHDkiDmVl5/fmQNYWJg5cEfR16Kp6H1QkHkrlEfz9T050qErFfWylRa6zhLM7BkZOpKcrKioKHmVFESKLoss7dgVWSaZXa6n3keXnm42QFc8BsDHp+TAFRpqNjartfSvlZ3HPXMAcE4iVAFw8PI6OTZF69Zl26ZoPIqSwlbRVNr848fNv6Hz8qTDh82pIiwWKSTEOXSVFsDOFs5q3N/Dp/ay1alT7s0LbTat+fZbDSxteP7qUtSQigLWqaNMnj6vtGUZGSdHnDx2zJzcwde3eNDy9T05+fg4v6+KycfH/IEubbJYzry8vOsVFso/NdX8X5XAQOdzLNoPANRihCoAlXLqeBTx8eXb1jCkzMyzB7GiW5CK/m4+9X1BgbkfV9ym5OXlHM7OdBVaaVNoaA3oNfNEpzak+vUrtg/DMHvgzhS68vPNKS/P+Wtl5hVdflnEZjt5D905wldS/zOuUEKgdNVrb2/zvbd38ddnWlbZbSyWk6GzPK8rsk1ReLVaeW4f4KEIVQDcxmI52VvUqFH5tzcM8yqy0gJXae9LWlZY6Hw/WWUUBbOzTaXdHhQSwt9NFVLUZRkSUvERJyuisLB44Cp6nZd3MmSVZSq6lrayU9H/NtjtpU9nW16OdQ27XUZBgbzs9pI/o3MwaFYZHx+zJ9Tfv+SvZ1pWkW28vMzvf9FU1B7ONK8i6/zvvcVmU+zvv8uSl1exEU0r2ita1OsfEOD8bMSiqWgev5xRCkIVgBrLYjn5711Z7xsriWGYg/6dGrIqMuXlmfvLzDSnv/+ueE1lDWZn6jHj3/5q4u19siGeowpsNn377bcaOGCAfL28Sg6JVfHaZjNDbWGhOf9Mr8+2vLyvSwsI5X19tvsPi33YBeZ04kTVfDPdzEfSRe4u4kx8fYsHrdLel2Xe/y6fLbGtnakdVmZdu9255/XUryXNc9XXqCjpAo8aGsmlCFUAznkWi3kbSGCgFBNT8f3k5VU8kBVNubnmvlwRzIKDzx6+goO9tH17Ix05YpGfX/F/W0//N7a092Wdx+01tZzFcvKbfQ6HzHIrS29OUY9oXp75i6Kkr1WxzG4v/dLF8s4rw7p2ScfT0hQREVHy4DlVxW43zzcnxzz3U59rmJ9/cr2iMF8Vj8Wo7fr3lxYtcncVVYZQBQAu4udn/kdcVFTF95GXV3Jv2ZnGaygtmBWNuH7w4JmO6C3p/IoXXAHe3icHzfP3d77aqKT3rlqvKDSWFPoIenCrU8PFOa7QZtPPnjB4zqmKRlQ6NWiVFL5Of1+WeUW9RuW5z6889wGW9P7U3rGins+SenVP/eqKdZo1c/d3skoRqgDAg/j5nXzQckXl55c9gKWl2XXgQLLq1o2W3e7l+Hfx1OnUfy/LOu/0sRtOVVh48u8JT+HlVbYeuYq8P/02jbN9Pds6/N0NVDNv75OXMwClIFQBQC1jtZY9mNlshfr22181cOBA+fq67q/1oquVSgtgJV3FVB2vS2O3O1/h48ms1rMHs1N75YoGyzt1On3e2d6fbR3Jol27wrVhg3n807ctbSoaRA8AajpCFQDA5U69tcaTFD3G6vSgV1XvbTYzrJ16xU9Zv576uqDg5DkUDS7oWbd0+EjqWaEtS7oX72z36lVmcuX+KjMKPGESqF087J87AACqTtGjfmqagoKTvW1lCWR5ecV7B4tC3pneV3wdQxkZubJa/VVQYCkxbJY22nrRrR1Fo2eeK07t7StrICu6F7Ho2dJFr09/78rXFovzIHKumk4fnO5kO7Bo/fo45eRY5O9f8ceNleU14RauRKgCAMDD+ficfDayJ7LZCvTtt9/97zLSkgcXsNtLvhS0PNPp2xeNqF6ZfVZkOnVk96Kvpb0+tZfxVEX7wul8JHWptqN5eZ0MW0XPWD51OvXZy1W1/NQBEMs6ryLbFH09dbCg079W5vW5/mxqQhUAAKhyRX9EesqAbtWl6P5CVzyW69RnSpf0+mzLy7JdeZ/RXBRKyjOd3lvkPDCdXampxxQeHukYPMcVjxQrTdEzrHk2tWsUDc5TUui69FLpjTfcXWHVIVQBAABUkZr26C7DcA5Z0pkCkOsvnzMHz/nFpYPnFPWSnu3Z0EWPBjt9Km1+ZZeX9gzp07+6elnRI8/KGszPFMpPH+CnsFDKzjan0zVs6JJvp8ciVAEAAECSGZKKehhqi3O1l7Q6GIbZy1eWIBYe7u5qqxahCgAAAEC5WSzm5X1WqxQS4u5q3ItHCAIAAABAJRCqAAAAAKASCFUAAAAAUAmEKgAAAACoBEIVAAAAAFQCoQoAAAAAKoFQBQAAAACVQKgCAAAAgEogVAEAAABAJRCqAAAAAKASCFUAAAAAUAmEKgAAAACoBEIVAAAAAFQCoQoAAAAAKsHH3QVUNcMwJEkZGRlurkSy2WzKzs5WRkaGfH193V0OajDaElyFtgRXoB3BVWhLcJXS2lJRJijKCK5S60NVZmamJCk+Pt7NlQAAAADwBJmZmQoLC3PZ/iyGq2Oah7Hb7Tp06JBCQkJksVjcWktGRobi4+N14MABhYaGurUW1Gy0JbgKbQmuQDuCq9CW4CqltSXDMJSZmam4uDh5ebnuTqha31Pl5eWlBg0auLsMJ6GhofyigEvQluAqtCW4Au0IrkJbgquU1JZc2UNVhIEqAAAAAKASCFUAAAAAUAmEqmrk5+enadOmyc/Pz92loIajLcFVaEtwBdoRXIW2BFep7rZU6weqAAAAAICqRE8VAAAAAFQCoQoAAAAAKoFQBQAAAACVQKgCAAAAgEogVFWj2bNnq3HjxvL391fXrl3166+/urskuNGPP/6oQYMGKS4uThaLRQsWLHBabhiGpk6dqtjYWAUEBKhv377auXOn0zqpqakaOXKkQkNDFR4ernHjxikrK8tpnT/++EOXXnqp/P39FR8fr2eeeaaqTw3VaNasWerSpYtCQkIUFRWlwYMHa/v27U7r5Obmavz48YqMjFRwcLCGDRum5ORkp3X279+vK6+8UoGBgYqKitIDDzyggoICp3WWL1+uCy64QH5+fmrevLkSExOr+vRQjd544w116NDB8aDMhIQELVy40LGcdoSKeOqpp2SxWDRx4kTHPNoSymr69OmyWCxOU+vWrR3LPaotGagW8+bNM6xWq/HOO+8YW7ZsMW699VYjPDzcSE5OdndpcJNvv/3WeOSRR4z58+cbkozPP//caflTTz1lhIWFGQsWLDA2btxoXH311UaTJk2MnJwcxzpXXHGF0bFjR2P16tXGTz/9ZDRv3twYMWKEY3l6eroRHR1tjBw50ti8ebPx0UcfGQEBAcabb75ZXaeJKta/f39jzpw5xubNm40NGzYYAwcONBo2bGhkZWU51rn99tuN+Ph4Y+nSpcZvv/1mdOvWzbj44osdywsKCozzzjvP6Nu3r7F+/Xrj22+/NerWrWs8/PDDjnX27NljBAYGGvfdd5+xdetW49VXXzW8vb2NRYsWVev5oup8+eWXxjfffGPs2LHD2L59u/Gvf/3L8PX1NTZv3mwYBu0I5ffrr78ajRs3Njp06GDcc889jvm0JZTVtGnTjHbt2hlJSUmO6ejRo47lntSWCFXV5KKLLjLGjx/veF9YWGjExcUZs2bNcmNV8BSnhyq73W7ExMQYzz77rGNeWlqa4efnZ3z00UeGYRjG1q1bDUnG2rVrHessXLjQsFgsxsGDBw3DMIzXX3/diIiIMPLy8hzrTJ482WjVqlUVnxHc5ciRI4YkY8WKFYZhmO3G19fX+OSTTxzrbNu2zZBkrFq1yjAMM+B7eXkZhw8fdqzzxhtvGKGhoY628+CDDxrt2rVzOtbw4cON/v37V/UpwY0iIiKMt956i3aEcsvMzDRatGhhLFmyxOjZs6cjVNGWUB7Tpk0zOnbsWOIyT2tLXP5XDfLz87Vu3Tr17dvXMc/Ly0t9+/bVqlWr3FgZPNXevXt1+PBhpzYTFhamrl27OtrMqlWrFB4ergsvvNCxTt++feXl5aU1a9Y41unRo4esVqtjnf79+2v79u06fvx4NZ0NqlN6erokqU6dOpKkdevWyWazObWl1q1bq2HDhk5tqX379oqOjnas079/f2VkZGjLli2OdU7dR9E6/A6rnQoLCzVv3jydOHFCCQkJtCOU2/jx43XllVcW+37TllBeO3fuVFxcnJo2baqRI0dq//79kjyvLRGqqkFKSooKCwudvqGSFB0drcOHD7upKniyonZxpjZz+PBhRUVFOS338fFRnTp1nNYpaR+nHgO1h91u18SJE9W9e3edd955kszvs9VqVXh4uNO6p7els7WT0tbJyMhQTk5OVZwO3GDTpk0KDg6Wn5+fbr/9dn3++edq27Yt7QjlMm/ePP3++++aNWtWsWW0JZRH165dlZiYqEWLFumNN97Q3r17demllyozM9Pj2pJPeU8OAOCZxo8fr82bN+vnn392dymooVq1aqUNGzYoPT1dn376qUaPHq0VK1a4uyzUIAcOHNA999yjJUuWyN/f393loIYbMGCA43WHDh3UtWtXNWrUSP/9738VEBDgxsqKo6eqGtStW1fe3t7FRiNJTk5WTEyMm6qCJytqF2dqMzExMTpy5IjT8oKCAqWmpjqtU9I+Tj0GaocJEybo66+/1rJly9SgQQPH/JiYGOXn5ystLc1p/dPb0tnaSWnrhIaGetw/bKg4q9Wq5s2bq3Pnzpo1a5Y6duyol19+mXaEMlu3bp2OHDmiCy64QD4+PvLx8dGKFSv0yiuvyMfHR9HR0bQlVFh4eLhatmypXbt2edzvJUJVNbBarercubOWLl3qmGe327V06VIlJCS4sTJ4qiZNmigmJsapzWRkZGjNmjWONpOQkKC0tDStW7fOsc4PP/wgu92url27Otb58ccfZbPZHOssWbJErVq1UkRERDWdDaqSYRiaMGGCPv/8c/3www9q0qSJ0/LOnTvL19fXqS1t375d+/fvd2pLmzZtcgrpS5YsUWhoqNq2betY59R9FK3D77DazW63Ky8vj3aEMuvTp482bdqkDRs2OKYLL7xQI0eOdLymLaGisrKytHv3bsXGxnre76VyDWuBCps3b57h5+dnJCYmGlu3bjVuu+02Izw83Gk0EpxbMjMzjfXr1xvr1683JBkvvPCCsX79euOvv/4yDMMcUj08PNz44osvjD/++MO45pprShxSvVOnTsaaNWuMn3/+2WjRooXTkOppaWlGdHS0cdNNNxmbN2825s2bZwQGBjKkei1yxx13GGFhYcby5cudhpzNzs52rHP77bcbDRs2NH744Qfjt99+MxISEoyEhATH8qIhZy+//HJjw4YNxqJFi4x69eqVOOTsAw88YGzbts2YPXs2wxfXMg899JCxYsUKY+/evcYff/xhPPTQQ4bFYjG+++47wzBoR6i4U0f/MwzaEsru/vvvN5YvX27s3bvXWLlypdG3b1+jbt26xpEjRwzD8Ky2RKiqRq+++qrRsGFDw2q1GhdddJGxevVqd5cEN1q2bJkhqdg0evRowzDMYdUfffRRIzo62vDz8zP69OljbN++3Wkfx44dM0aMGGEEBwcboaGhxtixY43MzEyndTZu3Ghccsklhp+fn1G/fn3jqaeeqq5TRDUoqQ1JMubMmeNYJycnx7jzzjuNiIgIIzAw0BgyZIiRlJTktJ99+/YZAwYMMAICAoy6desa999/v2Gz2ZzWWbZsmXH++ecbVqvVaNq0qdMxUPPdfPPNRqNGjQyr1WrUq1fP6NOnjyNQGQbtCBV3eqiiLaGshg8fbsTGxhpWq9WoX7++MXz4cGPXrl2O5Z7UliyGYRjl69sCAAAAABThnioAAAAAqARCFQAAAABUAqEKAAAAACqBUAUAAAAAlUCoAgAAAIBKIFQBAAAAQCUQqgAAAACgEghVAAAAAFAJhCoAwDmtcePGeumll9xdBgCgBiNUAQCqzZgxYzR48GBJUq9evTRx4sRqO3ZiYqLCw8OLzV+7dq1uu+22aqsDAFD7+Li7AAAAKiM/P19Wq7XC29erV8+F1QAAzkX0VAEAqt2YMWO0YsUKvfzyy7JYLLJYLNq3b58kafPmzRowYICCg4MVHR2tm266SSkpKY5te/XqpQkTJmjixImqW7eu+vfvL0l64YUX1L59ewUFBSk+Pl533nmnsrKyJEnLly/X2LFjlZ6e7jje9OnTJRW//G///v265pprFBwcrNDQUF133XVKTk52LJ8+fbrOP/98vf/++2rcuLHCwsJ0/fXXKzMz07HOp59+qvbt2ysgIECRkZHq27evTpw4UUWfJgDA3QhVAIBq9/LLLyshIUG33nqrkpKSlJSUpPj4eKWlpal3797q1KmTfvvtNy1atEjJycm67rrrnLZ/9913ZbVatXLlSv373/+WJHl5eemVV17Rli1b9O677+qHH37Qgw8+KEm6+OKL9dJLLyk0NNRxvEmTJhWry26365prrlFqaqpWrFihJUuWaM+ePRo+fLjTert379aCBQv09ddf6+uvv9aKFSv01FNPSZKSkpI0YsQI3Xzzzdq2bZuWL1+uoUOHyjCMqvgoAQAegMv/AADVLiwsTFarVYGBgYqJiXHMf+2119SpUyc9+eSTjnnvvPOO4uPjtWPHDrVs2VKS1KJFCz3zzDNO+zz1/qzGjRvr8ccf1+23367XX39dVqtVYWFhslgsTsc73dKlS7Vp0ybt3btX8fHxkqT33ntP7dq109q1a9WlSxdJZvhKTExUSEiIJOmmm27S0qVL9cQTTygpKUkFBQUaOnSoGjVqJElq3759JT4tAICno6cKAOAxNm7cqGXLlik4ONgxtW7dWpLZO1Skc+fOxbb9/vvv1adPH9WvX18hISG66aabdOzYMWVnZ5f5+Nu2bVN8fLwjUElS27ZtFR4erm3btjnmNW7c2BGoJCk2NlZHjhyRJHXs2FF9+vRR+/btde211+o///mPjh8/XvYPAQBQ4xCqAAAeIysrS4MGDdKGDRucpp07d6pHjx6O9YKCgpy227dvn6666ip16NBBn332mdatW6fZs2dLMgeycDVfX1+n9xaLRXa7XZLk7e2tJUuWaOHChWrbtq1effVVtWrVSnv37nV5HQAAz0CoAgC4hdVqVWFhodO8Cy64QFu2bFHjxo3VvHlzp+n0IHWqdevWyW636/nnn1e3bt3UsmVLHTp06KzHO12bNm104MABHThwwDFv69atSktLU9u2bct8bhaLRd27d9eMGTO0fv16Wa1Wff7552XeHgBQsxCqAABu0bhxY61Zs0b79u1TSkqK7Ha7xo8fr9TUVI0YMUJr167V7t27tXjxYo0dO/aMgah58+ay2Wx69dVXtWfPHr3//vuOASxOPV5WVpaWLl2qlJSUEi8L7Nu3r9q3b6+RI0fq999/16+//qpRo0apZ8+euvDCC8t0XmvWrNGTTz6p3377Tfv379f8+fN19OhRtWnTpnwfEACgxiBUAQDcYtKkSfL29lbbtm1Vr1497d+/X3FxcVq5cqUKCwt1+eWXq3379po4caLCw8Pl5VX6P1kdO3bUCy+8oKefflrnnXeePvzwQ82aNctpnYsvvli33367hg8frnr16hUb6EIye5i++OILRUREqEePHurbt6+aNm2qjz/+uMznFRoaqh9//FEDBw5Uy5YtNWXKFD3//PMaMGBA2T8cAECNYjEY4xUAAAAAKoyeKgAAAACoBEIVAAAAAFQCoQoAAAAAKoFQBQAAAACVQKgCAAAAgEogVAEAAABAJRCqAAAAAKASCFUAAAAAUAmEKgAAAACoBEIVAAAAAFQCoQoAAAAAKuH/AaquIVpYGuXLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1000x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Train model\n",
    "\n",
    "iterations = np.empty(int(max_iters/eval_iters))\n",
    "iterations[:] = np.nan\n",
    "train_loss = np.empty(int(max_iters/eval_iters))\n",
    "train_loss[:] = np.nan\n",
    "val_loss = np.empty(int(max_iters/eval_iters))\n",
    "val_loss[:] = np.nan\n",
    "\n",
    "for i in range(max_iters):\n",
    "    train_one_batch(model, optimizer)\n",
    "    if i % eval_iters == 0:\n",
    "        out = estimate_loss()\n",
    "        iterations[int(i/eval_iters)] = i\n",
    "        train_loss[int(i/eval_iters)] = torch.exp(out['train'])\n",
    "        val_loss[int(i/eval_iters)] = torch.exp(out['val'])\n",
    "        print(f\"First {i} batches: training perplexity: {torch.exp(out['train']):.3f}, validation perplexity: {torch.exp(out['val']):.3f}\")\n",
    "        \n",
    "# Visualize results\n",
    "plt.figure(figsize=(10, 5))\n",
    "plt.plot(iterations, train_loss, label='Training Perplexity', color='blue')\n",
    "plt.plot(iterations, val_loss, label='Validation Perplexity', color='red')\n",
    "plt.title('Training and Validation Perplexity')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Perplexity')\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "feceb325-c3b8-4fe8-ab56-6b0fd2242963",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello, my name is wite wereing years officers,\n",
      "How words, child friend! all be tI have a cause o'er-love be any inkee\n"
     ]
    }
   ],
   "source": [
    "# Output some text\n",
    "\n",
    "model.eval()\n",
    "prompt = \"Hello, my name is\"\n",
    "max_new_tokens = 100\n",
    "ix = torch.tensor([encode(prompt)], dtype=torch.long)\n",
    "X =  ix.to('cuda')\n",
    "output = model.generate(X, max_new_tokens)\n",
    "[output] = output.tolist()\n",
    "response = decode(output)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa232d79-8d54-4927-8bd3-15bf01d549bd",
   "metadata": {},
   "source": [
    "Increase the number of decoder blocks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "f16b80b9-1b3a-4427-ad96-5b6175ff4c80",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_layer = 8\n",
    "\n",
    "model = BigramLanguageModel()\n",
    "model.to('cuda')\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "dfead583-0dc7-48f4-a77b-862be1524f65",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 0 batches: training perplexity: 39.619, validation perplexity: 40.505\n",
      "First 200 batches: training perplexity: 11.009, validation perplexity: 11.302\n",
      "First 400 batches: training perplexity: 9.303, validation perplexity: 9.530\n",
      "First 600 batches: training perplexity: 7.939, validation perplexity: 8.116\n",
      "First 800 batches: training perplexity: 7.129, validation perplexity: 7.664\n",
      "First 1000 batches: training perplexity: 6.556, validation perplexity: 7.178\n",
      "First 1200 batches: training perplexity: 6.239, validation perplexity: 6.943\n",
      "First 1400 batches: training perplexity: 5.927, validation perplexity: 6.751\n",
      "First 1600 batches: training perplexity: 5.693, validation perplexity: 6.481\n",
      "First 1800 batches: training perplexity: 5.553, validation perplexity: 6.468\n",
      "First 2000 batches: training perplexity: 5.414, validation perplexity: 6.341\n",
      "First 2200 batches: training perplexity: 5.257, validation perplexity: 6.270\n",
      "First 2400 batches: training perplexity: 5.167, validation perplexity: 6.187\n",
      "First 2600 batches: training perplexity: 5.096, validation perplexity: 6.085\n",
      "First 2800 batches: training perplexity: 5.044, validation perplexity: 5.960\n",
      "First 3000 batches: training perplexity: 4.951, validation perplexity: 5.773\n",
      "First 3200 batches: training perplexity: 4.950, validation perplexity: 5.858\n",
      "First 3400 batches: training perplexity: 4.900, validation perplexity: 5.890\n",
      "First 3600 batches: training perplexity: 4.792, validation perplexity: 5.593\n",
      "First 3800 batches: training perplexity: 4.748, validation perplexity: 5.661\n",
      "First 4000 batches: training perplexity: 4.705, validation perplexity: 5.559\n",
      "First 4200 batches: training perplexity: 4.616, validation perplexity: 5.608\n",
      "First 4400 batches: training perplexity: 4.692, validation perplexity: 5.529\n",
      "First 4600 batches: training perplexity: 4.563, validation perplexity: 5.475\n",
      "First 4800 batches: training perplexity: 4.527, validation perplexity: 5.498\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1UAAAHWCAYAAACfRKOZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB+a0lEQVR4nO3deXhM1/8H8PedycxkT4jIJhF7UFtRDS1K7FXKr1VVRJVqaauq1UURVaqLalV1F1SqXy26orGr2orYdyFkFWTfJjPn98ftTDLZl0lmJt6v57lPcu/cuffM5Ih553PvOZIQQoCIiIiIiIiqRGHpBhAREREREdkyhioiIiIiIqJqYKgiIiIiIiKqBoYqIiIiIiKiamCoIiIiIiIiqgaGKiIiIiIiompgqCIiIiIiIqoGhioiIiIiIqJqYKgiIiIiIiKqBoYqIqIKCg0NRWBgYJWeO2/ePEiSZN4GWZmrV69CkiSEh4fX+rklScK8efOM6+Hh4ZAkCVevXi33uYGBgQgNDTVre6rTV+qioj8fc7sb/n0RkXVjqCIimydJUoWWXbt2Wbqpd70XX3wRkiTh0qVLpe7z1ltvQZIknDhxohZbVnlxcXGYN28eoqKiLN0UI0OwNSxKpRIBAQF49NFHraqdtWHhwoXYtGmTpZtBRHcJhioisnlr1qwxWfr161fi9tatW1frPF9//TXOnz9fpefOnj0b2dnZ1Tp/XTBmzBgAQERERKn7/PDDD2jXrh3at29f5fOMHTsW2dnZaNy4cZWPUZ64uDiEhYWVGFaq01fMYfTo0VizZg2+++47PPnkk9ixYwfuv//+OhusSvr3xVBFRLXJztINICKqrqeeespk/cCBA4iMjCy2vaisrCw4OjpW+DwqlapK7QMAOzs72NnxV263bt3QvHlz/PDDD5gzZ06xx/fv34/o6Gi899571TqPUqmEUqms1jGqozp9xRzuvfdek/7fo0cPPPLII1ixYgW+/PLLah07MzMTTk5O1W2iWfHfFxFZGitVRHRX6N27N+655x4cOXIEPXv2hKOjI958800AwC+//IIhQ4bA19cXGo0GzZo1wzvvvAOdTmdyjKL3yRgutfrwww/x1VdfoVmzZtBoNOjatSsOHz5s8tyS7vmQJAnTpk3Dpk2bcM8990Cj0aBt27bYsmVLsfbv2rULXbp0gb29PZo1a4Yvv/yywveR7N27F4899hgCAgKg0Wjg7++Pl19+udhf9kNDQ+Hs7IzY2FgMHz4czs7O8PT0xMyZM4u9FykpKQgNDYWbmxvc3d0xfvx4pKSklNsWQK5WnTt3DkePHi32WEREBCRJwujRo5GXl4c5c+agc+fOcHNzg5OTEx588EHs3Lmz3HOUdE+VEAILFixAo0aN4OjoiIceeginT58u9tzbt29j5syZaNeuHZydneHq6opBgwbh+PHjxn127dqFrl27AgAmTJhgvNzOcD9ZSfdUZWZm4pVXXoG/vz80Gg1atWqFDz/8EEIIk/0q0y8qqk+fPgCA6Oho47aDBw9i4MCBcHNzg6OjI3r16oV9+/aZPM/Qx86cOYMnn3wS9erVwwMPPGB8jc7Ozrhy5QoGDBgAJycn+Pr6Yv78+cVeU0liY2Px9NNPw8vLy/gav/vuO+Pj2dnZCAoKQlBQkElfvX37Nnx8fNC9e3djvyz6b0GSJGRmZmLVqlXGn01oaCh27twJSZKwcePGYu0x9L39+/dX5C0lIjLBP+sQ0V3j1q1bGDRoEJ544gk89dRT8PLyAiB/AHd2dsaMGTPg7OyMHTt2YM6cOUhLS8MHH3xQ7nEjIiKQnp6OZ599FpIk4f3338eIESNw5cqVcisWf//9NzZs2IDnn38eLi4u+PTTTzFy5EjExMTAw8MDAHDs2DEMHDgQPj4+CAsLg06nw/z58+Hp6Vmh171+/XpkZWXhueeeg4eHBw4dOoRly5bhxo0bWL9+vcm+Op0OAwYMQLdu3fDhhx9i27Zt+Oijj9CsWTM899xzAORwMmzYMPz999+YMmUKWrdujY0bN2L8+PEVas+YMWMQFhaGiIgI3HvvvSbn/t///ocHH3wQAQEBSE5OxjfffIPRo0dj0qRJSE9Px7fffosBAwbg0KFD6NixY4XOZzBnzhwsWLAAgwcPxuDBg3H06FH0798feXl5JvtduXIFmzZtwmOPPYYmTZogMTERX375JXr16oUzZ87A19cXrVu3xvz58zFnzhxMnjwZDz74IACge/fuJZ5bCIFHHnkEO3fuxMSJE9GxY0ds3boVr776KmJjY/Hxxx+b7F+RflEZly9fBgDjc3fs2IFBgwahc+fOmDt3LhQKBVauXIk+ffpg7969uO+++0ye/9hjj6FFixZYuHChSWDS6XQYOHAg7r//frz//vvYsmUL5s6di/z8fMyfP7/U9iQmJuL+++83BkhPT09s3rwZEydORFpaGqZPnw4HBwesWrUKPXr0wFtvvYUlS5YAAKZOnYrU1FSEh4eXWo1cs2YNnnnmGdx3332YPHkyAKBZs2a4//774e/vj7Vr1+LRRx81ec7atWvRrFkzBAcHV/LdJSICIIiI6pipU6eKor/eevXqJQCIL774otj+WVlZxbY9++yzwtHRUeTk5Bi3jR8/XjRu3Ni4Hh0dLQAIDw8Pcfv2beP2X375RQAQv/32m3Hb3Llzi7UJgFCr1eLSpUvGbcePHxcAxLJly4zbhg4dKhwdHUVsbKxx28WLF4WdnV2xY5akpNe3aNEiIUmSuHbtmsnrAyDmz59vsm+nTp1E586djeubNm0SAMT7779v3Jafny8efPBBAUCsXLmy3DZ17dpVNGrUSOh0OuO2LVu2CADiyy+/NB4zNzfX5Hl37twRXl5e4umnnzbZDkDMnTvXuL5y5UoBQERHRwshhEhKShJqtVoMGTJE6PV6435vvvmmACDGjx9v3JaTk2PSLiHkn7VGozF5bw4fPlzq6y3aVwzv2YIFC0z2+7//+z8hSZJJH6hovyiJoU+GhYWJmzdvioSEBLFr1y7RqVMnAUD8/PPPQq/XixYtWogBAwaYvBdZWVmiSZMmol+/fsZthn47evToEl8jAPHCCy8Yt+n1ejFkyBChVqvFzZs3TV5T4Z/PxIkThY+Pj0hOTjY55hNPPCHc3NxM+uwbb7whFAqF2LNnj1i/fr0AIJYuXWryvJL+fTk5OZn8XAsfT6PRiJSUFOO2pKQkYWdnZ9JGIqLK4OV/RHTX0Gg0mDBhQrHtDg4Oxu/T09ORnJyMBx98EFlZWTh37ly5xx01ahTq1atnXDdULa5cuVLuc0NCQtCsWTPjevv27eHq6mp8rk6nw7Zt2zB8+HD4+voa92vevDkGDRpU7vEB09eXmZmJ5ORkdO/eHUIIHDt2rNj+U6ZMMVl/8MEHTV7Ln3/+CTs7O2PlCpDvYXrhhRcq1B5Avg/uxo0b2LNnj3FbREQE1Go1HnvsMeMx1Wo1AECv1+P27dvIz89Hly5dSrx0sCzbtm1DXl4eXnjhBZPLxKZPn15sX41GA4VC/u9Rp9Ph1q1bcHZ2RqtWrSp9XoM///wTSqUSL774osn2V155BUIIbN682WR7ef2iPHPnzoWnpye8vb3Ru3dvXL58GYsXL8aIESMQFRWFixcv4sknn8StW7eQnJyM5ORkZGZmom/fvtizZw/0er3J8Yr2icKmTZtm/N5QecrLy8O2bdtK3F8IgZ9//hlDhw6FEMJ4/uTkZAwYMACpqakm7/O8efPQtm1bjB8/Hs8//zx69epV7H2sjHHjxiE3Nxc//fSTcduPP/6I/Pz8cu/DJCIqDS//I6K7hp+fn/FDemGnT5/G7NmzsWPHDqSlpZk8lpqaWu5xAwICTNYNAevOnTuVfq7h+YbnJiUlITs7G82bNy+2X0nbShITE4M5c+bg119/Ldamoq/P3t6+2GWFhdsDANeuXYOPjw+cnZ1N9mvVqlWF2gMATzzxBGbMmIGIiAj07t0bOTk52LhxIwYNGmQSUFetWoWPPvoI586dg1arNW5v0qRJhc9laDMAtGjRwmS7p6enyfkAOcB98skn+PzzzxEdHW1yP1lVLr0znN/X1xcuLi4m2w0jUhraZ1BevyjP5MmT8dhjj0GhUMDd3R1t27aFRqMBAFy8eBEAyrxcMzU11eR9Ke39VigUaNq0qcm2li1bAkCpc4TdvHkTKSkp+Oqrr/DVV1+VuE9SUpLxe7Vaje+++w5du3aFvb09Vq5cWa05qYKCgtC1a1esXbsWEydOBCBf+nf//fdX+N8UEVFRDFVEdNcoXLExSElJQa9eveDq6or58+ejWbNmsLe3x9GjRzFr1qxif7EvSWn3dYgK3KxfnedWhE6nQ79+/XD79m3MmjULQUFBcHJyQmxsLEJDQ4u9vtoaMa9hw4bo168ffv75Zyxfvhy//fYb0tPTjUOuA8D333+P0NBQDB8+HK+++ioaNmwIpVKJRYsWGe8RqgkLFy7E22+/jaeffhrvvPMO6tevD4VCgenTp1eoP5hDdftFixYtEBISUuJjhtfwwQcflHpfWtHAXNK/naoynP+pp54qNdgVHU5/69atAICcnBxcvHix0qG6qHHjxuGll17CjRs3kJubiwMHDuCzzz6r1jGJ6O7GUEVEd7Vdu3bh1q1b2LBhA3r27GncXniUNEtq2LAh7O3tS5wst6wJdA1OnjyJCxcuYNWqVRg3bpxxe2RkZJXb1LhxY2zfvh0ZGRkmH74rOy/TmDFjsGXLFmzevBkRERFwdXXF0KFDjY//9NNPaNq0KTZs2GBSmZg7d26V2gzIVZrClZWbN28Wq/789NNPeOihh/Dtt9+abE9JSUGDBg2M65WpljRu3Bjbtm1Denq6SbXKcHlpTc6nVZThskJXV9dSg1dF6fV6XLlyxVidAoALFy4AQLHRDw08PT3h4uICnU5XofOfOHEC8+fPx4QJExAVFYVnnnkGJ0+ehJubW5nPK+vnY6iU/vDDD8jOzoZKpcKoUaPKbQsRUWl4TxUR3dUMFYHCFYC8vDx8/vnnlmqSCaVSiZCQEGzatAlxcXHG7ZcuXSp2H05pzwdMX58QAp988kmV2zR48GDk5+djxYoVxm06nQ7Lli2r1HGGDx8OR0dHfP7559i8eTNGjBgBe3v7Mtt+8ODBKg15HRISApVKhWXLlpkcb+nSpcX2VSqVxSpC69evR2xsrMk2w1xNFRlKfvDgwdDpdMWqIR9//DEkSarw/XHm0LlzZzRr1gwffvghMjIyij1+8+bNSh2v8GsSQuCzzz6DSqVC3759S9xfqVRi5MiR+Pnnn3Hq1Kkyz6/VahEaGgpfX1988sknCA8PR2JiIl5++eVy2+Xk5FTqz6ZBgwYYNGgQvv/+e6xduxYDBw40CcxERJXFShUR3dW6d++OevXqYfz48XjxxRchSRLWrFljtsvvzGHevHn466+/0KNHDzz33HPGD+f33HMPoqKiynxuUFAQmjVrhpkzZyI2Nhaurq74+eefK3xvTkmGDh2KHj164PXXX8fVq1fRpk0bbNiwoUL3nxXm7OyM4cOHIyIiAgBMLv0DgIcffhgbNmzAo48+iiFDhiA6OhpffPEF2rRpU2IYKIthvq1Fixbh4YcfxuDBg3Hs2DFs3ry52Ifphx9+2FgZ6d69O06ePIm1a9cWu3eoWbNmcHd3xxdffAEXFxc4OTmhW7duJV6aNnToUDz00EN46623cPXqVXTo0AF//fUXfvnlF0yfPt1kUIqaplAo8M0332DQoEFo27YtJkyYAD8/P8TGxmLnzp1wdXXFb7/9VqFj2dvbY8uWLRg/fjy6deuGzZs3448//sCbb75Z5pD/7733Hnbu3Ilu3bph0qRJaNOmDW7fvo2jR49i27ZtuH37NgBgwYIFiIqKwvbt2+Hi4oL27dtjzpw5mD17Nv7v//4PgwcPLvUcnTt3xrZt27BkyRL4+vqiSZMm6Natm/HxcePG4f/+7/8AAO+8806FXi8RUWlYqSKiu5qHhwd+//13+Pj4YPbs2fjwww/Rr18/vP/++5ZumlHnzp2xefNm1KtXD2+//Ta+/fZbzJ8/H3379jWp7JREpVLht99+Q8eOHbFo0SKEhYWhRYsWWL16dZXbo1Ao8Ouvv2LMmDH4/vvv8dZbb8HPzw+rVq2q9LEMQcrHx8c4Qa1BaGgoFi5ciOPHj+PFF1/E1q1b8f3336NLly5VaveCBQsQFhaGY8eO4dVXX8Xly5fx119/GStOBm+++SZeeeUVbN26FS+99BKOHj2KP/74A/7+/ib7qVQqrFq1CkqlElOmTMHo0aOxe/fuEs9teM+mT5+O33//HdOnT8eZM2fwwQcfGOdfqk29e/fG/v370aVLF3z22Wd44YUXEB4eDm9v7wpVgQyUSiW2bNmChIQEvPrqqzh8+DDmzp1bbkjx8vLCoUOHMGHCBGzYsAHTpk3DJ598gtu3b2Px4sUAgKNHj2LhwoWYNm0aHnroIeNzX3/9dXTt2hWTJk0qs0q4ZMkSdO7cGbNnz8bo0aNNKquAHHTr1asHNzc3PPLIIxV+zUREJZGENf05loiIKmz48OE4ffq0cTQ3otoUGhqKn376qdJVQ2uRn58PX19fDB06tNj9c0RElcVKFRGRDcjOzjZZv3jxIv7880/07t3bMg0isnGbNm3CzZs3TQZwISKqKt5TRURkA5o2bYrQ0FA0bdoU165dw4oVK6BWq/Haa69ZumlENuXgwYM4ceIE3nnnHXTq1Am9evWydJOIqA5gqCIisgEDBw7EDz/8gISEBGg0GgQHB2PhwoXFJrMlorKtWLEC33//PTp27Ijw8HBLN4eI6gjeU0VERERERFQNvKeKiIiIiIioGhiqiIiIiIiIqqHO31Ol1+sRFxcHFxcXSJJk6eYQEREREZGFCCGQnp4OX19fKBTmqy/V+VAVFxdXbMJGIiIiIiK6e12/fh2NGjUy2/HqfKhycXEBIL9xrq6uFm2LVqvFX3/9hf79+0OlUlm0LWTb2JfIXNiXyBzYj8hc2JfIXErrS2lpafD39zdmBHOp86HKcMmfq6urVYQqR0dHuLq68hcFVQv7EpkL+xKZA/sRmQv7EplLeX3J3LcFcaAKIiIiIiKiamCoIiIiIiIiqgaGKiIiIiIiomqo8/dUEREREd2tdDodtFqtpZtRYVqtFnZ2dsjJyYFOp7N0c8gGKZVK2NnVfsRhqCIiIiKqgzIyMnDjxg0IISzdlAoTQsDb2xvXr1/n/KJUZY6OjvD09KzVczJUEREREdUxOp0ON27cMH64tJWAotfrkZGRAWdnZ7NOzEp3ByEE8vLycPPmTcTExNTquRmqiIiIiOoYrVYLIQQ8PT3h4OBg6eZUmF6vR15eHuzt7RmqqEocHBygUqlw9epVKJXKWjsveysRERFRHWUrFSoiczIE8trs/wxVRERERERE1cBQRUREREREVA0MVURERERUZwUGBmLp0qUV3n/Xrl2QJAkpKSk11iZLCw8Ph7u7u1mPKUkSNm3aZNZj2hKGKiIiIiKyOEmSoFQqUa9ePSiVSkiSZLLMmzevSsc9fPgwJk+eXOH9u3fvjvj4eLi5uVXpfBVlCG+GxcvLCyNHjsSVK1dq9Lw1JT4+HoMGDQIAXL16FZIkISoqyrKNqkUc/Y+IiIiILC4+Ph56vR7p6enYvHkz5s6di/Pnzxsfd3Z2Nn4vhIBOp6vQJK+Vna9IrVbD29u7Us+pjvPnz8PFxQUXL17E5MmTMXToUJw4caJKI9dptVqoVKoaaGX5avM9s0asVNUWIaDs1w8hzz4LxMdbujVERER0FxECyMy0zFLRuYe9vb3h7e0NLy8vuLq6QpIk47Zz587BxcUFmzdvRufOnaHRaPD333/j8uXLGDZsGLy8vODs7IyuXbti27ZtJsctevmfJEn45ptv8Oijj8LR0REtWrTAr7/+any86OV/hkvltm7ditatW8PZ2RkDBw5EfKHPc/n5+XjxxRfh7u4ODw8PzJo1C+PHj8fw4cPLfd0NGzaEj48PevbsiTlz5uDMmTO4dOkSAOCXX37BvffeC3t7ezRt2hRhYWHIz883eS0rVqzAI488AicnJ7z77rvG9v/xxx9o37497O3tcf/99+PUqVNltqOsc82fPx++vr64deuWcf8hQ4bgoYcegl6vN7bFcPlfkyZNAACdOnWCJEno3bs39uzZA5VKhYSEBJPzTp8+HQ8++GC575O1Y6iqLZIE6dIlOCUmQoqLs3RriIiI6C6SlQU4O1tmycoy3+t4/fXX8d577+Hs2bNo3749MjIyMHjwYGzfvh3Hjh3DwIEDMXTo0HInfg0LC8Pjjz+OEydOYPDgwRgzZgxu375dxvuXhQ8//BBr1qzBnj17EBMTg5kzZxofX7x4MdauXYuVK1di3759SEtLq9L9RYY5xfLy8rB3716MGzcOL730Es6cOYMvv/wS4eHhePfdd02eM2/ePDz66KM4efIknn76aeP2V199FR999BEOHz4MT09PDB06FFqttsTzlneut956C4GBgXjmmWcAAMuXL8c///yDVatWlTif2KFDhwAA27ZtQ3x8PDZs2ICePXuiadOmWLNmjXE/rVaLtWvXmrTbZgkrsWjRIgFAvPTSS8Zt2dnZ4vnnnxf169cXTk5OYsSIESIhIaFSx01NTRUARGpqqplbXHm6rl2FAIT2p58s3RSycXl5eWLTpk0iLy/P0k0hG8e+RObAfmR9srOzxZkzZ0R2drYQQoiMDCHkmlHtLxkZFW+3TqcTd+7cEd9++61wc3Mzbt+5c6cAIDZt2lTuMdq2bSuWLVtmXG/cuLH4+OOPjesAxOzZs43rGRkZAoDYvHmzybnu3LkjhBBi5cqVAoC4dOmS8TnLly8XXl5exnUvLy/xwQcfGNfz8/NFQECAGDZsWKntLHqeuLg40b17d+Hn5ydyc3NF3759xcKFC02es2bNGuHj42PyWqZPn17icdetW2fcduvWLeHg4CB+/PFH42sq/P5W5FyXL18WLi4uYtasWcLBwUGsXbvWZH8AYuPGjUIIIaKjowUAcezYMZN9Fi9eLFq3bm1c//nnn4Wzs7PIqEwnqYDs7Gxx+vRp8fvvvxf7vVRT2cAq7qk6fPgwvvzyS7Rv395k+8svv4w//vgD69evh5ubG6ZNm4YRI0Zg3759FmppNfn4AAAkXv5HREREtcjREcjIsNy5zaVLly4m6xkZGZg3bx7++OMPxMfHIz8/H9nZ2eVWqgp/5nRycoKrqyuSkpJK3d/R0RHNmjUzrvv4+Bj3T01NRWJiIu677z7j40qlEp07dzZeGleWRo0aQQiBrKwsdOjQAT///DPUajWOHz+Offv2mVSmdDodcnJykJWVBcf/3tii74lBcHCw8fv69eujVatWOHv2bIn7VuRcTZs2xYcffohnn30Wo0aNwpNPPlnuaysqNDQUs2fPxoEDB3D//fcjPDwcjz/+OJycnCp9LGtj8VCVkZGBMWPG4Ouvv8aCBQuM21NTU/Htt98iIiICffr0AQCsXLkSrVu3Nv4gbI3w85O/iY21bEOIiIjoriJJQB343Frsw/fMmTMRGRmJDz/8EM2bN4eDgwP+7//+D3l5eWUep+hgDpIklRmAStpfVPRmsXLs3bsXrq6uaNiwIVxcXIzbMzIyEBYWhhEjRhR7jr29vfF7cwSSip5rz549UCqVuHr1KvLz8ys0UEhhDRs2xNChQ7Fy5Uo0adIEmzdvxq5du6rbfKtg8VA1depUDBkyBCEhISah6siRI9BqtQgJCTFuCwoKQkBAAPbv319qqMrNzUVubq5xPS0tDYB8zWZp15HWFuHlBSUAERtr8baQbTP0H/Yjqi72JTIH9iPro9VqIYSAXq+vULXEWhQNKoa2F/5a+PXs27cP48ePx7BhwwDI4eDq1avG1174uIXXS3pfDNuKnqtoG4q2x8XFBV5eXjh06BAeeOABAHKV5+jRo+jQoUOp779he+PGjY1zRhXe995778W5c+fQtGnTMp9f9LUYvv/nn3/QqFEjAMCdO3dw4cIFtGrVqsTXVJFz/fjjj9iwYQN27NiBJ554AvPnzy82zL3h2IawpdVqi73+p59+GmPGjIGfnx+aNWuG4OBgs/dRvV5v7EtFfy/V1O8pi4aqdevW4ejRozh8+HCxxxISEqBWq4tNTObl5VVs1JDCFi1ahLCwsGLb//rrL2OZ1FL8b93CvQBunzyJ/X/+adG2UN0QGRlp6SZQHcG+RObAfmQ97Ozs4O3tjYyMjHKrNtYoJycHQgjjH8ez/hvtIj093WRghMDAQPz000946KGHAAALFy6EXq9HXl6e8bl6vR45OTnGdQDIzs42WRdCGPcpeq6ibTE8Hyj44/0zzzyDRYsWwdfXFy1atMBXX32F27dvQ6fTmTyvsNJek8GMGTPwxBNPwMvLC4888ggUCgVOnTqFs2fPYvbs2aW+FsNxw8LC4ODgAE9PTyxYsAD169dHnz59kJaWVuw1lXeu2NhYPP/885g3bx7at2+PZcuW4YknnsCDDz6Irl27FmuLvb09HBwc8Msvv8DNzQ0ajcY471dwcDCcnZ3x7rvv4o033ij1/amOvLw85OTkACj+eynLnCOnFGKxUHX9+nW89NJLiIyMNCkrVtcbb7yBGTNmGNfT0tLg7++P/v37w9XV1WznqQqdQgEsW4YGeXkYPHiwRdtCtk2r1SIyMhL9+vWz2HwUVDewL5E5sB9Zn5ycHFy/fh3Ozs5m/ZxV04QQSE9Ph729PSRJMn52M/xh3MXFxeTz3CeffIJnnnkGAwYMQIMGDfDaa68hOzsbarXauJ9CoYC9vb3J8xwcHEzWJUky7lP0XEXbYng+AOO2OXPmICUlBc899xyUSiUmTZqEAQMGQKlUlvr5s7TXZPDoo4/i119/xYIFC/DJJ59ApVIhKCgITz/9dJmvxXDcxYsX480338TFixfRsWNH/Pbbb2jQoAEAFHtNZZ3LxcUFL730Eu677z688sorkCQJjz76KKZMmYLnnnsOR48eNc4hVrgtS5cuxYIFC7Bo0SI8+OCD2LFjh7GNEyZMwKJFizBp0qQa+Xyek5Nj7PdFfy/VRIgDAEmY64LQStq0aRMeffRRk4nNdDodJEmCQqHA1q1bERISgjt37phUqxo3bozp06fj5ZdfrtB50tLS4ObmhtTUVIuGKiGAK79FodmwThD16kEqY9hOovJotVr8+eefGDx4MD/AULWwL5E5sB9Zn5ycHERHR6NJkyY2Far0ej3S0tLg6upaYvXGVuj1erRu3RqPP/443nnnnVo9965du/DQQw8V+wxtTSZOnIibN2+azA9mTjk5Obhy5Qqio6PRv3//YqGqJrKBxSpVffv2xcmTJ022TZgwAUFBQZg1axb8/f2hUqmwfft2jBw5EoA843RMTIzJaCa25MEnGiMOgHTnDpCdDfz3Vw4iIiIisl3Xrl3DX3/9hV69eiE3NxefffYZoqOjqzRCXl2WmpqKkydPIiIiosYClaVYLFS5uLjgnnvuMdnm5OQEDw8P4/aJEydixowZqF+/PlxdXfHCCy8gODjYJkf+kyTAydcNWZcd4IhsID4eKOVmQCIiIiKyHQqFAuHh4Zg5cyaEELjnnnuwbds2tG7d2tJNsyrDhg3DoUOHMGXKFPTr18/SzTEri4/+V5aPP/4YCoUCI0eORG5uLgYMGIDPP//c0s2qMr9GQOxlP7TAJXlYdYYqIiIiIpvn7+9vNfOo9u7d22zDvZtbXRk+vSRWFaqKvtH29vZYvnw5li9fbpkGmZmvLxAHXzlUxcVZujlERERERGQGtnsHoA3y9RWIBScAJiIiIiKqSxiqapGfn1ypAsBKFRERERFRHcFQVYtYqSIiIiIiqnsYqmoRK1VERERERHUPQ1Ut8vUVxlAlWKkiIiIiIqoTGKpqkbd3oUpVbBxgpcNdEhEREdmq3r17Y/r06cb1wMBALF26tMznSJKETZs2Vfvc5jqOtZo3bx46duxotuNdvXoVkiQhKirKbMe0FIaqWqRSAdnu9QEAUk42kJJi2QYRERERWYmhQ4di0KBBJT62d+9eSJKEEydOVPq4hw8fxuTJk6vbPBOlhYv4+PhSX4O5hIeHQ5IkSJIEhUKBRo0aYcKECUhKSqrR89YEf39/xMfH45577gEgT68kSRJSbPAzMkNVLXP0kHALcrDifVVEREREsokTJ2Lbtm2ILeEWiZUrV6JLly5o3759pY/r6ekJR0dHczSxXN7e3tBoNDV+HldXV8THx+PGjRv4+uuvsXnzZowdO7bKx9NqtWZsXcUplUp4e3vDzs6qps6tEoaqWubhkc3BKoiIiKh2CQFkZlpmqeDtDg8//DA8PT3xww8/mGzPyMjA+vXrMXHiRNy6dQujR4+Gn58fHB0d0a5du2L7F1X08r+LFy+iZ8+esLe3R5s2bRAZGVnsObNmzULLli3h6OiIpk2b4u233zYGj/DwcISFheH48ePGilF4eDiA4pf/nTx5En369IGDgwM8PDwwefJkZGRkGB8PDQ3F8OHD8eGHH8LHxwceHh6YOnVquSFHkiR4e3vD19cXgwYNwosvvoht27YhOzsbAPDNN9+gdevWsLe3R1BQED7//HPjcw2X3P3444/o1asX7O3tsXbtWoSHh8Pd3R2bNm1CixYtYG9vjwEDBuD69etltqWscz399NNo3749cnNzAQB5eXno1KkTxo0bZ9KWqKgoXL16FQ899BAAoF69epAkCaGhoVi9ejU8PDyMxzAYPnx4tYKkudl+LLQx9evnIBZ+aIdTHFadiIiIakdWFuDsbJlzZ2QATk7l7mZnZ4exY8ciIiICYWFhxu3r16+HTqfD6NGjkZGRgc6dO2PWrFlwdXXFH3/8gbFjx6JZs2a47777yj2HXq/HiBEj4OXlhYMHDyI1NdXk/isDFxcXhIeHw9fXFydPnsSkSZPg4uKC1157DaNGjcKpU6ewZcsWbNu2DQDg5uZW7BiZmZkYMGAAgoODcfjwYSQlJeGZZ57BtGnTjCEMAHbu3AkfHx/s3LkTly5dwqhRo9CxY0dMmjSp3Ndj4ODgAL1ej/z8fKxduxZz5szBZ599hk6dOuHYsWOYNGkSnJycMH78eONzXn/9dXz00Ufo1KkT7O3tsXXrVmRlZeHdd9/F6tWroVar8fzzz+OJJ57Avn37Sjxveef69NNP0aFDB7z++uv4+OOP8dZbbyElJQWfffZZsWP5+/vj559/xsiRI3H+/Hm4urrCwcEBarUaL774In799Vc89thjAICkpCT88ccf+Ouvvyr8HtU0hqpa5uGRw0oVERERUQkmTJiADz/8ELt370afPn0AyJf+jRw5Em5ubnBzc8PMmTON+7/wwgvYunUr/ve//1UoVG3btg3nzp3D1q1b4esrfx5buHBhsfugZs+ebfw+MDAQM2fOxLp16/Daa6/BwcEBzs7OsLOzg7e3d6nnioiIQE5ODlavXg2n/0LlZ599hqFDh2Lx4sXw8vICIFdlPvvsMyiVSgQFBWHIkCHYvn17hUPVxYsX8cUXX6BLly5wcXHB3Llz8dFHH2HEiBEAgCZNmuDMmTP48ssvTULV9OnTjfsYaLVafPbZZ+jWrRsAYNWqVWjdujUOHTpU4vtb3rmcnZ3x/fffo1evXnBxccHSpUuxc+dOuLq6FjuWUqlE/fryLTINGzaEu7u78bEnn3wSK1euNIaq77//HgEBAejdu3eF3qPawFBVy+rXz+YEwERERFS7HB3lipGlzl1BQUFBuO+++7By5Ur06dMHly5dwt69ezF//nwAgE6nw8KFC/G///0PsbGxyMvLQ25uboXvmTp79iz8/f2NgQoAgoODi+33448/4tNPP8Xly5eRkZGB/Pz8EoNAeefq0KGDMVABQI8ePaDX63H+/HljqGrbti2USqVxHx8fH5w8ebLMY6empsLZ2Rl6vR45OTl44IEH8M033yAzMxOXL1/GxIkTTUJZfn5+sWpaly5dih3Xzs4OXbt2Na4HBQXB3d0dZ8+eLRaqKnqu4OBgzJw5E++88w5mzZqFBx54oMzXVpJJkyaha9euiI2NhZ+fH8LDwxEaGgpJkip9rJrCUFXLWKkiIiKiWidJFboEzxqMHTsWs2bNwueff46VK1eiWbNm6NWrFwDggw8+wCeffIKlS5eiXbt2cHJywvTp05GXl2e28+/fvx9jxoxBWFgYBgwYADc3N6xbtw4fffSR2c5RmEqlMlmXJAl6vb7M57i4uODo0aNQKBTw8fGBg4MDACAxMREA8PXXXxurTQaFgxsAk7BXFYZ7w8o7l16vx759+6BUKnHp0qUqnatTp07o0KEDVq9ejf79++P06dP4448/qt74GsBQVcs8PHJw1DhXFStVRERERIUNHz4cb7zxBiIiIrB69Wo899xzxorEvn37MGzYMDz11FMA5A/sFy5cQJs2bSp07NatW+P69euIj4+Hj48PAODAgQMm+/zzzz9o3Lgx3nrrLeO2a9eumeyjVquh0+nKPVd4eDgyMzONAWbfvn1QKBRo1apVhdpbGoVCgebNmxfb7uXlBV9fX1y5cgVjxoyp9HHz8/Px77//GqtS58+fR0pKClq3bl3lc33wwQc4d+4cdu/ejQEDBmDlypWYMGFCifuq1WoAKPG9feaZZ7B06VLExsYiJCQE/v7+lX59NYmj/9Wywpf/6WNZqSIiIiIqzNnZGY8//jjeeOMNxMfHIzQ01PhYixYtEBkZiX/++Qdnz57Fs88+a6zOVERISAhatmyJ8ePH4/jx49i7d69JeDKcIyYmBuvWrcPly5fx6aefYuPGjSb7BAYGIjo6GlFRUUhOTi42Mh0AjBkzBvb29hg/fjxOnTqFnTt34oUXXsDYsWONl/7VhLCwMCxatAiffvopLly4gJMnT2LlypVYsmRJuc9VqVR44YUXcPDgQRw5cgShoaG4//77S71frbxzHTt2DHPmzME333yDHj16YMmSJXjppZdw5cqVEo/XuHFjSJKE33//HTdv3jQZKfHJJ580DiH/9NNPV+GdqVkMVbXM0TEfKQ7yX0akxASgnL9yEBEREd1tnn76ady5cwcDBgwwuf9p9uzZuPfeezFgwAD07t0b3t7eGD58eIWPq1AosHHjRmRnZ+O+++7DM888g3fffddkn0ceeQQvv/wypk2bho4dO+Kff/7B22+/bbLPyJEjMXDgQDz00EMlDgMPAI6Ojti6dStu376Nrl274v/+7//Qt2/fEke+M6dnnnkG33zzDVauXIl27dqhV69eCA8PR5MmTcp9rqOjI2bNmoUnn3wSPXr0gLOzM3788ccqnSsnJwdPPfUUQkNDMXToUADA5MmT8dBDD2Hs2LElVqP8/PwQFhaG119/HV5eXpg2bZrxMTc3N4wcORLOzs6V+pnXFkmICk4eYKPS0tLg5uaG1NTUSt9gaG5arRZ//vknZs0cgtOXNFBCL99X9V/5maiiDH1p8ODBxa7FJqoM9iUyB/Yj65OTk4Po6Gg0adIE9vb2lm5Ohen1eqSlpcHV1RUKBf/2X5vCw8Mxffp0pKSkWLopperbty/atm2LTz/9tMz9cnJycOXKFURHR6N///4mv5dqKhuwt1qATyMFEvDfEJy8r4qIiIiIqFR37tzBxo0bsWvXLkydOtXSzSkRB6qwAF9fIBZ+8EMcRwAkIiIiIipDp06dcOfOHSxevLjag3zUFIYqC/D1FRxWnYiIiIisRmhoqMmgINbk6tWrlm5CuXj5nwX4+YETABMRERER1REMVRbAShURERHVhjo+HhlRiSzR7xmqLICVKiIiIqpJSqUSAJCXl2fhlhDVvqysLAAlTyJcU3hPlQUUrlSJuDhIFm4PERER1S12dnZwdHTEzZs3oVKpbGZ4cr1ej7y8POTk5NhMm8l6CCGQlZWFpKQkuLq61mrFiqHKAry9gQTJFxCAuBHLUEVERERmJUkSfHx8EB0djWvXrlm6ORUmhEB2djYcHBwgSfyERFXj7u4ODw+PWj0nQ5UF2NkB2oZ+QCKguHMbyMkBbGhiPiIiIrJ+arUaLVq0sKlLALVaLfbs2YOePXtyImmqEpVKBaVSCa1WW6vnZaiyEOdG7shOtIcDcuTBKpo2tXSTiIiIqI5RKBSwt6E/3CqVSuTn58Pe3p6himwKL1a1EL9GUsFgFRwBkIiIiIjIZjFUWYifHwqGVecIgERERERENouhykJMhlVnpYqIiIiIyGYxVFmISaWKoYqIiIiIyGYxVFkIJwAmIiIiIqobGKoshJUqIiIiIqK6gaHKQgpXqvQ3WKkiIiIiIrJVDFUW4uoKpDoWqlQJYdkGERERERFRlTBUWZDkJ4cqRXYWkJpq4dYQEREREVFVMFRZUAN/B9xGPXmF91UREREREdkkhioL4gTARERERES2z6KhasWKFWjfvj1cXV3h6uqK4OBgbN682fh47969IUmSyTJlyhQLtti8OAEwEREREZHts7PkyRs1aoT33nsPLVq0gBACq1atwrBhw3Ds2DG0bdsWADBp0iTMnz/f+BxHR0dLNdfsWKkiIiIiIrJ9Fg1VQ4cONVl/9913sWLFChw4cMAYqhwdHeHt7W2J5tU4Pz/gNCtVREREREQ2zaKhqjCdTof169cjMzMTwcHBxu1r167F999/D29vbwwdOhRvv/12mdWq3Nxc5ObmGtfT0tIAAFqtFlqttuZeQAUYzm/46uUlIfK/SpX+xg3oLNw+sh1F+xJRVbEvkTmwH5G5sC+RuZTWl2qqb0lCWHaCpJMnTyI4OBg5OTlwdnZGREQEBg8eDAD46quv0LhxY/j6+uLEiROYNWsW7rvvPmzYsKHU482bNw9hYWHFtkdERFjdpYO3btnjl4k5+AXDcbtFC+z94ANLN4mIiIiIqM7KysrCk08+idTUVLi6uprtuBYPVXl5eYiJiUFqaip++uknfPPNN9i9ezfatGlTbN8dO3agb9++uHTpEpo1a1bi8UqqVPn7+yM5Odmsb1xVaLVaREZGol+/flCpVMjPB3o7HcNB0Q35Po0grl2xaPvIdhTtS0RVxb5E5sB+RObCvkTmUlpfSktLQ4MGDcweqix++Z9arUbz5s0BAJ07d8bhw4fxySef4Msvvyy2b7du3QCgzFCl0Wig0WiKbVepVFbzj9PQFpUKyG/oByQCyqR4SAoFoFRaunlkQ6ypX5NtY18ic2A/InNhXyJzKdqXaqpfWd08VXq93qTSVFhUVBQAwMfHpxZbVLPU/l7QQQFJpwNu3rR0c4iIiIiIqJIsWql64403MGjQIAQEBCA9PR0RERHYtWsXtm7disuXLxvvr/Lw8MCJEyfw8ssvo2fPnmjfvr0lm21W3o3skPivF3wRLw+rXkdHOiQiIiIiqqssGqqSkpIwbtw4xMfHw83NDe3bt8fWrVvRr18/XL9+Hdu2bcPSpUuRmZkJf39/jBw5ErNnz7Zkk83OMAGwL+LlYdU7d7Z0k4iIiIiIqBIsGqq+/fbbUh/z9/fH7t27a7E1lsEJgImIiIiIbJvV3VN1tzFUqgBwAmAiIiIiIhvEUGVhJpUqhioiIiIiIpvDUGVhJpUqXv5HRERERGRzGKosrHClSneDlSoiIiIiIlvDUGVhLi5AiqNcqRI3WKkiIiIiIrI1DFVWQPKTK1V2KbeAUiY+JiIiIiIi68RQZQWc/eshBxp5hYNVEBERERHZFIYqK+DXSOKw6kRERERENoqhygpwAmAiIiIiItvFUGUFOAEwEREREZHtYqiyAqxUERERERHZLoYqK8BKFRERERGR7WKosgKFK1X6WIYqIiIiIiJbwlBlBby8gASFXKnSx/DyPyIiIiIiW8JQZQWUSkDrKVeqpIQ4QAgLt4iIiIiIiCqKocpKKBrJoUqZnQmkpVm4NUREREREVFEMVVaiQYAj7sBdXuFgFURERERENoOhykpwWHUiIiIiItvEUGUlOKw6EREREZFtYqiyEqxUERERERHZJoYqK8FKFRERERGRbWKoshKsVBERERER2SaGKitRuFKVf52VKiIiIiIiW8FQZSWcnYFUJ7lSJW4wVBERERER2QqGKisifORKlfJmPKDXW7g1RERERERUEQxVVkQT4AU9JCh0+cDNm5ZuDhERERERVQBDlRXx8bdDIrzkFQ5WQURERERkExiqrAiHVSciIiIisj0MVVaEw6oTEREREdkehiorwkoVEREREZHtYaiyIqxUERERERHZHoYqK1K4UqWPZaWKiIiIiMgWMFRZkYYNgUSFXKnKv8ZKFRERERGRLWCosiJKJZDnKVeqpHhWqoiIiIiIbAFDlZWR/ORKlSolGcjNtXBriIiIiIioPAxVVsalcX3kQCOvxMdbtjFERERERFQuhior49dIKhgBkMOqExERERFZPYYqK8Nh1YmIiIiIbItFQ9WKFSvQvn17uLq6wtXVFcHBwdi8ebPx8ZycHEydOhUeHh5wdnbGyJEjkZiYaMEW1zxOAExEREREZFssGqoaNWqE9957D0eOHMG///6LPn36YNiwYTh9+jQA4OWXX8Zvv/2G9evXY/fu3YiLi8OIESMs2eQax0oVEREREZFtsbPkyYcOHWqy/u6772LFihU4cOAAGjVqhG+//RYRERHo06cPAGDlypVo3bo1Dhw4gPvvv7/EY+bm5iK30Kh5aWlpAACtVgutVltDr6RiDOcvqx0NGxZUqnQ3bkBv4TaTdapIXyKqCPYlMgf2IzIX9iUyl9L6Uk31LYuGqsJ0Oh3Wr1+PzMxMBAcH48iRI9BqtQgJCTHuExQUhICAAOzfv7/UULVo0SKEhYUV2/7XX3/B0dGxxtpfGZGRkaU+lpOjNFaqkqNO4cCff9ZWs8gGldWXiCqDfYnMgf2IzIV9icylaF/KysqqkfNYPFSdPHkSwcHByMnJgbOzMzZu3Ig2bdogKioKarUa7u7uJvt7eXkhISGh1OO98cYbmDFjhnE9LS0N/v7+6N+/P1xdXWvqZVSIVqtFZGQk+vXrB5VKVep+q532AZmAW1YOBg8eXIstJFtR0b5EVB72JTIH9iMyF/YlMpfS+pLhKjZzs3ioatWqFaKiopCamoqffvoJ48ePx+7du6t8PI1GA41GU2y7SqWymn+c5bVF7+0HXAbsEuNgZ2cHSFItto5siTX1a7Jt7EtkDuxHZC7sS2QuRftSTfUri4cqtVqN5s2bAwA6d+6Mw4cP45NPPsGoUaOQl5eHlJQUk2pVYmIivL29LdTa2qFq7CuHqpxMID0dsHCFjYiIiIiISmd181Tp9Xrk5uaic+fOUKlU2L59u/Gx8+fPIyYmBsHBwRZsYc3zCHBCCtzkFQ6rTkRERERk1SxaqXrjjTcwaNAgBAQEID09HREREdi1axe2bt0KNzc3TJw4ETNmzED9+vXh6uqKF154AcHBwaUOUlFXGIZVd0eqPKx6UJClm0RERERERKWwaKhKSkrCuHHjEB8fDzc3N7Rv3x5bt25Fv379AAAff/wxFAoFRo4cidzcXAwYMACff/65JZtcKwwTALfBWVaqiIiIiIisnEVD1bffflvm4/b29li+fDmWL19eSy2yDpwAmIiIiIjIdljdPVVUUKkCwEoVEREREZGVY6iyQoUrVfrrrFQREREREVkzhior1LAhkKCQK1XaGFaqiIiIiIisGUOVFVIogLwGvKeKiIiIiMgWMFRZKz+5UqVKjgf0egs3hoiIiIiISsNQZaUcAr2ghwSFLh+4edPSzSEiIiIiolIwVFkpb38VEuElr3AEQCIiIiIiq8VQZaVM5qpiqCIiIiIisloMVVaKEwATEREREdkGhiorxQmAiYiIiIhsA0OVlSpcqRI3WKkiIiIiIrJWDFVWqnClKp8TABMRERERWS2GKivl6AikOcmVqvwYVqqIiIiIiKwVQ5UV03nLlSplAitVRERERETWiqHKiin95UqVOvUmkJtr4dYQEREREVFJGKqsmEugB3KhllcSEizbGCIiIiIiKhFDlRXzayRxrioiIiIiIivHUGXFOFcVEREREZH1Y6iyYoXnqmKoIiIiIiKyTgxVVswkVPHyPyIiIiIiq8RQZcUKX/6nv8FKFRERERGRNWKosmKenkCSUq5U5UWzUkVEREREZI0YqqyYQgHkeMiVKhHLShURERERkTViqLJ2vnKlyi6JlSoiIiIiImvEUGXl1IFyqFLlZADp6RZuDRERERERFcVQZeU8GjsjFa7yCkcAJCIiIiKyOgxVVo4TABMRERERWTeGKivHuaqIiIiIiKwbQ5WVMwlVrFQREREREVkdhiorV/jyPw6rTkRERERkfRiqrFzhSpX2Ki//IyIiIiKyNgxVVs7BAUh1kitV+TGsVBERERERWRuGKhug85IrVVI8K1VERERERNaGocoGSI3kSpX6Vjyg11u4NUREREREVFiVQtXKlSuRlZVl7rZQKRybekMPCUqdFkhOtnRziIiIiIiokCqFqtdffx3e3t6YOHEi/vnnH3O3iYrw9lchCQ3lFQ6rTkRERERkVaoUqmJjY7Fq1SokJyejd+/eCAoKwuLFi5GQkGDu9hE4ATARERERkTWrUqiys7PDo48+il9++QXXr1/HpEmTsHbtWgQEBOCRRx7BL7/8An0F7v1ZtGgRunbtChcXFzRs2BDDhw/H+fPnTfbp3bs3JEkyWaZMmVKVZtuswnNVsVJFRERERGRdqj1QhZeXFx544AEEBwdDoVDg5MmTGD9+PJo1a4Zdu3aV+dzdu3dj6tSpOHDgACIjI6HVatG/f39kZmaa7Ddp0iTEx8cbl/fff7+6zbYprFQREREREVkvu6o+MTExEWvWrMHKlStx5coVDB8+HL///jtCQkKQmZmJ+fPnY/z48bh27Vqpx9iyZYvJenh4OBo2bIgjR46gZ8+exu2Ojo7w9vaualNtXuFQpbseB6WF20NERERERAWqFKqGDh2KrVu3omXLlpg0aRLGjRuH+vXrGx93cnLCK6+8gg8++KBSx01NTQUAk2MBwNq1a/H999/D29sbQ4cOxdtvvw1HR8cSj5Gbm4vc3FzjelpaGgBAq9VCq9VWqj3mZjh/Zdvh5gYkKH0BHZBzJRZqC78Osryq9iWiotiXyBzYj8hc2JfIXErrSzXVtyQhhKjskyZOnIhnnnkGwcHBpe4jhEBMTAwaN25coWPq9Xo88sgjSElJwd9//23c/tVXX6Fx48bw9fXFiRMnMGvWLNx3333YsGFDiceZN28ewsLCim2PiIgoNYjZgnXjdPghbSQSfZrjwIoPLd0cIiIiIiKbk5WVhSeffBKpqalwdXU123GrFKpWr16NUaNGQaPRmGzPy8vDunXrMG7cuEo35LnnnsPmzZvx999/o1GjRqXut2PHDvTt2xeXLl1Cs2bNij1eUqXK398fycnJZn3jqkKr1SIyMhL9+vWDSqWq1HMndjmF1SfuRY5rQyiTb9RQC8lWVKcvERXGvkTmwH5E5sK+ROZSWl9KS0tDgwYNzB6qqnT534QJEzBw4EA0bNjQZHt6ejomTJhQ6VA1bdo0/P7779izZ0+ZgQoAunXrBgClhiqNRlMs7AGASqWymn+cVWmLKrARcAKwT0sChADU6hpqHdkSa+rXZNvYl8gc2I/IXNiXyFyK9qWa6ldVGv1PCAFJkoptv3HjBtzc3Cp1nGnTpmHjxo3YsWMHmjRpUu5zoqKiAAA+Pj4VPk9d4NqkAfLwXyfgfGBERERERFajUpWqTp06GeeK6tu3L+zsCp6u0+kQHR2NgQMHVvh4U6dORUREBH755Re4uLgYJw92c3ODg4MDLl++jIiICAwePBgeHh44ceIEXn75ZfTs2RPt27evTNNtnl8jCXHwRSCuycOqBwRYuklERERERIRKhqrhw4cDkKtFAwYMgLOzs/ExtVqNwMBAjBw5ssLHW7FiBQB5gt/CVq5cidDQUKjVamzbtg1Lly5FZmYm/P39MXLkSMyePbsyza4TDBMAB+IaJwAmIiIiIrIilQpVc+fOBQAEBgZi1KhRsLe3r9bJyxsjw9/fH7t3767WOeoKTgBMRERERGSdqnRP1fjx46sdqKhyCocqEctKFRERERGRtahwpap+/fq4cOECGjRogHr16pU4UIXB7du3zdI4KuDrK1/+BwB50bEoPr4hERERERFZQoVD1ccffwwXFxfj92WFKjI/BwcgzckXyAS01+IYqoiIiIiIrESFQ9X48eON34eGhtZEW6gc2oZ+QDSAeF7+R0RERERkLap0T1V4eHiJ2/Pz8/HGG29Upz1UBslPvqdKfZMDVRARERERWYsqhaoXX3wRjz32GO7cuWPcdv78eXTr1g0//PCD2RpHpjRN/gtVOelAerqFW0NEREREREAVQ9WxY8dw48YNtGvXDpGRkVi+fDnuvfdeBAUF4fjx4+ZuI/2nQRMXpEG+r41zVRERERERWYdKzVNl0KxZM+zbtw/Tp0/HwIEDoVQqsWrVKowePdrc7aNCDBMAu+KcHKpatbJ0k4iIiIiI7npVqlQBwB9//IF169YhODgY7u7u+PbbbxHH6kmN4gTARERERETWp0qh6tlnn8Vjjz2GWbNmYe/evThx4gTUajXatWuH//3vf+ZuI/3HUKkCwMv/iIiIiIisRJVC1b59+3Dw4EG88sorkCQJ3t7e+PPPPzF//nw8/fTT5m4j/adwpSo/hpUqIiIiIiJrUKV7qo4cOQKNpvj0s1OnTkVISEi1G0Ula9AASFL6Ajog90pc1X54RERERERkVlWqVGk0Gly+fBmzZ8/G6NGjkZSUBADYvHkz8vPzzdpAKiBJQHZ9+fI/3XVWqoiIiIiIrEGVQtXu3bvRrl07HDx4EBs2bEBGRgYA4Pjx45g7d65ZG0im9N7y5X/KRN5TRURERERkDaoUql5//XUsWLAAkZGRUKvVxu19+vTBgQMHzNY4Kk4ZIFeq7G/HAUJYuDVERERERFSlUHXy5Ek8+uijxbY3bNgQycnJ1W4Ulc6pmTcAQKnTAnyviYiIiIgsrkqhyt3dHfHx8cW2Hzt2DH5+ftVuFJXOO0CNRDSUVzisOhERERGRxVUpVD3xxBOYNWsWEhISIEkS9Ho99u3bh5kzZ2LcuHHmbiMVwgmAiYiIiIisS5VC1cKFCxEUFAR/f39kZGSgTZs26NmzJ7p3747Zs2ebu41UCCcAJiIiIiKyLlWa6kitVuPrr7/G22+/jVOnTiEjIwOdOnVCixYtzN0+KsLPDzj7X6VK3IiFZOH2EBERERHd7ao1f2xAQAACAgLM1RaqAF/fgkpVbnQc7C3cHiIiIiKiu12FQ9WMGTMqfNAlS5ZUqTFUPnt7IM3JF8gE8qJjGaqIiIiIiCyswqHq2LFjFdpPknhBWk3TesqhSvCeKiIiIiIii6twqNq5c2dNtoMqw88PuAqokzj6HxERERGRpVVp9L/Crl+/juvXr5ujLVRB6kB5oAqH9CRAq7Vwa4iIiIiI7m5VClX5+fl4++234ebmhsDAQAQGBsLNzQ2zZ8+Glh/ya5xbswbIg0peSUiwbGOIiIiIiO5yVRr974UXXsCGDRvw/vvvIzg4GACwf/9+zJs3D7du3cKKFSvM2kgy5dtIgXj4oDFi5AmA/f0t3SQiIiIiortWlUJVREQE1q1bh0GDBhm3tW/fHv7+/hg9ejRDVQ0zTADcGDGcAJiIiIiIyMKqdPmfRqNBYGBgse1NmjSBWq2ubpuoHH5+QNx/EwAjloNVEBERERFZUpVC1bRp0/DOO+8gNzfXuC03Nxfvvvsupk2bZrbGUckMlSoAyI9hpYqIiIiIyJKqdPnfsWPHsH37djRq1AgdOnQAABw/fhx5eXno27cvRowYYdx3w4YN5mkpGXl4AElKX0AH5FyOhbOlG0REREREdBerUqhyd3fHyJEjTbb5c7CEWiNJQHY9XyCZlSoiIiIiIkurdKgSQiAsLAyenp5wcHCoiTZRBei8/YBkQBHPe6qIiIiIiCyp0vdUCSHQvHlz3LhxoybaQxWk9JcHqtDcZqWKiIiIiMiSKh2qFAoFWrRogVu3btVEe6iC7JvJA1VoctKAjAwLt4aIiIiI6O5VpdH/3nvvPbz66qs4deqUudtDFdSgiQvSDUNUcK4qIiIiIiKLqdJAFePGjUNWVhY6dOgAtVpd7N6q27dvm6VxVDrDsOpBOC+HqpYtLd0kIiIiIqK7UpVC1dKlS83cDKoswwTAQTjPCYCJiIiIiCyoSqFq/PjxZjn5okWLsGHDBpw7dw4ODg7o3r07Fi9ejFatWhn3ycnJwSuvvIJ169YhNzcXAwYMwOeffw4vLy+ztMFW+fkBf/83AbCIjYNk4fYQEREREd2tqnRPFQBcvnwZs2fPxujRo5GUlAQA2Lx5M06fPl3hY+zevRtTp07FgQMHEBkZCa1Wi/79+yMzM9O4z8svv4zffvsN69evx+7duxEXF2cyufDdytdXrlQBQM4VVqqIiIiIiCylSqFq9+7daNeuHQ4ePIgNGzYg47/R544fP465c+dW+DhbtmxBaGgo2rZtiw4dOiA8PBwxMTE4cuQIACA1NRXffvstlixZgj59+qBz585YuXIl/vnnHxw4cKAqTa8zNBog1UmuVOVGc6AKIiIiIiJLqdLlf6+//joWLFiAGTNmwMXFxbi9T58++Oyzz6rcmNTUVABA/fr1AQBHjhyBVqtFSEiIcZ+goCAEBARg//79uP/++4sdIzc3F7m5ucb1tLQ0AIBWq4VWq61y28zBcH5ztSPXwwfIBPQxNyz+2qh2mbsv0d2LfYnMgf2IzIV9icyltL5UU32rSqHq5MmTiIiIKLa9YcOGSE5OrlJD9Ho9pk+fjh49euCee+4BACQkJECtVsPd3d1kXy8vLyQkJJR4nEWLFiEsLKzY9r/++guOjo5Vapu5RUZGmuU4dxzsAQD62Gv4888/zXJMsi3m6ktE7EtkDuxHZC7sS2QuRftSVlZWjZynSqHK3d0d8fHxaNKkicn2Y8eOwc/Pr0oNmTp1Kk6dOoW///67Ss83eOONNzBjxgzjelpaGvz9/dG/f3+4urpW69jVpdVqERkZiX79+kGlUlX7ePsibgDnAfesZAweNAiQOFzF3cLcfYnuXuxLZA7sR2Qu7EtkLqX1JcNVbOZWpVD1xBNPYNasWVi/fj0kSYJer8e+ffswc+ZMjBs3rtLHmzZtGn7//Xfs2bMHjRo1Mm739vZGXl4eUlJSTKpViYmJ8Pb2LvFYGo0GGo2m2HaVSmU1/zjN1RaXlnKAtdPlAWlpQIMG1T4m2RZr6tdk29iXyBzYj8hc2JfIXIr2pZrqV1UaqGLhwoVo3bo1AgICkJGRgTZt2qBnz57o3r07Zs+eXeHjCCEwbdo0bNy4ETt27ChW+ercuTNUKhW2b99u3Hb+/HnExMQgODi4Kk2vU3waq5EET3mFc1UREREREVlEpSpVer0eH3zwAX799Vfk5eVh7NixGDlyJDIyMtCpUye0aNGiUiefOnUqIiIi8Msvv8DFxcV4n5SbmxscHBzg5uaGiRMnYsaMGahfvz5cXV3xwgsvIDg4uMRBKu42hgmAG+ImEBcHdOhg6SYREREREd11KhWq3n33XcybNw8hISFwcHBAREQEhBD47rvvqnTyFStWAAB69+5tsn3lypUIDQ0FAHz88cdQKBQYOXKkyeS/JIeq6/BDRxyXQxUREREREdW6SoWq1atX4/PPP8ezzz4LANi2bRuGDBmCb775BgpF5a8kFEKUu4+9vT2WL1+O5cuXV/r4dZ2fH3DwvwmA86/FVu0GOSIiIiIiqpZKJaGYmBgMHjzYuB4SEgJJkhDHKolF1K8PJCrlwSqyLvFnQERERERkCZUKVfn5+bC3tzfZplKpOEGbhUgSkF2voFJFRERERES1r1JXjAkhEBoaajJkeU5ODqZMmQInJyfjtg0bNpivhVQmrZcfkAwgnpUqIiIiIiJLqFSoGj9+fLFtTz31lNkaQ5WnbOQLnAY0yaxUERERERFZQqVC1cqVK2uqHVRFmiby5X8O6UmAVgtwojwiIiIiolpVpcl/yXq4NfeEFnZQQAD/zfNFRERERES1h6HKxvn5KxAPH3mFozASEREREdU6hiob5+cHxEIeVh2xvK+KiIiIiKi2MVTZOD8/IO6/CYBFLCtVRERERES1jaHKxvn6FlSqsi8zVBERERER1TaGKhunVgOpTnKlKucyL/8jIiIiIqptDFV1QK6HXKnS32ClioiIiIiotjFU1QHCR65UKRNZqSIiIiIiqm0MVXWAXcB/EwDfZqWKiIiIiKi2MVTVAQ7N5cv/7HNTgcxMC7eGiIiIiOjuwlBVB3g2dUEGnOQVTgBMRERERFSrGKrqAL9GEicAJiIiIiKyEIaqOqDwBMCsVBERERER1S6GqjrAz69gAmDtVVaqiIiIiIhqE0NVHVCvHpCklCtVWZdYqSIiIiIiqk0MVXWAJAGZ7nKlKu8qQxURERERUW1iqKoj8hsa7qni5X9ERERERLWJoaqu8JMrVaqbrFQREREREdUmhqo6QtNErlQ5pcYBQli4NUREREREdw+GqjrCpaUPAEClywVu37Zwa4iIiIiI7h4MVXWEd2MNbqKBvMIJgImIiIiIag1DVR3BCYCJiIiIiCyDoaqOKDwBsLjBShURERERUW1hqKojfHwKKlWcAJiIiIiIqPYwVNURajWQ4ihXqrIvsVJFRERERFRbGKrqkFwPuVKlu85KFRERERFRbWGoqkN03nKlSpHAUEVEREREVFsYquoQpb9cqbK/zcv/iIiIiIhqC0NVHeLQTA5VThmJQH6+hVtDRERERHR3YKiqQ9xbNkQ+lFBAAAkJlm4OEREREdFdgaGqDvHzVyAePvIKJwAmIiIiIqoVDFV1SOEJgBHL+6qIiIiIiGqDRUPVnj17MHToUPj6+kKSJGzatMnk8dDQUEiSZLIMHDjQMo21AX5+BRMA511lpYqIiIiIqDZYNFRlZmaiQ4cOWL58ean7DBw4EPHx8cblhx9+qMUW2hZ3dyBRKVeqMi+wUkVEREREVBvsLHnyQYMGYdCgQWXuo9Fo4O3tXUstsm2SBGS6+QK3gVxWqoiIiIiIaoVFQ1VF7Nq1Cw0bNkS9evXQp08fLFiwAB4eHqXun5ubi9zcXON6WloaAECr1UKr1dZ4e8tiOH9NtiPP0we4DYjrNyz+eqnm1EZforsD+xKZA/sRmQv7EplLaX2ppvqWJIQQNXLkSpIkCRs3bsTw4cON29atWwdHR0c0adIEly9fxptvvglnZ2fs378fSqWyxOPMmzcPYWFhxbZHRETA0dGxpppvNXa/nYklJ8cg1r0Z/g3/yNLNISIiIiKyGllZWXjyySeRmpoKV1dXsx3XqkNVUVeuXEGzZs2wbds29O3bt8R9SqpU+fv7Izk52axvXFVotVpERkaiX79+UKlUNXKOJZPOY9aqdsjSuEOVnlQj5yDLq42+RHcH9iUyB/YjMhf2JTKX0vpSWloaGjRoYPZQZfWX/xXWtGlTNGjQAJcuXSo1VGk0Gmg0mmLbVSqV1fzjrMm2uAb5AwAcc1MArRa4C6pzdzNr6tdk29iXyBzYj8hc2JfIXIr2pZrqVzY1T9WNGzdw69Yt+Pj4WLopVsuzmSsy8V+Q4gTAREREREQ1zqKhKiMjA1FRUYiKigIAREdHIyoqCjExMcjIyMCrr76KAwcO4OrVq9i+fTuGDRuG5s2bY8CAAZZstlXzayRxAmAiIiIiolpk0VD177//olOnTujUqRMAYMaMGejUqRPmzJkDpVKJEydO4JFHHkHLli0xceJEdO7cGXv37i3x8j6SFZ4AWH+DlSoiIiIioppm0XuqevfujbLGydi6dWsttqZu8PEB9v5Xqcq8GAsXC7eHiIiIiKius6l7qqh8KhWQ6ihXqrIvsVJFRERERFTTGKrqoKz6cqUq/xrvqSIiIiIiqmkMVXWQzkuuVEkJrFQREREREdU0hqo6SNFIrlRpklmpIiIiIiKqaQxVdZBDM7lS5ZIeB5QxEAgREREREVUfQ1Ud5NJKDlUqXS5w546FW0NEREREVLcxVNVBPoEaJMNDXuEEwERERERENYqhqg4qPAEw4jhYBRERERFRTWKoqoP8/IDY/yYAzotmpYqIiIiIqCYxVNVBbm5AklKuVKWfZ6WKiIiIiKgmMVTVQZIEpLvJlarcK6xUERERERHVJIaqOirPQ65U6W+wUkVEREREVJMYquoo4StXquwSWakiIiIiIqpJDFV1lKqxXKlySGGlioiIiIioJjFU1VGOLeRKlUtmIpCfb+HWEBERERHVXQxVdVS9lp7IhxIK6IHEREs3h4iIiIiozmKoqqP8ApRIgLe8wgmAiYiIiIhqDENVHVV4AmD9dQ5WQURERERUUxiq6ihvbyAO8mAVGRdYqSIiIiIiqikMVXWUSgXccZQrVVkXWakiIiIiIqopDFV1WLa7XKnKu8ZKFRERERFRTWGoqsO0DeVKlRTHShURERERUU1hqKrDFI3kSpU6mZUqIiIiIqKawlBVh2maypUq51RWqoiIiIiIagpDVR3m0kquVDnlpQBZWZZtDBERERFRHcVQVYd5NndDJhzlFU4ATERERERUIxiq6jC/RpJxriqGKiIiIiKimsFQVYf5+RVMAJwbzVBFRERERFQTGKrqMFdXIFEpD1aRuS8KEMKyDSIiIiIiqoMYquowSQIS3VoCAOp/vRgYMgS4cMHCrSIiIiIiqlsYquq4P+6ZhUV4HTqlCti8GbjnHmDWLCA93dJNIyIiIiKqExiq6jjPAAe8iUV485HT0A0aAmi1wPvvAy1bAmvWAHq9pZtIRERERGTTGKrquHvvlb++v7EFWpz7HX+//jtE8+ZAQgIwbhzwwAPAkSOWbSQRERERkQ1jqKrjXnoJCA8HfH2B6GjgwfeGoG/DU7gx7T3AyQnYvx/o2hWYPBm4edPSzSUiIiIisjkMVXWcQgGMHy+PTzFnDuDgAOz8RwP/z2bhxYEXkDniKXlUwK+/li8JXLYMyM+3dLOJiIiIiGwGQ9VdwskJCAuTw9XYsfK2ZT/7wnPzGnwT+jd0HToBKSnAiy8CnToBO3datL1ERERERLaCoeou06gRsHo1cOgQ0KMHkJ0NTArvgcCkw/gn9EsIDw/g1CmgTx/g8ceBmBhLN5mIiIiIyKoxVN2lunYF9u4F/vc/IDAQuBGvRI/wyejb6AJiH50mXze4fj0QFATMny+nLyIiIiIiKsaioWrPnj0YOnQofH19IUkSNm3aZPK4EAJz5syBj48PHBwcEBISgosXL1qmsXWQJAGPPQacPQssXgy4uAA7j9dHo43LMOOhY8i+r5ccpubOBdq0ATZulO+/IiIiIiIiI4uGqszMTHTo0AHLly8v8fH3338fn376Kb744gscPHgQTk5OGDBgAHJycmq5pXWbvT3w2mvApUvAlClykerj7e3hfmwn1jz8I/S+jYCrV4ERI4D+/eUURkREREREACwcqgYNGoQFCxbg0UcfLfaYEAJLly7F7NmzMWzYMLRv3x6rV69GXFxcsYoWmUfDhsCKFcDx43J2ytNKGPf742iSew7/DpwNodEA27YB7dsDM2YAqamWbjIRERERkcXZWboBpYmOjkZCQgJCQkKM29zc3NCtWzfs378fTzzxRInPy83NRW5urnE9LS0NAKDVaqHVamu20eUwnN/S7ShPq1bAb78BW7dKeO01Jc6dc0LXLe8gpGkoVjd8BT4HfgE+/hhi7VroFiyAGDdOLm9RrbGVvkTWj32JzIH9iMyFfYnMpbS+VFN9SxLCOm6SkSQJGzduxPDhwwEA//zzD3r06IG4uDj4+PgY93v88cchSRJ+/PHHEo8zb948hIWFFdseEREBR0fHGml7XZafL+GvvwLxww+tkJ6uAQA81+xHvJv+KuolXQcA3GnRAicnTcKdli0t2VQiIiIiojJlZWXhySefRGpqKlxdXc123DoXqkqqVPn7+yM5Odmsb1xVaLVaREZGol+/flCpVBZtS2WlpACLFinw2WcKaLUS7BW5WN1tGUaenA9FRjoAQD9uHHTvvgt4eVm2sXcBW+5LZF3Yl8gc2I/IXNiXyFxK60tpaWlo0KCB2UOV1V7+5+3tDQBITEw0CVWJiYno2LFjqc/TaDTQaDTFtqtUKqv5x2lNbakoT09gyRLg+eflQS02btTg8f0z0dz5KWzo8gba/RsOxerVUGzaJI8W+MILgI29Rltki32JrBP7EpkD+xGZC/sSmUvRvlRT/cpqb4Rp0qQJvL29sX37duO2tLQ0HDx4EMHBwRZs2d2teXNgwwZg506gUyfgUoY32v+7EiN8D+B2865AWhrwyivyYBa//cYh2ImIiIiozrNoqMrIyEBUVBSioqIAyINTREVFISYmBpIkYfr06ViwYAF+/fVXnDx5EuPGjYOvr6/xEkGynN69gX//BVauBHx8gI1x3dDg0gG81+JbaOt5AufOAY88AnToAKxdC+TnW7rJREREREQ1wqKh6t9//0WnTp3QqVMnAMCMGTPQqVMnzJkzBwDw2muv4YUXXsDkyZPRtWtXZGRkYMuWLbC3t7dks+k/CgUQGgpcuADMmQPYOyjwxsWn4XnnAv5o+xr0Ts7AyZPAU08BLVoAn38uTyZMRERERFSHWDRU9e7dG0KIYkt4eDgAefCK+fPnIyEhATk5Odi2bRtacoQ5q+PsDISFAefPy/kpFe54+PRieOXEYH3HBXLl6upVYOpUoHFjYOFCeeQLIiIiIqI6wGrvqSLb4+8PrFkDHDoEPPQQkKyrh8ej3oLbnatYFvQZshs2Bm7eBN56CwgIkEe8iIuzdLOJiIiIiKqFoYrMrmtXYMcO4PBh4IkngDylI148NxWuSRfxeqPvcafRPUB6OvDBB0CTJsDkycDFi5ZuNhERERFRlTBUUY3p0gX44Qfg0iVg+nRA46TC4htjUP/GCYyr/zuuBz4A5OUBX38NtGoFPP44cOSIpZtNRERERFQpDFVU4wIDgY8/Bm7cABYvBnx9Jay5PQQBV/ein/1enGrysDz0+vr1chLr318udXE4diIiIiKyAQxVVGvc3eXbqKKjgVWr5KmstuU8gHbRv6GDdAJ7Gj8FoVQCkZFA375At27ypFh6vaWbTkRERERUKoYqqnVqNTBuHBAVBfz1l1yYOiHaode1NWiiu4QNvtOgU9vLN2WNHAm0aQN89518qSARERERkZVhqCKLkSSgXz9g61bg+HE5aMWpAjEybhl88q7hc4/ZyHV0l8dqnzgRaNoUWLJEHuSCiIiIiMhKMFSRVWjfXr4kMDoamDULyHNriKm33kGDrBjMdfoQaS6+QGws8Mor8lxXc+bIw7MTEREREVkYQxVZFT8/4L33gOvX5cEtPBq7YH7mK/BMv4Ipdt8g0a0lcOcO8M47crh68UXg2jVLN5uIiIiI7mIMVWSVXFzkYdgvXQLWrQPad9Hgy/yJ8E09g//DT7jk1hnIzgaWLQOaNZOvHTx1ytLNJiIiIqK7EEMVWTU7O2DUKODQIWD3bmDIUCV+xki0SD2MvtiGQ64hgE4HrFkDtGsHhIQAr74qz321ezcQH8+h2YmIiIioRtlZugFEFSFJQM+e8nLuHLBkiYTVq/uiW1pfdMa/mO+4GIOyf4a0fTuwfbvpk11cgJYtC5ZWreSvLVoArq6WeUFEREREVGcwVJHNCQoCvvpKvq3q88+B5cu7YMit9WiOixiq/gu9/C6ig/15+GZcgCr2KqT0dODIEXkpysen5MDVpIk89jsRERERUTkYqshmeXkBYWHyaIGrVgFLlrTAx5da4OPogn2a+ObiifuuYEDgedzrfAEu8RfkIdovXACSkuTLA+Pj5UsFC1Mq5WBlCFmFA5evr1w6IyIiIiICQxXVAY6OwHPPAc8+K893FRkpL3v3AtFxGiza1BqL0BqAPHR7v35AyFtAz/YpcIy9WBCyCi+ZmfIoGZcuAX/8YXpCJyf50sHCgat1a7mE5uRkgXeAiIiIiCyJoYrqDIUC6NRJXl57TR4ccN++gpB17Bhw4oS8fPQRoFa7o0ePrujXrytChgL33isXqCAEEBdXELAKh64rV+TAFRUlL0UFBMgBq00b+ath8fCo5XeDiIiIiGoLQxXVWQ4O8mCAISHA4sXyXME7dhSErJgYYOdOeXnzTaB+faBPH6BfPwkhIX5o+pAf8NBDpgfVauVgVbiqde6cvCQlyQeNiQG2bjV9nqdn8aDVpg0vJSQiIiKqAxiq6K7h6SkPzz5qlFyMungR2LZNDlg7dgC3bwM//SQvANC0qXypYL9+craqXx+ASiVf9teqVfET3LoFnD1bsJw5I3+NiZET3e7dxe/dcnExDVmG75s0+a9sRkRERETWjqGK7kqSVHA71PPPA/n5wOHDBSFr/365IPXll/KiUACdOxeErOBgQKMpclAPD+CBB+SlsIwM+RJCQ8gyLJcuAenp8iRchw6ZPkejkRtXtLrVsmUJJyYiIiIiS2KoIoI8yXBwsLy8/bacdXbvLghZZ87IoevwYWDhQnlwjJ49/xv0IgRo27aMwpKzs5zIOnc23Z6XJ5fLila2zp8HcnKAkyflpTCFAmjWDMpWrXAPAMWxY/IwiA0ayKGuQYOC7xm+iIiIiGoFQxVRCVxcgIcflhcAiI2VA5YhZCUmAlu2yAsgh6wOHYCOHeWBMjp2BNq1A+ztyziJWi2nsbZtTbfrdMC1a8UrW2fPAqmpwMWLUFy8iGYA8PvvZb8IQ8gqb/HwkBc7/kogIiIiqix+giKqAD8/YPx4eRECOHVKDlfbtgF79sgDAu7fLy8GSqU8yrphRMKOHeWlfv1yTqZUyjd0NW1akOoA+cTx8cDZs9CdOoXLe/eiubs7FLdvA8nJ8j1dycnyotfL5bb0dCA6uvRzFeXuXn74cncH6tWTv7q7y+FNoaj4OYiIiIjqGIYqokqSJLkK1a4dMGOGXFi6eFEesj0qSv567JicbU6flpfvvy94fkBAQcgyBC5//woMAihJ8miBvr7Q9+yJs4GBaDJ4MBQqlel+er1c0TIErIosd+7IoS0lRV4uXar4G6JQAG5uxcNWSd+XtM3eniMgEhERkU1jqCKqJkNFKigIGD1a3maY6qpw0IqKkge/MIy6/ssvBceoX7+gkmUIXEFBVbwaT6GQA0u9evIkxRWh08nBqrzwdetWQfC6cwfIzZVD3J078lKZqpiBWl1+GPP0lO8dMywNG8rPIyIiIrICDFVENUCS5EsG/fxMr+BLSQGOHzcNWqdPy8O579ghLwYajVwNK3z5YPv2gJNTDTRYqSy4xK8ycnIKAlbhsFXe94ZFp5MH7EhMlJfKqFfPNGQVDl1FFweHyh2biIiIqBIYqohqkbs70KuXvBjk5srBqnDQioqSR2L/9195MTAMBd+hgxKS1Arx8RICAgoCnIdHLV9JZ28PeHvLS2UJIb/I8oLYnTvyxMqG4JWUJI+Bb3js3Lnyz+XsXHboKhzMXFx4OSIRERFVCkMVkYVpNMC998qLgV4vXypY9D6thAR5xPXz5xUAgvDjj6bHUqvl264MIaukxde3nFEJa4skyQHGxUW+qayiDJcbFg5aZS25uXJ4y8gALl8u//j29qbhqrRFoTD/40qlPJSkk5PpUpltRe+xIyIiohrHUEVkhRQKoHlzeXnssYLtCQlyyDpyRIc9e67Dzi4AcXEKxMYCN2/KV9JdvSovZfHwKAhYpYWvBg2stGCjUBQMAd+6ddn7CiGPgFiR8JWUJAevnBx5SHtbpVKVHb6KrCvs7dH0yhUorl2TL5NUq6u32NlZacchIiKqOQxVRDbE2xsYOBDo21ePe+45jsGD/aBSycOZ5+bKoSs2tuwlJ0ceb+LWLeDEidLPVVLVy7Be+KujYy29+KqQJMDVVV4qMmhHZmZBBSwzUw5lpS16vfkfz88HsrLkcxu+Fl7K2qbTya9Bqy24fLIClADaVfX9LU1FA5hGU3uLSsWwR0RENYahiqiO0GiAxo3lpTRCyFfOxcbKoxOWFrySkipe9XJ3LzlsFf7q5WUj8wo7OQFNmsiLLRFC/oFVJHwV2aZPT0dsdDT8GjSAIj9fPk5Zi1Zrup6bW7w9hsesjUYjX97p4FCwVHa9ss9RKi39qomIqBbYwsccIjITSZKHb69fXx5ZsDR5efI8wyWFr8LrWVkFRZEzZ8o+r5dX2cHL11duF4sJVSBJBRWZcmeXNqXTanH0zz/hXdKcZxUhRMEojpVZcnNrfsnPN22rYXtqauVfZ1WpVHI513DZZXW/Ft3Ged6IiKwCQxURFaNWV6zqlZZWELJK+xofL3/mTkiQlyNHSj+mRmOc39gYtHx95YH5DCO+GxZXV36WtAqSJJch7eys71pQvd40ZOXkyEt2dsFXw2LO9cJVOq1WDnE1FeQkqeQA5uJSMN+bYXLushZnZ/l+RSqg18uLTlewGNbVavk9IyL6D0MVEVWJJMmf1dzcyh4vQqeTB9EoL3zduiV/7o2OrtgcwnZ28lgVRcNW4aXo487ODGJ3FYWi4DK82qTTyZ3ZELKK3idX1a+FvzcENyEKHqsOwz/o8sJXafsUfo8Nl6MWrRzm5Jh3m2F7fn7p4aek9YruWx4nJ8DHR158fQu+L7rN3Z2/eIjuAgxVRFSjlMqCqawKDxtfVE6OXNUqKXQlJxcst27Jnx/z8ys/Z7BaXXboKrwY/njv7Cx/XuRnIqoww9D4NVm5MwxqUloAS0+Xq2NFJ9wuvKSmyjdZ5uXJQagSA5wUZSdJGKLRQClEyffZ1UWZmcClS/JSFo2m/ODl4yP/QmK1kMhmMVQRkVWwt6/4GBHZ2XK4Khq2Cq8XXXJy5M+OcXHyUhmSJP9R2hCyCi+lbS/vcScnOeQRVYmdXcHIltWVk1N28CovmGVnQxICdjk5pbfV3t50NMai6yVtq8g+hpEdlUo5kCiVZX9vrv1ycuTrmQ1/CYqPN10M21JS5JBZkVF/VCr5r0+lBS8vr4IqYNEBY0raVt11w7bCFV/DICyFB2Mp6fuKbjN8X5X7OfV6+Y8LhkWrNV0vb3vRx3Q6ua9qNKajk5b1vVrNv7iREUMVEdkcBwegUSN5qaisrOJBq6wglppacEWVEAXzB5uTSlX5IFbeNicn/rGbKsnevqCcXBW5udAmJ2P3b7+hV0gIVC4upqGnLnZIlUq+b628qRqys8sOX4btycnyB/zr1+XlbqNUGoOWnYMDQvLyYKfRlB2ChLB0q2UlTRFR0VBm+GpnJ/cpw9eKfF+dfe3sCu4ZLLoYpvyozlLaMerXBzp2tPRPrMYwVBHRXcHREQgIkJeK0uvlz0SGQFV0ycws/bGy9jHcDqPVyldf3blj/tdakQAmX9qowNWrjZGWJqFePflzoqur/NXwPQeYozJpNEDDhsj08ZFHt6lK1aGucnCoWAk+L0++lrlopavwkpQkB1S1Wn6Pi877VhPbVKqCX4SFB2MpacCXij5e+KuBTme8N1AC4FSd97xwcDAsJW0rul2plH8p5+aa3hNY9P7AovfbGap65v6rW100cCCwebOlW1FjGKqIiEqhUBSMYu3lZb7jGqaUKhq60tNN10sLbSVtz8yUP/sABbfa3LxZkdYoAXTEihVl7KE0DVnV+d7JiQGNqBi1GvD3l5e7heH+uyKhS5uRgf27dqF7r16ws7evXECqjaqoYQqJkgJXWWGstMe02oLFUI0r/H1J2yrzeOHqXkVIkvw+lrSU9VhFlrKGFK4DrDpUzZs3D2FhYSbbWrVqhXPnzlmoRURE1Wf4Q3C9euY7phDyZ5KKBjDD96mpely+nAhHRy9kZCiQni4PlZ+eXvCHV52uWmMYmJCkgqqZvX3BLTJlfTXXY4bFzo7BjsjiJKngH2rhX4ZaLe4kJEB07mydVU+l0jKjilaXYU5BrVb+Wlpo4i/HKrPqUAUAbdu2xbZt24zrdnZW32QiolonSQX/z3t6Vvx5Wq0Of/55CIMHD4ZKZfpXXr2+oIJmWAyBqyrfGy61N6xbUtErnQovhe9Br+xS0nOL3tZQ9I/tlX2sLt6iREQ1rPCcglQjrP6dtbOzg3dVb54lIqIqUyjMN8CcEPIliYZAlZlpOv1QWV+r+ljhfUq7DcIWGT4blRXG1OqCIkDRqp25t2s0ln5HiIgsz+pD1cWLF+Hr6wt7e3sEBwdj0aJFCCjjTvPc3FzkFpojIy0tDQCg1Wqh1WprvL1lMZzf0u0g28e+ROZSm31JrZan4vHwqPFTFZOfX3D7giFQGW5tkEeOloqNJl18tGnTfQzPLzimVOw5ubnFBy3TaiXjNsPVOEUHNzN81euLX4ojRMGtEtZCrbaDUjkYKpUdgJJHZSvvqqKyHi/rMaXSNFQWXbezE6WOS1DymAai3H3UasDBQZiMDF6wiBK2scJYUfz/jcyltL5UU31LEsJaxqQsbvPmzcjIyECrVq0QHx+PsLAwxMbG4tSpU3BxcSnxOSXdhwUAERERcKzJiRiJiIjMTK8HdDoJer0C+fkSdDoJOp2iyNfi27RaRaFFibw806+Gx/LylCZfS9pW+DHDMfLylBCC915Uhp2dDhqNDmq1Hmq14XvDYrqt4DG9cR+NRgelUkCpFFAoBJRKPRQKw/fC5DGFQl9ov/Ifk7fB5DGGQKqrsrKy8OSTTyI1NRWu5rgU4z9WHaqKSklJQePGjbFkyRJMnDixxH1KqlT5+/sjOTnZrG9cVWi1WkRGRqJfv35QWePNl2Qz2JfIXNiXqCqEKKj+yQOk5GP79n3o0aNHifc+l/dJo6zHy3tMpyuYuzU/XypWGSxpvaBCKFVojtiiVca8PNPRwbOygOxsyWQE8exsuXppqyTJtGJX2veG9YLtotT9FYrSj2OoKAJ6xMVdRatWgXBwUBS65FSYXNZqOgiNKHWAGqWy5t4jvd50ML+iFXBD9bqkfeRKtgSlEnB0FMaRZp2c5Gpn4XUnJ+scs8Palfb/W1paGho0aGD2UGX1l/8V5u7ujpYtW+LSpUul7qPRaKAp4QJvlUplNR8YrKktZNvYl8hc2JeostRqeU40QP6A6OOTiaAgO/ajQnQ605BV0iIHsvIfz8kpHvIMgbKq62WNsi2EVMXLTKsbJJUAyplUuRLs7Co2aqhaLb8/pYWkkr4WvVezJqlUKBa0Ci+G+Qkrsjg4mIbiyiy2ODhg0f/faup3lE2FqoyMDFy+fBljx461dFOIiIiIyqRUFky0ba30+tIDmOG+v6KPl7Wtst8X3Zabq8OFC1fg59cUeXnKKg1eY5izDyg4dmZmzb+XhjmUDaOAFh4NtOg2w9dC8x4bl6ysgu8NwU2rNd/UFtUhSVULY0ol8MADwBdfWLb9NcmqQ9XMmTMxdOhQNG7cGHFxcZg7dy6USiVGjx5t6aYRERER2TyFQv5wby20Wj3+/PMMBg8OhEpVtWv38vPLD2FFt+XmFlS1ygtCJX2tiekOhCiYLL60pXAAq+h+WVlyWCtpKRxIS2tTZeYSLqyMcebqBKsOVTdu3MDo0aNx69YteHp64oEHHsCBAwfgWZlJWIiIiIjormFnZ/0VwoqQpIJJy+vXr51zCmEYIKf0xVBVrOxSW6/BUqw6VK1bt87STSAiIiIiuisUvryPKocDZhIREREREVUDQxUREREREVE1MFQRERERERFVA0MVERERERFRNTBUERERERERVQNDFRERERERUTUwVBEREREREVUDQxUREREREVE1MFQRERERERFVA0MVERERERFRNTBUERERERERVQNDFRERERERUTUwVBEREREREVUDQxUREREREVE12Fm6ATVNCAEASEtLs3BLAK1Wi6ysLKSlpUGlUlm6OWTD2JfIXNiXyBzYj8hc2JfIXErrS4ZMYMgI5lLnQ1V6ejoAwN/f38ItISIiIiIia5Ceng43NzezHU8S5o5pVkav1yMuLg4uLi6QJMmibUlLS4O/vz+uX78OV1dXi7aFbBv7EpkL+xKZA/sRmQv7EplLaX1JCIH09HT4+vpCoTDfnVB1vlKlUCjQqFEjSzfDhKurK39RkFmwL5G5sC+RObAfkbmwL5G5lNSXzFmhMuBAFURERERERNXAUEVERERERFQNDFW1SKPRYO7cudBoNJZuCtk49iUyF/YlMgf2IzIX9iUyl9ruS3V+oAoiIiIiIqKaxEoVERERERFRNTBUERERERERVQNDFRERERERUTUwVBEREREREVUDQ1UtWr58OQIDA2Fvb49u3brh0KFDlm4SWdCePXswdOhQ+Pr6QpIkbNq0yeRxIQTmzJkDHx8fODg4ICQkBBcvXjTZ5/bt2xgzZgxcXV3h7u6OiRMnIiMjw2SfEydO4MEHH4S9vT38/f3x/vvv1/RLo1q0aNEidO3aFS4uLmjYsCGGDx+O8+fPm+yTk5ODqVOnwsPDA87Ozhg5ciQSExNN9omJicGQIUPg6OiIhg0b4tVXX0V+fr7JPrt27cK9994LjUaD5s2bIzw8vKZfHtWiFStWoH379saJMoODg7F582bj4+xHVBXvvfceJEnC9OnTjdvYl6ii5s2bB0mSTJagoCDj41bVlwTVinXr1gm1Wi2+++47cfr0aTFp0iTh7u4uEhMTLd00spA///xTvPXWW2LDhg0CgNi4caPJ4++9955wc3MTmzZtEsePHxePPPKIaNKkicjOzjbuM3DgQNGhQwdx4MABsXfvXtG8eXMxevRo4+OpqanCy8tLjBkzRpw6dUr88MMPwsHBQXz55Ze19TKphg0YMECsXLlSnDp1SkRFRYnBgweLgIAAkZGRYdxnypQpwt/fX2zfvl38+++/4v777xfdu3c3Pp6fny/uueceERISIo4dOyb+/PNP0aBBA/HGG28Y97ly5YpwdHQUM2bMEGfOnBHLli0TSqVSbNmypVZfL9WcX3/9Vfzxxx/iwoUL4vz58+LNN98UKpVKnDp1SgjBfkSVd+jQIREYGCjat28vXnrpJeN29iWqqLlz54q2bduK+Ph443Lz5k3j49bUlxiqasl9990npk6dalzX6XTC19dXLFq0yIKtImtRNFTp9Xrh7e0tPvjgA+O2lJQUodFoxA8//CCEEOLMmTMCgDh8+LBxn82bNwtJkkRsbKwQQojPP/9c1KtXT+Tm5hr3mTVrlmjVqlUNvyKylKSkJAFA7N69Wwgh9xuVSiXWr19v3Ofs2bMCgNi/f78QQg74CoVCJCQkGPdZsWKFcHV1Nfad1157TbRt29bkXKNGjRIDBgyo6ZdEFlSvXj3xzTffsB9RpaWnp4sWLVqIyMhI0atXL2OoYl+iypg7d67o0KFDiY9ZW1/i5X+1IC8vD0eOHEFISIhxm0KhQEhICPbv32/BlpG1io6ORkJCgkmfcXNzQ7du3Yx9Zv/+/XB3d0eXLl2M+4SEhEChUODgwYPGfXr27Am1Wm3cZ8CAATh//jzu3LlTS6+GalNqaioAoH79+gCAI0eOQKvVmvSloKAgBAQEmPSldu3awcvLy7jPgAEDkJaWhtOnTxv3KXwMwz78HVY36XQ6rFu3DpmZmQgODmY/okqbOnUqhgwZUuznzb5ElXXx4kX4+vqiadOmGDNmDGJiYgBYX19iqKoFycnJ0Ol0Jj9QAPDy8kJCQoKFWkXWzNAvyuozCQkJaNiwocnjdnZ2qF+/vsk+JR2j8Dmo7tDr9Zg+fTp69OiBe+65B4D8c1ar1XB3dzfZt2hfKq+flLZPWloasrOza+LlkAWcPHkSzs7O0Gg0mDJlCjZu3Ig2bdqwH1GlrFu3DkePHsWiRYuKPca+RJXRrVs3hIeHY8uWLVixYgWio6Px4IMPIj093er6kl1lXxwREVmnqVOn4tSpU/j7778t3RSyUa1atUJUVBRSU1Px008/Yfz48di9e7elm0U25Pr163jppZcQGRkJe3t7SzeHbNygQYOM37dv3x7dunVD48aN8b///Q8ODg4WbFlxrFTVggYNGkCpVBYbjSQxMRHe3t4WahVZM0O/KKvPeHt7IykpyeTx/Px83L5922Sfko5R+BxUN0ybNg2///47du7ciUaNGhm3e3t7Iy8vDykpKSb7F+1L5fWT0vZxdXW1uv/YqOrUajWaN2+Ozp07Y9GiRejQoQM++eQT9iOqsCNHjiApKQn33nsv7OzsYGdnh927d+PTTz+FnZ0dvLy82Jeoytzd3dGyZUtcunTJ6n4vMVTVArVajc6dO2P79u3GbXq9Htu3b0dwcLAFW0bWqkmTJvD29jbpM2lpaTh48KCxzwQHByMlJQVHjhwx7rNjxw7o9Xp069bNuM+ePXug1WqN+0RGRqJVq1aoV69eLb0aqklCCEybNg0bN27Ejh070KRJE5PHO3fuDJVKZdKXzp8/j5iYGJO+dPLkSZOQHhkZCVdXV7Rp08a4T+FjGPbh77C6Ta/XIzc3l/2IKqxv3744efIkoqKijEuXLl0wZswY4/fsS1RVGRkZuHz5Mnx8fKzv91KlhrWgKlu3bp3QaDQiPDxcnDlzRkyePFm4u7ubjEZCd5f09HRx7NgxcezYMQFALFmyRBw7dkxcu3ZNCCEPqe7u7i5++eUXceLECTFs2LASh1Tv1KmTOHjwoPj7779FixYtTIZUT0lJEV5eXmLs2LHi1KlTYt26dcLR0ZFDqtchzz33nHBzcxO7du0yGXI2KyvLuM+UKVNEQECA2LFjh/j3339FcHCwCA4ONj5uGHK2f//+IioqSmzZskV4enqWOOTsq6++Ks6ePSuWL1/O4YvrmNdff13s3r1bREdHixMnTojXX39dSJIk/vrrLyEE+xFVXeHR/4RgX6KKe+WVV8SuXbtEdHS02LdvnwgJCRENGjQQSUlJQgjr6ksMVbVo2bJlIiAgQKjVanHfffeJAwcOWLpJZEE7d+4UAIot48ePF0LIw6q//fbbwsvLS2g0GtG3b19x/vx5k2PcunVLjB49Wjg7OwtXV1cxYcIEkZ6ebrLP8ePHxQMPPCA0Go3w8/MT7733Xm29RKoFJfUhAGLlypXGfbKzs8Xzzz8v6tWrJxwdHcWjjz4q4uPjTY5z9epVMWjQIOHg4CAaNGggXnnlFaHVak322blzp+jYsaNQq9WiadOmJucg2/f000+Lxo0bC7VaLTw9PUXfvn2NgUoI9iOquqKhin2JKmrUqFHCx8dHqNVq4efnJ0aNGiUuXbpkfNya+pIkhBCVq20RERERERGRAe+pIiIiIiIiqgaGKiIiIiIiompgqCIiIiIiIqoGhioiIiIiIqJqYKgiIiIiIiKqBoYqIiIiIiKiamCoIiIiIiIiqgaGKiIiIiIiompgqCIiortaYGAgli5daulmEBGRDWOoIiKiWhMaGorhw4cDAHr37o3p06fX2rnDw8Ph7u5ebPvhw4cxefLkWmsHERHVPXaWbgAREVF15OXlQa1WV/n5np6eZmwNERHdjVipIiKiWhcaGordu3fjk08+gSRJkCQJV69eBQCcOnUKgwYNgrOzM7y8vDB27FgkJycbn9u7d29MmzYN06dPR4MGDTBgwAAAwJIlS9CuXTs4OTnB398fzz//PDIyMgAAu3btwoQJE5Cammo837x58wAUv/wvJiYGw4YNg7OzM1xdXfH4448jMTHR+Pi8efPQsWNHrFmzBoGBgXBzc8MTTzyB9PR04z4//fQT2rVrBwcHB3h4eCAkJASZmZk19G4SEZGlMVQREVGt++STTxAcHIxJkyYhPj4e8fHx8Pf3R0pKCvr06YNOnTrh33//xZYtW5CYmIjHH3/c5PmrVq2CWq3Gvn378MUXXwAAFAoFPv30U5w+fRqrVq3Cjh078NprrwEAunfvjqVLl8LV1dV4vpkzZxZrl16vx7Bhw3D79m3s3r0bkZGRuHLlCkaNGmWy3+XLl7Fp0yb8/vvv+P3337F792689957AID4+HiMHj0aTz/9NM6ePYtdu3ZhxIgREELUxFtJRERWgJf/ERFRrXNzc4NarYajoyO8vb2N2z/77DN06tQJCxcuNG777rvv4O/vjwsXLqBly5YAgBYtWuD99983OWbh+7MCAwOxYMECTJkyBZ9//jnUajXc3NwgSZLJ+Yravn07Tp48iejoaPj7+wMAVq9ejbZt2+Lw4cPo2rUrADl8hYeHw8XFBQAwduxYbN++He+++y7i4+ORn5+PESNGoHHjxgCAdu3aVePdIiIia8dKFRERWY3jx49j586dcHZ2Ni5BQUEA5OqQQefOnYs9d9u2bejbty/8/Pzg4uKCsWPH4tatW8jKyqrw+c+ePQt/f39joAKANm3awN3dHWfPnjVuCwwMNAYqAPDx8UFSUhIAoEOHDujbty/atWuHxx57DF9//TXu3LlT8TeBiIhsDkMVERFZjYyMDAwdOhRRUVEmy8WLF9GzZ0/jfk5OTibPu3r1Kh5++GG0b98eP//8M44cOYLly5cDkAeyMDeVSmWyLkkS9Ho9AECpVCIyMhKbN29GmzZtsGzZMrRq1QrR0dFmbwcREVkHhioiIrIItVoNnU5nsu3ee+/F6dOnERgYiObNm5ssRYNUYUeOHIFer8dHH32E+++/Hy1btkRcXFy55yuqdevWuH79Oq5fv27cdubMGaSkpKBNmzYVfm2SJKFHjx4ICwvDsWPHoFarsXHjxgo/n4iIbAtDFRERWURgYCAOHjyIq1evIjk5GXq9HlOnTsXt27cxevRoHD58GJcvX8bWrVsxYcKEMgNR8+bNodVqsWzZMly5cgVr1qwxDmBR+HwZGRnYvn07kpOTS7wsMCQkBO3atcOYMWNw9OhRHDp0COPGjUOvXr3QpUuXCr2ugwcPYuHChfj3338RExODDRs24ObNm2jdunXl3iAiIrIZDFVERGQRM2fOhFKpRJs2beDp6YmYmBj4+vpi37590Ol06N+/P9q1a4fp06fD3d0dCkXp/2V16NABS5YsweLFi3HPPfdg7dq1WLRokck+3bt3x5QpUzBq1Ch4enoWG+gCkCtMv/zyC+rVq4eePXsiJCQETZs2xY8//ljh1+Xq6oo9e/Zg8ODBaNmyJWbPno2PPvoIgwYNqvibQ0RENkUSHOOViIiIiIioylipIiIiIiIiqgaGKiIiIiIiompgqCIiIiIiIqoGhioiIiIiIqJqYKgiIiIiIiKqBoYqIiIiIiKiamCoIiIiIiIiqgaGKiIiIiIiompgqCIiIiIiIqoGhioiIiIiIqJqYKgiIiIiIiKqhv8HABCVDIdZlkcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Train model\n",
    "\n",
    "iterations = np.empty(int(max_iters/eval_iters))\n",
    "iterations[:] = np.nan\n",
    "train_loss = np.empty(int(max_iters/eval_iters))\n",
    "train_loss[:] = np.nan\n",
    "val_loss = np.empty(int(max_iters/eval_iters))\n",
    "val_loss[:] = np.nan\n",
    "\n",
    "for i in range(max_iters):\n",
    "    train_one_batch(model, optimizer)\n",
    "    if i % eval_iters == 0:\n",
    "        out = estimate_loss()\n",
    "        iterations[int(i/eval_iters)] = i\n",
    "        train_loss[int(i/eval_iters)] = torch.exp(out['train'])\n",
    "        val_loss[int(i/eval_iters)] = torch.exp(out['val'])\n",
    "        print(f\"First {i} batches: training perplexity: {torch.exp(out['train']):.3f}, validation perplexity: {torch.exp(out['val']):.3f}\")\n",
    "        \n",
    "# Visualize results\n",
    "plt.figure(figsize=(10, 5))\n",
    "plt.plot(iterations, train_loss, label='Training Perplexity', color='blue')\n",
    "plt.plot(iterations, val_loss, label='Validation Perplexity', color='red')\n",
    "plt.title('Training and Validation Perplexity')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Perplexity')\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "2b13e014-85ad-4ead-a1db-fc64c4927929",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello, my name is my lament tell ten thing brother to-nigw.\n",
      "\n",
      "HENRY VI:\n",
      "tha doth adment, we thank the lands!\n",
      "And colit\n"
     ]
    }
   ],
   "source": [
    "# Output some text\n",
    "\n",
    "model.eval()\n",
    "prompt = \"Hello, my name is\"\n",
    "max_new_tokens = 100\n",
    "ix = torch.tensor([encode(prompt)], dtype=torch.long)\n",
    "X =  ix.to('cuda')\n",
    "output = model.generate(X, max_new_tokens)\n",
    "[output] = output.tolist()\n",
    "response = decode(output)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0db71c83-e0ec-4f42-ae2f-4d35f447f79f",
   "metadata": {},
   "source": [
    "Simultaneously increase the number of heads and the number of decoder blocks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "5f3fed51-1c6b-4214-8dac-745353f0e765",
   "metadata": {},
   "outputs": [],
   "source": [
    "n_head = 8\n",
    "n_layer = 8\n",
    "\n",
    "model = BigramLanguageModel()\n",
    "model.to('cuda')\n",
    "optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "5e62e07c-8ccd-45f5-b943-dfe4c6f6af32",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 0 batches: training perplexity: 41.698, validation perplexity: 42.393\n",
      "First 200 batches: training perplexity: 11.059, validation perplexity: 11.008\n",
      "First 400 batches: training perplexity: 8.993, validation perplexity: 9.227\n",
      "First 600 batches: training perplexity: 7.793, validation perplexity: 8.375\n",
      "First 800 batches: training perplexity: 7.075, validation perplexity: 7.552\n",
      "First 1000 batches: training perplexity: 6.652, validation perplexity: 7.376\n",
      "First 1200 batches: training perplexity: 6.206, validation perplexity: 6.940\n",
      "First 1400 batches: training perplexity: 5.921, validation perplexity: 6.693\n",
      "First 1600 batches: training perplexity: 5.725, validation perplexity: 6.699\n",
      "First 1800 batches: training perplexity: 5.573, validation perplexity: 6.456\n",
      "First 2000 batches: training perplexity: 5.485, validation perplexity: 6.215\n",
      "First 2200 batches: training perplexity: 5.284, validation perplexity: 6.121\n",
      "First 2400 batches: training perplexity: 5.218, validation perplexity: 6.026\n",
      "First 2600 batches: training perplexity: 5.132, validation perplexity: 6.000\n",
      "First 2800 batches: training perplexity: 4.973, validation perplexity: 5.733\n",
      "First 3000 batches: training perplexity: 4.947, validation perplexity: 5.858\n",
      "First 3200 batches: training perplexity: 4.962, validation perplexity: 5.811\n",
      "First 3400 batches: training perplexity: 4.867, validation perplexity: 5.711\n",
      "First 3600 batches: training perplexity: 4.837, validation perplexity: 5.675\n",
      "First 3800 batches: training perplexity: 4.710, validation perplexity: 5.604\n",
      "First 4000 batches: training perplexity: 4.697, validation perplexity: 5.646\n",
      "First 4200 batches: training perplexity: 4.708, validation perplexity: 5.536\n",
      "First 4400 batches: training perplexity: 4.653, validation perplexity: 5.512\n",
      "First 4600 batches: training perplexity: 4.642, validation perplexity: 5.557\n",
      "First 4800 batches: training perplexity: 4.596, validation perplexity: 5.482\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1UAAAHWCAYAAACfRKOZAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/P9b71AAAACXBIWXMAAA9hAAAPYQGoP6dpAAB/T0lEQVR4nO3dd3hTZf8G8PskTdKRpi2lezDKRvYsKCAUCmgFREVAoIgivoAi4osDgfKi4EIUETdFpeIPZKjMshUBEShTEUqR0UUp3StNzu+PY9Kmg660Jy3357rO1Zz9TXhaevc55zmCKIoiiIiIiIiIqFoUchdARERERERUnzFUERERERER1QBDFRERERERUQ0wVBEREREREdUAQxUREREREVENMFQRERERERHVAEMVERERERFRDTBUERERERER1QBDFRERERERUQ0wVBERVVJ4eDiaNm1arX0XLlwIQRCsW5CNuXLlCgRBQGRkZJ2fWxAELFy40DwfGRkJQRBw5cqVCvdt2rQpwsPDrVpPTdpKQ1Ty38fa7obvLyKybQxVRFTvCYJQqWn//v1yl3rXe+655yAIAi5dulTuNq+99hoEQcDp06frsLKqi4+Px8KFCxETEyN3KWamYGualEolAgMDMWrUKJuqsy68+eab2Lx5s9xlENFdgqGKiOq9b775xmIaPHhwmcvbtm1bo/N8/vnnuHDhQrX2nTdvHnJzc2t0/oZg/PjxAICoqKhyt/nuu+/QoUMHdOzYsdrnmTBhAnJzc9GkSZNqH6Mi8fHxiIiIKDOs1KStWMPYsWPxzTff4KuvvsK4ceOwd+9e9O7du8EGq7K+vxiqiKgu2cldABFRTT3xxBMW80eOHEF0dHSp5SXl5OTA0dGx0udRqVTVqg8A7OzsYGfHH7m9evVCixYt8N1332H+/Pml1h8+fBhxcXFYunRpjc6jVCqhVCprdIyaqElbsYauXbtatP++ffvioYcewqpVq/Dpp5/W6NjZ2dlwcnKqaYlWxe8vIpIbe6qI6K4wYMAA3HPPPTh+/Dj69esHR0dHvPrqqwCALVu24IEHHoCvry80Gg2CgoLwv//9DwaDweIYJe+TMV1q9e677+Kzzz5DUFAQNBoNevTogWPHjlnsW9Y9H4IgYMaMGdi8eTPuueceaDQatG/fHjt27ChV//79+9G9e3fY29sjKCgIn376aaXvI/nll1/w6KOPIjAwEBqNBgEBAXjhhRdK/WU/PDwcWq0WN27cwMiRI6HVauHh4YE5c+aU+izS0tIQHh4OFxcXuLq6YtKkSUhLS6uwFkDqrfrrr79w4sSJUuuioqIgCALGjh2LgoICzJ8/H926dYOLiwucnJxw3333Yd++fRWeo6x7qkRRxOLFi+Hv7w9HR0fcf//9OHfuXKl9U1NTMWfOHHTo0AFarRY6nQ7Dhg3DqVOnzNvs378fPXr0AABMnjzZfLmd6X6ysu6pys7OxosvvoiAgABoNBq0bt0a7777LkRRtNiuKu2isgYOHAgAiIuLMy87evQohg4dChcXFzg6OqJ///44dOiQxX6mNnb+/HmMGzcObm5uuPfee83vUavV4vLlywgNDYWTkxN8fX2xaNGiUu+pLDdu3MCTTz4JLy8v83v86quvzOtzc3PRpk0btGnTxqKtpqamwsfHB3369DG3y5LfC4IgIDs7G2vWrDH/24SHh2Pfvn0QBAGbNm0qVY+p7R0+fLgyHykRkQX+WYeI7hq3bt3CsGHD8Pjjj+OJJ56Al5cXAOkXcK1Wi9mzZ0Or1WLv3r2YP38+MjIy8M4771R43KioKGRmZuKZZ56BIAh4++238fDDD+Py5csV9lj8+uuv2LhxI/7zn//A2dkZH374IUaPHo2rV6/C3d0dAHDy5EkMHToUPj4+iIiIgMFgwKJFi+Dh4VGp971+/Xrk5OTg2Wefhbu7O37//XesWLEC169fx/r16y22NRgMCA0NRa9evfDuu+9i9+7deO+99xAUFIRnn30WgBRORowYgV9//RXTpk1D27ZtsWnTJkyaNKlS9YwfPx4RERGIiopC165dLc79f//3f7jvvvsQGBiIlJQUfPHFFxg7diyefvppZGZm4ssvv0RoaCh+//13dO7cuVLnM5k/fz4WL16M4cOHY/jw4Thx4gSGDBmCgoICi+0uX76MzZs349FHH0WzZs2QlJSETz/9FP3798f58+fh6+uLtm3bYtGiRZg/fz6mTp2K++67DwDQp0+fMs8tiiIeeugh7Nu3D1OmTEHnzp2xc+dOvPTSS7hx4wbef/99i+0r0y6qIjY2FgDM++7duxfDhg1Dt27dsGDBAigUCqxevRoDBw7EL7/8gp49e1rs/+ijj6Jly5Z48803LQKTwWDA0KFD0bt3b7z99tvYsWMHFixYgMLCQixatKjcepKSktC7d29zgPTw8MD27dsxZcoUZGRkYNasWXBwcMCaNWvQt29fvPbaa1i2bBkAYPr06UhPT0dkZGS5vZHffPMNnnrqKfTs2RNTp04FAAQFBaF3794ICAjA2rVrMWrUKIt91q5di6CgIAQHB1fx0yUiAiASETUw06dPF0v+eOvfv78IQPzkk09KbZ+Tk1Nq2TPPPCM6OjqKeXl55mWTJk0SmzRpYp6Pi4sTAYju7u5iamqqefmWLVtEAOJPP/1kXrZgwYJSNQEQ1Wq1eOnSJfOyU6dOiQDEFStWmJeFhYWJjo6O4o0bN8zLLl68KNrZ2ZU6ZlnKen9LliwRBUEQ//nnH4v3B0BctGiRxbZdunQRu3XrZp7fvHmzCEB8++23zcsKCwvF++67TwQgrl69usKaevToIfr7+4sGg8G8bMeOHSIA8dNPPzUfMz8/32K/27dvi15eXuKTTz5psRyAuGDBAvP86tWrRQBiXFycKIqimJycLKrVavGBBx4QjUajebtXX31VBCBOmjTJvCwvL8+iLlGU/q01Go3FZ3Ps2LFy32/JtmL6zBYvXmyx3SOPPCIKgmDRBirbLspiapMRERHizZs3xcTERHH//v1ily5dRADiDz/8IBqNRrFly5ZiaGioxWeRk5MjNmvWTBw8eLB5mandjh07tsz3CECcOXOmeZnRaBQfeOABUa1Wizdv3rR4T8X/faZMmSL6+PiIKSkpFsd8/PHHRRcXF4s2+8orr4gKhUI8ePCguH79ehGAuHz5cov9yvr+cnJysvh3LX48jUYjpqWlmZclJyeLdnZ2FjUSEVUFL/8joruGRqPB5MmTSy13cHAwv87MzERKSgruu+8+5OTk4K+//qrwuGPGjIGbm5t53tRrcfny5Qr3DQkJQVBQkHm+Y8eO0Ol05n0NBgN2796NkSNHwtfX17xdixYtMGzYsAqPD1i+v+zsbKSkpKBPnz4QRREnT54stf20adMs5u+77z6L97Jt2zbY2dmZe64A6R6mmTNnVqoeQLoP7vr16zh48KB5WVRUFNRqNR599FHzMdVqNQDAaDQiNTUVhYWF6N69e5mXDt7J7t27UVBQgJkzZ1pcJjZr1qxS22o0GigU0n+PBoMBt27dglarRevWrat8XpNt27ZBqVTiueees1j+4osvQhRFbN++3WJ5Re2iIgsWLICHhwe8vb0xYMAAxMbG4q233sLDDz+MmJgYXLx4EePGjcOtW7eQkpKClJQUZGdnY9CgQTh48CCMRqPF8Uq2ieJmzJhhfm3qeSooKMDu3bvL3F4URfzwww8ICwuDKIrm86ekpCA0NBTp6ekWn/PChQvRvn17TJo0Cf/5z3/Qv3//Up9jVUycOBH5+fnYsGGDedn333+PwsLCCu/DJCIqDy//I6K7hp+fn/mX9OLOnTuHefPmYe/evcjIyLBYl56eXuFxAwMDLeZNAev27dtV3te0v2nf5ORk5ObmokWLFqW2K2tZWa5evYr58+fjxx9/LFVTyfdnb29f6rLC4vUAwD///AMfHx9otVqL7Vq3bl2pegDg8ccfx+zZsxEVFYUBAwYgLy8PmzZtwrBhwywC6po1a/Dee+/hr7/+gl6vNy9v1qxZpc9lqhkAWrZsabHcw8PD4nyAFOA++OADfPzxx4iLi7O4n6w6l96Zzu/r6wtnZ2eL5aYRKU31mVTULioydepUPProo1AoFHB1dUX79u2h0WgAABcvXgSAO16umZ6ebvG5lPd5KxQKNG/e3GJZq1atAKDcZ4TdvHkTaWlp+Oyzz/DZZ5+VuU1ycrL5tVqtxldffYUePXrA3t4eq1evrtEzqdq0aYMePXpg7dq1mDJlCgDp0r/evXtX+nuKiKgkhioiumsU77ExSUtLQ//+/aHT6bBo0SIEBQXB3t4eJ06cwNy5c0v9xb4s5d3XIVbiZv2a7FsZBoMBgwcPRmpqKubOnYs2bdrAyckJN27cQHh4eKn3V1cj5nl6emLw4MH44YcfsHLlSvz000/IzMw0D7kOAN9++y3Cw8MxcuRIvPTSS/D09IRSqcSSJUvM9wjVhjfffBOvv/46nnzySfzvf/9Do0aNoFAoMGvWrEq1B2uoabto2bIlQkJCylxneg/vvPNOufellQzMZX3vVJfp/E888US5wa7kcPo7d+4EAOTl5eHixYtVDtUlTZw4Ec8//zyuX7+O/Px8HDlyBB999FGNjklEdzeGKiK6q+3fvx+3bt3Cxo0b0a9fP/Py4qOkycnT0xP29vZlPiz3Tg/QNTlz5gz+/vtvrFmzBhMnTjQvj46OrnZNTZo0wZ49e5CVlWXxy3dVn8s0fvx47NixA9u3b0dUVBR0Oh3CwsLM6zds2IDmzZtj48aNFj0TCxYsqFbNgNRLU7xn5ebNm6V6fzZs2ID7778fX375pcXytLQ0NG7c2Dxfld6SJk2aYPfu3cjMzLTorTJdXlqbz9MqyXRZoU6nKzd4VZbRaMTly5fNvVMA8PfffwNAqdEPTTw8PODs7AyDwVCp858+fRqLFi3C5MmTERMTg6eeegpnzpyBi4vLHfe707+Pqaf0u+++Q25uLlQqFcaMGVNhLURE5eE9VUR0VzP1CBTvASgoKMDHH38sV0kWlEolQkJCsHnzZsTHx5uXX7p0qdR9OOXtD1i+P1EU8cEHH1S7puHDh6OwsBCrVq0yLzMYDFixYkWVjjNy5Eg4Ojri448/xvbt2/Hwww/D3t7+jrUfPXq0WkNeh4SEQKVSYcWKFRbHW758ealtlUplqR6h9evX48aNGxbLTM9qqsxQ8sOHD4fBYCjVG/L+++9DEIRK3x9nDd26dUNQUBDeffddZGVllVp/8+bNKh2v+HsSRREfffQRVCoVBg0aVOb2SqUSo0ePxg8//ICzZ8/e8fx6vR7h4eHw9fXFBx98gMjISCQlJeGFF16osC4nJ6dy/20aN26MYcOG4dtvv8XatWsxdOhQi8BMRFRV7Kkiortanz594ObmhkmTJuG5556DIAj45ptvrHb5nTUsXLgQu3btQt++ffHss8+afzm/5557EBMTc8d927Rpg6CgIMyZMwc3btyATqfDDz/8UOl7c8oSFhaGvn374uWXX8aVK1fQrl07bNy4sVL3nxWn1WoxcuRIREVFAYDFpX8A8OCDD2Ljxo0YNWoUHnjgAcTFxeGTTz5Bu3btygwDd2J63taSJUvw4IMPYvjw4Th58iS2b99e6pfpBx980Nwz0qdPH5w5cwZr164tde9QUFAQXF1d8cknn8DZ2RlOTk7o1atXmZemhYWF4f7778drr72GK1euoFOnTti1axe2bNmCWbNmWQxKUdsUCgW++OILDBs2DO3bt8fkyZPh5+eHGzduYN++fdDpdPjpp58qdSx7e3vs2LEDkyZNQq9evbB9+3Zs3boVr7766h2H/F+6dCn27duHXr164emnn0a7du2QmpqKEydOYPfu3UhNTQUALF68GDExMdizZw+cnZ3RsWNHzJ8/H/PmzcMjjzyC4cOHl3uObt26Yffu3Vi2bBl8fX3RrFkz9OrVy7x+4sSJeOSRRwAA//vf/yr1fomIysOeKiK6q7m7u+Pnn3+Gj48P5s2bh3fffReDBw/G22+/LXdpZt26dcP27dvh5uaG119/HV9++SUWLVqEQYMGWfTslEWlUuGnn35C586dsWTJEkRERKBly5b4+uuvq12PQqHAjz/+iPHjx+Pbb7/Fa6+9Bj8/P6xZs6bKxzIFKR8fH/MDak3Cw8Px5ptv4tSpU3juueewc+dOfPvtt+jevXu16l68eDEiIiJw8uRJvPTSS4iNjcWuXbvMPU4mr776Kl588UXs3LkTzz//PE6cOIGtW7ciICDAYjuVSoU1a9ZAqVRi2rRpGDt2LA4cOFDmuU2f2axZs/Dzzz9j1qxZOH/+PN555x3z85fq0oABA3D48GF0794dH330EWbOnInIyEh4e3tXqhfIRKlUYseOHUhMTMRLL72EY8eOYcGCBRWGFC8vL/z++++YPHkyNm7ciBkzZuCDDz5Aamoq3nrrLQDAiRMn8Oabb2LGjBm4//77zfu+/PLL6NGjB55++uk79hIuW7YM3bp1w7x58zB27FiLnlVACrpubm5wcXHBQw89VOn3TERUFkG0pT/HEhFRpY0cORLnzp0zj+ZGVJfCw8OxYcOGKvca2orCwkL4+voiLCys1P1zRERVxZ4qIqJ6IDc312L+4sWL2LZtGwYMGCBPQUT13ObNm3Hz5k2LAVyIiKqL91QREdUDzZs3R3h4OJo3b45//vkHq1atglqtxn//+1+5SyOqV44ePYrTp0/jf//7H7p06YL+/fvLXRIRNQAMVURE9cDQoUPx3XffITExERqNBsHBwXjzzTdLPcyWiO5s1apV+Pbbb9G5c2dERkbKXQ4RNRC8p4qIiIiIiKgGeE8VERERERFRDTBUERERERER1UCDv6fKaDQiPj4ezs7OEARB7nKIiIiIiEgmoigiMzMTvr6+UCis17/U4ENVfHx8qQc2EhERERHR3evatWvw9/e32vEafKhydnYGIH1wOp1O1lr0ej127dqFIUOGQKVSyVoL1W9sS2QtbEtkDWxHZC1sS2Qt5bWljIwMBAQEmDOCtTT4UGW65E+n09lEqHJ0dIROp+MPCqoRtiWyFrYlsga2I7IWtiWylorakrVvC+JAFURERERERDXAUEVERERERFQDDFVEREREREQ10ODvqSIiIiK6WxkMBuj1ernLqDS9Xg87Ozvk5eXBYDDIXQ7VQ0qlEnZ2dR9xGKqIiIiIGqCsrCxcv34doijKXUqliaIIb29vXLt2jc8XpWpzdHSEh4dHnZ6ToYqIiIiogTEYDLh+/br5l8v6ElCMRiOysrKg1Wqt+mBWujuIooiCggLcvHkTV69erdNzM1QRERERNTB6vR6iKMLDwwMODg5yl1NpRqMRBQUFsLe3Z6iianFwcIBKpcKVK1egVCrr7LxsrUREREQNVH3poSKyJlMgr8v2z1BFRERERERUAwxVRERERERENcBQRUREREQNVtOmTbF8+fJKb79//34IgoC0tLRaq0lukZGRcHV1teoxBUHA5s2brXrM+oShioiIiIhkJwgClEol3NzcoFQqIQiCxbRw4cJqHffYsWOYOnVqpbfv06cPEhIS4OLiUq3zVZYpvJkmLy8vjB49GpcvX67V89aWhIQEDBs2DABw5coVCIKAmJgYeYuqQxz9j4iIiIhkl5CQAKPRiMzMTGzfvh0LFizAhQsXzOu1Wq35tSiKMBgMlXrIa1WfV6RWq+Ht7V2lfWriwoULcHZ2xsWLFzF16lSEhYXh9OnT1Rq5Tq/XQ6VS1UKVFavLz8wWsaeqrogilCEhCHnmGSA+Xu5qiIiI6C4iikB2tjxTZZ897O3tDW9vb3h5eUGn00EQBPOyv/76C87Ozti+fTu6desGjUaDX3/9FbGxsRgxYgS8vLyg1WrRo0cP7N692+K4JS//EwQBX3zxBUaNGgVHR0e0bNkSP/74o3l9ycv/TJfK7dy5E23btoVWq8XQoUORkJBg3qewsBDPPfccXF1d4e7ujrlz52LSpEkYOXJkhe/b09MTPj4+6NevH+bPn4/z58/j0qVLAIAtW7aga9eusLe3R/PmzREREYHCwkKL97Jq1So89NBDcHJywhtvvGGuf+vWrejYsSPs7e3Ru3dvnD179o513OlcixYtgq+vL27dumXe/oEHHsD9998Po9ForsV0+V+zZs0AAF26dIEgCBgwYAAOHjwIlUqFxMREi/POmjUL9913X4Wfk61jqKorggAhNhZOSUkQin0TEhEREdW2nBxAq5Vnysmx3vt4+eWXsXTpUvz555/o2LEjsrKyMHz4cOzZswcnT57E0KFDERYWVuGDXyMiIvDYY4/h9OnTGD58OMaPH4/U1NQ7fH45ePfdd/HNN9/g4MGDuHr1KubMmWNe/9Zbb2Ht2rVYvXo1Dh06hIyMjGrdX2R6plhBQQF++eUXTJw4Ec8//zzOnz+PTz/9FJGRkXjjjTcs9lm4cCFGjRqFM2fO4MknnzQvf+mll/Dee+/h2LFj8PDwQFhYGPR6fZnnrehcr732Gpo2bYqnnnoKALBy5Ur89ttvWLNmTZnPE/v9998BALt370ZCQgI2btyIfv36oXnz5vjmm2/M2+n1eqxdu9ai7npLbODS09NFAGJ6errcpYiG7t1FERD1P/wgdylUzxUUFIibN28WCwoK5C6F6jm2JbIGtiPbk5ubK54/f17Mzc0VRVEUs7JEUeozqvspK6vydRsMBvH27dvil19+Kbq4uJiX79u3TwQgbt68ucJjtG/fXlyxYoV5vkmTJuL7779vngcgzps3zzyflZUlAhC3b99uca7bt2+LoiiKq1evFgGIly5dMu+zcuVK0cvLyzzv5eUlvvPOO+b5wsJCMTAwUBwxYkS5dZY8T3x8vNinTx/Rz89PzM/PFwcNGiS++eabFvt88803oo+Pj8V7mTVrVpnHXbdunXnZrVu3RAcHB/H77783v6fin29lzhUbGys6OzuLc+fOFR0cHMS1a9dabA9A3LRpkyiKohgXFycCEE+ePGmxzVtvvSW2bdvWPP/DDz+IWq1WzKpKI6mE3Nxc8dy5c+LPP/9c6udSbWUD3lNVl/691pQ9VURERFSXHB2BrCz5zm0t3bt3t5jPysrCwoULsXXrViQkJKCwsBC5ubkV9lR17NjR/NrJyQk6nQ7Jycnlbu/o6IigoCDzvI+Pj3n79PR0JCUloWfPnub1SqUS3bp1M18adyf+/v4QRRE5OTno1KkTfvjhB6jVapw6dQqHDh2y6JkyGAzIy8tDTk4OHP/9YEt+JibBwcHm140aNULr1q3x559/lrltZc7VvHlzvPvuu3jmmWcwZswYjBs3rsL3VlJ4eDjmzZuHI0eOoHfv3oiMjMRjjz0GJyenKh/L1jBU1SHR11d6wXuqiIiIqA4JAtAAfm8t9cv3nDlzEB0djXfffRctWrSAg4MDHnnkERQUFNzxOCUHcxAE4Y4BqKztxcreLFaBX375BTqdDp6ennB2djYvz8rKQkREBB5++OFS+9jb25tfWyOQVPZcBw8ehFKpxJUrV1BYWFipgUKK8/T0RFhYGFavXo1mzZph+/bt2L9/f03LtwkMVXXJxwcAe6qIiIiIrOHQoUMIDw/HqFGjAEjh4MqVK3Vag4uLC7y8vHDs2DH069cPgNTLc+LECXTu3LnC/Zs1a1bmM6O6du2KCxcuoEWLFtWq68iRIwgMDAQA3L59G3///Tfatm1b5raVOdf333+PjRs3Yv/+/Xjsscfwv//9DxEREWVuq1arAUifQ0lPPfUUxo4dC39/fwQFBaFv375VfWs2iaGqjogicDzBFz0BGK4lcIQQIiIiohpq2bIlNm7ciLCwMAiCgNdff71Sl9xZ28yZM7FkyRK0aNECbdq0wYoVK3D79m0IglDtY86fPx8PPvggAgMD8cgjj0ChUODUqVM4e/YsFi9eXOH+ixYtgru7O7y8vPDaa6+hcePG5Y5GWNG5rl+/jmeffRZvvfUW7r33XqxevRoPPvgghg0bht69e5c6nqenJxwcHLBjxw74+/vD3t7e/Nyv0NBQ6HQ6LF68GIsWLar252Nr+Lt9HREE4J21/gCAwmvsqSIiIiKqqWXLlsHNzQ19+vRBWFgYQkND0bVr1zqvY+7cuRg7diwmTpyI4OBgaLVahIaGWlw6V1WhoaH4+eefsWvXLvTo0QO9e/fG+++/jyZNmlRq/6VLl+L5559Ht27dkJiYiJ9++sncg1SVc4miiPDwcPTs2RMzZswwb//ss8/iiSeeQFYZN+vZ2dnhww8/xKeffgpfX1+MGDHCvE6hUCA8PBwGgwETJ06sxidjmwTRWheE2qiMjAy4uLggPT0dOp1O1lpGNY/BprguyHf1hOZ2kqy1UP2m1+uxbds2DB8+XLaH/FHDwLZE1sB2ZHvy8vIQFxeHZs2a1egX+7pmNBqRkZEBnU5X5lDd9YXRaETbtm3Nl8nVpf379+P+++/H7du3y7ys0BZMmTIFN2/etHg+mDXl5eXh8uXLiIuLw5AhQyx+LtVWNuDlf3VIGeADxAGatGRArwf4Hw8RERFRvffPP/9g165d6N+/P/Lz8/HRRx8hLi6uWiPkNWTp6ek4c+YMoqKiai1QyaX+/gmgHnIKdIfelGNLPE2aiIiIiOonhUKByMhI9OjRA3379sWZM2ewe/fucgeGuFuNGDECQ4YMwbRp0zB48GC5y7Eq9lTVIR8/AQnwQSCuScOqBwTIXRIRERER1VBAQAAOHTokdxkAgAEDBlhtuHdrayjDp5eFPVV1yNcXiAefVUVERERE1JAwVNUhX1+RoYqIiIiIqIFhqKpDFj1VfAAwEREREVGDwFBVh4r3VIk32FNFRERERNQQMFTVIS8vIFHwAQAU/MNQRURERETUEDBU1SE7OyDDqTEAwHiNoYqIiIiIqCFgqKpjOa6NAADKJIYqIiIiImsbMGAAZs2aZZ5v2rQpli9ffsd9BEHA5s2ba3xuax3HVi1cuBCdO3e22vGuXLkCQRAQExNjtWPKxWZC1dKlSyEIgsU3QV5eHqZPnw53d3dotVqMHj0aSUlJ8hVpBXoPVwCAOvMWkJ8vbzFERERENiIsLAzDhg0rc90vv/wCQRBw+vTpKh/32LFjmDp1ak3Ls1BeuEhISCj3PVhLZGQkBEGAIAhQKBTw9/fH5MmTkZycXKvnrQ0BAQFISEjAPffcA0B6jpUgCEhLS5O3sGqwiVB17NgxfPrpp+jYsaPF8hdeeAE//fQT1q9fjwMHDiA+Ph4PP/ywTFVah52nA/KhlmY4AiARERERAGDKlCnYvXs3bty4UWrd6tWr0b1791K/K1aGh4cHHB0drVFihby9vaHRaGr9PDqdDgkJCbh+/To+//xzbN++HRMmTKj28fR6vRWrqzylUglvb2/Y2dnJcn5rkj1UZWVlYfz48fj888/h5uZmXp6eno4vv/wSy5Ytw8CBA9GtWzesXr0av/32G44cOSJjxTXj1iifw6oTERFR3RJFIDtbnkkUK1Xigw8+CA8PD3z33XcWy7OysrB+/XpMmTIFt27dwtixY+Hn5wdHR0d06NCh1PYllbz87+LFi+jXrx/s7e3Rrl07REdHl9pn7ty5aNWqFRwdHdG8eXO8/vrr5uARGRmJiIgInDp1ytxjFBkZCaD05X9nzpzBwIED4eDgAHd3d0ydOhVZWVnm9eHh4Rg5ciTeffdd+Pj4wN3dHdOnT68w5AiCAG9vb/j6+mLYsGF47rnnsHv3buTm5gIAvvjiC7Rt2xb29vZo06YNPv74Y/O+pkvuvv/+e/Tv3x/29vZYu3YtIiMj4erqis2bN6Nly5awt7dHaGgorl27dsda7nSuJ598Eh07dkT+v1dnFRQUoEuXLpg4caJFLTExMbhy5Qruv/9+AICbmxsEQUB4eDi+/vpruLu7m49hMnLkyBoFSWuTPRZOnz4dDzzwAEJCQrB48WLz8uPHj0Ov1yMkJMS8rE2bNggMDMThw4fRu3fvMo+Xn59v8aFnZGQAkBK4XCncRK/Xw909D/HwRTNcQeHVqxC7d5e1JqqfTG1Z7jZN9R/bElkD25Ht0ev1EEURRqMRRqMRyM6GQqeTpRZjRgbg5FThdgqFAk888QSioqKwcOFC8/Lvv/8eBoMBY8aMQVZWFrp27YqXXnoJOp0O27Ztw4QJE9CsWTP07NnTvI/pvZecNxqNePjhh+Hl5YXDhw8jPT0ds2fPluo0fVYAtFotvvrqK/j6+uLMmTN45plnoNVq8dJLL+HRRx/FmTNnsHPnTuzatQsA4OLiYt7XdJzs7GyEhoaid+/eOHr0KJKTkzF16lRMnz4dq1evNte1b98+eHt7Y8+ePbh06RLGjh2Ljh074umnny778yx2HhN7e3sYjUYUFBRgw4YNmD9/Pj788EN06dIFJ0+exDPPPAMHBwdMmjTJvN/LL7+Md955B1999RXs7e2xc+dO5OTk4I033kBkZCTUajVmzJiBxx9/HL/88ou53uLnXrt27R3PtXz5cnTp0gVz587FsmXL8OqrryItLQ0ffvihxedtNBrh5+eH9evX49FHH8Wff/4JnU4HBwcHqNVqPPfcc9i8eTMeffRRAEBycjK2bt2KHTt2WHwOxT8jU60lfy7V1s8pWUPVunXrcOLECRw7dqzUusTERKjVari6ulos9/LyQmJiYrnHXLJkCSIiIkot37VrV511/d5Jo0ae5p6q87t3I87eXuaKqD4r669rRNXBtkTWwHZkO+zs7ODt7Y2srCwUFBQA2dlwlamWjIwMwGCo1LaPPfYY3nvvPezYsQP33nsvAODLL79EWFgYBEGAs7OzRdiYOHEitm7dirVr16JNmzYAgMLCQhQUFJj/sG40GpGXl4eMjAzs3bsXf/31F/7v//4PPj7SY25effVVPProo8jNzTXvM3PmTPM5+vfvj+nTp2PdunV45plnAAAqlQqCIJh/tyz+x3vTcdasWYPc3FysWLECTk5OCAwMxNKlSzF27Fi89tpr8PT0hF6vh4uLC9544w0olUr4+vpiyJAh2LlzJ8aMGVPmZ5SXlwdRFM21xsbGYtWqVejSpQtEUcSCBQuwaNEic8dESEgInn32WaxatQqjRo0y95Q988wzFp0XeXl50Ov1WLJkCdq3bw8AWLFiBXr16oV9+/ahW7duyM/Ph8FgMJ+7onMBwKpVq/Dggw9CrVbjgw8+wI8//mhuF6ZasrOzkZ2dDft/fy92cHCw+GxHjx6NL774AqGhoeY24e/vj65du5prKa6goAB5eXkASv9cysnJKfNzrSnZQtW1a9fw/PPPIzo62vwBWsMrr7xi/osDIP2DBQQEYMiQIdDJ9BcaE71ejytXjiAB0jdxezc3tB0+XNaaqH7S6/WIjo7G4MGDoVKp5C6H6jG2JbIGtiPbk5eXh2vXrkGr1Uq/Zzk7Sz1GMtA5OgKCUKltu3Xrhp49e+L777/H8OHDcenSJRw+fBiLFy+GTqeDwWDAkiVLsH79ety4cQMFBQXIz8+HTqcz/55nZ2cHtVptnlcoFLC3t4dOp8PVq1cREBCA1q1bm885aNAgANIv8qZ9vv/+e3z00UeIjY1FVlYWCgsLLc6h0WigVCrL/N3SdJwrV66gc+fO5vAGAIMHD4bRaER8fDxatGgBlUqFe+65x+IWmICAAJw9e7bc31vt7e2RkZEBf39/c2C899578dlnn0GpVCIuLg7PPfecxeBvhYWFcHFxgU6ng1arBQD07dvX4hz29vaws7PDgAEDoFBIdwh1794drq6uuHr1Ku6//36L952dnV3huQApaL344otYunQp/vvf/5qDEQBzLU5OTtDpdOYg5ezsbFHbf/7zH/Tq1QuZmZnw8/PD999/j8mTJ8PFxaXMzygvL8+cL0r+XCorhFmDbKHq+PHjSE5ORteuXc3LDAYDDh48iI8++gg7d+5EQUEB0tLSLHqrkpKS4O3tXe5xNRpNmTcIqlQqm/hB36hRHs6Z7qmKT7KJmqj+spV2TfUf2xJZA9uR7TAYDObR4Uy/IMPZWd6iKsFoNGLChAmYO3cuPv74Y6xZswZBQUG4//77IQgC3n77bXz44YdYvnw5OnToACcnJ8yaNQt6vb7ofQLm915yXvg33BVfZ3pt+qwOHz6MCRMmICIiAqGhoXBxccG6devw3nvvmbct6zjFj1fZcwmCALVaXWobo9FY5rFN652dnXHixAkoFAr4+PjAwcEBAMyjZH/++efo1auXxX5KpdKiPTg7O9+xtorek6nH507nAqR/099++w1KpRKxsbF3PGd5NXTr1g2dOnXCt99+iyFDhuDcuXPYunXrHT8jU60lfy7V1s8o2ULVoEGDcObMGYtlkydPRps2bTB37lwEBARApVJhz549GD16NADgwoULuHr1KoKDg+Uo2SqcnfW4aecDFAIFV+LhIHdBRERERDZk5MiReOWVVxAVFYWvv/4azz77rPkX5EOHDmHEiBF44oknAEi/sP/9999o165dpY7dtm1bXLt2DQkJCeYepJIDoP32229o0qQJXnvtNfOyf/75x2IbtVoNQwWXNLZt2xaRkZHIzs6G07/3lB06dAgKhcKip6w6FAoFWrRoUWq5l5cXfH19cfnyZYwfP77Kxy0sLMQff/xhvj/twoULSEtLQ9u2bat9rnfeeQd//fUXDhw4gNDQUKxevRqTJ08uc1u1Whohu6zP9qmnnsLy5ctx48YNhISEICAgoMrvrzbJFqqcnZ3NY9KbODk5wd3d3bx8ypQpmD17Nho1agSdToeZM2ciODi43EEq6gNBAAoa+wKJgHiDDwAmIiIiKk6r1eKxxx7DK6+8goyMDISHh5vXtWzZEhs2bMBvv/0GNzc3LFu2DElJSZUOVSEhIWjVqhUmTZqEd955BxkZGRbhyXSOq1evYt26dejRowe2bt2KTZs2WWzTtGlTxMXFISYmBv7+/nB2di51pdT48eOxYMECTJo0CQsXLsTNmzcxc+ZMTJgwAV5eXtX7cCohIiICzz33HFxcXDB06FDk5+fjjz/+wO3bty1ukSmLSqXCzJkz8eGHH8LOzg4zZsxA7969LQYBqcq5Tp48ifnz52PDhg3o27cvli1bhueffx79+/dH8+bNSx2vSZMmEAQBP//8M4YPHw4HBwfzJYLjxo3DnDlz8Pnnn+Prr7+u+QdlZbIPqX4n77//Ph588EGMHj0a/fr1g7e3NzZu3Ch3WTUm/vuXEbubHFKdiIiIqKQnn3wSt2/fRmhoKHx9fc3L582bh65duyI0NBQDBgyAt7c3Ro4cWenjKhQKbNq0Cbm5uejZsyeeeuopvPHGGxbbPPTQQ3jhhRcwY8YMdO7cGb/99htef/11i21Gjx6NoUOH4v777y9zGHgAcHR0xM6dO5GamooePXrgkUcewaBBg/DRRx9V7cOooqeeegpffPEFVq9ejQ4dOqB///6IjIxEs2bNKtzX0dERc+fOxbhx49C3b19otVp8//331TpXXl4ennjiCYSHhyMsLAwAMHXqVNx///2YMGFCmb1Rfn5+iIiIwMsvvwwvLy/MmDHDvM7FxQWjR4+GVqut0r95XRFEsZIPD6inMjIy4OLigvT0dJsYqGLbtm3Y+NV9WPOju7QwJwdw4EWAVDWmtjR8+HDev0A1wrZE1sB2ZHvy8vIQFxeHZs2aWXVAsNpmNBqRkZEBnU5X7v0yVDsiIyMxa9YspKWlyV1KuQYNGoT27dvjww8/vON2eXl5uHz5MuLi4jBkyJBSA1XURjZga5WBW1Mdckx3U/EBwERERERE5bp9+zY2bdqE/fv3Y/r06XKXUybZH/57N/LxFZAAHwThMhAfD5RxTSkREREREQFdunTB7du38dZbb9V4kI/awlAlA19fEfHwLQpVREREREQyCg8PtxgUxJZcuXJF7hIqxMv/ZODrC8Sbn1XFUEVEREREVJ8xVMnA1FMFcFh1IiIiqj0NfDwyojLJ0e4ZqmRQvKdKf5UDVRAREZF1KZVKAEBBQYHMlRDVvZycHABlP0S4tvCeKhk4OgLpjr5ADqD/Jx5quQsiIiKiBsXOzg6Ojo64efMmVCpVvRme3Gg0oqCgAHl5efWmZrIdoigiJycHycnJ0Ol0ddpjxVAlE72HL/APeE8VERERWZ0gCPDx8UFcXBz++ecfucupNFEUkZubCwcHBwiCIHc5VE+5urrC3d29Ts/JUCUTwU8KVaoUhioiIiKyPrVajZYtW9arSwD1ej0OHjyIfv368UHSVC0qlQpKpRJ6vb5Oz8tQJRN1Ex/gN0CdmwFkZQFardwlERERUQOjUChgb28vdxmVplQqUVhYCHt7e4Yqqld4sapM3Js6IwtO0kwCB6sgIiIiIqqvGKpk4usn8FlVREREREQNAEOVTPz8ij0AmD1VRERERET1FkOVTIo/q4o9VURERERE9RdDlUyK91SJNxiqiIiIiIjqK4YqmXh5AQmCFKry4hiqiIiIiIjqK4YqmdjZAbkuPgCAwqsMVURERERE9RVDlYwKPaWeKiGBoYqIiIiIqL5iqJKRwl8KVZpbDFVERERERPUVQ5WM7JtJl/+p8rOBzEyZqyEiIiIioupgqJJR46ZapEMnzXBYdSIiIiKieomhSkZ8VhURERERUf3HUCWj4s+qYqgiIiIiIqqfGKpk5OsLJEC6r4qhioiIiIiofmKoklHxnqrCawxVRERERET1EUOVjNzcgGSlFKryLzNUERERERHVRwxVMhIEIK/Rvz1V1xNkroaIiIiIiKqDoUpmRm8pVCkT2VNFRERERFQfMVTJTBkghSpNajwgijJXQ0REREREVcVQJTPHIGn0P5U+F0hPl7kaIiIiIiKqKoYqmXkEOuA2XKUZDqtORERERFTvMFTJjA8AJiIiIiKq3xiqZObry1BFRERERFSfMVTJrHhPlRjPYdWJiIiIiOobhiqZFe+pKrjCnioiIiIiovqGoUpmjo7AbQcpVOXHMVQREREREdU3soaqVatWoWPHjtDpdNDpdAgODsb27dvN6wcMGABBECymadOmyVhx7Shwl0KV8TpDFRERERFRfWMn58n9/f2xdOlStGzZEqIoYs2aNRgxYgROnjyJ9u3bAwCefvppLFq0yLyPo6OjXOXWHh8f4DqgTGaoIiIiIiKqb2QNVWFhYRbzb7zxBlatWoUjR46YQ5WjoyO8vb3lKK/O2AX6AscAh9vxgCgCgiB3SUREREREVEmyhqriDAYD1q9fj+zsbAQHB5uXr127Ft9++y28vb0RFhaG119//Y69Vfn5+cjPzzfPZ2RkAAD0ej30en3tvYFKMJ2/ZB2app4AADtDAfRJSYC7e53XRvVLeW2JqKrYlsga2I7IWtiWyFrKa0u11bYEURTFWjlyJZ05cwbBwcHIy8uDVqtFVFQUhg8fDgD47LPP0KRJE/j6+uL06dOYO3cuevbsiY0bN5Z7vIULFyIiIqLU8qioKJu9dHDbtmZ447OBaIxb2PvBB8hs0kTukoiIiIiIGpycnByMGzcO6enp0Ol0Vjuu7KGqoKAAV69eRXp6OjZs2IAvvvgCBw4cQLt27Uptu3fvXgwaNAiXLl1CUFBQmccrq6cqICAAKSkpVv3gqkOv1yM6OhqDBw+GSqUyL9+yRUCrR7uiI86gcOtWiIMHy1gl1QfltSWiqmJbImtgOyJrYVsiaymvLWVkZKBx48ZWD1WyX/6nVqvRokULAEC3bt1w7NgxfPDBB/j0009LbdurVy8AuGOo0mg00Gg0pZarVCqb+eYsWUtgoPSsqo44A7vkZMBG6iTbZ0vtmuo3tiWyBrYjsha2JbKWkm2pttqVzT2nymg0WvQ0FRcTEwMA8PHxqcOKap+fX9EDgI03OAIgEREREVF9ImtP1SuvvIJhw4YhMDAQmZmZiIqKwv79+7Fz507Exsaa769yd3fH6dOn8cILL6Bfv37o2LGjnGVbnZcXkAgpKObGxsNJ5nqIiIiIiKjyZA1VycnJmDhxIhISEuDi4oKOHTti586dGDx4MK5du4bdu3dj+fLlyM7ORkBAAEaPHo158+bJWXKtsLMDsnS+QAagv8KeKiIiIiKi+kTWUPXll1+Wuy4gIAAHDhyow2rkVdBYClUiL/8jIiIiIqpXbO6eqruV4CfdU2WXkiBzJUREREREVBUMVTZC1UQKVQ5pCYDRKHM1RERERERUWQxVNkLbwhsAYGfUA7duyVwNERERERFVFkOVjfAOUCEJntJMPO+rIiIiIiKqLxiqbISfH5Dw77DqDFVERERERPUHQ5WN8PUtegAwQxURERERUf3BUGUj/PyKQlXhVYYqIiIiIqL6gqHKRri5AclKKVTlXGKoIiIiIiKqLxiqbIQgADluUqjSX+OzqoiIiIiI6guGKhtS6MF7qoiIiIiI6huGKhui8JdClTqFoYqIiIiIqL5gqLIhmqbSkOqOGYmAwSBzNUREREREVBkMVTbEuYUXjBCgFA3AzZtyl0NERERERJXAUGVDfALskAQvaYb3VRERERER1QsMVTak+LOqGKqIiIiIiOoHhiob4utbFKrEeA6rTkRERERUHzBU2ZDioSo/jj1VRERERET1AUOVDXF0BFLtpVCVG8tQRURERERUHzBU2Zh8N2lY9cJrDFVERERERPUBQ5WNMXhJPVWKBIYqIiIiIqL6gKHKxij8pVClSWWoIiIiIiKqDxiqbIx9cylUOWYmAYWFMldDREREREQVYaiyMS4tPFAIJRQQgeRkucshIiIiIqIKMFTZGN8AJRLhLc3wAcBERERERDaPocrGFH9WFUMVEREREZHtY6iyMX5+QAKkYdWN1xmqiIiIiIhsHUOVjfHyAhL+7anKucRQRURERERk6xiqbIydHZChlUJV3mWGKiIiIiIiW8dQZYPy3aVQZeDlf0RERERENo+hygYZvaVQpUxOkLkSIiIiIiKqCEOVDVIGSKHKPpU9VUREREREto6hygY5tpBClTY7GdDrZa6GiIiIiIjuhKHKBrkGuUMPO2kmMVHeYoiIiIiI6I4YqmyQX4DC/KwqPgCYiIiIiMi2MVTZIF9fIP7fZ1UxVBERERER2TZZQ9WqVavQsWNH6HQ66HQ6BAcHY/v27eb1eXl5mD59Otzd3aHVajF69GgkJSXJWHHd8PMrClX6qwxVRERERES2TNZQ5e/vj6VLl+L48eP4448/MHDgQIwYMQLnzp0DALzwwgv46aefsH79ehw4cADx8fF4+OGH5Sy5Tri5AUkKKVRlX+Sw6kREREREtsxOzpOHhYVZzL/xxhtYtWoVjhw5An9/f3z55ZeIiorCwIEDAQCrV69G27ZtceTIEfTu3VuOkuuEIADZLr7AbSA/jj1VRERERES2TNZQVZzBYMD69euRnZ2N4OBgHD9+HHq9HiEhIeZt2rRpg8DAQBw+fLjcUJWfn4/8/HzzfEZGBgBAr9dDL/Pw5KbzV6aOfHdv4DZguH5D9rrJ9lSlLRHdCdsSWQPbEVkL2xJZS3ltqbbaluyh6syZMwgODkZeXh60Wi02bdqEdu3aISYmBmq1Gq6urhbbe3l5IfEOw4wvWbIEERERpZbv2rULjo6O1i6/WqKjoyvcJlWjBgAYrsVh27ZttV0S1VOVaUtElcG2RNbAdkTWwrZE1lKyLeXk5NTKeWQPVa1bt0ZMTAzS09OxYcMGTJo0CQcOHKj28V555RXMnj3bPJ+RkYGAgAAMGTIEOp3OGiVXm16vR3R0NAYPHgyVSnXHbf9afw44BzTKTcXw4cPrqEKqL6rSlojuhG2JrIHtiKyFbYmspby2ZLqKzdpkD1VqtRotWrQAAHTr1g3Hjh3DBx98gDFjxqCgoABpaWkWvVVJSUnw9vYu93gajQYajabUcpVKZTPfnJWpRdsqAADglHcLMBqBMt4TkS21a6rf2JbIGtiOyFrYlshaSral2mpXNvecKqPRiPz8fHTr1g0qlQp79uwxr7tw4QKuXr2K4OBgGSusG41aNEI+pEsAkcARAImIiIiIbJWsPVWvvPIKhg0bhsDAQGRmZiIqKgr79+/Hzp074eLigilTpmD27Nlo1KgRdDodZs6cieDg4AY98p+Jn7+AePiiGa5IoappU7lLIiIiIiKiMsgaqpKTkzFx4kQkJCTAxcUFHTt2xM6dOzF48GAAwPvvvw+FQoHRo0cjPz8foaGh+Pjjj+Usuc74+sIcqsQb8RDkLoiIiIiIiMoka6j68ssv77je3t4eK1euxMqVK+uoItvh6wuchPQA4LzL8XCQuR4iIiIiIiqbzd1TRRJHRyBV7QMAyL7IBwATEREREdkqhiobluMq9VQV/MNQRURERERkqxiqbJjeQwpViGeoIiIiIiKyVQxVNkzwk0KV6iZDFRERERGRrWKosmGqJlKockznc6qIiIiIiGwVQ5UN07aSQpVT/m0gN1fmaoiIiIiIqCwMVTascZALckyDqSewt4qIiIiIyBYxVNkwP38BCZCGVedgFUREREREtomhyob5+gLx/z4A2HidoYqIiIiIyBYxVNkwLy8g4d9QlfU3QxURERERkS1iqLJhdnZAmpMUqnIuMVQREREREdkihiobl+sqharCaxyogoiIiIjIFjFU2TiDlxSqkMCeKiIiIiIiW8RQZeMEPylUaVIYqoiIiIiIbBFDlY3TNPv3AcAZDFVERERERLaIocrGObeSnlPlqM8AsrJkroaIiIiIiEpiqLJxnkHOyIKTNJPAwSqIiIiIiGwNQ5WN8/UTzA8ARjwvASQiIiIisjUMVTbOzw/mUFXwD3uqiIiIiIhsDUOVjXNzA5IUUqjKvMCeKiIiIiIiW8NQZeMEAch0lkJV7mWGKiIiIiIiW8NQVQ/kNZJCleEaQxURERERka2pVqhavXo1cnJyrF0LlcPoJQ2rrkhkqCIiIiIisjXVClUvv/wyvL29MWXKFPz222/WrolKUAZIPVX2qQxVRERERES2plqh6saNG1izZg1SUlIwYMAAtGnTBm+99RYSExOtXR8BsG8uhSrnTIYqIiIiIiJbU61QZWdnh1GjRmHLli24du0ann76aaxduxaBgYF46KGHsGXLFhiNRmvXetfStZYu/7MvzAYyM2WuhoiIiIiIiqvxQBVeXl649957ERwcDIVCgTNnzmDSpEkICgrC/v37rVAieQVpkQ6dNMMHABMRERER2ZRqh6qkpCS8++67aN++PQYMGICMjAz8/PPPiIuLw40bN/DYY49h0qRJ1qz1ruXrW/QAYPEGQxURERERkS2pVqgKCwtDQEAAIiMj8fTTT+PGjRv47rvvEBISAgBwcnLCiy++iGvXrlm12LtV8VCVG8tQRURERERkS+yqs5OnpycOHDiA4ODgcrfx8PBAXFxctQujIo6OwC2VD6AHMi/Ew1HugoiIiIiIyKxaPVX9+/dH165dSy0vKCjA119/DQAQBAFNmjSpWXVklqWTeqry49hTRURERERkS6oVqiZPnoz09PRSyzMzMzF58uQaF0Wl5btLocpwnaGKiIiIiMiWVCtUiaIIQRBKLb9+/TpcXFxqXBSVJvpIoUqZnCBzJUREREREVFyV7qnq0qULBEGAIAgYNGgQ7OyKdjcYDIiLi8PQoUOtXiQBdoFSqHJIZU8VEREREZEtqVKoGjlyJAAgJiYGoaGh0Gq15nVqtRpNmzbF6NGjrVogSRxbSKFKlx0PiCJQRk8hERERERHVvSqFqgULFgAAmjZtijFjxsDe3r5GJ1+yZAk2btyIv/76Cw4ODujTpw/eeusttG7d2rzNgAEDcODAAYv9nnnmGXzyySc1Ond9o2vtAwDQGHKB9HTA1VXegoiIiIiICEA176maNGlSjQMVABw4cADTp0/HkSNHEB0dDb1ejyFDhiA7O9tiu6effhoJCQnm6e23367xuesbn+YOuA1XaSaelwASEREREdmKSvdUNWrUCH///TcaN24MNze3MgeqMElNTa3UMXfs2GExHxkZCU9PTxw/fhz9+vUzL3d0dIS3t3dlS22QTA8AdkMaDNfioWzXTu6SiIiIiIgIVQhV77//Ppydnc2v7xSqqss0THujRo0slq9duxbffvstvL29ERYWhtdffx2OjmU/Ajc/Px/5+fnm+YyMDACAXq+HXq+3es1VYTp/depo1Aj4E75oj/NIO38VuoHyvheSV03aElFxbEtkDWxHZC1sS2Qt5bWl2mpbgiiKYq0cuYqMRiMeeughpKWl4ddffzUv/+yzz9CkSRP4+vri9OnTmDt3Lnr27ImNGzeWeZyFCxciIiKi1PKoqKhyg1h9kfPYeowtWIuDw6fi9tThcpdDRERERFSv5OTkYNy4cUhPT4dOp7PacasVqiIjIxEeHl5qeWFhIV5//XUsWbKkyoU8++yz2L59O3799Vf4+/uXu93evXsxaNAgXLp0CUFBQaXWl9VTFRAQgJSUFKt+cNWh1+sRHR2NwYMHQ6VSVXn/bwNfx+TEt3DpgRlosmlZLVRI9UVN2xKRCdsSWQPbEVkL2xJZS3ltKSMjA40bN7Z6qKrS6H8mzz33HLZu3YrPPvsMbm5uAIALFy5g3LhxuHXrVpVD1YwZM/Dzzz/j4MGDdwxUANCrVy8AKDdUaTQaaDSaUstVKpXNfHNWtxZ9Yz8gERASEm3mvZC8bKldU/3GtkTWwHZE1sK2RNZSsi3VVruq1uh/J0+exPXr19GhQwdER0dj5cqV6Nq1K9q0aYNTp05V+jiiKGLGjBnYtGkT9u7di2bNmlW4T0xMDADAx8enOqXXb77Ss6pUyRz9j4iIiIjIVlSrpyooKAiHDh3CrFmzMHToUCiVSqxZswZjx46t0nGmT5+OqKgobNmyBc7OzkhMTAQAuLi4wMHBAbGxsYiKisLw4cPh7u6O06dP44UXXkC/fv3QsWPH6pRer6mbSEHSIY2hioiIiIjIVlSrpwoAtm7dinXr1iE4OBiurq748ssvEV/F5yetWrUK6enpGDBgAHx8fMzT999/DwBQq9XYvXs3hgwZgjZt2uDFF1/E6NGj8dNPP1W37HrNqaXUU+WaHQ/YxvgiRERERER3vWr1VD3zzDNYs2YN3njjDcyePRtJSUl48skn0aFDB6xatQqPPfZYpY5T0RgZAQEBOHDgQHVKbJBc2kg9VSqxAEhNBdzdZa6IiIiIiIiq1VN16NAhHD16FC+++CIEQYC3tze2bduGRYsW4cknn7R2jfQvn6YapODfIJWQIG8xREREREQEoJqh6vjx4+jUqVOp5dOnT8fx48drXBSVzc8PiId0CWDBFd5XRURERERkC6oVqjQaDWJjYzFv3jyMHTsWycnJAIDt27ejsLDQqgVSETc3IFGQQlX6nwxVRERERES2oFqh6sCBA+jQoQOOHj2KjRs3IisrCwBw6tQpLFiwwKoFUhFBANKdpFCVfZGhioiIiIjIFlQrVL388stYvHgxoqOjoVarzcsHDhyII0eOWK04Ki3XVRqsQn+VoYqIiIiIyBZUK1SdOXMGo0aNKrXc09MTKSkpNS6KylfoKfVUoYrD1xMRERERUe2oVqhydXVFQhmjz508eRJ+fn41LorKJ/hJoUqdwlBFRERERGQLqhWqHn/8ccydOxeJiYkQBAFGoxGHDh3CnDlzMHHiRGvXSMWom0qhyimdoYqIiIiIyBZUK1S9+eabaNOmDQICApCVlYV27dqhX79+6NOnD+bNm2ftGqkYbSspVLnkJgJGo8zVEBERERGRXXV2UqvV+Pzzz/H666/j7NmzyMrKQpcuXdCyZUtr10clNGrnDQBQiXrg1i3Aw0PmioiIiIiI7m7VClUmgYGBCAwMtFYtVAk+gSokwRNeSIZ4Ix4CQxURERERkawqHapmz55d6YMuW7asWsVQxXx9gb/hAy8kI/tiPLSdO8ldEhERERHRXa3SoerkyZOV2k4QhGoXQxVzdARu2vkChaeQ8Vc8tHIXRERERER0l6t0qNq3b19t1kFVkKH1BdKAnEscAZCIiIiISG7VGv2vuGvXruHatWvWqIUqKddNGgHQcI2hioiIiIhIbtUKVYWFhXj99dfh4uKCpk2bomnTpnBxccG8efOg1+utXSOVYPCSQhUSSz+AmYiIiIiI6la1Rv+bOXMmNm7ciLfffhvBwcEAgMOHD2PhwoW4desWVq1aZdUiyZLCXwpVmlvsqSIiIiIiklu1QlVUVBTWrVuHYcOGmZd17NgRAQEBGDt2LENVLbNvLoUqbQZDFRERERGR3Kp1+Z9Go0HTpk1LLW/WrBnUanVNa6IKOLfyAQC45iUCBoPM1RARERER3d2qFapmzJiB//3vf8jPzzcvy8/PxxtvvIEZM2ZYrTgqm3s7LxghwA4G4OZNucshIiIiIrqrVevyv5MnT2LPnj3w9/dHp07Sw2dPnTqFgoICDBo0CA8//LB5240bN1qnUjLzDbRDErzgg0QYrsVD6e0td0lERERERHetaoUqV1dXjB492mJZQECAVQqiinl5AafgCx8kIv3PeDTq0VXukoiIiIiI7lpVDlWiKCIiIgIeHh5wcHCojZqoAnZ2QKrGF8g/gcy/E9BI7oKIiIiIiO5iVb6nShRFtGjRAtevX6+NeqiSMp2lEQBzL3MEQCIiIiIiOVU5VCkUCrRs2RK3bt2qjXqokvLcpVBlvMZQRUREREQkp2qN/rd06VK89NJLOHv2rLXroUoSvaVh1RVJDFVERERERHKq1kAVEydORE5ODjp16gS1Wl3q3qrU1FSrFEflUwZIPVX2qQxVRERERERyqlaoWr58uZXLoKpyDJJClS6ToYqIiIiISE7VClWTJk2ydh1URc6tpVDlWpAEFBZKQwISEREREVGdq9Y9VQAQGxuLefPmYezYsUhOTgYAbN++HefOnbNacVQ+j3YeKIQSCojAv58/ERERERHVvWqFqgMHDqBDhw44evQoNm7ciKysLADAqVOnsGDBAqsWSGXzDVAiEd4AgPw4XgJIRERERCSXaoWql19+GYsXL0Z0dDTUarV5+cCBA3HkyBGrFUflc3MDEgXpEsDb5xiqiIiIiIjkUq1QdebMGYwaNarUck9PT6SkpNS4KKqYIAC3HaRh1bP+ZqgiIiIiIpJLtUKVq6srEhISSi0/efIk/Pz8alwUVU62Tuqp4uV/RERERETyqVaoevzxxzF37lwkJiZCEAQYjUYcOnQIc+bMwcSJE61dI5WjoLEUqow3GKqIiIiIiORSrVD15ptvom3btggMDERWVhbatWuHfv36oU+fPpg3b561a6RyiD5SqLJLZqgiIiIiIpJLlUKV0WjEW2+9hfvvvx8nT57EhAkT8PPPP+Pbb7/FX3/9hW+++QZKpbLSx1uyZAl69OgBZ2dneHp6YuTIkbhw4YLFNnl5eZg+fTrc3d2h1WoxevRoJCUlVaXsBssuUApVDrdLX4pJRERERER1o0qh6o033sCrr74KrVYLPz8/REVFYcOGDXjsscfQsmXLKp/8wIEDmD59Oo4cOYLo6Gjo9XoMGTIE2dnZ5m1eeOEF/PTTT1i/fj0OHDiA+Ph4PPzww1U+V0Pk1FIKVbps9lQREREREcnFriobf/311/j444/xzDPPAAB2796NBx54AF988QUUiqpfSbhjxw6L+cjISHh6euL48ePo168f0tPT8eWXXyIqKgoDBw4EAKxevRpt27bFkSNH0Lt37yqfsyFxaSuFqkb6ZECvB1QqmSsiIiIiIrr7VClUXb16FcOHDzfPh4SEQBAExMfHw9/fv8bFpKenAwAaNWoEADh+/Dj0ej1CQkLM27Rp0waBgYE4fPhwmaEqPz8f+fn55vmMjAwAgF6vh16vr3GNNWE6v7XqcA3SQQ87qFCIgqvXIAQGWOW4ZPus3Zbo7sW2RNbAdkTWwrZE1lJeW6qttlWlUFVYWAh7e3uLZSqVyirFGY1GzJo1C3379sU999wDAEhMTIRarYarq6vFtl5eXkhMTCzzOEuWLEFERESp5bt27YKjo2ON67SG6OhoqxwnP18JJ/ggENew99vNyO8cZJXjUv1hrbZExLZE1sB2RNbCtkTWUrIt5eTk1Mp5qhSqRFFEeHg4NBqNeVleXh6mTZsGJycn87KNGzdWuZDp06fj7Nmz+PXXX6u8b3GvvPIKZs+ebZ7PyMhAQEAAhgwZAp1OV6Nj15Rer0d0dDQGDx4MlZUu1YtR+iLQcA2tdQHwL9aLSA1bbbQlujuxLZE1sB2RtbAtkbWU15ZMV7FZW5VC1aRJk0ote+KJJ2pcxIwZM/Dzzz/j4MGDFpcRent7o6CgAGlpaRa9VUlJSfD29i7zWBqNxiL0mahUKpv55rRmLemOvkAmkHc52WbeH9UdW2rXVL+xLZE1sB2RtbAtkbWUbEu11a6qFKpWr15t1ZOLooiZM2di06ZN2L9/P5o1a2axvlu3blCpVNizZw9Gjx4NALhw4QKuXr2K4OBgq9ZSX2W7SqFK/w+HVSciIiIikkOVQpW1TZ8+HVFRUdiyZQucnZ3N90m5uLjAwcEBLi4umDJlCmbPno1GjRpBp9Nh5syZCA4OvutH/jPRe/gC1wDxBodVJyIiIiKSg6yhatWqVQCAAQMGWCxfvXo1wsPDAQDvv/8+FAoFRo8ejfz8fISGhuLjjz+u40ptmK8vcAJQpTBUERERERHJQdZQJYpihdvY29tj5cqVWLlyZR1UVP9omvgAABzTGKqIiIiIiORQ9Sf2kk1xaik9ANg1h6GKiIiIiEgODFX1nGu7f0NV4S2g2EOPiYiIiIiobjBU1XNebRshH2oAgOE6RwAkIiIiIqprDFX1nJe3gHhIvVW3zzNUERERERHVNYaqes7ODkhRSaEq/U/eV0VEREREVNcYqhqAdK0UqnIuMVQREREREdU1hqoGIM9VGlZdf5WhioiIiIiorjFUNQCFnlJPlRDPUEVEREREVNcYqhoAwU8KVepbDFVERERERHWNoaoB0DSTQpU2naGKiIiIiKiuMVQ1ANpW/z4AOI9DqhMRERER1TWGqgag0T1SqHIx3AZyc2WuhoiIiIjo7sJQ1QB4t3ZBDhwAAPlX2FtFRERERFSXGKoaALdGAhIFaVj1lDO8r4qIiIiIqC4xVDUAggDc0kiXAGb+xVBFRERERFSXGKoaiEytFKpyYxmqiIiIiIjqEkNVA5HXSApVhmsMVUREREREdYmhqoEweEmhSkjkQBVERERERHWJoaqBUARIoUqTyp4qIiIiIqK6xFDVQDg0l0KVcwZDFRERERFRXWKoaiCcW0lDqjfKZ6giIiIiIqpLDFUNhHuHf3uqjBkQM7NkroaIiIiI6O7BUNVA+LRyRhacAACZf3OwCiIiIiKiusJQ1UA4OApIVEi9VbfO8BJAIiIiIqK6wlDVgNy2l0IVe6qIiIiIiOoOQ1UDkqWTQlVeHHuqiIiIiIjqCkNVA5LvLoUq43WGKiIiIiKiusJQ1YAYfaRQpUxiqCIiIiIiqisMVQ2IXYD0rCqHVIYqIiIiIqK6wlDVgDg0l3qqdFkMVUREREREdYWhqgFxaSuFKvcChioiIiIiorrCUNWANO4gXf7nJGbDkJYpczVERERERHcHhqoGxLO5FunQAeADgImIiIiI6gpDVQNiZwckK6VLAG+fY6giIiIiIqoLDFUNTJqjFKqy/maoIiIiIiKqC7KGqoMHDyIsLAy+vr4QBAGbN2+2WB8eHg5BECymoUOHylNsPZGtk+6rKrjCUEVEREREVBdkDVXZ2dno1KkTVq5cWe42Q4cORUJCgnn67rvv6rDC+kfvIfVUiTcYqoiIiIiI6oKdnCcfNmwYhg0bdsdtNBoNvL2966ii+k/08QViALtkhioiIiIioroga6iqjP3798PT0xNubm4YOHAgFi9eDHd393K3z8/PR35+vnk+IyMDAKDX66HX62u93jsxnb8261AGeAEAHNISZH+/VHvqoi3R3YFtiayB7YishW2JrKW8tlRbbUsQRVGslSNXkSAI2LRpE0aOHGletm7dOjg6OqJZs2aIjY3Fq6++Cq1Wi8OHD0OpVJZ5nIULFyIiIqLU8qioKDg6OtZW+Tbj1pZ4PLn6P7hi1wynNrwvdzlERERERDYjJycH48aNQ3p6OnQ6ndWOa9OhqqTLly8jKCgIu3fvxqBBg8rcpqyeqoCAAKSkpFj1g6sOvV6P6OhoDB48GCqVqlbO8feOy2j/UBvkwAGq/DRAEGrlPCSvumhLdHdgWyJrYDsia2FbImspry1lZGSgcePGVg9VNn/5X3HNmzdH48aNcenSpXJDlUajgUajKbVcpVLZzDdnbdbi0zUQAOCIXOTeyoGDj2utnIdsgy21a6rf2JbIGtiOyFrYlshaSral2mpX9eo5VdevX8etW7fg4+Mjdyk2y83XAbfhCgBIOc3BKoiIiIiIapusoSorKwsxMTGIiYkBAMTFxSEmJgZXr15FVlYWXnrpJRw5cgRXrlzBnj17MGLECLRo0QKhoaFylm3TBAG4qZKGVU/7k6GKiIiIiKi2yRqq/vjjD3Tp0gVdunQBAMyePRtdunTB/PnzoVQqcfr0aTz00ENo1aoVpkyZgm7duuGXX34p8/I+KpLuJIWqnIsMVUREREREtU3We6oGDBiAO42TsXPnzjqspuHIcfUF0oCCfxLkLoWIiIiIqMGrV/dUUeUUekg9VYhnTxURERERUW1jqGqABD8pVKlTGKqIiIiIiGobQ1UDpG4qhSrHdIYqIiIiIqLaxlDVAGlbSkPOu+YwVBERERER1TaGqgbIrb3UU+VZGA/RWP5AIEREREREVHMMVQ2QZyepp0qDAmRcSZW5GiIiIiKiho2hqgFycNXgluAOALh5msOqExERERHVJoaqBuqWWroEMP1P3ldFRERERFSbGKoaqAytFKpyYxmqiIiIiIhqE0NVA5XrJoWqwqsMVUREREREtYmhqoEyeEqDVQgJDFVERERERLWJoaqBUvhLPVWaWwxVRERERES1iaGqgbJvLoUqpwyGKiIiIiKi2sRQ1UBpW0mhqlEeQxURERERUW1iqGqgGt0jhSoPQyIMeqPM1RARERERNVwMVQ1U43u8AQBq6JFy4ZbM1RARERERNVwMVQ2UnYMKNxWeAICU07wEkIiIiIiotjBUNWC3NdKw6pkXGKqIiIiIiGoLQ1UDluks3VeVG8tQRURERERUWxiqGrC8RlKoMl5nqCIiIiIiqi0MVQ2Y0VsKVYpEhioiIiIiotrCUNWAKQOkUKW5nSBzJUREREREDRdDVQNm31wKVc6Z7KkiIiIiIqotDFUNmK6tFKrc8xiqiIiIiIhqC0NVA+beXhpS3VNMRG6WQeZqiIiIiIgaJoaqBsy1tReMEGAHA5LP3ZS7HCIiIiKiBomhqgETVHZIUXoBAG6d4SWARERERES1gaGqgbvtIN1XlfU3QxURERERUW1gqGrgsnRSqMr8m8OqExERERHVBoaqBk7wlUKV35aVeP+xw8jKkrkgIiIiIqIGhqGqgWu74FHk2zmiM07hhfV9sM/rcfz6TZzcZRERERERNRgMVQ2cw4Mh0PxzEddDn4QRAsJyvkePiW2wveNcpP2TLnd5RERERET1HkPV3cDXF/47vkTurydwwX8gNCjAsDNvw9C8BU5NWwUUFspdIRERERFRvcVQdRdx6tsZra/uxrmlP+GyqjXcjSno9Ol/cN29I9K/2waIotwlEhERERHVOwxVdxtBQPu5D8In5Qw2D/4IKXCHf8afcBn3ABI6hkI8fUbuComIiIiI6hVZQ9XBgwcRFhYGX19fCIKAzZs3W6wXRRHz58+Hj48PHBwcEBISgosXL8pTbAPjoFNh5K7puL7vEiIbz0E+1PA5Gw1jp87IGj8VSEyUu0QiIiIionpB1lCVnZ2NTp06YeXKlWWuf/vtt/Hhhx/ik08+wdGjR+Hk5ITQ0FDk5eXVcaUNV+cBrhh34x18/sKf2KB4FEoYoY36HAVNW0Jc/AaQmyt3iURERERENk3WUDVs2DAsXrwYo0aNKrVOFEUsX74c8+bNw4gRI9CxY0d8/fXXiI+PL9WjRTWjVgMzljVHuzP/h6ntfsVR9IQ6PwvC6/OgD2oNfPstYDTKXSYRERERkU2yk7uA8sTFxSExMREhISHmZS4uLujVqxcOHz6Mxx9/vMz98vPzkZ+fb57PyMgAAOj1euj1+totugKm88tdR3latgQ+PN4TH334Kz6etx6L9K+gScJVYMIEGN9fDuO770C89165yyTYflui+oNtiayB7YishW2JrKW8tlRbbctmQ1Xiv/f0eHl5WSz38vIyryvLkiVLEBERUWr5rl274OjoaN0iqyk6OlruEu6oZWtA+6EHHvnoF4ScW4tX8SacTxyHYuBAxAcH49zEicjx8ZG7TILttyWqP9iWyBrYjsha2JbIWkq2pZycnFo5j82Gqup65ZVXMHv2bPN8RkYGAgICMGTIEOh0Ohkrk5JxdHQ0Bg8eDJVKJWstlfHkk8BXX72ELi9NxpyshXgan8P38GH4/PEHjDNmwPjyy4Cbm9xl3pXqW1si28W2RNbAdkTWwrZE1lJeWzJdxWZtNhuqvL29AQBJSUnwKdYrkpSUhM6dO5e7n0ajgUajKbVcpVLZzDenLdVSkWnTgAcf9Ma0aZ/go60z8C7mYKh+J5Tvvw/lmjXAwoXSRvXk/TQ09aktkW1jWyJrYDsia2FbImsp2ZZqq13Z7HOqmjVrBm9vb+zZs8e8LCMjA0ePHkVwcLCMld19/P2Bn34CXl17D55w34Gh2I6zaA+kpgLPPQfccw/w4498eDARERER3ZVkDVVZWVmIiYlBTEwMAGlwipiYGFy9ehWCIGDWrFlYvHgxfvzxR5w5cwYTJ06Er68vRo4cKWfZdyVBAMaNA86fB1zHDEVnxOAZfIJbSg/g77+BESOAQYOAkyflLpWIiIiIqE7JGqr++OMPdOnSBV26dAEAzJ49G126dMH8+fMBAP/9738xc+ZMTJ06FT169EBWVhZ27NgBe3t7Ocu+q3l6AuvWAT9stsNPPs+gmeESluAV6JUaYN8+oFs36Was+Hi5SyUiIiIiqhOyhqoBAwZAFMVSU2RkJABAEAQsWrQIiYmJyMvLw+7du9GqVSs5S6Z/jRgBnDsHPPqkDq/iTbQwXMAWx7HSJYCrV0vjs0+fLl0WWEs3BBIRERER2QKbvaeKbJ+bG/Dll8CuXYCiaROMzIlCLxzBRc8+QE4O8PHHUvpq1Ai4915pUItDhwA+e4KIiIiIGhCGKqqxwYOBM2ekMSuOCb3QKvlXjHfbhj/vfxbGoBaAwSCFqYgIKVy5u0th66OPgL/+4gAXRERERFSvMVSRVWi1wAcfAL/8ArRuLSDq9jC02/cxXJMvYu5jcbg093OIjz0mBarMTOmywJkzgbZtgSZNpPuwvvsOSE6W+60QEREREVUJQxVZVd++QEwMsGQJ0Ly5lJ/e/r+maPnWU2gT8z2WvJCM5G1/SBsMGgRoNMC1a9J9WOPGAV5eQJcuwEsvSdcV5ubK/ZaIiIiIiO6IoYqszt4eePll4OJF4MABIDwccHKSRl5/dZ4CPg92w9D9L+P7p3cjLz4V2LkTmDMH6NRJOkBMDPDuu0BoqHTjVkgI8NZbwPHjgNEo51sjIiIiIiqFoYpqjUIB9OsndUIlJABffQXcd5+Ui3buBB5/HPBt4YjpW4bgjzHvQDwZAyQlAVFRwOTJ0lOH8/OBPXuklNa9uzSm+5gxwBdfAP/8I/dbJCIiIiJiqKK64ews5aSDB6UerNdekzLT7dvSIIE9egAdOwLLvvVE8qCxUgK7elUayGLFCuChh6SD3LoF/N//AU8/DTRtCrRqBfznP8CmTUBamtxvk4iIiIjuQgxVVOdatAAWLwauXCnqsdJogLNngRdfBPz8gJEjgc1bBOibtwZmzAC2bJEC1a+/SkOz9+0LKJVSQlu1Cnj4YWkQjN69pcS2dy+QlyfzOyUiIiKiuwFDFclGqQSGDJEG/UtMlLJRz55AYaGUoUaNkgLW7NnSkO1QqaQwtWCBFK5SU6UNZ8wA2rSRris8ehR4801pEAxXV+nrG28AR45IByYiIiIisjKGKrIJrq7AtGlSJjp7Vhq3wssLuHkTeP996dLA7t2BlSulLAUA0OmkywJXrAD+/FO6XHD1amDCBMDXV7ofa+9eYN48IDhYeghxWJh0wNOnOegFEREREVkFQxXZnPbtgXfekUZa//FH6co+lUoa/G/GDMDHRxqrYscO6bnCZgEB0lCDX38NXL8u3Y+1ciUwerQUqDIzgZ9/lrq+OnUCvL2lA336KXDpEh9CTERERETVwlBFNkulkjqWfvgBuHEDWL5cykIFBdJYFcOGSc8NfvVVabh2C4IAtG4tDWKxYYPU5XXihJTWhg2Txni/eVM60LRpQMuW0sEmTwa++UY6IRERERFRJTBUUb3g4QE8/7z0CKsTJ4CZM6XOpxs3pOcIt24t3W61ahVw/nwZV/YpFNJDhefMAbZtk64h/PVXICIC6N8fUKulrrHISGDiRGlowjZtgOnTpVRnvuaQiIiIiMiSndwFEFVVly7S9M47wE8/SbdR7dgB/PabNAHSPVrBwUCfPtLUsyeg1RY7iFotpbC+fYH584GcHODQIemZWHv3StcaXrggTR9/LPV8de4sDXwxcKD0wC2LAxIRERHR3YqhiuotjQZ45BFpSkiQrtrbvl0a7CItTXq9fbu0rUIhXTpoCll9+khX+wnCvwdzdAQGD5YmQHqA1oEDUsDas0fq/jp5UprefRewswN69ZJC1oABUs+WTidN9vbFDkxEREREDR1DFTUIPj7Af/8rTXq9NLifqefqt9+kgQFNmWjlyqJ9ioesLl2koAYAcHOTHpY1cqQ0n5goBSxTyLpyRerZOnQIWLTIshg7OylcubgUBS3TVJVlDGdERERE9QJDFTU4KhXQrZs0zZwpLbt+HTh8uChknTgh9W798IM0AVKg6t69KGQFB0vDugOQRgocN06aACAuruhSwcOHpXuuMjOlEQQLC6X5mt6HZQpnZYQvhVaL1hkZEFJTpXu/goKkG88YwoiIiIjqHEMV3RX8/YFHH5UmAMjNBf74w7I3KyWlqPPJJCjIsjerfXvpocVo1gx46ilpMjEagexsID0dyMgomkrOl7Ws+HwlwpkSQBsA+P77ooVarVRwUBDQokXR66Agabh5pbIWPlkiIiIiYqiiu5KDgzTWxH33SfOiKD2qqnjIOncOiI2Vpm++kbZzdgZ69y4KWb16SR1IAKQbt5ydpakmKhHODKmpuH7kCAL0eihiY6WuuKws4NQpaSpJpQKaNi0dtoKCgObNpUsNiYiIiKhaGKqIIF0117KlNE2aJC1LS5MGvTCFrCNHpE6k6GhpMu3Xvn3RFXjNmxdNgYHSFXxVVolwZtTrEbNtG3yHD4dCpQLy8qRLEk0pMDZWSomxsdJyvR64eFGayuLvXzpsmQKYq2s13gQRERHR3YOhiqgcrq5AaKg0AYDBAJw9a9mbdfmytOzs2dL7K5XSCIPFg5ZpsnpWsbcH2raVppIMBqknq2TYMk2ZmdL669elEQ9LatSoKGg1aybdu9W4celJq+U9XURERHRXYqgiqiSlUhqWvVMn4NlnpWWJidK9WZcuSQGr+JSfX/S6LG5upYOW6XVAQDV7ucorvEkTaRo40HKdKEo3k5UVtmJjgaSkovu6jh2783nUailcubuXHbrKmhwdrfQmiYiIiOTDUEVUA97ewIMPll5uNEqjCxYPWbGxRa+TkqRHYR0/Lk0lmXJQyUsKTZOTk5XegCBIPU8eHtLNYiVlZhYVHxsrjU1/65YUxEzTzZvS5YcFBUB8vDRVloND2WGreDBzd5cSpihK+5T8Wtaymn411Va8DgeHyr8vIiIiuqswVBHVAoUC8POTJtNgGMVlZxfdAlWyhysuruJerkaN7ODu3g/ffadEq1bS7U+myaojqzs7F3XP3UlOjmXQqsyk10vDMF67Jk22rngANIWtkuGv5Gv2xBEREd0VGKqIZODkBNxzjzSVZDRKnT3l9XIlJwOpqQJSU93KHHfC2dkyZBWffHxq6bYnR0dpZI7AwMptL4rSaIWVCV+3bkkfClBUfG1+FQSpvpycol656gbA4r1dFQWw4kGM96YRERHVKwxVRDZGoZAG4/P3B/r1K70+Kwu4cEGPDRtOwsWlG+LilLh0Sbol6to16Yq9kyelqSRHx6KB/UpO/v7SueuEIBSNcNisWR2dtJpEUfpQi1/2WJnXpiBmGgSksjSassPWnb46OzOIERERyYihiqie0WqBjh2B69cTMHy4ESpV0UN9TSOrm0JW8enKFanz5cwZaSpJo5Hu1yoetFq2lL5adeCM+kYQAJ1OmiobAIv3xFUljBUUSNd+3rghTZWlUknhqipBzMWlDlM0ERFRw3a3/ppE1CDdaWT1ggLgn3/KDlym0Qr//FOaSlKppDzRrJk0OIeXV9mTu7s0yMZdrzo9caYgdutWUciq6GtKipSk9XppKMrExMrXqFQCjRrBrlEj9C8shPLtt6UGZG8vJeySr2uyrPhrlYq9akRE1OAwVBHdJdTqogccl1RYKF06WFbgio2VAtfff0vTnSgU0kAZ5YWu4pOHx13c+1WW4kGsadPK71f83q/iX+8UyLKypOeX3bwJ4eZNuALSP3RduVPoqmhdTbbTaKQwqVCUPZnuqSMiIqoi/kpDRLCzK+qJGjzYcp3RKF2JZrqEMCmp7Mk0noRpviKCIPVsFQ9anp5lB7BGjTh+Q7kcHaUpIKDy++Tnm0NWYVISjh08iB4dOsDOYJB6vvLypG2Kf63ssrLWFRRYnt+03hYJQlHIulMAK28qax8XF6lxe3iU/7VxY6kXj4iI6iWGKiK6I4VC+n29ot/ZCwulR1aVF7qKTzdvSgHMdAXbuXMV12FnB7i6Sr+furpaTiWXlbWNVstbiMw0GsDXF/D1hajXIzknB+Lw4bX3S73RKAWryoS08rap7nbFXxd/Dll5RFHqxTMYpMsq65KbW8Xhy/TV9Py22mI0SgOt5ORUalJkZqL12bNQHD8utS87O2lSKotel7esptvw0lIisgEMVURkFXZ20pDtPj4Vb2swSB0llQlgyclSYCssLAph1SEIlmGronCm00lBzNlZ+mp6zc6EalAoii7Dk4soSo3IaLzzZDBUvE1V9iksBNLSpL8kJCeX/TUlRdr29m1punCh4vdj6uotL3g5OFQpFCEnR3qAnul1FXsSlQDaVOsfxooEwfKST9Pk4FD28ppso1ZLoc7UM2l6XdZUvAeToY+owWKoIqI6p1RKv/d5egIdOtx5W1GUftdLSwPS06WvxaeSy8rapqBAOo5pvibU6tJhq6zwVdbrstY5OfHesjohCLabiA0GIDX1zsGr+NfUVKlBm/7KUNboMtZkb190mWlZk5MTjPb2+CcxEYGBgVCawqRpMhgs58taVpltylpmeoYdIH0mubnSZKtKBrA7BbKy1qnV0qRSFb22xvydtgGKPndTL27xf4vyllV1+39fKwoK0O7yZSh+/136QengUBRoTa9LTiXXsdeSZMD/yonIpglCUQjx96/eMfLyKg5jxZfdvi09miorS5oyM4tuCyookH6nTU2t8Vszs7cvClim8RRKTsXHWqjMVNntlUrAYOAvH7JSKqUeJg8PoF27ircvLJS6eu8UvnJypAZ1pzD0byC643oHh0pdN2vQ63F62zb4Dx8OZV2GV6NRukzTdIlnbq7lJaLlTZXZ7k7b5OcXBQFTb6XBULl6TTVTmZQAWgLA5s3VP4hCUb0wVtX7KIvfg1ndydR+Sv7RoKzX1lomiqVDdU0mjaby2zbgIYIZqoiowbO3l4aC9/au/jEKCqQes+JhyxS4ynpdmXWFhdKxTb+n3bxpnfdbNSoAD0GrFeHqKt3WY7oEsvjrO807O/N+tTplZ1c0isvdTqEo+guBi4vc1VgGrLJCV03X6fXSDyPTVNvzpr8mFb+HzdRrVkvLDAoF4mJj0czXF8r8/KLeR1PILT5fcp3pvkmjUfqBnZ0tX1ug0gYPBnbtkruKWsNQRURUCaY/srm5Wed4oij9vlI8cGVnW47dUJmpptuafgfJyhKQlQVcv17192Ia4K68AFZWGHNyKrrCyHSVUcnXdna8gofqGVPvg61ealoPGPV6nNu2DU2q2utp+qFaXuC6Uxgzva7q/ZQ1nQwGqb2UNSBLyfBZ1dd3Wi8IZQfoykz5+VXbvnivrOly0gbKpkPVwoULERERYbGsdevW+Ouvv2SqiIjIOgSh6A/s7u7y1CCKQE6OHhs37kb37iHIzlaZL380XRJZ/HVZ6/LzLcdYsDZT0LpT+KrKa9OVKpWZKrst74kjsgHFf6i6uspdDZmIYlGAq8wIrPWYzf9X0L59e+zevds8b8f/vYiIrEIQpOCg0xWgRYvq/XG9+P1qlQ1jaWlSz5xeXzQVFJR9S4ppvS0zXYFWXghzcip6rrNpoJLi83dax0cBEFG9ZvqPpoH3UgH1IFTZ2dnBuwo3QuTn5yM/P988n5GRAQDQ6/XQy/w/s+n8ctdB9R/bEllLTduSUin1tFmjt800aJzpihFT2Cr5urBQKHed5SSU2sZ05Yr0VbC4FNK0XJqEMpYVzRuNgkXdtTnonJOTWGLUSNEigGm1okUY02pF8/gS1XuOsQhBqNo+BoMe6elq3Lihh52d9JmIYumprOWVXVY0CRbzdnaieXAW0wAtptf8O2z9w//fyFrKa0u11bYEUbTdvriFCxfinXfegYuLC+zt7REcHIwlS5YgMDDwjvuUvGQQAKKiouDo6Fib5RIR0V3CYBCg1ysspsJCZaller0C+fl2yM0tmvLyLOdLTnl5SuTm2sFoZBdVTSkURqhURqjVRqhUBqhUpnnTa8O/66RldnZF64rvU7SuaJk08JsIQRCLhdGSr0VzSK1o26LtUc5r8d86RN5rSFQDOTk5GDduHNLT06HT6ax2XJsOVdu3b0dWVhZat26NhIQERERE4MaNGzh79iycnZ3L3KesnqqAgACkpKRY9YOrDr1ej+joaAwePBgq3sRKNcC2RNbCtmSbRFG6tDIzs2jKyhJKjCgpFFtXNJ+dLfWaiWLx5xIL5d4rb+oZqsnzj4v33AH4NyTAYjKNQG3NZYDUu2kahCUvT+rJbMiUStFiNHwHB8DRUbSYl16LJealZZbzgIODWGpkfTmv1OLPJLKW8tpSRkYGGjdubPVQZdMd48OGDTO/7tixI3r16oUmTZrg//7v/zBlypQy99FoNNBoNKWWq1Qqm/nmtKVaqH5jWyJrYVuyPdL9bnJXUTkFBXps3boNDzwwHGq1CoB8wcZgKApYxR8rVdHrqmxXUFB2CC0+EnplllVmfen3VxSmi1j387azKwpYpufoljWVFXZruh5Q4vbtvnjnHXsAinIvAwXudIlo1bcHpEteTSOP2tkVvS751ZrrTI9tutPnVNH6qm5T3oCDVZk3DSJYH5T8/622/q+z6VBVkqurK1q1aoVLly7JXQoRERH9q/gvyHJTKosCQX1X/B6zwkKpBzInp2jKzracL2tZZbfJzi4KcYWFQEaGNNU9BYDGcpyYqqj4I8YqE8ruvRdYtUruqmtPvQpVWVlZiI2NxYQJE+QuhYiIiKhWFe/RsbOTBt+w1rPySjKNfF0yaBUWlt+7c+cBRSq3Tcn1en0hTpw4ga5du0Klsqtx70xl9wGkHsLCQtOAOJavS3611rrKfIamf5+a9NoV38bUG2qqp/jrsubLY3omtekZ0RUJCKham6xvbDpUzZkzB2FhYWjSpAni4+OxYMECKJVKjB07Vu7SiIiIiBqM4iNfy/mYJ71ehL19AoYPF/kMZRth6imtbAgrb762/iBgK2w6VF2/fh1jx47FrVu34OHhgXvvvRdHjhyBh4eH3KURERERETV4CsVd86ipGrHpULVu3Tq5SyAiIiIiIrojPgSDiIiIiIioBhiqiIiIiIiIaoChioiIiIiIqAYYqoiIiIiIiGqAoYqIiIiIiKgGGKqIiIiIiIhqgKGKiIiIiIioBhiqiIiIiIiIaoChioiIiIiIqAYYqoiIiIiIiGqAoYqIiIiIiKgGGKqIiIiIiIhqgKGKiIiIiIioBuzkLqC2iaIIAMjIyJC5EkCv1yMnJwcZGRlQqVRyl0P1GNsSWQvbElkD2xFZC9sSWUt5bcmUCUwZwVoafKjKzMwEAAQEBMhcCRERERER2YLMzEy4uLhY7XiCaO2YZmOMRiPi4+Ph7OwMQRBkrSUjIwMBAQG4du0adDqdrLVQ/ca2RNbCtkTWwHZE1sK2RNZSXlsSRRGZmZnw9fWFQmG9O6EafE+VQqGAv7+/3GVY0Ol0/EFBVsG2RNbCtkTWwHZE1sK2RNZSVluyZg+VCQeqICIiIiIiqgGGKiIiIiIiohpgqKpDGo0GCxYsgEajkbsUqufYlsha2JbIGtiOyFrYlsha6rotNfiBKoiIiIiIiGoTe6qIiIiIiIhqgKGKiIiIiIioBhiqiIiIiIiIaoChioiIiIiIqAYYqurQypUr0bRpU9jb26NXr174/fff5S6JZHTw4EGEhYXB19cXgiBg8+bNFutFUcT8+fPh4+MDBwcHhISE4OLFixbbpKamYvz48dDpdHB1dcWUKVOQlZVlsc3p06dx3333wd7eHgEBAXj77bdr+61RHVqyZAl69OgBZ2dneHp6YuTIkbhw4YLFNnl5eZg+fTrc3d2h1WoxevRoJCUlWWxz9epVPPDAA3B0dISnpydeeuklFBYWWmyzf/9+dO3aFRqNBi1atEBkZGRtvz2qQ6tWrULHjh3ND8oMDg7G9u3bzevZjqg6li5dCkEQMGvWLPMytiWqrIULF0IQBIupTZs25vU21ZZEqhPr1q0T1Wq1+NVXX4nnzp0Tn376adHV1VVMSkqSuzSSybZt28TXXntN3LhxowhA3LRpk8X6pUuXii4uLuLmzZvFU6dOiQ899JDYrFkzMTc317zN0KFDxU6dOolHjhwRf/nlF7FFixbi2LFjzevT09NFLy8vcfz48eLZs2fF7777TnRwcBA//fTTunqbVMtCQ0PF1atXi2fPnhVjYmLE4cOHi4GBgWJWVpZ5m2nTpokBAQHinj17xD/++EPs3bu32KdPH/P6wsJC8Z577hFDQkLEkydPitu2bRMbN24svvLKK+ZtLl++LDo6OoqzZ88Wz58/L65YsUJUKpXijh076vT9Uu358ccfxa1bt4p///23eOHCBfHVV18VVSqVePbsWVEU2Y6o6n7//XexadOmYseOHcXnn3/evJxtiSprwYIFYvv27cWEhATzdPPmTfN6W2pLDFV1pGfPnuL06dPN8waDQfT19RWXLFkiY1VkK0qGKqPRKHp7e4vvvPOOeVlaWpqo0WjE7777ThRFUTx//rwIQDx27Jh5m+3bt4uCIIg3btwQRVEUP/74Y9HNzU3Mz883bzN37lyxdevWtfyOSC7JyckiAPHAgQOiKErtRqVSievXrzdv8+eff4oAxMOHD4uiKAV8hUIhJiYmmrdZtWqVqNPpzG3nv//9r9i+fXuLc40ZM0YMDQ2t7bdEMnJzcxO/+OILtiOqsszMTLFly5ZidHS02L9/f3OoYluiqliwYIHYqVOnMtfZWlvi5X91oKCgAMePH0dISIh5mUKhQEhICA4fPixjZWSr4uLikJiYaNFmXFxc0KtXL3ObOXz4MFxdXdG9e3fzNiEhIVAoFDh69Kh5m379+kGtVpu3CQ0NxYULF3D79u06ejdUl9LT0wEAjRo1AgAcP34cer3eoi21adMGgYGBFm2pQ4cO8PLyMm8TGhqKjIwMnDt3zrxN8WOYtuHPsIbJYDBg3bp1yM7ORnBwMNsRVdn06dPxwAMPlPr3Zluiqrp48SJ8fX3RvHlzjB8/HlevXgVge22JoaoOpKSkwGAwWPyDAoCXlxcSExNlqopsmald3KnNJCYmwtPT02K9nZ0dGjVqZLFNWccofg5qOIxGI2bNmoW+ffvinnvuASD9O6vVari6ulpsW7ItVdROytsmIyMDubm5tfF2SAZnzpyBVquFRqPBtGnTsGnTJrRr147tiKpk3bp1OHHiBJYsWVJqHdsSVUWvXr0QGRmJHTt2YNWqVYiLi8N9992HzMxMm2tLdlV9c0REZJumT5+Os2fP4tdff5W7FKqnWrdujZiYGKSnp2PDhg2YNGkSDhw4IHdZVI9cu3YNzz//PKKjo2Fvby93OVTPDRs2zPy6Y8eO6NWrF5o0aYL/+7//g4ODg4yVlcaeqjrQuHFjKJXKUqORJCUlwdvbW6aqyJaZ2sWd2oy3tzeSk5Mt1hcWFiI1NdVim7KOUfwc1DDMmDEDP//8M/bt2wd/f3/zcm9vbxQUFCAtLc1i+5JtqaJ2Ut42Op3O5v5jo+pTq9Vo0aIFunXrhiVLlqBTp0744IMP2I6o0o4fP47k5GR07doVdnZ2sLOzw4EDB/Dhhx/Czs4OXl5ebEtUba6urmjVqhUuXbpkcz+XGKrqgFqtRrdu3bBnzx7zMqPRiD179iA4OFjGyshWNWvWDN7e3hZtJiMjA0ePHjW3meDgYKSlpeH48ePmbfbu3Quj0YhevXqZtzl48CD0er15m+joaLRu3Rpubm519G6oNomiiBkzZmDTpk3Yu3cvmjVrZrG+W7duUKlUFm3pwoULuHr1qkVbOnPmjEVIj46Ohk6nQ7t27czbFD+GaRv+DGvYjEYj8vPz2Y6o0gYNGoQzZ84gJibGPHXv3h3jx483v2ZbourKyspCbGwsfHx8bO/nUpWGtaBqW7dunajRaMTIyEjx/Pnz4tSpU0VXV1eL0Ujo7pKZmSmePHlSPHnypAhAXLZsmXjy5Enxn3/+EUVRGlLd1dVV3LJli3j69GlxxIgRZQ6p3qVLF/Ho0aPir7/+KrZs2dJiSPW0tDTRy8tLnDBhgnj27Flx3bp1oqOjI4dUb0CeffZZ0cXFRdy/f7/FkLM5OTnmbaZNmyYGBgaKe/fuFf/44w8xODhYDA4ONq83DTk7ZMgQMSYmRtyxY4fo4eFR5pCzL730kvjnn3+KK1eu5PDFDczLL78sHjhwQIyLixNPnz4tvvzyy6IgCOKuXbtEUWQ7ouorPvqfKLItUeW9+OKL4v79+8W4uDjx0KFDYkhIiNi4cWMxOTlZFEXbaksMVXVoxYoVYmBgoKhWq8WePXuKR44ckbskktG+fftEAKWmSZMmiaIoDav++uuvi15eXqJGoxEHDRokXrhwweIYt27dEseOHStqtVpRp9OJkydPFjMzMy22OXXqlHjvvfeKGo1G9PPzE5cuXVpXb5HqQFltCIC4evVq8za5ubnif/7zH9HNzU10dHQUR40aJSYkJFgc58qVK+KwYcNEBwcHsXHjxuKLL74o6vV6i2327dsndu7cWVSr1WLz5s0tzkH135NPPik2adJEVKvVooeHhzho0CBzoBJFtiOqvpKhim2JKmvMmDGij4+PqFarRT8/P3HMmDHipUuXzOttqS0JoiiKVevbIiIiIiIiIhPeU0VERERERFQDDFVEREREREQ1wFBFRERERERUAwxVRERERERENcBQRUREREREVAMMVURERERERDXAUEVERERERFQDDFVEREREREQ1wFBFRER3taZNm2L58uVyl0FERPUYQxUREdWZ8PBwjBw5EgAwYMAAzJo1q87OHRkZCVdX11LLjx07hqlTp9ZZHURE1PDYyV0AERFRTRQUFECtVld7fw8PDytWQ0REdyP2VBERUZ0LDw/HgQMH8MEHH0AQBAiCgCtXrgAAzp49i2HDhkGr1cLLywsTJkxASkqKed8BAwZgxowZmDVrFho3bozQ0FAAwLJly9ChQwc4OTkhICAA//nPf5CVlQUA2L9/PyZPnoz09HTz+RYuXAig9OV/V69exYgRI6DVaqHT6fDYY48hKSnJvH7hwoXo3LkzvvnmGzRt2hQuLi54/PHHkZmZad5mw4YN6NChAxwcHODu7o6QkBBkZ2fX0qdJRERyY6giIqI698EHHyA4OBhPP/00EhISkJCQgICAAKSlpWHgwIHo0qUL/vjjD+zYsQNJSUl47LHHLPZfs2YN1Go1Dh06hE8++QQAoFAo8OGHH+LcuXNYs2YN9u7di//+978AgD59+mD58uXQ6XTm882ZM6dUXUajESNGjEBqaioOHDiA6OhoXL58GWPGjLHYLjY2Fps3b8bPP/+Mn3/+GQcOHMDSpUsBAAkJCRg7diyefPJJ/Pnnn9i/fz8efvhhiKJYGx8lERHZAF7+R0REdc7FxQVqtRqOjo7w9vY2L//oo4/QpUsXvPnmm+ZlX331FQICAvD333+jVatWAICWLVvi7bfftjhm8fuzmjZtisWLF2PatGn4+OOPoVar4eLiAkEQLM5X0p49e3DmzBnExcUhICAAAPD111+jffv2OHbsGHr06AFACl+RkZFwdnYGAEyYMAF79uzBG2+8gYSEBBQWFuLhhx9GkyZNAAAdOnSowadFRES2jj1VRERkM06dOoV9+/ZBq9WapzZt2gCQeodMunXrVmrf3bt3Y9CgQfDz84OzszMmTJiAW7duIScnp9Ln//PPPxEQEGAOVADQrl07uLq64s8//zQva9q0qTlQAYCPjw+Sk5MBAJ06dcKgQYPQoUMHPProo/j8889x+/btyn8IRERU7zBUERGRzcjKykJYWBhiYmIsposXL6Jfv37m7ZycnCz2u3LlCh588EF07NgRP/zwA44fP46VK1cCkAaysDaVSmUxLwgCjEYjAECpVCI6Ohrbt29Hu3btsGLFCrRu3RpxcXFWr4OIiGwDQxUREclCrVbDYDBYLOvatSvOnTuHpk2bokWLFhZTySBV3PHjx2E0GvHee++hd+/eaNWqFeLj4ys8X0lt27bFtWvXcO3aNfOy8+fPIy0tDe3atav0exMEAX379kVERAROnjwJtVqNTZs2VXp/IiKqXxiqiIhIFk2bNsXRo0dx5coVpKSkwGg0Yvr06UhNTcXYsWNx7NgxxMbGYufOnZg8efIdA1GLFi2g1+uxYsUKXL58Gd988415AIvi58vKysKePXuQkpJS5mWBISEh6NChA8aPH48TJ07g999/x8SJE9G/f3907969Uu/r6NGjePPNN/HHH3/g6tWr2LhxI27evIm2bdtW7QMiIqJ6g6GKiIhkMWfOHCiVSrRr1w4eHh64evUqfH19cejQIRgMBgwZMgQdOnTArFmz4OrqCoWi/P+yOnXqhGXLluGtt97CPffcg7Vr12LJkiUW2/Tp0wfTpk3DmDFj4OHhUWqgC0DqYdqyZQvc3NzQr18/hISEoHnz5vj+++8r/b50Oh0OHjyI4cOHo1WrVpg3bx7ee+89DBs2rPIfDhER1SuCyDFeiYiIiIiIqo09VURERERERDXAUEVERERERFQDDFVEREREREQ1wFBFRERERERUAwxVRERERERENcBQRUREREREVAMMVURERERERDXAUEVERERERFQDDFVEREREREQ1wFBFRERERERUAwxVRERERERENfD/DhA6RpYTec8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1000x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Train model\n",
    "\n",
    "iterations = np.empty(int(max_iters/eval_iters))\n",
    "iterations[:] = np.nan\n",
    "train_loss = np.empty(int(max_iters/eval_iters))\n",
    "train_loss[:] = np.nan\n",
    "val_loss = np.empty(int(max_iters/eval_iters))\n",
    "val_loss[:] = np.nan\n",
    "\n",
    "for i in range(max_iters):\n",
    "    train_one_batch(model, optimizer)\n",
    "    if i % eval_iters == 0:\n",
    "        out = estimate_loss()\n",
    "        iterations[int(i/eval_iters)] = i\n",
    "        train_loss[int(i/eval_iters)] = torch.exp(out['train'])\n",
    "        val_loss[int(i/eval_iters)] = torch.exp(out['val'])\n",
    "        print(f\"First {i} batches: training perplexity: {torch.exp(out['train']):.3f}, validation perplexity: {torch.exp(out['val']):.3f}\")\n",
    "        \n",
    "# Visualize results\n",
    "plt.figure(figsize=(10, 5))\n",
    "plt.plot(iterations, train_loss, label='Training Perplexity', color='blue')\n",
    "plt.plot(iterations, val_loss, label='Validation Perplexity', color='red')\n",
    "plt.title('Training and Validation Perplexity')\n",
    "plt.xlabel('Iterations')\n",
    "plt.ylabel('Perplexity')\n",
    "plt.legend()\n",
    "plt.grid(True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "119c8322-ea37-48e3-84d9-9feceddc192b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Hello, my name is teaches;\n",
      "He friend crought?\n",
      "\n",
      "Second Kingbolant:\n",
      "By the joyques, Watchmond Mybract thy begrea\n",
      "Good v\n"
     ]
    }
   ],
   "source": [
    "# Output some text\n",
    "\n",
    "model.eval()\n",
    "prompt = \"Hello, my name is\"\n",
    "max_new_tokens = 100\n",
    "ix = torch.tensor([encode(prompt)], dtype=torch.long)\n",
    "X =  ix.to('cuda')\n",
    "output = model.generate(X, max_new_tokens)\n",
    "[output] = output.tolist()\n",
    "response = decode(output)\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60f4a4e5-75da-457d-b481-95368dfc4be7",
   "metadata": {},
   "source": [
    "Increasing any of the three hyperparameters lead to lower perplexity on the validation set, with the number of heads and number of decoder blocks resulting in slightly larger improvements. The training time increases notably as the number of model paramters grows. None of the generated texts make sense, and only some of the words generated are real words."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "datascience/conda-2023-01-10",
   "language": "python",
   "name": "conda-2023-01-10"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}