9_nlp_with_tensorflow/5_using_pretrained_bert_for_text_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d7d558d1",
   "metadata": {},
   "source": [
    "<table align=\"left\">\n",
    "  <td>\n",
    "    <a href=\"https://colab.research.google.com/github/nyandwi/machine_learning_complete/blob/main/9_nlp_with_tensorflow/5_using_pretrained_bert_for_text_classification.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "  </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b55d6622",
   "metadata": {},
   "source": [
    "*This notebook was created by [Jean de Dieu Nyandwi](https://twitter.com/jeande_d) for the love of machine learning community. For any feedback, errors or suggestion, he can be reached on email (johnjw7084 at gmail dot com), [Twitter](https://twitter.com/jeande_d), or [LinkedIn](https://linkedin.com/in/nyandwi).*"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "304ad719",
   "metadata": {
    "id": "304ad719"
   },
   "source": [
    "<a name='0'></a>\n",
    "# Using Pretrained BERT for Text Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6a095ea",
   "metadata": {
    "id": "b6a095ea"
   },
   "source": [
    "## Contents \n",
    "\n",
    "* [1. Intro to BERT](#1)\n",
    "* [2. In Practice: Finetuning a Pretrained BERT](#2)\n",
    "   * [2.1 Getting the data](#2-1)\n",
    "   * [2.2 Preparing the data](#2-2)\n",
    "   * [2.3 Getting the BERT and its preprocesing model from TensorFlow Hub](#2-3)\n",
    "   * [2.4 Building a news classifier](#2-4)\n",
    "   * [2.5 Training a finetuned BERT model](#2-4)\n",
    "   * [2.6 Visualizing the model results](#2-4)\n",
    "   * [2.7 Performing the inference on new texts](#2-4)\n",
    "\n",
    "* [3. Futher learning](#3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "NpH129c1ipzK",
   "metadata": {
    "id": "NpH129c1ipzK"
   },
   "source": [
    "<a name='1'></a>\n",
    "## 1. Intro to BERT"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "qxOWDkU-Z3jj",
   "metadata": {
    "id": "qxOWDkU-Z3jj"
   },
   "source": [
    "BERT stands for Bidirectional Encoder Representations from Transformers. It is one of the recent state of the art language models that can be used for variety of tasks such as question answering, text classification, machine translation and so forth. \n",
    "\n",
    "As it sounds and hence the name, BERT is based on Transformers. Transformers are a neural network type architecture that uses attention mechanisms([Attention is All you Need](https://arxiv.org/pdf/1706.03762.pdf))\n",
    "\n",
    "BERT is deeply bidirectional: it captures the full context of the input text (converted to tokens) from both directions. The traditional language models (e.g [Word2vec](https://arxiv.org/abs/1301.3781)) are undirectional and thus, they don't take into account the context of each occurence of a given word. \n",
    "\n",
    "To quote its [paper](https://arxiv.org/pdf/1810.04805.pdf), BERT also uses a `next word prediction` task that jointly pretrains text-pair representations. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6b0vYfygj3nk",
   "metadata": {
    "id": "6b0vYfygj3nk"
   },
   "source": [
    "TensorFlow Hub contains many version of [BERT](https://tfhub.dev/google/collections/bert/1) such as: [Small BERT](https://tfhub.dev/google/collections/bert/1), [Albert](https://tfhub.dev/google/collections/albert/1), [Electra](https://tfhub.dev/google/collections/albert/1), [BERT Expert](https://tfhub.dev/google/collections/experts/bert/1). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pJsFn5Jukzvi",
   "metadata": {
    "id": "pJsFn5Jukzvi"
   },
   "source": [
    "<a name='2'></a>\n",
    "\n",
    "## 2. In Practice: Finetuning a Pretrained BERT"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hROK2TTUtEm5",
   "metadata": {
    "id": "hROK2TTUtEm5"
   },
   "source": [
    "We are going to use `ag_news_subset` dataset that is available in [TensorFlow datasets](https://www.tensorflow.org/datasets/catalog/ag_news_subset). TensorFlow datasets is a collection of awesome datasets that can be used right away with little preparations. \n",
    "\n",
    "AG is a collection of more than 1 million news articles gathered from more than 2000 news sources by ComeToMyHead in more than 1 year of activity. \n",
    "\n",
    "The AG dataset contains 4 classes that are: World(0), Sports(1), Business(2), Sci/Tech(3). The total number of training samples is 120,000 and testing 7,600. Each class contains 30,000 training samples and 1,900 testing samples. \n",
    "\n",
    "You can learn more about the dataset [here](http://groups.di.unipi.it/~gulli/AG_corpus_of_news_articles.html).\n",
    "\n",
    "\n",
    "Let's get the dataset from TensorFlow datasets."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "BwdYMcP3vb4K",
   "metadata": {
    "id": "BwdYMcP3vb4K"
   },
   "source": [
    "<a name='2-1'></a>\n",
    "\n",
    "## 2.1 Getting the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wkrr7SMgwXwt",
   "metadata": {
    "id": "wkrr7SMgwXwt"
   },
   "source": [
    "#### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "a2PFMTFVwPkY",
   "metadata": {
    "id": "a2PFMTFVwPkY"
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import tensorflow_datasets as tfds\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow_hub as hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "nSyKsBxnXEY3",
   "metadata": {
    "id": "nSyKsBxnXEY3"
   },
   "source": [
    "The dataset that we are going to download has only one version so far. Let's specify it so that when it is updated, our lab will not be affected. If loading the data fails, run the cell again. It happens sometime when loading datasets from TF datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "6JIKIZDRQwVV",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 316,
     "referenced_widgets": [
      "ffe4fbd6097e414b99cb4c4780557553",
      "40ad8f616a684bedb5dc40639541e466",
      "515f8749c3cd420d85289ac142257240",
      "377499c0d2c94299926b820deee71819",
      "d6a78c6422574503a85c7bbc8be3321c",
      "e9c1537f77814146b98ed4e0772626e2",
      "feb1ffd9c3504e238531a5d8b3ee9f8a",
      "75c874a4d3884c8cbbc26ffa57d97b6f",
      "64fe9627b18644668b6f386b72f9ec98",
      "3211dd4a60b04381a2ce78c2278b5023",
      "c17648de297344679cdec0b3c734b223",
      "2c48c62afa474d1a9a62b6daf6931d40",
      "12b0c1fcd12c4d4fa1dfdc1593ff3972",
      "d8deccabcafc4e649e46fca288bf95dd",
      "ed825f71daa6460da541326d01b098e9",
      "25f4bb62d1ab47779bfbe78dfe58fdac",
      "a13335e07e85455e8366cb35077de77e",
      "1bcd2619383e41c6ae1ae9108530d3e8",
      "9cd6636a115c404099bf30a71eb44738",
      "a9a512b8b9254d91a5530852bd8033df",
      "85b1127de5c245f1ad61b33bae41279e",
      "b99b0cff30294582ae1e66e2a633869b",
      "2ac92ba33fcd419991df92f41de01c65",
      "cde24def6f35489391282d157e6a3550",
      "66370f990ba64dc1a28c984e344b23ac",
      "66e634e821fc4c55a84f314a472170f0",
      "d14cd8661cae4f7a8c6785f56b398dc6",
      "b0a973b98e5d4541a0c677c874e64c50",
      "354f3b9cc77c4615a762df558e5c5267",
      "046226810bd141599517fba1ded1ed8f",
      "aa33d0e6a828439db44ff7bb83e728a3",
      "a3c813936264408d99f3119fd9499df2",
      "97b8666b250d46c9b8d8f463464ce793",
      "f9ed0b3303a549b6a1620a7b0f43ad45",
      "f9b9ef9699704cbbb8920fef2e60fe71",
      "8fa058e1bdd7435b890054f3f0aa06df",
      "d63a80ebc12a4adfbb6238033f91c638",
      "b18fb0dc468b42ea8afd392b9cb4f727",
      "27ffec67ddd849778376db18144d6f38",
      "c4b2124409f64d0ca42ffb44056504de",
      "6c57f7f09d7d4e6a89e4b02ce378aae0",
      "b67c6831b62e4e28ad081517a6ea06eb",
      "0ba653c10cc449368a8910a5854ddb71",
      "f11bb92965034840b21e3a050c7ad8a6",
      "8ffdc8e410ff467f826aa116f517658e",
      "6578461a0b3040418cf2fec36cc26bd4",
      "68f17020d75d4e2e834536fa9838a76b",
      "7fbfd0d4356349f99215c963d9d0f58b",
      "3fb758f51cb2447ba8e5767fcc95ea2b",
      "ea1ab1d0e9244801852bfd6840d8cbc4",
      "c1cf085a00d148b4b583950ac8f06073",
      "2fc4685daa1f4c8dbd3833fb607aa650",
      "8e85031ac9fc4d9d9f6a55e8172e4df6",
      "0cc6e536dabf4e3397f1d8db0765bb63",
      "f704ed4f5e254d04b2d8d1fb36f77e1a",
      "566bb79ab07f4e208fc43e0643d76bc3",
      "0ca9a37cd38b4ce8ba1d6535c2d85712",
      "85842ecd8d1f44a58fbcea48c39b904b",
      "d3a0f7aba98d4742a8cdcb7c315390c3",
      "d25d9a0a261b4c2db8f7020b7ec2cc9f",
      "7ee38e0deb744620a1eb7c866c24f640",
      "6e8761c0b1ad41f29dd174a2d4373fe5",
      "69e2be6bc02b4283aeedc41683d2c75d",
      "519c55322a6940e789892928e9e62e24",
      "f0ed3f707f0140af8a7dfcb5b843254c",
      "5f5ba5d5d3be47b69202839d3ce3dd27",
      "80af01b6d7a54fa99744a536f5580529",
      "fe9f20e182a2454c804e68c317834c96",
      "7374cef848094e08bf146366013931bf",
      "d4cb8bcdf308470c9d7a5949b5181736",
      "feca4ddf88df4f72ac3dfd6fb124ccf3",
      "4ce8e034694f4ae3a84560d66705ddab",
      "0d8bf4e088a849b6a0fb83f7bbeba65d",
      "9c3c2794090b410da00dd380ae00e34e",
      "a50d20a0b4fe4bd2bcdcff21b9eb64ba",
      "98e6f46668294d71bea96540e5cc3122",
      "cebb666792534b11ab7696da849ee17d"
     ]
    },
    "id": "6JIKIZDRQwVV",
    "outputId": "c25b028f-d238-4abb-e5ab-46d25b6dee3c"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mDownloading and preparing dataset ag_news_subset/1.0.0 (download: 11.24 MiB, generated: 35.79 MiB, total: 47.03 MiB) to /root/tensorflow_datasets/ag_news_subset/1.0.0...\u001b[0m\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ffe4fbd6097e414b99cb4c4780557553",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dl Completed...: 0 url [00:00, ? url/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2c48c62afa474d1a9a62b6daf6931d40",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dl Size...: 0 MiB [00:00, ? MiB/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2ac92ba33fcd419991df92f41de01c65",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Extraction completed...: 0 file [00:00, ? file/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f9ed0b3303a549b6a1620a7b0f43ad45",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shuffling and writing examples to /root/tensorflow_datasets/ag_news_subset/1.0.0.incompleteIWSQ9J/ag_news_subset-train.tfrecord\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8ffdc8e410ff467f826aa116f517658e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/120000 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "566bb79ab07f4e208fc43e0643d76bc3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "0 examples [00:00, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shuffling and writing examples to /root/tensorflow_datasets/ag_news_subset/1.0.0.incompleteIWSQ9J/ag_news_subset-test.tfrecord\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "80af01b6d7a54fa99744a536f5580529",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/7600 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mDataset ag_news_subset downloaded and prepared to /root/tensorflow_datasets/ag_news_subset/1.0.0. Subsequent calls will reuse this data.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "(train_data, val_data), info = tfds.load('ag_news_subset:1.0.0', #version 1.0.0\n",
    "                                         split=('train', 'test'),\n",
    "                                         with_info=True, \n",
    "                                         as_supervised=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "378IQW1eSVlz",
   "metadata": {
    "id": "378IQW1eSVlz"
   },
   "source": [
    "Now that we have the data, we can look into some examples. This is always a good practice. \n",
    "\n",
    "Printing `info` that we loaded with the dataset, we can see pretty much everything we would need to know about the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "kEVqTS2aRiOF",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "kEVqTS2aRiOF",
    "outputId": "8ef7c8af-32ad-44e4-fefb-9356fe5709e3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tfds.core.DatasetInfo(\n",
      "    name='ag_news_subset',\n",
      "    version=1.0.0,\n",
      "    description='AG is a collection of more than 1 million news articles.\n",
      "News articles have been gathered from more than 2000  news sources by ComeToMyHead in more than 1 year of activity.\n",
      "ComeToMyHead is an academic news search engine which has been running since July, 2004.\n",
      "The dataset is provided by the academic comunity for research purposes in data mining (clustering, classification, etc),\n",
      "information retrieval (ranking, search, etc), xml, data compression, data streaming,\n",
      "and any other non-commercial activity.\n",
      "For more information, please refer to the link http://www.di.unipi.it/~gulli/AG_corpus_of_news_articles.html .\n",
      "\n",
      "The AG's news topic classification dataset is constructed by Xiang Zhang (xiang.zhang@nyu.edu) from the dataset above.\n",
      "It is used as a text classification benchmark in the following paper:\n",
      "Xiang Zhang, Junbo Zhao, Yann LeCun. Character-level Convolutional Networks for Text Classification. Advances in Neural Information Processing Systems 28 (NIPS 2015).\n",
      "\n",
      "The AG's news topic classification dataset is constructed by choosing 4 largest classes from the original corpus.\n",
      "Each class contains 30,000 training samples and 1,900 testing samples.\n",
      "The total number of training samples is 120,000 and testing 7,600.',\n",
      "    homepage='https://arxiv.org/abs/1509.01626',\n",
      "    features=FeaturesDict({\n",
      "        'description': Text(shape=(), dtype=tf.string),\n",
      "        'label': ClassLabel(shape=(), dtype=tf.int64, num_classes=4),\n",
      "        'title': Text(shape=(), dtype=tf.string),\n",
      "    }),\n",
      "    total_num_examples=127600,\n",
      "    splits={\n",
      "        'test': 7600,\n",
      "        'train': 120000,\n",
      "    },\n",
      "    supervised_keys=('description', 'label'),\n",
      "    citation=\"\"\"@misc{zhang2015characterlevel,\n",
      "        title={Character-level Convolutional Networks for Text Classification},\n",
      "        author={Xiang Zhang and Junbo Zhao and Yann LeCun},\n",
      "        year={2015},\n",
      "        eprint={1509.01626},\n",
      "        archivePrefix={arXiv},\n",
      "        primaryClass={cs.LG}\n",
      "    }\"\"\",\n",
      "    redistribution_info=,\n",
      ")\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(info)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "03IBh4d1S1eJ",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "03IBh4d1S1eJ",
    "outputId": "dc971fd9-14f1-47aa-b36c-2b87f561571d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The news are grouped into 4 classes that are :['World', 'Sports', 'Business', 'Sci/Tech']\n"
     ]
    }
   ],
   "source": [
    "# Displaying the classes\n",
    "\n",
    "class_names = info.features['label'].names\n",
    "num_classes = info.features['label'].num_classes\n",
    "\n",
    "print(f'The news are grouped into {num_classes} classes that are :{class_names}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "uUzqSrGwY7dU",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "uUzqSrGwY7dU",
    "outputId": "40230be6-3bc1-4506-aa19-f16e2dd2f39e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number of training samples: 120000 \n",
      "The number of validation samples: 7600\n"
     ]
    }
   ],
   "source": [
    "num_train = info.splits['train'].num_examples\n",
    "num_val = info.splits['test'].num_examples\n",
    "\n",
    "print(f'The number of training samples: {num_train} \\nThe number of validation samples: {num_val}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "_AGay5wWZ_xs",
   "metadata": {
    "id": "_AGay5wWZ_xs"
   },
   "source": [
    "We can also display the first 10 news samples. We can use [`tfds.as_dataframe`](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe) to display them as dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "kGj0TRaXaHG1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 359
    },
    "id": "kGj0TRaXaHG1",
    "outputId": "a301ab2d-8a29-455f-bd13-248bbd4438f5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>description</th>\n",
       "      <th>label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>b'AMD #39;s new dual-core Opteron chip is desi...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b'Reuters - Major League Baseball\\\\Monday anno...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>b'President Bush #39;s  quot;revenue-neutral q...</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>b'Britain will run out of leading scientists u...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>b'London, England (Sports Network) - England m...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>b'TOKYO - Sony Corp. is banking on the \\\\$3 bi...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>b'Giant pandas may well prefer bamboo to lapto...</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>b'VILNIUS, Lithuania - Lithuania #39;s main pa...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>b'Witnesses in the trial of a US soldier charg...</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>b'Dan Olsen of Ponte Vedra Beach, Fla., shot a...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         description  label\n",
       "0  b'AMD #39;s new dual-core Opteron chip is desi...      3\n",
       "1  b'Reuters - Major League Baseball\\\\Monday anno...      1\n",
       "2  b'President Bush #39;s  quot;revenue-neutral q...      2\n",
       "3  b'Britain will run out of leading scientists u...      3\n",
       "4  b'London, England (Sports Network) - England m...      1\n",
       "5  b'TOKYO - Sony Corp. is banking on the \\\\$3 bi...      0\n",
       "6  b'Giant pandas may well prefer bamboo to lapto...      3\n",
       "7  b'VILNIUS, Lithuania - Lithuania #39;s main pa...      0\n",
       "8  b'Witnesses in the trial of a US soldier charg...      0\n",
       "9  b'Dan Olsen of Ponte Vedra Beach, Fla., shot a...      1"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "news_df = tfds.as_dataframe(train_data.take(10), info)\n",
    "\n",
    "news_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ksGa0Z4JaJDx",
   "metadata": {
    "id": "ksGa0Z4JaJDx"
   },
   "source": [
    "Let's display some full news."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "7Dq5HE_2Y2tm",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "7Dq5HE_2Y2tm",
    "outputId": "c412b9af-7ccd-41bd-9ce0-4c35d3877950"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample news 0\n",
      "   Label: 3 World\n",
      "   Description: b'AMD #39;s new dual-core Opteron chip is designed mainly for corporate computing applications, including databases, Web services, and financial transactions.'\n",
      "----------\n",
      "\n",
      "Sample news 1\n",
      "   Label: 1 Sports\n",
      "   Description: b'Reuters - Major League Baseball\\\\Monday announced a decision on the appeal filed by Chicago Cubs\\\\pitcher Kerry Wood regarding a suspension stemming from an\\\\incident earlier this season.'\n",
      "----------\n",
      "\n",
      "Sample news 2\n",
      "   Label: 2 Business\n",
      "   Description: b'President Bush #39;s  quot;revenue-neutral quot; tax reform needs losers to balance its winners, and people claiming the federal deduction for state and local taxes may be in administration planners #39; sights, news reports say.'\n",
      "----------\n",
      "\n",
      "Sample news 3\n",
      "   Label: 3 Sci/Tech\n",
      "   Description: b'Britain will run out of leading scientists unless science education is improved, says Professor Colin Pillinger.'\n",
      "----------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i in range (0,4):\n",
    "\n",
    "  print(f\"Sample news {i}\\n \\\n",
    "  Label: {news_df['label'][i]} {(class_names[i])}\\n \\\n",
    "  Description: {news_df['description'][i]}\\n----------\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "omL7uadpazuc",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "omL7uadpazuc",
    "outputId": "c178f9a3-45b5-4514-df0e-9de2989d25b0"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['description', 'label'], dtype='object')"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "news_df.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "F-jPA6TEd3sw",
   "metadata": {
    "id": "F-jPA6TEd3sw"
   },
   "source": [
    "Now that we understand the data, let's prepare it before we can use LSTMs to classify the news.  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "EKjyyw21bRzM",
   "metadata": {
    "id": "EKjyyw21bRzM"
   },
   "source": [
    "<a name='2-2'></a>\n",
    "\n",
    "## 2.2 Preparing the Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "w-KMfmo0dOWc",
   "metadata": {
    "id": "w-KMfmo0dOWc"
   },
   "source": [
    "For this practice of using a pretrained BERT, we will not need to vectorize the data. We will feed the sentences directly to the model. \n",
    "\n",
    "Let's just only shuffle and batch the training data. For validation data, we don't shuffle. We only batch it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "i3e-BTCUfmdr",
   "metadata": {
    "id": "i3e-BTCUfmdr"
   },
   "outputs": [],
   "source": [
    "buffer_size = 1000\n",
    "batch_size = 32\n",
    "\n",
    "train_data = train_data.shuffle(buffer_size)\n",
    "train_data = train_data.batch(batch_size).prefetch(1)\n",
    "val_data = val_data.batch(batch_size).prefetch(1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "E0x15xiSogyS",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "E0x15xiSogyS",
    "outputId": "b1b5c652-56aa-40c9-b818-25de49ee6ea1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample news\n",
      "----\n",
      " [b'RAMALLAH: Palestinian leader Yasser Arafat is  quot;very, very sick quot; and the ex-guerrilla who for decades symbolised a struggle for statehood is slipping in and out of consciousness, officials and medics said today.'\n",
      " b'AOL said it was intent on making travel an important part of its content expansion, which increasingly includes general Web properties that all Internet users can access as well as content just '\n",
      " b'AP - WHAT WAS FOUND: A microscopic diamond embedded in kimberlite, the molten rock in which diamonds are found, in Montana.'\n",
      " b'The US military in Iraq has denied reports that the former Iraqi leader Saddam Hussein is on hunger strike. But a spokesman told the BBC that some of his former aides who are also in custody, including  quot;Chemical '] \n",
      "----\n",
      "Corresponding labels: [0 2 3 0]\n"
     ]
    }
   ],
   "source": [
    "for news, label in train_data.take(1):\n",
    "\n",
    "  print(f'Sample news\\n----\\n {news.numpy()[:4]} \\n----\\nCorresponding labels: {label.numpy()[:4]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "B0vO0wXq7FO8",
   "metadata": {
    "id": "B0vO0wXq7FO8"
   },
   "source": [
    "<a name='2-3'></a>\n",
    "\n",
    "## 2.3 Getting the BERT Model and its Preprocessing Model from TF Hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "kUnQsfjf7eAf",
   "metadata": {
    "id": "kUnQsfjf7eAf"
   },
   "source": [
    "There many versions of BERT that are optimized for different use cases. The choice of a given version is going to depend on computation time & speed, accuracy, and the size of that particular BERT version. \n",
    "\n",
    "For our case, we can use [SmallBERT](https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3). It is also worth mentioning that each BERT model has its preprocessing model that will convert the texts into tokens. The BERT models and their corresponding preprocesing models are pretty detailed on [TensorFlow Hub BERT models page](https://tfhub.dev/google/collections/bert/1). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "-ZrJ7X2x9ZQV",
   "metadata": {
    "id": "-ZrJ7X2x9ZQV"
   },
   "source": [
    "The preprocessing model use [TensorFlow Text](https://www.tensorflow.org/text) operations. We have to install it. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "uM8eTKO79YkG",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "uM8eTKO79YkG",
    "outputId": "d1ca81b0-c26f-4030-9d70-49fd0f47d1bb"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow_text\n",
      "  Downloading tensorflow_text-2.6.0-cp37-cp37m-manylinux1_x86_64.whl (4.4 MB)\n",
      "\u001b[K     |████████████████████████████████| 4.4 MB 5.1 MB/s \n",
      "\u001b[?25hRequirement already satisfied: tensorflow-hub>=0.8.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow_text) (0.12.0)\n",
      "Requirement already satisfied: tensorflow<2.7,>=2.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow_text) (2.6.0)\n",
      "Requirement already satisfied: clang~=5.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (5.0)\n",
      "Requirement already satisfied: absl-py~=0.10 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (0.12.0)\n",
      "Requirement already satisfied: google-pasta~=0.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (0.2.0)\n",
      "Requirement already satisfied: flatbuffers~=1.12.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (1.12)\n",
      "Requirement already satisfied: termcolor~=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (1.1.0)\n",
      "Requirement already satisfied: six~=1.15.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (1.15.0)\n",
      "Requirement already satisfied: tensorboard~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (2.6.0)\n",
      "Requirement already satisfied: tensorflow-estimator~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (2.6.0)\n",
      "Requirement already satisfied: wheel~=0.35 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (0.37.0)\n",
      "Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (3.17.3)\n",
      "Requirement already satisfied: keras-preprocessing~=1.1.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (1.1.2)\n",
      "Requirement already satisfied: wrapt~=1.12.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (1.12.1)\n",
      "Requirement already satisfied: typing-extensions~=3.7.4 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (3.7.4.3)\n",
      "Requirement already satisfied: astunparse~=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (1.6.3)\n",
      "Requirement already satisfied: gast==0.4.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (0.4.0)\n",
      "Requirement already satisfied: numpy~=1.19.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (1.19.5)\n",
      "Requirement already satisfied: h5py~=3.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (3.1.0)\n",
      "Requirement already satisfied: keras~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (2.6.0)\n",
      "Requirement already satisfied: opt-einsum~=3.3.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (3.3.0)\n",
      "Requirement already satisfied: grpcio<2.0,>=1.37.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow<2.7,>=2.6.0->tensorflow_text) (1.39.0)\n",
      "Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py~=3.1.0->tensorflow<2.7,>=2.6.0->tensorflow_text) (1.5.2)\n",
      "Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (2.23.0)\n",
      "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (1.8.0)\n",
      "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (0.4.5)\n",
      "Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (57.4.0)\n",
      "Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (3.3.4)\n",
      "Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (1.34.0)\n",
      "Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (0.6.1)\n",
      "Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (1.0.1)\n",
      "Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (4.2.2)\n",
      "Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (4.7.2)\n",
      "Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (0.2.8)\n",
      "Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (1.3.0)\n",
      "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (4.6.4)\n",
      "Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (0.4.8)\n",
      "Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (2.10)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (2021.5.30)\n",
      "Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (3.0.4)\n",
      "Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (1.24.3)\n",
      "Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (3.1.1)\n",
      "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->markdown>=2.6.8->tensorboard~=2.6->tensorflow<2.7,>=2.6.0->tensorflow_text) (3.5.0)\n",
      "Installing collected packages: tensorflow-text\n",
      "Successfully installed tensorflow-text-2.6.0\n"
     ]
    }
   ],
   "source": [
    "!pip install tensorflow_text\n",
    "import tensorflow_text as text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "zb7F36kY6quG",
   "metadata": {
    "id": "zb7F36kY6quG"
   },
   "outputs": [],
   "source": [
    "bert_handle = 'https://tfhub.dev/tensorflow/small_bert/bert_en_uncased_L-4_H-512_A-8/2'\n",
    "preprocessing_model = 'https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "VGHlY82i_Gde",
   "metadata": {
    "id": "VGHlY82i_Gde"
   },
   "source": [
    "Let's wrap `preprocessing_model` into a [KerasLayer](https://www.tensorflow.org/hub/api_docs/python/hub/KerasLayer) and test it on some sample sentences. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "-8iJ0dhE_WHh",
   "metadata": {
    "id": "-8iJ0dhE_WHh"
   },
   "outputs": [],
   "source": [
    "preprocess_layer = hub.KerasLayer(preprocessing_model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "IYjYF5jDEEdk",
   "metadata": {
    "id": "IYjYF5jDEEdk"
   },
   "source": [
    "The output of [preprocess model](https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3) is made of 3 elements or keys that are:   `input_word_ids`, `input_mask`, `input_type_ids`. Here are what they mean.\n",
    "\n",
    "* `input_word_ids`: has the token ids of the input sequences.\n",
    "* `input_mask`: has value 1 at the position of all input tokens present before padding and value 0 for the padding tokens.\n",
    "\n",
    "* `input_type_ids`: has the index of the input segment that gave rise to the input token at the respective position. The first input segment (index 0) includes the start-of-sequence token and its end-of-segment token. The second segment (index 1, if present) includes its end-of-segment token. Padding tokens get index 0 again.\n",
    "\n",
    "Source: [TF Hub doc](https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "NpLEBPAK_sEn",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "NpLEBPAK_sEn",
    "outputId": "23141b2a-b577-4baf-c0f3-7dd340317ebe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Keys       : ['input_mask', 'input_word_ids', 'input_type_ids']\n",
      "Shape      : (1, 128)\n",
      "Word Ids   : [  101  6627 11256  1024  1996]\n",
      "Input Mask : [1 1 1 1 1]\n",
      "Type Ids   : [0 0 0 0 0]\n"
     ]
    }
   ],
   "source": [
    "sample_news = ['Tech rumors: The tech giant Apple is working on its self driving car']\n",
    "\n",
    "preprocessed_news = preprocess_layer(sample_news)\n",
    "\n",
    "print(f'Keys       : {list(preprocessed_news.keys())}')\n",
    "print(f'Shape      : {preprocessed_news[\"input_word_ids\"].shape}')\n",
    "print(f'Word Ids   : {preprocessed_news[\"input_word_ids\"][0, :5]}')\n",
    "print(f'Input Mask : {preprocessed_news[\"input_mask\"][0, :5]}')\n",
    "print(f'Type Ids   : {preprocessed_news[\"input_type_ids\"][0, :5]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fQqja4ZQF0W1",
   "metadata": {
    "id": "fQqja4ZQF0W1"
   },
   "source": [
    "We can also see what kinds of the output the BERT model will give if we feed it the preprocessed texts. \n",
    "\n",
    "The bert model expect 3 input tensors: `input_word_ids`, `input_mask`, `input_type_ids`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "-hrBJ4HQAzCg",
   "metadata": {
    "id": "-hrBJ4HQAzCg"
   },
   "outputs": [],
   "source": [
    "bert_model = hub.KerasLayer(bert_handle)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tH8A0H_IHHE3",
   "metadata": {
    "id": "tH8A0H_IHHE3"
   },
   "source": [
    "The `bert_model` returns 2 main keys: `pooled_output`, `sequence_output`. \n",
    "\n",
    "Here are what they mean:\n",
    "\n",
    "* `pooled_output` represents the  input sequence.\n",
    "* `sequence_output` denotes each input token in the context. \n",
    "\n",
    "Any of those keys can be used as input to the rest of the model. We will see that later.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "G0ONfy_dGg2s",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "G0ONfy_dGg2s",
    "outputId": "8ce38ddf-ea7b-41aa-e54a-6d19149134af"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Pooled output shape:(1, 512)\n",
      "Pooled output values:[ 0.944598    0.9861868  -0.15102872  0.17372097 -0.6424564 ]\n",
      "Sequence output shape:(1, 128, 512)\n",
      "Sequence output values:[[-0.60826176  0.2623035   1.2436793  ... -0.8821719   0.36095393\n",
      "  -0.00184657]\n",
      " [ 0.33820462  0.07993765  1.1374032  ... -1.6438915  -1.0649079\n",
      "  -0.7371688 ]\n",
      " [-0.6054905  -0.14994524  0.8464851  ... -0.74730563  0.84758896\n",
      "   0.13153419]\n",
      " [ 0.06868989  0.6969774   1.5265335  ... -0.7680908  -0.30986136\n",
      "   0.25017208]\n",
      " [-1.2314899   1.4334055  -0.37746784 ... -1.3110946  -0.01984588\n",
      "   0.7904873 ]]\n"
     ]
    }
   ],
   "source": [
    "bert_outputs = bert_model(preprocessed_news)\n",
    "\n",
    "print(f'Pooled output shape:{bert_outputs[\"pooled_output\"].shape}')\n",
    "print(f'Pooled output values:{bert_outputs[\"pooled_output\"][0, :5]}')\n",
    "print(f'Sequence output shape:{bert_outputs[\"sequence_output\"].shape}')\n",
    "print(f'Sequence output values:{bert_outputs[\"sequence_output\"][0, :5]}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ZfgJaX3bJzko",
   "metadata": {
    "id": "ZfgJaX3bJzko"
   },
   "source": [
    "Now that we understand what the ins and outs of [bert model](https://tfhub.dev/tensorflow/small_bert/bert_en_uncased_L-4_H-512_A-8/2) and its [preprocessor](https://tfhub.dev/tensorflow/bert_en_uncased_preprocess/3), let's build the rest of the model. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "k1_z6U5WA9Pq",
   "metadata": {
    "id": "k1_z6U5WA9Pq"
   },
   "source": [
    "<a name='2-4'></a>\n",
    "\n",
    "## 2.4 Building a News Classifier with BERT"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "wyPUdJ8aBQmZ",
   "metadata": {
    "id": "wyPUdJ8aBQmZ"
   },
   "source": [
    "Since the main purpose of this practice it to finetune BERT, we are going to add few more layers to it. \n",
    "\n",
    "We will add dropout for regularization, and dense layers for classification purpose. \n",
    "\n",
    "We have been using [Keras Sequential API](https://www.tensorflow.org/api_docs/python/tf/keras/Sequential). There is also a Functional API. This API type make it easy to build flexible models where you can arrange layers in a way you want. Also, Functional API is a suitable API type for the models that take multiple inputs and outputs. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5sqJYq7wBCWO",
   "metadata": {
    "id": "5sqJYq7wBCWO"
   },
   "outputs": [],
   "source": [
    "input_text = tf.keras.layers.Input(shape=(), dtype=tf.string, name='Input')\n",
    "\n",
    "# A preprocesing model before layer BERT\n",
    "preprocessing_layer = hub.KerasLayer(preprocessing_model, name='preprocessing_layer')\n",
    "\n",
    "bert_input = preprocessing_layer(input_text)\n",
    "\n",
    "# Getting a Bert model, set trainable to True\n",
    "bert_encoder = hub.KerasLayer(bert_handle, trainable=True, name='bert_encoder')\n",
    "\n",
    "bert_outputs = bert_encoder(bert_input)\n",
    "\n",
    "# For finetuning, we take pooled_output\n",
    "pooled_bert_output = bert_outputs['pooled_output']\n",
    "\n",
    "# Adding a dense layer \n",
    "\n",
    "dense_net = tf.keras.layers.Dense(16, activation='relu', name='fully_connected_layer')(pooled_bert_output)\n",
    "# Add dropout layer for regularization\n",
    "dense_net = tf.keras.layers.Dropout(0.2)(dense_net)\n",
    "\n",
    "# Last dense layer for classification purpose\n",
    "final_output = tf.keras.layers.Dense(4, activation='softmax', name='classifier')(dense_net)\n",
    "\n",
    "# Combine input and output\n",
    "\n",
    "news_classifier = tf.keras.Model(input_text, final_output)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bjMQN3ssJE9z",
   "metadata": {
    "id": "bjMQN3ssJE9z"
   },
   "source": [
    "We can display the model summary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "-UcdFImJIzRC",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "-UcdFImJIzRC",
    "outputId": "24cf89b5-6765-4701-d817-1aa9e755b5c6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"model\"\n",
      "__________________________________________________________________________________________________\n",
      "Layer (type)                    Output Shape         Param #     Connected to                     \n",
      "==================================================================================================\n",
      "Input (InputLayer)              [(None,)]            0                                            \n",
      "__________________________________________________________________________________________________\n",
      "preprocessing_layer (KerasLayer {'input_word_ids': ( 0           Input[0][0]                      \n",
      "__________________________________________________________________________________________________\n",
      "bert_encoder (KerasLayer)       {'encoder_outputs':  28763649    preprocessing_layer[0][0]        \n",
      "                                                                 preprocessing_layer[0][1]        \n",
      "                                                                 preprocessing_layer[0][2]        \n",
      "__________________________________________________________________________________________________\n",
      "fully_connected_layer (Dense)   (None, 16)           8208        bert_encoder[0][5]               \n",
      "__________________________________________________________________________________________________\n",
      "dropout (Dropout)               (None, 16)           0           fully_connected_layer[0][0]      \n",
      "__________________________________________________________________________________________________\n",
      "classifier (Dense)              (None, 4)            68          dropout[0][0]                    \n",
      "==================================================================================================\n",
      "Total params: 28,771,925\n",
      "Trainable params: 28,771,924\n",
      "Non-trainable params: 1\n",
      "__________________________________________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "news_classifier.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hJV8VbEDJKbh",
   "metadata": {
    "id": "hJV8VbEDJKbh"
   },
   "source": [
    "Or plot it with `tf.keras.utils.plot_model()`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "c2SVEGm_JJQP",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 53
    },
    "id": "c2SVEGm_JJQP",
    "outputId": "9d1ac06a-813d-4152-9934-15fe7b4ce32d"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABxUAAABLCAIAAAA5yk7FAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deXwTZf4H8Cc9klAoKe2WUtpSsIA1LUuhuLgtsotXcS0gLgWk3Ai6qxVEEFaBglW5FEtXxQKuBcshlfUolMtytAjiyiHYglCkJ4VCS++75PfH/JxXTCaTJzOTTBI+778gmcx8+8zzfJ8jMxOFTqcjAAAAAAAAAAAAAGDERe4AAAAAAAAAAAAAAOwU1k8BAAAAAAAAAAAAuGH9FAAAAAAAAAAAAIAb1k8BAAAAAAAAAAAAuGH9FAAAAAAAAAAAAICbm8H/6+rq9u/fL0so4MQ0Gs0TTzwhdxQWOHjwYE1NjdxRAICtjRw50tPTU+4oaOXn5+fl5ckdBYCDCQsL02q1ckdBCyNzAEeBIQQAelhwJhz1Wfd7SKNgDVqtVudQHCjvA4CE8vLy5E4/FkhMTJS7wAAcT2Jiotxt1wIYmQM4CgwhANDDgjMxrs+G158y4uLi4uLibBwcOKv58+fLHYIQgYGB69atkzsKALCRjIyMjIwMuaMQYt26dYGBgXJHAeAASktLHXRMgpE5gD3DEAIAPSw4E1P1mXv9VKvVog6BVJYvXy53CEJ07doVrQDg3uG4X0HHxMTgknkAGvn5+XKHIBBG5gD2DEMIAPSw4ExM1Wf8fhQAAAAAAAAAAAAAN6yfAgAAAAAAAAAAAHDD+ikAAAAAAAAAAAAAN6yfAgAAAAAAAAAAAHDD+ikAAAAAAAAAAAAAN6yfAgAAAAAAAAAAAHDD+ikAAAAAAAAAAAAAN6yfAgAAAAAAAAAAAHDD+ikAAAAAAAAAAAAAN6yfAgAAAAAAAAAAAHDD+ikAAAAAAAAAAAAAN6yfAgAAAAAAAAAAAHDD+ikAAAAAAAAAAAAAN6yfAgAAAAAAAAAAAHDD+ikAAAAAAAAAAAAAN6yfAgAAAAAAAAAAAHDD+ikAAAAAAAAAAAAAN6yfAgAAAAAAAAAAAHDD+ikA3EM++eSTfv36ubi4KBSKwsJCucMBsJ3o6Gi1Wq1QKD7++GO5Y7Frb7zxRkBAgEKh+MMf/iB3LAAC3bp1a9KkSV5eXp07d164cCHNR4z7R/SYNiM47eAcgVzQVwLokz0bC+j3QQBR66fFxcVDhw719PRUKBQKhcLf33/69OkSBWYLdht/bm7uwIEDmYmuQqHo2bPn8OHDq6ur5Y4LQKza2lqtVhsSEnLz5k1ZApg1a9aVK1eGDx8uy9EF0M8GLi4uvXv3Xr58OftuWloaM3hVq9WDBw++du2avBEiX9mz7777Tr/ygClvv/12WVlZ586dbXM4tHGwhjFjxpSUlBQWFq5Zs6akpITmI8b9o8P1mI5LcNpxuHNkkE8Yrq6uGo2mX79+f//73z/99NOGhga5wwTzbNxX2glUYDDFZtnY1GzauN+32bz7nmoXotZPe/XqderUqX379hFCfHx8ysvL09LSpInLJuw2/ocffvinn35KTk4mhERHR1+/fj0nJ8fLy0vuuBzV3r1779y5I3cUQAghOp1Op9PJHYUj0c8GUVFRhYWF+msr06dPnz59ularvXbt2pkzZ/r06SNvhMhXYhw+fPj69etyRwG2hjZ+T2ltbf36669bWlqsepQffvjh5MmTY8eO9fLyio+PVyqVVj0cAD2DfMIMC6urq0+cODFv3rzLly/PnDkzNDT022+/lTtSB3P58uUffvhB7iicHyqwPcvNzS0qKpI7CqvjnE1z9vs2m3ffU+3Cee7ff/3111etWiV3FMI5evz2bMmSJX5+fmPGjNm5c2djY6Pc4dzTNBrNxYsXr1696ufnJ3cszmDZsmVZWVnHjh3z9/eXOxYQa8OGDUFBQY888sjmzZut+pWPi4tVun70YtaANu5kWlpann76aT8/v+eee+7w4cMdHR3WOMqlS5cIIZ6enoQQLy+vrVu3WuMoYAA5UDBPT8+wsLAXX3zx3LlzCxYsKC0tjY2NzcnJkTsuR3Lu3LmhQ4f27dt32bJlFy9elDschyFJs3W+Cuyg2Sw9Pb1Pnz7Dhg378MMPb926JXc41sI5m+bs9+Wddztfu2A4yfppUVHR+++/L3cUwjl6/Pavra0tMzPz2Wef9fX1nTx58t69e9va2uQOCsQqKCiYPn36rl275A5EHklJSQcPHjxy5AiePOU07t69e/To0dmzZ1v1Kx+1Wi35Pp2vF3N3d5c7BLRxp1VTU/PJJ588+uijAQEBr7zyiuRXjZWXlxNCFAqFtLsFHpLkQHtIO/JydXVdu3btM88809LS8vzzz7e3t8sdkYO5evVqUlKSVquNiIhYs2ZNcXGxtY/o0JVW8qGLc1Rghx7R6XS6EydOvPTSS/7+/k8++eTWrVtra2vlDsoW7Lnfd452wbLK+mlGRoafn59CofjnP/+5b9++uLg4X1/fbt26xcfHszV427ZtPj4+CoVi5syZ06ZNCwgIUKlUoaGhycnJ+pcZL1myRKPRKBSKJUuWMK+8/fbb/v7+CoVi2LBh7DYRERHNzc3/+te/FApFYGAgIaSjo2PkyJGBgYF5eXmSB08ZP03wpuK3VHx8vFar9fLy8vDwCA8P37hxI/P6ggULlEqlQqFQKpUPPvgg8yyM9PT0kJAQ5pGvWVlZzJZFRUVTpkzp3r27h4fHwIED16xZc/fuXULI5s2bu3fvrlAoEhIS6urqVq5cqdVq161bJyBIGSkUCua8NDY2btu2LTY21tfXd+rUqZmZmSKv/qCsyTzFaKrkxe+5oqLi5ZdfDgkJUalUXl5eI0aM+Prrr43/hMbGxhUrVoSHh3fq1MnDwyMgIGDYsGFswzEVHiGkqalpyZIlffv2VavVfn5+0dHR3333Hc9bV65ciYiIcHV1VSgUBQUFzJaULY4Q8uuvv86aNYvZZ7du3QYNGhQaGtqpU6fx48eLOYMM8S2Ip6xMnSPBaYoQsmrVqm+//fbQoUOm7qIV1qJNlQPhPd2WQr7iwbRu9isfHx+fUaNGZWRktLa2SnWIlJSUkJCQLl26qNXqP/7xjykpKQZ391havGJ6MVPHos8MxFwSo8mEFRUVCQkJvXv3ViqVGo1mwIABTU1N9NGijRtAG6dUUVGRnJw8dOjQ4ODguXPnnj17VuQOz58//9BDD73xxhuEkNmzZzPPIBs5ciTNcJSSRd0iDTHtlzJR0OcTnjEPT6imciD/3ijTjmCyNEMxSY+VlJRECLl06ZLBlUoi+wv+FMd/shzOhQsXFi1aFBwcPGTIkPXr10v19EP+Ssufn/nbMuWsh2ZXYibgVqrAYiZr9CVjdleC11UcC1MsHR0dBw4cmDZtWvfu3UeNGrV161aprkjg77OMCRt3mXrLeDbN2e//+OOPnPNuYtuhrD7OxC5+FiBDYtf9HlMuiYmJOmq5ubmEEB8fH/0X//e//xFCtFrt7t27b9++fefOnRdeeIEQ8tprr7HbZGZmEkJiY2OvXr3a3Nz8yy+/PPvss4SQRYsW6e9qx44dhJA33niDfYW5Hpt9toJOp9u0aRMhZOXKlewrV65cYU+VpfHTBE8ZP03wnPHrdLoNGzYYb2mKSqVKS0u7efNmeXl5VFQUIeTAgQPMW++99x4hZNSoUfrb3759W61Wl5SUMP+9fv16jx49fH19T5061dTUtGLFCkLI66+/rl8gAwYMiIiIeOqpp/r167d27VqaqBharVar1dJvL7mIiAjjb2PYV3r27Pnyyy/n5ubqf8SimClrMmcx8pe8mD3fvn27d+/eQUFBx44da25uLiwsnDRpEiGE6W5Z9fX1kZGRarV6y5Yt9fX1VVVVBw4cCAgIOHLkiM5cxZgxY4aPj8/JkycbGxvPnDkTGRn52WefmX0rJiaGEHLlyhWD+PlbXHl5eY8ePYKCgo4fP97Q0HDhwoWIiAhCyLFjxyhPk76//OUvhJBr166xr4hsQWbLivMc0acpg2ywdu3axx57rKGhwdT2gls0TznwnFPjCPnZZ75KTEwkhOTl5dFsbA3jxo3j6aw1Gs2UKVO++eab9vZ2wTGvXLmSEJKQkFBWVtba2nrp0qX4+HhCyEsvvcRuI6x4OXsxs2iOZbYv5k9iNJnw1q1bffr06dKly65du5qampqbm0tKSjp16mQwsEEbd/Q2rhM0ypWWqQth2DHJAw88kJiYWFBQICZmpqVv2rSJfYVyOKrj6h+NX6HsFmmIb7+UiYJmM/6Kxx+qcQ7k3xtl2qFkfI50MjVDyqRnNp/cd999hJAlS5awr4jvL3hSHP/Oacg+hPj88895couLi0t0dHRqampNTY3gmGkqramKQdOWKWc9NLsSPAG3XgUWM1mjKRnKXQlbV6Ekew87Z84cnlbg6enJDKRbW1sFx8zfEegsnGMKm0rruGbTxv0+55a2HMoaM2gXkswCrJfYTdUN666fTpgwgX2loqKCOR/sK0wumDt3LvtKS0tLUFCQq6traWkp+6Kwdt7e3v7oo4/26NHjp59+sjR+muAp47fZ+umwYcPYfzMPvFiwYAHz39u3b6tUKjc3t5s3b7LbfPDBByNHjmT/y1THtLQ09pXevXt7eHjU1tayBTJ69Oi7d+/qdLqDBw86wfqpQUolhAQHBy9atOjSpUuWxkxZkzmLkb/kxew5ISGBEJKens5u1tbW5ufnp1Qqb9y4wb7IfFu1evVq/b/ohRdeYLoBnvA6Ojo6d+4cFxfHvrVz504mYfG8pTO9fsrf4pYuXUoIeffdd9lXvvjiC0LItGnTdJYz7ttEtiCdoEZEn6b0swFzT82IESPa2tpMbS+4RZsqB/5zqnOKfCX75Id//ZTzKx9h66cbNmxgX2lrawsJCSGEXLx4kXlFWPEKG23THMtsX8yfxGgy4YsvvkiM5hidO3c2GNigjTt6G9fZwezO7I2E7OOJIyMjk5OTb9y4YYfrp5TdIg3x7ZcyUdBsxl/x+EM1zoH8e6NMO5Q4109laYaUSc9sPnniiScIITNmzLAoHp7zy5/i+HdOQ/YhhKn1U4NRhEqlio2N3bJlS0NDg6Ux01RaUxWDpi1TznpodiV4Am69CixmskZTMpS7ujfXTw1069Ztzpw5ubm5d+/etTRm/o5AZ8kcU/BUWidi/dRmQ1lOBu1C/CzAqondVN1wo6lkkvD19SWE1NXV8WyjVCpjY2M3bNiQlZU1e/ZsMYdzdXWV8Be+aIInksZvKWYVmMFEy95P4ePjM3bs2J07d6anp8+fP595MT09/dVXX2U/wuTlp556in0lKipq+/bt33///eOPP8680qlTJ6brffzxx9kXKTU0NGRkZAj4uyTBfz+U7rd7H4qLi1evXr169erIyMjKykqNRiP4iDw1waAYmdsi+Ete2J579uxpsGc3N7cxY8Zs3Ljx66+/ZjuY9PR0QsjkyZP1d7V48eIuXboQ3orx6KOP6nS6nJycoqKi4OBgQsiECRPYzXjeomHc4kpLSwkh+neRMP9mXhdPZAsiQhuRpWnqgw8++Oabb5RK5ZEjRxYuXGjq+USCWzRPOYg8p/rsOV8dOHBA5P0pgl2/fp3nXTZTlZeXp6SkpKSk9O/fn2nmYri5uY0ePfr999/PysoKDQ0l1u8O9NEcSx9nX8yfxJgvWvgz4c6dOwkhEydOFB8t2rjZo8jexgkh+fn5co1JzN6gzd5ZdubMmdOnTy9YsIC5XEXCJ3iIR9kt0hDffo1RDtqNN+OvePyhGuPfG2XaEUOWZijV/Iv5FRT6s2O8B+Pzy5PiLN25KTIOIb7//nv+DZhRREtLy549e/bs2ZOQkMB8e0r/HDP6SmtcMYS1Zc5Zj7BdUbJeBWYIm6wZMy4ZqxaLRWTsYa9du0azWXV19caNGzdu3Ni7d29meZGepR0BETruknBIps82Q1lTDNqFJLMAGyR2A7ZbP2XojJ5gYqBPnz7kt18Qszdmgyf2Gv/s2bN37tyZlpbGDJIKCgquXr06evRo5t3GxsaysjJ3d3f9X6jw9vYmhJSVlUkSwK1btyR5SKUwlL+8wZ7f06dPE0IqKirefPPN119/3c1NSDOhqQnCSp5mz9XV1eXl5V26dDF4cF7v3r0JIezvctbW1hYVFXl6ehqswjAJiD88FxeX8ePHp6WlabXaSZMmTZs2jX10Ds9bFtFvcUFBQYQQ/QfhFxUVkd+vqFoPfwsiNmlEhJC8vLz6+vrs7OzU1NR//OMfycnJQ4YMYe6/1meNYKQ6pzTkzVfsTNL2lEolzWZsu7h8+fLly5cJIUlJSZs3b+7cubOw4zJ30/z666/EVjWZIfhY+pmBP4nRZMLy8vLKykpvb2/+HydFG5eQ7GOSjIwMGb/TpcTU8/b2duZJYUxleOaZZ+SO6/+Z7RZpiG+/PGgG7fqb8Vc8/lCN8e+NMu1YlezNkF9DQwMhhL2UQXx/wZPiJPxjZRxCWKq2tpZ54PLEiRMzMjK0Wi3/9mIqrZi2bDDrEZkWbMagAnMS+bfol4xdFYuMPSzlnJ1NC4WFhampqYSQL7/8ct68eaYeN8+ytCPgZ4OptAG7SuySzAJsk9gN2Hr91CxmAmmNXxy2DVnir6mpeeeddw4ePFhSUtLY2NjS0mKwwYgRI/r27XvhwoXTp09HRkamp6fHx8ezc3WmKre1tRnf5N7c3CxJhL6+vmvXrpVkVwIsXbq0srLS7Gbsb0xFRkaWlpZqNJply5YJPihNTRBW8jR7rq+vJ4QYfxXGdAzMu4QQ5kZCU8suZsPbsGFDYGDgxx9/vHnz5s2bNz/44IPbt2/v27cv/1vCTJ06dd26dSkpKdHR0YMGDSooKHjrrbeUSiVzM5FIIlsQsUkjIoSEhYUtXryYEPLCCy+cOnUqLS1t9uzZWq120KBB+psJDoa/HKQ6p3aer9atWyfXw/KTk5NPnDhhdjM2UzHXnx49enTp0qWCF0/Jb8NN5uo229RkhiTH4k9iNJmwpqaGmJvkSBWtWWjjYv46enFxcXFxcZLsylJNTU3Tpk2j2ZJp6W5ublFRUTk5OfPmzbOfxVNC0S3SEN9+JcRf8fhDtXRvlGlHDPtvhvyY78iZr/ekisdUipPwj5VxCHHy5EmLfjO9a9euISEhZ8+e3blzp9nFU0LdV3IS05YNZj02TguCGVRgTiL/Fv2SsatikbGH3bRp06FDh8xuxg6kmetPU1NTx44da3bxlJjrszgJHndJPpUmdpbYpQrGBondgN2tn9pgSGFVssQfGxt7/PjxtWvXMr9NdvToUf0LlQkhCoXiueeeW7x4cVpaWmRk5LZt27788kv2XSYLeHp6mn0sl2DMkymstHOz3nnnHZ532Rzaq1eviRMnzpgx4/777w8LCxN5UJqaIKzkafZs6raRO3fusO+y/6iqqhIWnlqtTkpKWrZs2bfffrtq1aqcnJzx48efOXOG/y1hQkJCDh069Nhjj8XGxtbX1/fo0WPIkCFbtmwxWFYQRmQLIjZpRAY2bNjw008/nT17duzYsadPn/bx8REfDH85SHVO7TxfxcTE0EwkrGHXrl0877KZyt/ff9y4cXFxccOGDVu+fPnRo0dFHre6upr89pWsLWuyJMfiT2I0mVCtVhOKm6PRxqU6CpG7jRNCtFqtXGOSuro6/vVTFxcX5hb+wYMHT5kyZeLEiZWVlWFhYZYuTVqb2W6Rhvj2KyH+iscMvUyFauneCgsLiZWfyWD/zZBHZWUlc3fF8OHDJYzHVIqT8I+VcQhh9oJrZhShUqkef/zxuLi4cePGrVmz5uzZs66urjT7p+wrOYlpywazHhunBWGMKzAnkX+LfsnYVbHI2MNS3mbu5eUVFxc3ZcqU6OjoixcvMpeg0uDvszgJHndJPpUmdpbYpQrGBondgIu0uxOPuQp94MCB7Cs8v/xjhwzit1Lw586dY5+7UVZWdvz48YceemjBggXBwcEeHh7sjw/omz59uru7+44dO3Jycry9vcPDw9m3PDw8/P396+rqmHX6ewR7avz9/ZkfYyksLFy1atX9998vyf6Na7IxYSVPs2eNRhMQENDQ0GDQlZaUlBBC2NVhjUbTs2fP1tZWZigvLDx3d/cnn3wyOztbq9WePXv25s2bNG8JkJaW9vzzz1dVVbW2thYXF//3v/+VZPFUfAsicjQitVq9e/dub2/voqKiCRMm6D++SlgwlOUg4JwiX0mia9euzM+GFhcXr1+/XsJ7q5mHtUVERBDbFq8kx+JPYjSZMDAwUKVS3bhxg//BlGjjPNDGxWPHJPfff39iYmJBQcGPP/44d+5cqe7vtsZwlL9bpCG+/UqIv+Lxh2rp3ijTjmCO3gzT0tI6OjrCw8PZnk7CeIxTnHPnHKbtu7i4REVFpaamVlRUZGZmTp061cPDw6L9iKm0YtqywayHclfyrh4YV2BOIlOcfsnQ78qx1lWkwvzVnp6ezED65s2bqampw4YNs7Q0LO0IxI+7pJ1K21VilzYYWyZ2+1o/ra+v379/v6enp/7CPHPNBfP9iSnM7Yft7e3sKx0dHTExMYGBgT///LPV4jVkHD9N8IQrfn45OTnsV0lMN2b2mnM/P79Ro0ZVVlbOmjVr5syZBu8++eST5LcnT98jrLcYQUzUZE6Wljz9npkHzu7bt499pbW1NSsrS6VS6T+hjNnso48+sjS85uZm/XsJ3dzctFqtm5tbt27deN6i+Rs5ffrpp6mpqeHh4cb3oIkkSQsighqRyDTVp0+fbdu2ubi4ZGdnv/baayKD4S8HMecU+UoYZlSnVqtjY2N37dpVUVGxdevWUaNGUV4qQqm+vj4rK0uj0bApRVjxWtqLiTmWAf4kZjYTurm5PfLIIx0dHWavnkMbNwVtXDB28hYUFPTyyy+fOXMmPz9/+fLlzO+6SIhyOGoRs90iDZHtV1r8FY8/VOMcyLM3+rQjjIzNUPz869y5c2+++aZSqWR+BFxkPCz+FOd8OYddphk8eHBycvL169ePHz8+Z86crl27CtuhyEorrC1zznpodiV4Am69CsxJcIozLhnKXQlbV3FQTA+rUqmeeuqpLVu23LhxgxlIu7u7C94nf0dgQPC4yxpTaYbth7IMznYhPvHKktjlXz+9c+dOdXV1R0dHUVFRfHz8nTt31q9fr5/cIyIiOnfu/MUXX5w9e7atre327dsnT5402Em/fv0IIQcOHKitra2qqmpsbLx27drBgwfLysq++uorGeOnCZ4zfs5j6XS6pqam0tLSffv2sXOV4ODgHj165Obm/vDDD42NjefPn9dPnfqYn+e7fv36s88+a/DWkiVLvL2958+fv3379pqamvb29hs3bnz33XeCisQe6XQ6Jod6eHjEx8fv2bPn1q1b0i5GmK3JnGhKXvCe77vvvldeeeXIkSMtLS2FhYWTJ08uLi5evXo18+t1jGXLlvXr1++9995bsWJFYWFhe3t7Y2PjpUuXfvnlF7PhffPNN+vXry8tLW1qasrKytq/f/+ECROYewx53hKGeYLtzJkz1Wq1QqFQKBQuLi6BgYEzZsy4deuW4N0SiVoQEdSIxKepkSNHLl++nBCybt267du3iwnGbDlYek6Rr4RhMpW7u/uoUaN27NhRWVmZmZkZFxcn4d27FRUVtbW1HR0dv/7668SJE6urqz/66CP2HAkrXspezIAkp9JsEjObCZOSktRq9bx58/bu3cv8OMDMmTONv69GGzeANi5e9+7d582bd+rUqaKiovXr10tyUwUnyuGopfi7RRri26+E+Csef6jGOZB/b5RpRxgZm6FFSa+tra2urq6trY0Q0tjY+PPPP69YseLhhx9WqVR79+596KGHxMejjyfFOV/OGTBgwOrVq4uKiiS8kl1MpaVvy2ZnPTS7EjwBt14FFlksZkuGclfC1lXM/iF2hRlIu7q6xsTEbNmyRfA115z4OwIDYsZdkk+lGTYYytK3C0kSrwyJXfd7zH18iYmJOgrFxcV//vOf2UFzz549Z86cqdPpMjIyunfvTghxdXX905/+VF1dfejQIeZH4ggh4eHhZWVlOp0uMzOTEOLv79+jRw93d/cuXboMHz587969xgfavXt3v379VCpVWFjYa6+9lp+fz6yeDB48uLKyktnm+eef79KlC3MRTUlJSXt7+2OPPebv73/hwgWL4qcMnj5+muAN4t+xY8egQYNUKhXPiVuxYgX72aNHjw4cOFCtVgcFBb344ovMlwNKpfKZZ57Rj6SjoyM4OHjq1KmcpXHlypXJkyf37NnTzc1No9FERUWtXLlSp9Nt3ryZSbuurq5hYWG7d+82WSFM0Gq1Wq3W0k9JKCIiwt3dffTo0Tt27GhoaKD5iEUx09QEnmI0VfLi93zz5s2XXnqpV69e7u7uGo3mr3/961dffWUcf1VV1eLFix944AG1Wq1UKoODgydNmnT+/Hn+8Nra2iZNmhQaGurh4eHm5tarV6958+bV1dXxvJWenh4aGsp8Je7t7R0bG6ujThfNzc2mpmejR4+mPFM6ne4///lP//79mZ7Vx8fn6aef1knUgnjKytQ5oklTubm5bDZQKBS9evUaN26c/gZ3796NjY1l3g0NDc3OzhYWDH858Jxu/QgdOl8lJiYSQvLy8ug/Iq1x48a5uLiMGDFi06ZNVVVVNB+xNOZLly69+uqrUVFRvr6+zNezMTExhw8fNthMWPEa9MKUIZk6Fn1frDOXxGgy4ffff//EE094enqqVKqIiIjPPvuMeaDBAw88cPnyZcElgzZuV21cZ+Eo1xqY53BpNJpZs2ZlZ2e3t7eb/YhFMbe1tcXExHTq1IkQolarhwwZkpKSwrxldjhq3D9y9pj6zHaLNMS0X8pEQZ9PeIZkZkM1zoH8e6NMO/xMnSO5mqHZpHfixIno6GgfHx+1Wu3u7s5eJqlWqwMCAv72t7+lpKQwmYc+Hprzy5Pi+HdOSfYhxOeff04ICQkJWbp0aX5+Ps1HBMTMX2n587PZvph+ZYCmWxcwAadZQO833a8AAAT5SURBVBBWgUVO1ihLhnLeJ2BdhbMojMnew86ZM0ehUERHR3/wwQcVFRU0HxEQs6mOwNI5poCptE6nM55Nm+r3OefdOusMZQUndpGzAKsmdlN1Q9T6qUhMLli4cKENjmUNjhj/8OHDjx07ZuODyr5+umfPHsrFCJaA9VNr1ARHrGNWUlpaOmDAgOPHj7Ov3L179+eff/b19VUqlW1tbbYJQ5YWdM+ycWnLPvnJzs7WXxOkIXvMAGLYPqPKPrtraWn56quvmpub6T8ie8z80C06OpxBScjeHf/yyy+nTp2y6COyx2wAsx5THKVkZO+tcnJyCgsLLfqI7DGD3TJVN9x4riawDebiXsflQPEXFBRUV1fz/xSgUzL7qFBJWK8mOFAds54pU6YEBARER0ezrygUirCwsPDw8OPHj9vmaej3bAuSxT1Y2o888ojcIQDYzj3YxgkhSqVyzJgxckchmXvzJDoTnEGn0b9/f7lDkAZmPaagZMx6+OGH5Q4BnJ/8zz/V6XRyhyCKncefk5PD/hLfW2+9tWDBAnnjcWLWqwl2Xsds49SpU5xPAjp9+vTIkSNdXV3PnTunMOfpp5+29LhoQbaE0nYOVmqM4ATQxp2A2ZOIDCAJ6xUjmiHYM8x6TEHJANgD+a8/Bas6fPhwfn7+pEmTPv74Y+ZJ0nJHBCBETEzMF198ERMTExsb6+npWVVVlZ2dnZSU5O3t/e9//5sQEhERYY2BBVqQLaG0nYOVGiM4AbRxJ2D2JCIDSMJ6xYhmCAAAIIxs15+uXbt2ypQphJAPP/xwwIAB+/fvlysSYRwlfh8fn1deecXPz+/06dOZmZm2uc35nmK9muAodcw20tPTFy1a9O677wYEBCiVyv79+69fv37y5Mnnz58PDg623nHRgmwJpQ3g3NDGnQBOoqPDGQQ7hFmPKSgZALsi2/WnCxcuXLhwoVxHF89R4k9ISEhISJA7CmdmvZrgKHXMNjw8PBITE5lH3dsSWpAtobQBnBvauBPASXR0OINghzDrMQUlA2BX5H/+KQAAAAAAAAAAAIB9wvopAAAAAAAAAAAAADesnwIAAAAAAAAAAABww/opAAAAAAAAAAAAADesnwIAAAAAAAAAAABww/opAAAAAAAAAAAAADesnwIAAAAAAAAAAABww/opAAAAAAAAAAAAADesnwIAAAAAAAAAAABww/opAAAAAAAAAAAAADesnwIAAAAAAAAAAABww/opAAAAAAAAAAAAADesnwIAAAAAAAAAAABww/opAAAAAAAAAAAAADesnwIAAAAAAAAAAABww/opAAAAAAAAAAAAADc3zlfz8/MzMjJsHAo4q9ra2q5du8odhcVqa2vRCgDuHfn5+XKHINCBAwfy8vLkjgLAAZSWlsodgkAYmQPYMwwhANDDgjMxVZ+5108zMjJQh0BCWq1W7hAsVlpaOn78eLmjAAAwY/78+XKHAADWhZE5AFgDhhAA6GGBnuH6aVBQ0K5du2QJBZyYRqOROwTLvP/++zU1NXJHAQC2FhQUJHcIFhg/fnxYWJjcUQA4GMdqNRiZAzgKDCEAHKtSoYcFfsb1WaHT6WQJBQAAAAAAAAAAAMDO4fejAAAAAAAAAAAAALhh/RQAAAAAAAAAAACAG9ZPAQAAAAAAAAAAALhh/RQAAAAAAAAAAACAG9ZPAQAAAAAAAAAAALj9HwtQos4Q+4//AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.keras.utils.plot_model(news_classifier, rankdir='LR', dpi=120)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Nsubq7bBJlsC",
   "metadata": {
    "id": "Nsubq7bBJlsC"
   },
   "source": [
    "We can change the model plot orientation by setting the `rankdir` to either `TB` (top-bottom) or `LR` (left-right)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "9ixf5A8uJS4i",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 700
    },
    "id": "9ixf5A8uJS4i",
    "outputId": "02a711ea-9ccc-4fc4-d36b-0f9a8f1f953a"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAKrCAYAAADPplemAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzde1xUdf4/8NeBYRgHaBDiDuKNfdiAgZl+XVTKstRES1ohwco0H/nN6Jt9ybXkl7rWWrqatW32LXfXLewi+d3KDfNeaBfra3nP0hJCRBSUi1zk9v790TI5DpcBBj4Mvp6Pxzwees7nnPP+nPPhxZkz5wyaiAiIiKirZbqoroCI6GrFACYiUoQBTESkCAOYiEgRBjARkSI61QU4k/Lycnz88ceqyyDqlkwmE26//XbVZTgVjbeh2e/o0aOIjIxUXQZRt2Q2m3HkyBHVZTiTTJ4Bt8PUqVMxdepU1WUQdRuPP/646hKcEgO4HcxmMwOY6DKLFy9WXYJT4odwRESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCAO4iP//8M/7jP/4DXl5e0DQNmqYhKCgIM2bMUF1al+hJ/d+9ezeio6NhMBgsfQkODkZcXBxKSkpUl0dOhH+Us4v06dMHe/fuxZ49ezB69Gj4+vqioKBAdVldpif1f/To0Thw4ABeffVV/Od//idGjhyJPXv2qC6LnBDPgKlVTz31FJ577jnVZShztfefOg8DmFqUm5uLF154QXUZylzt/afOxQDuJjIzMxEQEABN0/Dwww9j8+bNmDp1Kvz8/NC7d2+kpKSgrKzM0n79+vXw9fWFpmmYOXMm7r//foSEhMDd3R2DBg3C6tWrISJW20hPT4fJZIKmaUhPT7dMf/bZZxEUFARN0zBq1Cir9jExMaiursaTTz4JTdMQGhoKAKivr8f48eMRGhqKI0eOdGnf29P/tva9tf53VEpKCsxmM7y9vWE0GhEVFYXXXnvNMj8tLQ16vR6apkGv12PYsGHIy8sDAGRkZGDAgAGW6+hZWVlW687NzcW9994Lf39/GI1GREdHY/ny5WhoaAAArF27Fv7+/tA0DampqSgvL8eyZctgNpuxatUqh/SP7CRktyNHjggAWbRoUbvXsXv3bgEgvr6+NvO+/vprASBms1k2btwoRUVFcuHCBZkzZ44AkPnz51u137RpkwCQ+Ph4+fHHH6W6ulq+//57mTZtmgCQ3//+9zbbePvttwWALFy40Gr6uXPnBICMHDnSavrrr78uAGTZsmVW048fPy4ABIAsXbq0w/1va9/b0/+29r2l/ouIrFmzptnlWuPu7i7r1q2TwsJCKSgokNjYWAEgW7ZssbRZuXKlAJBJkybZLF9UVCQGg0Hy8vKspp8+fVoCAwPFz89P9u7dK1VVVbJkyRIBIE899ZSlXeP+Hjx4sMTExMjEiRMlIiJCVqxY0ea+iIiYzWYxm83tWvYqtoFnwN3Q4MGDkZCQAF9fX3h7e+MPf/gDAGDz5s1Nth8wYAD69+8Pd3d3/OY3v8G6desQFhaGP/3pT8jPz++UGvv164dbb70VgYGBmDx5ssPW29a+A2r631HDhg3D/fffD39/fwQGBmLOnDkAgG3btlna3H///XB3d8fmzZtx9uxZq+Xfeecd3HzzzTZn5H/4wx9w5swZrFixAsOHD4fBYMDTTz+Nvn37YvXq1SgvL7dq369fP3zzzTf417/+hb/85S+d1FtqDgPYCfj5+QGAzQ9Pc/R6PeLj41FfX2/z9tRRXF1dsX37dhQUFOD666/vlG0Abe870DX976jdu3db/b+xn1VVVZZpvr6+mDJlCurq6pCRkWHVPiMjA7NmzbJZ76ZNmwAAEydOtJoeGxuLyspKfPnll1bTe/XqBU3TAAC33XYb0tLS2tkjag8GsBORK67ptqRfv34AgGPHjnVWOV2qLX0Hek7/Z8+eDQBYt26dZdqJEyfw448/2rzzqKysRH5+Ptzc3HDttddazfPx8QGAbvuO4GrF+4B7KL1eD+CXH8qrUXfuf2lpKf74xz9i69atyMvLQ2VlJS5dutRk2zFjxmDgwIE4dOgQ9u3bh6FDhyIjIwMpKSmWPjaqqKgAANTW1lrOaq9UXV3t2M5Qh/AMuIcqLS0FAJhMJsWVqNGd+x8fH4/ly5cjJSUF+/btQ1FRkeXSwZU0TcODDz4I4Nez4PXr1zd5+cHDwwMA4OXlBRFp8tV4rZm6BwZwD9X41js6OtpqenNnRj1NU/1X0ff9+/dj+vTplv/n5+djz549GDFiBNLS0hAeHg6j0QgXl+Z/FGfMmAE3Nze8/fbbyM7Oho+PD6KiomzaGY1GBAUFoby83HI2TN0bA7gHunjxIj7++GN4eXnZfBjj6+sLALhw4YJd69LpfrlKVVdXZzW9vr4e48aNQ2hoKA4fPuyAqh2nuf63te9A8/23V3Z2Nry8vCz/b/yQzdvb2+51BAQEYNKkSSguLsasWbMwc+bMZttOmDABAPDee++1q17qWgzgHuDChQsoKSlBfX09cnNzkZKSggsXLuDFF1/ENddcY9U2JiYGHh4eeO+99/Dtt9+itrYWRUVF+OKLL5pcd0REBABgy5YtKCsrw/nz51FZWYmTJ09i69atyM/Px/vvv9/pfWyJvf1va9+B5vvfEhFBVVUVTp06hc2bN1sFcHh4OAIDA7F792589dVXqKysxMGDB1u8zQ749cO406dPY9q0ac22S09Ph4+PDx5//HG89dZbKC0tRV1dHc6cOYPPPvusxW2QAspuQXZCHXkQ4+eff5bf/va34uXlZXmAITg4WGbOnCkiIpmZmeLv7y8AxNXVVYYPHy4lJSWybds26devn2WZqKgoyc/PF5FfH0QICgqSwMBAcXNzE09PT4mLi5OPPvqo2Vo2btwoERER4u7uLpGRkTJ//nw5evSoaJomLi4ucsMNN0hxcbGl/UMPPSSenp5iMplk4sSJkpeXJ3V1dTJ27FgJCgqSQ4cOdaj/7el7e/vf1r431f+3335bhgwZIu7u7pbaWnotWbLEan2ffPKJREdHi8FgkLCwMJk7d64cPnxYAIher5eEhASbuuvr6yU8PFzuu+++Vvf18ePHZfr06RIcHCw6nU5MJpPExsZaHiZZu3at+Pn5WfZ3ZGSkbNy4sdX1toQPYrTLBgZwGzjiSThHagygJ554QnUpSlxt/Y+Li5NPP/1UdRlNYgC3C5+E6wlqa2tVl6DU1dD/EydOoKSkBHFxcapLIQdiAPcA0saHFHqantj/7Oxsq6f/nnnmGT6l1gMxgIm6oZ07d2L9+vUoKyvD8uXLkZOTY3U7G/UMDGAntWLFCtx7770AgL/85S8YPHgwPv74Y8VVdZ2e3n9fX1/MmzcPAQEB2LdvHzZt2nTV3MN9NdGkJ75/6yRHjx5FZGQkFi1ahMWLF6suh6jbiIyMBIAOfzf0VSaTZ8BERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRXSqC3BGR48eRWZmpuoyiLqNsrIyXHPNNarLcDr8ixht0PgXMYjIltls5l/EaJtMngG3QVhYGDZs2KC6DPq3xMRE/Pa3v8W8efNUl0IATCaT6hKcDs+AyWlpmobExES8++67qkshag/+TTgiIlUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFdKoLIGpJZWUl/vznPzc7/9ixY3j++edtpvfp0wfTpk3rzNKIOowBTN2a0WhERkYGDh8+3OT8gwcP4uDBgzbTFy9e3MmVEXUcL0FQt9eeM9nk5OROqITIsRjA1O2lpKRA0zS72rq4uODGG29EREREJ1dF1HEMYOr2wsPDMWLECLtCuKGhgWe/5DQYwOQUpk2bBhFptZ2LiwsSExO7oCKijmMAk1NISkqCq6tri200TcPNN9+MkJCQLqqKqGMYwOQU/P39ccstt7R4GUJEeOsZORUGMDmN1i5DuLm5ISEhoQsrIuoYBjA5jbvvvhsGg6HJeZqm4Y477oCPj08XV0XUfgxgchrXXHMN7rjjjibn8fIDOSMGMDmV5kLWaDQiPj6+i6sh6hgGMDmV+Ph4eHl52XwYl5CQAA8PD0VVEbUPA5icisFgQEJCgs2Hcbz8QM6IAUxO5/Kw1TQN3t7eGDt2rMKKiNqHAUxO59Zbb4Wfnx80TYOI4J577oFer1ddFlGbMYDJ6eh0OiQlJVkuQ/C7H8hZadLEne35+fmYN2+einqI7FJcXIydO3eiV69emDhxot3flkbU1cLCwrBy5cqmZmU2GcBHjx5FZGRk51dGRNTDmc1mHDlypKlZmS3+RYxFixbxLwtQt/Xkk08iKSkJMTExqkshalJrJ7L8k0TktB599FEEBQWpLoOo3fghHDkthi85OwYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMHUrf/3rXxEREQEXFxdomoacnBzVJRF1GgbwVaCsrAxmsxkDBgxAYWGh6nJaNGvWLBw/fhxxcXGqS+lUu3fvRnR0NAwGAzRNg4uLC/r27WvzNxjXrVuHkJAQaJoGg8GAG264ASdPnlRTtB2u7JemaQgODkZcXBxKSkpUl9ftMICvAiKCJv74NSk0evRoHDhwAKtXrwYAxMbGIicnxyaAZ8yYgRkzZsBsNuPkyZP45ptv0K9fPwUV2+fKfo0cORKnT59GdnY2vL29FVfX/fCPcl4FTCYTvvvuO9VlUDs8/fTTyMrKwqeffoprr71WdTnkYDwDJoc5ceIEZsyYgQ0bNqgupUdYunQptm7dil27djF8eyiHB/D69evh6+sLTdMwc+ZM3H///QgJCYG7uzsGDRqE1atX27wdXrt2Lfz9/aFpGlJTU1FeXo5ly5bBbDZj1apVAIDc3Fzce++98Pf3h9FoRHR0NJYvX46GhoZO3e7Zs2fx6KOPYsCAAXB3d4e3tzfGjBmDDz74oNl9UFlZiSVLliAqKgq9evWC0WhESEgIRo0ahSNHjljatdYnAKiqqkJ6ejoGDhwIg8GAgIAAjBw5Ep999pld848fP46YmBi4urpC0zScOHECAJCZmYmAgABomoaHH34YmzdvxtSpU+Hn54fevXsjJSUFZWVlTfbvp59+wqxZsyzb7N27N4YMGYJBgwahV69eSExMbH2gtFNKSgrMZjO8vb1hNBoRFRWF1157zTI/LS0Ner0emqZBr9dj2LBhyMvLAwBkZGRgwIAB0DQNQUFByMrKsizX2rGwZ6zU19dj/PjxCA0NtTrO7fHcc89h+/bt2LZtm11v3R1Rvz37F2h9zHVUZx1jR+4nh5EmHDlyRADIokWLmprdqk2bNgkAiY+Plx9//FGqq6vl+++/l2nTpgkA+f3vf2+zzNdffy0AZPDgwRITEyMTJ06UiIgIWbFihZw+fVoCAwPFz89P9u7dK1VVVbJkyRIBIE899VSnbbeoqEj69u0rYWFh8umnn0p1dbXk5ORIcnKyAJDVq1fbrO/ixYsydOhQMRgM8o9//EMuXrwo58+fly1btkhISIjs2rVLRMTuPj3wwAPi6+srX3zxhVRWVso333wjQ4cOlTfffNOu+Y3GjRsnAOT48eM2fTebzbJx40YpKiqSCxcuyJw5cwSAzJ8/36Z/BQUFEhgYKGFhYbJnzx6pqKiQQ4cOSUxMjACQTz/9tJXRYZ+bbrpJAMjJkyetpru7u8u6deuksLBQCgoKJDY2VgDIli1bLG1WrlwpAGTSpEk26y0qKhKDwSB5eXmWafYei5bGiojI8ePHBYAAkKVLl9rVzzVr1ggAGTlypGXaihUrZOzYsVJRUWHXOhxVv4h9+9eeMddUv+zVGcfY0fvJXmazWcxmc3OzN3RqAP/Xf/2X1fRLly5JWFiYuLq6yqlTp6zmNXZ68uTJ0tDQICIiW7dulRUrVlgCYd26dVbL9O3bV4xGo5SVlXXKdlNTUwWAZGRkWC1TW1srAQEBotfr5cyZM1bzFi5cKADk+eeft9kvc+bMsQSwPX2qr68XDw8PmTp1qlWbd955R958881W51+upQBOSkqyanv27FnLALzS//t//08AyJ/+9Cer6e+9954AkPvvv99mmfZoLoBHjRpl9f833nhDAEhaWpplWlFRkbi7u4tOp5PCwkKr9i+//LKMHz/eapq946ulsSIiUldXJ7feeqsEBgbKgQMH7OrnlUH1wgsvCAAZM2aM1NbW2rUOR9Uv0vr+tXfMdSSAO+MYizh2P9mrtQDu0mvAer0e8fHxqK+vt3lr0KhXr17QNA0AcNtttyEtLQ2bNm0CAEycONGqbWxsLCorK/Hll192ynbfe++9Jrer0+lw5513oqamxuZSREZGBgBg+vTpNttYsGABBg8eDAB290lEkJ2djdzcXEubpKQky/pbm98efn5+AIDy8nKbeadOnQIAhIaGWk1v/H/j/M6ye/duq/831lpVVWWZ5uvriylTpqCurs5yPBplZGRg1qxZVtPaOr6aGisA4Orqiu3bt6OgoADXX399m/v28ssv48MPP4Rer8euXbvwxBNP2LWco+oH7Nu/nTHmLtcZxxhw7H5ylC7/EK7xFppjx47Z1b6yshL5+flwc3Oz+SDCx8cHAJCfn+/w7ZaUlKCgoACenp5NXoPr27cvAFjdXVBWVobc3Fx4eXkhODjYZpnw8HD4+vra3ScXFxckJiaisLAQZrMZs2fPxp49eyxtW5vfUdLErWthYWEAgJ9//tlqeuMP45XBrMrs2bMB/HIfbaMTJ07gxx9/xOTJky3THDW+OurIkSO4ePEiduzYgRdffBEAsHr1aqxfv77F5bq6/s4ec21h7zEGus9xvlKXB7Berwfwyw6xR0VFBQCgtrbWcmN34+vll18GAFRXVzt8uxcvXgQAeHp6Njm/MZQb2wGwfGjl4eHR4rrb0qc1a9YgPT0dRqMRa9euxejRozF8+HDLh2mtzXe0++67D56ennjppZfw+eefo6qqCocOHcIzzzwDvV6PuXPndsp2AaC0tBS///3vMWTIEFx77bUwGo02ZzONxowZg4EDB+LQoUPYt28fgF/OjFJSUixjAXDc+OqoyMhILFiwAJqmYc6cOZgxYwaAX0Lm22+/bXY5R9Zv7/7tzDHXGccY6D7H+UpdHsClpaUAfrk31R6NYebl5WV5oODK15w5cxy+XS8vLwBNvw0HgAsXLli1u/zf58+fb3HdbemTwWDA0qVLcfr0aWRlZSEuLg5ff/215U6D1uY72oABA7Bt2zZcuHAB8fHxMJlMmDhxIgYOHIgvv/wSw4YN65TtAkB8fDyWL1+OlJQU7Nu3D0VFRZa3lVfSNA0PPvgggF/PkNavX2/z1tRR48vR1qxZgyFDhqCqqgpTpkxBcXFxk+0cWb+9+7czx1xnHGOg+x7nLg/gxksA0dHRdrU3Go0ICgpCeXm55bdYV2zXZDIhJCQEFRUVTYZw420vkZGRVssEBwejpqamxe8waE+f3NzcMGHCBOzYsQNmsxnffvut1WPFrc13pHXr1uGhhx7C+fPnUVNTg59//hn/+7//iyFDhnTK9oBf3h7u2bMHI0aMQFpaGsLDw2E0GuHi0vwQnjFjBtzc3PD2228jOzsbPj4+iIqKsmrjqPHlaAaDARs3boSPjw9yc3ORlJSE+vp6m3aOqr89+7ejY27//v1W14076xgD3fc4d2kAX7x4ER9//DG8vLyafVvRlAkTJgCA5UOxrtpu42/0zZs3W02vqalBVlYW3N3dba41NS7zyiuvtLhue/pUXV2NhIQEq2k6nQ5msxk6nQ69evVqcX7v3r1brKE9/v73v+N//ud/EBUVhUuXLjl8/c1p/ACmLY+zBgQEYNKkSSguLsasWbMwc+bMJtt1dHw1qq+vx7hx4xAaGorDhw93aF3AL59brF+/Hi4uLtixYwfmz5/fZDtH1G/v/m1tTLZlzGVnZ1u9g+zMYww47jg7UqcG8IULF1BSUoL6+nrk5uYiJSUFFy5cwIsvvohrrrnG7vWkp6fDx8cHjz/+ON566y2Ulpairq4OZ86cafLmb0dut3///pg3bx527dqFS5cuIScnB9OnT8fPP/+M559/3vIJbaOnn34aERERWLlyJZYsWYKcnBzU1dWhsrISx44dw/fff9+mPn344Yd48cUXcerUKVRVVSErKwsff/wxkpKSoNfrW53vaI1vhWfOnGn1hSsuLi4IDQ3FAw88gHPnzjl8u+Hh4QgMDMTu3bvx1VdfobKyEgcPHrT55Xilxg9qTp8+jWnTpjXZpq3jqzknT57E1q1bkZ+fj/fff9/+zrVg/Pjxlu+HWLVqFd56661Oqb8t+7cjY05EUFVVhVOnTmHz5s1WAdyZxxhw3HF2qKZuTnPUfcBBQUESGBgobm5u4unpKXFxcfLRRx/ZtF+7dq34+fkJAHF1dZXIyEjZuHGjVZvjx4/L9OnTJTg4WHQ6nZhMJomNjZVly5Z16nYLCwvlkUcekT59+oibm5uYTCa5+eab5f3332+2/+fPn5cFCxbIddddJwaDQfR6vYSHh0tycrIcPHjQ7j7V1tZKcnKyDBo0SIxGo+h0OunTp4889thjUl5e3up8EZGMjAwZNGiQuLi4CADx8fGR+Ph4yczMFH9/f0vfhw8fLiUlJbJt2zbp16+f5YGCqKgoyc/Pt9RcXV1tebCludfkyZNbGh4t+tvf/ia/+c1vRNM0ASC+vr5y1113iYjIJ598ItHR0WIwGCQsLEzmzp0rhw8fFgCi1+slISHBZn319fUSHh4u9913X4vbbe1Y2DNW6urqZOzYsRIUFCSHDh1qcXu7d++WIUOGiLu7uwAQTdOkT58+8rvf/c6mbUNDg8THx1vaDRo0SHbs2OHw+u3Zv62NuSv71dpryZIlba7hSvYeY0ftp7ZQ+iDGE0880a7l20vVdq8mp06dksGDB8uePXuspjc0NMjhw4fFz89P9Hq93Q8RdIW4uDiHPaFH3VN3PcZKH8Sora3tzNV3u+1eDe69916EhIRg5MiRVtM1TUNkZCSioqIgIpYb2FU7ceIESkpKevz3C1/NnPkYd2oAi6LvoFW13avB3r17ba57N7pw4QL27duH8ePH49ChQzb3W7b0uuuuuxxSX3Z2ttVdK88880ynPMFE6vSkY8yvo6Q2GTduHN577z2sX78epaWlaGhoQFFREd59912MHj0aPj4++POf/4yYmJhm77ds6uWoD6127tyJ9evXo6ysDMuXL7d8aEo9R486xk1dmOjINeDly5eLt7e3ABCdTidRUVGyefPmNq/HWbZ7tamoqJDFixdLTEyMeHh4iKurq/Tu3Vt++9vfyrJlyyxfaKLKSy+9JAaDQQwGgyQmJiqvhxzPmY5xa9eANRHb9+tHjx5FZGQkFi1aZPMnUoiIyD6ND2o18/3QmbwEQUSkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpIiupZmZmZk4evRoV9VCRNSjnDp1CqGhoc3ObzGAjx49ygAmIuokTf5NOCJnoGkaEhMT8e6776ouhag9+DfhiIhUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpoomIqC6CqDmVlZW49dZbUVtbazPv8OHDMJlMCAsLs5kXFxeHVatWdUWJRO2VqVNdAVFLjEYjfHx8kJWV1eT8s2fP4uzZszbT586d29mlEXUYL0FQtzdt2rQ2tXd3d0dCQkInVUPkOAxg6vamTJkCo9FoV1tN0xAfHw+TydTJVRF1HAOYuj0PDw/Ex8fb1VZE2nzGTKQKA5icgr2h6uXlhTvuuKOTqyFyDAYwOYU77rgD3t7e0DStxXZ33303evXq1UVVEXUMA5icgl6vx913343W7prk5QdyJgxgchothaumafD19cUtt9zShRURdQwDmJzGmDFjEBwc3ORlCBFBcnIydDre2k7OgwFMTsPFxQWJiYnNXobg5QdyNgxgcipNhaymaejTpw9GjBihoCKi9mMAk1MZPnw4Bg4cCBeXX4euiGD69Omt3iFB1N0wgMnpTJs2DQ0NDTbTiJwNA5icTnJysuXfLi4uuO666xAVFaWwIqL2YQCT0xk0aBCio6MBAA0NDZg+fbriiojahwFMTqnxLFjTNF5+IKfVbb+QPTExEZmZmarLICInt2HDBkydOlV1GU3p/l/I3k13HHUDu3btQlhYGAYOHKi6FOqGTp06hS+++EJ1GS3q9gG8YcMG1SVQN/Xaa6/hzjvvREBAgOpSqBvKzMxkABN1lpkzZ/LRY3Jq/BCOnBbDl5wdA5iISBEGMBGRIgxgIiJFGMBERIowgPmqUTkAACAASURBVImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkSI8L4JEjR8JgMEDTNLz66quqy7nqLVy4ECEhIdA0Dddee63qcoi6lR4XwJ999hkWL16sugz6t2effRb5+fnw8PBQXYrF7t27ER0dbflF7eLigr59+9qMm3Xr1ll+eRgMBtxwww04efKkmqJbcWWfNE1DcHAw4uLiUFJSoro8akaPC2Ci1owePRoHDhzA6tWrAQCxsbHIycmxCeAZM2ZgxowZMJvNOHnyJL755hv069dPQcWtu7JPI0eOxOnTp5GdnQ1vb2/F1VFzenQAu7io6d5TTz2F5557Tsm2yXGefvppZGVl4dNPP0VQUJDqcqgH6tEBbDAYunybubm5eOGFF7p8u92dm5ub6hLaZOnSpdi6dSt27drFa9fUaXp0AL/00ksYMGAAPD09YTAYcP311+Oll16CiDTZPjc3F/feey/8/f1hNBoRHR2N5cuXo6GhAQCwdu1a+Pv7Q9M0pKamory8HMuWLYPZbMaqVauQnp6OmJgYVFdX48knn4SmaQgNDW13/a3Vk5mZiYCAAGiahocffhibN2/G1KlT4efnh969eyMlJQVlZWVNrruyshJLlixBVFQUevXqBaPRiJCQEIwaNQpHjhyxtDt79iweffRRDBgwAO7u7vD29saYMWPwwQcfNLnes2fPIjU1FX379oVer4fJZMLgwYNRVVXV7n62tt/r6+sxfvx4hIaGWtXeXs899xy2b9+Obdu2tfr2vaO1N0pJSYHZbIa3tzeMRiOioqLw2muvWW2rqqoK6enpGDhwIAwGAwICAjBy5Eh89tlnHe6zvXWkpaVBr9dD0zTo9XoMGzYMeXl5AICMjAwMGDAAmqYhKCgIWVlZDt9PPY50U1OnTpX2lrds2TIBIKmpqZKfny81NTVy7NgxSUlJEQDyyCOP2Cxz+vRpCQwMFD8/P9m7d69UVVXJkiVLBIA89dRTlnZff/21AJDBgwdLTEyMTJw4USIiImTFihUiIvL6668LAFm2bFn7Ot7Oesxms2zcuFGKiorkwoULMmfOHAEg8+fPt1n3xYsXZejQoWIwGOQf//iHXLx4Uc6fPy9btmyRkJAQ2bVrl4iIFBUVSd++fSUsLEw+/fRTqa6ulpycHElOThYAsnr1aqv1njt3Tvr16yeenp6yYcMGqaqqkurqasnLy5NevXqJr69vp+z348ePCwABIEuXLrV7H69Zs0YAyMiRIy3TVqxYIWPHjpWKiopWl3fUmBERcXd3l3Xr1klhYaEUFBRIbGysAJAtW7ZY2jzwwAPi6+srX3zxhVRWVso333wjQ4cOlTfffLPFPrWFPXWsXLlSAMikSZNsli8qKhKDwSB5eXmdsp/aYsOGDQJANmzY0K7lu8CGHh3Aa9assZpeW1srAwYMEADy3XffWc1rDKx169ZZTe/bt68YjUYpKysTkV8HyeTJk6WhoUFERLZu3erwAG5rPUlJSVbtzp49axnMV1q4cKEAkOeff77J7TYGcGpqqgCQjIwMqza1tbUSEBAger1ezpw5Y5k+d+7cJoNZRMTDw6PJAHbEfq+rq5Nbb71VAgMD5cCBAzbbaM6VYfXCCy8IABkzZozU1ta2uryjxoyIyKhRo6zW8cYbbwgASUtLExGR+vp68fDwkKlTp1q1e+eddxwawK3VIfJLyLq7u4tOp5PCwkKr9i+//LKMHz/eapoj91NbOEMA9+hLEFfS6XSYPHkyAFi9PQKATZs2AQAmTpxoNT02NhaVlZX48ssvrab36tULmqYBAG677TakpaU5tNa21nMlPz8/AEB5ebnNvIyMDADA9OnTbeYtWLAAgwcPBgC89957Tdag0+lw5513oqamxupSxDvvvAMAuOeee1qs7XKO2O+urq7Yvn07CgoKcP3119u97cu9/PLL+PDDD6HX67Fr1y488cQTXVJ7o927d1u1bTx+l1+6ERFkZ2cjNzfXMi0pKanJ49he9tTh6+uLKVOmoK6uzjKWGmVkZGDWrFlW07rbz1Z3clUFMAD0798fAPDTTz9ZplVWViI/Px9ubm42H7j4+PgAAPLz87usRkfWI1dc7y4rK0Nubi68vLwQHBxs0z48PBy+vr4oKSlBQUEBPD09m7wO2rdvXwDAd999BwAoKChAcXExfHx8EBAQYFdt3WW/HzlyBBcvXsSOHTvw4osvAgBWr16N9evXN7tMV9fu4uKCxMREFBYWwmw2Y/bs2dizZ4/D1t9Ws2fPBvDLvdKNTpw4gR9//NFykgN0n2PcXelUF9DVdLpfulxTU2OZVlFRAQCora21/Oa9UnV1decX1wX1NH4o19qDERcvXgQAeHp6Njm/MZQb25WWlgIATCaT3bV0l/0eGRmJBQsWAADmzJmDvXv3Yt26dZg9ezbMZjOGDBlis4wjay8tLcUf//hHbN26FXl5eaisrMSlS5ds2q1ZswahoaF49dVXsXbtWqxduxbDhg3DW2+9hYEDB9rb3Q7XAQBjxozBwIEDcejQIezbtw9Dhw5FRkYGUlJSoNfrLe26yzHurq66M+DGp4Iaf/sCv4aRl5cXRKTJ15w5c7qsxs6sx8vLCwBw/vx5u9o1dQkDAC5cuGDVrvGWv8t/sbWmu+33RmvWrMGQIUNQVVWFKVOmoLi42KaNI2uPj4/H8uXLkZKSgn379qGoqMjytv1yBoMBS5cuxenTp5GVlYW4uDh8/fXXSExM7FiH21gHAGiahgcffBDAr2fB69evt7n80F2PcXdx1QVw421KMTExlmlGoxFBQUEoLy+3/MZWqTPrMZlMCA4ORk1NDXJyclpsFxISgoqKiiZDuPHWo8jISABAaGgo3N3dcebMmRZvObtcd9vvjQwGAzZu3AgfHx/k5uYiKSkJ9fX1Vm0cVXt+fj727NmDESNGIC0tDeHh4TAajS0+ROTm5oYJEyZgx44dMJvN+Pbbb1FYWGj3Nvfv329z3bg9dcyYMQNubm54++23kZ2dDR8fH0RFRVm16a7HuLu4qgL44sWLyMrKgslksvlAYMKECQB+/eCpvRovcdTV1XVoPY6qpymNZ0yvvPKKXe02b95sNb2mpgZZWVlwd3e3XO/T6XS45ZZbUF9fj3/+85921+KIftbX12PcuHEIDQ3F4cOH272ey/Xr1w/r16+Hi4sLduzYgfnz59u0cUTtjb+sWrvfuLq6GgkJCVbTdDodzGYzdDodevfubfc2s7OzLe9c2lrH5QICAjBp0iQUFxdj1qxZmDlzZpPtOnMsO7seHcBnz55FWVkZ6uvr8dNPP+Gee+5BSUkJXnnlFZsBmJ6eDh8fHzz++ON46623UFpairq6Opw5c6ZNN7pHREQAALZs2YKysjKcP38elZWVba7dUfU05emnn0ZERARWrlyJJUuWICcnB3V1daisrMSxY8fw/fffW2ro378/5s2bh127duHSpUvIycnB9OnT8fPPP+P555+3fEoO/PL0mMFgwGOPPYaPPvrI8oHfzJkzmz37cUQ/T548ia1btyI/Px/vv/9+h/bN5caPH2/5fohVq1bhrbfecnjt4eHhCAwMxO7du/HVV1+hsrISBw8etPmlBwAffvghXnzxRZw6dQpVVVXIysrCxx9/jKSkJKvrrk0REVRVVeHUqVPYvHmzzfhvSx2Xa/ww7vTp05g2bVqTbTpzLDu9Tr/TrZ06ch/wsWPH5L//+78lNjZW/Pz8RKfTSe/evWXcuHGyc+fOZpc7fvy4TJ8+XYKDg0Wn04nJZJLY2FjLPb1r164VPz8/ASCurq4SGRkpGzdutFnPQw89JJ6enmIymWTixIlWN6W3RWv1ZGZmir+/v6We4cOHS0lJiWzbtk369etneTghKipK8vPzrdZ9/vx5WbBggVx33XViMBhEr9dLeHi4JCcny8GDBy3tCgsL5ZFHHpE+ffqIm5ubmEwmufnmm+X9999vsuYvv/xSbr/9dvHy8hJ3d3eJiYmRN998U2JiYgSAXHfddfLDDz84dL/X1dXJ2LFjJSgoSA4dOtTqft29e7cMGTJE3N3dBYBomiZ9+vSR3/3udzZtGxoaJD4+3tJu0KBBsmPHDofVLiLyySefSHR0tBgMBgkLC5O5c+fK4cOHBYDo9XpJSEiQ2tpaSU5OlkGDBonRaBSdTid9+vSRxx57TMrLy2361NpryZIlNn21p44r1dfXS3h4uNx3330t7nNH/Wy1hTPcB6yJNPNcrmKJiYnIzMxs9rFhIuoebrrpJixduhRxcXGqS7GSmZmJxMREbNiwAVOnTlVdTlMye/QlCCLqXCdOnEBJSUm3C19nwQDuZPv377d8QbY9r7vuukt1yUTNys7Otror5plnnunRT6p1tqvuQYyuFhMTw8so1GPs3LkTR48eRXJyMl599VXLh7LUPjwDJiK7+fr6Yt68eQgICMC+ffuwadOmZp9wo9bxDJiI7JaamorU1FTVZfQYPAMmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJSpNv/RQz+uRMi6qm6bQDHxsaqLoG6uczMTISFhWHEiBGqS6FuLCwsTHUJzdKEf7KXnJSmaUhMTMS7776ruhSi9sjkNWAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFdKoLIGpJbW0t8vLymp1fUVGBn376yWa6Xq9HaGhoZ5ZG1GGaiIjqIoiaU1tbi6CgIBQXF7dpuVmzZmHt2rWdVBWRQ2TyEgR1a25ubvjd737X5uWmTZvWCdUQORYDmLq9toSppmkICAjAzTff3HkFETkIA5i6vbi4OISFhUHTtFbbigiSk5Ph6uraBZURdQwDmLo9TdOQlJQEez+u4OUHchYMYHIK9oSqpmno378/brzxxi6oiKjjGMDkFG644QZcd911cHFpfsiKCKZPn27XpQqi7oABTE4jKSkJDQ0NrbYhchYMYHIaLZ3duri4ICYmBmazuYurImo/BjA5jQEDBmDo0KFNhnBDQwOSk5MVVEXUfgxgcirTpk1r8m4ITdOQmJiooCKi9mMAk1OZNm2azQdxmqZh9OjRCA8PV1QVUfswgMmpBAUF4aabbrK6DCEivPeXnBIDmJzOlZchXF1dkZCQoLAiovZhAJPTufvuu+Hm5gbgl8sP48aNg7+/v+KqiNqOAUxOx8fHBxMmTADw63c/EDkjh34h+9atW1FaWurIVRI1qV+/fgB++eL1hoYGZGZmKq6IrgaRkZGOvddcHMhsNgsAvvjii68e+Vq0aJEjI3ODw/8kUWhoKFatWuXo1RLZeOmllxAbG8sv36FOd+rUKTz++OMOX6/DA/iaa67B1KlTHb1aIhuBgYEYPnw43N3dVZdCPdzRo0c7Zb38o5zktEaPHq26BKIO4V0QRESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREinS7AD537hySk5Ph7e0NDw8PPPHEE21ex1//+ldERETAxcUFmqYhJyfHrnnUMyxcuBAhISHQNA3XXntth9bF8UKdqdsF8J133om8vDzk5ORg+fLlyMvLa/M6Zs2ahePHjyMuLq5N86hnePbZZ5Gfnw8PD48Or+tqGS+7d+9GdHQ0DAYDNE2zerm6usJkMiEiIgJ33303/v73v6OiokJ1yT1Ctwrgr776Cl988QWmTJkCb29vpKSkQK/Xqy6LqMcbPXo0Dhw4gNWrVwMARo4cCRGBiKCkpASff/45HnvsMfzwww+YOXMmBg0ahO3btyuu2vl1qwA+duwYAMDLywsA4O3tjTfeeENlSdTJnnrqKTz33HOqy6AWeHl5ITIyEnPnzsX+/fuRlpaGU6dOIT4+HtnZ2arLc2rdKoALCgoAAJqmKa6EukJubi5eeOGFTlu/m5tbp637auXq6ooVK1YgISEBly5dwkMPPYS6ujrVZTmtbhHABw8exIgRI7Bw4UIAwOzZsy3Xn/7v//4P6enpMJlM0DQN6enpluWeffZZBAUFQdM0jBo1qkM1pKWlQa/XQ9M06PV6DBs2zHL9OSMjAwMGDICmaQgKCkJWVlaHtlVZWYklS5YgKioKvXr1gtFoREhICEaNGoUjR45Y2p09exaPPvooBgwYAHd3d3h7e2PMmDH44IMPrNaXmZmJgIAAaJqGhx9+GJs3b8bUqVPh5+eH3r17IyUlBWVlZR1eBvglNO+99174+/vDaDQiOjoay5cvR0NDQ5v6mJ6ejpiYGFRXV+PJJ5+EpmkIDQ1t17bOnj2L1NRU9O3bF3q9HiaTCYMHD0ZVVVW7jk9bpKSkwGw2w9vbG0ajEVFRUXjttdcs89s7rlrr+9q1a+Hv7w9N05Camory8nIsW7YMZrMZq1atAgDU19dj/PjxCA0NtRpXjrB06VIAv7xrbeosuLX62zP+qqqqkJ6ejoEDB8JgMCAgIAAjR47EZ5991qZtdyviQGazWcxmc7uXX7ZsmQCQ119/3Wbe22+/LQBk4cKFVtPPnTsnAGTkyJE2y9x0000CQE6ePGnXvJUrVwoAmTRpkk37oqIiMRgMkpeX1/aOXebixYsydOhQMRgM8o9//EMuXrwo58+fly1btkhISIjs2rXLsr2+fftKWFiYfPrpp1JdXS05OTmSnJwsAGT16tVW6/36668FgJjNZtm4caMUFRXJhQsXZM6cOQJA5s+fb1NLW5c5ffq0BAYGip+fn+zdu1eqqqpkyZIlAkCeeuqpNvfx9ddfFwCybNkym9rs3da5c+ekX79+4unpKRs2bJCqqiqprq6WvLw86dWrl/j6+rb3UFlpbiy5u7vLunXrpLCwUAoKCiQ2NlYAyJYtWyxt2jqu7O174/EbPHiwxMTEyMSJEyUiIkJWrFghIiLHjx8XAAJAli5dalc/16xZ0+zP05X69+8vACQ9Pd1qelvrt3f8PfDAA+Lr6ytffPGFVFZWyjfffCNDhw6VN998s83bbqsjR44IAFm0aFG719GEDQzgy+YVFRWJu7u76HQ6KSwstGr/8ssvy/jx49vXscssXLhQAMjzzz9vM2/OnDmWcEpNTRUAkpGRYdWmtrZWAgICRK/Xy5kzZyzTGwdzUlKSVfuzZ89afkiv1NZlGn8w1q1bZzW9b9++YjQapaysrE19bCmA7d3W3Llzm/yFJCLi4eHR6QE8atQoq/+/8cYbAkDS0tIs09o6ruzte+Pxmzx5sjQ0NIiIyNatWy0BXFdXJ7feeqsEBgbKgQMH7OpnWwL49ttvFwDywAMPdKh+e8ZffX29eHh4yNSpU63avvPOO1YBbO+226qzArhbXILoLnx9fTFlyhTU1dUhIyPDal5GRgZmzZrV4W00rnf69Ok28xYsWIDBgwcDAN577z0AwMSJE63a6HQ63HnnnaipqbG5FNEUPz8/AEB5ebndNTa3zKZNm5qsKTY2FpWVlfjyyy8B2N/Hlti7rXfeeQcAcM8997S6zs6we/duq/837rvLL3+0dVzZ2/dGvXr1snxucttttyEtLQ3AL9drt2/fjoKCAlx//fXt7WKzGj8sb+84aU5z409EkJ2djdzcXMu0pKQkq3HW0W13NQbwFWbPng0AWLdunWXaiRMn8OOPP2Ly5MkdWndZWRlyc3Ph5eWF4OBgm/nh4eHw9fVFSUkJCgoK4OnpCW9vb5t2ffv2BQB89913dm9bRNpc7+XLVFZWIj8/H25ubjYPN/j4+AAA8vPz7e5jS+zdVkFBAYqLi+Hj44OAgIA2968r2Tuu7O17d9B4L7DJZLJMc2T9l48/FxcXJCYmorCwEGazGbNnz8aePXus2jvTvmukU11AdzNmzBgMHDgQhw4dwr59+zB06FBkZGQ45J7kxg8VWntA4OLFiwAAT0/PJuc3hnJju67Q+MNWW1vb7F0q1dXVdvfREdsqLS0FYB0AXam0tBR//OMfsXXrVuTl5aGyshKXLl1qsq2948revncHjWei/fv3t0zrzPrXrFmD0NBQvPrqq1i7di3Wrl2LYcOG4a233sLAgQOdat814hnwFTRNw4MPPgjg17OV9evXO+TyQ+NbtvPnz9vVrrnLBhcuXLBq1xUaA9XLy8tyg/6Vrzlz5tjdR0dsy2AwAABqamo62Lv2iY+Px/Lly5GSkoJ9+/ahqKjI8hb4SvaOK3v7rlpxcTF++OEHALB6SrAz6zcYDFi6dClOnz6NrKwsxMXF4euvv0ZiYmKnb7uzOE0Ad+W9wTNmzICbmxvefvttZGdnw8fHB1FRUR1er8lkQnBwMGpqalr8TgGTyYSQkBBUVFQ0GcKNtzFFRkZ2uCZ7GY1GBAUFoby8vMXHUO3toyO2FRoaCnd3d5w5c6ZLbjm7XH5+Pvbs2YMRI0YgLS0N4eHhMBqNcHFp/kfKnnFlb99VW7duHerr6xEVFWV1C2hX1O/m5oYJEyZgx44dMJvN+Pbbb1FYWOg0++5yThPAjdcNG8/+OlNAQAAmTZqE4uJizJo1CzNnznTYuht/W7/yyit2tdu8ebPV9JqaGmRlZcHd3b3D16TbasKECQB+/YCwOfb2Uaf75QpYUzfy27MtnU6HW265BfX19fjnP//Z4rYcrTHwm7pG3xx7x5W9+7k19fX1GDduHEJDQ3H48OEOrety+/fvxx/+8Afo9Xq8/vrrNvMdVf/lqqurkZCQYDVNp9PBbDZDp9Ohd+/enbbtTuXIeyo68za0c+fOiYeHh/j7+8s333wjNTU1cu7cOfnwww8ddhva5TZv3iwAxGg0Smlpabv7dKXz589LRESEuLi4yOLFi+XkyZNSW1srFRUV8t1338mxY8dERKS4uFj69+8vwcHBsnPnTqmurpaTJ0/K1KlTW7wP+MpbekREAEh4eLjN9LYu89NPP4mPj4/4+PjI+vXrpaSkRGpra6WgoED27NnT5j7u2bNHAMioUaOktLRUiouLpaKiok3b+r//+z8xGAzi5+cn//rXv6S0tFRycnLkgQceEACddhtaTU2NBAYGioeHh+zdu1cqKirkwIED8uijjwoAmTt3bpPrsWdc2dv3lo6fSMfuAx4+fLiUlZVJTU2NiIhUVFTIoUOHZPHixeLp6Sl+fn6ybdu2Tqv/yvFXVVUlrq6usnr1asnLy5PKykr56KOPxNPTU1JSUtq87bbq0fcB19bWyrhx46RXr14CQAwGg9x4443y0ksvWbXbuHGjREREiLu7u0RGRsr8+fPl6NGjommauLi4yA033CDFxcXyt7/9TX7zm9+IpmmWH8K77rpLRKTFeZerr6+X8PBwue+++9q3M1pw/vx5WbBggVx33XViMBhEr9dLeHi4JCcny8GDBy3tCgsL5ZFHHpE+ffqIm5ubmEwmufnmm+X999+3Wl9mZqb4+/sLAHF1dZXhw4dLSUmJbNu2Tfr162f5IYyKipL8/Px2LyPyyw/19OnTJTg4WHQ6nZhMJomNjbW5l9fePj700EPi6ekpJpNJJk6caPVAgr3b+vLLL+X2228XLy8vcXd3l5iYGHnzzTclJiZGAMh1110nP/zwQ7uOVUvj5ZNPPpHo6GgxGAwSFhYmc+fOlcOHDwsA0ev1kpCQYLM+e8dVa31fu3at+Pn5WY5fZGSkbNy40WoddXV1MnbsWAkKCpJDhw61uL3PP/9cRo4cKb6+vmIwGMTNzU1cXFws48BgMEhISIjccccd8tJLL0l5eXmH6m/r+KutrZXk5GQZNGiQGI1G0el00qdPH3nsscdsarF33LRFZwWwJtKO+5Oa0XhN0tGPPapy0003YenSpT3+qwipa3FcOZ+jR48iMjISixYtwuLFix212kynuQbc1U6cOIGSkhL+kJBDcVzR5RjA/5adnW11x8EzzzxjeaLoSvv377f50uqWXnfddVdXdYOaoeqYtWVc0dWHD2L8286dO3H06FEkJyfj1VdfRU5OTpOP0gJATExMu54sI3VUHbO2jCu6+vAM+N98fX0xb948BAQEYN++fdi0aRO/l5g6jOOKWsIz4H9LTU1Famqq6jKoh+G4opbwDJiISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkiMO/kL2srAyZmZmOXi0RkTKnTp3qlPU6PIBPnTqFxMRER6+WiKjHcWgAv/DCCygtLXXkKomalZiYiN/+9reYN2+e6lLoKhEZGenQ9Tk0gG+//XZHro6oVWFhYZg6darqMojahR/CEREpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJqLEDaQAAF6xJREFUiBRhABMRKaJTXQBRSyorK/HnP/+52fnHjh3D888/bzO9T58+mDZtWmeWRtRhDGDq1oxGIzIyMnD48OEm5x88eBAHDx60mb548eJOroyo43gJgrq99pzJJicnd0IlRI7FAKZuLyUlBZqm2dXWxcUFN954IyIiIjq5KqKOYwBTtxceHo4RI0bYFcINDQ08+yWnwQAmpzBt2jSISKvtXFxckJiY2AUVEXUcA5icQlJSElxdXVtso2kabr75ZoSEhHRRVUQdwwAmp+Dv749bbrmlxcsQIsJbz8ipMIDJabR2GcLNzQ0JCQldWBFRxzCAyWncfffdMBgMTc7TNA133HEHfHx8urgqovZjAJPTuOaaa3DHHXc0OY+XH8gZMYDJqTQXskajEfHx8V1cDVHHMIDJqcTHx8PLy8vmw7iEhAR4eHgoqoqofRjA5FQMBgMSEhJsPozj5QdyRgxgcjqXh62mafD29sbYsWMVVkTUPgxgcjq33nor/Pz8oGkaRAT33HMP9Hq96rKI2owBTE5Hp9MhKSnJchmC3/1AzkoTex6wJwBAfn4+5s2bp7oMAlBcXIydO3eiV69emDhxot3flkadJywsDCtXrlRdhjPJZAC3wdGjRxEZGam6DKJuyWw248iRI6rLcCaZvATRDosWLYKI8KX4tWDBAnz77bfK6+BLYDabVf9YOiX+SSJyWo8++iiCgoJUl0HUbjwDJqfF8CVnxwAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAVmzhwoUICQmBpmm49tprVZdDRF2IAazYs88+i/z8fHh4eKguxent3r0b0dHRMBgM0DTN6uXq6gqTyYSIiAjcfffd+Pvf/46KigrVJdNVjgFMPcbo0aNx4MABrF69GgAwcuRIyx+NLCkpweeff47HHnsMP/zwA2bOnIlBgwZh+/btiqumqxkDmLqFp556Cs8991ynrd/LywuRkZGYO3cu9u/fj7S0NJw6dQrx8fHIzs7utO12d52936llDOBuws3NTXUJyuTm5uKFF17osu25urpixYoVSEhIwKVLl/DQQw+hrq6uy7bfXXT1fidbDOAudvbsWaSmpqJv377Q6/UwmUwYPHgwqqqqbNquXbsW/v7+0DQNqampKC8vx7Jly2A2m7Fq1SrL+h599FEMGDAA7u7u8Pb2xpgxY/DBBx/YrG/9+vXw9fWFpmmYOXMm7r//foSEhMDd3R2DBg3C6tWrISJN1mzvNtLT02EymaBpGtLT0y3Tn332WQQFBUHTNIwaNcqqfUxMDKqrq/Hkk09C0zSEhoYCAOrr6zF+/HiEhobiyJEjbd/ZrVi6dCkA4NixY5azYHv2eVv2SXv3eVu20dZ93rhMc/udupCQ3Y4c+f/t3X9M1PUfB/Dn5zi444AdggjyQwSl6YFB2ZyJ0g8tS5imDky0Nm1Ol7W5hj+WrETbWFKt3Nfc1Ip+rB+Qy1mhlmWlmWmZKfmLnDhEREUFFIkfvr5/tMPOA7k77njf4fOxucnn8/583i9en+PJ5z73ueMvASAvv/yyS9tfuHBBEhISJDg4WEpKSuT69evS3NwsVVVVEhgYKOHh4Xbb7N+/XwDIiBEjJC0tTTIzMyUpKUmKiork4sWLMnjwYImLi5Mff/xRmpubpbKyUnJzcwWAvPnmm3b7+/LLLwWAZGVlycmTJ6W5uVmOHz8uM2fOFACydOlSm/GuzPHJJ58IAFm+fLnd9w9A0tPTbZZv2LBBAEhhYaHN8oqKCgEgAGTVqlUO93ndunWdztOZxMREASD5+fkdy27Xc1d64mzPXZnD2Z6LdN13V1gsFrFYLD3ezx2mhGfAvWjFihU4deoUXnnlFWRnZ8NoNMJgMCA2NhY63e0PRUJCAg4cOICvvvoKa9euBQAUFBSgsrIShYWFyMjIgMFgQHx8PN5//31ERkZiyZIlqK2t7XR/Q4YMQWJiIgwGA+666y4UFxcjLi4Or732GqqrqzvG9WSOnkpISMD48eMRFRWFyZMne2SOoUOHAoDN9/zf+W/tOeB6TxzteU/mIN/CAO5Fn376KQDgySefdHrbwMBAaJoGAHjkkUeQl5eHzz//HACQmZlpM1av12PKlCloaWnp9DJBZwICApCVlYX29naUlZV1LHfnHM7y8/PDjh07UFNTg7vvvtsjc4SEhAAAGhsb7dZ11nPAfT3pqufunIO8GwO4l9TU1KCurg5hYWGIjIzs8f6uXLmCmpoaBAcHIzQ01G794MGDAQBHjx51eJ8JCQkA/r0m6qk5vI31XmCz2ezQeHf35Naee2IO8l4M4F5SX18PwPEf9O5cvXoVABAcHNzpeusPrnWcIwICAgAATU1NHpvD25w+fRoAkJiY6NB4d/fk1p57Yg7yXgzgXmI0GgEALS0tbtnf7Z46A8Dly5dtxjni1l8SnpjDm9TV1eHEiRMAgIyMDIe2cXdPOvvF3Nf7TjcxgHtJbGwsDAYDzp071+ktZ84ym82IiYnBtWvXOv1BraqqAgAkJyc7vE/r0+DU1NQezWG9burtiouL0d7ejpSUFLvbtLri7r7f2nNX5/CVnpMtBnAv0ev1ePjhh9He3o4vvvjCLfvMyckBAGzdutVmeUtLC8rKymAwGBy+e+Dq1avYtm0bQkJCbF74cWWO8PBwADfP1Lqj1+sBwO7NEO3t7Zg4cSJiY2NRXl7u0L4cdfDgQaxcuRIBAQHYsGGDU9u6q+9d9dyVOZztOdB136kXqb4Rzpf09D7g3377TYxGo0RERMhXX30l9fX1UllZKXPmzBEAt70PeMaMGXbr6urqJDExUaKjo+X777+X5uZmOXXqlGRnZ3d7H/DTTz8tly9flra2NqmsrJTJkycLAHn33Xd7PMeFCxckKChIBgwYIAcOHJCWlha5cOGCbNmypdN7Unfv3i0AZOzYsVJfXy91dXVy7dq1Ht8HPGrUKGloaJCWlhYREbl27ZocPnxYVqxYIcHBwRIRESHffvutUz13pSfO9tyVOZzt+e367greB+ySEgawE3oawCIie/fulUcffVRCQkLEYDBIWlqafPjhh5KWliYAZPjw4XLixAkREdm4caNEREQIAPHz85Pk5GTZtGmTzf5qa2vlueeek0GDBom/v7+YzWZ58MEHZfPmzZ3Obw2DgQMHSlRUlPj7+0twcLBkZGTI119/3ek2zs4hIrJp0yZJSkoSg8EgycnJsmTJEjly5IhomiY6nU7uvfdeqaur6xg/f/58CQ4OFrPZLJmZmVJVVSVtbW0yYcIEGThwoBw+fLjb3u7Zs0fS09MlPDxcjEaj+Pv7i06n6whxo9EoMTExMmnSJFmzZo00Njba7cORnjvbE1d67uwcrvRcpPO+u4IB7BIGsDPcEcCqWcNg8eLFqku5Y9wJPWcAu4TvhLtTtba2qi7hjsOe060YwHco6eIDYMhz2HO6FQOYiEgRBvAdpKioCE899RQAYO3atRgxYgS2bdumuKq+jT2n29GEz4scduTIESQnJ+Pll1/GihUrVJdD5DWsbwrxxOc292GlPAMmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJSRK+6AF9UWlqKI0eOqC6DyGucOXMGsbGxqsvwOQxgFxw5coQBTEQ9xr8JRz5L0zTk5OTgs88+U10KkSv4N+GIiFRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREijCAiYgUYQATESmiiYioLoKoK01NTRg/fjxaW1vt1pWXl8NsNiMuLs5uXUZGBt54443eKJHIVaV61RUQ3Y7JZEJYWBjKyso6XX/+/HmcP3/ebvnChQs9XRpRj/ESBHm9mTNnOjXeYDBg2rRpHqqGyH0YwOT1pk6dCpPJ5NBYTdOQlZUFs9ns4aqIeo4BTF4vKCgIWVlZDo0VEafPmIlUYQCTT3A0VENCQjBp0iQPV0PkHgxg8gmTJk1CaGgoNE277bjp06cjMDCwl6oi6hkGMPmEgIAATJ8+Hd3dNcnLD+RLGMDkM24XrpqmITw8HA8//HAvVkTUMwxg8hkPPfQQoqOjO70MISLIzc2FXs9b28l3MIDJZ+h0OuTk5HR5GYKXH8jXMIDJp3QWspqmYdCgQRg9erSCiohcxwAmnzJq1CgMHToUOt3Nh66IYPbs2d3eIUHkbRjA5HNmzpyJGzdu2C0j8jUMYPI5ubm5Hf/X6XQYPnw4UlJSFFZE5BoGMPmcYcOGITU1FQBw48YNzJ49W3FFRK5hAJNPsp4Fa5rGyw/ks/iB7B7AF4Oor8nOzkZJSYnqMvoafiC7p8TGxuL+++9XXUaftnPnTsTFxWHo0KGqS+nTSktLVZfQZzGAPeT+++/nGYOHrV+/HlOmTEFkZKTqUvo0PqPzHAYw+ay5c+fyrcfk0/giHPkshi/5OgYwEZEiDGAiIkUYwEREijCAiYgUYQATESnCACYiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGsA965513kJSUBJ1OB03TUFlZqbqkLl24cAG5ubkIDQ1FUFAQFi9erLokIq/BAPZBzzzzDCoqKpCRkaG6FDQ0NMBisWDIkCGora21Wz9lyhRUVVWhsrISq1evRlVVlUPb9aZdu3YhNTUVRqMRmqbZ/PPz84PZbEZSUhKmT5+O9957D9euXVNaL/UdDGDqERGBiHS6bt++ffjll18wdepUhIaGYtasWQgICOh2u942btw4/Pnnn3jzzTcBAOnp6R31XblyBXv27MGiRYtw4sQJzJ07F8OGDcOOHTsUV019AQOYesRsNuPo0aM4efKk3V8nPnbsGAAgJCQEABAaGooPPvig2+28SUhICJKTk7Fw4UIcPHgQeXl5OHPmDLKysvDTTz+pLo98HAOYPKampgZA3/mz5n5+figqKsK0adPwzz//YP78+Whra1NdFvkwBrAXampqQkFBAVJSUhAYGAiTyYSYmBiMHTsWf/31V7fbz5o1CxaLBaGhoTCZTEhJScH69ettxly/fh35+fkYOnQojEYjIiMjkZ6ejp9//tnhMRUVFUhLS4Ofnx80TcPff/8NADh06BBGjx6N5cuXAwDmzZvXcU31t99+63I7q9OnT+Opp57CgAEDYDKZkJqaitWrV+PGjRsAgI0bN2LAgAHQNA3PP/88GhsbUVhYCIvFgjfeeAPt7e147LHHEBsb61C/nLVq1SoA/57h33oW3F3tpaWliIyMhKZpePbZZ7F161ZkZ2cjIiIC/fr1w6xZs9DQ0GA3pyPHy5H5ycsIuR0Ayc7Odmnbq1evysiRI8VoNMr7778vV69elUuXLsn27dslJiZGdu7c2TH2gQceEABy6tQpm30YDAYpLi6W2tpaqampkTFjxggA2b59e8eYOXPmSHh4uPzyyy/S1NQkBw4ckJEjR8qHH37o1BgRkYkTJwoAqaiosFleWFgoAGTDhg2dfq+dbXf27FmJioqSiIgI+fXXX+X69etSUFAgAOTFF1/sGLd//34BICNGjJC0tDTJzMyUpKQkKSoqkoqKCgEgAGTVqlUO937dunUCQNLT07sdm5iYKAAkPz/f5dotFots2rRJLl68KJcvX5YFCxYIAFmyZIndfI4cC0fnd1ZPHs90WyUMYA/oyQN2+fLlAkBeffVVu3ULFixwKIDHjh1r8/UHH3wgACQvL09ERNrb2yUoKMiuxk8//bTjB9qRMVbuDGBrCBUXF9uMHTx4sJhMJmloaBCRmyE2efJkuXHjhoiIfPPNN1JUVCRtbW0yfvx4iYqKkj///LPTuTvjTAA/+uijAkDmzJnjcu0zZsywGXf+/PmOXyr/5eixcHR+ZzGAPaaElyC8zEcffQQAmD17tt26ZcuWYcSIEd3uY9euXTZfR0REAPj3aayViOCnn37C6dOnO5bNmDHDZl5Hxrjbl19+CQDIzMy0WT5mzBg0NTVh7969NssDAwM7rjE/8sgjyMvLg5+fH3bs2IGamhrcfffdHqnT+sJiY2Ojy7Xfynqc/rtPK0eORU/np97HAPYiDQ0NOH36NEJCQhAdHW23Pj4+HuHh4T2eR6fTIScnB7W1tbBYLJg3bx52797t9Bh3a2pqQnV1Nfz9/dG/f3+bdWFhYQCA6upqj9bgKOu9wGazGYB7a5dbbs9z5Fj4Uu/oJgawF7G++BIUFOTyPurr67F06VLcc8896N+/P0wmk90ZEQCsW7cO+fn5MJlM2LhxI8aNG4dRo0bZvCDmyBh3soZaa2ur3Rsi/ve//wEAmpubPTK3s6xnoomJiQA8X3t3x8KXekc3MYC9iPVp7aVLl1zeR1ZWFlavXo1Zs2bh999/x8WLFzuemv6X0WjEqlWrcPbsWZSVlSEjIwP79+9HTk6OU2PcyfqLJyQkpOONELf+W7BggUfmdkZdXR1OnDgBAB3vRvR07d0dC1/pHdliAHsRs9mM6OhotLS0uPT5DtXV1di9ezdGjx6NvLw8xMfHw2QyQafr+jD7+/vj8ccfx3fffQeLxYI//vjD7q3BjoxxB5PJhIEDB6KxsdGr3+5bXFyM9vZ2pKSkYOzYsQB6r/aujoWv9I5sMYC9jPWM5u2333Z6W+uLbKGhobcd19zcjGnTptks0+v1sFgs0Ov16Nevn0NjPOHxxx8HAHz++ecu76O9vR0TJ05EbGwsysvL3VUaAODgwYNYuXIlAgICsGHDBpt17qi9M44eC0/NT57DAPYyL730EpKSkvD666+joKAAlZWVaGtrQ1NTE44dO4bjx493uW18fDyioqKwa9cu7Nu3D01NTTh06BC2bt1qN3bLli146623cObMGVy/fh1lZWXYtm0bZsyY0fF5DY6Mcbf8/HyEhYXhhRdewMcff4z6+nq0tbXh3Llzdm866MqpU6fwzTffoLq6Gps3b3a6htbWVjQ2NqK1tRXAvy9wlZeXo6CgAOPGjYPBYMDXX3+N0aNHu732rjhyLDw5P3lIb970dqdAD++bvHTpkixbtkyGDx8uRqNRAgICJD4+XnJzc+XQoUPy7rvvyl133SWapgkACQ8PlyeeeEJERH744QdJTU0Vo9EocXFxsnDhQikvLxcAEhAQINOmTZPW1lbJzc2VYcOGiclkEr1eL4MGDZJFixZJY2OjiIhDYz766CMZNmyY6HQ6ASBhYWGSlZUlra2tMnHiRAkMDBQAYjQa5b777pM1a9bcdjuriooKmT17tkRHR4terxez2SxjxoyRwsJCERHZuHGjRERECADx8/OT5ORk2bRpU8f2bW1tMmHCBBk4cKAcPny4237v2bNH0tPTJTw8XIxGo/j7+3fUZq0/JiZGJk2aJGvWrOn4/jvTXe2lpaUyYMCAjtpHjRolV65ckW+//VYSEhI65kxJSZHq6mqHj4Wj87uip49n6lKJJuIlH0nVh2iahuzsbJSUlKguhajH+Hj2mFJegiAiUoQBTESkCAOYiEgRBjARkSIMYCIiRRjARESKMICJiBRhABMRKcIAJiJShAFMRKQIA5iISBEGMBGRIgxgIiJFGMBERIowgImIFGEAExEpwgAmIlKEAUxEpAgDmIhIEQYwEZEiDGAiIkUYwEREiuhVF9BXlZaWQtM01WUQkRdjAHtAdna26hKI3GrMmDGqS+iTNBER1UUQEd2BSnkNmIhIEQYwEZEiDGAiIkUYwEREijCAiYgU+T/g0gofJBQ+TgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.keras.utils.plot_model(news_classifier, rankdir='TB', dpi=120)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rMP5y7SwKJS9",
   "metadata": {
    "id": "rMP5y7SwKJS9"
   },
   "source": [
    "<a name='2-5'></a>\n",
    "\n",
    "## 2.5 Training a Finetuned BERT"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "oDYULiuLKPcD",
   "metadata": {
    "id": "oDYULiuLKPcD"
   },
   "source": [
    "Before training a model, we have to compile it. When compiling the model, we specify the loss and optimization function. \n",
    "\n",
    "Loss function is there to measure how far the predictions are from the actual outputs, and optimizer to reduce the loss. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "lh2sQNQ5JlWz",
   "metadata": {
    "id": "lh2sQNQ5JlWz"
   },
   "outputs": [],
   "source": [
    "news_classifier.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=3e-5), \n",
    "                        loss='sparse_categorical_crossentropy', \n",
    "                        metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "HTSFsQe0K9Mz",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "HTSFsQe0K9Mz",
    "outputId": "4bd3722d-d2d8-4601-f692-43b35e73e43e"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/15\n",
      "117/117 [==============================] - 872s 7s/step - loss: 0.9071 - accuracy: 0.5710 - val_loss: 0.7042 - val_accuracy: 0.6205\n",
      "Epoch 2/15\n",
      "117/117 [==============================] - 865s 7s/step - loss: 0.6153 - accuracy: 0.7639 - val_loss: 0.4422 - val_accuracy: 0.8839\n",
      "Epoch 3/15\n",
      "117/117 [==============================] - 867s 7s/step - loss: 0.4662 - accuracy: 0.8288 - val_loss: 0.3497 - val_accuracy: 0.8929\n",
      "Epoch 4/15\n",
      "117/117 [==============================] - 865s 7s/step - loss: 0.4057 - accuracy: 0.8632 - val_loss: 0.3616 - val_accuracy: 0.8973\n",
      "Epoch 5/15\n",
      "117/117 [==============================] - 866s 7s/step - loss: 0.3775 - accuracy: 0.8659 - val_loss: 0.3295 - val_accuracy: 0.9107\n",
      "Epoch 6/15\n",
      "117/117 [==============================] - 863s 7s/step - loss: 0.3795 - accuracy: 0.8657 - val_loss: 0.3112 - val_accuracy: 0.9062\n",
      "Epoch 7/15\n",
      "117/117 [==============================] - 859s 7s/step - loss: 0.3613 - accuracy: 0.8707 - val_loss: 0.3071 - val_accuracy: 0.9107\n",
      "Epoch 8/15\n",
      "117/117 [==============================] - 862s 7s/step - loss: 0.3399 - accuracy: 0.8835 - val_loss: 0.3287 - val_accuracy: 0.9107\n",
      "Epoch 9/15\n",
      "117/117 [==============================] - 858s 7s/step - loss: 0.3523 - accuracy: 0.8798 - val_loss: 0.3150 - val_accuracy: 0.9062\n",
      "Epoch 10/15\n",
      "117/117 [==============================] - 858s 7s/step - loss: 0.3322 - accuracy: 0.8897 - val_loss: 0.2895 - val_accuracy: 0.9196\n",
      "Epoch 11/15\n",
      "117/117 [==============================] - 858s 7s/step - loss: 0.3323 - accuracy: 0.8897 - val_loss: 0.2966 - val_accuracy: 0.9062\n",
      "Epoch 12/15\n",
      "117/117 [==============================] - 857s 7s/step - loss: 0.3362 - accuracy: 0.8862 - val_loss: 0.2875 - val_accuracy: 0.9018\n",
      "Epoch 13/15\n",
      "117/117 [==============================] - 856s 7s/step - loss: 0.3317 - accuracy: 0.8902 - val_loss: 0.2764 - val_accuracy: 0.9196\n",
      "Epoch 14/15\n",
      "117/117 [==============================] - 859s 7s/step - loss: 0.3574 - accuracy: 0.8801 - val_loss: 0.2794 - val_accuracy: 0.9018\n",
      "Epoch 15/15\n",
      "117/117 [==============================] - 861s 7s/step - loss: 0.3299 - accuracy: 0.8870 - val_loss: 0.2577 - val_accuracy: 0.9107\n"
     ]
    }
   ],
   "source": [
    "# Train the model\n",
    "\n",
    "batch_size = 32\n",
    "\n",
    "train_steps = int(len(train_data)/batch_size)\n",
    "val_steps = int(len(val_data)/batch_size)\n",
    "\n",
    "history = news_classifier.fit(train_data, \n",
    "                    epochs=15,\n",
    "                    validation_data=val_data,\n",
    "                    steps_per_epoch=train_steps,\n",
    "                    validation_steps=val_steps\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "Ahee5IyNNyFp",
   "metadata": {
    "id": "Ahee5IyNNyFp"
   },
   "source": [
    "<a name='2-6'></a>\n",
    "\n",
    "## 2.6 Visualizing the Model Results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "OQWZqy0RN7tl",
   "metadata": {
    "id": "OQWZqy0RN7tl"
   },
   "source": [
    "It is always a good practice to plot the model results. When doing error analysis, plotting learning curve becomes helpful. \n",
    "\n",
    "Let's plot the loss and accuracy over on the course of epochs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "9o6jE3u3Lhkp",
   "metadata": {
    "id": "9o6jE3u3Lhkp"
   },
   "outputs": [],
   "source": [
    "# function to plot accuracy and loss\n",
    "\n",
    "def plot_acc_loss(history):\n",
    "\n",
    "  model_history = history.history\n",
    "  acc = model_history['accuracy']\n",
    "  val_acc = model_history['val_accuracy']\n",
    "  loss = model_history['loss']\n",
    "  val_loss = model_history['val_loss']\n",
    "  epochs = history.epoch\n",
    "\n",
    "  plt.figure(figsize=(10,5))\n",
    "  plt.plot(epochs, acc, 'r', label='Training Accuracy')\n",
    "  plt.plot(epochs, val_acc, 'g', label='Validation Accuracy')\n",
    "  plt.title('Training and validation accuracy')\n",
    "  plt.xlabel('Epochs')\n",
    "  plt.ylabel('Accuracy')\n",
    "  plt.legend(loc=0)\n",
    "\n",
    "  # Create a new figure with plt.figure()\n",
    "  plt.figure()\n",
    "\n",
    "  plt.figure(figsize=(10,5))\n",
    "  plt.plot(epochs, loss, 'b', label='Training Loss')\n",
    "  plt.plot(epochs, val_loss, 'y', label='Validation Loss')\n",
    "  plt.title('Training and Validation Loss')\n",
    "  plt.xlabel('Epochs')\n",
    "  plt.ylabel('Loss')\n",
    "  plt.legend(loc=0)\n",
    "  plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "AwDAEUfAN_BD",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 700
    },
    "id": "AwDAEUfAN_BD",
    "outputId": "608ec765-f238-4f20-b850-d1c2645cd169"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAFNCAYAAABFbcjcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXxU9b3/8dcnC0sCYcvGTtDIlhgQKrizaNVqtW6VUG2trUs3q7e93ra3i9e299fe3tvtdrm1rbW2GtfaS1uv1mFxt4ITEAggCCiBBEJYQ0K2+f7+OJMwCQmZhJnMTPJ+Ph7zmJkz55z5zBDIm+9yvuacQ0RERETiQ1KsCxARERGR4xTOREREROKIwpmIiIhIHFE4ExEREYkjCmciIiIicUThTERERCSOKJyJ9BNm9n9m9olI7xtLZrbDzC6OwnmdmZ0efPw/ZvaNcPbtwft8zMz+3tM6RaRvMl3nTCR+mVlNyNM0oB5oDj6/wzn3SO9XFT/MbAfwaeecL8LndUC+c25rpPY1s0nAdiDVOdcUiTpFpG9KiXUBItI559yQlscnCyJmlqJf+BIv9PMocmrUrSmSgMxsvpmVm9m/mFkl8DszG2FmfzWzKjM7EHw8LuSYlWb26eDjW8zsFTP7z+C+283s8h7um2dmL5nZETPzmdnPzeyPndQdTo3fNrNXg+f7u5llhrx+s5m9Z2bVZvavJ/l+5ppZpZklh2y7xszeDj4+28xeN7ODZlZhZj8zswGdnOshM/tOyPN/Dh6z28xubbfvFWZWamaHzWynmd0X8vJLwfuDZlZjZue0fLchx59rZqvM7FDw/txwv5tufs8jzex3wc9wwMz+HPLa1Wa2JvgZ3jWzy4Lb23Qhm9l9LX/OZjYp2L37KTN7H1ge3P5k8M/hUPBnZEbI8YPN7L+Cf56Hgj9jg83sb2b2hXaf520zu6ajzyrSFymciSSuXGAkMBG4He/v8++CzycAdcDPTnL8XGAzkAn8B/BbM7Me7Pso8CYwCrgPuPkk7xlOjUuATwLZwADgywBmNh34ZfD8Y4LvN44OOOf+ARwFFrY776PBx83APcHPcw6wCPjsSeomWMNlwXouAfKB9uPdjgIfB4YDVwCfMbOPBF+7MHg/3Dk3xDn3ertzjwT+Bvw0+Nl+CPzNzEa1+wwnfDcd6Op7/gNeN/mM4Ll+FKzhbOBh4J+Dn+FCYEdn30cHLgKmAZcGn/8f3veUDfiB0G74/wRmA+fi/RzfCwSA3wM3texkZkXAWLzvRqRfUDgTSVwB4FvOuXrnXJ1zrto597RzrtY5dwT4Lt4vy86855z7tXOuGe8X4mggpzv7mtkE4APAN51zDc65V4Clnb1hmDX+zjn3jnOuDngCmBncfj3wV+fcS865euAbwe+gMyVAMYCZDQU+FNyGc+4t59wbzrkm59wO4Fcd1NGRjwbrW++cO4oXRkM/30rn3DrnXMA593bw/cI5L3hhbotz7g/BukqATcCHQ/bp7Ltp42Tfs5mNBi4H7nTOHXDONTrnXgwe+ingQefcC8HPsMs5tynM+gHuc84dDdaHc+5B59yR4J/XfUCRmQ0zsyTgVuCLwfdods69FtxvKXCGmeUHz3kz8LhzrqEbdYgkNIUzkcRV5Zw71vLEzNLM7FfBbqLDeN1ow0O79tqpbHngnKsNPhzSzX3HAPtDtgHs7KzgMGusDHlcG1LTmNBzB8NRdWfvhddKdq2ZDQSuBfzOufeCdZwR7OqrDNbx73itaF1pUwPwXrvPN9fMVgS7Ew8Bd4Z53pZzv9du23t4rUYtOvtu2ujiex6P92d2oINDxwPvhllvR1q/GzNLNrPvBbtGD3O8BS4zeBvU0XsFf6YfB24KhrhivJY+kX5D4UwkcbWfav0lYAow1zmXwfFutM66KiOhAhhpZmkh28afZP9TqbEi9NzB9xzV2c7OuTK8cHM5bbs0wese3YQ3yzID+FpPasDrMgz1KF7Lz3jn3DDgf0LO29XU+N143ZChJgC7wqirvZN9zzvx/syGd3DcTuC0Ts55FK8rtEVuB/uEfsYlwNV4Xb/DgEkhNewDjp3kvX4PfAyvu7m2fRewSF+ncCbSdwzFG1t0MDh+6VvRfsNgS9Rq4D4zG2Bm59C2Gy6SNT4FXGlm5wcH799P1/+GPQp8ES+cPNmujsNAjZlNBT4TZg1PALeY2fRgOGxf/1C8VqljwfFbS0Jeq8Lrhp3cybmfxevOW2JmKWZ2IzAd+GuYtbWvo8Pv2TlXgTcW7BfBiQOpZtYS3n4LfNLMFplZkpmNDX4/AGuAxcH95+B1M3dVQz1e62YaXutkSw0B4EHgh2Y2JtjKdk6wlZNgGAsA/4VazaQfUjgT6Tt+DAzGa5V4A3iul973Y3iD6quB7+B1SdV3sm+Pa3TObQA+hxe4KoADQHkXh7WM+VrunNsXsv3LeMHpCPDrYM3h1PB/wc+wHNgavA/1WeB+MzsCfBMvzLUcW4s39utV82aJzmt37mrgSrxWr2q8AfJXtqs7XF19zzcDjXith3uBu4M1vIk34eBHwCHgRY635n0Dr6XrAPBvtG2J7MjDeC2Xu4CyYB2hvgysA1YB+4Hv0/Z30sNAIdDhzF+RvkwXoRWRiDKzx4FNzrmot9xJ32VmHwdud86dH+taRHqbWs5E5JSY2QfM7LRgN9hleOOM/tzVcSKdCXYZfxZ4INa1iMSCwpmInKpcYCVQg3eNrs8450pjWpEkLDO7FG983h667joV6ZPUrSkiIiISR9RyJiIiIhJHFM5ERERE4khKrAuIlMzMTDdp0qRYlyEiIiLSpbfeemufcy6ro9f6TDibNGkSq1evjnUZIiIiIl0ys/bLtbVSt6aIiIhIHFE4ExEREYkjCmciIiIicUThTERERCSOKJyJiIiIxBGFMxEREZE4onAmIiIiEkcUzkRERETiiMKZiIiISBxROBMRkR45UHeApZuX8sK7L1DbWBvrciQM1bXVPFX2FO/ufzfWpchJ9Jnlm0REJLrqGut45f1XWLZ9Gcu2L8Nf4SfgAgAMSB7AuePPZVHeIhblLeIDYz9ASpJ+xcSDmoYalm5eyqPrHuX5d5+nKdAEwNljz2ZJwRI+OuOjjB46OsZVSihzzsW6hoiYM2eO09qaIiKR0xRoYvXu1Szb5oWx13a+Rn1zPSlJKcwbN49FeYtYmLeQ2sba1n3WVK7B4Rg6YCgXTbqoNawVZBdgZrH+SP1GQ3MDz219jpL1JSzdvJTaxlrGZ4xnccFirjzjSv5R/g9K1pdQWllKkiWxYNICiguKuXbatYwYPCLW5fcLZvaWc25Oh68pnImICIBzjg1VG1qD1ovvvcjh+sMAzMyd2Rq0Lph4AUMGDOnwHPtq97Fi+4rW1rWt+7cCkJ2ezcK8ha3nyBuR12ufq79oDjTz4nsvUrKuhKc2PsXBYwfJTMvkhuk3sKRwCeeOP5ckazuaadO+TZSsK6FkfQlb9m9hQPIALj/9cooLivnwlA+TlpoWo0/T9ymciYhIh3Yc3NEaxpZvX86eo3sAOG3EaV6QmryIBZMWkJWe1aPzv3/o/dbzL9u+jMqaSgDyhue1nn9h3kKy07Mj9pn6E+ccq3avomRdCY9veJyKmgqGDBjCNVOvobigmIsnX0xqcmpY53mr4i1K1pXw2IbH2H1kN+mp6Xxk6kdYUriESyZfEtZ5JHwKZyIiAkDV0SpW7FjBsm3L8G33se3ANgBy0nNYNHlRa8vWxOETI/7ezjk27tvYGtZW7ljJofpDABRmF3Lx5ItZlLeICydeyNCBQyP+/n3JxqqNPLruUUrWl/DugXcZkDyAK/KvoLigmCvOuOKUWryaA828/P7LPLruUZ4qe4oDxw4wavAorp9+PUsKl3D+hPNPaIGT7lM4E4kT9U31JCcla6B0J5oCTRypPxLrMtpIsiQyBmYk7HipmoYaXnrvpdZAtHbPWgAyBmYwf9L81jA2PWt6r3/GpkAT/go/vm0+lm1fxqvvv9o6pu3ssWe31jZv3DwGpgzs1dri0fuH3uex9Y/x6LpHWbtnLUmWxMK8hSwpWMI1065h+KDhEX/PhuYGnt/6PCXrS/jfzf9LbWMt4zLGsXjGYooLi5mVOyth/250pinQRHVtNTlDcqL6PgpnIjFS31TPG+VvtHbpvLnrTZoDzYxKG0V2ejY56Tlkp2ef+HjI8cedje1JBM45ahpq2Ht0L3uP7mXP0T3HH9fsYW9tyOOje6muq451yR1KTUolKz2r9c8oZ0gO2Wkn/lnlpOeQlZ7FgOQBMau1obnB+5kLhrF/7PoHTYEmBiYPPD6bcvIi5oyZE3f/SahrrOO1na+1/n1ZvXs1ARdgcMpgLph4QWtYm5k7k+Sk5FiX2yuqjlbxZNmTPLruUV7d+SoA88bNY0nBEm6YcQO5Q3J7rZajDUdZunkpJetLeG7rczQGGjlj1BksKVhCcWExZ4w6o9dqiQTnHLuP7Gbd3nWs37uedXvXsW7POsqqypiaOZU1d66J6vsrnIn0kuZAM2sq17T+cnn5vZepa6ojyZKYM2YOCyYtYFDKoDZBpSWYtHTvtJeWmtZxgOsgzI0aPCrqv7SaA83sq913QuBq+Rx7a0MeH91LXVNdh+cZPmh4h59p2KBhGPHzP/GmQFPr520TLo/u4VjTsQ6PGT5oeId/Vu2DXHZ69im3ygVcgLWVa1tbn15+/2VqG2tJsiRmj57dGsbOG38eg1MH9/h9YuHgsYO8uOPF1r9PZVVlAIwYNIIFeQtYlLeIiydfTP7I/D7VenO4/jB/3vRnStaX8MK7L9DsminILqC4oJjFBYuZPGJyrEtkf91+ni57mpL1JazcsRKHY/bo2RQXFHNjwY2MyxgX6xLbOHjsIOv3rvdC2J51rYHswLEDrfuMGTqGwuxCCrILOGv0WSwpXBLVmhTORKLEOceW/VtafzGu2L6i9S/79Kzprf/Tv2jSRV12OdQ31bf+4u8oCLRpcTq6l2bXfMI5kiyJzLTME0NBJ61yLeNSjjYcDbt1a1/tPhwn/ruRkpRy0hAZ+r5ZaVkJ300V2irY0Z9P+5DaWavgwOSBrd9LaKtcR0EuMy2TlKQUtu7f2hpYVmxf0XruaZnTWsPY/Enzo9LNFUsVRypYvn05y7Yvw7fNx87DOwEYlzGu9e/aosmLGDN0TIwr7b5jTcd4dsuzlKwv4a/v/JVjTceYNHwSxQXFFBcUU5hTGOsSO7Xr8C6e2PAEj65/lNW7V2MYF068kOKCYq6ffj2j0kb1Wi31TfVs2reptRVsfZUXxlp+VsDr0i/ILqAwu7A1jBVkF/RqnaBwJhJRu4/sbjP7rPxwOQDjM8a3DmhemLcwqhd1DLgAB48dbPPLv00LVrtAdaSh43FcQwYMwTnH0cajHb6eMTAj7O7XEYNG9KnWi0hrbG5kX+2+sILcnqN7aGhu6PA8QwYMoaahBjgeSi6efDEL8xYmZCjpKedcpyF1fMZ4cofkdvkz2xJ2Y6Up0MTy7cspWV/Cnzb+icP1h8lOz+bGGTdSXFDMvHHzEu7v1JbqLZSs9y7NsWnfJlKSUrj0tEspLijm6qlXR2yYRsAF2H5g+wkh7J3qd1r/45qalMq0rGltQlhhTiHjM8bHxfcas3BmZpcBPwGSgd84577X7vWJwINAFrAfuMk5Vx587RPA14O7fsc59/uTvZfCmUTLgboDrNyxsvWXwKZ9mwAYNXhUa9fKorxFnD7y9Lj4C9+Rusa6Dlvl9tTswcw6bN3KTs9mUMqgWJfeLznnOFx/uMNWueq6aq+FbPKiPteddypauneXbV/Gur3r2obfo3tpDDSecIxhjEobFdZ/PnLSc0gfkH7KdTrneKP8DR5d9yhPlD3B3qN7yRiYwbXTrmVJwRIW5C2Iu7GAPeGcY+2etTy67lEeW/8YOw/vZHDKYK6achVLCpdw6WmXht16vqdmz/FxYcEuyQ1VG9osGTZ5xOQ2rWGFOYXkj8yP68t/xCScmVky8A5wCVAOrAKKnXNlIfs8CfzVOfd7M1sIfNI5d7OZjQRWA3MAB7wFzHbOHWj/Pi0UziRSOluiJi01jQsnXtgaxopyizSdXCQBOOc4eOxg2OMkuxr/2dVwgZz0HEYOHtlm/Oe6Peu8oLLhMXYc3MGglEFcecaVFBcU86H8D/Xp/wgFXIBX33+VkvUlPFn2JPtq9zF80HCum3YdSwqXcNHEi0hOSqamoabDcWFVtVWt58pKy6Iwp7BNa9iM7BkJOXEqVuHsHOA+59ylwedfBXDO/b+QfTYAlznndpr3379DzrkMMysG5jvn7gju9ytgpXOupLP3UziTngpdosa33cdrO1+jobmhzRI1i/IWMXfc3JjOwhOR3nGs6RhVR6vCGoPZ1fjPnPQcGpob2Fy9mWRL5pLTLqG4oJiPTP0IGQMzYvDpYquxuRHfNh8l60t4ZtMz1DTUkDskl8Epg9l+cHvrfmmpaa0tYa0tYjmFfepixScLZ9FsOx0L7Ax5Xg7MbbfPWuBavK7Pa4ChZjaqk2PHRq9U6U+6WqLmC2d/ocslakSk7xqUMojxw8Yzftj4LvcNuAAH6g6cdGJIfVM9d829ixum39DjlRb6itTkVC7Pv5zL8y+ntrGWv73zN57a+BQAt866tTWETRo+qV/3TMS6Y/vLwM/M7BbgJWAXcOJ/QTphZrcDtwNMmDAhGvVJH3GyJWoWz1h8ykvUiEj/lGRJjEobxai0UUzLmhbrchJKWmoaN8y4gRtm3BDrUuJONMPZLiD0vx3jgttaOed247WcYWZDgOuccwfNbBcwv92xK9u/gXPuAeAB8Lo1I1i7JBDnHEcajpzQ1bDn6B7eO/geK99b2atL1IiIiJyKaIazVUC+meXhhbLFQJsruplZJrDfORcAvoo3cxPgeeDfzWxE8PkHg69LP9Hmwp8dXS4ipPtg79G9nV4MdNTgUZw34Ty+OPeLMVuiRkREpDuiFs6cc01m9nm8oJUMPOic22Bm9wOrnXNL8VrH/p+ZObxuzc8Fj91vZt/GC3gA9zvn9kerVukdRxuOnjhLqpOLrVbXVnd4odPUpNQ2M6OmZ03vdOZUZlqmBvCLiEjC0UVo5ZQ55yg/XM76vet5p/qdTgNX6DVpQg0bOKztVPSQq6O3D1zDBg5Ty5eIiCS8WM3WlD7oQN2BNgvErq/yrklz8NjB1n2SLblNoMofld/pcjTZ6dkJv4yPiIhIJCmcSYeONR1jY9XG41dlDoaxXUeOz+kYNnAYhTmFLJ6xuPWigFMzpzIqbVS/ngIt0m84B5WVcKDT64PHRmoqjBgBw4dDSj/9NRcIwKFDsG/fibeDB2HyZJg9G2bM8L4viSv99KdWWjQHmtl2YNsJIWzL/i0EXACAAckDmJ41nQV5C9pclXlcxjh1MYr0F87Btm1QWgp+//H7vXtjXdnJZWR4QW3kSO8+9HFH21oeZ2RAvPz75hzU1HQctDq7VVdDcydXpkpK8sIbwMCBcOaZXlCbPRvOOgsKCmCAxuvGksac9RPOOSprKo8HsGAY27B3A3VNdYC3ztzkEZNbW8FarsqcPyq/T6z1JiJhamqCzZvbhrA1a7yWGPBao6ZP936Rz5oFubmxrbe9+nqvNe/AAdi//8THLfcNHS8uD3gBpiWsdTfcpaWdPNgdO9a9oLVvn/eZOpKcDJmZJ96ysjrenpkJgwfDu+/CW28dv/n9x/98BwyAwsK2ga2w0Atyfd2BA953U1sLF14Y1beK2cLnvUnh7Lgj9UdaQ1hoa1h1XXXrPjnpORTmFFKQVdAaxqZnTY/Iwr4ikkCOHYP169u2iK1d620HGDQIioq8ENYSxgoKvO2JzDmoqzsxsHUU4tpvO3jweMtTRwYMaBva0tK8Y1uC1tGjnR87cmR4AavlNmxYZFr4WlpG2we2li7r1FTvz70lsM2e7QW2RPs5CASgvNwLYNu2efeht4PB8dNTpsCmTVEtReGsj2oONFNWVXZCCHvv0Hut+6SnprdZl6ylRUxXwhfph44c8YJXaItYWZnXUgZeV15oCDvrLO+XVH8dt9WZQMD7LsMNdEePwqhRXQetESPi67t2DnbsaBvY3nrL+1zg1TpjRtvAduaZXstcLNXVwfbtbUNXSxDbvr1ti2lKCkycCKeddvw2eTLk53thNIoUzvqoW/58C79f+3sAUpJSmDJqygldkhOHT9TgfJHu2r0bXnwRVq70uvOGDPF+uY4c6d23f9zyPJ5+uVZXHw9gLWFsyxbvFy5AdnbbEDZrFuTled15Ip1xDt5773jLWktg27fPez052evyDg1sRUVe62Eka6iuPjF4tdx27267/9ChbYNXaBAbPz5mf2cVzvqoiT+eSP7IfH506Y+YkjlFF1wV6amKCi+ItdzeecfbnpEBc+Z4432qq73b/v2dD7QGb4ZgZ+Gts+dDh/a8a8o575dRaGtYaSm8//7xfSZOPLFFbPTo+BnwLonNOdi5s2136FtvHZ8skpQE06a1DWwzZ0L6SYbRNDV552wfvFqeHz7cdv8xY04MXi1hLDMzLn/WFc76oOraajJ/kMkPLvkBXz73y7EuRySxVFa2DWObN3vbMzK8QcDz53u3mTO9loBQznm/GFrCWktg6+p5y2DrjqSkdN0q1/I4I8P75RQaxqqqvPOYwRlntA1hM2d6x4n0Judg164Tu0T37PFeT0qCqVO9n9HZs73/8IQGsR07jne3gzfmLS/vxOB12mne9ki2zPUSXYS2DyqtLAXgrNFnxbgSkQRQWXm8m3LlyuMDfYcO9cLYpz99PIx11cVh5g3CHjbM++UQrsZGb/xROGGuZZxPdfXxgfntpaR4Y2KuvPJ4GCsq8rpgRWLNDMaN825XX318++7dbcPasmXwxz96rw0f7oWts86C669vG8TGjj3xP0p9mMJZgvJX+AGYmTszxpWIxKE9e9qGsY0bve1Dh8IFF8Ctt8KCBeGFsUhJTfXGeWVnd++42tq24e3gQZg0yRuI3R8ubSB9y5gx3u3DHz6+bc8e7+/HyJGxqyvOKJwlKH+Fn0nDJzFysH6YRdi793gYW7HieBgbMsQLY7fc4oWxWbPiZ8B+uNLSvNu4cbGuRCQ6cnJiXUHcSbB/paRFaWWpujSl/6qq8sLYihVeICsr87YPGQLnn++Fsfnzve6RRAtjItLv6V+tBHS4/jDvVL/DzWfeHOtSRHpHSxhr6abcsMHbnp7utYx9/OPHw5jWCRSRBKdwloDWVq4FNBlA+rB9+9qGsfXrve3p6V7L2E03eWFs9myFMRHpcxTOEpBmakpCc867rMTu3d71xVruWx5v3Ajr1nn7pqV5YWzJEi+MzZmjMCYifZ7CWQLyV/jJHZJL7pA4W2xY+jfnvEtFtA9bHd13dHmI9PTjF5JcvPh4GBugiyuLSP+icJaA/BV+tZpJ72lZKqWjVq722+rrTzw+I8O7Gv3o0XDOOd79mDFt70eP9i5zISIiCmeJpq6xjrKqMq6ecnXXO0vsOOctdnzkiHc1+SNHjt+am72rY5t5t+487ulxJzvH4cOdh63du70LuIYuFNxi2LDj4er8808MWy2PT7ZEi4iInEDhLMGs37ueZtfMrNGzYl1K3xMIQE3N8RDVEqpCw1VHjzt7PVGXRhsx4niwmj//xLA1Zgzk5ibkcikiIolA4SzBtKwMoG7NMBw9CiUl3tWnwwlXNTXhnTclxeuCy8jw7ocO9QLNhAltt3X2ODnZC26BgHd/Ko9P9RyBgHdtsNAWr0GDovvnIiIiJ6VwlmD8FX5GDBrBxGETY11KfHv1Ve9CpFu3es9TU48HpJb7zExv8HlnQaqzcDVokNclKCIiEgUKZwmmZWUAUzjoWF0dfOMb8MMfwsSJ4PN546G0BqGIiCSIpFgXIOFrbG7k7T1vMytX48069I9/eFeI/6//gjvu8K6VtWiRgpmIiCQUhbMEsnHfRuqb6zXerL36evja1+Dcc71xZn//O/zyl95YKhERkQSjbs0EUlqhlQFO4PfDJz7hLe9z661ed+awYbGuSkREpMfUcpZA/BV+0lPTyR+VH+tSYq+hAb71LTj7bNi/H/72N/jtbxXMREQk4UU1nJnZZWa22cy2mtlXOnh9gpmtMLNSM3vbzD4U3D7JzOrMbE3w9j/RrDNR+Cv9zMydSZL180z99tswdy7cf7+35uL69fChD8W6KhERkYiI2m95M0sGfg5cDkwHis1servdvg484ZybBSwGfhHy2rvOuZnB253RqjNRBFyANZVr+neXZlMTfPe73nqLu3fDn/8MDz/sXWNMRESkj4jmmLOzga3OuW0AZvYYcDVQFrKPAzKCj4cBu6NYT0Lbun8rNQ01/XemZlmZN7Zs9WpvUez//m/vOmUiIiJ9TDT7x8YCO0Oelwe3hboPuMnMyoFngS+EvJYX7O580cwuiGKdCaHfrgzQ3Aw/+IF3iYwdO+CJJ7yr/iuYiYhIHxXrwUvFwEPOuXHAh4A/mFkSUAFMCHZ3/hPwqJlltD/YzG43s9VmtrqqqqpXC+9tpRWlDEgewPSs9j3Dfdg778AFF8C993pjytavhxtuiHVVIiIiURXNcLYLGB/yfFxwW6hPAU8AOOdeBwYBmc65eudcdXD7W8C7wBnt38A594Bzbo5zbk5WVlYUPkL88Ff6KcwuJDU5NdalRF8gAD/+MRQVwaZN8Mgj8PTTkJMT68pERESiLprhbBWQb2Z5ZjYAb8D/0nb7vA8sAjCzaXjhrMrMsoITCjCzyUA+sC2KtcY15xz+Cn//6NJ8912YPx/uuQcuvhg2bPBmZGq5KhER6SeiFs6cc03A54HngY14szI3mNn9ZnZVcLcvAbeZ2VqgBLjFOeeAC4G3zWwN8BRwp3Nuf7RqjXfvH3qf/XX7+3Y4CwTgF7/wWsvefhseegiWLoXRo2NdmYiISK+K6goBzrln8Qb6h277ZsjjMuC8Do57Gng6mrUlktJKb2WAPjtT87334FOfgmXL4IMf9C4mO25crKsSERGJiVhPCJAw+Cv8JFsyZ+acGetSIss5+M1voLDQW7T8gQfguecUzEREpF/T2poJwF/hZ20IGWkAACAASURBVFrWNAanDo51KZFTXg6f/jQ8/zwsWAAPPgiTJsW6KhERkZhTy1kCKK0s7Ttdms7B738PBQXw8svws5+Bz6dgJiIiEqSWszhXWVPJ7iO7+8ZkgIoKuOMO+MtfvOuX/e53cNppsa5KREQkrqjlLM6VVniTARI6nDnnXdW/oABeeAF++ENYuVLBTEREpAMKZ3GuZabmzNyZMa6kh/bu9a7qv2QJnHEGrFnjXcMsST96IiIiHdFvyDjnr/Bz+sjTyRh4wupV8e+pp2DGDK8b8/vfh1degSlTYl2ViIhIXFM4i3MJuTJAdTUUF3stZhMngt/vrY+ZnBzrykREROKewlkcO1B3gO0Ht3NWbgKFs6VLvdayp5+Gb38bXn/dey4iIiJhUTiLY2sq1wAwa3SCXEbj+9+Hq6+G3FxYtQq+/nVI7QcLtYuIiESQLqURx/wVfiBBlm167jn46lfhxhvh4YdhwIBYVyQiIpKQ1HIWx0orSxmfMZ6s9KxYl3Jy27d7szELC711MRXMREREekzhLI75K/zx36VZWwvXXutdy+xPf4L09FhXJCIiktAUzuLU0YajbNq3Kb4nAzgHd94Ja9fCI4/oorIiIiIRoHAWp9buWYvDxfdlNH7xC/jDH+Bb34IPfSjW1YiIiPQJCmdxqmXZprjt1nz1Vbj7brjySvjGN2JdjYiISJ+hcBan/BV+stKyGDt0bKxLOVFFxfELzP7hD1qKSUREJIJ0KY045a/0VgYws1iX0lZDgxfMDh2C55+H4cNjXZGIiEifoiaPOFTfVM+GvRvi8/pmX/6y16X5m994l84QERGRiFI4i0MbqjbQGGiMv8kAf/wj/Pd/e2PNiotjXY2IiEifpHAWh1pWBoircLZ2Ldx+O1x4IfzHf8S6GhERkT5L4SwOlVaUkjEwg7wRebEuxbN/P1xzDYwYAU88ofUyRUREokgTAuKQv9LPrNxZJFkcZOfmZvjYx6C8HF56CXJyYl2RiIhInxYHv/0lVHOgmbWVa+OnS/Pf/s1b1PynP4V582JdjYiISJ+ncBZnNldvpq6pLj5mav7lL/Dtb8MnPwl33BHrakRERPoFhbM4EzeTAbZsgZtugrPOgp//HOLtemsiIiJ9VFTDmZldZmabzWyrmX2lg9cnmNkKMys1s7fN7EMhr301eNxmM7s0mnXGE3+Fn8Epg5mSOSV2RdTUeBMAUlPhT3+CwYNjV4uIiEg/E7UJAWaWDPwcuAQoB1aZ2VLnXFnIbl8HnnDO/dLMpgPPApOCjxcDM4AxgM/MznDONUer3nhRWllKUW4RKUkxmqvhHHz607BxozfWbOLE2NQhIiLST0Wz5exsYKtzbptzrgF4DLi63T4OyAg+HgbsDj6+GnjMOVfvnNsObA2er08LuAD+Cn9sx5v96Efw+OPw3e/CJZfErg4REZF+KprhbCywM+R5eXBbqPuAm8ysHK/V7AvdOLbP2X5gO4frD8duvNnKlXDvvV6X5r/8S2xqEBER6ediPSGgGHjIOTcO+BDwB7PwL+5lZreb2WozW11VVRW1IntLTCcDlJfDRz8K+fnw0EOaACAiIhIj0Qxnu4DxIc/HBbeF+hTwBIBz7nVgEJAZ5rE45x5wzs1xzs3JysqKYOmxUVpZSkpSCjOyZvTuG9fXw3XXQV2dNwEgI6PrY0RERCQqohnOVgH5ZpZnZgPwBvgvbbfP+8AiADObhhfOqoL7LTazgWaWB+QDb0ax1rjgr/BTkF3AwJSBvfvGX/wivPkm/P73MG1a7763iIiItBG1cOacawI+DzwPbMSblbnBzO43s6uCu30JuM3M1gIlwC3OswGvRa0MeA74XF+fqemcw1/h56zcXu7SfPBB+NWvvDFm117bu+8tIiIiJ4jq9Rqcc8/iDfQP3fbNkMdlwHmdHPtd4LvRrC+e7D6ym6raKmaN7sWZmqtXw2c/CxdfDN/5Tu+9r4iIiHQq1hMCJKjXJwNUVXktZTk5UFICKTG6rpqIiIi0od/IccJf4ccwinKKov9mTU1QXAx798Irr0BmZvTfU0RERMKicBYnSitLmZI5hfQB6dF/s69/HZYt88abzZkT/fcTERGRsKlbM074K/y906X59NPw/e/DHXfAJz8Z/fcTERGRblE4iwP7avex8/DO6M/U3LgRbrkF5s6Fn/wkuu8lIiIiPaJwFgdKK0oBojtT8/Bhb1mmtDR46ikY2MvXUhMREZGwaMxZHGiZqRm1Bc+d81rMtm71xpqNGxed9xEREZFTpnAWB/yVfvKG5zFi8IjovMH3vw/PPAM//CFcdFF03kNEREQiQt2acaC0ojR6XZp//zv867/CjTfC3XdH5z1EREQkYhTOYuxw/WG27N8SnckAO3Z41zObPh1++1swi/x7iIiISEQpnMXYmso1QBRWBqirg+uug+Zm+NOfIL0Xrp8mIiIip0xjzmKsdTJAJLs1nfPWzPT74S9/gfz8yJ1bREREokotZzFWWlnK6CGjyR2SG7mT/upX8NBD8M1vwpVXRu68IiIiEnVdhjMz+7CZKcRFScRXBnj9dbjrLrj8cvjWtyJ3XhEREekV4YSuG4EtZvYfZjY12gX1J3WNdWys2hi5cFZZCddfD+PHwx//CEnK1CIiIommyzFnzrmbzCwDKAYeMjMH/A4occ4diXaBfdm6vetods2RufhsY6N3uYwDB7zWs5EjT/2cIiIi0uvCalpxzh0GngIeA0YD1wB+M/tCFGvr81omA0Sk5ezee+Gll+DXv4aiolM/n4iIiMREOGPOrjKzZ4CVQCpwtnPucqAI+FJ0y+vb/BV+Rg4eyYRhE07tRCUl8OMfe2PNPvaxyBQnIiIiMRHOpTSuA37knHspdKNzrtbMPhWdsvqH0spSZuXOwk7l4rBvvw2f+hRccAH8539GrjgRERGJiXC6Ne8D3mx5YmaDzWwSgHNuWVSq6gcamxt5e8/bp9alefAgXHstDB8OTzwBqamRK1BERERiIpxw9iQQCHneHNwmp6CsqoyG5oZTC2df+hK8/z489RTkRvA6aSIiIhIz4YSzFOdcQ8uT4OMB0SupfyitLAXo+UzNQAD+93+9tTPPPTeClYmIiEgshRPOqszsqpYnZnY1sC96JfUP/go/QwYMIX9UD5dWWrsWqqvhkksiW5iIiIjEVDgTAu4EHjGznwEG7AQ+HtWq+gF/hZ+ZuTNJ6uniCz6fd79wYeSKEhERkZgL5yK07wLzzGxI8HlN1Kvq4wIuwJrKNdw669aen8Tng+nTYcyYyBUmIiIiMRdOyxlmdgUwAxjUctkH59z9UayrT9tSvYWjjUd7Phmgvh5efhluuy2yhYmIiEjMhXMR2v/BW1/zC3jdmjcAE6NcV592yisDvP461NXBxRdHsCoRERGJB+EMeDrXOfdx4IBz7t+Ac4Azwjm5mV1mZpvNbKuZfaWD139kZmuCt3fM7GDIa80hry0N9wMlgtLKUgYmD2Ra5rSencDng+RkuOiiyBYmIiIiMRdOt+ax4H2tmY0BqvHW1zwpM0sGfg5cApQDq8xsqXOurGUf59w9Ift/AQi9rkSdc25mGPUlHH+Fn8KcQlKTe3jRWJ8P5s6FjIzIFiYiIiIxF07L2V/MbDjwA8AP7AAeDeO4s4GtzrltwWujPQZcfZL9i4GSMM6b0Jxz+Cv8nJXbwy7Ngwdh1Sp1aYqIiPRRJ205M7MkYJlz7iDwtJn9FRjknDsUxrnH4l12o0U5MLeT95kI5AHLQzYPMrPVQBPwPefcn8N4z7j33qH3OHDsALNG9/DisytXehegXbQoonWJiIhIfDhpOHPOBczs5wS7G51z9UB9FOpYDDzlnGsO2TbRObfLzCYDy81sXfCyHq3M7HbgdoAJEyZEoazIK63wVgbo8WQAnw/S0mDevAhWJSIiIvEinG7NZWZ2nbVcQyN8u4DxIc/HBbd1ZDHtujSdc7uC99uAlbQdj9ayzwPOuTnOuTlZWVndLC82/BV+ki2ZwuzCnp3A5/MmAgzQCloiIiJ9UTjh7A68hc7rzeywmR0xs8NhHLcKyDezPDMbgBfATph1aWZTgRHA6yHbRpjZwODjTOA8oKz9sYnIX+lnetZ0BqcO7v7B5eWwebPGm4mIiPRh4awQMLQnJ3bONZnZ54HngWTgQefcBjO7H1jtnGsJaouBx5xzLuTwacCvzCyAFyC/FzrLM5GVVpRyyWk9XA9z2TLvXuFMRESkz+oynJnZhR1td8691NWxzrlngWfbbftmu+f3dXDca0AP+/3iV8WRCipqKno+U9Png+xsKCiIbGEiIiISN8K5ztk/hzwehHeJjLcArbjdTaWVpzAZwDkvnC1aBEk9XCxdRERE4l443ZofDn1uZuOBH0etoj6sZaZmUW5R9w8uK4PKSnVpioiI9HE9aYIpxxsTJt3kr/STPzKfjIE9uLK/z+fd6/pmIiIifVo4Y87+G2gZrJ8EzMRbKUC6yV/hZ+7YDq/D2zWfD04/HSZqzXkREZG+LJwxZ6tDHjcBJc65V6NUT591oO4AOw7u4M7Zd3b/4MZGb2WAm26KeF0iIiISX8IJZ08Bx1qu3m9myWaW5pyrjW5pfcspTQZYtQpqajTeTEREpB8Ia4UAIPSKqYMBX3TK6bv8FV5PcI/W1PT5wAwWLIhwVSIiIhJvwglng5xzNS1Pgo/ToldS31RaWcr4jPFkpmV2/2CfD2bPhpEjI1+YiIiIxJVwwtlRM2vtizOz2UBd9Erqm/wV/p51adbUwOuvq0tTRESknwhnzNndwJNmthswIBe4MapV9TE1DTVs3reZ4oLi7h/80kvQ1KRLaIiIiPQT4VyEdlVwcfIpwU2bnXON0S2rb1lbuRaHY1ZuD8ebDRwI550X+cJEREQk7nTZrWlmnwPSnXPrnXPrgSFm9tnol9Z3nNJMTZ8Pzj8fBg/uel8RERFJeOGMObvNOXew5Ylz7gBwW/RK6nv8FX6y07MZM3RM9w7cswfWrdN4MxERkX4knHCWbGbW8sTMkoEB0Sup7/FX+JmVO4uQrzE8y5d79wpnIiIi/UY44ew54HEzW2Rmi4AS4P+iW1bfUd9Uz4aqDT3v0hwxAmb1YKyaiIiIJKRwZmv+C3A70LLu0Nt4MzYlDOv3rqcp0NT9cOYcvPACLFwIycnRKU5ERETiTpctZ865APAPYAdwNrAQ2BjdsvqO1pUBujtTc+tW2LlTl9AQERHpZzptOTOzM4Di4G0f8DiAc05rCHVDaWUpwwYOY/KIyd070BdcIUvjzURERPqVk3VrbgJeBq50zm0FMLN7eqWqPsRf4WfW6B5MBvD5YMIEOP306BQmIiIicelk3ZrXAhXACjP7dXAyQDcTRv/WFGhi7Z61nJXbzfFmzc2wYoXXatbdUCciIiIJrdNw5pz7s3NuMTAVWIG3jFO2mf3SzD7YWwUmss37NnOs6RizRndzvFlpKRw4oC5NERGRfiicCQFHnXOPOuc+DIwDSvFmcEoXWiYDdHumZst4s4ULI1yRiIiIxLtwrnPWyjl3wDn3gHNOUwjD4K/wMzhlMFNGTel651A+H5x5JuTkRKcwERERiVvdCmfSPaWVpRTlFpGc1I3rlNXVwSuv6BIaIiIi/ZTCWZQEXIDSytLuTwZ49VWor9d4MxERkX5K4SxKth3YxuH6wz0bb5aSAhdeGJ3CREREJK5FNZyZ2WVmttnMtprZVzp4/UdmtiZ4e8fMDoa89gkz2xK8fSKadUZD68oA3Z2puWwZnHMODBkShapEREQk3oWztmaPmFky8HPgEqAcWGVmS51zZS37OOfuCdn/C8Cs4OORwLeAOYAD3goeeyBa9UZaaUUpqUmpzMiaEf5B+/fDW2/BffdFrS4RERGJb9FsOTsb2Oqc2+acawAeA64+yf7FQEnw8aXAC865/cFA9gJwWRRrjTh/pZ+C7AIGpgwM/6AVK7wFzzXeTEREpN+KZjgbC+wMeV4e3HYCM5sI5AHLu3tsPHLOecs2dXexc58Phg6FD3wgOoWJiIhI3IuXCQGLgaecc83dOcjMbjez1Wa2uqqqKkqldd+uI7vYV7uvZ5MBLroIUlOjU5iIiIjEvWiGs13A+JDn44LbOrKY412aYR8bvCDuHOfcnKysrFMsN3J6tDLAjh2wdau6NEVERPq5aIazVUC+meWZ2QC8ALa0/U5mNhUYAbwesvl54INmNsLMRgAfDG5LCP4KP4ZxZs6Z4R+0bJl3r3AmIiLSr0VttqZzrsnMPo8XqpKBB51zG8zsfmC1c64lqC0GHnPOuZBj95vZt/ECHsD9zrn90ao10korS5maOZX0AenhH7RsGeTmwvTp0StMRERE4l7UwhmAc+5Z4Nl2277Z7vl9nRz7IPBg1IqLIn+Fn4smXhT+AYGAN97s0kvBLHqFiYiISNyLlwkBfUbV0SrKD5d3b6bm+vVQVaUuTREREVE4i7TSylKgm5MBfD7vXoudi4iI9HsKZxHWo2WbfD6YOhXGjYtSVSIiIpIoFM4izF/hJ294HsMHDQ/vgIYGePFFtZqJiIgIoHAWcaWVpd3r0nzjDait1XgzERERARTOIurQsUNs3b+1++PNkpJg/vyo1SUiIiKJQ+EsgtZUrgG6ORlg2TJvLc3hYXaDioiISJ+mcBZBrZMBwr2MxuHD8I9/qEtTREREWimcRVBpZSljho4hZ0hOeAe8+CI0NyuciYiISCuFswjyV/i7P95s8GA455zoFSUiIiIJReEsQmoba9m4b2P3Vgbw+eCCC2DgwOgVJiIiIglF4SxC1u1ZR8AFwm85270bysrUpSkiIiJtKJxFSMtkgLDD2bJl3r3CmYiIiIRQOIsQf4WfkYNHMj5jfHgHLFsGo0ZBUVF0CxMREZGEonAWIS0rA5hZ1zs75403W7TIuwCtiIiISJCSQQQ0NDewbu86zsoNs0tz82bYtUtdmiIiInIChbMIKKsqo6G5gVmjw5yp6fN59wpnIiIi0o7CWQSUVpQC3ZgM4PNBXp53ExEREQmhcBYB/go/QwYM4fSRp3e9c1MTrFihVjMRERHpkMJZBPgr/czMnUmShfF1rl7trampcCYiIiIdUDg7Rc2BZtZWrg1/MkDL9c0WLoxeUSIiIpKwFM5O0Zb9WzjaeLR7481mzYLMzOgWJiIiIglJ4ewUtawMENZMzaNH4bXX1KUpIiIinVI4O0X+Cj8DkwcyLXNa1zu/8go0NCiciYiISKcUzk5RaWUpZ+acSWpyatc7+3wwYACcf370CxMREZGEpHB2Cpxz+Cv83Rtvdu65kJYW3cJEREQkYSmcnYIdB3dw8NhBZuWGMd6sqgrWrFGXpoiIiJxUVMOZmV1mZpvNbKuZfaWTfT5qZmVmtsHMHg3Z3mxma4K3pdGss6dKK7uxMsCKFd69wpmIiIicREq0TmxmycDPgUuAcmCVmS11zpWF7JMPfBU4zzl3wMyyQ05R55ybGa36IsFf4SfZkinMKex6Z58Phg2D2bOjX5iIiIgkrGi2nJ0NbHXObXPONQCPAVe32+c24OfOuQMAzrm9Uawn4vwVfqZnTWdQyqCud/b5YMECSIlaHhYREZE+IJrhbCywM+R5eXBbqDOAM8zsVTN7w8wuC3ltkJmtDm7/SEdvYGa3B/dZXVVVFdnqw1BaWRpel+a2bbB9u7o0RUREpEuxbsZJAfKB+cA44CUzK3TOHQQmOud2mdlkYLmZrXPOvRt6sHPuAeABgDlz5rjeLLziSAWVNZXhhTOfz7tftCi6RYmIiEjCi2bL2S5gfMjzccFtocqBpc65RufcduAdvLCGc25X8H4bsBIIY0pk72ldGSCcmZo+H4wdC1OmRLkqERERSXTRDGergHwzyzOzAcBioP2syz/jtZphZpl43ZzbzGyEmQ0M2X4eUEYcaZmpOTO3izkLgQAsX+51aZr1QmUiIiKSyKLWremcazKzzwPPA8nAg865DWZ2P7DaObc0+NoHzawMaAb+2TlXbWbnAr8yswBegPxe6CzPeOCv8HPGqDMYOnDoyXdcuxaqqzXeTERERMIS1TFnzrlngWfbbftmyGMH/FPwFrrPa0AY16eIHX+Fn3nj5nW9o8abiYiISDdohYAe2F+3n/cOvRf+ZIAZM2D06OgXJiIiIglP4awHSivCXBng2DF4+WV1aYqIiEjYFM56IOyZmq+/DnV1CmciIiISNoWzHvBX+pkwbAKj0kadfEefD5KT4cILe6cwERERSXgKZz1QWhHmygA+H8ydCxkZ0S9KRERE+gSFs246Un+Ed6rf6bpL8+BBWL1aXZoiIiLSLQpn3bR2z1ocruuWs5UrvQvQKpyJiIhINyicdVPYMzV9PkhP97o1RURERMKkcNZN/ko/2enZjB7SxXXLfD646CIYMKB3ChMREZE+QeGsm/wVfs4afRZ2snUyd+6EzZvVpSkiIiLdpnDWDceajlFWVcZZuV10aS5b5t1rySYRERHpJoWzbli/dz1NgabwxptlZ0NBQe8UJiIiIn2Gwlk3tK4MMPokl9Fwzms5W7QIkvT1ioiISPcoPXRDaUUpwwYOI294Xuc7lZVBZaXGm4mIiEiPKJx1g78yjMkAPp93r3AmIiIiPaBwFqamQBNv73m765UBfD7Iz4cJE3qnMBEREelTUmJdQKJISUqh/J5ymgJNne/U2OitDHDzzb1Wl4iIiPQtCmfdMCpt1Ml3ePNNqKnRJTRERESkx9StGUk+H5jBggWxrkREREQSlMJZJPl8MHs2jBwZ60pEREQkQSmcRUpNDbzxhmZpioiIyClROIuUl16CpiaFMxERETklCmeR4vPBoEFw3nmxrkREREQSmMJZpPh8cP75XkATERER6SGFs0iorIR163QJDRERETllus5ZJCxf7t1rvJmISL/W2NhIeXk5x44di3UpEicGDRrEuHHjSE1NDfuYqIYzM7sM+AmQDPzGOfe9Dvb5KHAf4IC1zrklwe2fAL4e3O07zrnfR7PWU+LzwYgRMKuLpZ1ERKRPKy8vZ+jQoUyaNOnk6zBLv+Cco7q6mvLycvLy8sI+LmrhzMySgZ8DlwDlwCozW+qcKwvZJx/4KnCec+6AmWUHt48EvgXMwQttbwWPPRCtenvMOS+cLVwIycmxrkZERGLo2LFjCmbSyswYNWoUVVVV3ToummPOzga2Oue2OecagMeAq9vtcxvw85bQ5ZzbG9x+KfCCc25/8LUXgMuiWGvPbd0KO3eqS1NERAAUzKSNnvw8RDOcjQV2hjwvD24LdQZwhpm9amZvBLtBwz02Pvh83r3CmYiIxFh1dTUzZ85k5syZ5ObmMnbs2NbnDQ0NJz129erV3HXXXV2+x7nnnhupcgG4++67GTt2LIFAIKLnTWSxnhCQAuQD84FxwEtmVhjuwWZ2O3A7wIQJE6JRX9d8Ppg4EU47LTbvLyIiEjRq1CjWrFkDwH333ceQIUP48pe/3Pp6U1MTKSkd/+qfM2cOc+bM6fI9XnvttcgUCwQCAZ555hnGjx/Piy++yIIorU19ss8dj6LZcrYLGB/yfFxwW6hyYKlzrtE5tx14By+shXMszrkHnHNznHNzsrKyIlp8WJqbvZmaixZ5C56LiIjEmVtuuYU777yTuXPncu+99/Lmm29yzjnnMGvWLM4991w2b94MwMqVK7nyyisBL9jdeuutzJ8/n8mTJ/PTn/609XxDhgxp3X/+/Plcf/31TJ06lY997GM45wB49tlnmTp1KrNnz+auu+5qPW97K1euZMaMGXzmM5+hpKSkdfuePXu45pprKCoqoqioqDUQPvzww5x55pkUFRVx8803t36+p556qsP6LrjgAq666iqmT58OwEc+8hFmz57NjBkzeOCBB1qPee655zjrrLMoKipi0aJFBAIB8vPzW8eKBQIBTj/99G6PHeupaMbIVUC+meXhBavFwJJ2+/wZKAZ+Z2aZeN2c24B3gX83sxHB/T6IN3Egvvj9cPCgujRFROREd98NwVasiJk5E378424fVl5ezmuvvUZycjKHDx/m5ZdfJiUlBZ/Px9e+9jWefvrpE47ZtGkTK1as4MiRI0yZMoXPfOYzJ1wOorS0lA0bNjBmzBjOO+88Xn31VebMmcMdd9zBSy+9RF5eHsXFxZ3WVVJSQnFxMVdffTVf+9rXaGxsJDU1lbvuuouLLrqIZ555hubmZmpqatiwYQPf+c53eO2118jMzGT//v1dfm6/38/69etbZ0o++OCDjBw5krq6Oj7wgQ9w3XXXEQgEuO2221rr3b9/P0lJSdx000088sgj3H333fh8PoqKiuithqCotZw555qAzwPPAxuBJ5xzG8zsfjO7Krjb80C1mZUBK4B/ds5VO+f2A9/GC3irgPuD2+JLy3izhQtjW4eIiMhJ3HDDDSQHryhw6NAhbrjhBgoKCrjnnnvYsGFDh8dcccUVDBw4kMzMTLKzs9mzZ88J+5x99tmMGzeOpKQkZs6cyY4dO9i0aROTJ09uDUSdhbOGhgaeffZZPvKRj5CRkcHcuXN5/vnnAVi+fDmf+cxnAEhOTmbYsGEsX76cG264gczMTABGjhzZ5ec+++yz21zC4qc//SlFRUXMmzePnTt3smXLFt544w0uvPDC1v1aznvrrbfy8MMPA16o++QnP9nl+0VKVDtgnXPPAs+22/bNkMcO+Kfgrf2xDwIPRrO+U7ZsGZx5JuTkxLoSERGJNz1o4YqW9PT01sff+MY3WLBgAc888ww7duxg/vz5HR4zcODA1sfJyck0NTX1aJ/OPP/88xw8eJDCQm+oeW1tLYMHD+60C7QzKSkprZMJAoFAm4kPoZ975cqV+Hw+Xn/9ddLS0pg/f/5JLxY8fvx4cnJyWL58OW+++SaPPPJIt+o6FVq+qafq6uCVV9SlKSIiCeXQoUOMHetdAOGhhx6K+PmnTJnCtm3b4R4G8QAAEChJREFU2LFjBwCPP/54h/uVlJTwm9/8hh07drBjxw62b9/OCy+8QG1tLYsWLeKXv/wlAM3NzRw6dIiFCxfy5JNPUl1dDdDarTlp0iTeeustAJYuXUpjY2OH73fo0CFGjBhBWloamzZt4o033gBg3rx5vPTSS2zfvr3NeQE+/elPc9NNN7VpeewNCmc99eqrUF+vcCYiIgnl3nvv5atf/SqzZs3qVktXuAYPHswvfvELLrvsMmbPns3QoUMZNmxYm31qa2t57rnnuOKKK1q3paenc/755/OXv/yFn/zkJ6xYsYLCwkJmz55NWVkZM2bM4F//9V+56KKLKCoq4p/+yet0u+2223jxxRcpKiri9ddfb9NaFuqyyy6jqamJadOm8ZWvfIV58+YBkJWVxQMPPMC1115LUVERN954Y+sxV111FTU1Nb3apQlgLTMrEt2cOXPc6tWre+8Nv/IV+OEPYf9+CM4MERGR/m3jxo1MmzYt1mXEXE1NDUOGDME5x+c+9zny8/O55557Yl1Wt61evZp77rmHl19++ZTO09HPhZm95Zzr8NolajnrKZ8PzjlHwUxERKSdX//618ycOZP/3979B1dV5nccf38JqSH8pqEsJbTJVBDJhksSgbT8JtsOCpPwU4yLmrIyyrRqcMcuqw44KpVZmRZplQ6iIm4mGaVLChVQkhBgBqhIyq8gVFjiij8Q4hChAQnJ0z/u5TaB/DbkHMPnNZO55zz33nO+9zu5N988z3PPk5CQQEVFBY888ojXIbXYsmXLmDlzJi+99FK7n1s9Z61RXg59+8Jzz8HixU0+XEREbg3qOZP6qOesPWzfHlzwXPPNREREpI2pOGuNwkLo3h1GjPA6EhEREelgVJy1RkEBTJgA110pWUREROSHUnHWUmVlcOKEhjRFRETkplBx1lKFhcFbFWciIuIzEydODC+BdM2KFSvCSyHVZ8KECVz7Qt0999zD+fPnb3jMc889x/Llyxs9d35+PkePHg3vL168mIJryxy2gezsbAYMGBBeDaAjU3HWUgUF0L8/6Ns4IiLiM5mZmeTl5dVpy8vLa3Tx8do2b95Mr169WnXu64uz559/np+1UUdGTU0NGzZsYODAgezYsaNNjlmfm3FR3tZQcdYSNTXBnrO0NDDzOhoREZE6Zs2axfvvvx9eX7KsrIwvv/ySsWPHsmDBAu666y4SEhJYsmRJvc+Pi4vj3LlzACxdupTBgwczZswYjh8/Hn7M66+/zogRIwgEAsycOZPKykp2797Nxo0beeqppxg+fDgnT54kKyuL9evXA1BYWEhSUhKJiYnMmzeP77//Pny+JUuWkJycTGJiIseOHas3ruLiYhISEliwYAG5ubnh9jNnzjB9+nQCgQCBQIDdu3cDsG7dOoYNG0YgEOCBBx4AqBMPQLfQdUqLi4sZO3Ys6enpDB06FIBp06aRkpJCQkICq1evDj9n69atJCcnEwgESEtLo6amhkGDBnH27FkgWETefvvt4f3WuqkLn3c4hw/D2bMa0hQRkSZlb83mwNcH2vSYw38ynBWTG15QvU+fPowcOZItW7aQkZFBXl4e9957L2bG0qVL6dOnD9XV1aSlpXHo0CGGDRtW73H2799PXl4eBw4c4OrVqyQnJ5OSkgLAjBkzmD9/PgDPPvssb7zxBo899hjp6elMnTqVWbNm1TnW5cuXycrKorCwkMGDB/Pggw+yatUqsrOzAYiJiaGkpITXXnuN5cuXs2bNmhviyc3NJTMzk4yMDJ5++mmqqqqIjIzk8ccfZ/z48WzYsIHq6mouXrxIaWkpL774Irt37yYmJqbOWpkNKSkp4ciRI8THxwPw5ptv0qdPHy5dusSIESOYOXMmNTU1zJ8/n507dxIfH8+3335Lp06dmDt3Ljk5OWRnZ1NQUEAgEKBv375NnrMx6jlriWtj52lp3sYhIiLSgNpDm7WHNN99912Sk5NJSkqitLS0zhDk9Xbt2sX06dOJjo6mR48epKenh+87cuQIY8eOJTExkZycHEpLSxuN5/jx48THxzN48GAAHnroIXbu3Bm+f8aMGQCkpKSEF0uv7cqVK2zevJlp06bRo0cPRo0aFZ5XV1RUFJ5PFxERQc+ePSkqKmL27NnExMQAwYK1KSNHjgwXZgArV64kEAiQmprK559/zqeffsrevXsZN25c+HHXjjtv3jzWrVsHBIu6tliHUz1nLVFYCEOGQGys15GIiIjPNdbDdTNlZGSwcOFCSkpKqKysJCUlhVOnTrF8+XL27dtH7969ycrK4vLly606flZWFvn5+QQCAdauXUtxcfEPive2224DgsVVfXO+PvjgA86fP09iYiIQXDS9S5cuTJ06tUXn6dy5c/jLBDU1NeGhX6DOYunFxcUUFBSwZ88eoqOjmTBhQqO5GjhwIP369aOoqIiPPvqInJycFsVVH/WcNdeVK7Bjh4Y0RUTE17p168bEiROZN29euNfsu+++o2vXrvTs2ZMzZ86wZcuWRo8xbtw48vPzuXTpEhcuXGDTpk3h+y5cuED//v2pqqqqU4h0796dCxcu3HCsO+64g7KyMk6cOAHAO++8w/jx45v9enJzc1mzZg1lZWWUlZVx6tQptm3bRmVlJWlpaaxatQqA6upqKioqmDRpEu+99x7l5eUA4WHNuLg49u/fD8DGjRupqqqq93wVFRX07t2b6Ohojh07xt69ewFITU1l586dnDp1qs5xAR5++GHmzp3L7NmziYiIaPZra4iKs+aKjIRDh+DJJ72OREREpFGZmZkcPHgwXJwFAgGSkpIYMmQI999/P6NHj270+cnJycyZM4dAIMDdd9/NiFor4rzwwguMGjWK0aNHM2TIkHD7fffdx8svv0xSUhInT54Mt0dFRfHWW28xe/ZsEhMT6dSpE48++mizXkdlZSVbt25lypQp4bauXbsyZswYNm3axCuvvML27dtJTEwkJSWFo0ePkpCQwDPPPMP48eMJBAI8Gfq7PX/+fHbs2EEgEGDPnj11estqmzx5MlevXuXOO+9k0aJFpKamAtC3b19Wr17NjBkzCAQCzJkzJ/yc9PR0Ll682CZDmqCFz0VERNqMFj6/NX388ccsXLiQXbt21Xt/Sxc+15wzERERkVZatmwZq1atapO5ZtdoWFNERESklRYtWsRnn33GmDFj2uyYKs5EREREfETFmYiISBvqKHO5pW205vdBxZmIiEgbiYqKory8XAWaAMHCrLy8nKioqBY9T18IEBERaSOxsbGcPn36B6+tKB1HVFQUsS28eL2KMxERkTYSGRlZZxkgkdbQsKaIiIiIj6g4ExEREfERFWciIiIiPtJhlm8ys7PAZ+1wqhjgXDuc58dK+WmactQ45adpylHjlJ+mKUeNa4/8/Llzrm99d3SY4qy9mNnHDa2FJcpPcyhHjVN+mqYcNU75aZpy1Div86NhTREREREfUXEmIiIi4iMqzlputdcB+Jzy0zTlqHHKT9OUo8YpP01TjhrnaX4050xERETER9RzJiIiIuIjKs6aycwmm9lxMzthZou8jsdvzGygmW03s6NmVmpmT3gdkx+ZWYSZ/beZ/afXsfiRmfUys/VmdszMPjGzv/Q6Jj8xs4Wh99cRM8s1s5atptwBmdmbZvaNmR2p1dbHzLaZ2aeh295exuilBvLzcug9dsjMNphZLy9j9Fp9Oap13y/NzJlZTHvGpOKsGcwsAngVuBsYCmSa2VBvo/Kdq8AvnXNDgVTg75Sjej0BfOJ1ED72CrDVOTcECKBchZnZAOBx4C7n3E+BCOA+b6PyhbXA5OvaFgGFzrlBQGFo/1a1lhvzsw34qXNuGPA/wK/bOyifWcuNOcLMBgJ/A/yhvQNScdY8I4ETzrnfO+euAHlAhscx+Ypz7ivnXElo+wLBP6oDvI3KX8wsFpgCrPE6Fj8ys57AOOANAOfcFefceW+j8p3OQBcz6wxEA196HI/nnHM7gW+va84A3g5tvw1Ma9egfKS+/DjnPnTOXQ3t7gVi2z0wH2ngdwjgn4F/ANp9cr6Ks+YZAHxea/80KjwaZGZxQBLwX95G4jsrCL7Ra7wOxKfigbPAW6Gh3zVm1tXroPzCOfcFsJzgf/FfARXOuQ+9jcq3+jnnvgptfw308zIYn5sHbPE6CL8xswzgC+fcQS/Or+JM2pSZdQP+Hch2zn3ndTx+YWZTgW+cc/u9jsXHOgPJwCrnXBLwv9zaw1F1hOZNZRAsYv8U6Gpmc72Nyv9c8JIEuixBPczsGYJTUnK8jsVPzCwaeBpY7FUMKs6a5wtgYK392FCb1GJmkQQLsxzn3O+8jsdnRgPpZlZGcFh8kpn91tuQfOc0cNo5d63HdT3BYk2Cfgaccs6ddc5VAb8D/srjmPzqjJn1BwjdfuNxPL5jZlnAVODnTtfUut5fEPwn6GDoMzsWKDGzn7RXACrOmmcfMMjM4s3sjwhOwt3ocUy+YmZGcK7QJ865f/I6Hr9xzv3aORfrnIsj+PtT5JxTr0ctzrmvgc/N7I5QUxpw1MOQ/OYPQKqZRYfeb2noCxMN2Qg8FNp+CPgPD2PxHTObTHCKRbpzrtLrePzGOXfYOfcnzrm40Gf2aSA59BnVLlScNUNo4uTfAx8Q/DB81zlX6m1UvjMaeIBgj9CB0M89XgclPzqPATlmdggYDvyjx/H4RqhHcT1QAhwm+Pl9y1/l3cxygT3AHWZ22sx+ASwD/trMPiXY47jMyxi91EB+/hXoDmwLfVb/m6dBeqyBHHkbk3ozRURERPxDPWciIiIiPqLiTERERMRHVJyJiIiI+IiKMxEREREfUXEmIiIi4iMqzkSkQzOz6lqXdzlgZm226oCZxZnZkbY6nogIBJdLERHpyC4554Z7HYSISHOp50xEbklmVmZmvzGzw2b2kZndHmqPM7MiMztkZoVm9meh9n5mtsHMDoZ+ri2dFGFmr5tZqZl9aGZdQo9/3MyOho6T59HLFJEfIRVnItLRdbluWHNOrfsqnHOJBK+YviLU9i/A2865YQQXhF4Zal8J7HDOBQiu+XltlZBBwKvOuQTgPDAz1L4ISAod59Gb9eJEpOPRCgEi0qGZ2UXnXLd62suASc6535tZJPC1c+6Pzewc0N85VxVq/8o5F2NmZ4FY59z3tY4RB2xzzg0K7f8KiHTOvWhmW4GLQD6Q75y7eJNfqoh0EOo5E5FbmWtguyW+r7Vdzf/P5Z0CvEqwl22fmWmOr4g0i4ozEbmVzal1uye0vRu4L7T9c2BXaLsQWABgZhFm1rOhg5pZJ2Cgc2478CugJ3BD752ISH30n5yIdHRdzOxArf2tzrlrl9PobWaHCPZ+ZYbaHgPeMrOngLPA34banwBWm9kvCPaQLQC+auCcEcBvQwWcASudc+fb7BWJSIemOWcicksKzTm7yzl3zutYRERq07CmiIiIiI+o50xERETER9RzJiIiIuIjKs5EREREfETFmYiIiIiPqDgTERER8REVZyIiIiI+ouJMRERExEf+D8ZiJ7cJRhBvAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmEAAAFNCAYAAABIc7ibAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXhV1b3/8fc340nIQAhhHp0YZAgYoIIDWAcUCmhSK1Vb7FWrtVI7ake9Wqu39drWVtuf1g63gzhVq0XrPOMAIiCgKAJCGMOUgZB5/f7YJ8lJCJnIzj4Jn9fznCc55+yz1zcbNB/WWnstc84hIiIiIp0rJugCRERERI5GCmEiIiIiAVAIExEREQmAQpiIiIhIABTCRERERAKgECYiIiISAIUwEcHMnjazL3f0sUEys01mdqYP533ZzC4Pf3+xmT3bmmPb0c4QMysxs9j21ioi0U0hTKSLCv+Crn3UmNnBiOcXt+VczrlznXN/6ehjo5GZ3WBmrzbxem8zqzCzMa09l3Pu7865szuorgah0Tm32TmX4pyr7ojzN2rLmdlxHX1eEWkbhTCRLir8CzrFOZcCbAY+F/Ha32uPM7O44KqMSn8DpprZ8EavXwS875xbHUBNInIUUggT6WbMbLqZ5ZvZ9Wa2A/iTmWWY2b/NrMDM9oW/HxTxmcghtgVm9rqZ3RE+dqOZndvOY4eb2atmVmxmz5vZ3Wb2t8PU3ZoabzGzN8Lne9bMeke8f6mZfWpme8zsh4e7Ps65fOBF4NJGb30J+L+W6mhU8wIzez3i+Vlm9qGZFZrZbwGLeO9YM3sxXN9uM/u7mfUMv/dXYAjwZLgn83tmNizcYxUXPmaAmT1hZnvNbL2ZXRFx7pvM7CEz+7/wtVljZjmHuwaHY2bp4XMUhK/lj8wsJvzecWb2Svhn221mD4ZfNzP7pZntMrMiM3u/Lb2JIkczhTCR7qkf0AsYClyJ99/6n8LPhwAHgd828/kpwDqgN/Bz4H4zs3Yc+w/gHSATuIlDg0+k1tT4ReAyoA+QAHwHwMxGA78Ln39AuL0mg1PYXyJrMbMRQHa43rZeq9pz9Ab+CfwI71p8AkyLPAS4LVzfKGAw3jXBOXcpDXszf95EE4uA/PDn84CfmdkZEe/PCR/TE3iiNTU34TdAOnAMcDpeML0s/N4twLNABt61/U349bOB04ATwp+9ENjTjrZFjjoKYSLdUw1wo3Ou3Dl30Dm3xzn3qHOu1DlXDNyK90v2cD51zt0Xno/0F6A/0Lctx5rZEGAS8BPnXIVz7nW8cNCkVtb4J+fcR865g8BDeMEJvFDyb+fcq865cuDH4WtwOI+Fa5wafv4l4GnnXEE7rlWt84A1zrlHnHOVwK+AHRE/33rn3HPhP5MC4M5WnhczG4wX6K53zpU551YAfwjXXet159xT4T+HvwLjW3PuiDZi8YZkv++cK3bObQL+l/qwWokXTAeEa3g94vVUYCRgzrkPnHPb29K2yNFKIUykeypwzpXVPjGzZDP7f+EhpiLgVaCnHf7Ou8jwUBr+NqWNxw4A9ka8BrDlcAW3ssYdEd+XRtQ0IPLczrkDNNMbE67pYeBL4V67i4H/a0MdTWlcg4t8bmZ9zWyRmW0Nn/dveD1mrVF7LYsjXvsUGBjxvPG1CVnb5gP2BuLD522qje/h9ea9Ex7u/AqAc+5FvF63u4FdZnavmaW1oV2Ro5ZCmEj35Bo9/zYwApjinEvDGz6CiDlLPtgO9DKz5IjXBjdz/JHUuD3y3OE2M1v4zF/whs7OwuvJefII62hcg9Hw5/0Z3p/L2PB5L2l0zsZ/ZpG24V3L1IjXhgBbW6ipLXZT39t1SBvOuR3OuSuccwOArwL3WPgOS+fcXc65k4DReMOS3+3AukS6LYUwkaNDKt7cpv1m1gu40e8GnXOfAsuAm8wswcxOBj7nU42PALPN7BQzSwBupuX/v70G7AfuBRY55yqOsI7FwIlmdkG4B2oh3ty8WqlACVBoZgM5NKjsxJuLdQjn3BZgCXCbmYXMbBzwX3i9ae2VED5XyMxC4dceAm41s1QzGwp8q7YNM/t8xA0K+/BCY42ZTTKzKWYWDxwAymh+KFhEwhTCRI4OvwKS8Ho73gL+00ntXgycjDc0+FPgQaD8MMe2u0bn3BrgGryJ9dvxQkJ+C59xeEOQQ8Nfj6gO59xu4PPA7Xg/7/HAGxGH/DcwESjEC2z/bHSK24Afmdl+M/tOE03MB4bh9Yo9hjfn7/nW1HYYa/DCZu3jMuBavCC1AXgd73r+MXz8JOBtMyvBm9v3DefcBiANuA/vmn+K97P/4gjqEjlqmPf/IRER/4WXNfjQOed7T5yISLRTT5iI+CY8VHWsmcWY2UxgLvB40HWJiEQDraQtIn7qhzfslok3PHi1c+69YEsSEYkOGo4UERERCYCGI0VEREQCoBAmIiIiEoAuNyesd+/ebtiwYUGXISIiItKid999d7dzLqup97pcCBs2bBjLli0LugwRERGRFpnZp4d7T8ORIiIiIgFQCBMREREJgEKYiIiISAC63JwwERGR7qyyspL8/HzKysqCLkXaIBQKMWjQIOLj41v9GYUwERGRKJKfn09qairDhg3DzIIuR1rBOceePXvIz89n+PDhrf6chiNFRESiSFlZGZmZmQpgXYiZkZmZ2ebeS4UwERGRKKMA1vW0589MIUxERETq7Nmzh+zsbLKzs+nXrx8DBw6se15RUdHsZ5ctW8bChQtbbGPq1KkdUuvLL7/M7NmzO+RcQdCcMBEREamTmZnJihUrALjppptISUnhO9/5Tt37VVVVxMU1HR9ycnLIyclpsY0lS5Z0TLFdnK89YWY208zWmdl6M7uhifeHmtkLZrbKzF42s0F+1tMaW7fCnXdCZWXQlYiIiESHBQsWcNVVVzFlyhS+973v8c4773DyySczYcIEpk6dyrp164CGPVM33XQTX/nKV5g+fTrHHHMMd911V935UlJS6o6fPn06eXl5jBw5kosvvhjnHABPPfUUI0eO5KSTTmLhwoVt6vF64IEHGDt2LGPGjOH6668HoLq6mgULFjBmzBjGjh3LL3/5SwDuuusuRo8ezbhx47jooouO/GK1gW89YWYWC9wNnAXkA0vN7Ann3NqIw+4A/s859xczOwO4DbjUr5pa49134dvfhjFj4Oyzg6xEREQkeuTn57NkyRJiY2MpKiritddeIy4ujueff54f/OAHPProo4d85sMPP+Sll16iuLiYESNGcPXVVx+yhMN7773HmjVrGDBgANOmTeONN94gJyeHr371q7z66qsMHz6c+fPnt7rObdu2cf311/Puu++SkZHB2WefzeOPP87gwYPZunUrq1evBmD//v0A3H777WzcuJHExMS61zqLn8ORk4H1zrkNAGa2CJgLRIaw0cC3wt+/BDzuYz2tcvbZkJICjz6qECYiIsG67joIjwx2mOxs+NWv2v65z3/+88TGxgJQWFjIl7/8ZT7++GPMjMrDDB/NmjWLxMREEhMT6dOnDzt37mTQoIaDXpMnT657LTs7m02bNpGSksIxxxxTt9zD/Pnzuffee1tV59KlS5k+fTpZWd6e2RdffDGvvvoqP/7xj9mwYQPXXnsts2bN4uzwL/lx48Zx8cUXM2/ePObNm9f2C3ME/ByOHAhsiXieH34t0krggvD35wOpZpbpY00tCoVg9mx47DGorg6yEhERkejRo0ePuu9//OMfM2PGDFavXs2TTz552KUZEhMT676PjY2lqqqqXcd0hIyMDFauXMn06dP5/e9/z+WXXw7A4sWLueaaa1i+fDmTJk3yrf2mBD0x/zvAb81sAfAqsBU4JPqY2ZXAlQBDhgzxvajcXFi0CF57DaZP9705ERGRJrWnx6ozFBYWMnCg16/y5z//ucPPP2LECDZs2MCmTZsYNmwYDz74YKs/O3nyZBYuXMju3bvJyMjggQce4Nprr2X37t0kJCSQm5vLiBEjuOSSS6ipqWHLli3MmDGDU045hUWLFlFSUkLPnj07/Gdqip8hbCswOOL5oPBrdZxz2wj3hJlZCpDrnDtkQNY5dy9wL0BOTo7zq+Ba554LSUnwyCMKYSIiIo1973vf48tf/jI//elPmTVrVoefPykpiXvuuYeZM2fSo0cPJk2adNhjX3jhhQZDnA8//DC33347M2bMwDnHrFmzmDt3LitXruSyyy6jpqYGgNtuu43q6mouueQSCgsLcc6xcOHCTgtgAFZ7F0KHn9gsDvgI+Cxe+FoKfNE5tybimN7AXudcjZndClQ7537S3HlzcnLcsmXLfKk5Um4uvPkm5OdDjFZTExGRTvLBBx8watSooMsIXElJCSkpKTjnuOaaazj++OP55je/GXRZzWrqz87M3nXONbluh2/xwjlXBXwdeAb4AHjIObfGzG42sznhw6YD68zsI6AvcKtf9bRVXh5s3+4FMREREelc9913H9nZ2Zx44okUFhby1a9+NeiSOpxvPWF+6ayesKIiyMqCr30NwkuJiIiI+E49YV1X1PSEdXVpaXDOOd5SFV0sp4qIiEgXoBDWjLw82LIFli4NuhIRERHpbhTCmvG5z0FcnNcbJiIiItKRFMKakZEBZ57pLVWhIUkRERHpSAphLcjNhQ0bOn7bCBERkWg0Y8YMnnnmmQav/epXv+Lqq68+7GemT59O7U1z5513XpN7MN50003ccccdzbb9+OOPs3Zt/e6GP/nJT3j++efbUn6TIjcWjyYKYS2YNw9iYzUkKSIiR4f58+ezaNGiBq8tWrSo1ZtoP/XUU+1e8LRxCLv55ps588wz23WurkAhrAW9e8Ppp2tIUkREjg55eXksXryYiooKADZt2sS2bds49dRTufrqq8nJyeHEE0/kxhtvbPLzw4YNY/fu3QDceuutnHDCCZxyyimsW7eu7pj77ruPSZMmMX78eHJzcyktLWXJkiU88cQTfPe73yU7O5tPPvmEBQsW8MgjjwDeyvgTJkxg7NixfOUrX6G8vLyuvRtvvJGJEycyduxYPvzww1b/rA888ABjx45lzJgxXH/99QBUV1ezYMECxowZw9ixY/lleJ2qu+66i9GjRzNu3DguuuiiNl7VpimEtUJeHqxbBxHhXEREpFvq1asXkydP5umnnwa8XrALL7wQM+PWW29l2bJlrFq1ildeeYVVq1Yd9jzvvvsuixYtYsWKFTz11FMsjVhq4IILLmDp0qWsXLmSUaNGcf/99zN16lTmzJnDL37xC1asWMGxxx5bd3xZWRkLFizgwQcf5P3336eqqorf/e53de/37t2b5cuXc/XVV7c45Flr27ZtXH/99bz44ousWLGCpUuX8vjjj7NixQq2bt3K6tWref/997nssssAuP3223nvvfdYtWoVv//979t0TQ8n6A28u4Tzz4drrvF6w048MehqRETkaPHxx9dRUtKxk5JTUrI5/vjmdwavHZKcO3cuixYt4v777wfgoYce4t5776Wqqort27ezdu1axo0b1+Q5XnvtNc4//3ySk5MBmDNnTt17q1ev5kc/+hH79++npKSEc845p9l61q1bx/DhwznhhBMA+PKXv8zdd9/NddddB3ihDuCkk07in//8ZyuuAixdupTp06eTlZUFwMUXX8yrr77Kj3/8YzZs2MC1117LrFmzOPvsswEYN24cF198MfPmzWPevHmtaqMl6glrhX794JRTNC9MRESODnPnzuWFF15g+fLllJaWctJJJ7Fx40buuOMOXnjhBVatWsWsWbMoKytr1/kXLFjAb3/7W95//31uvPHGdp+nVmJiIgCxsbFUVVUd0bkyMjJYuXIl06dP5/e//z2XX345AIsXL+aaa65h+fLlTJo06YjbAfWEtVpeHnzjG/DRRxAO4iIiIr5qqcfKLykpKcyYMYOvfOUrdRPyi4qK6NGjB+np6ezcuZOnn36a6dOnH/Ycp512GgsWLOD73/8+VVVVPPnkk3X7PxYXF9O/f38qKyv5+9//zsCBAwFITU2luLj4kHONGDGCTZs2sX79eo477jj++te/cvrppx/Rzzh58mQWLlzI7t27ycjI4IEHHuDaa69l9+7dJCQkkJuby4gRI7jkkkuoqalhy5YtzJgxg1NOOYVFixZRUlLS7hsQaimEtdIFF3gh7NFH4fvfD7oaERERf82fP5/zzz+/7k7J8ePHM2HCBEaOHMngwYOZNm1as5+fOHEiX/jCFxg/fjx9+vRh0qRJde/dcsstTJkyhaysLKZMmVIXvC666CKuuOIK7rrrrroJ+QChUIg//elPfP7zn6eqqopJkyZx1VVXtenneeGFFxg0aFDd84cffpjbb7+dGTNm4Jxj1qxZzJ07l5UrV3LZZZdRU1MDwG233UZ1dTWXXHIJhYWFOOdYuHDhEQcw0AbebXLyyVBRAe++G0jzIiJyFNAG3l2XNvD2UW4uLF/uLd4qIiIiciQUwtogN9f72sobL0REREQOSyGsDYYPh4kTvaUqRERERI6EQlgb5eXB22/Dli1BVyIiIt1VV5uvLe37M1MIayMNSYqIiJ9CoRB79uxREOtCnHPs2bOHUCjUps9piYo2OuEEGDvWW6riG98IuhoREeluBg0aRH5+PgUFBUGXIm0QCoUaLIHRGgph7ZCXBzfdBDt2eKvpi4iIdJT4+HiGDx8edBnSCTQc2Q65ueAcPPZY0JWIiIhIV6UQ1g6jR8PIkbpLUkRERNpPIawdzLzesJdfBg3Zi4iISHsohLVTXh7U1MC//hV0JSIiItIVKYS10/jxcMwxGpIUERGR9lEIayczrzfshRdg376gqxEREZGuRiHsCOTlQVUVPPFE0JWIiIhIV6MQdgRycmDIEG/hVhEREZG2UAg7ArV3ST7zDBQVBV2NiIiIdCW+hjAzm2lm68xsvZnd0MT7Q8zsJTN7z8xWmdl5ftbjh9xcqKiAxYuDrkRERES6Et9CmJnFAncD5wKjgflmNrrRYT8CHnLOTQAuAu7xqx6/nHwy9O+vuyRFRESkbfzsCZsMrHfObXDOVQCLgLmNjnFAWvj7dGCbj/X4IiYGLrgAnn4aDhwIuhoRERHpKvwMYQOBLRHP88OvRboJuMTM8oGngGt9rMc3eXlw8KAXxERERERaI+iJ+fOBPzvnBgHnAX81s0NqMrMrzWyZmS0riMJ9gk49FbKyNCQpIiIirednCNsKDI54Pij8WqT/Ah4CcM69CYSA3o1P5Jy71zmX45zLycrK8qnc9ouNhfPP9ybnHzwYdDUiIiLSFfgZwpYCx5vZcDNLwJt433hZ083AZwHMbBReCIu+rq5WyMuDkhJ49tmgKxEREZGuwLcQ5pyrAr4OPAN8gHcX5Bozu9nM5oQP+zZwhZmtBB4AFjjnnF81+Wn6dMjI0MKtIiIi0jpxfp7cOfcU3oT7yNd+EvH9WmCanzV0lvh4mDcP/vlPKC+HxMSgKxIREZFoFvTE/G4lNxcKC71NvUVERESaoxDWgc48E9LSNCQpIiIiLVMI60CJifC5z8Hjj0NlZdDViIiISDRTCOtgeXmwdy+88krQlYiIiEg0UwjrYOecAz16aOFWERERaZ5CWAdLSoJZs+Cxx6C6OuhqREREJFophPkgLw927YLXXw+6EhEREYlWCmE+OPdcCIV0l6SIiIgcnkKYD1JSvCD26KNQUxN0NSIiIhKNFMJ8kpsL27bBW28FXYmIiIhEI4Uwn8yeDQkJGpIUERGRpimE+SQ9Hc46ywthXXNLchEREfGTQpiP8vLg00/h3XeDrkRERESijUKYj+bMgbg4LdwqIiIih1II81GvXnDGGRqSFBERkUMphPksLw/Wr4dVq4KuRERERKKJQpjP5s2DmBjdJSkiIiINKYT5LCsLTj9d88JERESkIYWwTpCbCx98AGvXBl2JiIiIRAuFsE5w/vlgpiFJERERqacQ1gkGDIBp0zQkKSIiIvUUwjpJbq53h+THHwddiYiIiEQDhbBOcsEF3lcNSYqIiAgohHWaIUNg8mSFMBEREfEohHWivDxYtgw2bQq6EhEREQmaQlgnys31vqo3TERERBTCOtExx8CECQphIiIiohDW6XJz4c03IT8/6EpEREQkSAphnSwvz/v62GPB1iEiIiLB8jWEmdlMM1tnZuvN7IYm3v+lma0IPz4ys/1+1hMNRoyAMWO0cKuIiMjRzrcQZmaxwN3AucBoYL6ZjY48xjn3TedctnMuG/gN8E+/6mmtmpoq9u17Aeecb23k5sJrr8HOnb41ISIiIlHOz56wycB659wG51wFsAiY28zx84EHfKynVXbteoCVK8+kqOht39rIywPnNCQpIiJyNPMzhA0EtkQ8zw+/dggzGwoMB170sZ5W6d17LjExIXbu/KtvbZx4Ipxwgu6SFBEROZpFy8T8i4BHnHPVTb1pZlea2TIzW1ZQUOBrIXFxafTuPY9duxZRU1PhSxtmXm/YSy/B7t2+NCEiIiJRzs8QthUYHPF8UPi1plxEM0ORzrl7nXM5zrmcrKysDiyxaX37XkpV1V727HnKtzZyc6G6Gv71L9+aEBERkSjmZwhbChxvZsPNLAEvaD3R+CAzGwlkAG/6WEubZGScTXx8H1+HJCdMgOHDNSQpIiJytPIthDnnqoCvA88AHwAPOefWmNnNZjYn4tCLgEXOz9sR2ygmJo4+feazZ8+/qazc50sbtUOSzz8P+7v9whwiIiLSmK9zwpxzTznnTnDOHeucuzX82k+cc09EHHOTc+6QNcSC1q/fpThXQUHBQ761kZsLlZXw5JO+NSEiIiJRKlom5kedlJSJJCePYscO/4YkJ0+GwYO1cKuIiMjRSCHsMMyMvn0vpajoDQ4e3OBTG3DBBfDMM1Bc7EsTIiIiEqUUwprRt+/FAOzc+Tff2sjLg/JyWLzYtyZEREQkCimENSMUGkLPntPZufOvvm1jNHUq9OunuyRFRESONgphLejb91IOHlzv2zZGMTHekORTT8GBA740ISIiIlFIIawFWVl5vm9jlJsLpaXwn//41oSIiIhEGYWwFsTFpZGZOdfXbYxOOw1699aQpIiIyNFEIawV+vX7ElVVe9m792lfzh8XB+ef760XVlbmSxMiIiISZRTCWqF2GyM/1wzLzYWSEnjuOd+aEBERkSiiENYK9dsYPenbNkZnnAEZGVq4VURE5GihENZK9dsYPezL+ePjYc4ceOIJqPBn6pmIiIhEEYWwVqrdxsjPuyTz8rzNvF980bcmREREJEoohLVS7TZGhYWvc/DgRl/aOOssSE3VkKSIiMjRQCGsDfzexigxET73OXj8caiq8qUJERERiRIKYW3QGdsY5ebCnj3wyiu+nF5ERESihEJYG3nbGH3s2zZGM2dCcrIWbhUREenuFMLayO9tjJKTYdYs+Oc/obralyZEREQkCiiEtVFnbGOUmws7d8KSJb6cXkRERKKAQlg79Ot3qa/bGJ13HoRCuktSRESkO1MIawdvG6Ms37YxSk2Fc87xhiRranxpQkRERAKmENYOMTHxvm9jlJcH+fnwzju+nF5EREQCphDWTn37+ruN0ezZ3lZGGpIUERHpnhTC2ik19SSSk0f6dpdkz57eCvqPPgo+LUkmIiIiAVIIa6fO2MYoLw82bYLly305vYiIiARIIewI+L2N0Zw5EBurhVtFRES6I4WwIxAKDSU9/XTftjHKzIQzzvDmhWlIUkREpHtRCDtC/fp52xgVF/tzG2NuLnz8Maxe7cvpRUREJCAKYUeodhsjv9YMO/987y7J227z5fQiIiISEIWwIxQXl+7rNkZ9+sAPfwgPPABP+7NAv4iIiATA1xBmZjPNbJ2ZrTezGw5zzIVmttbM1pjZP/ysxy/eNkZ72Lv3P76c/4YbYNQouOoqKCnxpQkRERHpZL6FMDOLBe4GzgVGA/PNbHSjY44Hvg9Mc86dCFznVz1+qt3GyK81wxIT4Q9/gC1b4Ec/8qUJERER6WStCmFm1sPMYsLfn2Bmc8wsvoWPTQbWO+c2OOcqgEXA3EbHXAHc7ZzbB+Cc29W28qND7TZGu3c/SWXlfl/amDoVrr4a7roL3n7blyZERESkE7W2J+xVIGRmA4FngUuBP7fwmYHAlojn+eHXIp0AnGBmb5jZW2Y2s5X1RB1vG6Ny37YxAm9y/oABcMUVUFnpWzMiIiLSCVobwsw5VwpcANzjnPs8cGIHtB8HHA9MB+YD95lZz0MaN7vSzJaZ2bKCgoIOaLbj+b2NEUBaGtxzD7z/PvziF741IyIiIp2g1SHMzE4GLgYWh1+LbeEzW4HBEc8HhV+LlA884ZyrdM5tBD7CC2UNOOfudc7lOOdysrKyWlly56rfxug137YxAm8V/bw8uPlm+Ogj35oRERERn7U2hF2HN4H+MefcGjM7Bniphc8sBY43s+FmlgBcBDzR6JjH8XrBMLPeeMOTG1pZU9TxexujWr/5DSQlwZVXQk2Nr02JiIiIT1oVwpxzrzjn5jjn/ic8QX+3c25hC5+pAr4OPAN8ADwUDnA3m9mc8GHPAHvMbC1eqPuuc25Pu3+agPm9jVGtfv284chXXoH77/etGREREfGRtSYshNfvugqoxuvhSgN+7Zzr9JlJOTk5btmyZZ3dbKtt334/69ZdzsSJb5GWNsW3dpzz9pV87z344APo39+3pkRERKSdzOxd51xOU++1djhytHOuCJgHPA0Mx7tDUhrxexujWmZw771QVgbXXutrUyIiIuKD1oaw+PC6YPMIT6QH/Btv68K8bYzm+LaNUaTjj4cbb4RHH4V//cvXpkRERKSDtTaE/T9gE9ADeNXMhgJFfhXV1fXt6+82RpG+8x0YNw6+9jUoLPS9OREREekgrZ2Yf5dzbqBz7jzn+RSY4XNtXVavXuf4uo1RpPh4uO8+2LEDvv9935sTERGRDtLabYvSzezO2gVTzex/8XrFpAneNkYX+bqNUaTJk2HhQvjd7+CNN3xvTkRERDpAa4cj/wgUAxeGH0XAn/wqqjvojG2MIt1yCwwdCpdfDuXlndKkiIiIHIHWhrBjnXM3hjfj3uCc+2/gGD8L6+pSU3NIShrRKUOSACkpXiM/xlYAACAASURBVE/Yhx96e0yKiIhIdGttCDtoZqfUPjGzacBBf0rqHsyMfv1qtzHa1CltnnsufPGL8LOfwZo1ndKkiIiItFNrQ9hVwN1mtsnMNgG/Bb7qW1XdRJ8+nbONUaRf/crb6PuKK7SlkYiISDRr7d2RK51z44FxwDjn3ATgDF8r6waSkoaRnn6a79sYRcrKgjvvhDff9IYnRUREJDq1ticMAOdcUXjlfIBv+VBPt9Ov35c4ePAjiouXdlqbl14KZ53lLVmxZUunNSsiIiJt0KYQ1oh1WBXdWO02Rp01QR+8LY1+/3uoqoJrrvH2mRQREZHociQhTL/aW6HhNkaVndbuMcd4y1Y8+SQ88kinNSsiIiKt1GwIM7NiMytq4lEMDOikGru8vn0vpbJyd6dsYxTpG9+Ak07yNvjet69TmxYREZEWNBvCnHOpzrm0Jh6pzrm4ziqyq+vMbYwixcV5Wxrt3g3f/W6nNi0iIiItOJLhSGml+m2MnuiUbYwiTZgA3/423H8/vPRSpzYtIiIizVAI6ySdvY1RpBtv9OaIXXklHNQSuyIiIlFBIayTdPY2RpGSk+Hee2H9em+yvoiIiARPIayTBLGNUaTPfhYWLICf/xxWruz05kVERKQRhbBOFMQ2RpHuuAMyM+Hyy6G6OpASREREJEwhrBMFsY1RpMxM+PWvYdkyuOuuTm9eREREIiiEdbK+fS/t9G2MIn3hCzBrFvzoR7BpUyAliIiICAphnS4rKw+zxEAm6IO3pdE990BMDFx1lbY0EhERCYpCWCeLj+9J796dv41RpCFD4Gc/g2eegX/8I5ASREREjnoKYQEIahujSF/7GkyZAtdd562oLyIiIp1LISwAvXrNJD6+d2BDkgCxsfCHP8D+/fCtbwVWhoiIyFFLISwAQW5jFGnMGLjhBvjrX72hSREREek8CmEBqd/G6JFA6/jhD2HECG+S/oEDgZYiIiJyVFEIC0hq6iSSkk4IdEgSIBTytjTatMnbY1JEREQ6h68hzMxmmtk6M1tvZjc08f4CMyswsxXhx+V+1hNNzIy+fS+lsPDVQLYxinTaad7m3r/8pbeQq4iIiPjPtxBmZrHA3cC5wGhgvpmNbuLQB51z2eHHH/yqJxr17XsJALt2/T3gSuB//gf69oUrroDKYFbOEBEROar42RM2GVjvnNvgnKsAFgFzfWyvy6ndxmjHjmC2MYrUsyf89rewYgXceWegpYiIiBwV/AxhA4EtEc/zw681lmtmq8zsETMb7GM9UcnbxmgdxcXBjwNecAGcfz7cdBOsXx90NSIiIt1b0BPznwSGOefGAc8Bf2nqIDO70syWmdmygoKCTi3Qb0FvY9TYb38LCQnw1a9qSyMRERE/+RnCtgKRPVuDwq/Vcc7tcc6Vh5/+ATipqRM55+51zuU453KysrJ8KTYo9dsYPRDYNkaRBgyAn/8cXnwR/vznoKsRERHpvvwMYUuB481suJklABcBT0QeYGb9I57OAT7wsZ6oFQ3bGEW64go49VT49rdh586gqxEREemefAthzrkq4OvAM3jh6iHn3Bozu9nM5oQPW2hma8xsJbAQWOBXPdEsGrYxihQT460dduAAfOMbQVcjIiLSPVnQd+W1VU5OjlvWDRez+vjja9m27T6mTt1BfHzPoMsB4JZb4Cc/gSefhNmzg65GRESk6zGzd51zOU29F/TEfAmLlm2MIl1/PZx4Ilx9NRQXB12NiIhI96IQFiWiZRujSAkJcN99sHUr/OAHQVcjIiLSvSiERYlo2sYo0sknwzXXwN13w5tvBl2NiIhI96EQFkWiaRujSD/7GQwc6N01WVERdDUiIiLdg0JYFPG2MTo1KrYxipSaCr/7HaxZ4+0xKSIiIkdOISzKRNM2RpFmz4YvfAF++lP48MOgqxEREen6FMKiTFbW56NqG6NIv/419OjhDUvW1ARdjYiISNemEBZlvG2MPseuXYuiYhujSH37wv/+L7z+Otx6q4KYiIjIkVAIi0LeNkYF7N37TNClHGLBArjgAm8R1+nTNTQpIiLSXgphUahXr5nExWVG5ZCkGTzyCPzxj7B6NYwfD//931Be3vJnRUREpJ5CWBSKiUmgT5+L2L37X1RVFQZdziHM4LLL4IMPIDcXbroJJkyAN94IujIREZGuQyEsSvXrF33bGDXWty/84x+weLG32fcpp3hbHBVGX24UERGJOgphUSo1dTJJSSewY0f0DUk2dt553hpi3/wm3HsvjB4Njz0WdFUiIiLRTSEsStVvY/QKZWWfBl1Oi1JS4M474a23ICvLm7x//vnevpMiIiJyKIWwKFa7jdHOndG1jVFzJk2CpUu9lfX/8x8YNQruuUfLWYiIiDSmEBbFarcx2rkzurYxakl8PHzve97dk1OmeBuAn3qqN2QpIiIiHoWwKNe376WUln5IUdHbQZfSZsceC88+C3/5C6xb591B+ZOfQFlZ0JWJiIgETyEsymVlfZ64uF6sXXshpaXrgy6nzczgS1/ylrO48EK45RbIzoZXXw26MhERkWAphEW5+PiejB//AtXVpaxYcRoHDnTNJeqzsuBvf/PmiZWXw+mnw5VXwv79QVcmIiISDIWwLiA1NZvs7JdxroYVK06npGR10CW12znneHPFvv1tuP9+b+L+ww9DF5ryJiIi0iEUwrqIlJQxZGe/jFkcK1ZMp7j4vaBLarcePeCOO7y7KPv394Yp586FLVuCrkxERKTzKIR1IT16jCQ7+xViY5NZufIMioreCbqkIzJxIrzzjhfInn/eW+T1N7+B6uqgKxMREfGfQlgXk5x8HNnZrxIXl8HKlWdSWLgk6JKOSFycNzS5Zg1MnQoLF8K0afD++0FXJiIi4i+FsC4oKWkY2dmvkpDQj5Urz2b//leCLumIDR/uTdr/29/gk0+8XrIf/lDLWYiISPelENZFhUKDyM5+hVBoCKtWncvevc8HXdIRM4OLL/aWs/jiF+FnP4Nx4+Cll4KuTEREpOMphHVhiYn9yc5+maSk43j//dns2fNU0CV1iN69vQVen3vOmx92xhnwX/8Fe/cGXZmIiEjHUQjr4hIS+pCd/RI9epzI6tXzKCh4POiSOsyZZ3pzw66/3gtlo0bBokVazkJERLoHhbBuID4+k/HjXyAlZSJr136eXbseCrqkDpOcDLffDsuWwZAhMH8+zJ4Nn34adGUiIiJHRiGsm/BW1n+WtLTPsHbtfHbs+FvQJXWo7Gx46y345S/hlVfgxBPhV7/SchYiItJ1+RrCzGymma0zs/VmdkMzx+WamTOzHD/r6e7i4tIYO/ZpevY8nQ8//BLbt/8x6JI6VGwsXHedt5zFaafBN78Jn/kMrFwZdGUiIiJt51sIM7NY4G7gXGA0MN/MRjdxXCrwDeBtv2o5msTFpTB27L/JyDiLdev+i61bfxd0SR1u6FBYvBgeeAA2b4aTToJrroGXX4bKyqCrExERaR0/e8ImA+udcxuccxXAImBuE8fdAvwPoBWhOkhsbDJjxvyLzMzZfPzx18jP/3XQJXU4M7joIm85iwUL4A9/gBkzvI3CL7rIW29s9+6gqxQRETk8P0PYQCByN8D88Gt1zGwiMNg5t9jHOo5KsbEhTjzxUXr3voD1669j8+afB12SL3r18gLY7t3wz39Cbq7XI3bppdC3r7f6/m23eXdZ6q5KERGJJoFNzDezGOBO4NutOPZKM1tmZssKCgr8L66biIlJYPToRfTpcxEbNlzPpk23BF2Sb1JT4fzz4f77Yds2b0/KH/8YysvhBz/wFn0dNswbtnz6aTh4MOiKRUTkaGfOp+4BMzsZuMk5d074+fcBnHO3hZ+nA58AJeGP9AP2AnOcc8sOd96cnBy3bNlh35YmOFfNhx9+hZ07/48hQ37I8OG3YGZBl9Vptm2Dp56Cf//bWwC2tBSSkrx1yGbPhlmzYODAls8jIiLSVmb2rnOuyRsP/QxhccBHwGeBrcBS4IvOuTWHOf5l4DvNBTBQCGsv52r46KOvsn37Hxg06Nsce+wvjqogVquszBuuXLwYnnyyfr2xCRO8QDZ7NuTkQIwWbxERkQ7QXAjz7VeNc64K+DrwDPAB8JBzbo2Z3Wxmc/xqV5pmFsMJJ/w/Bgy4hvz8/2X9+oU4VxN0WZ0uFIKZM+E3v4GNG2H1am8x2B494NZbYcoU6N8fLrsMHn0UioqCrlhERLor33rC/KKesCPjnOOTT75Dfv6d9O9/BSec8Hu86XmyZw8884w3bPn007B/P8THw+mn1w9bHndc0FWKiEhXEshwpF8Uwo6cc46NG3/E5s0/o2/fLzNy5P14y7pJraoqWLLEC2T//re3FAbAiBH1w5bTpnkhTURE5HAUwuQQzjk+/fQWNm26kT595jNy5P8RExMXdFlRa8MGbx7Zv//tzSmrqID0dDjnHC+QnXsu9O4ddJUiIhJtFMLksDZv/h82bLiB3r1zGT36H8TEJARdUtQrLobnn/cC2eLFsHOnt3jsySfX95KNGeO9Fsk5r4etstILcUfyta2fqanxhlInTPAeQ4ceWp+IiHQ8hTBp1pYtv+STT75FZubnOPHEh4mJSQy6pC6jpgaWL68ftnz3Xe/1zExvr8vGgchPZpCQ4A2RJiQ0/N452LTJqxegZ09vU/TaUDZhAowcCXHqDBUR6VAKYdKirVvv4eOPryEj4xzGjHmM2NikoEvqkmrXJHvnHW+Zi8gg5PfX2Bam9ZWWejsHvPcerFjhfV21ylu2A7w7R8eObRjMxo6F5GT/r5uISHelECatsn37/axbdwU9e85g7NgniI3tEXRJ4rOqKli3zgtkkY/9+733Y2K8mxEig9mECd52USIi0jKFMGm1HTv+yocfLiA9fRpjxy4mLi416JKkkznnLWJb21tW+8jPrz9myJD6QFY7rDl4sOaZiYg0phAmbbJr14OsXXsxaWmTGDv2aeLjewZdkkSBgoJDg9lHH9VvjJ6Zeeg8sxNOaHmYVESkO1MIkzYrKHiMtWu/QI8e4xg//lni4zX+JIcqKamfZ1b7eP9970YE8OaTjRvXMJyNHevNPxMRORoohEm77NmzmNWrc0lOHsH48c+TkJAVaD3V1QcpL99MWdkmDh7cSFnZprpHTEwCvXtfQFZWHqHQoEDrPNpVVnqL20beALBiBRQWeu/HxsKoUTBpEkyd6i3tMWqU9usUke5JIUzabe/eZ1m9eh6h0HDGj3+BxMR+vrVVXV1WF7LKyhqGrLKyTVRU7GhwvFk8iYlDCIWGUVm5mwMHVgKQnn4KWVkXkpWVS2LiAN/qldZzzturs7a3bPlyePtt2LvXe79nT/jMZ7xANnUqTJ4MaWnB1iwiHaekxPvH2ZgxkHSU3XyvECZHZN++l3n//dkkJg4kO/tFEhMHtus8NTXllJVtjghWGxuFrO0NjjeLC4es4YRCww55JCb2b7DdUmnpOnbtepiCgoc4cOB9wEhPP5U+fb5AVlYuCQl9j+QySAdzDj7+2Nse6s03va9r1nivm3nDlrWh7OSTvcVmNfFfpGuorvbWTXz2WXjuOe+/8cpKL4CdcQacd573GDYs6Er9pxAmR6yw8A1WrTqX+PgssrNfJBQaesgxXsja0qgHa2NEyNrW4Pj6kHVowPJC1oB272l54MAHFBQ8xK5dD1Ja+gEQQ8+ep9f1kAU9tCpNKyz0eshqQ9lbb0FRkfde794NQ9mkSd1vDTPnvB0YNm70dmbIzoY+fYKuSqR1NmzwAtdzz8GLL8K+fd7rEybAWWd5X5cs8XYa2bDBe2/UqPpAdsop3pqH3Y1CmHSIoqJ3WLXqHGJj0xg69AeUl+c3mJ/lhazIv0+xhEKHD1kJCQM6Zb/KAwfWsGuXF8gOHlwHxJCRcQZZWRfSu/f5JCRo08doVVMDa9fWh7I33/TWNQNvdf/x4+tD2dSp3tIZ0d5bVlTkhazax4YNDZ8fPNjw+GHDvOHZ2sfEidBDS/hJFNi/3wtbtcHrk0+81wcN8kLXWWfBZz976D8kanvBn3rKe7zyinczT0qK95nzzvP24x3YvkGXqKMQJh2muHg5K1eeTVXVHryQNbiZkDUwqjYFd85x4MD77Nr1EAUFD3Lw4HogloyMz9KnjxfIdBdo9Nuzx+shqw1lb7/t7QYA0L9/w1A2cSIkdvIuXBUVsHnzoQGr9vs9exoen5YGw4d7j2OOqf8+OdmbO/fOO95j0ybv+JgYb15NZDA78URtOSX+q6z0/turDV3vvOP9QyklBaZP9wLU2Wd7Czy35R9DJSVemKsNZVu2eK+PH1/fS/aZz3Tdv+MKYdKhqqqKqKzcS2LioKgKWW3hnKOkZCUFBQ+ya9dDlJVtwCyOjIyzwj1kc4mPzwi6TGmFqipvWYzIuWUbN3rvJSTASSc1HMYccIT3ajgHO3Y0HbA2bvQWta3doxO8baWGDasPV43DVq9erfuFtWsXLF1aH8reeaf+xobkZO/njAxm2qRdjpRzXs/zc895c7teftkLTDEx3nSAs8/2gteUKR03jOicNze0NpC9/ro3v6xnTzjnHC+QzZzZtYbpFcJEmuEFsuXhHrKHKCvbhFk8vXqdEw5kc4iLSw+6TGmDHTu8QFYbypYtg/Jy772hQxuGsvHjvaAUqbCw6aHCDRu8Hqna/TZrDRhwaLiq/X7AAH8WrHXOG/6JDGXLl9f/nFlZDUPZpEnegroizSkogBdeqJ9QX7tTxrHH1g8xzpgBGZ30b9T9++H55+tD2c6d3j8ucnK8QDZrlvcPkGhe4kYhTKSVnHMUFy+tC2Tl5VswS6BXr5n06XMhmZlztJVTF1RR4S2NURvKliyBrVu995KSvIDSp0990KqdUFwrPb3pgDV8uNfLFS2Lz1ZUwOrVXiB7+23v6wcf1O9qcOyxXq9FbTDLzj76lguQhsrKvN6m2iHG997zXu/Z05vPVRu8jjkm2DrB62FescILY4sXe3/HnfP+wXHuuV4oO/vszguIraUQJtIOztVQVPRO+C7Lh6io2IpZIpmZ55GVdSGZmbOJi0sJukxppy1bGk7437//8L1Z0fY/9bYoKvKWCojsMavt3YiL83Y0iOwxGzlSW021hnPeTRSlpXDgQP28xNRUb45USkp0zmFyDlatqg9dr77qBbG4OK93uDZ05eRE/9+D3bvhmWe8UPaf/3jD8zEx3s9RO5ds3Ljgh+UVwkSOkBfI3gz3kD1MRcV2YmKSyMycFQ5k5xEbq1vWpGvYts2bX1bbW7Z0af1SIKmp3i/gyGA2cGDwv8jaqrKyYUCK/L6lr609piWJifWBLCWlYUBr/Gjte42Hzltj27b60PXcc978QvCWh6gNXaef7rXTVVVXe3+Xa4ctly/3Xh8woD6QnXlmMD+jQphIB3KuhsLC18OB7BEqK3cSE5NMZuZs+vT5Ar16nUtsrMZ4pOuoqfE2Y4/sLVuxwgsy4N11OnFiw3XZIn91NP410tzz9r7X1LHl5YcPSbW1t5aZ9/MlJ3tLgLTla+33znkT1xs/ioubfr32vcbLkjQnIaH14W3/fi90rV3rfTYrywsiZ5/tfR3UjXd4277d6x176ilvfltRkRdgTz21PpSNHNk5/7hQCBPxiXPV7N//GgUFD1JQ8CiVlQXExPQgM/M80tKmkJIygZSUCbrTMopVVx+kuHgZ1dVFJCUdRyg0nJiYbrhiZBuVl8PKlfW9ZatWHRpsIn+BNf5l1tzz9r7X+Hko1HwoasvXUCi43r7qai88tiW4tfR6aanXE3faafW9XePGRfcEdr9UVnrTDmp7yVav9l4fNgxuvhkuvdTf9hXCRDpBTU0VhYWvsGvXQ+zZs5iKiq1174VCw8KBbCKpqV4wS0joj3W1MZ5uoLx8K4WFSygqWkJh4RJKSpbjXFXEETGEQsNISjqe5OTjSUqqf4RCw7rssixydKmu9nrmonFeWtA2b4ann/YC2WWXwbx5/ranECYSgIqKXZSUvEdx8XuUlLxHScny8AKxnvj4PqSmTqzrLUtJmUBS0jGYHYX/VPVJTU0VBw6sCoeuNygsXEJ5+WYAYmJCpKZOJj19KmlpU4mPz+TgwfUcPPgxpaUfc/Cg96iuLq47n1kcodDwulAWGdJCoSHt3mZLRLovhTCRKFFVVURJycpwOFtOScl7lJaureuJiY1NIyUlm5SUCXUBLTl5lHpfWqmyci9FRW/V9XQVFb1NTY03gzohYSDp6dPqQldKyvgWhx2dc1RW7moQyiJDWu25AcwSSEo6pkHPWW1IS0wcpHAtcpRSCBOJYtXVZZSWrqkLZd5jJTU13mxds0RSUsaSkjIxHM4m0KPHuKN+8r9zjtLSdXXDikVFS8KbtQPEkpo6gbS0qXWhKxQa3OHtV1RsP6TnzHusp6amfkXXmJgQodCxhwxvJicfT0LCAA1Li3RjCmEiXYxz1ZSWrjtkOLOqan/4iBiSk0fVzS+rvwGgZ6B1+6m6+gBFRUsjQtebVFV5+/bExfUiLe1k0tOnkp4+jdTUnECXDHGuhvLyrYf0nHmPT3Cuou7YmJhkkpKOO2R4Mzl5lDaXF+kGFMJEugHnHGVln9YFstpwVlGxre6YUGh4g6HMlJQJJCb2D7Dq9isr29Kgl6u4+D2gGoDk5FENermSk0/oMsN9zlVTVra5yYBWVraxwU0CPXqMIyPjTDIyzqRnz9O0Fp1IF6QQJtKNVVTsjOgt8x4NbwDoS0pKNvHxvYmN7VH3iInpQWxscsT3jd+LfJ7ka8ipqamkpGRFg9BVXu4t6x4Tk0xa2pSI0PUZ4uN7+VZLkGpqqigr28TBgx9TUrKCffuep7DwdZyrwCyetLSp9Op1FhkZZ5KScpLmCop0AQphIkeZqqrCiBsA3uPAgfepqtpPdfUBamoOUF19AGjbf/sxMbWBLbmFwNb4efIh78XEJHHw4Md1oau4eGndHLjExKF1PVzp6VPp0WPcUR02qqtLKSx8nX37nmffvucoKVkBQGxsOhkZZ4R7ys4iKek4zS0TiUKBhTAzmwn8GogF/uCcu73R+1cB1+CNMZQAVzrn1jZ3ToUwkSPnnKOmpiwcykqprj7QIKA19TzyuOaPLcW58lbVYRZHSsrEutCVlnYyoVA3Xsa7A1RUFLB//4vs3fsc+/Y9V7fkRmLiEDIyzgqHss+SkJAVcKUiAgGFMPMWzPkIOAvIB5YC8yNDlpmlOeeKwt/PAb7mnJvZ3HkVwkSiX01NVQvhrpTExEGkpk466u/yPBLOOQ4eXB/uJXue/ftfrLt5IyUlu24+WXr6qcTGJrdwNhHxQ3MhzM8+/snAeufchnARi4C5QF0Iqw1gYT1o6/iIiESlmJg4YmLSiItLC7qUbs3MSE727qocOPBqnKumuPhd9u17jn37nic//9ds2XIHZgmkp0+r6ylLTZ2ohWVFooCfIWwgsCXieT4wpfFBZnYN8C0gATijqROZ2ZXAlQBDhgzp8EJFRLoDs1jS0iaTljaZoUN/SHX1Afbvf61uPtnGjT9g48YfEBeXQc+e3nyyXr3OIhQ6pkvNJ6uuLqWyci+xsT2Ii+vZpWoXieTncGQeMNM5d3n4+aXAFOfc1w9z/BeBc5xzX27uvBqOFBFpn4qKnezb92K4p+y5ujtQQ6FhdRP8e/Y8o9PWJ3Ouhqqq/VRW7qGycg9VVXvqvj/0+e661yIXwjVLJCGhH4mJ/UlI6E9CQr9DviYm9ic+vg8xMfGd8nOJRApqOHIrELlE9aDwa4ezCPidj/WIiBzVEhL60rfvfPr2nR+eT/YR+/Y9z969z7Fr10Ns3/4HwEhJmRAxn+yUVs3bq6kpbyZAHe75XqDmMGeMIT6+F3FxmcTHZxIKDSM19STi4zPDr/WiurqEioodVFRsp6JiB6WlH7F//yt1i/g2ZMTH924yoDUObnFxqUdwlUVaz8+esDi8ifmfxQtfS4EvOufWRBxzvHPu4/D3nwNuPFxarKWeMBGRjldTU0Vx8bK6+WRFRW/iXCVmiaSnn0LPnqfhXPVhA1V1dclhzx0TkxQRnuof3vPeTbyWSVxcervXpqupKaeiYmeDgFZevj3iee33O3Cusol6exw2oNV+9XrXemtunbQoyCUqzgN+hbdExR+dc7ea2c3AMufcE2b2a+BMoBLYB3w9MqQ1RSFMRMR/VVUlFBa+Wjef7MCB1QDExfUkPr73YQJV0wErWu+A9YZD9zUKaA2DWu171dWFTZwhloSEPhHBbCCJiYMJhYaQmDgk/HUQMTGJnf6zSfTQYq0iInJEqqpKiIkJHbUL51ZXl4Z71+qDWuPetfLyrVRW7jrks/HxfRsFsyGEQoPrnsfHZ3WZbbek7YKaEyYiIt1EXFxK0CUEKjY2maSk4SQlDW/2uOrqMsrL8ykv30xZ2WbKyzdTXr6FsrLNlJauYe/ep6mpKW3wGbPEcCgb3GRQS0wcfNRf/+5KIUxERKSDxMaGSE4+juTk45p83zlHVdW+uoDWOKjt3/8C5eXbaHzDQlxcr3A4axzUvNcSEvr71kvpLb5cFvE42Oj54R4NjwNHjx7jSU+fpm22whTCREREOomZER/fi/j4XqSmZjd5TE1NJRUV2ygr29IgqJWVbaas7FMKC1+r2xmhXiyJiQMbBLX4+N44V9Hq0FRd3fRx3s6CR/IzxxMTE8K56rpewPj4rLr9YdPTp5GSchKxsaEjaqcrUggTERGJIjEx8YRCQwmFhh72mKqqonDv2aFBrajobcrLH4m489OIiUkKz+lr+hEfn05i4uHfj4kJERvb/DnqH5HHJdbdQepcDaWlH1BYuITCwjcoKlrCnj3/8iq0BFJTTyI9fVpdOEtI6Ov3pQ6cJuaLiIh0M87VUF3t3UxhFh+1Q38VFbsoKnqTwsI3KCxcQnHxUpyrACAUOpb09Gmkp08lLW0aPXqM7pI3/ItwfAAACAdJREFUMOjuSBEREYl6NTXlFBcvr+spKyx8o+6O09jYdNLTTw73lE0jNXVyl7hhQXdHioiISNSLiUkkPf1k0tNPBrwbGcrKNtT1lBUVvcGmTTcCDoglJWV8XU9ZevpUQqGutb+0esJERESky6is3E9R0Vt1PWVFRW9TU3MAgMTEQXU9ZWlpU0lJGR/4nqHqCRMREZFuIT6+J5mZM8nMnAl4S2gcOLCqrqessHAJBQUPARATk0xa2uS6nrK0tJOJj88IsvwGFMJERESky4qJiSM1dSKpqROBrwNQVpYf0VO2hM2bb6d2qY3k5NF1PWUZGZ8lFBocWO0ajhQREZFurbr6AEVFS+t6yoqKllBVtZ/hw29j6NAbfG1bw5EiIiJy1IqN7UFGxnQyMqYDtWuWfUhcXM9A61IIExERkaOKWQw9eowOugy63qpnIiIiIt2AQpiIiIhIABTCRERERAKgECYiIiISAIUwERERkQAohImIiIgEQCFMREREJAAKYSIiIiIBUAgTERERCYBCmIiIiEgAutwG3mZWAHzqczO9gd0+t9HV6Ro1T9enZbpGzdP1aZmuUfN0fVrWGddoqHMuq6k3ulwI6wxmtuxwO56LR9eoebo+LdM1ap6uT8t0jZqn69OyoK+RhiNFREREAqAQJiIiIhIAhbCm3Rt0AV2ArlHzdH1apmvUPF2flukaNU/Xp2WBXiPNCRMREREJgHrCRERERP5/e/cfqmdZx3H8/enMcNOYv2jZzuJIDWWZOpGwhP7QipXigv5QsbDyH6V0hZTToL8ipKJsNYqy3MChxPIXQWtjigb5i9Z+T1Ns6FlnbRKzZuGvPv1xXwefc3YeaDCf69qzzwsOz3Vf53Cfz3Nxnvt8n+u+nvuuIEXYNJKWSHpG0nOSltfO0xJJCyQ9LGmnpB2SltXO1CpJI5L+LOm3tbO0RtJJktZKelrSLkkfqZ2pNZK+Vl5j2yXdLen42plqk/QrSfskbe/pO0XSBknPlseTa2asqc/4fK+8zrZKuk/SSTUz1jbTGPV87yZJlnTaIDOlCOshaQRYCXwKWARcJWlR3VRNeQO4yfYi4ELgyxmfvpYBu2qHaNSPgHW2zwLOJeM0haT5wI3ABbbPBkaAK+umasIqYMm0vuXARtsLgY1l+1i1ikPHZwNwtu1zgL8Atww6VGNWcegYIWkB8EnghUEHShE21YeB52w/b/s14B5gaeVMzbA9YXtTaf+L7p/n/Lqp2iNpFLgUuKN2ltZImgt8DPglgO3XbB+om6pJs4DZkmYBc4C/Vc5Tne1HgX9M614KrC7t1cBnBhqqITONj+31tt8om48DowMP1pA+f0MAPwS+AQx8kXyKsKnmAy/2bI+TImNGksaAxcATdZM06Xa6F/R/awdp0BnAfuDOcrr2Dkkn1A7VEtt7gO/TvSufAF62vb5uqmbNsz1R2nuBeTXDNO5LwO9qh2iNpKXAHttbavz+FGFx2CSdCPwG+Krtf9bO0xJJlwH7bP+pdpZGzQLOB35qezHwCsf2KaRDlHVNS+kK1vcCJ0j6XN1U7XP3Uf983H8Gkr5Jt5xkTe0sLZE0B7gV+FatDCnCptoDLOjZHi19UUg6jq4AW2P73tp5GnQRcLmk3XSnsy+WdFfdSE0ZB8ZtT86grqUryuItHwf+anu/7deBe4GPVs7Uqr9LOh2gPO6rnKc5kr4AXAZc7VyTarr3073Z2VKO2aPAJknvGVSAFGFTPQUslHSGpHfSLYZ9sHKmZkgS3VqeXbZ/UDtPi2zfYnvU9hjd389DtjOLUdjeC7wo6czSdQmws2KkFr0AXChpTnnNXUI+vNDPg8A1pX0N8EDFLM2RtIRuacTltv9dO09rbG+z/W7bY+WYPQ6cX45TA5EirEdZwPgV4Pd0B71f295RN1VTLgI+Tze7s7l8fbp2qDjq3ACskbQVOA/4TuU8TSmzhGuBTcA2uuP0MX/lc0l3A48BZ0oal3QtcBvwCUnP0s0g3lYzY019xucnwLuADeV4/bOqISvrM0Z1M2V2MiIiImLwMhMWERERUUGKsIiIiIgKUoRFREREVJAiLCIiIqKCFGERERERFaQIi4ijnqQ3ey6bslnSEbsKv6QxSduP1P4iIibNqh0gIuII+I/t82qHiIg4HJkJi4ihJWm3pO9K2ibpSUkfKP1jkh6StFXSRknvK/3zJN0naUv5mrxd0IikX0jaIWm9pNnl52+UtLPs555KTzMijlIpwiJiGMyedjryip7vvWz7Q3RXD7+99P0YWG37HLqbGq8o/SuAR2yfS3dPy8k7ZiwEVtr+IHAA+GzpXw4sLvu57u16chExnHLF/Ig46kk6aPvEGfp3Axfbfr7cfH6v7VMlvQScbvv10j9h+zRJ+4FR26/27GMM2GB7Ydm+GTjO9rclrQMOAvcD99s++DY/1YgYIpkJi4hh5z7tw/FqT/tN3lpPeymwkm7W7ClJWWcbEf+3FGERMeyu6Hl8rLT/CFxZ2lcDfyjtjcD1AJJGJM3tt1NJ7wAW2H4YuBmYCxwyGxcR0U/etUXEMJgtaXPP9jrbk5epOFnSVrrZrKtK3w3AnZK+DuwHvlj6lwE/l3Qt3YzX9cBEn985AtxVCjUBK2wfOGLPKCKGXtaERcTQKmvCLrD9Uu0sERHT5XRkRERERAWZCYuIiIioIDNhERERERWkCIuIiIioIEVYRERERAUpwiIiIiIqSBEWERERUUGKsIiIiIgK/gcU3NAYSj03oQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 720x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_acc_loss(history)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pbHUMaOjBcpA",
   "metadata": {
    "id": "pbHUMaOjBcpA"
   },
   "source": [
    "Looking on the learning curve, we could have trained the model for fewer epochs, like 9 or 10 because after 9, there seems to be no improvements. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "DGX0X6PVOE8K",
   "metadata": {
    "id": "DGX0X6PVOE8K"
   },
   "source": [
    "<a name='2-7'></a>\n",
    "\n",
    "## 2.7 Performing Inference on New Texts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "xgBvWH3VOLdK",
   "metadata": {
    "id": "xgBvWH3VOLdK"
   },
   "outputs": [],
   "source": [
    "def predict(model, sample_news, class_names):\n",
    "\n",
    "  # Convert sample news into array\n",
    "\n",
    "  sample_news = np.array(sample_news)\n",
    "\n",
    "  # Predict the news type\n",
    "\n",
    "  preds = model.predict(sample_news)\n",
    "\n",
    "  pred_class = np.argmax(preds[0])\n",
    "\n",
    "  print(f'predicted class: {pred_class} \\nPredicted Class name: {class_names[pred_class]}')\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "vx4pDaAeOPJi",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "vx4pDaAeOPJi",
    "outputId": "424b1617-29ff-4760-cec6-64d52700241a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted class: 3 \n",
      "Predicted Class name: Sci/Tech\n"
     ]
    }
   ],
   "source": [
    "sample_news = ['Tesla, a self driving car company is also planning to make a humanoid robot. This humanoid robot appeared dancing in the latest Tesla AI day']\n",
    "\n",
    "predict(news_classifier, sample_news, class_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "6RrCqt3PORiJ",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "6RrCqt3PORiJ",
    "outputId": "958f502e-cc79-4108-b7d7-f693be545c70"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted class: 1 \n",
      "Predicted Class name: Sports\n"
     ]
    }
   ],
   "source": [
    "sample_news = [\"In the last weeks, there has been many transfer suprises in footbal. Ronaldo went back to Old Trafford, \"\n",
    "                \"while Messi went to Paris Saint Germain to join his former colleague Neymar.\"\n",
    "                \"We can't wait to see these two clubs will perform in upcoming leagues\"]\n",
    "            \n",
    "\n",
    "predict(news_classifier, sample_news, class_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "NP4Luis7OT8C",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "NP4Luis7OT8C",
    "outputId": "660d8c99-63a6-4908-ef71-6275bd186d81"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted class: 3 \n",
      "Predicted Class name: Sci/Tech\n"
     ]
    }
   ],
   "source": [
    "sample_news = [\"In the latest tech news: The tech giant NVIDIA has acquired ARM, a microproccessor company\"]\n",
    "            \n",
    "\n",
    "predict(news_classifier, sample_news, class_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "_lgKvb97BHNP",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "_lgKvb97BHNP",
    "outputId": "58a11fff-ee45-4f20-9b6f-f34e3fc57f53"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predicted class: 2 \n",
      "Predicted Class name: Business\n"
     ]
    }
   ],
   "source": [
    "sample_news = [\"Salesforce reported making over 1B sales in last quarter\"]\n",
    "            \n",
    "\n",
    "predict(news_classifier, sample_news, class_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "LmMjvJEoOaPZ",
   "metadata": {
    "id": "LmMjvJEoOaPZ"
   },
   "source": [
    "<a name='3'></a>\n",
    "\n",
    "# 3. Further Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "UUPjzSNoPqht",
   "metadata": {
    "id": "UUPjzSNoPqht"
   },
   "source": [
    "We have learned about BERT (Bidirectional Encoder Representation Transformer) and how to finetune it to model basic NLP tasks such as text classification. \n",
    "\n",
    "If you would like to learn more about BERT and Text Classification, here are resources to use:\n",
    "\n",
    "* [TensorFlow official text classification with BERT tutorial](https://tfhub.dev/google/collections/bert/1)\n",
    "* [BERT: Pre-training of Deep Bidirectional Transformers for\n",
    "Language Understanding](https://arxiv.org/pdf/1810.04805.pdf)\n",
    "* [BERT Models in TensorFlow Hub](https://www.tensorflow.org/text/tutorials/classify_text_with_bert)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tF8_wNZsQsZw",
   "metadata": {
    "id": "tF8_wNZsQsZw"
   },
   "source": [
    "## [BACK TO TOP](#0)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "9.5 Using Pretrained BERT for Text Classification.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3.7.10 64-bit ('tensor': conda)",
   "language": "python",
   "name": "python3710jvsc74a57bd034ac5db714c5906ee087fcf6e2d00ee4febf096586592b6ba3662ed3b7e7a5f6"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "046226810bd141599517fba1ded1ed8f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "0ba653c10cc449368a8910a5854ddb71": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0ca9a37cd38b4ce8ba1d6535c2d85712": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "0cc6e536dabf4e3397f1d8db0765bb63": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0d8bf4e088a849b6a0fb83f7bbeba65d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "12b0c1fcd12c4d4fa1dfdc1593ff3972": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1bcd2619383e41c6ae1ae9108530d3e8": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "25f4bb62d1ab47779bfbe78dfe58fdac": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b99b0cff30294582ae1e66e2a633869b",
      "placeholder": "​",
      "style": "IPY_MODEL_85b1127de5c245f1ad61b33bae41279e",
      "value": " 0/0 [00:00&lt;?, ? MiB/s]"
     }
    },
    "27ffec67ddd849778376db18144d6f38": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "2ac92ba33fcd419991df92f41de01c65": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_66370f990ba64dc1a28c984e344b23ac",
       "IPY_MODEL_66e634e821fc4c55a84f314a472170f0",
       "IPY_MODEL_d14cd8661cae4f7a8c6785f56b398dc6"
      ],
      "layout": "IPY_MODEL_cde24def6f35489391282d157e6a3550"
     }
    },
    "2c48c62afa474d1a9a62b6daf6931d40": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_d8deccabcafc4e649e46fca288bf95dd",
       "IPY_MODEL_ed825f71daa6460da541326d01b098e9",
       "IPY_MODEL_25f4bb62d1ab47779bfbe78dfe58fdac"
      ],
      "layout": "IPY_MODEL_12b0c1fcd12c4d4fa1dfdc1593ff3972"
     }
    },
    "2fc4685daa1f4c8dbd3833fb607aa650": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "3211dd4a60b04381a2ce78c2278b5023": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "354f3b9cc77c4615a762df558e5c5267": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "377499c0d2c94299926b820deee71819": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_64fe9627b18644668b6f386b72f9ec98",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_75c874a4d3884c8cbbc26ffa57d97b6f",
      "value": 0
     }
    },
    "3fb758f51cb2447ba8e5767fcc95ea2b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f704ed4f5e254d04b2d8d1fb36f77e1a",
      "placeholder": "​",
      "style": "IPY_MODEL_0cc6e536dabf4e3397f1d8db0765bb63",
      "value": " 119999/120000 [00:00&lt;00:00, 247720.17 examples/s]"
     }
    },
    "40ad8f616a684bedb5dc40639541e466": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "4ce8e034694f4ae3a84560d66705ddab": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "515f8749c3cd420d85289ac142257240": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_feb1ffd9c3504e238531a5d8b3ee9f8a",
      "placeholder": "​",
      "style": "IPY_MODEL_e9c1537f77814146b98ed4e0772626e2",
      "value": "Dl Completed...: "
     }
    },
    "519c55322a6940e789892928e9e62e24": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "566bb79ab07f4e208fc43e0643d76bc3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_85842ecd8d1f44a58fbcea48c39b904b",
       "IPY_MODEL_d3a0f7aba98d4742a8cdcb7c315390c3",
       "IPY_MODEL_d25d9a0a261b4c2db8f7020b7ec2cc9f"
      ],
      "layout": "IPY_MODEL_0ca9a37cd38b4ce8ba1d6535c2d85712"
     }
    },
    "5f5ba5d5d3be47b69202839d3ce3dd27": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "64fe9627b18644668b6f386b72f9ec98": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "6578461a0b3040418cf2fec36cc26bd4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "66370f990ba64dc1a28c984e344b23ac": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_354f3b9cc77c4615a762df558e5c5267",
      "placeholder": "​",
      "style": "IPY_MODEL_b0a973b98e5d4541a0c677c874e64c50",
      "value": "Extraction completed...: 100%"
     }
    },
    "66e634e821fc4c55a84f314a472170f0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_aa33d0e6a828439db44ff7bb83e728a3",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_046226810bd141599517fba1ded1ed8f",
      "value": 1
     }
    },
    "68f17020d75d4e2e834536fa9838a76b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c1cf085a00d148b4b583950ac8f06073",
      "placeholder": "​",
      "style": "IPY_MODEL_ea1ab1d0e9244801852bfd6840d8cbc4",
      "value": "100%"
     }
    },
    "69e2be6bc02b4283aeedc41683d2c75d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "6c57f7f09d7d4e6a89e4b02ce378aae0": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "6e8761c0b1ad41f29dd174a2d4373fe5": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "7374cef848094e08bf146366013931bf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_0d8bf4e088a849b6a0fb83f7bbeba65d",
      "placeholder": "​",
      "style": "IPY_MODEL_4ce8e034694f4ae3a84560d66705ddab",
      "value": "100%"
     }
    },
    "75c874a4d3884c8cbbc26ffa57d97b6f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "7ee38e0deb744620a1eb7c866c24f640": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "7fbfd0d4356349f99215c963d9d0f58b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "danger",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_8e85031ac9fc4d9d9f6a55e8172e4df6",
      "max": 120000,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_2fc4685daa1f4c8dbd3833fb607aa650",
      "value": 119999
     }
    },
    "80af01b6d7a54fa99744a536f5580529": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_7374cef848094e08bf146366013931bf",
       "IPY_MODEL_d4cb8bcdf308470c9d7a5949b5181736",
       "IPY_MODEL_feca4ddf88df4f72ac3dfd6fb124ccf3"
      ],
      "layout": "IPY_MODEL_fe9f20e182a2454c804e68c317834c96"
     }
    },
    "85842ecd8d1f44a58fbcea48c39b904b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_6e8761c0b1ad41f29dd174a2d4373fe5",
      "placeholder": "​",
      "style": "IPY_MODEL_7ee38e0deb744620a1eb7c866c24f640",
      "value": ""
     }
    },
    "85b1127de5c245f1ad61b33bae41279e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8e85031ac9fc4d9d9f6a55e8172e4df6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "8fa058e1bdd7435b890054f3f0aa06df": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c4b2124409f64d0ca42ffb44056504de",
      "placeholder": "​",
      "style": "IPY_MODEL_27ffec67ddd849778376db18144d6f38",
      "value": ""
     }
    },
    "8ffdc8e410ff467f826aa116f517658e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_68f17020d75d4e2e834536fa9838a76b",
       "IPY_MODEL_7fbfd0d4356349f99215c963d9d0f58b",
       "IPY_MODEL_3fb758f51cb2447ba8e5767fcc95ea2b"
      ],
      "layout": "IPY_MODEL_6578461a0b3040418cf2fec36cc26bd4"
     }
    },
    "97b8666b250d46c9b8d8f463464ce793": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "98e6f46668294d71bea96540e5cc3122": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "9c3c2794090b410da00dd380ae00e34e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "9cd6636a115c404099bf30a71eb44738": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a13335e07e85455e8366cb35077de77e": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a3c813936264408d99f3119fd9499df2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "a50d20a0b4fe4bd2bcdcff21b9eb64ba": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "a9a512b8b9254d91a5530852bd8033df": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "aa33d0e6a828439db44ff7bb83e728a3": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "b0a973b98e5d4541a0c677c874e64c50": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "b18fb0dc468b42ea8afd392b9cb4f727": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_f11bb92965034840b21e3a050c7ad8a6",
      "placeholder": "​",
      "style": "IPY_MODEL_0ba653c10cc449368a8910a5854ddb71",
      "value": " 119746/0 [00:33&lt;00:00, 3615.91 examples/s]"
     }
    },
    "b67c6831b62e4e28ad081517a6ea06eb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "b99b0cff30294582ae1e66e2a633869b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c17648de297344679cdec0b3c734b223": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c1cf085a00d148b4b583950ac8f06073": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "c4b2124409f64d0ca42ffb44056504de": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cde24def6f35489391282d157e6a3550": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "cebb666792534b11ab7696da849ee17d": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "d14cd8661cae4f7a8c6785f56b398dc6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_97b8666b250d46c9b8d8f463464ce793",
      "placeholder": "​",
      "style": "IPY_MODEL_a3c813936264408d99f3119fd9499df2",
      "value": " 1/1 [00:00&lt;00:00,  2.10 file/s]"
     }
    },
    "d25d9a0a261b4c2db8f7020b7ec2cc9f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_5f5ba5d5d3be47b69202839d3ce3dd27",
      "placeholder": "​",
      "style": "IPY_MODEL_f0ed3f707f0140af8a7dfcb5b843254c",
      "value": " 7256/0 [00:01&lt;00:00, 4090.46 examples/s]"
     }
    },
    "d3a0f7aba98d4742a8cdcb7c315390c3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_519c55322a6940e789892928e9e62e24",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_69e2be6bc02b4283aeedc41683d2c75d",
      "value": 1
     }
    },
    "d4cb8bcdf308470c9d7a5949b5181736": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "danger",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a50d20a0b4fe4bd2bcdcff21b9eb64ba",
      "max": 7600,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9c3c2794090b410da00dd380ae00e34e",
      "value": 7599
     }
    },
    "d63a80ebc12a4adfbb6238033f91c638": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "info",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_b67c6831b62e4e28ad081517a6ea06eb",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_6c57f7f09d7d4e6a89e4b02ce378aae0",
      "value": 1
     }
    },
    "d6a78c6422574503a85c7bbc8be3321c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_c17648de297344679cdec0b3c734b223",
      "placeholder": "​",
      "style": "IPY_MODEL_3211dd4a60b04381a2ce78c2278b5023",
      "value": " 0/0 [00:00&lt;?, ? url/s]"
     }
    },
    "d8deccabcafc4e649e46fca288bf95dd": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_1bcd2619383e41c6ae1ae9108530d3e8",
      "placeholder": "​",
      "style": "IPY_MODEL_a13335e07e85455e8366cb35077de77e",
      "value": "Dl Size...: "
     }
    },
    "e9c1537f77814146b98ed4e0772626e2": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ea1ab1d0e9244801852bfd6840d8cbc4": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ed825f71daa6460da541326d01b098e9": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "success",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_a9a512b8b9254d91a5530852bd8033df",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9cd6636a115c404099bf30a71eb44738",
      "value": 0
     }
    },
    "f0ed3f707f0140af8a7dfcb5b843254c": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "f11bb92965034840b21e3a050c7ad8a6": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f704ed4f5e254d04b2d8d1fb36f77e1a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f9b9ef9699704cbbb8920fef2e60fe71": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "f9ed0b3303a549b6a1620a7b0f43ad45": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8fa058e1bdd7435b890054f3f0aa06df",
       "IPY_MODEL_d63a80ebc12a4adfbb6238033f91c638",
       "IPY_MODEL_b18fb0dc468b42ea8afd392b9cb4f727"
      ],
      "layout": "IPY_MODEL_f9b9ef9699704cbbb8920fef2e60fe71"
     }
    },
    "fe9f20e182a2454c804e68c317834c96": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "feb1ffd9c3504e238531a5d8b3ee9f8a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "feca4ddf88df4f72ac3dfd6fb124ccf3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_cebb666792534b11ab7696da849ee17d",
      "placeholder": "​",
      "style": "IPY_MODEL_98e6f46668294d71bea96540e5cc3122",
      "value": " 7599/7600 [00:00&lt;00:00, 94341.21 examples/s]"
     }
    },
    "ffe4fbd6097e414b99cb4c4780557553": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_515f8749c3cd420d85289ac142257240",
       "IPY_MODEL_377499c0d2c94299926b820deee71819",
       "IPY_MODEL_d6a78c6422574503a85c7bbc8be3321c"
      ],
      "layout": "IPY_MODEL_40ad8f616a684bedb5dc40639541e466"
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}