Week 03 - Introduction to Computer Vision/1. Introduction to CNNs.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Convolutional Neural Networks\n",
    "<!--<badge>--><a href=\"https://colab.research.google.com/github/TheAIDojo/AI_4_Climate_Bootcamp/blob/main/Week 03 - Introduction to Computer Vision/1. Introduction to CNNs.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a><!--</badge>-->\n",
    "\n",
    "In this notebook, we will explore the basics of convolutional neural networks (CNNs) and their applications in the field of climate change. We will be using the popular Python libraries such as Numpy, Pandas, TensorFlow, Scikit Learn, and TensorBoard to build and train our models.\n",
    "\n",
    "CNNs are a type of neural network that are commonly used in image and video analysis, and have been shown to be highly effective in tasks such as object recognition, image classification, and object detection. In the context of climate change, CNNs can be used to analyze satellite imagery and other data to better understand and predict changes in the Earth's climate.\n",
    "\n",
    "Before diving into CNNs, it is important to have a basic understanding of neural networks and how they work. If you are new to neural networks, we recommend starting with a tutorial on basic neural networks before continuing with this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Table of Contents <a name=\"toc\"></a>\n",
    "* [Understanding the structure of a CNN](#intro)\n",
    "* [Fashion MNIST CNN](#cn_tf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Understanding the structure of a CNN <a name=\"intro\"></a>\n",
    "[Back to Top](#toc)\n",
    "\n",
    "CNNs consist of multiple layers, including convolutional layers, pooling layers, and fully connected layers. The convolutional layers are responsible for extracting features from the input image, such as edges and textures. The pooling layers are used to reduce the dimensionality of the feature maps, while the fully connected layers are used to classify the image based on the features extracted.\n",
    "\n",
    "During the training process, the CNN learns the weights and biases of the filters in the convolutional layers. These filters are used to convolve over the input image and extract features. The training process also involves adjusting the weights and biases of the fully connected layers to improve the classification accuracy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convolution Layers <a name=\"conv\"></a>\n",
    "[Back to Top](#toc)\n",
    "\n",
    "A convolutional filter is a small matrix that slides across the input image, performing a mathematical operation at each location. The output of the convolution is a new image, called a feature map, which highlights certain features of the input image.\n",
    "\n",
    "For example, a filter that detects edges in an image will produce a feature map where the pixels have high values where edges are present and low values where edges are not present. By applying multiple filters with different properties, a CNN can extract multiple features from the input image.\n",
    "\n",
    "The process of convolution can be thought of as a way to \"localize\" features in an image. By moving the filter across the image, the CNN can detect features at different positions in the image. This is in contrast to fully connected layers, which treat all pixels in the image as a single vector and do not take into account the spatial relationships between pixels.\n",
    "\n",
    "In Keras, the Conv2D layer is used to create convolutional layers in a CNN. The main parameters of this layer are the number of filters, the stride, and the padding. The number of filters determines how many feature maps are created by the convolution, the stride controls the step size of the filter when moving across the image, and the padding controls the amount of blank space added to the edges of the image before the convolution is applied."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pooling Layers <a name=\"pool\"></a>\n",
    "[Back to Top](#toc)\n",
    "\n",
    "Another important component of CNNs are pooling layers. Pooling layers are used to downsize the spatial dimensions of the feature maps, while keeping the most important features.\n",
    "\n",
    "The most common type of pooling is max pooling, in which the maximum value of a group of pixels is selected and used as the output. This has the effect of reducing the spatial resolution of the feature map, while retaining the most important features.\n",
    "\n",
    "There are also other types of pooling, such as average pooling, which takes the average value of a group of pixels, and min pooling, which takes the minimum value.\n",
    "\n",
    "The main advantage of using pooling layers is that they decrease the computational cost of the CNN and make it more robust to small translations of the input. By down-sampling the feature maps, pooling layers also make the CNN more tolerant to small variations in the position of the features.\n",
    "\n",
    "Pooling layers also make the CNN less sensitive to small changes in the input, which makes it more robust to noise and small variations in the position of the features. In addition, pooling layers also help to reduce overfitting by reducing the number of parameters in the model.\n",
    "\n",
    "We can stack multiple `Conv2D` and `MaxPooling` layers to create a deep CNN."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Fully Connected Layers (Dense) <a name=\"fcn\"></a>\n",
    "[Back to Top](#toc)\n",
    "\n",
    "After the convolutional and pooling layers, the final stage of a CNN is the fully connected layers. These layers take the output of the previous layers and use it to make a prediction.\n",
    "\n",
    "In fully connected layers, each neuron in the layer is connected to every neuron in the previous layer. This allows the network to learn complex, non-linear relationships between the features extracted by the convolutional and pooling layers.\n",
    "\n",
    "The output of the fully connected layers is usually passed through an activation function, such as sigmoid or softmax, to produce a probability distribution over the possible classes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fashion MNIST CNN <a name=\"cn_tf\"></a>\n",
    "[Back to Top](#toc)\n",
    "\n",
    "we will be working with the Fashion MNIST dataset, which consists of 60,000 28x28 grayscale images of clothing items and 10 classes. This dataset is a great starting point for learning about CNNs, as it is a relatively simple dataset with clear class labels.\n",
    "\n",
    "We will be using the TensorFlow and Keras libraries in Python to train and evaluate our CNNs. We will cover the different layers and techniques used in CNNs, such as convolutional layers, pooling layers, and fully connected layers.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import libraries\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (60000, 28, 28) y_train shape: (60000,)\n"
     ]
    }
   ],
   "source": [
    "# load the data\n",
    "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.fashion_mnist.load_data()\n",
    "\n",
    "# show the shape of the data\n",
    "print(\"x_train shape:\", x_train.shape, \"y_train shape:\", y_train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# create index labels dictionary\n",
    "index_labels = {\n",
    "    0: \"T-shirt/top\",\n",
    "    1: \"Trouser\",\n",
    "    2: \"Pullover\",\n",
    "    3: \"Dress\",\n",
    "    4: \"Coat\",\n",
    "    5: \"Sandal\",\n",
    "    6: \"Shirt\",\n",
    "    7: \"Sneaker\",\n",
    "    8: \"Bag\",\n",
    "    9: \"Ankle boot\",\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAx4AAAFQCAYAAAAx96gLAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABZjUlEQVR4nO3dd5hURdb48TNInkSGGRhmBERRQCSIYABckCCKmEUXWJXXNcKy5lcJ6hoX45rWAAZQVEQBFUERJYgkyQMIMsQhhyEP4f7+8Ee/3qozdNHTd6Zn+H6eZ59n61B9u6a7um6Xfc89cZ7neQIAAAAAASpR2AMAAAAAUPyx8QAAAAAQODYeAAAAAALHxgMAAABA4Nh4AAAAAAgcGw8AAAAAgWPjAQAAACBwbDwAAAAABK5kpA88evSobNiwQRITEyUuLi6aY0IR5Xme7N69W1JTU6VEiWD3tMw/mApy/okwB+HH/ENh4xyMwuQ6/yLeeGzYsEHS0tIifTiKsbVr10qtWrUCfQ7mH/JSEPNPhDkIHfMPhY1zMApTuPkX8cYjMTEx9ARJSUmRHgbFSE5OjqSlpYXmRpAKYv55nudrR/O/6ixbtsyK3XvvvVase/fuVqxx48a+dunSpa0+JUvaH+3MzEwrNm7cOF87IyPD6tO3b18rVqFCBStW2Apy/okUvTVwy5Ytvvbw4cOtPjfccIMVq169emBjEhFZsGCBr718+XKrT7du3axYqVKlAhtTJJh//ycrK8uKTZs2zdf+6quvrD4VK1a0Ytdff70VO/vss31tbc6MGTPGik2ePNmKlS9f3te+7rrrrD5/+9vfrFgsKm7n4JNBdna2r52SklJII8k/1/kX8cbj2JewpKQkJh18CuJn14KYf0FuPBISEqyYtlkoV65c2Me6bjzME6yI/eWtTJkyVh/t9Y3lz3xB/exf1NbAAwcO+Nply5a1+mgnjKD/NnM+a/NUG0OsbTyOYf7p88hcy7T3T1vL4uPjrZj592rrqbaWnXLKKVbMXCu1NTfWXt9wiss5+GSwZ88eX7s4vJbh5h/J5QAAAAACF/EvHkBRZf6SIaLv0F3+q9Gvv/5qxUaOHGnFRo0a5Wtr/+XN/C8fIiIPP/ywFdu+fXvYcbmqX7++rz1//nyrz1NPPWXFatSoYcU6duzoa//zn/+0+jRq1OhEh4gIaHPJvPTk/ffft/p8/PHHVqxq1apWzPwv09p/vdbGcPDgQSu2du1aX/uKK66w+mifl2uuucaKIVjffPONFXvhhResmParQW5urq+t/eKmXaKlXWq1adMmX1u7RFT71Ve7jCU5OdnX/uyzz6w+L774ohVr3769FXv55ZetGArexRdfbMV27Njha1epUsXq89Zbb1kxbW652LBhgxVr166dFdu/f7+vXbt2bavPt99+a8W0XwKLCn7xAAAAABA4Nh4AAAAAAsfGAwAAAEDgyPHAScf1jh85OTm+ds+ePa0+Wk6ElkNi3nVFuwZau5Wkdm374cOHfe1du3ZZfbQ7A2nHcnktzj33XCtm3iFJRGT69Om+tnbrygsuuMCKffjhh2HHgBOj3eXHvJb96aeftvr861//smJLly61YuY19lruhnbLZe1uR+a18l26dLH6aPkiCN7KlSt97REjRlh9tLwt87p1kT8Kzv2ZVmBMqwvhcpcfbR3T1juXu6NpuSGtWrWyYuvWrbNiZl7bkCFD7MEicOZcExHZunWrr71+/XqrjzaXtbX06quv9rW1c9iRI0esmJbXZK6Tu3fvtvoU5XwODb94AAAAAAgcGw8AAAAAgWPjAQAAACBwbDwAAAAABI7k8jBci82ZtAShqVOnWrHOnTtHNAYtcUlLiouU9pwm1yTtoqp79+6+9po1a6w+1atXt2La62K+X1rio0Z7n833pnLlyk6P07i8zxotOd5MnNNehylTplixzMxMK9agQYOIxoW8mQngWvL3nXfeacVeeeUVK1amTJnjHjuv4zdr1syK/e1vf/O1tSJyWhFDBM9MjnZ9H7TkXvOGFNoaqJ3DTj31VCtm3ihBu9mFtv5o89RlDIcOHbJiWmG5RYsW+drjxo2z+nTt2jXsGJA/lSpVsmKrVq3ytbXzplacd+PGjVbMXBO1m8wsWLDAimk3kDHnljau4oZfPAAAAAAEjo0HAAAAgMCx8QAAAAAQODYeAAAAAAJHcnkYWpKcmRS3YsUKq8/bb79txbSEXLMipVbZUqsc7ZJIriUOa3+P1s/l+GYCs2tCcyyaM2eOFTOTyatUqWL1MauI58Ws5KtVTXWp9itivzfa665VBdbk5ub62mYVXxG92nStWrXCjkujjUv7rFDxN/rM99Gs5Csikp6ebsW098Kcv1u2bLH6aMm32mfIHIf2mYr0JgjIn969e/vaL7zwgtVHSzjXbrph3nBFW2s0pUuXtmLafDNpVcrLly/v9JwuY9i5c6cVM9dFEskLR926da3YjBkzfG3t5gbmTTNcaWuddiOV1NRUK2ae9/ft2xfRGIoSfvEAAAAAEDg2HgAAAAACx8YDAAAAQODYeAAAAAAIHMnlYWiJu2ZS0qRJk6w+EydOtGJpaWlWzKykqiUWTZgwwYr16dPHipkJfVrlVteK2Xv27PG1taRgM1HP9dix6IcffrBi5nujVcfVXhctIdxMWnv22WetPikpKVZMmzMbNmwI+zhtDFoyp5lcbr7vIiJz5861Yi+//LIVM5NMtWq/2us1atQoK0ZyefS5fD63bdvmdCwzSbxGjRpWH20t026qYI5LW7e0GIJn3tikVatWVp8vv/zSirVs2dKKmTcN0OaHVnFaS+w21xrtpiza8bU1yayCvnnzZquPRrsZyNNPP+30WASrQYMGVsw8J2prinmzHxF9/mlVyU3anNRukmHOSe2mCMUNv3gAAAAACBwbDwAAAACBY+MBAAAAIHDkeIShXd9nmjVrlhXLysqyYtp192bskksusfr8+uuvVuz++++3Ys2bN/e1GzVqZPXRrn2cOXOmFTP/ptatW1t9zOt9c3JyrD5FxWeffWbFzGvPXYr5iejXFpvXEWs5Olouj1bY8Oabb/a133zzTavPWWedZcW0HBUzh6latWpWn3/84x9W7LXXXrNi5rWq2vNp19AuXbrUii1fvtzXrl+/vtUHJ8a8vtg1B0zLc9OKpwU5LtdCnQjWPffcY8VefPFFK6YVojTzMrS1QCvw53LNuzY/tMKGWj+Xa+x37dplxTp37mzFTobr84sClwK32rpm5jyK6DmU55xzjq+tve/aGLTvECbzu0JxxC8eAAAAAALHxgMAAABA4Nh4AAAAAAgcGw8AAAAAgSO5/E+04i5aoqNZHHD27NlWHy3ZaO/evVbMTKI12yIiLVq0sGL16tWzYmbxt+nTp1t9Pv/8cyumJUibhaPeeustq4+ZeK/9fUXF/PnzrZhZvE9LRjOLDOZFS040dezY0YolJCRYsczMTF/73//+t9Wne/fuVmzs2LFWzEy2NJPmRPQCgi5J9VqxQC2mFUn8+eeffW2Sy/PPXB+0uasVvdLmvfk+an209VRjJlxqCZjajQoQPHN90D7306ZNs2L/+7//G/bYWiK5VuRUK9RXrlw5X1ubf9rjzEKuIm4Jv1qfyy67LOzjUDi0hHBzbmnrk3ZzDW1Omjdv0QpTanNGSxw312GX+VjU8YsHAAAAgMCx8QAAAAAQODYeAAAAAALHxgMAAABA4E6a5HLXREcXjz76qK+dnZ3t9DitorWZzKQlv02dOtWKaQntZiJ806ZNrT6nnXZa2DGIiPznP//xtX///Xerz6hRo3ztolK5fOHChVZMq3Jrvi5aAqNrUmOlSpXCjmvx4sVWTJsP5nzTEjm1+a4lyZn9zKTuvGjJexs2bPC1tXml3azBTBQVEfnpp5987V69ejmNC3kzE4W1OaLFtGRHs1+kjxOxE5a1x2mfMwRPSyY3aWtBnTp1rNiqVat8be1GBomJiVZMuyGF+Vhtzmg35tiyZYsVc5l/tWvXtmKIXdr5PCsry9c+44wzrD7anNTWLC2Z3ORyvhWx57d5057iiF88AAAAAASOjQcAAACAwLHxAAAAABA4Nh4AAAAAAnfSJJdrSa2Rqlixoq+tJZdrCbNapWAzScmsLiyiJzxpCczm36glpWvVzLWEp02bNvnanTp1svoUVc8884wV017P+Ph4X9ulWreI/n6ZiWbazQG2bdtmxbZv327FzDljvlfa8+U1rtzcXF97586dVp+RI0dasR07dlgxc85rx9I+F1qi3pw5c6wY8sdMmtUqR2tJ3C5J4tqNBDQu67B2QwUULdo5xTy3aUnj2jlSSzg31y1tbXNN0nWZu9WqVXM6FmJDjRo1wvZxTRp3qSSurWvmzTzyiplrrvn9sjjiFw8AAAAAgWPjAQAAACBwbDwAAAAABO6kyfGIJvO6ftfrorXr281rEStXrmz1MQvfiOjXx5rXLLoWt9OOZV73um7dOqtPUdW6dWsrpuVJrFixwtfetWuX1UfL8dCKNJqvccuWLa0+2rXG2ntjxrS5pl2r6lK8TZszSUlJVqx+/fpWbO/evWHHpY0hNTXVil1xxRVWDPnjcq2y9v5rc9Ds53LsvJjXPWs5HtrnEwVPe5+1+VGzZk0rtmDBgrDH0t577fgHDhw44T4i+jnYzA/ZunWr1adWrVpWTGPOZZcCjCgYWh5QpMycDi3HQzufa/PBPCdq59vihl88AAAAAASOjQcAAACAwLHxAAAAABA4Nh4AAAAAAnfSZD6ZCTxaYpuWDKQV9NuwYYOvrSXEacWLzKJH2mPNonUielKzloRuJjprz5eQkGDFcnJyrFijRo18bTNxWMQugqe9VrHojjvucIqZRfJ+++03q8/rr79uxSZPnmzFKlWq5Gubr6+ISIUKFayY9h7mJ5HX5PK50JLytDnZuHFjX3vEiBH5HB0ipRV4NBPCtUR/LUkymvNNSwI2E3K1+abdxEFLHo5mAikil5GRYcXM+aetbdq8TU9Pt2Jmkq5WfFUrxKYl95rnapebcKDoibSItPY4c45ofVzXUrOf9h2wuOEXDwAAAACBY+MBAAAAIHBsPAAAAAAEjo0HAAAAgMCdNBlTZgKPVqFXSy4fOXKkFcvOzva1q1atavXRKoRrxzeTttesWWP1KVWqlBU7ePCgFTMT4LTq1dq4tEqtd955p689b948q4+ZFKq9pkWZmZx47rnnWn20GwtMmjTJipnzT3v/tAR+8zUW0RN0TVqCpBYzj6WNS5t/WmKvVhEehUObl2Ys0mRL18dq8y3S6unJyclWjETy2FW+fHkrpp3/TNraps0Zl8rlWnL5li1brJjLTVG0RHgULdp6FOnjXCrUa+uYNk/N2ObNm090iEUOv3gAAAAACBwbDwAAAACBY+MBAAAAIHBsPAAAAAAE7qRJLjeTgbTK4pqGDRtaMTNJU0vidk1eNxOJtIRJs+q1iJ50bI5DS1bWEu7S0tKsmFl1+r777rP6nHfeeb62VgG9qNASyMzXU5szWpJtYmKiFTPngzYXXJN9XaqmRpNr5Wqt8rrJNeEu6L+puNNev1i9+YPLjRcQG1xubCGiJ9uaN2HR1lPt/KQx1xrtWNqNVKpXr27FzITzk6Fy9Mkomsnl5jnRpbq5iP69zbx5S1ZW1gmOsOjhFw8AAAAAgWPjAQAAACBwbDwAAAAABC7wHA/tOjftWmPtOnLzsVohs/xcc+qic+fOViwhIcHXLleunNXHteCQed2rdg2gVqzNJUdF+5u110t7PxYsWOBrawW8ihPtGk1tvpnq1q1rxZKSkqxYpDlGLteORjMfQhuX61x2mSPa59ylsBhOjEs+h2uxNhfRPJbrHNH6uZ4PEBnX11zL99uxY4evrZ03t23b5jQO87y5b98+q8+uXbusmMu6q/2NWmFfTaTfMxA8lxwPl++hrsd2zbMz1zZyPAAAAAAgCth4AAAAAAgcGw8AAAAAgWPjAQAAACBwUc+EcimUVhgJWD/99JOvPWrUKKvP1KlTrVj58uWtWOXKlX1treCVllik/d3m8bXkI+34WsK5+ZyuhZC05GHzsZ9//rnV57LLLnM6flFlJpppc1lLkDQLTIrY75eWuK4VonRJWtP6uBQ90mgFLLXETe34JInHDpf1wbXolUtid36KE7rcLEGLaeuWNn8RPa7J+2byt4jIWWed5WvXrl3b6qOtNdp7umnTJl9bSxpPT093OpaZCJ+SkmL1Wb9+vRVD7Fq+fLkVM9cLbU1xvSGGuWa5FifU+pnfC7du3ep0rKKMXzwAAAAABI6NBwAAAIDAsfEAAAAAEDg2HgAAAAACF/Us70gTTLdv327FNmzY4GtrCUNmHxE9Edp8rJYArCUWaQnaZnXV1NRUq4+WxKYlD5tJctq4tIS71q1bW7Hdu3f72lOmTLH6aMmBWsVpM/l5xowZVp/izqUiuPZ6arFIE3tdxhVpQpzrc7pWu3dJPI1mlXXkzeW9dq2263r8aHE9dqSV0RE87dxTt25dX9s1+TsxMdGKmee6nTt3Wn20G8NoSejadwiTeZ4WEdm8ebMVq1atmq/tWukd0ZWZmWnFatWq5Wtrc0H7rqUxz3+ua5Z23jS/823cuNHqM336dCumfQcsKvgEAAAAAAgcGw8AAAAAgWPjAQAAACBwbDwAAAAABC7qyeU///yzrz1gwACrz5YtW6yYlhxmJmFpiVoVKlSwYlqCu5mgpiVxawlCWmVqM6ln5MiRVp8WLVpYMbNCqoidTJeVlWX10SxYsMCK7dmzx9c2k6lE9GR5LaFq7969EY0LerKiOU9dKz1HmhAeKe3YWpV1rd/hw4cDGRNOXH4qibswk9DzU7nXPJY2j7S/h/kWPPOcqyVGr1271ootWbLEitWpU8fX3rFjh9XHvHGLiEi9evWsmHl++v33360+FStWtGLaOdhFQkKCFRsxYoQV69evn69NInnh+P77762Yy01ZXG8GYK5jrjfl0I5vPlab76+//roVI7kcAAAAAI6DjQcAAACAwLHxAAAAABC4fOd4HDlyxHf9bd++fX3/rl3vXrKk/bTadXRaPoLp4MGDVkzLy9Bipl27dlmx1atXW7EHH3ww7LG1a/JSUlKsmJnjcfHFF1t9zMJLIiK//fabFTOvj9Wuzdeui9auOzTfI7Mw0skg0mJ3LkU0c3NzrZjLtaRazLUQnEs/bVxaPpR2fJdr7ikgWDC099qcl65zxKVQn+v7qvVzOb42Lm29TkpKchoH3LjkKHz77bdW7Mwzz7RiBw4c8LW190o739asWdOKLV261NfW1lwtx1HLjaxevbqvreWZaPki69evt2Lmefm0006z+iB4WsFj8zuNdr7KTyFAF9paZ34utPOtVkCwKOMXDwAAAACBY+MBAAAAIHBsPAAAAAAEjo0HAAAAgMDlO7l8xIgRvuRqMznMLBokYhf/ERHZvXu3FdOSvExagpCWdGgmmmkJa/v377diZuKZiEivXr187S+++MLqc9lll1mxVatWWTHztZgzZ47V54cffrBiWnKTmZSkJd5rycMaMxFLe5xZOEp7D09GWnKYmVSmJUO6Fi9yKbim3VhAS+w155HWR7sZhEYrAorCcejQIStmzq9oFv2LJm2+ac9nJmWicGgJ240bN7Zi5vzTzinaOUvjciMLl7VTxL7Bi1YQUUuEd0mOJ7m8cGgFj80bBORnXXM5b7oyPxfa99CNGzdaMe2zon33iEX84gEAAAAgcGw8AAAAAASOjQcAAACAwLHxAAAAABC4fCeXV61aVcqXLx9qm0ncWsKxlgBTu3ZtK2Y+VkuYzMnJsWKVKlWyYunp6WHHZSaZ5RUzE4O7d+9u9WnUqJEV0xKezAR67bWpUKGCFdOSh81xlS5d2uoTacVsLRFr+fLlvrZ204CTkUvlck2kyW7ajQZcE8LN47uOQZt/WlKcy7EQfS5VebV5Uxjvj8tc1eaba3I8oke7QUpKSooV0xL/ExISfG1tjmprp8u6os0h7bzmkrz+5+8zx2jJvdoNarZs2RL2+IiuHTt2WDHtfahWrZqvrc0Fbc5oN30x10mX71B5xcxxXHLJJVafTz75xIppNyJq3bq1FYtF/OIBAAAAIHBsPAAAAAAEjo0HAAAAgMCx8QAAAAAQuHwnl6empvqSxswkm7S0NOsxWhKylgxkJlVXrVrV6qPFtKQ1M4FH66MlxO3Zs8eKmUmZlStXtvosWbLEipnJdSJ2Ur1ZXTOvcWl/t5lgpyVkakl4LpUyk5OTrT7z5s0LO86TkZa06yLSxN78JNmaz+mSSCeiJ4Hu27cv4nEgurSbSJi099U1uTJIrjcz4GYWBU+r6q3NGe38as5J7XyhnZ+0m8qYtARj7Vja2myO9dRTT7X6/Pbbb07H2rVrl6+9fft2q4928xtE7tdff3XqZ84H7XuP6/pnzl1tvdXOkS5r27Jly6w+2lzLzMy0YiSXAwAAAMD/x8YDAAAAQODYeAAAAAAIXL5zPBo3bixJSUmhtllMb+jQodZjUlNTrVjdunWtmFm8T8u30K6t067dM68T1a5B1YoFav3M6/S0gkNaUSXt+kHzOkDt+bQCgi6FGbXHaTGt0KB53aFWOKp69eq+tkuhp6IkmsXUonmdvEtOh2ueiUsBQW3srtdPo3Bo66L5XmvvYWEU5TPnl3ZttJbjsXLlSit2zjnnRG9gsGjnJ2190M6JZg6YlruhnYu0+WCeS7XzoTa/tQK969ev97WbN29u9fnpp5+smHaON18fLfeEHI/oGjdunBWrUqWKFTPXEJd5JaJ/7zTXSe1zoT3uz9+VjzHnqVasUhvrwoULrVhRwS8eAAAAAALHxgMAAABA4Nh4AAAAAAgcGw8AAAAAgct3crnp4Ycf9rWbNGli9fn3v/9txbTkZbNInpYYrSWxacluZgFBl0JCInqypZmk6VIsSURPvjYf65rcqfUzXwst4U4raKQlVJkJTo0bN7b63HTTTb52Tk6O/M///I8+4CLIfI1dk821BMlIE+9dChppiWfaZ0A7lkn7G7W5pj2nS3J5NBP2kbcNGzaE7eNaLFKbN+Z77fq+usxLbb5picJaAimCtW3bNiumneu0AreLFi3ytbU1UStUqx3fnA+uN57RbiCzYMECX/vSSy+1+mjfPbTjm8nk2ncDRJd2kwntu4/5nUY7X2nFoLVk77Fjx/raXbt2tfqUK1fOimlFdrXC0i6PW7x4cdjHxSp+8QAAAAAQODYeAAAAAALHxgMAAABA4Nh4AAAAAAhcvpPLjx496ksYNBMFu3TpYj1Gi02aNMmKmYnqWVlZVp9du3ZZMS050Uwk0qqmulbyrVatmq+tJVbWqlXLimmJbWZiUX6qP5tJza6J9x06dLBiDRo08LVbt24d8bjglhDuWjXcjLkmkrvcuECby65V16lcHju0tcZc87T3WnsPXW4u4PreaxXIzce6Vg+uXbu203MierZs2WLFtPVBS9LduXOnr63NmdTUVCumJXFXrFjR146Pj3calwst2dd8PhH982OOIzs72+pz+umnRzQu6LTE7smTJ1sxcx3T1hktiVvjkhCufZ/U1j+Xx2nreaNGjcIeK1bxiwcAAACAwLHxAAAAABA4Nh4AAAAAAsfGAwAAAEDg8p1cXqJECaeKyOFcfPHFVmzGjBlhH7d06VIrpiXAmclh69ats/qkp6dbMa0Kdd26dcOOC0VfpFW2tQTJ3377zdfWEsi0z5EWM5MttT7a2LWYOQ7tpguuqFweO84991wrtnz5cl/bTPYV0ZMYNWaipjafI32vtYRcbY6TpFvw9u7da8W0m5iYFbw1Bw4csGLa+Var/m2e47VK6dpYte8GZkyrhO16sw5zzmsVtBFdffr0sWL/8z//Y8XM90u7AYJ20xeNy3feKlWqWDFtzTXnfE5OjtVHi/Xt2zfsGGIVv3gAAAAACBwbDwAAAACBY+MBAAAAIHD5zvEobGeccYZTzNSwYcMghgOo13GaBdC0XIpt27ZZMS1vwiyMlZ+8DPPafO35tGKY+/fvt2LatdEm12KHyB/tuvuePXv62j/88IPVZ+vWrVZMu1bevO7epTCWiD6/zDmYkZFh9dFyALW/EcEyc9VERE499VQrpuVvmLS1QCvgpuUdmQVtR4wYYfXRckP+8pe/hB2HNi5tTdfmX506dXztdu3aWX0QvAULFlixxo0bh31cmTJlnI6/efPmsH02btxoxbTPhbkmanlB3377rRXTcpKLCs74AAAAAALHxgMAAABA4Nh4AAAAAAgcGw8AAAAAgSvyyeVAUMyCQ64F0Zo2bWrFzjrrLF+7QoUKVh/XJHEz+TEhIcHqo41VK3hlJvZqid5a4rCWbKkVrTORSF4wtPfaTNLt3Lmz07G2b99uxczEyV27dll9tDlYo0aNsLFIixjm9ZyIntdee82KacUjtQTt6667ztfWbkahJcyuXbvWipkJ7c2bN7cH6+iqq64K2+eaa66J+PgoeI0aNbJi5noxZcoUq09mZqYVmzRpkhU7//zzw47hrrvusmJaUrr5uejSpUvYYxd1fAsAAAAAEDg2HgAAAAACx8YDAAAAQOAizvE4dr1cTk5O1AaDou3YXNCuvY62gph/keZ4HDx40Irl5uaG7RNpjod2jXU0czy0om/a+M3iXwW9NhTk/Pvz88TiGhjN/Aft7zMLYmpFBrXn04pjmYXYzM9KXmItx+NkmH9aUT7XHA9zbdHmgva3uPZD8TsHB0lbs7TCuNp52ZyT8fHxVh+X7wEihX/ejCbX+RfxxuPYC5+WlhbpIVBM7d69W5KTkwN/DhHmH2wFMf+OPY8IcxB+zD8UNs7Bwfr888+jdqyPP/44aseKFeHmX5wX4db46NGjsmHDBklMTOROIhCRP3a5u3fvltTU1MDvXsT8g6kg558IcxB+zD8UNs7BKEyu8y/ijQcAAAAAuCK5HAAAAEDg2HgAAAAACBwbDwAAAACBY+MBAAAAIHBsPAAAAAAEjo0HAAAAgMCx8QAAAAAQODYeAAAAAAJ3Um88Bg0aJE2aNMnz34cNGyYVKlTI13P07t1brrjiinwdA8Wfy1xjLiFIzEEAJyvWv4JTpDce06dPl1NOOUU6depU2EMpdG3btpV+/foV9jBOGnFxccf9X+/evaP+nC+99JIMGzbMaWxffPGF+m/Dhg2T8847T0REMjIy5MUXX4zeAFGgmIMo6nr37h2ar6VKlZLq1atLhw4d5N1335WjR48W9vAQw1j/iq6ShT2A/Hj33Xfl7rvvlrffflvWrFkjtWvXLuwh4SSRnZ0d+v8jR46UAQMGyLJly0KxcuXKRf05k5OTj/vvubm5Urp06eP2GTNmjHTr1i2aw0IhYQ6iOOjUqZMMHTpUjhw5Ips2bZLx48dL37595bPPPpMxY8ZIyZL215RDhw5JqVKlCmG0iBWsf0VXkf3FY+/evfLJJ5/I7bffLl27drV2oZMnT5a4uDj5/vvvpXnz5lK+fHlp3bq1b2KaVq1aJfXq1ZPbb789z//aMnbsWGnWrJmULVtW6tSpI4MHD5bDhw+HHe/gwYOlWrVqkpSUJLfddpvk5uaG/u3gwYNyzz33SLVq1aRs2bJywQUXyKxZs3yP//HHH+Xcc8+VMmXKSEpKijz44IOh5+3du7f8+OOP8tJLL4V2+1lZWWHHhMjVqFEj9L/k5GSJi4uzYqb58+dLu3btJDExUZKSkqRZs2Yye/ZsX59vv/1WGjRoIAkJCdKpUyff4mr+zNu2bVu56667pH///lKlShXp0KGDZGRkiIhI9+7dJS4uLtQWETlw4IBMmDBBLr/8cmnbtq2sXr1a/vGPf4TmzDGjRo2Ss846S8qUKSMZGRkyZMgQ3xgzMjLk8ccflx49ekhCQoKkpqbKK6+8ko9XE5FgDjIHi4MyZcpIjRo1pGbNmtK0aVN5+OGH5csvv5RvvvkmdF6Pi4uTN954Q7p16ybx8fHyxBNPiEj48/GgQYOkdu3aUqZMGUlNTZV77rkn9G+vvfaanHbaaVK2bFmpXr26XH311QX6dyN/WP+K8PrnFVHvvPOO17x5c8/zPG/s2LFeRkaGd/To0dC///DDD56IeC1btvQmT57sLV682Lvwwgu91q1bh/oMHDjQO/vssz3P87yFCxd6KSkp3oMPPhj696FDh3rJycmh9vjx472kpCRv2LBh3sqVK70JEyZ4GRkZ3qBBg/IcZ69evbyEhATvuuuu8xYtWuSNGzfOq1q1qvfwww+H+txzzz1eamqq9/XXX3uLFy/2evXq5VWsWNHbtm2b53met27dOq98+fLeHXfc4WVmZnqjR4/2qlSp4g0cONDzPM/buXOn16pVK69Pnz5edna2l52d7R0+fDji1xYnxpwneTnrrLO8m266ycvMzPSWL1/uffLJJ968efNCxyhVqpTXvn17b9asWd6cOXO8Bg0aeD169Ag9vlevXl63bt1C7TZt2ngJCQnefffd5y1dutTLzMz0Nm/e7ImIN3ToUC87O9vbvHlzqP+4ceO8unXrep7nedu2bfNq1arlPfbYY6E543meN3v2bK9EiRLeY4895i1btswbOnSoV65cOW/o0KGh46Snp3uJiYneU0895S1btsx7+eWXvVNOOcWbMGFCPl5F5AdzkDlYFJnz6c/OPvtsr3Pnzp7neZ6IeNWqVfPeeecdb+XKlV5WVlbY8/Gnn37qJSUleV9//bW3evVq75dffvH++9//ep7nebNmzfJOOeUUb8SIEV5WVpY3d+5c76WXXiqQvxnRx/pXtNa/IrvxaN26tffiiy96nud5hw4d8qpUqeJNnDgx9O/HNh7fffddKPbVV195IuLt37/f87z/23hMnz7dq1Spkvfcc8/5nsOczBdeeKH35JNP+vp88MEHXkpKSp7j7NWrl1epUiVv7969odjrr7/uJSQkeEeOHPH27NnjlSpVyhs+fHjo33Nzc73U1FTv2Wef9TzP8x5++GHv9NNP922sXn311dAxPO+PD0Dfvn2P+5ohGK6LXmJiojds2LA8jyEi3ooVK0KxV1991atevXqorS16TZo0sY4lIt7o0aOteJ8+fbz+/fuH2unp6d4LL7zg69OjRw+vQ4cOvth9993nnXnmmb7HderUydfnuuuuC31JQMFjDjIHi6LjbTyuu+46r0GDBp7n/TGf+vXr5/v3cOfjIUOGePXr1/dyc3OtY48aNcpLSkrycnJyovBXoLCx/hWt9a9IXmq1bNkymTlzplx//fUiIlKyZEm57rrr5N1337X6Nm7cOPT/U1JSRERk8+bNodiaNWukffv28sgjj8i999573OedM2eOPPbYY5KQkBD6X58+fSQ7O1v27duX5+POPvtsKV++fKjdqlUr2bNnj6xdu1ZWrlwphw4dkvPPPz/076VKlZJzzz1XMjMzRUQkMzNTWrVq5fsp7vzzz5c9e/bIunXrjjtmFI4/z5G///3vIiLSv39/ufXWW6V9+/by9NNPy8qVK32PKV++vNStWzfUTklJ8c1VTfPmzZ3G43mejB07Vi6//PLj9svMzPTNRZE/5tpvv/0mR44cCcVatWrl69OqVavQfEVsYA6iKPM8z3fOM+dZuPPxNddcI/v375c6depInz59ZPTo0aHLsDp06CDp6elSp04d+etf/yrDhw8/7jkcRQ/rX+wqksnl77zzjhw+fFhq1qwZinmeJ6VKlZIdO3ZIxYoVQ/E/J6AdW8T+nL9RtWpVSU1NlY8//lhuueUWSUpKyvN5jx49KoMHD5Yrr7zS+reyZcue8N8RFxcnnuf5xvbnv+dYzFyAj8W0xyE2zJs3L/T/j82pQYMGSY8ePeSrr76Sb775RgYOHCgff/yxdO/eXUTESpb88/zIS3x8vNN4Zs6cKbm5uXLBBRcct9/x5lo4zMXYwhxEUZaZmSmnnnpqqG3Os3Dn47S0NFm2bJlMnDhRvvvuO7njjjvkueeekx9//FESExNl7ty5MnnyZJkwYYIMGDBABg0aJLNmzcr3LfQRG1j/YleR+8Xj8OHD8v7778uQIUNk3rx5of/Nnz9f0tPTZfjw4Sd0vHLlysm4ceOkbNmy0rFjR9m9e3eefZs2bSrLli2TevXqWf8rUSLvl3L+/Pmyf//+UHvGjBmSkJAgtWrVknr16knp0qVl6tSpoX8/dOiQzJ49Wxo0aCAiImeeeaZMnz7dN/mmT58uiYmJoc1X6dKlfbthFK4/z41q1aqF4vXr15d//OMfMmHCBLnyyitl6NChUX/uUqVKWXPhyy+/lEsvvVROOeWUUEybM2eeeaZvLor8Mdfq16/ve+yMGTN8fWbMmCFnnHFGtP4ERAFzEEXVpEmTZOHChXLVVVfl2cflfFyuXDm5/PLL5eWXX5bJkyfLzz//LAsXLhSRP66UaN++vTz77LOyYMECycrKkkmTJhXI34fgsf7FriK38Rg3bpzs2LFDbrnlFmnYsKHvf1dffbW88847J3zM+Ph4+eqrr6RkyZLSuXNn2bNnj9pvwIAB8v7778ugQYNk8eLFkpmZKSNHjpRHHnnkuMfPzc2VW265RZYsWRLaZd91111SokQJiY+Pl9tvv13uu+8+GT9+vCxZskT69Okj+/btk1tuuUVERO644w5Zu3at3H333bJ06VL58ssvZeDAgdK/f//QApuRkSG//PKLZGVlydatW7kHeozZv3+/3HXXXTJ58mRZvXq1TJs2TWbNmhXaXEZTRkaGfP/997Jx40bZsWOHiOi38MvIyJCffvpJ1q9fL1u3bhURkX/+85/y/fffy+OPPy7Lly+X9957T/7zn/9YlyFOmzZNnn32WVm+fLm8+uqr8umnn0rfvn2j/rcgepiDiEUHDx6UjRs3yvr162Xu3Lny5JNPSrdu3aRr167Ss2fPPB8X7nw8bNgweeedd2TRokXy+++/ywcffCDlypWT9PR0GTdunLz88ssyb948Wb16tbz//vty9OhROf300wvqz0YBY/2LIQWcU5JvXbt29bp06aL+25w5czwR8ebMmRNKLt+xY0fo33/99VdPRLxVq1Z5nue/q5Xned7u3bu91q1bexdeeKG3Z88eNWFp/PjxXuvWrb1y5cp5SUlJ3rnnnhu6U4bmWDLSgAEDvMqVK3sJCQnerbfe6h04cCDUZ//+/d7dd9/tValSxStTpox3/vnnezNnzvQdZ/LkyV6LFi280qVLezVq1PAeeOAB79ChQ6F/X7ZsmXfeeed55cqV8/2NCJ5LYtvBgwe966+/3ktLS/NKly7tpaamenfddVfoRgfaMUaPHu39+SOqJbZpNxQYM2aMV69ePa9kyZJeenq6t2LFCq9MmTLe7t27ff1+/vlnr3Hjxl6ZMmV8z/PZZ595Z555pleqVCmvdu3a1k0X0tPTvcGDB3vXXnutV758ea969eqhGz2gcDAHmYNFUa9evTwR8UTEK1mypFe1alWvffv23rvvvhu6cYrn5Z2se7zz8ejRo72WLVt6SUlJXnx8vHfeeeeFbjYzZcoUr02bNl7FihW9cuXKeY0bN/ZGjhxZIH8zoo/1r2itf3Ge53jxGIAi6fnnn5fvvvtOvv7666gcLyMjQ/r16yf9+vWLyvFQ/DEHAZysWP/8itylVgBOTK1ateShhx4q7GHgJMYcBHCyYv3zK5J3tQLg7tprry3sIeAkxxwEcLJi/fPjUisAAAAAgYv4F4+jR4/Khg0bJDExscjcOxjB8jxPdu/eLampqce9vXA0MP9gKsj5J8IchB/zD4WNczAKk+v8i3jjsWHDBklLS4v04SjG1q5dK7Vq1Qr0OZh/yEtBzD8R5iB0zD8UNs7BKEzh5l/EG4/ExMTQExyv2rcr7YqvaO6it2zZ4mv/+OOPVp/33nvPiiUnJ1sx817fpUuXtvrs3LnTis2cOdOKtWjRwtceOHCg1adcuXJWzEXQr6kpJydH0tLSQnMjSNGefyj6CnL+iRTMHHS5Ejaan2mzcJWI+KpHH3OscOmJysrKsmK//vqrr32sinBRUxznH4oWzsEoTK7zL+KNx7GTXVJSUpHYeBw4cMDXLl++vNWnZEn75ShVqpQVK1OmzHHbecW045v9tNeyqGw8CuM5ojX/UHwU1M/+BTEHC3rjER8fb8W0k0ikf692LHMtLuqf5+I0/1A0cQ5GYQo3/7idLgAAAIDAsfEAAAAAELhCqeMR6SVAW7dutWIvvfSSFfvuu++smHmplXZJQW5urhWbNWuWFfv888+PO04R/RIt7broX375xddu3bq11adSpUpWrE2bNlbs7rvv9rUrVqwYdpwAYpe5VrreqWbdunVW7N133/W1hwwZYvXJyck5gdFFh/k3/fWvf7X6PPPMM1asb9++ET3f0aNHw44BABAMVlsAAAAAgWPjAQAAACBwbDwAAAAABK5QcjxcrVy50tfu2rWr1adGjRpWrEKFClbMzLk45ZRTrD7aLXCbN29uxfbs2RPRsbQcErO+yOHDh60+Bw8etGITJ060YtOmTfO1b7vtNqvPlVdeacUAFL5Icw/OOeccK/bbb79ZMXMd0W4prq2nZn6ciJ0/pq252dnZVmz//v1WzLxduPZ89957rxV78sknrdhf/vIXX3vEiBFWH+01Je8jdmk5oeb7pb1XrreULehbVk+fPt2Kabmdy5Yt87Xr168f6LjgrqDnTKRuuukmK9a/f38r1rRpUytmni+077SRYmUFAAAAEDg2HgAAAAACx8YDAAAAQODYeAAAAAAIXKEkl7sm3Tz00EO+dkpKitVHK5KnJWibz1mypP2nawlDZiK5iJ1k45pIvnfvXitmJr1r4ypbtqwV05Ihzed89dVXrT6XXHKJFUtISLBiAIKjrTUuycytWrWyYosWLbJi1atXt2Lm+qCtw9q6pa1JGzdu9LW1RHIzaVxEpHTp0lbMTCbX1jstpq3zH330ka+9b98+q88XX3xhxbTX3nyPYiFZFLr8vDfRfF8nT57say9cuNDqo9344eGHH7Zi5vybMGGC1SeaCb/FSaRFql0fZ8a0x0U6hkOHDlkxrSC1NreuvvpqX3v58uVWH+07rbYmBrne8YsHAAAAgMCx8QAAAAAQODYeAAAAAALHxgMAAABA4GKmcrmWnGgmMCYlJVl9tEQcLRnSTDLUEr2PHDlixbSq5GZMS0zUqu9qiY7mY7UkIm0MWkK4mYCp/Y1jxoyxYj169LBiAILjmrg3evRoX3vGjBlWn7S0NCum3XzCXCtdkibziplrsUt16bz6mWugtnZqY9DWytq1a/va3377rdXnm2++sWKdO3d2ek5ELtJkfa2fdk508f7771ux8847z9eeMmWK1efll1+2YqmpqVZs/vz5vrZWbVyrEv3iiy9asSZNmlgxuNHmTKTVxrXvhSZtrdNufqHdcMN8rLau/fTTT1ase/fuVsy8eccZZ5xh9dFuOqTRxhEt/OIBAAAAIHBsPAAAAAAEjo0HAAAAgMCx8QAAAAAQuJhJLt+xY4cVM5PLtYSygwcPWjEtidt8rFah16V6rYidDKQlJGmJRRqXCphasvyWLVusWJUqVXxt7W/87rvvrBjJ5UBwXG9aobnyyit9bfMzLiKye/duK1ahQgUrZiYLajfmcF3LzH4uVdfz4vJY17XZXPO016FLly5WTLu5SY0aNXxt7XXQ1mYUvMzMTCumvV9mZXERkdmzZ/va27dvt/r06tXLirVp08aKmYnj5rHziplJwSIiK1as8LXr1atn9YG7SG8W4bJWa31ck7PNtW3t2rVWH23NSkxMtGLmuWbIkCFWn5o1a1qxSKusR4pfPAAAAAAEjo0HAAAAgMCx8QAAAAAQuJi5QHXBggVWzLxG08z5ENELt2gxs7ieVvynbt26ViwjI8OKlS9f3tfWisLEx8dbMe2aPzNHZeHChVafsWPHWjHtOXfu3Olr79mzx+qjFRUEEBzXfI5u3bpZMTNHQSscmpWVFfZxIm7FSjUuBbSiScvncC0iZ6795lotYp8LRPRr/6+//vqwzwd3kV4zruVsTp8+3dc283FERJKTk63YzTffbMVeeOEFX1u7Br5///5WbPPmzVbM/Bu1Am5z5861YhMnTrRi5jwlxyN/zLUhP3lpmzZt8rW1vKBt27ZZsTlz5oQ9lpabVKlSJSumzfldu3b52s2bN7f6xAJ+8QAAAAAQODYeAAAAAALHxgMAAABA4Nh4AAAAAAhczCSXm4l8IiIXXnihrz18+HCrz6JFi6zYww8/bMW0JC8XWmLb/v37j9sW0ZO4Dxw4YMXMJHStmN9TTz1lxVq0aGHFzOR7LbHy999/t2IACt/PP/8cto9WMFXjkjipJfu6JgBrBaeixXVc2hjMv1srkqitw7NmzbJi5jkpyIJaJwPzJgWuNxHQbpJSpkwZX1v7HqDdMODNN9+0YuPHj/e1O3bsaPXRVKtWLWwfLQFdSxRev369FXv33Xd97fPPP9/q07Bhw7BjwB9c5t/KlSutWL9+/ayYeSMfrZjf4sWLrZh2U6MlS5b42m3btrX6aDc80M4F5ufCtZB1pMzX1PVGJPziAQAAACBwbDwAAAAABI6NBwAAAIDAsfEAAAAAELiYSS6///77rZiZ/NOuXTurzznnnGPFcnJyrJiZXK4lJiYlJVmxypUrWzGzKrBWAdg1GdKsNKklyWkVS7VEe7OqsTZ2M/kIBcMlGVebM1qylvm50B6nJZWVLBnZx92s+KqNIT+0BGBzrCdDYm+5cuWsWG5urq/t+h5q881cp1xedxG3hEGXKuJ5jcvlWBptjpvVnrUETPOGHiIiI0aMsGJDhgxxGgfcuKxbGu1zYc6jSZMmWX1uuukmK/bGG284PWe0aNWrte8nzZo1s2KlS5f2tbW5bB5/9+7dJzrEk4b2Pc1Ut25dKzZs2DArpn23ipaqVataMe2GGNqNBa677jpfW0tmd/lOofXT1m7zfOG6dvOLBwAAAIDAsfEAAAAAEDg2HgAAAAACx8YDAAAAQOBiJrlcqxb6/fff+9qjRo2y+kyYMMGK9erVy4q99tprvraZ1C0ismLFCiumVU01k+K0JEctcdNMFhOxk3q0hDitKubTTz9txczE8YoVK1p9Pv/8cys2ffp0K6ZVV0XkIk2O1hK6XI4VaSK5+TkREXniiSes2IYNGyI6vsYl6a+4mT9/vhXbsmWLFUtOTva1tSRDbV3R+pmJ11qSoWuSuNkvP9XGzX5aH20M2hw3H7tjxw6rj3aDjUg/L3AX6Rqonf8uuuii47bzsn//fitmfi5cx+kyl7Ozs60+2nlZu7FN586dwx5r9erVvrb2fQX5oyWSm+uRtpZGel7TbqKkfffV5tGPP/7oaz/wwANWH9cEcJd+kd7cgF88AAAAAASOjQcAAACAwLHxAAAAABC4mLmw9cEHH7Ri5nW3WjGUBg0aWLExY8ZYscceeyzsGLRr8rTrgV2ub9auGXbJBdm7d6/VxyxYKCLSsmVLK1ajRg1fW7tWUCtGSD5HwXPN3Yj02nOtINq8efOs2Keffuprm9c7i+gFjW644QYr9tFHH53ACP+PWSRPROTZZ5/1tR955JGIjh2rtLVAy2Mwaddwa8WftPllPqdrLoXWz7ymWRuD67FcriV2fZw5Lm1N18a6bt26sGNA7Ih0/mnMfq7XwLvQ8rbMQr8ibp9F7bNvnh+0dQX543Kuds3ncCns27NnT6uPeZ7Oa1xmnrKW06QV5NQsWbLE177zzjutPjVr1vS1tdxmDb94AAAAAAgcGw8AAAAAgWPjAQAAACBwbDwAAAAABC5mksu7d+9uxcwCgnPmzLH6mEV2REQuv/xyK7Z582Zfu3bt2lYfrQiMlixjJuxoj9NoicLly5f3tbUkJa0oi1k4SETkhRdeCNtn8uTJVuycc85xisGNSzKaa5Gq3377zYqZiWY///yz1UcrrFmnTh0rVqtWLV9bK9aVlZVlxb7++msrFqmPP/7Yiv3yyy9RO34smjt3rhXTkuxdiutpBQS1BELzxhWuCZHaXDWTeV36iOhrpUtBVtc11uynJVdqN0vQEn7NOajd0AOFwyUBXOujfS5c5lakhVy1m8W89957Vqxr165WrEePHr62NkfNv8f1cwJ3kRa+1GhrokmbC1qxwJ07d1oxsxCl+R1aRCQtLc2Kad+/TVoxVvMmNrt375ZPPvkk7LH4xQMAAABA4Nh4AAAAAAgcGw8AAAAAgWPjAQAAACBwMZNcnpmZacXMxGuzMreIyHnnnWfFpk2bZsUWLlzoa2sJQ66VTl0ShbVkNI1LBVbt7zYTz0REmjRp4mufeuqpVh8tsej0008PN8xiRXuftdfdTPbVkng1LsloWmLYww8/bMVGjhxpxeLj433tlJQUq8+5555rxbQbJezbt8/XPuOMM6w+69evt2KPPvqoFTOZN3QQ0f+e/v37W7GlS5f62tqNJZo1axZ2DLFKWx9cqnO7JoS7PKd2rAMHDoQdg4i9buVnDTRpxzp48KAVS05OtmJmdWctUV37u7Xjv/jii772Rx99ZPVB5InXscKc364J2i4J7pUrV7Zi2o1bZs+ebcVuu+02X3vlypVWn9atW/vaJJfnT6Rz2XU9j/RzoX1v0246tH37dl/7sssuczp+9erVrZi5TrZr187qY373ML+b5IVfPAAAAAAEjo0HAAAAgMCx8QAAAAAQODYeAAAAAAIXM8nlWuKUmSi1du1aq4+WeG0mpYvYiTJaFVAtGUirNu6SEO6abGkm92qJj1qSrvY3momVWlKwltS8ceNGK6ZVuS6KXJO+NK7J5CatWuioUaN8bbPip4hIpUqVrNhZZ51lxcw5uWvXLqtPTk6OFdOq9prJYFqSo/YZGz58uBV77rnnwj5fo0aNrJiW2GsmOWsV1Ysybf3RmGuNtj5o81Sb45EmnrredCNS5li1v0dbt7Q11rwhRIUKFaw+2t+jPaeWaA9bUUokd+GSNJ6XefPm+dpnn3221eeGG26wYuPGjbNi3377ra9tzm0RO+lYW/fhrqCrlLuaP3++FWvcuLEVy87O9rU//vhjq482RwYMGGDFzO+THTp0CDtOV/ziAQAAACBwbDwAAAAABI6NBwAAAIDAxUyOh3ZNctmyZX1tLd9Cu/bbzJsQsa+3067z1a6B1sZlPla7lk97nNbPPJZ2Hac21ipVqlgxk1lMRkQvqLVhwwYrVlxyPLRrNiO9hvfll1+2Yq+//roV27RpkxUzr8Vt2LCh1Ueb39qxTNrf6JpjZM7JqlWrWn1crxs2i1mNHj3a6XFPPPGEFXv11Vd97fT0dKvPhx9+6GtrBZVi1ZNPPmnFtPwNM6blw2ifc61wWaQF/YJmrrtavoX2mdVeC7NIppZLo50ftJy5L774wtcu6oXyoDPnn+v54ZlnnrFi5mfx73//u9Xngw8+sGLa57VLly6+dlZWltXH/KxEmpeIE2OuBdo6oH3X0uaW+VhtnSlTpowV0777RrrG/+tf/7Ji5vfOa665JqJja/jFAwAAAEDg2HgAAAAACBwbDwAAAACBY+MBAAAAIHAxk1yuJVCbiTJacnbFihWt2P79+62YS3K5a6Kg2c81kVdLHjUTJLWEJG2s1atXt2JmMr6WyKQdvygl5YYzd+5cX3vixIlWn2XLllkxrViYmXSvvU5agbJatWpZMbPIn5YYqxUC1JiJsNp76npzAzMZV+ujFQI055qIyC+//OJrp6SkWH327t1rxWrWrGnF6tev72trCcFvvfWWr629prHq999/t2JaAqH5N2k3n9AS77XXK1aTy02ua6f2eTTns7Y2u95YJCMjI+yxUPSZ50ktiXvQoEFWTFt3q1Wr5mubhWNFRE477TQrZs5bEfv8czImjptrgcv3xLyY57ZoFvhzeT4RtzWkefPmVqxdu3ZWzCww6Uo7h2jrn3lecbmhkSt+8QAAAAAQODYeAAAAAALHxgMAAABA4Nh4AAAAAAhczCSXa8zkLS0xp0aNGlZMS6x04ZqQa45LS3hyjZmJbVqSj0ZLRHVJxNKqY7s+Zyx68803fcnPn3/+ue/ftRsNaO+zlrhnJvzFx8c7HWvPnj1WzJxHWqVkLVFdS2A0PwdaYrw2Li352pwj2uulHV9LUEtOTva1tZsbaDeD0BKHzXEU5RsgrF+/3oppr7OWvGeuZdprpa1R2mfa7OdaiVt7H7X334U2VvP4rpV7tZsxmJ9j7SYI2lzS1sU1a9ZYseJMmzOuVbwLmjlWbc5oc1RbdzMzM33t++67z+pj3uxCRGTt2rVWbMiQIb626w0J5s2bZ8XMG1C0atXK6ViFzaWqt+vaY8ZidT5qXJPXr7zySl+7cePGVp+hQ4c6Hcs8n7t8fxXRb/pyzjnnOD1nJPjFAwAAAEDg2HgAAAAACBwbDwAAAACBY+MBAAAAIHAxk1weaVVYLVlVS54xaUk3WsKTlnRoJvC4JEXlxTy+luCnjUtLTjWTk10rOWvJw0XF9ddfL0lJSaF2ixYtfP8+bdo06zGLFi2yYqtXr7ZiZhLqjh07rD5axVmXObN582arz9atW62YS+KwlkSpjculwmtCQoIV05LqtWR8M/FP+wxoyb4uSaBacvGll17qa+/du1deeuklq19hmzJlilM/lyRuLblce023b99uxcz3zDWR3GUtC7qqt/b+a/PS/LxoN3rQzg/aa6jdnKM4c0ncda0SHfR8cLkpi5ZIrt3o4fnnn/e1L774YqvPL7/8YsU+/fTTsON0pb1e5t+k/T2xyPxbXBPJI7V06VIr9u6771ox86YBVatWdTq+tg6Y64z2HUpbUx555BErtmXLFl/bvEHOiXBJaNf6aH9j3bp1wx7LfG+dq8g79QIAAACAfGDjAQAAACBwbDwAAAAABC5mcjyiSbvezqVIlWvRP5Pr9You19Zp1zLv3LnTimk5HqeddpqvrRUl0q7Nd70uLxZ5nucbf8OGDX3/3rJlS6fjaPkwq1at8rVXrFhh9cnKyrJiGzZssGLmnHSdf9qcqVy5sq+dmJgYto+IXqDQLPqn9dGuLXa53ljL3XCda2YxPe16fvNzl5OT43TsgqblZWi0z745J7TXT1sftGvezdwj1/nmsi5qf6Pre22OVVtPXXNbzH5aXpbLawNd0LkbGpccAdfCcoMGDbJiqampvvaCBQusPiNHjnQ6fqS0z52Z86edu2PBoUOHfDmF5vul/W3a503Lf3j77bd9ba1gtMY8d4uIfPnll772smXLnI7lkg+srUVagUktL+jrr78OOwbt+96fCycf41JAUFsTtc/1BRdcEHZc5HgAAAAAiFlsPAAAAAAEjo0HAAAAgMCx8QAAAAAQuJjJqNMSZM3iT64FnbREHDPpUEtGcym+IuJWIEeLuRQ2dE3+1l6L2rVr+9qzZ8+2+mgJrFqyZVFRoUIFXwHBvXv3+v49OzvbeoxrAlSlSpV87bZt21p9tBsZuCQTa6+5luClvc/mc2rHci0qaB5LK7hmFjgSsYsrasfXXgftM7Bv3z4rZq4HWjJienq6r62NPRa0adPGqZ/2/ptrkktxShH9tTc/+9rjtDFo75kZ0xIdtfmmrbvm/NWeT/t7tHlvvl6uY4BbErd2I4NNmzZZMW3d1dZPF5EmtA8cONCKaZ8fM5l89OjRET2fiNs5XhuDNpe1grKxqFSpUs430DieuXPnWjFzbrmeI6tVq2bFzKK9Y8eOtfpcdtllYceZ1zhMN9xwgxXr1KmTFXMp1Ketr5HauHGjFdNu3tK6deuoPaeJXzwAAAAABI6NBwAAAIDAsfEAAAAAEDg2HgAAAAACVyjJ5Vriq5asYya7/TmJ+Hi0hEKXyrTaGFySEyOt0KsdS0twd034zMjI8LW1sWvH1/oVVWaSlJY05cq8SYFroqqW5GxWRnd9zbU5YybTuSbLuiS0azd5qFmzphVzuXlCpMnFWj/tfTQrDsdq5fKvvvrKqZ92YwkzpiX6V69e3elY5nvmuj5o71mkieou89l1vdOqBZvHcplbecVONi4Js0uWLLFiWoVm7Vxt3kSifPnyJzC641u/fr0Vmz59uhXTbgYyZcqUqI3DfA0jvWGNiMiaNWuiMqagTZs2zbc+m+O++uqrrcdon13thgSm5ORkK1axYkUrpiVjm+eQvn37Wn1ck8tN3bp1s2KLFy+2Ymb19MKwa9cuKxbpZ5HK5QAAAABiFhsPAAAAAIFj4wEAAAAgcGw8AAAAAASuUJLLtUQql8RuLclV45JE65r05VKVXOujHV+LuSRWaonxWuXo0047zdd2TR51TQg62ZgJaq7VQ7VkN5y8xo8f79RP+5ybSdza5/7111+3YjfeeKMVM9eDhIQEq4+2PmiJ6mY/lzU3L+axtARgLaYlSZpV4levXm31qVChgtO4TFqFbi2xv6B5nudbwyOt9O1SuTzIasb50adPHyu2fPlyKzZu3LhAxxHpjWe0z93SpUujMqagZWVl+c6Nt912m+/fH330Uesx2tqj3SDA7KdVSNdubqAdy3yNtRti3H///Vbs1ltvtWIPPPCAr/3DDz9Yfdq3b2/FKleubMUKmpbEr91UxoW5Pjiv+RE9GwAAAACcADYeAAAAAALHxgMAAABA4Aolx0OjXRtmXoNnFgzLi3a9sXl9n5b/4FLcSjuWxuV6WZHIrwnVrm8+66yzfG1t7FqMHA8gOGbxSBH9mlqzwJqI21rTvXt3K3bPPfdYsREjRvjaWr7I9u3brVhKSooV0/4mk1aUT1sDzeu2tQKc2rFatmxpxcyiYD/++KPTGFwKCI4ZM8aKabkFBS0uLi7ivA7zOOFo54ouXbpYMe0a+wcffNDX7tGjxwmMzu+xxx7ztbU8qn79+lmxRo0aRfycQdK+e+zYsaMQRnLibrzxRl/ByP/+97++f9eKTmp/m7bW1ahRw9fW1oadO3dasSpVqlgxM09Mm8vPPfecU6xq1aq+tpb/OXjwYCumMb+TueYfR0p7vSLNezPH6jp2fvEAAAAAEDg2HgAAAAACx8YDAAAAQODYeAAAAAAIXEwnl5vJP+np6U7HMotuidjJQFpyp0uCoYhd6Ms1iVtj/o1a0qZWPEtLsnIpsKj9jYcPHw77OACR0dY2LbE70gQ/zdNPP+0Uc6GtP+b4XW+mocXMAoV/TlQNgjZW7WYjZcuW9bXHjh1r9YmF5PIpU6ZIfHx8qG2+ntq5rlKlSlbsz8c4xjyXmq9JXrEVK1ZYsSFDhvjaWoG1atWqWbEJEyZYsZdeesnXbtu2rdUn0vkeTa5J/9r3Be17TFGQkZHha8+YMcPqU7t2bSuWm5trxcyindrrpBUj1L5HubwXWvFfl/fBTIIXcb+RQTRuDHGM+XdrSe/ajYlcCqFq5wHts++CXzwAAAAABI6NBwAAAIDAsfEAAAAAEDg2HgAAAAACVyjJ5Vpyn0v1bC1JTuOSoG1WyxUR2bZtmxUzE8lFIq82rjGTpbTEyr1791qx7OxsK2Ym+mivg5ZIriV1AYiOd955x4p9/vnnVkz7nBd0VVuNa0JxLDITXUVEtmzZYsW0xH7znHH++edHa1hRtWbNGl8SaVZWlu/fN2/ebD1Gu7mBdk40k221m5OkpaVZsZtuusmKNW7c2Nf+7rvvrD7Tp0+3YgsXLrRiF1xwga9tJq6L2En2Ivo5MRaSuLUk4I4dOxbCSPLvoYce8rU/+ugjq8/atWutmPY9yvzOp30/0t4/LWHb/O6j3VBCG4OW0G5+fkaMGGH10WjHiuaa7vJdVEsSd0kud71hkgt+8QAAAAAQODYeAAAAAALHxgMAAABA4Nh4AAAAAAhcoSSXHzlyxIppiWCRJnFfffXVViwnJ8fXNiuZ5zUul2rm2uNcE+jNxCItmT05OdmKNW/ePOy4tGRB7e/Rxg8gOrTE5dWrV1ux1q1bWzFz3erRo0fUxqXREghdYq7Vd136acmWWsylWnqnTp2sPm+//bYV27NnjxW79NJLfe0HHnjAHmwMuPHGG6NS7V27ucq6det87e3bt4ftI6K/N+ac1xLJzfkuItKlSxcrZn4OtAR3TSwkkmu05PLnn3/e13700UcLajj5Ylbs1ubC+PHjrdiAAQOs2KxZs3xtbX4UhgsvvNDXbteuXSGNxM8lUV373KWmpoZ9XDQrrPOLBwAAAIDAsfEAAAAAEDg2HgAAAAACVyg5Hvv377diLtcR79y50+n4ZgGbk5V2TZ72Oru+rgCio3bt2lZMK+RpFqrSrqfXaMUI4+Pjwz7ONb8iFmi5aWaOXJMmTcL2EdFzPO66667IB1cEVa5c2SmG6NMKXRbn+aflXmkx0/Lly63YnDlzrNiCBQus2Pr1631tLV9J+85Us2ZNK/bGG28cd5wibjm90eaSw3T//fdbsdNPPz3s47Q87EjF5hkFAAAAQLHCxgMAAABA4Nh4AAAAAAgcGw8AAAAAgSuU5PJKlSpZsfr161sxsyhQy5YtnY7vUmgwmsVQYpVWbGzVqlVWrFmzZgUxHAD/n7ZGPffcc1bMXCtTUlKcjh+rhdKiyWUN1wrFasXatNcrVpPqcXJ4/PHHC3sIMUf7nqjFbrjhhoIYznEVxndMl+ds3759RMd2KabtipUVAAAAQODYeAAAAAAIHBsPAAAAAIGLOMfj2DXKOTk5URnIwYMHrZhZUGvfvn1WH+35yfH4g/aaHjp0yIq5vq7hHHuMy+ufX9Gefyj6CnL+/fl5IpmD2hi1wqrmZ1MrdKc9/+HDh62YVjivKNMKCJrXIWtrm/baa4VVzSKM4d7nojT/UDxxDkZhcp1/cV6EM3TdunVW8jcgIrJ27VqpVatWoM/B/ENeCmL+iTAHoWP+obBxDkZhCjf/It54HD16VDZs2CCJiYknxa8HCM/zPNm9e7ekpqYGfkcY5h9MBTn/RJiD8GP+obBxDkZhcp1/EW88AAAAAMAVyeUAAAAAAsfGAwAAAEDg2HgAAAAACBwbDwAAAACBY+MBAAAAIHBsPAAAAAAEjo0HAAAAgMCx8TiOtm3bSr9+/ULtjIwMefHFFwttPMCJysrKkri4OJk3b15hDwVFDOsfiovevXvLFVdc4dyfdRPRxhz8P8V649G7d2+Ji4uTuLg4KVWqlNSpU0fuvfde2bt3b2EPDSeBzZs3y2233Sa1a9eWMmXKSI0aNaRjx47y888/F/bQcBJg/UOsYU1EYWMOFr6ShT2AoHXq1EmGDh0qhw4dkilTpsitt94qe/fulddff72whxaR3NxcKV26dGEPAw6uuuoqOXTokLz33ntSp04d2bRpk3z//feyffv2wh5avhw6dEhKlSpV2MOAA9Y/xJLiuiai6GAOFr5i/YuHiIR2tGlpadKjRw+58cYb5YsvvlB/9urXr5+0bdvW+dhr1qyRbt26SUJCgiQlJcm1114rmzZtEhGRZcuWSVxcnCxdutT3mOeff14yMjLE8zwREVmyZIl06dJFEhISpHr16vLXv/5Vtm7dGurftm1bueuuu6R///5SpUoV6dChQ2QvBArUzp07ZerUqfLMM89Iu3btJD09Xc4991x56KGH5NJLLxURkbi4OHn77bele/fuUr58eTnttNNkzJgxvuOEmx/jx4+XCy64QCpUqCCVK1eWrl27ysqVK/Mc19GjR6VPnz5Sv359Wb16tYiIjB07Vpo1ayZly5aVOnXqyODBg+Xw4cOhx8TFxckbb7wh3bp1k/j4eHniiSei+VIhQKx/iBUua+Lzzz8vjRo1kvj4eElLS5M77rhD9uzZEzrGsGHDpEKFCvLtt99KgwYNJCEhQTp16iTZ2dmhPkeOHJH+/fuH1sT7778/NN+OOdF1E8UDczA2FPuNh6lcuXJy6NChfB/H8zy54oorZPv27fLjjz/KxIkTZeXKlXLdddeJiMjpp58uzZo1k+HDh/seN2LECOnRo4fExcVJdna2tGnTRpo0aSKzZ8+W8ePHy6ZNm+Taa6/1Pea9996TkiVLyrRp0+TNN9/M99gRvISEBElISJAvvvhCDh48mGe/wYMHy7XXXisLFiyQLl26yI033hj6Ly8u82Pv3r3Sv39/mTVrlnz//fdSokQJ6d69uxw9etR6rtzcXLn22mtl9uzZMnXqVElPT5dvv/1WbrrpJrnnnntkyZIl8uabb8qwYcPkX//6l++xAwcOlG7dusnChQvl5ptvjtKrhILG+ofC4rImlihRQl5++WVZtGiRvPfeezJp0iS5//77fX327dsn//73v+WDDz6Qn376SdasWSP33ntv6N+HDBki7777rrzzzjsydepU2b59u4wePdp3jBNZN1F8MAdjhFeM9erVy+vWrVuo/csvv3iVK1f2rr32WuvfPM/z+vbt67Vp0ybUbtOmjde3b99QOz093XvhhRc8z/O8CRMmeKeccoq3Zs2a0L8vXrzYExFv5syZnud53vPPP+/VqVMn9O/Lli3zRMRbvHix53me9+ijj3qXXHKJbwxr1671RMRbtmxZaAxNmjSJ9CVAIfrss8+8ihUremXLlvVat27tPfTQQ978+fND/y4i3iOPPBJq79mzx4uLi/O++eYbz/Pc5odp8+bNnoh4Cxcu9DzP81atWuWJiDdlyhSvffv23vnnn+/t3Lkz1P/CCy/0nnzySd8xPvjgAy8lJcU3zn79+kX4KqCwsP4h1oRbE02ffPKJV7ly5VB76NChnoh4K1asCMVeffVVr3r16qF2SkqK9/TTT4fahw4d8mrVqmXN9z/La9389ddfI/grEcuYg4Wv2P/iMW7cOElISJCyZctKq1at5KKLLpJXXnkl38fNzMyUtLQ0SUtLC8XOPPNMqVChgmRmZoqIyPXXXy+rV6+WGTNmiIjI8OHDpUmTJnLmmWeKiMicOXPkhx9+CO3CExIS5IwzzhAR8f3k1rx583yPFwXvqquukg0bNsiYMWOkY8eOMnnyZGnatKkMGzYs1Kdx48ah/x8fHy+JiYmyefNmEXGbHytXrpQePXpInTp1JCkpSU499VQR+eMymD+74YYbZM+ePTJhwgRJTk4OxefMmSOPPfaY7zn69Okj2dnZsm/fvlA/5mDRxPqHWBJuTfzhhx+kQ4cOUrNmTUlMTJSePXvKtm3bfDdEKF++vNStWzfUTklJCa2Zu3btkuzsbGnVqlXo30uWLGnNIdd1E8UPc7DwFfvk8nbt2snrr78upUqVktTU1FBSbIkSJaxr7k7kEgTP8yQuLu648ZSUFGnXrp2MGDFCzjvvPPnoo4/ktttuC/U9evSoXHbZZfLMM89Yx0lJSQn9//j4eOdxIbaULVtWOnToIB06dJABAwbIrbfeKgMHDpTevXuLiFhJ2nFxcaGfWl3mx2WXXSZpaWny1ltvSWpqqhw9elQaNmwoubm5vv5dunSRDz/8UGbMmCEXX3xxKH706FEZPHiwXHnllerYj2EOFk2sf4g1ea2J7dq1ky5dusjf//53efzxx6VSpUoydepUueWWW3xzU1szzbkcjuu6ieKJOVi4iv3GIz4+XurVq2fFq1atKosWLfLF5s2b53y3njPPPFPWrFkja9euDf1XvyVLlsiuXbukQYMGoX433nijPPDAA3LDDTfIypUr5frrrw/9W9OmTWXUqFGSkZEhJUsW+7cC8se8+eKLL5z6hpsf27Ztk8zMTHnzzTflwgsvFBGRqVOnqse6/fbbpWHDhnL55ZfLV199JW3atAk9x7Jly9TPCIo+1j/EumNr4uzZs+Xw4cMyZMgQKVHij4sxPvnkkxM6VnJysqSkpMiMGTPkoosuEhGRw4cPy5w5c6Rp06YicmLrJk4OzMGCVewvtcrLxRdfLLNnz5b3339ffvvtNxk4cKB1Ij6e9u3bS+PGjeXGG2+UuXPnysyZM6Vnz57Spk0b309qV155peTk5Mjtt98u7dq1k5o1a4b+7c4775Tt27fLDTfcIDNnzpTff/9dJkyYIDfffLMcOXIkqn8vCta2bdvk4osvlg8//FAWLFggq1atkk8//VSeffZZ6datm9Mxws2PihUrSuXKleW///2vrFixQiZNmiT9+/fP83h33323PPHEE9K1a9fQIjdgwAB5//33ZdCgQbJ48WLJzMyUkSNHyiOPPBKV1wGxifUPBS3cmli3bl05fPiwvPLKK/L777/LBx98IG+88cYJP0/fvn3l6aefltGjR8vSpUvljjvukJ07d4b+/UTXTRQfzMHYcNJuPDp27CiPPvqo3H///dKiRQvZvXu39OzZ0/nxcXFx8sUXX0jFihXloosukvbt20udOnVk5MiRvn5JSUly2WWXyfz58+XGG2/0/VtqaqpMmzZNjhw5Ih07dpSGDRtK3759JTk5ObTbRtGUkJAgLVu2lBdeeEEuuugiadiwoTz66KPSp08f+c9//uN0jHDzo0SJEvLxxx/LnDlzpGHDhvKPf/xDnnvuueMes1+/fjJ48GDp0qWLTJ8+XTp27Cjjxo2TiRMnSosWLeS8886T559/XtLT06PxMiBGsf6hoIVbE5s0aSLPP/+8PPPMM9KwYUMZPny4PPXUUyf8PP/85z+lZ8+e0rt3b2nVqpUkJiZK9+7dQ/8eybqJ4oE5GBvivBO9MA0AAAAAThD/WQkAAABA4Nh4AAAAAAgcGw8AAAAAgWPjAQAAACBwbDwAAAAABI6NBwAAAIDAsfEAAAAAEDg2HgAAAAACx8YDAAAAQODYeAAAAAAIHBsPAAAAAIFj4wEAAAAgcP8Pp6Fm7Wd0RDEAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1000x1000 with 10 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot the first 10 images in the training set and their labels\n",
    "plt.figure(figsize=(10, 10))\n",
    "for i in range(10):\n",
    "    plt.subplot(5, 5, i + 1)\n",
    "    plt.xticks([])\n",
    "    plt.yticks([])\n",
    "    plt.grid(False)\n",
    "    plt.imshow(x_train[i], cmap=plt.cm.binary)\n",
    "    plt.xlabel(index_labels[y_train[i]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# normalize the data\n",
    "x_train = x_train / 255.0\n",
    "x_test = x_test / 255.0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# one-hot encode the labels\n",
    "y_train = tf.one_hot(y_train, depth=10)\n",
    "y_test = tf.one_hot(y_test, depth=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " conv2d_1 (Conv2D)           (None, 28, 28, 32)        320       \n",
      "                                                                 \n",
      " max_pooling2d_1 (MaxPooling  (None, 14, 14, 32)       0         \n",
      " 2D)                                                             \n",
      "                                                                 \n",
      " conv2d_2 (Conv2D)           (None, 14, 14, 64)        18496     \n",
      "                                                                 \n",
      " max_pooling2d_2 (MaxPooling  (None, 7, 7, 64)         0         \n",
      " 2D)                                                             \n",
      "                                                                 \n",
      " dropout (Dropout)           (None, 7, 7, 64)          0         \n",
      "                                                                 \n",
      " conv2d_3 (Conv2D)           (None, 7, 7, 128)         73856     \n",
      "                                                                 \n",
      " max_pooling2d_3 (MaxPooling  (None, 3, 3, 128)        0         \n",
      " 2D)                                                             \n",
      "                                                                 \n",
      " dropout_1 (Dropout)         (None, 3, 3, 128)         0         \n",
      "                                                                 \n",
      " flatten_1 (Flatten)         (None, 1152)              0         \n",
      "                                                                 \n",
      " dense_2 (Dense)             (None, 512)               590336    \n",
      "                                                                 \n",
      " dropout_2 (Dropout)         (None, 512)               0         \n",
      "                                                                 \n",
      " dense_3 (Dense)             (None, 10)                5130      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 688,138\n",
      "Trainable params: 688,138\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# rewrite the model in the previous cell in sequential API\n",
    "model = tf.keras.models.Sequential(\n",
    "    [\n",
    "        tf.keras.layers.Conv2D(\n",
    "            32,\n",
    "            3,\n",
    "            padding=\"same\",\n",
    "            activation=\"relu\",\n",
    "            kernel_initializer=\"he_normal\",\n",
    "            input_shape=(28, 28, 1),\n",
    "        ),\n",
    "        tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),\n",
    "        tf.keras.layers.Conv2D(64, 3, padding=\"same\", activation=\"relu\"),\n",
    "        tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),\n",
    "        tf.keras.layers.Dropout(0.3),\n",
    "        tf.keras.layers.Conv2D(128, 3, padding=\"same\", activation=\"relu\"),\n",
    "        tf.keras.layers.MaxPooling2D(pool_size=(2, 2)),\n",
    "        tf.keras.layers.Dropout(0.4),\n",
    "        tf.keras.layers.Flatten(),\n",
    "        tf.keras.layers.Dense(512, activation=\"relu\"),\n",
    "        tf.keras.layers.Dropout(0.25),\n",
    "        tf.keras.layers.Dense(10, activation=\"softmax\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "# print a summary of the model\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# compile the model\n",
    "model.compile(\n",
    "    optimizer=tf.keras.optimizers.Adam(),\n",
    "    loss=tf.keras.losses.CategoricalCrossentropy(),\n",
    "    metrics=[tf.keras.metrics.CategoricalAccuracy()],\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/20\n",
      "938/938 [==============================] - 22s 23ms/step - loss: 0.5125 - categorical_accuracy: 0.8126 - val_loss: 0.3362 - val_categorical_accuracy: 0.8799\n",
      "Epoch 2/20\n",
      "938/938 [==============================] - 20s 22ms/step - loss: 0.3384 - categorical_accuracy: 0.8769 - val_loss: 0.2897 - val_categorical_accuracy: 0.8944\n",
      "Epoch 3/20\n",
      "938/938 [==============================] - 21s 22ms/step - loss: 0.2952 - categorical_accuracy: 0.8911 - val_loss: 0.2621 - val_categorical_accuracy: 0.9065\n",
      "Epoch 4/20\n",
      "938/938 [==============================] - 21s 22ms/step - loss: 0.2701 - categorical_accuracy: 0.9006 - val_loss: 0.2491 - val_categorical_accuracy: 0.9045\n",
      "Epoch 5/20\n",
      "938/938 [==============================] - 22s 23ms/step - loss: 0.2541 - categorical_accuracy: 0.9059 - val_loss: 0.2416 - val_categorical_accuracy: 0.9139\n",
      "Epoch 6/20\n",
      "938/938 [==============================] - 22s 24ms/step - loss: 0.2404 - categorical_accuracy: 0.9117 - val_loss: 0.2441 - val_categorical_accuracy: 0.9112\n",
      "Epoch 7/20\n",
      "938/938 [==============================] - 22s 24ms/step - loss: 0.2291 - categorical_accuracy: 0.9153 - val_loss: 0.2316 - val_categorical_accuracy: 0.9140\n",
      "Epoch 8/20\n",
      "938/938 [==============================] - 23s 24ms/step - loss: 0.2210 - categorical_accuracy: 0.9176 - val_loss: 0.2152 - val_categorical_accuracy: 0.9208\n",
      "Epoch 9/20\n",
      "938/938 [==============================] - 23s 24ms/step - loss: 0.2123 - categorical_accuracy: 0.9206 - val_loss: 0.2399 - val_categorical_accuracy: 0.9117\n",
      "Epoch 10/20\n",
      "938/938 [==============================] - 23s 24ms/step - loss: 0.2057 - categorical_accuracy: 0.9233 - val_loss: 0.2150 - val_categorical_accuracy: 0.9204\n",
      "Epoch 11/20\n",
      "938/938 [==============================] - 23s 24ms/step - loss: 0.2010 - categorical_accuracy: 0.9254 - val_loss: 0.2195 - val_categorical_accuracy: 0.9206\n",
      "Epoch 12/20\n",
      "938/938 [==============================] - 23s 25ms/step - loss: 0.1957 - categorical_accuracy: 0.9270 - val_loss: 0.2159 - val_categorical_accuracy: 0.9237\n",
      "Epoch 13/20\n",
      "938/938 [==============================] - 24s 25ms/step - loss: 0.1898 - categorical_accuracy: 0.9287 - val_loss: 0.2092 - val_categorical_accuracy: 0.9280\n",
      "Epoch 14/20\n",
      "938/938 [==============================] - 24s 25ms/step - loss: 0.1840 - categorical_accuracy: 0.9308 - val_loss: 0.2094 - val_categorical_accuracy: 0.9261\n",
      "Epoch 15/20\n",
      "938/938 [==============================] - 24s 25ms/step - loss: 0.1826 - categorical_accuracy: 0.9319 - val_loss: 0.2098 - val_categorical_accuracy: 0.9243\n",
      "Epoch 16/20\n",
      "938/938 [==============================] - 24s 26ms/step - loss: 0.1789 - categorical_accuracy: 0.9335 - val_loss: 0.2086 - val_categorical_accuracy: 0.9260\n",
      "Epoch 17/20\n",
      "938/938 [==============================] - 23s 25ms/step - loss: 0.1743 - categorical_accuracy: 0.9350 - val_loss: 0.2078 - val_categorical_accuracy: 0.9252\n",
      "Epoch 18/20\n",
      "938/938 [==============================] - 23s 25ms/step - loss: 0.1713 - categorical_accuracy: 0.9348 - val_loss: 0.2113 - val_categorical_accuracy: 0.9265\n",
      "Epoch 19/20\n",
      "938/938 [==============================] - 22s 23ms/step - loss: 0.1693 - categorical_accuracy: 0.9358 - val_loss: 0.2037 - val_categorical_accuracy: 0.9270\n",
      "Epoch 20/20\n",
      "938/938 [==============================] - 23s 25ms/step - loss: 0.1662 - categorical_accuracy: 0.9371 - val_loss: 0.2066 - val_categorical_accuracy: 0.9255\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x2cb703b80>"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create an early stopping callback\n",
    "early_stopping = tf.keras.callbacks.EarlyStopping(monitor=\"val_loss\", patience=3)\n",
    "\n",
    "# train the model\n",
    "model.fit(\n",
    "    x_train,\n",
    "    y_train,\n",
    "    epochs=20,\n",
    "    batch_size=64,\n",
    "    validation_data=(x_test, y_test),\n",
    "    callbacks=[early_stopping],\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6 (default, Oct 18 2022, 12:41:40) \n[Clang 14.0.0 (clang-1400.0.29.202)]"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}