apphub/anomaly_detection/alocc/alocc.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Anomaly Detection with Fastestimator\n",
    "\n",
    "[[Paper](https://arxiv.org/pdf/1802.09088.pdf)] [[Notebook](https://github.com/fastestimator/fastestimator/blob/master/apphub/anomaly_detection/alocc/alocc.ipynb)] [[TF Implementation](https://github.com/fastestimator/fastestimator/blob/master/apphub/anomaly_detection/alocc/alocc_tf.py)] [[Torch Implementation](https://github.com/fastestimator/fastestimator/blob/master/apphub/anomaly_detection/alocc/alocc_torch.py)]\n",
    "\n",
    "In this notebook we will demonstrate how to do anomaly detection using one class classifier as described in <b>Adversarially Learned One-Class Classifier for Novelty Detection</b>. In real world, outliers or novelty class is often absent from the training dataset. Such problems can be efficiently modeled using one class classifiers.\n",
    "In the algorithm demonstrated below, two networks are trained to compete with each other where one network acts as a novelty detector and other enhances the inliers and distorts the outliers. We use images of digit \"1\" from MNIST dataset for training and images of other digits as outliers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tempfile\n",
    "\n",
    "import fastestimator as fe\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from fastestimator.backend import binary_crossentropy\n",
    "from fastestimator.op.numpyop import LambdaOp\n",
    "from fastestimator.op.numpyop.univariate import ExpandDims, Normalize\n",
    "from fastestimator.op.tensorop import TensorOp\n",
    "from fastestimator.op.tensorop.model import ModelOp, UpdateOp\n",
    "from fastestimator.trace import Trace\n",
    "from fastestimator.trace.io import BestModelSaver\n",
    "from fastestimator.util import ImageDisplay, GridDisplay, to_number\n",
    "from sklearn.metrics import auc, f1_score, roc_curve\n",
    "from tensorflow.keras import layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "# Parameters\n",
    "epochs=20\n",
    "batch_size=128\n",
    "train_steps_per_epoch=None\n",
    "save_dir=tempfile.mkdtemp()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Building Components\n",
    "\n",
    "### Downloading the data\n",
    "\n",
    "First, we will download training images using tensorflow API. We will use images of digit `1` for training and test images of `1` as inliers and images of other digits as outliers. Outliers comprise 50% of our validation dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_eval, y_eval) = tf.keras.datasets.mnist.load_data()\n",
    "\n",
    "# Create Training Dataset\n",
    "x_train, y_train = x_train[np.where((y_train == 1))], np.zeros(y_train[np.where((y_train == 1))].shape)\n",
    "train_data = fe.dataset.NumpyDataset({\"x\": x_train, \"y\": y_train})\n",
    "\n",
    "# Create Validation Dataset\n",
    "x_eval0, y_eval0 = x_eval[np.where((y_eval == 1))], np.ones(y_eval[np.where((y_eval == 1))].shape)\n",
    "x_eval1, y_eval1 = x_eval[np.where((y_eval != 1))], y_eval[np.where((y_eval != 1))]\n",
    "\n",
    "# Ensuring outliers comprise 50% of the dataset\n",
    "index = np.random.choice(x_eval1.shape[0], int(x_eval0.shape[0]), replace=False)\n",
    "x_eval1, y_eval1 = x_eval1[index], np.zeros(y_eval1[index].shape)\n",
    "\n",
    "x_eval, y_eval = np.concatenate([x_eval0, x_eval1]), np.concatenate([y_eval0, y_eval1])\n",
    "eval_data = fe.dataset.NumpyDataset({\"x\": x_eval, \"y\": y_eval})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Create `Pipeline`\n",
    "\n",
    "We will use the `LambdaOp` to add noise to the images during training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline = fe.Pipeline(\n",
    "    train_data=train_data,\n",
    "    eval_data=eval_data,\n",
    "    batch_size=batch_size,\n",
    "    ops=[\n",
    "        ExpandDims(inputs=\"x\", outputs=\"x\"),\n",
    "        Normalize(inputs=\"x\", outputs=\"x\", mean=1.0, std=1.0, max_pixel_value=127.5),\n",
    "        LambdaOp(fn=lambda x: x + np.random.normal(loc=0.0, scale=0.155, size=(28, 28, 1)),\n",
    "                 inputs=\"x\",\n",
    "                 outputs=\"x_w_noise\",\n",
    "                 mode=\"train\")\n",
    "    ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can visualize sample images from our `Pipeline` using the 'get_results' method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAFsCAYAAADSVYN+AAAgAElEQVR4Xu2deXxcdbn/v1naNEuTJk1KW9oGaktVblFAZROFFlA2K1UWBVRAf60gYKFSKCA7SNlp4RZZRSqbWCgoylYRkJcichFvZWsLdEm3NGmbNGmapL/XM3MnW2fmeZKcmTnnzHv+uVf6me/3e97fc555z5mZJzk7duzY4XhAAAIQgAAEIACBABHIQWACtFssFQIQgAAEIACBCAEEhhMBAhCAAAQgAIHAEUBgArdlLBgCEIAABCAAAQSGcwACEIAABCAAgcARQGACt2UsGAIQgAAEIAABBIZzAAIQgAAEIACBwBFAYAK3ZSwYAhCAAAQgAAEEhnMAAhCAAAQgAIHAEUBgArdlLBgCEIAABCAAAQSGcwACEIAABCAAgcARQGACt2UsGAIQgAAEIAABBIZzAAIQgAAEIACBwBFAYAK3ZSwYAhCAAAQgAAEEhnMAAhCAAAQgAIHAEUBgArdlLBgCEIAABCAAAQSGcwACEIAABCAAgcARQGACt2UsGAIQgAAEIAABBIZzAAIQgAAEIACBwBFAYAK3ZSwYAhCAAAQgAAEEhnMAAhCAAAQgAIHAEUBgArdlLBgCEIAABCAAAQSGcwACEIAABCAAgcARQGACt2UsGAIQgAAEIAABBIZzAAIQgAAEIACBwBFAYAK3ZSwYAhCAAAQgAAEEhnMAAhCAAAQgAIHAEUBgArdlLBgCEIAABCAAAQSGcwACEIAABCAAgcARQGACt2UsGAIQgAAEIAABBIZzAAIQgAAEIACBwBFAYAK3ZSwYAhCAAAQgAAEEhnMAAhCAAAQgAIHAEUBgArdlLBgCEIAABCAAAQSGcwACEIAABCAAgcARQGACt2UsGAIQgAAEIAABBIZzAAIQgAAEIACBwBFAYAK3ZSwYAhCAgDcEfvrzee75v/zDzb/+fHfwfhO9GZRRIJAmAghMmkAzDQQgAAG/Ebjk+nvdS6/90918+Vlu/30+67flsR4IJCWAwAT0BNnWst3tc8SPkq7+7Rfvdfl5eQE9QpYNgewjsOB3L7hrb3/IlZYUuRceu9kVFw3qgPD6P/7X/XDmDe7nM77nTpwyKfvgOOeoe1m57QkPGoEJ6PnQ2tbmbrnr8Y7VL1j4gsvNyXHf+ebkjv92/vQTXW5uTkCPkGVDIPsIxARGjnzm9BPdaScdicB0OQ2oe9l3TSQ7YgQmJOfDfkf/2OXl5bq/LrpjpyNqb9/hJk46zZ0//QT3wbJV7vm/vOE+Pa7a/fcvZrg7f/WUe+L3L7thleXulivOcuN3H9Xx/E1bGt09C37vnl38N7d+Q70rKy12e3xqtPvJace5z+85bqd5Grc2u5vuesw9//IbTv7/z4yvjsz5v+995H4x7zfu3psucPvv2/029dPP/dU9/OSL7r2lK9yOHTtc9ahd3JSvf9md8q3DuXsUknOTw7ATiAnMmF13cU3N29zzj9zoBgzIjwyQ7A7MwmdfcY88+ZJb+vEqJ9f77mNGuOOOPNh997jDdnoTc8VND7jHnv5zt0Ul+w6MXJcy/qNPLXafrFrrRCJGjxzmjjn8APf9478eqTvy+GjFGnf0qRe6L39portrzvlxD3r6rJvcK397xz274Honx9jfB3WvvwSD/XwEJtj717H6ZBeyhERgdh1e5Y6ctJ/71G4j3ayr73L7TNzDja0e4b52yBfdmRfd6vbf5zORL/PJQ4rWyWdd7d5estR96+ivuP/69Fi3pWGre/Spl9y6DXXut3df6cbtvms3emdedIt7+fW33aEH7u2+vN9Et3rNBvfEH/7iDth3T/fsS39zv547OzJn7HHbPU+4Xz70tDvgC3u6I776RZeflxt5/guvvOmOmryfu+HSH4dkdzgMCNgIxATmvGknuJvvesxdPvMH7vhjDkkqMDf89yPugUf/6L6096fdYQd/wQ3Iz3MvvvpP9+rf34k8V8bo+vj7W++6D5avjPyn1954J3LNJROYO+5fGHmj8/VDvxR5AyIfS/9ryVL3+DMvu+OPPcRddt73O4afdsFN7rU3/u3+9PANbtfhld3mXbu+zh124nlu/332dHffONMGRElR9zzBGNhBEJjAbl33hWsX8ucP/2GksP316Tsj//fgb57tNtZvca89Nc8NKStxp559rft45Rr3l4W3RwauWVvrLrh6vpvwqTHukp+e2jHZO/9Z5k768ZWRz+Dls/jY490PP3Hf+uHP3X57f8bdd8usjv8ee9co/6GrwHy8cm3k3dqxhx/orpvd/bs8F117t1v03GvuV7dd5L7wuQkh2SEOAwI6gZjAyLl/w52PuM0NW93vf/2LyF2UeHdg5M7l1DMujdwRffD22R13Q+QNyGkzrndv/M+77rd3XxG5GxrvMf/BRW7ufb9LKjCTjz/PDasqdw/feWm3IeTNx7JPatw1s37YMe8rf/uXmz7rZvejk49xP/3Rt7vl7/r10+72e59wc68+x0368j46DEOCumeAFOIIAhOSzbVcyBM/PTYiEfKYctrFbtPmRvfnJ26N/O8Zl81zi197y/3PC/fGJdLW1h65dSxfojvgmDMjv1i49+YLOrK/evxPbs4dD3d7xxj7RxEbEZyuAiPFT+7AzLv2XCfr6vp4698fOPl551k/+KY78wffDMkOcRgQ0AnEBOb+Wy509ZsbItflzZef6b52yJfiCkxEPh5c5OZcOt0dPXn/bhM88/zrbtY1d7kff2+K+8npx/VZYL469dyIoDxxz5WuvGxw0oMQcZI3JvIR8ouP39zxMbD89yNPnuW2b291zz1yY4fw6ESSJ6h7/SUY7OcjMMHev47VWy7kg7800c295tzIc+RdmxSTpx+8LvK/z7/iTvfcy2+4d166v2NMkY4HHvtjpHDW1m2OfKwUe+z9X+PdQ/Mu7vjf8h2XX//2ubjfc7ni5l+5xxYt7iYwUlilwCZ7yGf4V886IyQ7xGFAQCfQVWDk7uMx37vQFRUOitxFiXcHJtbH5an7r9npI93YXVH56Oemy87ss8Dc+/AfIh9nyToOOfBzkY+ED/riRLdLVXncMRf87nl37e0L3C1X/MQd8dUvRDJ/e+s/7vQZ17uzT5/qpn/vGzoIY4K6ZwQV0hgCE5KNtVzIhxzweXfrlT/pEJjWtna36IFr4grM+8tWuhOnXxGRljO+c1TkuyslxYVO7sScevY1rqfAXH7jA+7xZ/7sFtxxyU5f8L3ll4+7e37z+24Cc/Ylt7uXXv1n5GOokcOr4u5CZUVpwlvfIdk2DgMC3Qh0FRj5TotcU3Jt/fKGmZFfGfb8GfX/+9mNke+cyJd9R/b4zsknq9a5I0++wH31gM+5O6+b0WeBkSfKtfroosUREZE3PvI4eL+93OxzTt7py7hy9+XQb//U7fXZT7l7bvxZJCvfufvj4r9H7spUVpR5tuvUPc9QBnIgBCaQ27bzor2+kGO/VLhi5mnu28d8tWNC+WXSgceetZPASO8KKb5y61sKb9fH1bf+OvJLo64fIcW+5yIFTr7EywMCEHCRa0iupdh11NKy3R1+0szIl+2nnfINd8b5c7r1gZGPmJ57+R8u2R2YZF+It3wHpuu+NG9rcf94+73I3Vr5ZVLFkFK36IFrI79Q7Pq4bu4C99ATz7s//maOKystcYdMPdcdetDeCe8E9XXvqXt9JReO5yEw4dhH5/WF/KOZN7q//mPnd3axL+n1vANz94Jn3K13/9Zde9GP3JSvHdSN6nfOvCryq4WuAiO/mpBfT0w79Vh3zhnf6pZvam6J3PkpKiwIye5wGBCwEegpMPKs2Ec4PzvzpMgXe7s2srvzgSfdHQ886a6/eFrkZ81dH0/+8VV38S/uiVxfcp3Fe/RWYLqOIXdV5e7qbVed7Q47eN9uw8vdn6NOmRX5Mq981HTVLQ+m5Ev51D3beRXWFAITkp31+kK+8NpfukiPljsvjdwKlod8gVduWcuXbD8zrto9etdlHfTk1w4/+Okv3OSD93G3X3VOx3//5zvvR37hJI+uAiO/cvrad3/mygYXu8fvvsINr6roeM71dzwc+c6MfMcm0a8nQrJtHAYEuhGIJzANjU1u8gnnuV2qKtzSj1Z1E5ilH6923/j+7MgdUfkVUqxxpXzUe8rZ1zj51eAzD17ndhs9vE8CI/PNuPxOd9qJX4/0len6iL0JeeDWC90XP9/9rqvkfnzhLe69pZ9E7tK0bG/t+Ljayy2n7nlJM3hjITDB27PIiuUXQfJuLPaQ/iw5ubnuhGOjPSPkMeus73YUNPkZdW++A7P4r2+5n8y+zU38zFh37hnfijTVuvs3v3eTDto7IjYra9ZHvk8j/y6/TJA7JidNv9L9+73lkUK37157RDJPPvuq22ev8e4PL+7cByb2Dm7ELkPd94//misYOMDJHZ6XXnsr8jPL26862+Xk0Ek4oKcoy+4DgXgCI8PEvkcm/3/PPyUgdz7lDqj8MlB6Osl32+QjHnlTIV+YlS/Oxh7yHZYPl6/q+N+xPjAnTZnkxlaPjPx3kaBYR29pinfCtMvdso9Xu+9OPSzS6FKudfn5tnwsLG8wHpp7cdxfFcl3c+QNjzwuPvdU993jOruE9wENda+v0EL8PAQmoJvb278J0luBESzyGbb8okAaUElTKumOK/1f/vTnv7srb3kw8lNJ6ekgX+aTx4aNmyKf30sDLfnB0sRP7+7kzxnIX7uVAhvvC74y1oOPPxd5p9ba2uZ2Gz3CnfCNQyMNsqRfDQ8IZBOBRAIj15Z8F0a+ExPvbyH97g9/cb9Z+GJENOSxx9hR7jvHHbbTx7mXzrnPSTbZQwSm668RpYGlNLKTNgvra+tdbm6uGzl8qDvy0P3cqd8+otvfa+o57mEnnu82bW5wi397a+RHAP19UPf6SzBcz0dgwrWfvjwa+Yu38oU/aciV6Fa2LxfOoiAAgT4TiH1MfMKxh3ZrhtnnAQP2ROpe6jcMgUk946yZQb5sKJ+5X3XB6W5wSVHkuOUd01Enz3LNLS3ulYVz+eOSWXM2cKDZTuC8y+9wL77yT/fUA9eE+o0LdS9zZzoCkzn2oZv5kadeivzaYMKnRrupR33FyddXnvrTa5E/5njBWd+JfM+FBwQgEF4C8gcdX3/zf91Lr74V+RXjmd+f4s46LX4X4LBQoO5lbicRmMyxD+XMIizy3RkpZPJF47FjRkT+Iq78QUgeEIBAuAlIszrp6i1f7P/BiV+PNMHMhi/iU/cyc14jMJnhzqwQgAAEIAABCPSDAALTD3g8FQIQgAAEIACBzBBAYDLDnVkhAAEIQAACEOgHAQSmH/B4KgQgAAEIQAACmSGAwGSGO7NCAALORbq6rly5UmUhOcujoED/+1lNTU2WodzQoUPV3IABA9SMBPLy9KaMdXV16liNjY1qRgJlZfpffG5ubjaNVV5erua2b9+uZiRQU1Oj5qRRnvawHJ+MUVzc/Y9Mxht348aN2nSRf29tjf4V7mSP9vZ2LRL590GDBqk5S2bz5s3qOBKwnMvWsbZs2aLOaTlnZJBt27YlHauysjLpHiIw6lYQgAAEUkUAgekki8BEWSAwUQ4IjHMITKoqL+NCAAL9JoDAIDA9TyIEBoGJnRMITL9LLANAAAKpIoDAIDAITPyrizsw3IFJVd1lXAhAwAMCCAwCg8AgMIlKCXdgPCiyDAEBCKSGAAKDwCAwCAwCk5r6yqgQgEAKCSAwCAwCg8AgMCkssgwNAQikhgACg8AgMAgMApOa+sqoEIBACgkgMAgMAoPAIDApLLIMDQEIpIaACMzatWvVwdevX69mJGBpoGVtNjZy5Eh1zhUrVqgZCVga8VnWbmkiJvNZGvqVlJSY1m5pUmdtDmj5ZY2lOaC1oZ9lXda/lm3JWRoWCnTLugYPHqzuT1tbm5qRgIWXtTmg5VwuLS01rau2tjZprqKiwhUVFSXM0MjOhJkQBCCQCgIITCdVBCbKAoGJckBgnENgUlF1GRMCEPCEAAKDwPQ8kRAYBCZ2TiAwnpRZBoEABFJBAIFBYBCY+FcWd2C4A5OKmsuYEICARwQQGAQGgUFgEpUT7sB4VGgZBgIQ8J4AAoPAIDAIDALjfW1lRAhAIMUEEBgEBoFBYBCYFBdahocABLwngMAgMAgMAoPAeF9bGRECEEgxAQQGgUFgEBgEJsWFluEhAAHvCSAwCAwCg8AgMN7XVkaEAARSTEAEZvXq1eos9fX1akYCybp2xgawdIOV7IYNG9Q5Ld16ZRBL99Li4mJ1Pmvn3zFjxqhj5ebmqhkJfPLJJ2rOwl0GsXSElV+eaA+tg2tv9jo/P1+bLvLvls7Fzc3NprEsnXhbWlrUsSznjAxima+mpkadTwKWPj3V1dWmsTReVVVVLtkx0onXhJkQBCCQCgIITCdVy4sRAhPlhcBEOVjOGQQmFZWLMSEAgawngMAgMD0vAu7ARIlwB8Y57sBk/UsEACDgXwIIDAKDwMS/PhEYBMa/lZuVQQACke+G8B0Y+8cBfITER0hdywYfIVm+XUahhQAEIJACAggMd2C4A8MdmESlhY+QUlB0GRICEPCGAAKDwCAwCAwC4009ZRQIQCCNBBAYBAaBQWAQmDQWXaaCAAS8IYDAIDAIDAKDwHhTTxkFAhBIIwGrwFiaiMmyBw4cqK7e0phNBrHMOWTIEHU+CViahFkahFmbrll+wWLJyNotDe+szQErKytVXlpzMxnA0mRQcpaxhg0bpq5JAjk5OWrOss8yiOWrp2VlZep81vOvra1NHauurk7NSMCy15brUMbSzme5BgsKChKui0Z2pi0jBAEIpIIAAtNJFYGJsrBIBwITZYXAWFQwFZWLMSEAgawngMAgMD0vAgQmSoQ7MNG7oNyByfqXCQBAwJ8EEBgEBoGJf20iMAiMP6s2q4IABCIEEBgEBoFBYBKVQ+7A8EIBAQj4lgACg8AgMAgMAuPbEs3CIACBRAQQGAQGgUFgEBheIyAAgcARQGAQGAQGgUFgAle6WTAEIIDAIDAIDAKDwPBaAAEIBI4AAoPAIDAIDAITuNLNgiEAAREYS2dca8OuTZs2qVDLy8vVjAQaGhrU3IgRI9SMBLZs2aLmGhsb1Yw1YOmWWltbaxqusLBQzVl6t8gglu7GGzduVOezcm9vb1fHsswng+Tl5aljWZoRyiDJepvEJtm6das6n+W8kkF22203dSzr+ad1z5WJrF2etQ7Bcq0WFRUlXDudeNVtJQABCKSKAALTSdb6AmLZCwQmSgmBiXJAYCxXDRkIQAACvSCAwCAwPU8Xyx0R7sBEqXEHhj8l0ItySxQCEPCSAAKDwCAw8a8oPkJyjo+QvKy2jAUBCHhKAIFBYBAYBCZRUUFgPC23DAYBCHhJAIFBYBAYBAaB8bKqMhYEIJAWAggMAoPAIDAITFrKLZNAAAJeEkBgEBgEBoFBYLysqowFAQikhQACg8AgMAgMApOWcsskfiRw9913q8uaPn26mpGApW/E+++/r441fvx4NUMgynv58uUqiuLiYjUjAUtDNWuTrdLSUnXOZE22uj65vr5eHWvgwIFqxtrQb8WKFepYFRUVakYCWrMxyVgbuFmuL8uiLKws41jPGcktXLhQHXLmzJlqRgKWpniLFi1Sxxo9erSakYDlPLX86slaIy3zyVhaY0Np+Jfs3KKRnWn7CfmZAALj591JvjYEppOP5UUZgYnysrCyXhUW6UVgOmlaJBSBsZ595LKeAAIT3FMAgUFg+nL2IjBRatyBoZFdX64fnuMjAgiMjzajl0tBYBCYXp4y3IHpAgyBQWD6cv3wHB8RQGB8tBm9XAoCg8D08pRBYBCYDgJ8B6YvVw/P8RUBBMZX29GrxSAwCEyvTpj/C/MREh8hCQEEpi9XD8/xFQEExlfb0avFIDAITK9OGASmGy4+QuIjpL5cPzzHRwQQGB9tRi+XgsAgML08ZfgIiY+Q+AipLxcNz/EnAQTGn/tiWRUCg8BYzpOeGT5C4iMkPkLqy5XDc3xHAIHx3ZaYF4TAIDDmk6VLEIFBYBCYvlw5PCdtBBYvXmyaa8qUKWrO2qzK0nX0vffeU+cbN26cmiHgnHyCvWrVKhXF0KFD1YwEGhsb1ZylEZcM0tTUpI5l6bArg9TV1alj7bHHHmqmtbVVzUhg5MiRam7NmjVqRgLSDVV75ObmapHIv1s69lr2x3KdynyFhYXquu677z41I4HZs2erOWsDN8u59dBDD6nz7bfffmrGyt36bZKGhgZ1zmHDhqkZCWi1dMyYMa68vDzhWHyJ14SZUCYIIDCZoJ7eORGYTt4ITJQFAhPlgMA4h8Cktx4zm4cEEBgPYfp0KAQGgel5aiIwCEzsnEBgfFq4WZZOAIHRGQU9gcAgMAhM/KuYOzDcgQl6fc/q9SMw4d9+BAaBQWAQmESVjjsw4X8NCO0RIjCh3dqOA0NgEBgEBoFBYMJf67PuCBGY8G85AoPAIDAIDAIT/lqfdUeIwIR/yxEYBAaBQWAQmPDX+qw7QgQm/FuOwCAwCAwCg8CEv9Zn3RHOnz/fdMznnHOOKWcJWRpkac2XZB4a2VloRxvZrV27Vg1bGpLJIJYmW/n5+ep8ErA09rI2lrPMaWm2mKypV9eDysnJUY9xy5YtakYClmO0ZGSs4uJidU5LsztrJ15LY8P7779fXZMEbr75ZjVXWVmpZiTQ0tKi5hYsWKBm9tprLzUjgaVLl6o5a1PGsWPHqmNZGvVZzi1pyFhaWppwPhrZqVtBIFMEEJhMkU/fvAhMJ2sEJsoCgYlyQGCiHaURmPTVY2bykAAC4yFMnw6FwCAwPU9NBAaBiZ0TCIxPCzfL0gkgMDqjoCcQGAQGgYl/FXMHhjswQa/vWb1+BCb824/AIDAIDAKTqNJxByb8rwGhPUIEJrRb23FgCAwCg8AgMAhM+Gt91h0hAhP+LUdgEBgEBoFBYMJf67PuCBGY8G85AoPAIDAIDAIT/lqfdUeIwIR/yxEYBAaBQWAQmPDX+lAd4X/+8x/1eA455BA1I4G6ujo1t/fee6sZCTz//PNqztKMKy8vTx2HgHPt7e1u+fLlKorBgwerGQlYGhHKnJaH5ee81rEsjcssY1nOPTk2y1i5ubkWDM6SszTOk8k2bNigzjl8+HA1Y22U9tprr6ljnXHGGWpGApameF/5yldMY11yySVqbuLEiWqmqKhIzUhg1apVas5yzsggliaCBQUF6nwS2LZtW9JcVVVV0uaHNLIzYSbkNQEExmuiwRwPgencN8sLCAIT5YXARDkgMJZ+2cGsjazaxwQQGB9vThqXhsAgMD1PN+7ARIlwB8Y57sCksRgzlZ0AAmNnFeYkAoPAIDDxr3AEBoEJc+0P9LEhMIHePs8Wj8AgMAgMApOooHAHxrNSy0BeEkBgvKQZ3LEQGAQGgUFgEJjg1vCsXDkCk5XbvtNBIzAIDAKDwCAwvB4EigACE6jtStliERgEBoFBYBCYlJVYBk4FAQQmFVSDNyYCg8AgMAgMAhO82p3VK0Zgsnr7Ow4egUFgEBgEBoHh9cAXBJYuXWpax+TJk9Xc6tWr1Yw18Nxzz5mi1u6/psEIqQSkDZWlk3Jzc7M6lgQaGhrUnPyywfJobW1VYzU1NWpGAtXV1Wpu5cqVaqaiokLNSEDrcCqZrVu3msYqKSlRc5WVlWpGAvX19WrO0sX1zTffVMeRwPHHH6/mrF2zLd2gL7/8cnU+CRx66KFqznKe1tbWquNIwHLerFu3zjRWfn6+mrOcMzKIdv2MGjXKlZWVJZyPTrzqVhDoDQEEpje0yCIwnecAAhNlgcBEOSAwziEwvEaklQACk1bcgZ8MgUFgep7ECAwCEzsnEJjAl/hgHQACE6z9yvRqERgEBoGJfxVyB4Y7MJmuz1k3PwKTdVverwNGYBAYBAaBSVREuAPTr/LKk3tLAIHpLbHsziMwCAwCg8AgMNn9OuCbo0dgfLMVgVgIAoPAIDAIDAITiHId/kUiMOHfYy+PEIFBYBAYBAaB8bKqMlafCSAwfUaXlU9EYBAYBAaBQWCysvz776Avuugi06LmzJljyllCU6dOVWOPP/64miGQfgIiMJZmXIMGDTItbv369WouWWOsrk+2jGVpbiZjtrS0qOvavn27mrE2CLM0gRw3bpw6nwQsDf22bNliGis3N1fNWZrd3Xbbbeo4Enj55ZfV3PLly9WMBI4++mg1N3/+fDUjgcLCQjVnaUZYXFysjiMBC1Nrs8ghQ4aoc1rPB60p3tChQ12yY6SRnboVBHpDAIHpDS2yCEznOYDARFlYXmwRmCgrBEYqCA8IeEQAgfEIZJYMg8AgMD1PdQQmSoQ7MM5xByZLXgj8cpgIjF92IhjrQGAQGAQm/rWKwCAwwajiIVolAhOizUzDoSAwCAwCg8AkKjXcgUlDEWaKTgIIDGdDbwggMAgMAoPAIDC9qZpkU0YAgUkZ2lAOjMAgMAgMAoPAhLK8B++gEJjg7VkmV4zAIDAIDAKDwGSyCjN3BwEEhpOhNwQQGAQGgUFgEJjeVE2yKSOAwKQMbSgHRmAQGAQGgUFgQlne/XVQTU1N6oKsnULz8vLUsSoqKtSMBCzdNidMmGAai1B6CbS3tzvLn5+orq42LaytrU3Nvfvuu2pGAqNGjVJzAwcOVDMSWLVqlZrbZZdd1IylR4oMYrl2NmzYoM4nAUvnVet1b+lIbOnMvM8++5jWPmzYMDVnbYU2b948dayxY8eqGQlYOtVauk9buylbuG/cuNG09sbGRjWnddiNDaBdP5WVlXTiVWkTMBFAYEyYCPWCAALTCQuBibJAYKIcEBjnEJheFFOiyQkgMJwhXhNAYBCYnucUAoPAxM4JBMbripvF4yEwWbz5KTp0BAaBQWDiX1zcgeEOTIrKbnYOi8Bk576n8qgRGAQGgUFgEtUY7sCksvpm2dgITJZteBoOF4FBYBAYBAaBSRgmtKUAABmLSURBVEOxzfYpEJhsPwO8P34EBoFBYBAYBMb72sqIPQggMJwSXhNAYBAYBAaBQWC8rqyMtxMBBIaTwmsCCAwCg8AgMAiM15U1y8bbtGmTesRTpkxRM6+88oqakYClkd3cuXNNY02bNs2UI+Q/AtJEbPXq1erCvGzgtnXrVnU+CWzbtk3NWRvsWZp/NTQ0qPO1traqGQlYfsFiychYlgZn8mVLy2PZsmVqbMaMGWpmyZIlakYCRUVFau70009XMxI4++yz1VxhYaGakUBdXZ2aKy8vVzMffvihmpHA0KFD1Zz1fLD8zH3EiBHqfBLQznlpyJhsD3N2WNsQmpZDKKgEEJig7lyw143AdO6fVswlicBEeSEwUQ4IDAIT7FcAj1aPwHgEkmF6RQCBQWB6njDcgYkS4Q5M9E9icAemVyU1O8MITHbue6aPGoFBYBCY+FchAoPAZLo+B2Z+BCYwWxWqhSIwCAwCg8AkKmrcgQlVuU/dwSAwqWPLyIkJIDAIDAKDwCAwvEr0iwAC0y98PLmPBBAYBAaBQWAQmD4WUJ4WJYDAcCZkggACg8AgMAgMApOJ6huiORGYEG1mgA4FgUFgEBgEBoEJUNH241IRGD/uSvjXhMAgMAgMAoPAhL/Wp/QIn3nmGXX8qVOnqpm2tjY1I4GTTjpJzd13331qRgIFBQWmHCH/ERCBsfyJig0bNpgWb+nwbPl5qkxmmXPgwIGmdZWUlKi5NWvWqJkhQ4aoGQnk5OSoOUunYRmkuLhYHcvaYO+FF15Qx5o9e7aasXT0lUGOPvpodSxrndm+fbs6VmlpqZqRgOUN4+DBg9Wxampq1IwEysrK1Jyla7EMYunMbL0u5E+JJHtIB2H6wKhbRwCB4RzIBAEEppM6AhNlgcBEOSAw0T+BgMBkojIHbE4EJmAbFpLlIjAITM9TGYFBYGLnBAITkkKf6sNAYFJNmPHjEUBgEBgEJn5t4A4Md2B41TASQGCMoIh5SgCBQWAQGAQmUVHhDoyn5Ta8gyEw4d1bPx8ZAoPAIDAIDALj5yodgLUhMAHYpBAuEYFBYBAYBAaBCWFxT+chITDppM1cMQIIDAKDwCAwCAyvCf0igMD0Cx9P7iMBBAaBQWAQGASmjwU07E978803TYd4+OGHq7mGhgY1Y2kcJYMsWLBAHcvaWEkdiIBvCYjAWBpjNTc3m47B0mxswIABprF23XVXNVdXV6dmJGCZ03J91dfXm+azNJarqKgwjSVfpNQe1jpz2WWXaUO5Dz74QM0cdNBBakYC8+bNU3PLly9XMxKwNMwcOXKkaSxLE0FLw0VLQzzr+WdauLFJorWhaWNjY9Jp5dxL1kgxZ4dUEB6hJWAtLAhMaE8BXx8YAtO5PQhMlAUCE+WAwER/Ro3A+LqEp3ZxCExq+TJ6/wggMAhMzzMIgUFgYucEAtO/+hr4ZyMwgd/CUB8AAoPAIDDxL3HuwHAHJtTF33JwCIyFEplMEUBgEBgEBoFJVH+4A5OpyuyTeREYn2wEy4hLAIFBYBAYBAaB4QUiLgEEhhPDzwQQGAQGgUFgEBg/V+kMrg2BySB8plYJIDAIDAKDwCAwaqnMzgACk537HpSjRmAQGAQGgUFgglKxPVynpYmRtVmVV8uaMWOGaagbb7zRlCMUbgIiMOvXr1cPsr29Xc1IwNJAyzqWJWdto5Wsl0XswPLz89VjtDTqk0Hy8vLUsbQmYrEBamtr1bEmT56sZiRgaQZnWfvUqVNN81111VVqbt26dWpGApZmhIMGDTKNVVpaquY+/vhjNTN8+HA1IwHLa4Wl+aGMZWkwam3opzWClOtm4MCBCY+RRnam7fdnyHJSIjD+3DtWFSWAwHSeCQhMlAUCE+WAwLhIEzsEJqSvFghMSDc2iw4LgUFgep7uCAwCEzsnEJgQvxggMCHe3Cw5NAQGgUFg4l/s3IHhDkyoXwYQmFBvb1YcHAKDwCAwCEyiYscdmBC/DCAwId7cLDk0BAaBQWAQGAQmSwp+18NEYLJw00N2yAgMAoPAIDAITMgKu+VwEBgLJTJ+JoDAIDAIDAKDwPi5SqdobQhMisAybNoIIDAIDAKDwCAwaSu5/pkIgfHPXrCSvhFAYBAYBAaBQWD6Vj8D/axLL71UXf/111+vZrwM1NTUmIaTP5POAwIiMCtWrFBBlJSUqBkJNDU1qTlLRgYZPXq0Opb1fLd0L7W8IcnJyVHXJIERI0aouZUrV6oZCcybN0/NPfvss2pGAs3NzWpu8ODBauapp55SMxKw1BlrN+X6+np1zrKyMjUjgdzcXDVnaUK6ZMkSdRwJWDr2NjQ0mMaqqqpSc5s3b1YzEtA6F8t1QyM7E8rghRCY4O0ZK+5OAIHp5IHARFkgMFEOCEz0zxYgMCF91UBgQrqxWXRYCAwC0/N0R2AQmNg5gcCE+MUAgQnx5mbJoSEwCAwCE/9i5w4Md2BC/TKAwIR6e7Pi4BAYBAaBQWASFTvuwIT4ZQCBCfHmZsmhITAIDAKDwCAwWVLwux4mApOFmx6yQ0ZgEBgEBoFBYEJW2C2Hg8BYKJHxMwEEBoFBYBAYBMbPVTpFa0NgUgSWYdNGAIFBYBAYBAaBSVvJTf1Eq1evNk0yefJkNbd06VI1Yw2cdtppavSuu+5SMwQgECMgAmPpf2JtslVQUKDCtTYu2759uzqWpTGbDGL5RYllvtraWnVNEqiurlZzy5YtUzMSOP300005S6i4uFiNWZrPWZrryUSWhnF5eXnqmiSQrB9JbABLszvJDhgwQJ1zzZo1aiY/P1/NSGDMmDFq7qOPPlIzEth9993VXF1dnZqRgHbODxs2LGlfoJwd1qvZtBxCXhBAYLygyBhBIIDAdO6SVswlicBEeSEwUQ4IDALjuzqPwPhuS1hQigggMAhMz1OLOzBRItyBcY47MCkqvKkcFoFJJV3G9hMBBAaBQWDiX5EIDALjp1ptXgsCY0ZFMOAEEBgEBoFBYBKVMe7ABLDAIzAB3DSW3CcCCAwCg8AgMAhMn8qnP5+EwPhzX1iV9wQQGAQGgUFgEBjva2vGRkRgMoaeidNMAIFBYBAYBAaBSXPhTeV0CEwq6TK2nwggMAgMAoPAIDB+qsr9XAsC00+APD0wBBAYBAaBQWAQmMCUbH2hI0aM0EO9aGplGeyII45QYwsXLlQzlg6T6iAEsoaACIyly+769etNTMrKytRcW1ubmpGApUPr8uXLTWONGjVKzVka2bW0tKjjSMDS+XfKlCmmsd555x01V1JSomYkcMABB6i5n//852qmsrJSzUjA0pm5qanJNNaQIUPUnOWckUHWrl2rjtXa2qpmrGu3dBEuKipS55OApU+Ppbu2jKWtv6qqyiXr3kwnXtOWpTeEwKSXN7NljgAC08kegYmyQGCiHBAY5xCYzNXmPs+MwPQZHU8MGAEEBoHpecoiMAhM7JxAYAJW0GW5CEwAN40l94kAAoPAIDDxLx3uwHAHpk9FNdNPQmAyvQPMny4CCAwCg8AgMInqDXdg0lWJPZwHgfEQJkP5mgACg8AgMAgMAuPrMt27xSEwveNFOrgEEBgEBoFBYBCY4NbwnVaOwIRoMzmUpAQQGAQGgUFgEJgQvVAgMCHaTA4FgXHO0QcmehrQBybKgT4wUQ70gQnhC4S1GZI1Z0G0ePFiNXbQQQepGQIQ6A0Bayfe5uZm07CDBg1Sc9ameJZmaZYGYbKguro6dV2NjY1qxjpfTk6OOta0adPUjAQ+/PBDNffRRx+pGQnMnz9fzU2aNEnNWJsRWpquyTloeVgavVnP061bt6pTWvbay8ahtbW16pokUFhYqOas69J4yf7RyE7F7a+AVUysOcvRITAWSmS8JoDAdBJFYKIsEJgoBwQm2vUXgfG66qZ4PKuYWHOW5SIwFkpkvCaAwCAwPc8pBAaBiZ0TCIzXFTcN41nFxJqzLBmBsVAi4zUBBAaBQWDiX1XcgeEOjNf1Ni3jWcXEmrMsGoGxUCLjNQEEBoFBYBCYRHWFOzBeV9w0jGcVE2vOsmQExkKJjNcEEBgEBoFBYBAYrytrBseziok1ZzkUBMZCiYzXBBAYBAaBQWAQGK8rawbHs4qJNWc5FATGQomM1wQQGAQGgUFgEBivK2sGx7OKiTVnORQExkKJjNcEEBgEBoFBYBAYrytrBseziok1ZzkUBMZCiYzXBBAYBAaBQWAQGK8ra4rGmzlzpjryTTfdpGYkkJuba8pZQpZOoaWlpZahyEDATEAExtLAzdK5VCa1dEttb283rc/SsdfyU1eZzNI5dtiwYeq6Ghoa1IwErrvuOjV3xx13qBkJ7LvvvmrurbfeUjMSeP3119WchYOl47JMtG3bNnW+srIyNSMBrWtsb2pyfX29Oufw4cPVjDWwZcsWNWrtnmu5xlavXq3OJ4Hy8vKkOZkr2TWWs8PaR9m0HEIaAQRGI8S/ZxMBBKZzty0v3AhMlBcC07sqgcD0jhfpBAQQGE4NCHQSQGAQmJ7Xg0XkEJjeVREEpne8SCMwnAMQUAkgMAgMAhP/MuEjpOhHwnyEpJbR9AW4A5M+1szkfwIIDAKDwCAwiSoVAuOzGo7A+GxDWE5GCSAwCAwCg8AgMBktw/bJERg7K5LhJ4DAIDAIDAKDwASk1iMwAdkolpkWAggMAoPAIDAITFrKbf8nQWD6z5ARwkMAgUFgEBgEBoHJcE23Nu458MAD1ZWuWLFCzUigoKBAzV122WVqRgIWsfKy869pUYRCT0AEpqmpST1OSxMxdZD/C9TU1Jiira2tam633XZTMxKwNIq0XF8bNmwwzXfUUUepue3bt6sZCViaCM6ZM8c01pFHHqnmKioq1Ex+fr6akcDbb7+t5j772c+qGQlYforc0tJiGsvyU3FLYzlLoz5ZULqbnlquHVmX1ixy/PjxrrKyMiFTGtmZTjc9hMDojEhAoCcBBKaTCAITZYHARDkgMM4hMGl6zUBg0gSaaUJFAIFBYHqe0AgMAhM7JxCYNJV7BCZNoJkmVAQQGAQGgYl/SXMHhjswaSv2CEzaUDNRiAggMAgMAoPAJCpp3IFJU7FHYNIEmmlCRQCBQWAQGAQGgclwWUdgMrwBTB9IAggMAoPAIDAITIbLNwKT4Q1g+kASQGAQGAQGgUFgMly+EZgMbwDTB5IAAoPAIDAIDAKT4fL9wQcfmFaw5557qrm2tjY1I4EJEyaouSVLlqgZAhDIFAGrwFiaqckxDBw4UD0US0MyGWTQoEHqWBs3blQzEigrK1NzlmO0vlGaNGmSOl9JSYmakYBlrFtvvdU0lqXBmaUfjrWRneWXPJZ9loNbt26deoyW45NBLDXecs5Y99DSCHLTpk3q8UnAMtbo0aNNY2nXT1VVlSsuLk44Fo3sTJj1EAKjMyIBgZ4EEJhOIghMlAUCE+WAwDiHwKTpNQOBSRNopgkVAQQGgel5QiMwCEzsnEBg0lTuEZg0gWaaUBFAYBAYBCb+Jc0dGO7ApK3YIzBpQ81EISKAwCAwCAwCk6ikcQcmTcUegUkTaKYJFQEEBoFBYBAYBCbDZR2ByfAGMH0gCSAwCAwCg8AgMBku3whMhjeA6QNJAIFBYBAYBAaByXD5RmAyvAFMH0gCCAwCg8AgMAhMhss3ApPhDWD6QBJAYBAYBAaBQWAyXL4RmAxvANMHkkB7e7v7+OOP1bUPGTJEzUhAhEh7tLS0aJHIv1u6vW7YsME01siRI9WcpcPpM888o44jgVmzZqk5awfa/fffXx1r7ty5akYCBQUFam7btm1qxtqReMyYMepYDQ0NakYClq6+1k68OTk56pyWfjjJutR2nUDreCvZ8vJydU0SaGxsVHOWa0cG0c55foWkovYmgMB4w5FRsosAAtO531oxlyQCE+WFwEQ5IDCWtyzZVVP7dLQITJ+w8aQsJ4DAIDA9LwHuwESJcAeGRnZpe3lAYNKGmolCRACBQWAQmPgXNAKDwKSt1CMwaUPNRCEigMAgMAgMApOopPEdmDQVewQmTaCZJlQEEBgEBoFBYBCYDJd1BCbDG8D0gSSAwCAwCAwCg8BkuHwjMBneAKYPJAEEBoFBYBAYBCbD5RuByfAGMH0gCSAwCAwCg8AgMBku35bmPrLEU045RV3pokWL1IwEJkyYoOaWLFmiZghAIFMERGDefvttdfrq6mo1I4Ht27erOUsTMRnE0pTM8ksRGWvw4MHquiwN/dra2tRxJDB79mw1969//UvNWNf+6KOPmsayNCQsKipSx7KwkkFGjRqljlVYWKhmJGCppZb5ZCxLM8WSkhJ1XVu2bFEz1j3ctGmTaSxLk7qKigrTWFo/n+HDhye9dnJ20AfGBFoLITAaIf4dAjsTQGA6mVhelBGYKC8LKwSm89yyCDQCk8UVGoHJ4s3n0PtMAIFBYHqePNyBiRLhDoxz3IHpc2nt3RMRmN7xIg0BIYDAIDAITPxagMAgMGl7lUBg0oaaiUJEAIFBYBAYBCZRSeMOTJqKPQKTJtBMEyoCCAwCg8AgMAhMhss6ApPhDWD6QBJAYBAYBAaBQWAyXL4RmAxvANMHkgACg8AgMAgMApPh8o3AZHgDmD6QBBAYBAaBQWAQmAyXbwQmwxvA9IEkgMAgMAgMAoPABLJ8s2gIZDcB6aNZX1+vQrD229y8ebM6lqXPiAxiaexl6dYrY+26667qupqamtSMtYtwc3OzOpa1W6qlu3Ftba06nwQsXVwtzfqsbxgtnX9Foi0PC69169ZZhjJ1ebbs4YgRI0zzWXhVVVWZxrJ0zh47dqxpLG1d/ArJhJEQBCCQCQIITCd1BCbKAoGJckBg6AOTiZrMnBCAgJEAAoPA9DxVEBgEJnZOcAfGWEiJQQAC6SeAwCAwCEz86447MNyBSX9FZkYIQMBMAIFBYBAYBCZRweAOjLmUEoQABNJNAIFBYBAYBAaBSXflZT4IQKDfBBAYBAaBQWAQmH6XUgaAAATSTQCBQWAQGAQGgUl35WU+CECg3wQQGAQGgUFgEJh+l1IGgAAE0k1Amoi9//776rTjx49XMxJYtmyZmisvL1czErA0z6urqzONZQntsssuaszaOG/9+vXqWDU1NWpGApZmaXl5eaaxCgoK1JyloZqluZ5MZMk1NDSoa5KApU+PZQ9lrE8++USdc9y4cWrG0gRSBhkwYIA6VmlpqZqRwJo1a0w5S6iysjJpTM6XZGvP2WG5Si0rIQMBCECglwQQmE5glhc/BCbKyyIm1hwCE2WKwPSyeBGHAASymwACg8D0vAK4AxMlwh0Y57gDk92vDxw9BHxNAIFBYBCY+JcoAoPA+Lp4szgIZDsBBAaBQWAQmER1kDsw2f4KwfFDwMcEEBgEBoFBYBAYHxdplgYBCMQngMAgMAgMAoPA8AoBAQgEjgACg8AgMAgMAhO40s2CIQABBAaBQWAQGASG1wIIQCBwBBAYBAaBQWAQmMCVbhYMAQhIH80VK1aoICwdXGUQS1Myaydey7oGDhyorl0CQ4YMUXMic9rD0lFVxsjNzdWGMrGSQSwNzvLz89X5JNDW1qbmVq9erWaqq6vVjAQsXWMLCwtNY1m6DW/bts001rp169Sc5ZwpKipSx5GAZX+am5tNY1VUVKi5zZs3qxkJaA0JpSdQcXFxwrHoxGvCTAgCEEgFAQSmkyoCE2WBwEQ5IDDOITCpqLqMCQEIeEIAgUFgep5ICAwCEzsnEBhPyiyDQAACqSCAwCAwCEz8K4s7MNyBSUXNZUwIQMAjAggMAoPAIDCJygl3YDwqtAwDAQh4TwCBQWAQGAQGgfG+tjIiBCCQYgIIDAKDwCAwCEyKCy3DQwAC3hNAYBAYBAaBQWC8r62MCAEIpJgAAoPAIDAIDAKT4kLL8BCAAAQgAAEI+IcAjez8sxesBAIQgAAEIAABIwEExgiKGAQgAAEIQAAC/iGAwPhnL1gJBCAAAQhAAAJGAgiMERQxCEAAAhCAAAT8QwCB8c9esBIIQAACEIAABIwEEBgjKGIQgAAEIAABCPiHAALjn71gJRCAAAQgAAEIGAkgMEZQxCAAAQhAAAIQ8A8BBMY/e8FKIAABCEAAAhAwEkBgjKCIQQACEIAABCDgHwIIjH/2gpVAAAIQgAAEIGAkgMAYQRGDAAQgAAEIQMA/BBAY/+wFK4EABCAAAQhAwEgAgTGCIgYBCEAAAhCAgH8IIDD+2QtWAgEIQAACEICAkQACYwRFDAIQgAAEIAAB/xBAYPyzF6wEAhCAAAQgAAEjAQTGCIoYBCAAAQhAAAL+IYDA+GcvWAkEIAABCEAAAkYCCIwRFDEIQAACEIAABPxDAIHxz16wEghAAAIQgAAEjAQQGCMoYhCAAAQgAAEI+IcAAuOfvWAlEIAABCAAAQgYCSAwRlDEIAABCEAAAhDwDwEExj97wUogAAEIQAACEDASQGCMoIhBAAIQgAAEIOAfAgiMf/aClUAAAhCAAAQgYCSAwBhBEYMABCAAAQhAwD8EEBj/7AUrgQAEIAABCEDASACBMYIiBgEIQAACEICAfwggMP7ZC1YCAQhAAAIQgICRAAJjBEUMAhCAAAQgAAH/EEBg/LMXrAQCEIAABCAAASMBBMYIihgEIAABCEAAAv4hgMD4Zy9YCQQgAAEIQAACRgIIjBEUMQhAAAIQgAAE/EMAgfHPXrASCEAAAhCAAASMBBAYIyhiEIAABCAAAQj4hwAC45+9YCUQgAAEIAABCBgJ/H+8I4GqzMDOBQAAAABJRU5ErkJggg==",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sample_batch = pipeline.get_results()\n",
    "\n",
    "GridDisplay([ImageDisplay(image=sample_batch[\"x\"][0], color_map='greys', title=\"Image\"),\n",
    "             ImageDisplay(image=sample_batch[\"x_w_noise\"][0], color_map='greys', title=\"Noisy Image\")\n",
    "            ]).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Create `Network`\n",
    "\n",
    "The architecture of our model consists of an Autoencoder (ecoder-decoder) network and a Discriminator network.\n",
    "<BR>\n",
    "<p align=\"center\">\n",
    "<img src=\"https://github.com/fastestimator-util/fastestimator-misc/blob/master/resource/pictures/apphub/alocc/network_architecture.PNG?raw=true\" width=800 height=400>\n",
    "</p>\n",
    "<BR>\n",
    "[Credit: https://arxiv.org/pdf/1802.09088.pdf]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def reconstructor(input_shape=(28, 28, 1)):\n",
    "    model = tf.keras.Sequential()\n",
    "    # Encoder Block\n",
    "    model.add(\n",
    "        layers.Conv2D(32, (5, 5),\n",
    "                      strides=(2, 2),\n",
    "                      padding='same',\n",
    "                      kernel_initializer=tf.keras.initializers.TruncatedNormal(stddev=0.02),\n",
    "                      input_shape=input_shape))\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.LeakyReLU(0.2))\n",
    "    model.add(\n",
    "        layers.Conv2D(64, (5, 5),\n",
    "                      strides=(2, 2),\n",
    "                      kernel_initializer=tf.keras.initializers.TruncatedNormal(stddev=0.02),\n",
    "                      padding='same'))\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.LeakyReLU(0.2))\n",
    "    model.add(\n",
    "        layers.Conv2D(128, (5, 5),\n",
    "                      strides=(2, 2),\n",
    "                      kernel_initializer=tf.keras.initializers.TruncatedNormal(stddev=0.02),\n",
    "                      padding='same'))\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.LeakyReLU(0.2))\n",
    "\n",
    "    # Decoder Block\n",
    "    model.add(\n",
    "        layers.Conv2DTranspose(32, (5, 5),\n",
    "                               strides=(2, 2),\n",
    "                               output_padding=(0, 0),\n",
    "                               padding='same',\n",
    "                               kernel_initializer=tf.keras.initializers.RandomNormal(stddev=0.02)))\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.ReLU())\n",
    "    model.add(\n",
    "        layers.Conv2DTranspose(16, (5, 5),\n",
    "                               strides=(2, 2),\n",
    "                               padding='same',\n",
    "                               kernel_initializer=tf.keras.initializers.RandomNormal(stddev=0.02)))\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.ReLU())\n",
    "    model.add(\n",
    "        layers.Conv2DTranspose(1, (5, 5),\n",
    "                               strides=(2, 2),\n",
    "                               padding='same',\n",
    "                               kernel_initializer=tf.keras.initializers.RandomNormal(stddev=0.02),\n",
    "                               activation='tanh'))\n",
    "    return model\n",
    "\n",
    "\n",
    "def discriminator(input_shape=(28, 28, 1)):\n",
    "    model = tf.keras.Sequential()\n",
    "    model.add(\n",
    "        layers.Conv2D(16, (5, 5),\n",
    "                      strides=(2, 2),\n",
    "                      padding='same',\n",
    "                      kernel_initializer=tf.keras.initializers.TruncatedNormal(stddev=0.02),\n",
    "                      input_shape=input_shape))\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.LeakyReLU(0.2))\n",
    "    model.add(\n",
    "        layers.Conv2D(32, (5, 5),\n",
    "                      strides=(2, 2),\n",
    "                      padding='same',\n",
    "                      kernel_initializer=tf.keras.initializers.TruncatedNormal(stddev=0.02)))\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.LeakyReLU(0.2))\n",
    "    model.add(\n",
    "        layers.Conv2D(64, (5, 5),\n",
    "                      strides=(2, 2),\n",
    "                      padding='same',\n",
    "                      kernel_initializer=tf.keras.initializers.TruncatedNormal(stddev=0.02)))\n",
    "    model.add(layers.BatchNormalization())\n",
    "    model.add(layers.LeakyReLU(0.2))\n",
    "    model.add(\n",
    "        layers.Conv2D(128, (5, 5),\n",
    "                      strides=(2, 2),\n",
    "                      padding='same',\n",
    "                      kernel_initializer=tf.keras.initializers.TruncatedNormal(stddev=0.02)))\n",
    "    model.add(layers.LeakyReLU(0.2))\n",
    "    model.add(layers.Flatten())\n",
    "    model.add(layers.Dense(1, activation=\"sigmoid\"))\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-05-20 22:51:09.938098: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA\n",
      "To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
      "2022-05-20 22:51:10.451389: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1525] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 38420 MB memory:  -> device: 0, name: NVIDIA A100-SXM4-40GB, pci bus id: 0000:90:00.0, compute capability: 8.0\n"
     ]
    }
   ],
   "source": [
    "recon_model = fe.build(model_fn=reconstructor, optimizer_fn=lambda: tf.optimizers.RMSprop(2e-4), model_name=\"reconstructor\")\n",
    "disc_model = fe.build(model_fn=discriminator,\n",
    "                      optimizer_fn=lambda: tf.optimizers.RMSprop(1e-4),\n",
    "                      model_name=\"discriminator\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining Loss\n",
    "\n",
    "The losses of both the networks are smilar to a standarad GAN network with the exception of the autoencoder having and additional *reconstruction* loss term to enforce similarity between the input and the reconstructed image.\n",
    "We first define custom `TensorOp`s to calculate the losses of both the networks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "class RLoss(TensorOp):\n",
    "    def __init__(self, alpha=0.2, inputs=None, outputs=None, mode=None):\n",
    "        super().__init__(inputs, outputs, mode)\n",
    "        self.alpha = alpha\n",
    "\n",
    "    def forward(self, data, state):\n",
    "        fake_score, x_fake, x = data\n",
    "        recon_loss = binary_crossentropy(y_true=x, y_pred=x_fake, from_logits=True)\n",
    "        adv_loss = binary_crossentropy(y_pred=fake_score, y_true=tf.ones_like(fake_score), from_logits=True)\n",
    "        return adv_loss + self.alpha * recon_loss\n",
    "\n",
    "\n",
    "class DLoss(TensorOp):\n",
    "    def forward(self, data, state):\n",
    "        true_score, fake_score = data\n",
    "        real_loss = binary_crossentropy(y_pred=true_score, y_true=tf.ones_like(true_score), from_logits=True)\n",
    "        fake_loss = binary_crossentropy(y_pred=fake_score, y_true=tf.zeros_like(fake_score), from_logits=True)\n",
    "        total_loss = real_loss + fake_loss\n",
    "        return total_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now define the `Network` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-05-20 22:51:11.153783: W tensorflow/python/util/util.cc:368] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.\n"
     ]
    }
   ],
   "source": [
    "network = fe.Network(ops=[\n",
    "    ModelOp(model=recon_model, inputs=\"x_w_noise\", outputs=\"x_fake\", mode=\"train\"),\n",
    "    ModelOp(model=recon_model, inputs=\"x\", outputs=\"x_fake\", mode=\"eval\"),\n",
    "    ModelOp(model=disc_model, inputs=\"x_fake\", outputs=\"fake_score\"),\n",
    "    ModelOp(model=disc_model, inputs=\"x\", outputs=\"true_score\"),\n",
    "    RLoss(inputs=(\"fake_score\", \"x_fake\", \"x\"), outputs=\"rloss\"),\n",
    "    UpdateOp(model=recon_model, loss_name=\"rloss\"),\n",
    "    DLoss(inputs=(\"true_score\", \"fake_score\"), outputs=\"dloss\"),\n",
    "    UpdateOp(model=disc_model, loss_name=\"dloss\")\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we will also use the following traces:\n",
    "\n",
    "1. BestModelSaver for saving the best model. For illustration purpose, we will save these models in a temporary directory.\n",
    "2. A custom trace to calculate Area Under the Curve and F1-Score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "class F1AUCScores(Trace):\n",
    "    \"\"\"Computes F1-Score and AUC Score for a classification task and reports it back to the logger.\n",
    "    \"\"\"\n",
    "    def __init__(self, true_key, pred_key, mode=(\"eval\", \"test\"), output_name=[\"auc_score\", \"f1_score\"]):\n",
    "        super().__init__(inputs=(true_key, pred_key), outputs=output_name, mode=mode)\n",
    "        self.y_true = []\n",
    "        self.y_pred = []\n",
    "\n",
    "    @property\n",
    "    def true_key(self):\n",
    "        return self.inputs[0]\n",
    "\n",
    "    @property\n",
    "    def pred_key(self):\n",
    "        return self.inputs[1]\n",
    "\n",
    "    def on_epoch_begin(self, data):\n",
    "        self.y_true = []\n",
    "        self.y_pred = []\n",
    "\n",
    "    def on_batch_end(self, data):\n",
    "        y_true, y_pred = to_number(data[self.true_key]), to_number(data[self.pred_key])\n",
    "        assert y_pred.size == y_true.size\n",
    "        self.y_pred.extend(y_pred.ravel())\n",
    "        self.y_true.extend(y_true.ravel())\n",
    "\n",
    "    def on_epoch_end(self, data):\n",
    "        fpr, tpr, thresholds = roc_curve(self.y_true, self.y_pred, pos_label=1)\n",
    "        roc_auc = auc(fpr, tpr)\n",
    "        eer_threshold = thresholds[np.nanargmin(np.absolute((1 - tpr - fpr)))]\n",
    "        y_pred_class = np.copy(self.y_pred)\n",
    "        y_pred_class[y_pred_class >= eer_threshold] = 1\n",
    "        y_pred_class[y_pred_class < eer_threshold] = 0\n",
    "        f_score = f1_score(self.y_true, y_pred_class)\n",
    "\n",
    "        data.write_with_log(self.outputs[0], roc_auc)\n",
    "        data.write_with_log(self.outputs[1], f_score)\n",
    "        \n",
    "\n",
    "traces = [\n",
    "    F1AUCScores(true_key=\"y\", pred_key=\"fake_score\", mode=\"eval\", output_name=[\"auc_score\", \"f1_score\"]),\n",
    "    BestModelSaver(model=recon_model, save_dir=save_dir, metric='f1_score', save_best_mode='max', load_best_final=True),\n",
    "    BestModelSaver(model=disc_model, save_dir=save_dir, metric='f1_score', save_best_mode='max', load_best_final=True)\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Create `Estimator`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "estimator = fe.Estimator(pipeline=pipeline,\n",
    "                         network=network,\n",
    "                         epochs=epochs,\n",
    "                         traces=traces,\n",
    "                         train_steps_per_epoch=train_steps_per_epoch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    ______           __  ______     __  _                 __            \n",
      "   / ____/___ ______/ /_/ ____/____/ /_(_)___ ___  ____ _/ /_____  _____\n",
      "  / /_  / __ `/ ___/ __/ __/ / ___/ __/ / __ `__ \\/ __ `/ __/ __ \\/ ___/\n",
      " / __/ / /_/ (__  ) /_/ /___(__  ) /_/ / / / / / / /_/ / /_/ /_/ / /    \n",
      "/_/    \\__,_/____/\\__/_____/____/\\__/_/_/ /_/ /_/\\__,_/\\__/\\____/_/     \n",
      "                                                                        \n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/dispatch.py:1082: UserWarning:\n",
      "\n",
      "\"`binary_crossentropy` received `from_logits=True`, but the `output` argument was produced by a sigmoid or softmax activation and thus does not represent logits. Was this intended?\"\n",
      "\n",
      "2022-05-20 22:51:17.273882: I tensorflow/stream_executor/cuda/cuda_dnn.cc:368] Loaded cuDNN version 8100\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FastEstimator-Warn: the key 'y' is being pruned since it is unused outside of the Pipeline. To prevent this, you can declare the key as an input of a Trace or TensorOp.\n",
      "FastEstimator-Start: step: 1; logging_interval: 100; num_device: 1;\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-05-20 22:51:29.792069: I tensorflow/stream_executor/cuda/cuda_blas.cc:1786] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FastEstimator-Train: step: 1; dloss: 1.4055667; rloss: 0.9982634;\n",
      "FastEstimator-Train: step: 53; epoch: 1; epoch_time: 8.74 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 155.95;\n",
      "Eval Progress: 11/17; steps/sec: 185.13;\n",
      "Eval Progress: 17/17; steps/sec: 172.52;\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/reconstructor_best_f1_score.h5\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/discriminator_best_f1_score.h5\n",
      "FastEstimator-Eval: step: 53; epoch: 1; auc_score: 0.4215160395117313; dloss: 1.4071065; f1_score: 0.4431718061674009; max_f1_score: 0.4431718061674009; rloss: 1.0221571; since_best_f1_score: 0;\n",
      "FastEstimator-Train: step: 100; dloss: 0.18782794; rloss: 3.4576418; steps/sec: 12.69;\n",
      "FastEstimator-Train: step: 106; epoch: 2; epoch_time: 5.69 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 158.05;\n",
      "Eval Progress: 11/17; steps/sec: 237.7;\n",
      "Eval Progress: 17/17; steps/sec: 212.97;\n",
      "FastEstimator-Eval: step: 106; epoch: 2; auc_score: 0.0593130082089697; dloss: 1.7725676; f1_score: 0.12945838837516513; max_f1_score: 0.4431718061674009; rloss: 1.739687; since_best_f1_score: 1;\n",
      "FastEstimator-Train: step: 159; epoch: 3; epoch_time: 5.57 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 203.15;\n",
      "Eval Progress: 11/17; steps/sec: 257.38;\n",
      "Eval Progress: 17/17; steps/sec: 267.37;\n",
      "FastEstimator-Eval: step: 159; epoch: 3; auc_score: 0.3312002949795261; dloss: 3.8447435; f1_score: 0.3762114537444934; max_f1_score: 0.4431718061674009; rloss: 3.5073256; since_best_f1_score: 2;\n",
      "FastEstimator-Train: step: 200; dloss: 0.0074149473; rloss: 5.2848363; steps/sec: 8.9;\n",
      "FastEstimator-Train: step: 212; epoch: 4; epoch_time: 5.7 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 133.68;\n",
      "Eval Progress: 11/17; steps/sec: 182.39;\n",
      "Eval Progress: 17/17; steps/sec: 172.01;\n",
      "FastEstimator-Eval: step: 212; epoch: 4; auc_score: 0.12731859729472725; dloss: 10.529196; f1_score: 0.20000000000000004; max_f1_score: 0.4431718061674009; rloss: 6.2233853; since_best_f1_score: 3;\n",
      "FastEstimator-Train: step: 265; epoch: 5; epoch_time: 5.84 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 113.41;\n",
      "Eval Progress: 11/17; steps/sec: 165.91;\n",
      "Eval Progress: 17/17; steps/sec: 268.33;\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/reconstructor_best_f1_score.h5\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/discriminator_best_f1_score.h5\n",
      "FastEstimator-Eval: step: 265; epoch: 5; auc_score: 0.9694777697995305; dloss: 16.567978; f1_score: 0.9053280493174812; max_f1_score: 0.9053280493174812; rloss: 9.16089; since_best_f1_score: 0;\n",
      "FastEstimator-Train: step: 300; dloss: 9.0785594e-05; rloss: 9.5743; steps/sec: 8.69;\n",
      "FastEstimator-Train: step: 318; epoch: 6; epoch_time: 5.87 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 153.15;\n",
      "Eval Progress: 11/17; steps/sec: 223.88;\n",
      "Eval Progress: 17/17; steps/sec: 217.8;\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/reconstructor_best_f1_score.h5\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/discriminator_best_f1_score.h5\n",
      "FastEstimator-Eval: step: 318; epoch: 6; auc_score: 0.9818909740146323; dloss: 20.46954; f1_score: 0.9263992948435434; max_f1_score: 0.9263992948435434; rloss: 10.067438; since_best_f1_score: 0;\n",
      "FastEstimator-Train: step: 371; epoch: 7; epoch_time: 5.85 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 202.2;\n",
      "Eval Progress: 11/17; steps/sec: 238.39;\n",
      "Eval Progress: 17/17; steps/sec: 224.66;\n",
      "FastEstimator-Eval: step: 371; epoch: 7; auc_score: 0.6487768052941063; dloss: 17.003485; f1_score: 0.6144366197183099; max_f1_score: 0.9263992948435434; rloss: 13.551922; since_best_f1_score: 1;\n",
      "FastEstimator-Train: step: 400; dloss: 2.1421674e-06; rloss: 13.66077; steps/sec: 8.66;\n",
      "FastEstimator-Train: step: 424; epoch: 8; epoch_time: 5.64 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 139.75;\n",
      "Eval Progress: 11/17; steps/sec: 249.63;\n",
      "Eval Progress: 17/17; steps/sec: 213.55;\n",
      "FastEstimator-Eval: step: 424; epoch: 8; auc_score: 0.5244898988918861; dloss: 26.341074; f1_score: 0.5312775330396475; max_f1_score: 0.9263992948435434; rloss: 14.630283; since_best_f1_score: 2;\n",
      "FastEstimator-Train: step: 477; epoch: 9; epoch_time: 5.71 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 118.19;\n",
      "Eval Progress: 11/17; steps/sec: 188.76;\n",
      "Eval Progress: 17/17; steps/sec: 175.64;\n",
      "FastEstimator-Eval: step: 477; epoch: 9; auc_score: 0.8110687185856508; dloss: 21.545362; f1_score: 0.7548500881834214; max_f1_score: 0.9263992948435434; rloss: 15.638036; since_best_f1_score: 3;\n",
      "FastEstimator-Train: step: 500; dloss: 4.0664025e-05; rloss: 13.955134; steps/sec: 8.85;\n",
      "FastEstimator-Train: step: 530; epoch: 10; epoch_time: 5.67 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 169.3;\n",
      "Eval Progress: 11/17; steps/sec: 207.95;\n",
      "Eval Progress: 17/17; steps/sec: 202.13;\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/reconstructor_best_f1_score.h5\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/discriminator_best_f1_score.h5\n",
      "FastEstimator-Eval: step: 530; epoch: 10; auc_score: 0.9777368083991539; dloss: 17.341671; f1_score: 0.9463028169014085; max_f1_score: 0.9463028169014085; rloss: 12.420723; since_best_f1_score: 0;\n",
      "FastEstimator-Train: step: 583; epoch: 11; epoch_time: 5.64 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 128.68;\n",
      "Eval Progress: 11/17; steps/sec: 247.49;\n",
      "Eval Progress: 17/17; steps/sec: 240.88;\n",
      "FastEstimator-Eval: step: 583; epoch: 11; auc_score: 0.9810429078771178; dloss: 12.695894; f1_score: 0.9219920669898635; max_f1_score: 0.9463028169014085; rloss: 6.138731; since_best_f1_score: 1;\n",
      "FastEstimator-Train: step: 600; dloss: 0.1742793; rloss: 2.5519087; steps/sec: 8.71;\n",
      "FastEstimator-Train: step: 636; epoch: 12; epoch_time: 5.91 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 69.28;\n",
      "Eval Progress: 11/17; steps/sec: 187.24;\n",
      "Eval Progress: 17/17; steps/sec: 212.75;\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/reconstructor_best_f1_score.h5\n",
      "FastEstimator-BestModelSaver: Saved model to /tmp/tmp2w1fltl6/discriminator_best_f1_score.h5\n",
      "FastEstimator-Eval: step: 636; epoch: 12; auc_score: 0.99651613654447; dloss: 3.250877; f1_score: 0.9788546255506608; max_f1_score: 0.9788546255506608; rloss: 0.61733353; since_best_f1_score: 0;\n",
      "FastEstimator-Train: step: 689; epoch: 13; epoch_time: 5.54 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 170.37;\n",
      "Eval Progress: 11/17; steps/sec: 251.05;\n",
      "Eval Progress: 17/17; steps/sec: 223.27;\n",
      "FastEstimator-Eval: step: 689; epoch: 13; auc_score: 0.9530811775893187; dloss: 3.0862265; f1_score: 0.8974020255394101; max_f1_score: 0.9788546255506608; rloss: 2.3505096; since_best_f1_score: 1;\n",
      "FastEstimator-Train: step: 700; dloss: 0.09519278; rloss: 5.07388; steps/sec: 8.95;\n",
      "FastEstimator-Train: step: 742; epoch: 14; epoch_time: 5.68 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 189.04;\n",
      "Eval Progress: 11/17; steps/sec: 235.52;\n",
      "Eval Progress: 17/17; steps/sec: 271.66;\n",
      "FastEstimator-Eval: step: 742; epoch: 14; auc_score: 0.9910233072638709; dloss: 2.9711509; f1_score: 0.9616571176729837; max_f1_score: 0.9788546255506608; rloss: 0.5347926; since_best_f1_score: 2;\n",
      "FastEstimator-Train: step: 795; epoch: 15; epoch_time: 5.67 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 153.04;\n",
      "Eval Progress: 11/17; steps/sec: 247.51;\n",
      "Eval Progress: 17/17; steps/sec: 232.68;\n",
      "FastEstimator-Eval: step: 795; epoch: 15; auc_score: 0.9850724834559181; dloss: 1.6044965; f1_score: 0.9494060712714474; max_f1_score: 0.9788546255506608; rloss: 3.59912; since_best_f1_score: 3;\n",
      "FastEstimator-Train: step: 800; dloss: 0.30007884; rloss: 2.662114; steps/sec: 8.83;\n",
      "FastEstimator-Train: step: 848; epoch: 16; epoch_time: 5.9 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 72.03;\n",
      "Eval Progress: 11/17; steps/sec: 156.27;\n",
      "Eval Progress: 17/17; steps/sec: 209.58;\n",
      "FastEstimator-Eval: step: 848; epoch: 16; auc_score: 0.9890345242484814; dloss: 2.4346852; f1_score: 0.9379128137384413; max_f1_score: 0.9788546255506608; rloss: 1.1848502; since_best_f1_score: 4;\n",
      "FastEstimator-Train: step: 900; dloss: 2.2080877; rloss: 0.1052717; steps/sec: 15.36;\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FastEstimator-Train: step: 901; epoch: 17; epoch_time: 5.79 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 115.86;\n",
      "Eval Progress: 11/17; steps/sec: 181.74;\n",
      "Eval Progress: 17/17; steps/sec: 141.38;\n",
      "FastEstimator-Eval: step: 901; epoch: 17; auc_score: 0.9592586698752159; dloss: 1.7382531; f1_score: 0.8748898678414097; max_f1_score: 0.9788546255506608; rloss: 1.7491591; since_best_f1_score: 5;\n",
      "FastEstimator-Train: step: 954; epoch: 18; epoch_time: 5.87 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 88.9;\n",
      "Eval Progress: 11/17; steps/sec: 133.96;\n",
      "Eval Progress: 17/17; steps/sec: 209.24;\n",
      "FastEstimator-Eval: step: 954; epoch: 18; auc_score: 0.97444080032603; dloss: 2.4483051; f1_score: 0.9131776112825033; max_f1_score: 0.9788546255506608; rloss: 2.4951015; since_best_f1_score: 6;\n",
      "FastEstimator-Train: step: 1000; dloss: 1.7254086; rloss: 0.19289978; steps/sec: 8.57;\n",
      "FastEstimator-Train: step: 1007; epoch: 19; epoch_time: 5.89 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 115.95;\n",
      "Eval Progress: 11/17; steps/sec: 184.74;\n",
      "Eval Progress: 17/17; steps/sec: 213.9;\n",
      "FastEstimator-Eval: step: 1007; epoch: 19; auc_score: 0.9717246599002504; dloss: 1.33784; f1_score: 0.9040492957746478; max_f1_score: 0.9788546255506608; rloss: 0.684746; since_best_f1_score: 7;\n",
      "FastEstimator-Train: step: 1060; epoch: 20; epoch_time: 5.94 sec;\n",
      "Eval Progress: 1/17;\n",
      "Eval Progress: 5/17; steps/sec: 141.37;\n",
      "Eval Progress: 11/17; steps/sec: 212.39;\n",
      "Eval Progress: 17/17; steps/sec: 228.58;\n",
      "FastEstimator-Eval: step: 1060; epoch: 20; auc_score: 0.7852941062314424; dloss: 1.8164662; f1_score: 0.7163652404058226; max_f1_score: 0.9788546255506608; rloss: 1.6960378; since_best_f1_score: 8;\n",
      "FastEstimator-BestModelSaver: Restoring model from /tmp/tmp2w1fltl6/reconstructor_best_f1_score.h5\n",
      "FastEstimator-BestModelSaver: Restoring model from /tmp/tmp2w1fltl6/discriminator_best_f1_score.h5\n",
      "FastEstimator-Finish: step: 1060; discriminator_lr: 1e-04; reconstructor_lr: 0.0002; total_time: 212.81 sec;\n"
     ]
    }
   ],
   "source": [
    "estimator.fit()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inferencing\n",
    "\n",
    "Once the training is finished, we will apply the model to visualize the reconstructed image of the inliers and outliers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "idx0 = np.random.randint(len(x_eval0))\n",
    "idx1 = np.random.randint(len(x_eval1))\n",
    "\n",
    "data = [{\"x\": x_eval0[idx0]}, {\"x\": x_eval1[idx1]}]\n",
    "result = [pipeline.transform(data[i], mode=\"infer\") for i in range(len(data))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "network = fe.Network(ops=[\n",
    "    ModelOp(model=recon_model, inputs=\"x\", outputs=\"x_fake\"),\n",
    "    ModelOp(model=disc_model, inputs=\"x_fake\", outputs=\"fake_score\")\n",
    "])\n",
    "\n",
    "output_imgs = [network.transform(result[i], mode=\"infer\") for i in range(len(result))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAFsCAYAAADSVYN+AAAgAElEQVR4Xu2dB5hV1bWA18AwjaKgoKjEYEkk0SSAsWCNiopgLyiIQWyIElTsBgVRiVFBQbGCYlcsUVHy7CVY0IBGkog1iA1BxKFMgRnet871zNwZ78xe986duaf89/ve995z1t1n73/ts89/9zlnkbd+/fr1wgcCEIAABCAAAQiEiEAeAhOibNFVCEAAAhCAAAQ8AggMEwECEIAABCAAgdARQGBClzI6DAEIQAACEIAAAsMcgAAEIAABCEAgdAQQmNCljA5DAAIQgAAEIIDAMAcgAAEIQAACEAgdAQQmdCmjwxCAAAQgAAEIIDDMAQhAAAIQgAAEQkcAgQldyugwBCAAAQhAAAIIDHMAAhCAAAQgAIHQEUBgQpcyOgwBCEAAAhCAAALDHIAABCAAAQhAIHQEEJjQpYwOQwACEIAABCCAwDAHIAABCEAAAhAIHQEEJnQpo8MQgAAEIAABCCAwzAEIQAACEIAABEJHAIEJXcroMAQgAAEIQAACCAxzAAIQgAAEIACB0BFAYEKXMjoMAQhAAAIQgAACwxyAAAQgAAEIQCB0BBCY0KWMDkMAAhCAAAQggMAwByAAAQhAAAIQCB0BBCZ0KaPDEIAABCAAAQggMMwBCEAAAhCAAARCRwCBCV3K6DAEIAABCEAAAggMcwACEIAABCAAgdARQGBClzI6DAEIQAACEIAAAsMcgAAEIAABCEAgdAQQmNCljA5DAAIQgAAEIIDAMAcgAAEIQAACEAgdAQQmdCmjwxCAAAQgAAEIIDDMAQhAAAIQgAAEQkcAgQldyugwBCAAAQhAAAIIDHMAAhCAAAQgAIHQEUBgQpcyOgwBCEAAAhCAAALDHIAABCAAAQhAIHQEEJjQpYwOQwACEIAABCCAwDAHIAABCEAAAhAIHQEExpiyisq10mv/UxqNfu+FaZLfurWxxdyErfhhlex+2EiZPvEC2anndlnrRFT4ZA0IDUEAAhCAQLMSQGCMeNdVVcmkW2fWRN/3+PPSKi9Pjjts35r/Nnr4QGnVKs/YYm7CnnnhLTlv/M1y56QLsyowUeGTm6xwVAhAAAIQSJcAApMusR/jd+5/urRu3Upef/KmlC3MX/CRHH/mlXLhmYNk659vJjfPeFI++HiRlBQXyS69fiUXnDlIOm3Yvua7fvwZQw+TnXv9Sibe+rB88PHnUljQRvbu8zs5f8RxsuEG7Wri9W9HnnypHH9kX7lo5OA6fbjhjkfltnufqiMpn3+5RPoNvqDB0Z51ylFyyuABGdL46dca41NdvV522OdEGT38GPno0y/luVfflu222VJu/svZMnXGE/Lo069Il407yqRxZ8i23beoafyHlavljvueltkvvSVLl62QDTq0lV9s3U3OPPFw+d2vt/lJJ1avKZfrbn1YnnvlbdH/u8e2W3rH/PfC/8lfbrxfpl13vuzS+1d1vvfUs6/LA397QRZ+sljWr18vW26xiRx64O4e56DvrmUteTQEgRgT8H/kPXX3BNnqZ11jTCL4Q0dgMsyRS2AWLPxMBp42zrtALv7yWznntGPk5902lVfeeE8mT3tUeu3wC7lnysU1R9eL6jGnjZV9dusp//rvp3Lw/n2ka5eN5LW3/uX9T+/f/ELunlwbn67ArFy1Rp589nV57a335LW33pfBR/T1Ls7+p+f228ivfvHzDGmkJzAarQKz+aadpd8+O3uCd8EVt3pMttqyqxyw9+9lxEXXyy69esgtV4/2GleZGHzGFfLefz6RI/vvKdtvt5XomB564kX5dtn38sjtl8s23Tev05ERF03yeP+hT0/Zfecd5Ktvlsmjz7wqu/b+tcx+8S2Pvx7T//jit+uOv5b99/q95Ldu5X3/+df+KQftu7NcM+b0rPGhoeASGD1uqvz9pbk1HdRd1U4bdhCdF2efcrRs0rljcDvfxJ7p+fTCP+bJbdec28SWmu/rmp/ttvlZVn9wJfe2KQJzxfX3eD+AUn3uuPY8bw7xyR4BBCZDli6B8QVDf7U/OeOqOrIw7Oyr5a35/5XnH54oXbt08nrgx+v/raKiwqIf3a04afTVMnf+B3LvjZdIz+23rRNv3YHxh3nTnY97uxzZvoVUH6OLz+/6nixt8lvL609N9f73HoeNlOUrVsqcJ270dpqGjLxKFn3xjbz6+GSv6a+XfCfnX3GL/HLrn8mfzxpSc7j3//upHHv65TLw0H3k0rNPqPnvPs+de/aQ6ZNqd57eeOffcvK513hxyQKz6Isl0n/IhXJw3z4y4eK6zzpddNXt8uSzc2TGDRfJjr/9ZYYzhq+FhYBeIF95412ZOuEcr8v6fNeHny6W2++bJZt27iQzbxsrbdrkh2U4afXz/PG3yIrSVRkJjP7I0E9eXvPeRt/7yLNk8BH7BVJgPln0lXz1zXcehz+NmSw9f72NnHjsQd7/v8N23evsoqeVGIJTEkBgMpwYrgu0fwHts+P2cvu1dX/NXHvLQ3Lng7NTConuIjxx55V1evX47Nfkz1dPk3OHD5QTj+0XGYHZYbutanahDj3xEvmhdLW8/Oj13vjOvuxGeWnOfHn3+WkpM1RVVS363I1eXHYdMMK7LTdt4vk1sTNm/p/89aYHZOy5Q+XoAXvXaUNvvWl+kgVGb7npDsyNV40S7VfyR2/vnXXpjaK390YMPSzDGcPXwkJABeYfc9+Xt56+uU6XH3ziRRk/6W558OZLZYceiTmi0j3ptpkyd/5/Zcmy72WzTTaSYw75g/zx6APqXMh1vs6Y+XfRW5SLv/pW2rUtkZ179ZBRJx0pm226cc1x1q6rklvufsKLW7L0e2nfrkT22Pk33q3PjTttUBOnP4I6tG8r++3Z2xMrvWjq34efcIgcesBudfqtu7s33PGI/PvD/0lZWYW3E6w/fI44aM+auI8++0IOO/HPKVN001Vnebex/c+jT78ql117p7z+1E0y+4U3ZfqDs+WrJcu8H1t/u/OKmtu++iPlhKP293af/c/Lr78rZ1x8vTx062Wy/S+71/z3F16bJ3c9/Hf55H9ferutuht8xomH1dkhve+x5+Wqyfem7OPbs2/xbs/7H2teli3/QcZNnCGvv73Ak1Ldre25w7Yy7rq7pKm3kHY88FTZd/fecvWfT6vTZ5WcQ/54sYw/f5hcf/sjUlhYIFOu+JO3/rzz3kJvB/qKC06q+Y7+txvvfFwWfPCpqCNq/k49/mAvLvmjYx4/aYb8Y+4C7/GGfn/YSQ7Yeyc5afRf68xZ/c7/vTxX7p75rHz8I++dftdDzj19oNd2mD4ITIbZsgpM/Z0BPVyqZ1R84em/7y7y1zHD6/RKJ/AfR03wFsXzzzjO+1u6t5D8BoO0A7PHTjvIlCtHeV074qQxsnbtOm/R0I9eRJ595W15/8U7a1jomHWR012U774v9RY6/6M7U7pD5X/0GZd7Hnk25XMuumA9/ORLdQTmgitvlVnPvdHobDi83x51FpYMpw5fCziBhgTG/yFx301/9p65Utk4+pTL5IeVq2TcuSfKNj/fXF5+4z2ZMOVeOfPEI+S0IQfXjFTFR7+vF3PdxdML5zU3Pyhryirk8WnjpV3bYi/2wqtukxde+6dc/KfjZaeePeSzz7+WyyfOkKLCAnn0jstrdn5OPe9a7zmt3/xqaxk7eqgnMxOm3Cczn3pJZt39l5od3zVl5bLfwNHSd88d5YSjD/DaeWvef2TcdTNk4tgzZN89ennH1XNv8ddL5dxxU6VtSZGMO29YTd9116mkuLDm//d3PVWC9Fab/rDqvmVXWbW6zJMSfyxWgdGL6Tljp8qgw/eVAX37iO7fTHvgGXnlzffknimXeDsX+ildtcbbidW1QtfCow6u/WHSvdumNcKYTl5UBBd+ulgmXHSKbLXlZt7t+mn3Py1ff7u82QRGBfbAQefL7jvtIDeMHynHnDrWE+Hrxo7wbouPGjOl5ti6zvUdOFp2+/32MvKkI6WgTb488X9z5I77Z8mDN18mv/5l7W3/4RdMlHnvfyjjzz9Jfrl1N289U7affv51HWHU2+fnXn6zHHPw3t4PYuU6YfJ9nlg/NWOC92xhWD4ITIaZsgqM9RaPLySphMf/W/IFNAoCs/euv5PrLz+zRmDWVVXLk3cldp/qC8yHn34hA4eP86TlpOMO8n6Z6UKpv2yHjLzSu7WWLDBjr71LZs56WfyLTXKa9RfzHfc/XUdgRv55srz4j3nebajNNu2cclZs3KmD9yAwn2gT8AVmzpM3egOtqFgr73/wqYz563QpLir0hEN/4foXAn34fM9dflsDRX+9P/3Cm97tUP1V/83S5bLfMaO93bsRfzy0jpDfdu8sGXZcP+/C/8XXS+WA486T+g/Uv/7OAjnl3Gtl0rgzZf+9dvS+rwKjP2xefuwG6dCuxPtvn3/5rfQbfL5ceeHJctiBu3v/7b8fLZKjTrnsJ+eB/vdNOneq8yKBxuvtWG2vsWdgln63QvQ2jo7tybuukp9t3iXlhLAKjL5csGmXjt5tbf9TWblWLrzqdtm7z2/lkP1rd5T8cg2NvXRgzYs+E9f32HN/wvvSa6aL7jI11w7Ml98sk/2PPde7Fa5vsV484XaZ8/YCeeWxGzyx3euIUXLL1ed4O2/KQcVi4402lA3aJ8RCd55773+qnDns8JrbaP739EUMZeN/dOf4uVffqSMwA064yFs7H5g6pkb6VNgOPO68Om2G4SxHYDLMUnMJTKpf+W+/+4EMPesvMnTggXLe6cd6PW5MYPxbVKmecwnSDkw6AqMXhYefetn7pXvUgL1qsqZvJvU5+IyfCIxuNeuWcyoG/oN2ybeQ/OdceNAuwxMiQl+r/xCvDq1A3wbc9bdyyaghNbdyrpp8nzz4xAsy/9k7PKHxP/rgtz5ArreC9Zawf0FNJdPJ2FR69BmU5FtU+vfyikrpfcCpMuzYg7xbSb7AfLtshXfLxv/obsvv+w331ghdK/zvqtS0btVKjj74D95LBSpLyf1N7oNFYPQXu962dT3YbhEY/8JrfQvSIjDWvPi3s5KfOVQW3jrTjLeQfIHR3Wd9aUOPpS8nPDZtvLcbomyTZVWff7zn0Wflo0+/kJWr13g/4vR2+9BjDpTzRiSuB/r9QSPGe29yJsu0v2vo37Lz18szTjy8jkxrG/oSyeabbuwdOywfBCbDTDWXwNR/20i7509CvX2kW6f6+fizL0WfGzn20H1kTNLDq/o3fX7k2VfeSXnxDqvA6C9Q/SX63IPX1nlmQLd8deu0/g6MPheg95evuuiUnzwTcNyI8fKv/3xSZwfmrof+7m3p67b/n046ss6sKCuv9BaN5G30DKcNXwsBARWYV998T+68/kLvmYPRY2/ydivunnxRneda9Jezbuf/5IHe9eu920u3/nW0d5tAdwJ1R1Af5t96y80aJKBvAF0+6W6ZdfcE6V7v9d3f7DtMjuy/l1x2zh9rBEYvdio79QUm+Vk5/Zv+utZn7vTBZN3l0R0WfU5n5ElH/KQ0QDoCM+rkI71nMRr6WARGH54/6PgLvDVM1zLXxyIw1rz4wvjwrWPr3IrRWy96S7m5d2D0Dcs9dt7BE5gFC//nPRzuC4ze3tNnXHSXTX+86jzStUlv5+W1ypP+x1/o8fIFRufr6RdO+smLBi/OmS8jL7mhZgfGlycV2FataqVbua9bV+U926U7M2H5IDAZZqq5BEYnlm7L+g9T6YVT35p585//qbMNqAbe55AzvGJ0yVuvei91/2NHi150U+0++A+3Xn3JaTKg764Zjt79NRcfXdzS2YHRZwO8Gi1Tx3j3/fWji5lupetDtj222dLj43/8XSu9xz95/J9q/rveI9Y3nPSTvAOj99YPGHSet0078/Zx3kLhf66+6QHvmRm9RcUtJHfuwx5R/xkYfaD3tPOv8+ot6S3h5Hlx/2PPy2PTx6d882aTjTt6z5P4v/T1eY5eOyTeIkz18Xdq6u/A6HMyv+93mne7wL89oPPeKjDJx1KBefr5N+Xmu5+QgYf84Sc1pNIRmAvOOM57rqahT8/9T5EhR/at8xCvPjOjfP0dAX/XSH80JD8z1FCbFoHR89WSF//iri9Z6MsW/ue+x54T3cUJgsDo7SwVqjlP3iTFRQVeF31myTsw897/yLuV7kuRPxZfin3e+ozNLgNGyJCj9vcktv5Hn7HZomvqW+hBPK8RGGNW9L7jNVMfrInWiZHXqpX3IJT/ueCMQTWVeNN9RsWP1wXum6Xfy9ED9vLqwLz0+nz5v5ff9gxcf9Elf3RR1cVVt5X1LZyvlyyXKdMfk84bbejtVqQSGN150B2Ijhu0l0P27+MtsCo93363wnsSPtNPunzSFRjlcObFN3i/EPTNjbLyCrn9/qe9LVgVG12Y9Xka/buOTcXv2OGXi9bj0dtyurOlMX+b/Q/p9ZttRWs91K8Do8/F6PMxXTfZyNvp0iKCusOjC90+u6sIjWz2V0Qz5c/3skcg1UO8emtH66M8Pv2Kmmc+9Lw8Z+xNdaRae6H/XIeeD/5bQ3qrR39UnDSov4wcdkRNR3VuXvrX6d4tH60P4j+TcfapR8vJg/rXxPm/rqdOOFv22jXxrI1VYPT5if8t/sZ7niL5o7/KtbhjcokB/bvWWtIHfZPf6KtP1t8lcAmMPsuh5+dlo4fWNKEFOvUB3eS3kHQnuUO7tnXqYumzHyeMmiD77t6rzuvS+rCxrh3KUd+4SvWx5sUv7lm/LWWrz6QEQWB0J0jfkHrtb1NqhvrIrFe8t8CSBUZ32fY75hxv9zhZBHXN1LUzmfchQy/xynfUv57oA+PdNu8SqoKdCIxx3Uv33/rJVGC0wJw+ca6vzemrjSoYfffY0XvFzX+63++yPkynb9voboO+AbD1zzeXEUMPleXfrxQ1d30QT9uq/9HdBN2J0e1EvTeuhbn0odjkV/eMWGrC0uWTrsDoge599DnRX0f6eqneq9Vfw/rQsz5pr1vvuiCrhPmLtd5f12dhVPL0VoC+zaD/3IM+1Ka3mFI9k+C/Xrjwk8+9LdWfd+vq/VI5+uC9vXo1fKJPIJXAqOQPGHKh90yL1gPSWid6m+jY4eO8X8T64+UXW20hektEn0HT7fmHbrms5geNviH00JMveW/s6OvTKjW6U7C+uloeunVsze3JS/5yh3f7Vx8m17eV9DVXfWtOZej+m8bUtGcVGH09+azLpsiYs06Q3XbawfvnT97/4DO57JrpctqQQ2qelfGzqs+CPf/aOzL23BO9V8JLVyaeuUh+jdoqMPpg/L8Xfib33TRGdDfqzXn/9nY29EKZfEHVPmrNFL0lcli/Pbyu3H7fU96F+/6pl3pckz96oW7btljOHX6stz7qOqi7Bv4bOenkRXecFi3+xiu38LPNN/F+LOpbYPrmTiYCY6kD49/GsdxCuv/xF+TKG+7xfpzt3aenV1rioSdf9Iqj6np/45Wjap5n0h+mX3691HuIW4uU6s6NziW9jiTz9t/6UgE6rN/unrDMfmmu3Dzjb3L9uJE1b6aF4UxHYAKSJV949Kn05EJtAelepLqhNXX0uaKn7/lL6OoeRCoRAR2M6zXq5FtJutsy6faZ3m2iH0pXSeeNO8peu/zW2yFIfh1V35ab9sDT8vjsf8jXS5ZJp44dvNsWeksoub6LXnz1QqJVs7XCtO4mam2Sc4YfU/O2UTo7MBr72DOvil4ItTCkysgWXbvI4Qft4dVoqV90TneB9G2r9/7zsaxdWyWdN97Qq3x9+gm1b09ZBUYv1LpT8N6/P5GCgnzZueevvB1rrUtS/8eD/qjQ15f1bUPdAdp+u+4y6uSj6jyb4k8XLaOg8ve/xV97oqg/ZvSHSbJkWfOit47HTbzLKxSqO65aN2XPXX/rPTeiu2315ck1ZS2VeNMRGN1xuvKGe72SEtXr13tzRq8Ps198UybeOlO6bdal5kFubVd39OYt+EjalRR5RTlVglUkk+vz6BiUt/6zLB9+9oU37m27by4nDuzn7TSH6YPABCRbCEz2E6Fb1VqzQgtGaUEw/ehO0UGDL5Dyykp57fEpgf/HN7NPhRYhAIGoEtDnpZJfNvCf53l79q2RfAkBgQnITEZgsp8Iv3KqFnXSqqNa4VzfGtHKpMlvdGX/yLQIAQhAoGUJ6D8Ro1WctdRE79/80ttx0x0wfUPzustGtGxnWuhoCEwLgXYdBoFxEcrs7yos+uyMPsioD1bqvy476PD9vG1xPhCAAASiQkDXtxunP+4VUdTngvTWpL7pqa+7+zvQURmrPw4EJmoZZTwQgAAEIACBGBBAYGKQZIYIAQhAAAIQiBoBBCZqGWU8EIAABCAAgRgQQGBikGSGCAEIQAACEIgaAQQmahllPBAIEYHq6mqZP3++s8fbbLONM8YakJ+fbwotLy93xhUVFTljNGD16tXOOEu/Kisrne1oQJs2bZxx1raKi4udba1du9YZowHt27d3xmmtmmx9VqxY4WyqbdvEv/Ls+pSWlrpCTOPTRnTeuz4WDlbulnlaUVHh6pL395KSREmKxj7r1q1zhXh/d41Rc1NQkPgnFFJ9EBgTZoIgAIHmIIDA1FJFYBIsXBe1dOYhApOghcCkM2uIhQAEIGAggMAgMPWnCQJjFzl2YLI5WwwLFiEQgAAEfAIIDAKDwKReDyyXZgTGQon1FgIQgEAzEEBgEBgEBoFpaGnhGZhmWHRpEgIQyA4BBAaBQWAQGAQmO+sprUAAAi1IAIFBYBAYBAaBacFFl0NBAALZIYDAIDAIDAKDwGRnPaUVCECgBQkgMAgMAoPAIDAtuOhyKAhAIDsE9B2CxYsXOxvr2LGjM0YDLAXCWrdubWorLy/PGWeJ0UbKysqcbWWzYJyFw/fff+/skwa0a9fOGWcpnKeNFBYWOtuy9L2qqsrZjpW79T0WS786dOhg6pdl3lj6ZYkxdUhEsvlGk4WV9st1TH2It7E5QyE7a3aJgwAEsk4AgalFisAkWFgufghMghUCk00CWV/eaBACEIgyAQQGgak/vxEYu5xk8/Lt2g3x82Sp6mvJITswUV7ZGRsEYkAAgUFgEJjUJ7pFTiwx1mUEgbGSIg4CEIDAj1vgPAOTmArcQuIWUvKiYJETS4x1oUFgrKSIgwAEIIDA1JkDCAwCg8DUXRZ5iJfLBAQgEFgC3ELiFhK3kLiF1NAChcAEdummYxCAAAKDwCAwCAwCw7UAAhAIHQEEBoFBYBAYBCZ0SzcdhgAEEBgEBoFBYBAYrgUQgEDoCKjArFq1ytlvSwVXbcRS8bakpMR5PA1Yt26dM65Vq1bOGA347rvvnHGdOnVyxlj6pI1Y+mWtxGupLltRUeHsuwa0b9/eGWd5G8ZSi0QPtHLlSufxLKy0kUWLFjnb6t69uzNGAyorK51xljlvYaUHsjwgXlpa6uyTBljmQ3l5uamt/Pz8RuOUQWNVnqnEa8JMEAQg0BwEEJhaqghMgoXloozA2FkhMM2xctEmBCAQewIIDAJT/yRAYBJE2IFJMGAHJvaXCQBAIJgEEBgEBoFJfW4iMAhMMFdtegUBCHgEEBgEBoFBYBpaDtmB4UIBAQgElgACg8AgMAgMAhPYJZqOQQACDRFAYBAYBAaBQWC4RkAAAqEjgMAgMAgMAoPAhG7ppsMQgAACg8AgMAgMAsO1AAIQCB0BFRhLsbGCgoKsja2qqsrUlqXAmeWVXz1Y69atnce0HM/ZyI8BloJ3lqJ/2pyFV2Ovuib32fJmTV5ennOYVlZLly51tvXVV185YzSgS5cuzjj9xwctH0thOcsYLfNK+2NhapkzlrFZ54ylX8qpsXOfQnbWjBAHAQhknQACU4vUcsGyJsByMUJgEjQRmAQHy5yxzj+L9CIwVprEQQACgSSAwCAw9SemZbfAKnvswCToWpgiMIFcIukUBCAQVAIIDAKDwKQ+Oy2Sxi0kXUH4QAACEMgBAQQGgUFgEJiGlh6egcnBoswhIQABGwEEBoFBYBAYBMa2XhIFAQgEiAACg8AgMAgMAhOgRZmuQAACNgIIDAKDwCAwCIxtvSQKAhAIEAEEBoFBYBAYBCZAizJdgQAEbAQQGAQGgUFgEBjbekkUBCAQIAIqMGvWrHH2yPJKqTZSWVnpbMv66qmlym42KwRbx+gcoLEoWWlpqaUpsVTZzc/PN7VVVFTkjKuoqHDGWAulzZ8/39nWwoULnTEasNtuuznjNthgA2eMBljiLKws9V30eJa5VV5ebuq7JdfWmjKutrRyc2Pzj0q8ppQRBAEINAcBBCZ3OzAITII9ApPggMA0xwpHmxCAQGQJIDAITP3JzQ5Mggg7MCLswER26WdgEAg/AQQGgUFgUp/HCAwCE/4VnhFAIMIEEBgEBoFBYBpa4tiBifDiz9AgEHYCCAwCg8AgMAhM2Fdy+g+BGBJAYBAYBAaBQWBiuPgzZAiEnQACg8AgMAgMAhP2lZz+QyCGBBAYBAaBQWAQmBgu/pYhP/LII5YwOfroo51xvXr1csa88847zhgNsBZgMjVGUGgJqMBY609YBmkpoGUpUKfHqq6udh6ypQvZlZWVOfukAZbCZda+W4oDWlhpv9q3b+/s/+rVq50x1uMNGDDA2VZJSYkzRgOOOuooZ1yfPn2cMRrQtWtXZ5y1X86GjAHW4oCWQpB6Xls+rjy2bdtWGpunFLKzUA5xDAIT4uTFoOsITG2SLdKBwCR4uS58PlUExr6IIDB2VkS2EAEEpoVAc5iMCCAwCEz9icMOTIIIOzAi7MBktKxG50sITHRyGcWRIDAIDAKT+sxGYBCYKK75aY0JgUkLF8EtTACBQWAQGASmoWWHHZgWXpCDdjgEJmgZoT/JBBAYBAaBQWAQGK4LKQkgMEyMIBNAYBAYBAaBQWCCvErnsG8ITA7hc2gnAQQGgUFgEBgExrlUxjMAgYln3sMyagQGgUFgEBgEJiwrdgv3E4FpYXHaMfgAACAASURBVOAcLi0CCAwCg8AgMAhMWstmfIIvv/xy02DHjRvnjBsyZIgz5s4773TGaACVeE2YIh+kArN8+XLnOPVtBMvHUmXXOvcs9UiKi4st3TLFWArZVVRUmNqy8LK2VVhY6DzmmjVrnDEa0K5dO1OcK+j99993hXh/HzlypDNu5cqVzhgNOPzww51xBx54oDNGAzbbbDNn3CabbOKMsVSe1kYslXGt86GoqChr/XI1xFtILkIR/zsCE/EEh3x4CExtAhEY+2RGYBKsEBiLmtnnFZEBI4DABCwhdKcOAQQGgcnklEBgEBglwL+FlMnZE6LvIDAhSlYMu4rAIDCZTHsEBoFBYDI5c0L2HQQmZAmLWXcRGAQmkymPwCAwCEwmZ07IvoPAhCxhMesuAoPAZDLlERgEBoHJ5MwJ2XcQmJAlLGbdRWAQmEymPAKDwCAwmZw5IfsOAhOyhMWsuwgMApPJlEdgEBgEJpMzJ2TfQWBClrCYdReBQWAymfIIDAKDwGRy5oTsOw8++KCpx4MHD3bGLVq0yBmzxRZbOGMIgIBPQAVmxYoVTiAlJSXOGA0oKytzxllrZzgbEpGCggJLmFRVVTnjWrdu7YyxFOrTRizF51atWuU8ngZYiuJZq3FYiqBZ5sP06dNNfX/iiSeccUuXLnXGaED//v2dcWeffbYzxrvw5uU54zp37uyMsc5lS34sMdohS1x1dbWz7xrgOn90vrRp06bBtniN2oQ5vEEITHhzF4eeIzC1WUZgEiwQmAQHBEYEgYnDVaCRMSIwMZ8AAR8+AoPA1J+iCAwC488JBCbgC3hzdw+BaW7CtN8UAggMAoPApD6D2IFhB6Ypa2skvovARCKNkR0EAoPAIDAITEMLHDswkV36bQNDYGyciMoNAQQGgUFgEBgEJjfrb+CPisAEPkWx7iACg8AgMAgMAhPry0DDg0dgmBhBJoDAIDAIDAKDwAR5lc5h3xCYHMLn0E4CCAwCg8AgMAiMc6mMZ8AjjzxiGvjAgQOdcVOmTHHGjBgxwhlDAAR8AiowFRUVTiCWQnDORn4MyGZb1mNaxlhcXOxsztr3/Px8Z1vWoniW4nPWwmWW/i9evNjZ9zPPPNMZowELFy50xlmLbw4ZMsTZ1qBBg5wxGmApNGipC2Qp3KjHs+THkhtty1LY0Dq3GitS53NqbC5TyM403cIbhMCEN3dx6DkCU5tlBCbBAoFJcEBgEqKHwMThStDAGBGYGCc/BENHYBCY+tMUgUFg/DmBwIRgEW/OLiIwzUmXtptKAIFBYBCY1GcROzDswDR1fQ399xGY0Kcw0gNAYBAYBAaBaWiRYwcm0su/e3AIjJsREbkjgMAgMAgMAoPA5G4NDvSREZhApyf2nUNgEBgEBoFBYGJ/KUgNAIFhYgSZAAKDwCAwCAwCE+RVOod9Q2ByCJ9DOwkgMAgMAoPAIDDOpTKeAQhMPPMellEjMAgMAoPAIDBhWbFbuJ/ZFBhLRc5Vq1aZRpiXl2eKIyjaBFRgKisrnYNct26dM0YDLNVErXPPEmepcKr9slQmtbw2u2bNGhOHkpISZ5zleNqI5sj1sawN2kZ5ebmrKRkzZowzxlJhVxtZsGCBs6299trLGaMBhxxyiDNun332ccZogKWarWUuWyvxWnNt6XxBQYElzBTjmlta3LGxar1U4jVhDm8QAhPe3MWh5whMbZYtFxkEJsELgUlwQGBcChSHVTTCY0RgIpzcCAwNgUFg6k9jdmASRNiBEWEHJgKLfFOGgMA0hR7fbW4CCAwCg8CkPssQGASmudffwLePwAQ+RbHuIAKDwCAwCExDiyA7MLG+PIggMDGfAAEfPgKDwCAwCAwCE/CFOlfdQ2ByRZ7jWgggMAgMAoPAIDCW1TKGMQhMDJMeoiEjMAgMAoPAIDAhWrRbsqsITEvS5ljpEkBgEBgEBoFBYNJdOWMSv3z5ctNIO3fubIpzBS1btswV4v29Y8eOpjiCok1ABaa0tNQ5SGvxLEtdDEu9Fe2QpXheYWGhs+8aYClk16pVK2dbFRUVzhgN6NChgzPOMj5tJD8/39mW5Y0ZbcQSN2zYMOfx3njjDWeMBmy11VbOuOOPP94ZowF9+/Z1xm222WbOGA1YvXq1M66xAm7+l7NZF8jaVvv27Z19t84t15zXAokUsnPijm4AAhPd3EZhZAhMbRZdi7lGIjAJXghMgoNVOiyVma1tITBRWHlDMgYEJiSJimk3ERgEpv7UZwcmQYQdGBF2YGJ6YfCHjcDEfAIEfPgIDAKDwKQ+SREYBCbgy3fzdw+BaX7GHCFzAggMAoPAIDANrSDswGS+tkbimwhMJNIY2UEgMAgMAoPAIDCRXeKbNjAEpmn8+HbzEkBgEBgEBoFBYJp3nQ1t6whMaFMXi44jMAgMAoPAIDCxWO7THyQCkz4zvtFyBBAYBAaBQWAQmJZbc0N1JAQmVOmKXWcRGAQGgUFgEJjYLf22AesFwvLZc889nWGvv/66M2bmzJnOGA044ogjTHEERZuAzk9rAS0LiWxVvNVjWc4da1Xf6upqZ/fLy8udMdYAfXvD9bFU2NU28vLyXE2ZKg1rI0uWLHG2NWbMGGfMnDlznDEasPHGGzvjpk2b5ozRgC5dujjjLAXjrEyzWdjQUsk6W9VzdXyWvls4FBcXS2N9z1tvOUudaSMgqASs6UVggprBaPcLganNLwKTYIHAJDhYJMBamRmBifY6GtnRITCRTW0kBobAIDD1JzICg8D4c4IdmEgs85kPAoHJnB3fbH4CCAwCg8CkPs/YgRFBYJp/DQ70ERCYQKcn9p1DYBAYBAaBaWghRGBifolAYGI+AQI+fAQGgUFgEBgEJuALda66h8DkijzHtRBAYBAYBAaBQWAsq2UMYxCYGCY9RENGYBAYBAaBQWBCtGi3ZFcRmJakzbHSJYDAIDAIDAKDwKS7chJfh8CNN97oJDJq1ChnzMSJE50xGmBpy9QQQaEmoAJTVlaWtTFYaqlYC7hZCuzpQ4aWj+WNEkshMesPksLCQme3rG1VVVU527KMTxuZN2+es61x48Y5Yz7++GNnjAZY6ltddNFFprYstVS6du1qastScNHSkCU32k6bNm2czVVWVjpjNMDCwTq3XOeizuPG+k4hO1PKoh+EwEQ/x0EcIQJTmxUEJsECgbGfqQiMVZXsTIkMIQEEJoRJi0CXERgEpv40RmDsJzYCg8DYZ0uEIxGYCCc3wENDYBAYBCbzExSBQWAynz0R+iYCE6FkhmgoCAwCg8BkfsIiMAhM5rMnQt9EYCKUzBANBYFBYBCYzE9YBAaByXz2ROibCEyEkhmioSAwCAwCk/kJi8AgMJnPngh9E4GJUDJDNBQEBoFBYDI/YREYBCbz2ROhbyIwEUpmiIaCwCAwCEzmJywCg8BkPnsi9E0EJkLJDNFQEBgEBoHJ/IRFYBCYzGdPhL75+eefO0fTvXt3Z0zv3r2dMRowd+5cUxxB0Sagy09FRYVzkNaFevXq1c62XNU//Qas1WWdBxQRS5G6bB7PUi21urra0nVTjLWy7IwZM5ztTZ482RljqZKsjQwdOtTZ1jnnnOOM0QDLGNu1a2dqKy8vzxlnOZ71vGjdurXzeNYAy/mTrbmsla4bm8tU4rVmLeJxCEzEExzQ4SEwtYnJ1qKvLSIwCa4ITIIDAhPQBZBuZYcAApMdjrSSHgEEBoGpP2PYgUkQYQdGhB2Y9NbT2EYjMLFNfU4HjsAgMAhM6lMQgUFgcro4h+ngCEyYshWdviIwCAwCg8A0tKKxAxOdtb5ZR4LANCteGm+AAAKDwCAwCAwCwyWiSQQQmCbh48sZEkBgEBgEBoFBYDJcQPlaggACw0zIBQEEBoFBYBAYBCYXq2+EjonARCiZIRoKAoPAIDAIDAITokU7iF1FYIKYlej3SQVm5cqVzoFa61gsX77c2VanTp2cMRpQWVnpjLPWbikvL3e2VVRU5IyxvJmijbRp08bZlpWpsyER+fDDDy1hYnlFetasWc629OFOy2fSpEnOsP79+ztjNMBScNE6Hyx5tOTHUiDROh8s49O2CgsLnbys/XKNsaSkhEJ2TtoEcAuJOZATAghMLXYEJsECgUlwcF3cNcYqChahRWBysgRy0GwQYAcmGxRpI10CCAwCU3/OIDAIjD8n2IFJd0WNaTwCE9PE53jYCAwCg8CkPgnZgRFBYHK8QIfl8AhMWDIVrX4iMAgMAoPANLSqITDRWu+bbTQITLOhpeFGCCAwCAwCg8AgMFwmmkQAgWkSPr6cIQEEBoFBYBAYBCbDBZSvJQggMMyEXBBAYBAYBAaBQWBysfpG6JgITISSGaKhIDAIDAKDwCAwIVq0g9hVBCaIWYl+n1RgysrKnAOtqqpyxmjA6tWrnXGWQlzaiLUomfOAxiJollod1dXVlsOZ+p6fn29q67vvvnPGvfzyy84YDbj66qudcaWlpc6Yrbfe2hmjAcOHD3fG9evXzxmjATpXXR9LDrUNS2FDy/yzxLj67P/dWlOmoKDA2aS1LdebVjzE60RNgBJAYJgHuSCAwNRSt1z8EJgELwQmwQGBsWhlLlY2jtmiBBCYFsXNwX4kgMAgMPVPBnZg7HKCwCAwXEzYgWEO5IgAAoPAIDCpTz6LnFhirKe29bYPt5CsRIlrMQLswLQYag6URACBQWAQGASmoUWRZ2C4XJgIIDAmTARlmQACg8AgMAgMApPlhTVuzSEwcct4MMaLwCAwCAwCg8AEYz0ObS8QmNCmLtQdR2AQGAQGgUFgQr2M577zCEzucxDHHiAwCAwCg8AgMHFc/bM4ZgQmizBpykwAgUFgEBgEBoExL5kEpiKAwDAvckHAKjDWvlVWVjpDXdU//QYsbVleKdX21q5d6+yXvnHh+lj6pG1Y4qyv4Foqbdx9992urnt/v+2225xxRUVFzpiNNtrIGaMBlsq/PXr0MLVlyaF1Plgq8VrmqaVPOjhLri1zRtuy5MdSXVvbKi4ubpS9/r0xpnnrLbPTlF6CwkwAgQlz9sLbdwSmNncITIKF5QKJwCRYITAITHhX/yz2HIHJIkyaMhNAYBCY+pMFgUkQYQcmsUPDDox5OY1vIAIT39zncuQIDAKDwKQ+AxEYBCaXa3Oojo3AhCpdkeksAoPAIDAITEMLGjswkVnqm3cgCEzz8qX11AQQGAQGgUFgEBiuEE0igMA0CR9fzpAAAoPAIDAIDAKT4QLK1xIEEBhmQi4IIDAIDAKDwCAwuVh9I3RMBCZCyQzRUBAYBAaBQWAQmBAt2kHsKgITxKxEv09WgamurjbBsNTFsBT10oNZinHl5+eb+lVVVeWMcxX10gYs43Me6MeAZ555xhT64YcfOuMWLFjgjNGAf/3rX864bbfd1hlz2GGHOWM04KCDDnLGdenSxRmjARUVFc4469yyVC+x5Np6XljeaLLMUQVgactaFM9V+0jPiTZt2jTInUJ2zikZjwAEJh55DtooEZjajCAwCRYITIIDApN4jRqBCdqqHcD+IDABTEoMuoTAIDD1pzkCg8D4cwKBicFFIBtDRGCyQZE20iWAwCAwCEzqs4YdGHZg0l1PYxuPwMQ29TkdOAKDwCAwCExDixA7MDldnsNzcAQmPLmKUk8RGAQGgUFgEJgoreo5GAsCkwPoHFIQGAQGgUFgEBguBk0igMA0CR9fzpAAAoPAIDAIDAKT4QLK1xIEEBhmQi4IIDAIDAKDwCAwuVh9I3RMBCZCyQzRUBAYBAaBQWAQmBAt2kHsqqXaZo8ePZxdP+2005wxGjB16lRTHEHRJqAC88UXXzgH2alTJ2eMBqxZs8YZV1RU5IzRAEs1UUtFVW3LUjG1bdu2zn5Z2tFGVqxY4WzrnXfeccZogCVuzpw5prZWrVrljLOsIQcffLCzHQ1Yvny5M65bt27OGA1Yt26dM846tyyvSOfl5TmPZ2lHG7FUCLbMd20rmwUXGytSp8dSnhSyc04DAhAY5kAuCCAwtdQRmAQLBCbBAYFBYHKxJofymAhMKNMW+k4jMAhM/UmMwCAw/pxgByb0S3zLDACBaRnOHKUuAQQGgUFgUq8K7MCwA8P1wkgAgTGCIiyrBBAYBAaBQWAaWlTYgcnqchvdxhCY6OY2yCNDYBAYBAaBQWCCvEqHoG8ITAiSFMEuIjAIDAKDwCAwEVzcW3JICExL0uZYPgEEBoFBYBAYBIZrQpMIIDBNwseXMySAwCAwCAwCg8BkuIDytQSBt99+24lil112ccb07t3bGaMBc+fONcURFG0CKjClpaXOQebn5ztjNMBSbMxafM4SV1FRYepXYWGhKc4VZOmTtvH999+7mpKnnnrKGaMBs2fPdsYtXrzYGaMBvXr1csbtvvvuzphjjjnGGaMBluJsJSUlprYs7C1vDunBqqqqnMd0FXnTBqzzz1LIztInPWZBQYGz79aCi66GNDeNHS9vvSUrrqPw99ATQGBCn8JQDgCBSS9t1uUagUlwRWASHBCY9M4zokNGAIEJWcIi0l0EJr1EIjAJXuzAJDiwA2M9I9I7z4gOGQEEJmQJi0h3EZj0EmldrtmBYQcmeWaxA5PeeUZ0yAggMCFLWES6i8Ckl0gEhh2Y5BnDDoz1jEjvPCM6ZAQQmJAlLCLdRWDSS6R1uWYHhh0YdmDSO7eIDjEBBCbEyQtx1xGY9JKHwLADww5MLQHeQkpv/YhsNAIT2dQGemAITHrpQWAQGAQGgUlv1YhBNAITgyQHcIgITHpJQWAQGAQGgUlv1YhBNAITgyQHcIgITHpJQWAQGAQGgUlv1YhB9Oeff+4cZffu3Z0xVOJ1IiIgiYBekJctW+Zk0q5dO2eMBliqiVpeKdW2LEXQrBWCLW+LWPq1Zs0aE4d3333XGTdt2jRnjAbMmzfPGTdgwABnjAb07NnTGbfrrrs6Y7p16+aM0QDLw8wbbrihqa21a9c646wVl8vLy51tWSreWvqkB7JU9bWcO9qWZc5bKmJrW64xKs/GjsczMM5pFI8ABCYeeQ7aKBGY2owgMAkWCEyCg+virjEIjHVPMmgrH/3JKgEEJqs4acxIAIFBYOpPFQQGgfHnBDswxoU07mEITNxnQG7Gj8AgMAhM6nOPHRgRBCY363LojorAhC5lkegwAoPAIDAITEOLGQITiWW++QeBwDQ/Y47wUwIIDAKDwCAwCAxXhyYRQGCahI8vZ0gAgUFgEBgEBoHJcAHlawkCCAwzIRcEEBgEBoFBYBCYXKy+ETomAhOhZIZoKAgMAoPAIDAITIgW7SB2FYEJYlai3ycVmLKyMudArUW2LAW0LPVWtEN5eXnOflkDLAXoLMXNli5dajqkpbL2Aw88YGrLUtCvX79+praGDRvmjCsqKnLGFBcXO2M0wFLAzdSQiFjyY61KUl1d7TysZZ5az4vWrVs7j2cNsPTLUuxOj+fipQ/xNpZDCtlZsxbxOAQm4gkO6PAQmNrEWC6QCEyCFwKT4IDAuBQooAsf3couAQQmuzxpzUYAgUFg6s8UdmASRCw7HQgMAmNbaSMehcBEPMEBHR4Cg8AgMKlPTgQmUciOW0gBXbyD1C0EJkjZiE9fEBgEBoFBYBpa8RCY+FwLmjRSBKZJ+PhyhgQQGAQGgUFgEJgMF1C+liCAwDATckEAgUFgEBgEBoHJxeoboWMiMBFKZoiGgsAgMAgMAoPAhGjRDmJXEZggZiX6fUJgEBgEBoFBYKK/1jfrCBGYZsVL4w0QUIGx1DZp27atieHKlSudce3atXPGaMDatWudcdYCYaWlpc62LEXXPvvsM2c7GjB37lxn3PPPP++M0QB9kNL12XPPPV0h3t8HDx7sjFuyZIkzpnv37s4YDbC8ZGspKqdt/fDDD85jtm/f3hmjARUVFc44C3fLHNUDWeappQ6RtmU5F639ckHQYxUUFDQYRiE7F8GY/B2BiUmiAzZMBKY2IQhMggUCk+CAwCRkCYEJ2KIdxO4gMEHMSvT7hMAgMPVnOQKDwPhzAoGJ/jUgKyNEYLKCkUbSJIDAIDAITOqThh0YdmDSXE7jG47AxDf3uRw5AoPAIDAITENrEDswuVydQ3RsBCZEyYpQVxEYBAaBQWAQmAgt6rkYCgKTC+ocE4FBYBAYBAaB4VrQJAIITJPw8eUMCSAwCAwCg8AgMBkuoHwtQQCBYSbkggACg8AgMAgMApOL1TdCx0RgIpTMEA0FgUFgEBgEBoEJ0aIdxK4iMEHMSvT7pAJjqVJrrZZqibO8nqrkLdVELcfTtixF6tatW+dM+LJly5wxGjB//nxn3KxZs5wxGrDffvs54yx910YOOOAAZ1slJSXOGGsOLdzLysqcx7POh1atWpnaslTGzcvLc7ZlqTTsbOTHgDVr1phCLZV4KysrTW25OPAWkgkjQQgMcyAXBBCYWuoWCUBgErwQmAQHBCabBHKxAnLMrBBAYLKCkUbSJIDAIDD1pww7MAki7MBQyC7N5TS+4QhMfHOfy5EjMAgMApP6DERgEJhcrs2hOjYCE6p0RaazCAwCg8AgMA0taDwDE5mlvnkHgsA0L19aT00AgUFgEBgEBoHhCtEkAghMk/Dx5QwJIDAIDAKDwCAwGS6gfC1BAIFhJuSCAAKDwCAwCAwCk4vVN0LHRGAilMwQDQWBQWAQGAQGgQnRoh3ErlqKifXo0cPZ9UMPPdQZowFTp041xREUbQIqMCtXrnQO0lXwym/AUnwuPz/feTwNsNRlsbZVVVVlOqYraPXq1a4Q7++WQmLWtrp16+Y8poW7NtKuXTtnW8uXL3fGWKt/dOjQwdmWte+WooWWIm/ZnFsVFRXO8WmApcCe5RpgzaF1bhUXFzfafx7iNaWXIMvkRWCYJ9kmgMCkR9R6YUBgElwRmAQHBCa984zokBFAYEKWsIh0F4FJL5EITIIXOzAJDuzAWGdCeucZ0SEjgMCELGER6S4Ck14iERgEJnnGIDAITHorSESjEZiIJjbgw0Jg0ksQAoPAIDC1BPLWIzDprSARjUZgIprYgA8LgUkvQQgMAoPAIDDprRoxiEZgYpDkAA4RgUkvKQgMAoPAIDDprRoxiEZgYpDkAA4RgUkvKQgMAoPAIDDprRoxiEZgYpDkAA4RgUkvKQgMAoPAIDDprRoxiEZgYpDkAA4RgUkvKQgMAoPAIDDprRpEQwACzULAKjCWKqjaQUvBLmtVX0slXiuUvLw8Z6ilqq+VgyXOcjzttKWKsLUtJwRjFWHr68OWd1SsfW/Tpo2z+5YYbcQyHyzzz1pF2HJeWI7nBPBjgGX+aairEm9hYaE0xpS3kKwZIQ4CEMg6AQSmFqnlQmq9MFjiLMdDYGrzY5ETSwwCU8sUgcn6kkqDEIBASxFAYBCY+nPN8s8gsAOToMYOjGWPraVWM44DAQjEigACg8AgMKlPecstHQQGgYnVBYPBQiBIBBAYBAaBQWAaWpN4BiZIqzV9gQAE6hBAYBAYBAaBQWC4MEAAAqEjgMAgMAgMAoPAhG7ppsMQgAACg8AgMAgMAsO1AAIQCB0BBAaBQWAQGAQmdEs3HYYABLReyT//+U8niK222soZowFFRUXOOGutDssbHtaieGVlZc5+tW3b1hljecXYysF5sDQCli9fboq28LLUsCkpKTEdb+XKlc64DTfc0BmjAZa3gqxzy/IauOX9GktBPNPgRMTSJ23LMk/Ly8tNh3WdrwUFBdJYvSIK2ZkwEwQBCDQHAQSmlqrlwoDAJHghMAkOCIxF85pj5aJNCEAg9gQQGASm/knADkyCiOXSjMBYKMV+mQUABCDQHAQQGAQGgUl9ZlkuzQiMhVJzrFy0CQEIxJ4AAoPAIDAITEMLIc/AxP4SAQAIBJcAAoPAIDAIDAIT3DWankEAAg0QQGAQGAQGgUFguERAAAKhI4DAIDAIDAKDwIRu6abDEIAAAoPAIDAIDALDtQACEAgdAQQGgUFgEBgEJnRLNx2GAAT0JchFixY5QVgLl1kqoeqbDZZPVVWVM8xyPG3EUtXXcjxX5VK/w9mqeKvtWaoIO0H9GGDhZeFgfX3Ywt3K1FJE0DI+RWF5+bdVq1ZOrKWlpc4YDbDMBytTCy8Ld+1XY1V29e/FxcXSGFMq8ZrSTxAEINAcBBCYWqqWC7fl4mG9YFkKxiEwtflBYBIsLHMQgWmO1ZI2IQCBQBFAYBCY+hPSInLW3QLLhdRyQdY+IjAITKAWTzoDAQjklgACg8AgMKnPQW4hcQspt6szR4cABBolgMAgMAgMAtPQIsEzMFxAIACBwBJAYBAYBAaBQWACu0TTMQhAoCECCAwCg8AgMAgM1wgIQCB0BBAYBAaBQWAQmNAt3XQYAhBAYBAYBAaBQWC4FkAAAhCAAAQgEBsCFLKLTaoZKAQgAAEIQCA6BBCY6OSSkUAAAhCAAARiQwCBiU2qGSgEIAABCEAgOgQQmOjkkpFAAAIQgAAEYkMAgYlNqhkoBCAAAQhAIDoEEJjo5JKRQAACEIAABGJDAIGJTaoZKAQgAAEIQCA6BBCY6OSSkUAAAhCAAARiQwCBiU2qGSgEIAABCEAgOgQQmOjkkpFAAAIQgAAEYkMAgYlNqhkoBCAAAQhAIDoEEJjo5JKRQAACEIAABGJDAIGJTaoZKAQgAAEIQCA6BBCY6OSSkUAAAhCAAARiQwCBiU2qGSgEIAABCEAgOgQQmOjkkpFAAAIQgAAEYkMAgYlNqhkoBCAAAQhAIDoEEJjo5JKRQAACEIAABGJDAIGJTaoZKAQgAAEIQCA6BBCY6OSSkUAAAhCAAARiQwCBiU2qGSgEIAABCEAgOgQQmOjkkpFAAAIQgAAEYkMAgYlNqhkoBCAAAQhAIDoEEJjo5JKRQAACEIAABGJDAIGJTaoZKAQgAAEIQCA6BBCY6OSSkUAAAhCAAARiQwCBiU2qGSgEIAABCEAgOgQQmOjkkpFACZZ1UwAAALlJREFUAAIQgAAEYkMAgYlNqhkoBCAAAQhAIDoEEJjo5JKRQAACEIAABGJDAIGJTaoZKAQgAAEIQCA6BBCY6OSSkUAAAhCAAARiQwCBiU2qGSgEIAABCEAgOgQQmOjkkpFAAAIQgAAEYkMAgYlNqhkoBCAAAQhAIDoEEJjo5JKRQAACEIAABGJDAIGJTaoZKAQgAAEIQCA6BBCY6OSSkUAAAhCAAARiQwCBiU2qGSgEIAABCEAgOgT+H3XuVrn+eAESAAAAAElFTkSuQmCC",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjAAAAFsCAYAAADSVYN+AAAgAElEQVR4Xu2dB3hVVbaAdwIkIaGrFLGMXUZxrKAogqOC2EURuyJ2cRBRsDGABexdBBEVFURREcvYsY1Ynw7qWEdFERVBWkJJAuF961xvchNustbt95z7n+9733vPrLP3Pv/eZ5//7rPPIm/dunXrHAcEIAABCEAAAhDwEYE8BMZHvUVTIQABCEAAAhDwCCAwDAQIQAACEIAABHxHAIHxXZfRYAhAAAIQgAAEEBjGAAQgAAEIQAACviOAwPiuy2gwBCAAAQhAAAIIDGMAAhCAAAQgAAHfEUBgfNdlNBgCEIAABCAAAQSGMQABCEAAAhCAgO8IIDC+6zIaDAEIQAACEIAAAsMYgAAEIAABCEDAdwQQGN91GQ2GAAQgAAEIQACBYQxAAAIQgAAEIOA7AgiM77qMBkMAAhCAAAQggMAwBiAAAQhAAAIQ8B0BBMZ3XUaDIQABCEAAAhBAYBgDEIAABCAAAQj4jgAC47suo8EQgAAEIAABCCAwjAEIQAACEIAABHxHAIHxXZfRYAhAAAIQgAAEEBjGAAQgAAEIQAACviOAwPiuy2gwBCAAAQhAAAIIDGMAAhCAAAQgAAHfEUBgfNdlNBgCEIAABCAAAQSGMQABCEAAAhCAgO8IIDC+6zIaDAEIQAACEIAAAsMYgAAEIAABCEDAdwQQGN91GQ2GAAQgAAEIQACBYQxAAAIQgAAEIOA7AgiM77qMBkMAAhCAAAQggMAwBiAAAQhAAAIQ8B0BBMZ3XUaDIQABCEAAAhBAYBgDEIAABCAAAQj4jgAC47suo8EQgAAEIAABCCAwjAEIQAACEIAABHxHAIHxXZfRYAhAAAIQgAAEEBjGAAQgAAEIQAACviOAwPiuy2gwBCAAAQhAAAIIDGMAAhCAAAQgAAHfEUBgjF1WXlHpdu11ZoPRc16b5Bo3amQsMTNhS5eVuX2OvMDdf8tw12WX7ZPWiKDwSRoQCoIABCAAgZQSQGCMeNesXetunTC9OnrKjFddfl6eO/7I/av/29Bz+rv8/DxjiZkJ+9dr77tLrr7HPXDrpUkVmKDwyUyvUCsEIAABCMRKAIGJldif8V0POdc1apTvZj9zd9QSPvn8W3fSoGvdpYNOcFv9ZWN3z+Rn3Ff/+9EVNy1ye+76Vzd80AmuTavm1eeG488/7UjXdde/ulsmPO6++t9PrrCgievZbWc37LzjXauWzarj5W9Hn/FPd9LRB7rLLjixVhtuv+9Jd+8jz9aSlJ/mL3B9Thxe79VeeOYx7swTD42TxvqnNcSnqmqd6/z3AW7oOce6b7+f715560O3/dabu3uuG+LGTZ7pnnz+Tdd2w9bu1tHnu2222KS68GWlK9x9U553L7z+vlu4aKlr2aLEbbvVpm7QgKPczjtsvV4jVqxc7W6e8Lh75c0PnfzfnbbZ3Kvzv1/PddfdNdVNunmY23O3v9Y679mXZ7tHn37Nff3dPLdu3Tq3+Sbt3BEH7eNxzvbVtaR1HgVBIIcJhH/kPfvQWLflZh1ymET2XzoCE2cfaQLz+dc/uP5nj/YekPPm/+4uOvtY95dN27s3353j7pj0pNu187bu4Tsvr65dHqrHnj3K/X3vXdynX37vDuvVzXVou4F7+/1Pvf/Zbadt3UN31MTHKjClZSvdMy/Pdm+/P8e9/f5n7sS+B3oP5/Cxy45bu79u+5c4acQmMBItAtOx/Uauz9+7eoI3/JoJHpMtN+/gevfcw5132W1uz107ufHXD/UKF5k48fxr3JwvvnNHH7Kv23H7LZ1c02MzZ7nfFy1xT0y8ym29RcdaDTnvsls93vt128Xt07Wz++W3Re7Jf73l9tptB/fCrPc9/lJn+AiL31677+B69djDNW6U753/6tv/5w7ev6u7ccS5SeNDQdlLYOjoce7F1z+obqCsqrZp1cLJuBhyZj/XbqPW2dv4BFsm99Nr//7Y3XvjxQmWlLrTpX+233qzpP7gimxtIgJzzW0Pez+Aoh333XSJN4Y4kkcAgYmTpSYwYcGQX+3PTB5TSxZOH3K9e/+TL92rj9/iOrRt47UgHC//t4iKCIscsloxcOj17oNPvnKP3HWF22XHbWrFW1dgwpd59wMzvFWOZL9CqotR47PzgWe4Jo0budnPjvP+d/cjL3CLl5a6d2be5a00nXzBGPfjz7+5t2bc4RX964I/3LBrxrvtttrMXXnhydXVffbl9+64c69y/Y/4u/vnkFOq/3uYZ9ddOrn7b61ZeXr3o/+6My6+0YuLFJgff17gDjn5UnfYgd3c2Mtr73W6bMxE98zL77jJt1/mdv/bdnGOGE7zCwF5QL757n/cuLEXeU2W/V3ffD/PTZzynGu/URs3/d5RrkmTxn65nJjaOezq8W7p8rK4BEZ+ZMiRl5fa1+g9j77Qndj3gKwUmO9+/MX98tsfHod/jLjD7bLD1m7AcQd7/3/n7beotYoeU8cQHJUAAhPnwNAe0OEHaLfdd3QTb6r9a+am8Y+5B6a9EFVIZBVh5gPX1mrVjBfedldeP8ldfE5/N+C4PoERmM7bb1m9CnXEgCvcsuUr3BtP3uZd35CRd7nX3/nE/efVSVF7aO3aKif7buThsteh53mv5SbdMqw6dvL0l9wNdz/qRl18mut3aM9aZcirN+mfSIGRV26yAnPXmMFO2hV5yOu9C/95l5PXe+eddmScI4bT/EJABObfH3zm3n/+nlpNnjZzlrv61ofctHv+6Tp3Co0Rke5b753uPvjkS7dg0RK3cbsN3LGH7+dO7de71oNcxuvk6S86eUU575ffXbOSYtd1105u8MCj3cbtN6yup3LNWjf+oZle3IKFS1zzZsWue9edvFefG7ZpWR0nP4JaNC9xB+y7mydW8tCUv59zyuHuiN5712q3rO7eft8T7r/fzHWrVpV7K8Hyw6fvwftWx337w8/uyAFXRu2iu8dc6L3GDh9PPv+WG3nTA272s3e7F157z90/7QX3y4JF3o+tpx+4pvq1r/xIOeWYXt7qc/h4Y/Z/3PmX3+YemzDS7bjdFtX//bW3P3YPPv6i+27ufG+1VVaDzx9wZK0V0ilPverG3PFI1DZ++MJ47/V8+LD2y6LFy9zoWya72R9+7kmprNbu0nkbN/rmB12ir5B2P+gst/8+u7nrrzy7VptFcg4/9XJ39bDT3W0Tn3CFhQXuzmv+4c0/H8352luBvmb4wOpz5L/d9cAM9/lX3ztxROm/s046zIuLPOSar751svv3B5972xv67NfF9e7ZxQ0cekOtMSvnvPTGB+6h6S+7//3Ju8vOndzF5/b3yvbTgcDE2VtWgam7MiDVRdujEhaeQ/bf090w4pxarZIBfOrgsd6kOOz8472/xfoKKVxgNq3AdO/S2d157WCvaX0HjnCVlWu8SUMOeYi8/OaH7rNZD1SzkGuWSU5WUf5Ystyb6MKHrEzJClX4kD0uDz/xctR9LjJhPf7M67UEZvi1E9xzr7zb4Gg4qk/3WhNLnEOH07KcQH0CE/4hMeXuK709VyIb/c4c6ZaVlrnRFw9wW/+lo3vj3Tlu7J2PuEED+rqzTz6s+kpFfOR8eZjLKp48OG+8Z5pbuarczZh0tWtW0tSLvXTMve61t//PXf6Pk1yXXTq5H3761V11y2RXVFjgnrzvquqVn7Muucnbp7XTX7dyo4ae5snM2DunuOnPvu6ee+i66hXflatWuwP6D3UH7ru7O6Vfb6+c9z/+wo2+ebK7ZdT5bv/uu3r1yr0379eF7uLR41xJcZEbfcnp1W2XVafipoXV/3941VMkSF61yQ+rLTbv4MpWrPKkJHwtVoGRh+lFo8a5E47a3x16YDcn6zeTHv2Xe/O9Oe7hO6/wVi7kWF620luJlblC5sJjDqv5YbLFpu2rhTGWfhER/Pr7eW7sZWe6LTff2HtdP2nq8+7X3xenTGBEYA86YZjbp0tnd/vVF7hjzxrlifDNo87zXosPHnFndd0yzx3Yf6jbe48d3QUDj3YFTRq7mS+94+6b+pybds9It8N2Na/9zxl+i/v4s2/c1cMGuu222tSbz4Tt9z/9WksY5fX5xVfd4449rKf3g1i4jr1jiifWz04e6+0t9MuBwMTZU1aBsb7iCQtJNOEJ/y3yARoEgem5187utqsGVQvMmrVV7pkHQ6tPdQXmm+9/dv3PGe1Jy8DjD/Z+mclEKb9sT77gWu/VWqTAjLrpQTf9uTdc+GET2c3yi/m+qc/XEpgLrrzDzfr3x95rqI3bbxR1VGzYpoW3EZgj2ATCAvPOM3d5F1peXuk+++p7N+KG+13TokJPOOQXbvhBIJvP993zb9VQ5Nf786+9570OlV/1vy1c7A44dqi3enfeqUfUEvJ7H3nOnX58H+/B//OvC13v4y9xdTfUz/7oc3fmxTe5W0cPcr167O6dLwIjP2zeeOp216JZsffffpr/u+tz4jB37aVnuCMP2sf7b19++6M75syR690H8t/bbdSm1ocEEi+vY6W8hvbALPxjqZPXOHJtzzw4xm3WsW3UAWEVGPm4oH3b1t5r7fBRUVHpLh0z0fXs9jd3eK+aFaVwuoaGPjqw9ovsiTvwuIvX4/3PG+93ssqUqhWY+b8tcr2Ou9h7FS5fsV4+dqJ758PP3ZtP3e6JbY++g9346y/yVt6Eg4jFhhu0ci2bh8RCVp5363WWG3T6UdWv0cLnyYcYwiZ8yMrxK299VEtgDj3lMm/ufHTciGrpE2E76PhLapXph7scgYmzl1IlMNF+5X/4n6/caRde507rf5C75NzjvBY3JDDhV1TR9rlk0wpMLAIjD4XHn33D+6V7zKE9qntNvkzqdtj56wmMLDXLknM0BuGNdpGvkML7XNhoF+cNEaDT6m7ilUsrkK8B9/qbu2LwydWvcsbcMcVNm/ma++Tl+zyhCR+y8Vs2kMurYHklHH6gRpPpSGwiPbIHJfIVlfx9dXmF2633We704w72XiWFBeb3RUu9VzbhQ1Zb9uhzjjdHyFwRPlekplF+vut32H7eRwUiS5HtjWyDRWDkF7u8ttU2tlsEJvzgtX4FaREYa7+EX2dF7jkUFt48k8JXSGGBkdVn+WhD6pKPE56adLW3GiJsI2VV9j8+/OTL7tvvf3alK1Z6P+Lkdftpxx7kLjkv9DyQ808472rvS85ImQ6vGoZf2YXny/MHHFVLpqUM+YikY/sNvbr9ciAwcfZUqgSm7tdG0rzwIJTXR7J0Ksf/fpjvZN/IcUf83Y2I2Lwqf5P9Iy+/+VHUh7dfBUZ+gcov0Vem3VRrz4As+crSad0VGNkXIO+Xx1x25np7Ao4/72r36Rff1VqBefCxF70lfVn2/8fAo2uNilWrK7xJI3IZPc5hw2k+ICAC89Z7c9wDt13q7TkYOupub7XioTsuq7WvRX45y3L+eht6163zXi9NuGGo95pAVgJlRVA282+1+cb1EpAvgK669SH33ENj3RZ1Pt/daf/T3dGH9HAjLzq1WmDkYSeyU1dgIvfKyd/k17XsuZONybLKIysssk/ngoF910sNEIvADD7jaG8vRn2HRWBk8/zBJw335jCZy7TDIjDWfgkL4+MTRtV6FSOvXuSVcqpXYOQLy+5dO3sC8/nXc73N4WGBkdd7ssdFVtnkx6uMI5mb5HVeXn6eO+SkSz1eYYGR8Xrupbeu96HBrHc+cRdccXv1CkxYnkRg8/NrpFu4r1mz1tvbJSszfjkQmDh7KlUCIwNLlmXDm6nkwSlfzbz3f1/UWgYUA+92+PleMrrIpVd5l9rruKFOHrrRVh/Cm1uvv+Jsd+iBe8V59fppGh+Z3GJZgZG9AV6OlnEjvPf+cshkJkvpssm209abe3zCR3jVSt7x33H1P6r/u7wjli+c5IhcgZF3671PuMRbpp0+cbQ3UYSP6+9+1NszI6+oeIWk973fI+rugZENvWcPu9nLtySvhCPHxdSnXnVP3X911C9v2m3Y2ttPEv6lL/s5du0c+oow2hFeqam7AiP7ZPboc7b3uiD8ekDGvVVgIusSgXn+1ffcPQ/NdP0P32+9HFKxCMzw84/39tXUd+zS60x38tEH1trEK3tmhG94RSC8aiQ/GiL3DNVXpkVg5H619Ev44S4fWcjHFuFjylOvOFnFyQaBkddZIlTvPHO3a1pU4DUxzCxyBebjz771XqWHpSh8LWEpDvOWPTZ7HnqeO/mYXp7E1j1kj80mHaK/Qs/G+xqBMfaKvHe8cdy06mgZGHn5+d5GqPAx/PwTqjPxxrpHJRwvE9xvC5e4fof28PLAvD77E/fSGx96Bi6/6CIPmVRlcpVlZfkK59cFi92d9z/lNtqglbdaEU1gZOVBViBat2zuDu/VzZtgRXp+/2OptxM+3iNWPrEKjHAYdPnt3i8E+XJj1epyN3Hq894SrIiNTMyyn0b+Ltcm4nfcOVc5yccjr+VkZUtinn7h327XnbZxkuuhbh4Y2Rcj+2M6tNvAW+mSJIKywiMT3d/3ERG6IOWfiMbLn/OSRyDaJl55tSP5UWbcf031ng+5Ly8adXctqZZWyD/XIfdD+KshedUjPyoGnnCIu+D0vtUNlbH5zxvu9175SH6Q8J6MIWf1c2eccEh1XPjX9bixQ1yPvUJ7bawCI/sn5s77zdtPEXnIr3JJ7hiZYkD+LrmWZKNv5Bd9dcmGVwk0gZG9HHJ/jhx6WnURkqBTNuhGfoUkK8ktmpXUyoslez9OGTzW7b/PrrU+l5bNxjJ3CEf54iraYe2XcHLPumUJW9mTkg0CIytB8oXU20/fWX2pTzz3pvcVWKTAyCrbAcde5K0eR4qgzJkyd0byPvy0K7z0HXWfJ7JhfNOObX2VsBOBMc57sf5bP/EKjCSYkx3n8tmcfNoognFg9929T9zCu/vDTZbNdPK1jaw2yBcAW/2lozvvtCPc4iWlTsxdNuJJWXUPWU2QlRhZTpR345KYSzbFRn66Z8RSHRYrn1gFRip65MlXnPw6ks9L5V2t/BqWTc+y016W3mVCFgkLT9byfl32wojkyasA+ZpB/rkH2dQmr5ii7UkIf1749Xc/eUuqf9m0g/dLpd9hPb18NRzBJxBNYETyDz35Um9Pi+QDklwn8prouHNGe7+I5cfLtltu4uSViOxBk+X5x8aPrP5BI18IPfbM694XO/L5tEiNrBSsq6pyj00YVf168orr7vNe/8pmcvlaST5zla/mRIam3j2iujyrwMjnyReOvNONuPAUt3eXzt4/f/LZVz+4kTfe784++fDqvTLhXpW9YK++/ZEbdfEA75Pw5aWhPReRn1FbBUY2xv/36x/clLtHOFmNeu/j/3orG/KgjHygShslZ4q8EjmyT3evKROnPOs9uKeO+6fHNfKQB3VJSVN38TnHefOjzIOyahD+IieWfpEVpx/n/ealW9isYzvvx6J8BSZf7sQjMJY8MOHXOJZXSFNnvOauvf1h78dZz267eKklHntmlpccVeb7u64dXL2fSX6Yzv91obeJW5KUysqNjCV5jkTyDn/1JQJ0ZJ99PGF54fUP3D2Tn3a3jb6g+ss0P9zpCEyW9FJYeGRXemSitixpXqCaITl1ZF/R8w9f57u8B4HqiCy9GO0z6shXSbLacuvE6d5romXLy9xGG7Z2Pfb8m7dCEPk5qnwtN+nR592MF/7tfl2wyLVp3cJ7bSGvhCLzu8jDVx4kkjVbMkzLaqLkJrnonGOrvzaKZQVGYp/611tOHoSSGFJkZJMObd1RB3f3crTUTTonq0DytdWcL/7nKivXuo02bOVlvj73lJqvp6wCIw9qWSmY89/vXEFBY9d1l796K9aSl6Tujwf5USGfL8vXhrICtOP2W7jBZxxTa29KeLhIGgWRv7nzfvVEUX7MyA+TSMmy9ou8Oh59y4NeolBZcZW8Kfvu9Tdv34isttWVJ23IWjLxxiIwsuJ07e2PeCklqtat88aMPB9emPWeu2XCdLfpxm2rN3JLubKi9/Hn37pmxUVeUk6RYBHJyPw8cg3CW/5Zlm9++Nm77m226OgG9O/jrTT76UBgsqS3EJjkd4QsVUvOCkkYJQnB5JCVooNPHO5WV1S4t2fcmfX/+GbyqVAiBCAQVAKyXyryY4Pwfp4PX5gQyI8QEJgsGckITPI7Ipw5VZI6SdZRyXAuX41IZtLIL7qSXzMlQgACEEgvAfknYiSLs6Sa2G2n7bwVN1kBky80bx55Xnobk6baEJg0gdaqQWA0QvH9XYRF9s7IRkbZWCn/uuwJRx3gLYtzQAACEAgKAZnf7rp/hpdEUfYFyatJ+dJTPncPr0AH5VrD14HABK1HuR4IQAACEIBADhBAYHKgk7lECEAAAhCAQNAIIDBB61GuBwIQgAAEIJADBBCYHOhkLhECEIAABCAQNAIITNB6lOuBgI8IVFVVuU8//VRt8RZbbKHGSIDkOdGOJk2aaCHe31euXKnGNW3aVI2xltWokZ4osaKiwlSfpV2rVq1KWlnl5eWmspo3b67GyZhI1rFkyRK1KEubpJDly5erZbVs2VKNkQDLNVpiKisrTfUVFRWpcatXr1ZjJKBZs2ZqnLVd2v1aUlLiCgsL660PgVG7ggAIQCBVBBCYGrIITIiF5cFtHY8ITIgUAmMdMcRBAAIQMBJAYBCYukMFgbGLnHWlA4ExTkiEQQACELASQGAQGAQm+t1iETkERnsJZZ2JiIMABCAQIwEEBoFBYBCY+qYN9sDEOKESDgEIpI8AAoPAIDAIDAKTvjmXmiAAgSQRQGAQGAQGgUFgkjShUgwEIJA+AggMAoPAIDAITPrmXGqCAASSRACBQWAQGAQGgUnShEoxEIBA+gjINwTz5s1TK2zdurUaIwGWbxLy8/NNZVmC8vLyLGHOkiTM8qmr9asTS6MsidmkHEu7LDlspKyGkpKF27x27Vq1+ZYYKcSarE+t0DlnqbNVq1aWopxl3FjGsuVLJWmQpb41a9aY2m7pQ2u7tPEsSfNIZGfqFoIgAIF0E0BgaohbREGb8GPpPwQmFloITJgWAhPbuCEaAhAIKAEEBoGpO7QtKx2WGFZgasiyAhPQCZTLggAEMkcAgUFgEJjo9x+vkEL/7hKvkDI3P1MzBCDQAAEEBoFBYBCY+qYIBIbHBwQgkLUEEBgEBoFBYBCYrJ2iaRgEIFAfAQQGgUFgEBgEhmcEBCDgOwIIDAKDwCAwCIzvpm4aDAEIIDAIDAKDwCAwPAsgAAHfEUBgEBgEBoFBYHw3ddNgCEBABKa0tFQFYUnyJoWUl5cnrayKigq1LGsG2sWLF6tltWnTRo2xZktt3LixWtaSJUvUGAkoKSlR4yyZhqWQFi1aqGVZsucWFxer5UhAWVmZGmftw/nz56tlbbbZZmqMBFjGliVhnGW8S30WXpb70NqH1vGgsZf7vkmTJvUyzVtn+djc1CUEQQACEIiNAAJTwwuBCbFAYEIcEJjQP2OBwMQ2pxINAQikiQACg8DUHWoIDAITHhMITJomYqqBAARiJ4DAIDAITPT7hhUYVmBin1E5AwIQSBsBBAaBQWAQmPomHFZg0jYVUxEEIBArAQQGgUFgEBgEJtaZk3gIQCDjBBAYBAaBQWAQmIxPxTQAAhCIlQACg8AgMAgMAhPrzEk8BCCQcQIIDAKDwCAwCEzGp2IaAAEIxEpABGb58uXqaZYvMtRC/gxYu3atKTQ/P1+NsyaWy8vLU8uy1GeJkYos7bImG7PwaihXR+SFWxLsWVipMP8M+OOPP9RQa0K/1q1bq2VZEsZJIckaz1oiuHCDLePGMmakPEvquKqqKpWVJUCSKBYUFNQbSiI7C0ViIACBlBBAYGqwWh4ylhgEpoYpAhNiYRk3CExKpjgKhQAEgkoAgUFg6o5tVmDsdzsrMJb1IDtPIiEAAQiYCSAwCAwCY75d1gtEYBCY+EcPZ0IAAgkRQGAQGAQm/lsIgUFg4h89nAkBCCREAIFBYBCY+G8hBAaBiX/0cCYEIJAQAQQGgUFg4r+FEBgEJv7Rw5kQgEBCBBAYBAaBif8WQmAQmPhHD2dCAAIJEUBgEBgEJv5bCIFBYOIfPZwJAQgkRACBQWAQmPhvIQQGgYl/9KTozJUrV5pKnjFjhhr3xRdfqDESMGbMGDXu7LPPVmMqKyvVGAno3r27GteuXTs1RgK6deumxrVs2VKNISD9BGT6KSsrUyu2ZHCVQizjz1pWeXm52q6GsoRGnmyZZq0PI7VRxky8y5YtsxRlyhprzd1SVFSk1mnhbulnqei7775T65s7d64aIwGdO3dW41q1aqXGSIBkmNUO69jSypG/W8a8NTOzpSxrUjytLBkvDWV5JhOvpffTHIPAhIAjMGkeeBmoDoGpgY7AhFggMCEOCIxzCEwGJuVEq0RgEJhEx5BfzkdgEJi6YxWBQWDCYwKB8ctMHtFOBAaB8eGwjavJCAwCg8BEv3VYgWEFJq5JNdMnITAITKbHYLrqR2AQGAQGgalvvmEFJl0zcRLrQWAQmCQOp6wuCoFBYBAYBAaByeppOrbGITAITGwjxr/RCAwCg8AgMAiMf+fw9VqOwCAwARrODV4KAoPAIDAIDAIToBkfgUFgAjScERjnHHlgQsOAPDAhDuSBCXEgD0wAZ/oHH3zQdFUDBw40xQU9aN9991Uv8ZVXXlFjLDeUqRCCzATkwb5q1So13poozZJAq6qqSq1PAizJ0iwPZCnLUqflGi2fGEt9lpwy+fn5Jg6WOteuXWsqy5JQsrS0VC3L0iYp5KKLLlLLat68uRojAQcddJAaZ0l2J4VsuOGGalmWsWURY7WiPwOsTBtKLGetKxyn3RfNmjVrMJEiiexiJZ6GeAQmNsgITGy8sikaganpDQQmxAKBCXFAYJxDYLJptja2BYExgvozDIGJjVc2RSMwCEzd8YjAIDDhMYHAZNNsbWwLAmMEhcDEBioLoxEYBAaBiX5jsgLDCkwWTrZZu2AAACAASURBVNl6kxAYnVFkBCswsfHKpmgEBoFBYBCY+uYkVmCyabY2tgWBMYJiBSY2UFkYjcAgMAgMAoPAZOHkHG+TEJjYyLECExuvbIpGYBAYBAaBQWCyaVZOsC0ITGwAEZjYeGVTNAKDwCAwCAwCk02zcoJtQWBiA4jAxMYrm6IRGAQGgUFgEJhsmpUTbAsCExtABCY2XtkUjcAgMAgMAoPAZNOs3EBb5s6dq7a0a9euaowELFq0SI2zZOSUQh566CG1rGQGjB8/Xi3u7bffVmMkoEuXLmrcW2+9pcZIQDKzTJoqzPEgEZjFixerFCyp16WQiooKtSxrBtrVq1erZRUWFqoxEmDJmJrMRHYWXpbrk7ZbrrGsrMzEwZL11sLh448/NtV3/fXXq3GWvpFCevTooZbVs2dPNUYCWrdurca1a9dOjbFknraOP0tGbCmruLhYbZcli7UUot2LMo4bGn9k4lW7IrkBCEyIJwKT3HHl19IQmJqeszy4reneEZgQVwQmxMEiaQiMX2fRNLYbgUFg0jjcsr4qBAaBqTtILSLHCkyIGiswFjXL+mnQPw1EYBAY/4zW1LcUgUFgEJjo9xmvkEL/ajevkFI/D5trQGAQGPNgyYFABAaBQWAQmPqmOgQmyx4CCAwCk2VDMqPNQWAQGAQGgUFgMjoN2ytHYBAY+2gJfiQCg8AgMAgMAuOTuR6BQWB8MlTT0kwEBoFBYBAYBCYt023ilSAwCEzioyg4JSAwCAwCg8AgMD6Z01966SW1pQcffLAaYw0oKCgwhVpzAJgKMwStWLFCjZo3b54aIwEtW7ZU4zp06KDGEJB+AiIwS5cuVStu2rSpGiMBluRsa9euNZVl+ZzXmviwqqpKrTNZMVKR5b633INSliWRnZWpJT/NwoULVVZTp05VYyTg3XffVeOWLVumxkjAAQccoMadcsopaow1YIMNNlBDLYkb1UL+DLCMP2tZ1s+7i4qKGixS7vuGxjKJ7Kw9kqQ4BCYE0jJ5IjBJGnRZXAwCU9M5lgeIJQaBqWGKwNhvfuvYspSIwFgo+TAGgUFgfDhsU9ZkBAaBqTu4WIEJEWEFxjlWYFI29cZXMAKDwMQ3coJ5FgKDwCAw0e9tBAaBybpZH4FBYLJuUGawQQgMAoPAIDD1TUGswGRwco5WNQKDwGTZkMxocxAYBAaBQWAQmIxOw/bKERgExj5agh+JwCAwCAwCg8D4ZK5HYBAYnwzVtDQTgUFgEBgEBoFJy3SbeCUIDAKT+CgKTgkIDAKDwCAwCIxP5vQlS5aoLd12223VGAlYvHixGmdNtFVaWpq0stSCCIDAnwREYCzJ56w5KqQ87bCWZUnOlp+fr1Xn/b28vFyNsyTrs7RJKrIk4ausrFTbJAGWdlmZWvKD/PTTT2q7rr32WjVGAr7++ms1bqeddlJjJOCwww5T4/bff381RgIsyQEbNWqkllVWVqbGSIDlvrCOhxYtWqh1WvpZCtGeT8KpoRgS2aldkdwABCa5PCnN3wQQmJr+s4gCAhPihcCEOCAwFjXz9xyZVa1HYLKqO2hMhgkgMAhM3SHICkyICCswoZUqVmAyPElHVo/AZFFn0JSME0BgEBgEJvptiMAgMBmfoOs2AIHJui6hQRkkgMAgMAgMAlPfFMQKTAYn52hVIzBZ1iE0J6MEEBgEBoFBYBCYjE7D9soRGDsrIoNPAIFBYBAYBAaB8clcj8D4pKNoZloIIDAIDAKDwCAwaZluE68EgUmcISUEhwACg8AgMAgMAuOTOR2B8UlH0cy0EEBgEBgEBoFBYNIy3aankhEjRpgqGjNmjCnOErT33nurYTNnzlRjWrdurcYQAIEwAREYS5Zaa2ZPS6I3a/ZcS4osS4xcqyXbcOPGjdWBYSlHCikqKlLLsmTrlUIsTJs1a6bWJwGrVq1S46677jo15pdfflFjJODbb79V43r16qXGSMABBxygxu2www5qjLV/LNxXrlxpqs8ytqz3mGVsWT4Bl4Zr948kdywoKKj3GsnEa+r+9AYhMOnlTW2ZI4DA1LC3PGQQmBAvBCbEAYHRFChzc1vO1ozA5GzX59yFIzAITN1BzwpMiAgrMKF/h4sVGJ89FhAYn3UYzY2bAAKDwCAw0W8fBAaBiXtizeSJCEwm6VN3OgkgMAgMAoPA1DfnsAKTztk4SXUhMEkCSTFZTwCBQWAQGAQGgcn6qdreQATGzopIfxNAYBAYBAaBQWD8PY/Xaj0CE6DO5FIaJIDAIDAIDAKDwAToQYHABKgzuRQEhjww1WOAPDAhFJZcKmziZROvLx8flptcLqx3797q9b3zzjtqjDXgpZdeUkMtyZ7UQgjIGQKyArNs2TL1egsLC9UYCbDkSbEm2aqsrFTrbNKkiRqTzHZZrk/qsySUrKioMLXdkp/GmgTN8lAePny42q73339fjZGAnXfeWY076aST1BgJ2G677dS4tm3bqjESYEneaOFuzQPT0KfI4QaXlpaa2m4ZW9bxoN2Lsom3oXuMRHamLktvEAKTXt7UljkCCEwNe20yt4oQAlPDFIEJsUBgMjfH5VzNCEzOdXnOXjACg8DUHfyswISIsAITeoXECozPHg8IjM86jObGTQCBQWAQmOi3DwKDwMQ9sWbyRAQmk/SpO50EEBgEBoFBYOqbc1iBSedsnKS6EJgkgaSYrCeAwCAwCAwCg8Bk/VRtbyACY2dFpL8JIDAIDAKDwCAw/p7Ha7UegQlQZ3IpDRJAYBAYBAaBQWAC9KBAYALUmVwKApPE/DTkgQkNJ/LAhDiQB0Z+AnFkFQEEJqu6g8akkAArMKzAsALDCgwrMCmcZLO16MWLF6tNu/LKK9UYCZgwYYIaN2DAADVm8ODBaowEdO7c2RRHULAJiMCsWLFCvci8vDw1RgIsmV7z8/OTVlYy22XJzmr9vSlfb2iH5TNdKaOqqkorylmyFkshCxYsUMu65ZZb1JhPP/1UjZGAzTbbTI0bO3asGiMBxcXFapw1Y7QlaaElxroCY+lry70jACztst4XWlnCvKEkfGTiVYdk9gYgMNnbN7TMRgCBqeGEwIRYIDAhDtrDXWIQGKvS2+YjotJIAIFJI2yqSgkBBAaBqTuwEBgEJjwmWIFJybSbHYUiMNnRD7QifgIIDAKDwES/f1iBCb224xVS/PNrVp+JwGR199A4AwEEBoFBYBCY+qYKBMYwifo1BIHxa8/R7jABBAaBQWAQGAQmB58JCEwOdnrALhmBQWAQGAQGgQnYxG65HATGQomYbCaAwCAwCAwCg8Bk8yydorYhMCkCS7FpI4DAIDAIDAKDwKRtys2eipYsWaI2pmfPnmqMBHz55ZdqnCXRUceOHdVyJGDQoEFq3LBhw9QYAvxNQATGksvCmnyuoqJCBWL5ukMKKSsrU8uyJIxTC/kzwJIwbs2aNabiioqK1DhLfVKIJUmdtX8sCejuuecete3z5s1TYyTAMv+dfvrpprIsGUfatm1rKsvKXivMOh4sY96Sh0jaYxnzlmeFlNWkSZMGL1HqaiiGRHbaCMnivyMwWdw5NM1EAIGpwWR5qFkfWAhMiCsCE+KAwJimI4LSSQCBSSdt6koFAQQGgak7rliBsd9pVqFFYOxMiUwTAQQmTaCpJmUEEBgEBoGJ//ZCYCwv9uLny5kpJIDApBAuRaeFAAKDwCAw8d9qCAwCE//oyfCZCEyGO4DqEyaAwCAwCEz8txECg8DEP3oyfCYCk+EOoPqECSAwCAwCE/9thMAgMPGPngyficBkuAOoPmECCAwCg8DEfxshMAhM/KMnw2ciMBnuAKpPmAACg8AgMPHfRggMAhP/6MnwmQhMhjuA6hMmgMAgMAhM/LcRAoPAxD96fHDmggULTK3s37+/GldYWKjGzJ49W42RgJ133lmNu+qqq9QYCdhvv/1McQRlHwGZfiwZQK2ZPVevXq1eZOPGjdUYCbBMjZYYKctyjQUFBWq7rBws96ola7E0KJkJ9mbMmKFe4+TJk9WY5cuXqzESMGTIEDWub9++aowErFixQo1r3ry5GiMBlrwslgzI1vGQl5entsvSz1KIZZxaMzNrjSopKWmwPjLxagR9/ncExucdGPDmIzA1HWx5MFgfWAhMiCsCE+KAwAR8Ig3q5SEwQe3ZYFwXAoPA1B3JrMCEiLAC4xwrMMGY5+O+CgQmbnScmAYCCAwCg8BEv9EQGAQmDVNwdleBwGR3/+R66xAYBAaBQWDqmwdZgcnxJwQCk+MDIMsvH4FBYBAYBAaByfKJOlPNQ2AyRZ56LQQQGAQGgUFgEBjLbJmDMQhMDna6jy4ZgUFgEBgEBoHx0aSdzqYiMOmkTV2xEkBgEBgEBoFBYGKdOYmvRWDhwoUqkeuuu06Nuffee9UYCbAkEysqKjKV9c0336hx1gRTakEEJJWACIwlKZkl8Zc0zJKduk2bNqZrsCR6sybsKi0tVeuUDYvaYWmTlGHhZYmRsiwJzubOnas13fv71KlT1biZM2eqMRtttJEaIwFjxoxR47p27arGSMDKlSvVOCtTyxdGltwtycwLZEm2KAAsOYYs1ydlNWnSpEGmbOJVhxwBQgCBYRxkggACU0MdgQmxQGBCHBAYPqPOxJzsyzoRGF92m+8bjcAgMHUHMQKDwITHBCswvp/i03MBCEx6OFNLbQIIDAKDwESfFViBYQWG54WRAAJjBEVYUgkgMAgMAoPA1DepsAKT1Ok2uIUhMMHt22y+MgQGgUFgEBgEJptnaR+0DYHxQScFsIkIDAKDwCAwCEwAJ/d0XhICk07a1BUmgMAgMAgMAoPA8ExIiAACkxA+To6TAAKDwCAwCAwCE+cEyml2ApasvuPGjTMVeM0115jiLEH9+vVTwywJtKQQa2IytUICTAREYCwJwtasWWMqb9WqVWpc06ZN1ZhkB1iShFmSoFmSyknbhat2WL5ykTIsyQE/+ugjrTrv7xMnTlTjFi1apMbstttuaowEnHbaaWpc586d1RhrgCVBp5RlGaeWucgSI/VZ+tqafK6goEDFYU2wp/FiE6+KmgArAQTGSoo4KwEEpoYUAhNigcCEOFjkxBKDwFhnI+ICTQCBCXT3ZuTiEBgEpu7AQ2AQmPCYYAUmI9NyMCtFYILZr5m8KgQGgUFgot+BltUVSwwrMJmc4ag7awggMFnTFYFpCAKDwCAwCEx9ExorMIGZ6jN/IQhM5vsgaC1AYBAYBAaBQWCCNrNn4fUgMFnYKT5vEgKDwCAwCAwC4/OJ3A/NR2D80Ev+aiMCg8AgMAgMAuOveduXrUVgfNltWd1oBAaBQWAQGAQmq6fpYDQOgQlGP2bTVSAwCAwCg8AgMNk0K+dwW3766SfT1d90001qnCVrpxRSUVGhljVy5Eg1RgKGDBmixjVv3lyNIcBGQASmrKxMDbZ+LmoZC1r2z3BjVq9erbarqKhIjbGOUUuGYEtGX6nP0nZLtl4py8L+xRdfNHGYNm2aGmfJ9Nq+fXu1HAkYNGiQGrfllluqMdY+tLRdyrJkn7aMU2v23CZNmqjXaB1bhYWFalnWzNkar+LiYtdQTN466yhWm0wABJxDYBgFsRBAYGpoITAhFtpDTWIQmBArBAaBiWW+JVYhgMAwRGIhgMAgMHXHCwITIsIKjHOswMQymxKbMAEEJmGEOVUAAoPAIDDRb3kEBoHJqYdBNlwsApMNveCfNiAwCAwCg8DUN2OxAuOfuTwQLUVgAtGNabsIBAaBQWAQGAQmbVMuFTVEAIFhfMRCAIFBYBAYBAaBiWXWJDZlBBCYlKENZMEIDAKDwCAwCEwgp3f/XRQC478+y2SLERgEBoFBYBCYTM7C1J0SApbMv1LxxhtvnLT6b7jhBrWsoUOHqjEE2AiIwKxYsUINrqqqUmMkwJIXw/J1h5S1atUqtU5rWZb2Wz4fXrt2rdomCbBkx5g1a5aprG+//VaN+/nnn9UYCZgzZ44at80226gxffr0UWMkYM8991Tj2rZtq8ZYx4Ml6Z+UZUn0ZhnLpoYbP8m2jFGpzzLmrWVpCfZKSkpIZGftZOKyiwACk139kYrWIDA1VBGYEAsEJsQBgXEOgUnFrEuZaSGAwKQFc0YrQWAQmLoDEIFBYMJjAoHJ6PRM5YkQQGASoeePcxEYBAaBiX6vsgLDCow/ZnFaGZUAAhP8gYHAIDAIDAJT30zHCkzwnwGBvUIEJrBdW31hCAwCg8AgMAhM8Of6nLtCBCb4XY7AIDAIDAKDwAR/rs+5K0Rggt/lCAwCg8AgMAhM8Of6nLtCBCb4XY7AIDAIDAKDwAR/rs+5K0Rggt/lCAwCg8AgMAhM8Of6nLvCZcuWma7Zkm3zm2++MZXVr18/NW7atGlqDAE2AiIwln9+YoMNNjAVuHr1ajWuqKhIjZEAy2esloy3UpYl82qzZs3Udlkz8ZaWlqplvffee2qMBFgy8X700UemssrLy9W4E044QY3p2bOnGiMBixcvVuPat2+vxkhARUWFGldcXKzGWMvSstRay5E4S/bcsrIyU9st49Qy3qUyLXlj06ZNycRr6hWCso4AApN1XZL0BiEwNUgtDwYEJsQLgQlxsEgVApP0aYsCIaATQGB0Rn6PQGAQmLpjmBWYEBFWYJxjBcbvM3wOtx+BCX7nIzAIDAIT/T5HYBCY4D8BAnyFCEyAO/fPS0NgEBgEBoGpb6ZjBSb4z4DAXiECE9iurb4wBAaBQWAQGAQm+HN9zl0hAhP8LkdgEBgEBoFBYII/1+fcFSIwwe9yBAaBQWAQGAQm+HN9zl0hAhP8LkdgEBgEBoFBYII/1+fcFVoF5rLLLlPZTJkyRY2RgFGjRqlxQ4YMUWMIsBEQgVm6dKkabPkiQwqx5klRK3TOWZLUWRLnSV2yGVE7LG3Py8vTivH+bkng9tJLL5nKeuONN9S433//XY2RgJ122kmN69KlixrTq1cvNUYCqqqq1Lhkjq38/Hy1Pus4LSwsVMtKZh6YZCWfs16fxGm8SkpKSGSnjgICspIAApOV3ZLURiEwNTgRmBALBCbEAYFxDoFJ6nRLYekkgMCkk3Zm6kJgEJi6Iw+BQWDCYwKBycy8TK1JIIDAJAFilheBwCAwCEz0m5QVGFZgsnz6pnkNEUBggj8+EBgEBoFBYOqb6ViBCf4zILBXiMAEtmurLwyBQWAQGAQGgQn+XJ9zV4jABL/LERgEBoFBYBCY4M/1OXeFCEzwuxyBQWAQGAQGgQn+XJ9zV4jABL/LERgEBoFBYBCY4M/1OXeFCEzwuxyBQWAQGAQGgcnwXL9kyRJTC3777Tc1rnHjxmqMBGy66aZqXKNGjdQYCbBmozQVZgiaPXu2GjV+/Hg1RgKsWXYthb377rtqmCVPhVoIAR4BERhLFtfmzZubiFmyiVrvL0uWU2tZ5eXlavu1rKRSQGlpqVqOBHz55Zdq3KOPPqrGSMAnn3yixvXp00eNkYBOnTqpcXvssYca065dOzVGAizzcqtWrUxlVVZWqnGWT5+lEMt4KCoqUuuzjFEppKCgQC3Lcn1SiKVdlqSMUpZ2/0hdDT2b8tZZ8mWrl06A5UYRSghMaKwgMNwzCEztMYDAhHggMCEOFlFAYBCYpDxJEJjYMCIwsfEKajQrMDU9i8AgMJH3OQITkjhWYNIw+yMwsUFGYGLjFdRoBAaBqTu2WYFhBSY8JhCYNM38CExsoBGY2HgFNRqBQWAQmOh3NyswrMCkbd5HYGJDjcDExiuo0QgMAoPAIDD1zW+swKRp5kdgYgONwMTGK6jRCAwCg8AgMAhMhmd4BCa2DkBgYuMV1GgEBoFBYBAYBCbDMzwCE1sHIDCx8QpqNAKDwCAwCAwCE+cMb0koNHr0aLX0qVOnqjESMG/ePDXOmnxuk002SVpZ/fr1U8uaM2eOGtOhQwc1RgIee+wxNW7lypVqjARYNrs9/fTTprJ69uypxqU76Z/aIB8HiMBY+rmqqsp0lZZEdpbPlU2V/ZmIzxK7evVqNcwSY/2h9Pnnn6v1TZ8+XY2RAMsc2bt3b1NZffv2VeMs91dxcbFajgRYEriZCnLOWfrHWpYl0ZuW5E3qst4X1meKpf2W+8cSI3VpWVyaNm3KZ9QNdYrl5kRgQgQRmBAHywRrmQiICU1gCExoJFgekAhMiBUCE+KAwGgKFPBZFoEJdTArMCEOrMCk94ZHYGp4IzD2HwgIDAIjBHL+nxJAYBCYyEc2AoPAxELA+vvPIieWGFZgWIGJHJ+swFjvwFjuah/FIjAIDAKTuRuWFRhWYOqOPssrWlZgWIFhBca4QY09MKGbhT0w9iXuzCmBv2pGYBAYBCb6PcsmXufYxKvM56zAsALDCkzmpAeBQWAQGASmvhkIgUFgTE8nNvGGMLEHxjRckhaEwCAwCAwCg8DEOaWyAsMKDCswcd48STgNgUFgEBgEBoFJwmRaXxEXXnihWvpTTz2lxkjA/PnzTXHpDurYsaNaZbrb3qVLF7VNEvDKK6+occ2aNVNjCEg/ARGYBQsWqBU3b95cjZGA0tJSNa6kpESNkYCKigo1zpogzNIuS9K1H3/8UW2TBHz22Wdq3KxZs9QYCbDw2nPPPU1lHXrooWrcr7/+qsZss802aowEWL7SscRIWZYvwFq0aGFql+WLM3l9oh2WMSplFBYWakWZEhZKIZZ2WRYGpKy8vLwG2yXzdkNtz/nPqNVedc4hMCFKCIxltBATCwEEpoYWAhNigcCEOFhEAYHJ8c+oLZMtAoPAWMYJMbETQGAQmLqjBoFBYMJjghWY2OfU9c5AYBCYJAwjiohCAIFBYBCY6FMDKzDOITBJeGwgMAhMEoYRRSAwDY4BXiHxCilygCAwCExSHhoIDAKTlIFEIesRYAWGFRhWYFiBqW9qZAUmCQ8NBAaBScIwoghWYFiB4SskbwzwFVLoVuArpDQ8FhAYBCYNwywnq2AFhhUYVmBYgWEFJoXTPwKDwKRweOV00QgMAoPAIDAITAofAwgMApPC4ZXTRSMwCAwCg8AgMCl8DFiyCpaVlZlacMcdd6hxp556qhojAU888YQa9/TTT6sxEjBp0iQ17vXXX1djLFlVpZBOnTqpZR1++OFqjAQUFxeb4gjKPgIiMEuXLk1awyxZVS1f+0iDKisr1XZZ02jl5+erZVnabmX16aefqvW9+OKLaowE9OjRQ42zcujevbtaVlFRkRpj+UJHCmnSpIla1ooVK9QYCVi7dq0aZ+lnKUTb+yExln+NWm1QDAGW55wUZ+kfa1kaB8nATSbeGDoxWqilMxCYEDkEJsHBlmOnIzA1HY7AhFhYHpAITPInCstzzto/1rIQmOT343olWjoDgUFg0jAUA1cFAoPA1B3UCEyICCswzrECk4QpH4EJQeQVUhIGE0XUIoDAIDAITPRJAYFBYJLyuEBgEJikDCQKWY8AAoPAIDAITH1TIyswSXhoIDAITBKGEUVEIYDAIDAIDAKDwKTw8YDAIDApHF45XTQCg8AgMAgMApPCxwACg8CkcHjldNEIDAKDwCAwCEwKHwMIDAKTwuGV00UjMAgMAoPAIDA5/Rjg4iHgTwIiMKWlpWrjrQnCLMnnLMnNpEHpLqtRo0Yqh5UrV6oxEmD5xwItP8ykrLZt26p1WlhJIbIpUzsWL16shZgSwUkh8q8Za8eaNWu0EO/vljw91qSalqR4lnFq6Wdpu2VsWe5DKcuSg8c6TjVeJSUlrqHEk3nrrCkUTV1MEAQgAAE7AQSmhpXlIWN9MFgebAhMiD0CE+KAwNjnLSIhAAEIOAQGgal7G7ACEyLCCoxzrMDwkIAABLKWAAKDwCAw0W9PBAaBydqJm4ZBAAKOFZiIQcArpBAMVmBYgQnfFqzA8JSAAASylgArMKzAsALDCkx9ExQCk7VTNw2DAAQQGAQGgUFgEBieBRCAgO8IIDAIDAKDwCAwvpu6aTAEIIDAIDAIDAKDwPAsgAAEfEcAgUFgEBgEBoHx3dRNgyEAARGY5cuXqyAsWVClEMuXPJYYKcuS4CwvL09tuzXA0i4rB0vbG8pwGtlmS9bYxo0bmy7TwsuShM+a+dfCy5rlWcsaax1/EmfhYOFu5ZCs+qTtlty3Fu5SVlFRUYPjRv7e0OfkZOI13XYEQQACqSCAwMS2AmN9MCAwIa4WXghMiJVFmBCYVMyClAkBCPiSAAKDwNQduKzA2IWCFRjLepAvp0YaDQEIZDsBBAaBQWCi36WWFREEBoHJ9jme9kEgsAQQGAQGgUFg6pvg2AMT2KmfC4OA/wkgMAgMAoPAIDD+n8u5AgjkHAEEBoFBYBAYBCbnpn4uGAL+J4DAIDAIDAKDwPh/LucKIJBzBBAYBAaBQWAQmJyb+rlgCPifgOTp+PDDD9UL2XrrrdUYCSgsLFTjGkqMFXmyJZeKNYeI5dNg+Zd3tcNSjpRhSbqWzO83li5dqjXd+7uFl+XrG8v1SX2lpaVqu1q3bq3GSIBlPFjHlrUfTQ0zBFm4W9tkGaerVq0ytEpPZCf3c0NJEklkZ8JMEAQgkAoCCEwNVcuDwfqQsTzgEZgQewQmxME6tizjFIFJxWxJmRCAQFYRQGAQmLoDkhWY5N+irMAknyklQgACOU4AgUFgEJjUTwIITOoZUwMEIJBjBBAYBAaBSf1Nj8CknjE1QAACOUYAgUFgEJjU3/QITOoZUwMEIJBjBBAYBAaBSf1Nj8CknjE1QAACOUYAgUFgEJjU3/QITOoZUwMEIJBjBBAYBAaBSf1Nj8CknjE1QAACOUYAgUFgEJjU3/QITOoZUwMEIJBjBERg5s6dq161JTGbFNKoUSO1rKZNm6oxEmDJR1JQUGAqq7KyUo0TFtphrc/CwXJ90p7y8nKtWea/WzLVWjLeWpPwWcqyjgdLHzaUNTYSUl5enplZQ4FlZWWmciwCYyrI6dlzpRwLK4nTeEnfNDTmycRr7TXidcyFnAAAA0NJREFUIACBpBNAYGqQIjAhFhbpQGBCrBAY60hI+tRFgRCAQK4TQGAQmLr3AAJjnxUQGATGPlqIhAAEkkoAgUFgEJj4bykEBoGJf/RwJgQgkBABBAaBQWDiv4UQGAQm/tHDmRCAQEIEEBgEBoGJ/xZCYBCY+EcPZ0IAAgkRQGAQGAQm/lsIgUFg4h89nAkBCCREAIFBYBCY+G8hBAaBiX/0cCYEIJAQAQQGgUFg4r+FEBgEJv7Rw5kQgAAEIAABCGSEAInsMoKdSiEAAQhAAAIQSIQAApMIPc6FAAQgAAEIQCAjBBCYjGCnUghAAAIQgAAEEiGAwCRCj3MhAAEIQAACEMgIAQQmI9ipFAIQgAAEIACBRAggMInQ41wIQAACEIAABDJCAIHJCHYqhQAEIAABCEAgEQIITCL0OBcCEIAABCAAgYwQQGAygp1KIQABCEAAAhBIhAACkwg9zoUABCAAAQhAICMEEJiMYKdSCEAAAhCAAAQSIYDAJEKPcyEAAQhAAAIQyAgBBCYj2KkUAhCAAAQgAIFECCAwidDjXAhAAAIQgAAEMkIAgckIdiqFAAQgAAEIQCARAghMIvQ4FwIQgAAEIACBjBBAYDKCnUohAAEIQAACEEiEAAKTCD3OhQAEIAABCEAgIwQQmIxgp1IIQAACEIAABBIhgMAkQo9zIQABCEAAAhDICAEEJiPYqRQCEIAABCAAgUQIIDCJ0ONcCEAAAhCAAAQyQgCByQh2KoUABCAAAQhAIBECCEwi9DgXAhCAAAQgAIGMEEBgMoKdSiEAAQhAAAIQSIQAApMIPc6FAAQgAAEIQCAjBBCYjGCnUghAAAIQgAAEEiGAwCRCj3MhAAEIQAACEMgIAQQmI9ipFAIQgAAEIACBRAggMInQ41wIQAACEIAABDJCAIHJCHYqhQAEIAABCEAgEQIITCL0OBcCEIAABCAAgYwQQGAygp1KIQABCEAAAhBIhAACkwg9zoUABCAAAQhAICMEEJiMYKdSCEAAAhCAAAQSIYDAJEKPcyEAAQhAAAIQyAgBBCYj2KkUAhCAAAQgAIFECCAwidDjXAhAAAIQgAAEMkIAgckIdiqFAAQgAAEIQCARAv8PTptGBIy/7GcAAAAASUVORK5CYII=",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "base_image = output_imgs[0][\"x\"].numpy()\n",
    "anomaly_image = output_imgs[1][\"x\"].numpy()\n",
    "\n",
    "recon_base_image = output_imgs[0][\"x_fake\"].numpy()\n",
    "recon_anomaly_image = output_imgs[1][\"x_fake\"].numpy()\n",
    "\n",
    "GridDisplay([ImageDisplay(image=base_image[0], color_map='greys', title=\"Input Image\"),\n",
    "             ImageDisplay(image=recon_base_image[0], color_map='greys', title=\"Reconstructed Image\")\n",
    "            ]).show()\n",
    "\n",
    "GridDisplay([ImageDisplay(image=anomaly_image[0], color_map='greys', title=\"Input Image\"),\n",
    "             ImageDisplay(image=recon_anomaly_image[0], color_map='greys', title=\"Reconstructed Image\")\n",
    "            ]).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the network is trained on inliers, so it's able to properly reconstruct them but does a poor job at reconstructing the outliers, thereby making it easier for discriminator to detect the outliers. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using your own dataset\n",
    "This example assumes each sample contains a gray-scale image array such as:\n",
    "```\n",
    "{\"x\": np.ones([28, 28])}\n",
    "```\n",
    "\n",
    "If you would like to read image from disk, you can create a `Dataset` that produces the file path and use `ReadImage` Op in `Pipeline` to read image from disk. \n",
    "\n",
    "If your image is 3-channel, then you can remove the `ExpandDims` and adjust the input shape in model definition."
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "73b4b63fab75275cc22557d8583617f6e244a28eab482e785f2d595a4709032f"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}