GAN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 基于MindSpore框架的GAN案例实现\n",
    "\n",
    "在下面的教程中，我们将通过示例代码说明生成对抗网络GAN如何设置网络、优化器、如何计算损失函数以及如何初始化模型权重。本教程使用的是[MNIST手写数字数据集](http://yann.lecun.com/exdb/mnist/)，共有70000张手写数字图片，包含60000张训练样本和10000张测试样本，图片大小均为28\\*28。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1 模型讲解\n",
    "\n",
    "生成式对抗网络（Generative Adversarial Networks，GAN）是一种生成式机器学习模型，是近年来复杂分布上无监督学习最具前景的方法之一。\n",
    "\n",
    "最初，GAN由Ian J. Goodfellow于2014年发明，并在论文[《Generative Adversarial Nets》](https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf)中首次进行了描述，其主要由两个不同的模型共同组成——**生成器**(enerative Model)和**判别器**(Discriminative Model)：\n",
    "\n",
    "- 生成器的任务是生成看起来像训练图像的“假”图像；\n",
    "- 判别器需要判断从生成器输出的图像是真实的训练图像还是虚假的图像。\n",
    "\n",
    "GAN通过设计生成模型和判别模型这两个模块，使其互相博弈学习产生了相当好的输出。\n",
    "\n",
    "\n",
    "\n",
    "## 1.1 模型简介\n",
    "\n",
    "GAN模型的核心在于提出了通过对抗过程来估计生成模型这一全新框架。在这个框架中，将会同时训练两个模型——捕捉数据分布的生成模型 $G$ 和估计样本是否来自训练数据的判别模型 $D$ 。\n",
    "\n",
    "在训练过程中，生成器会不断尝试通过生成更好的假图像来骗过判别器，而判别器在这过程中也会逐步提升判别能力。这种博弈的平衡点是，当生成器生成的假图像和训练数据图像的分布完全一致时，判别器拥有50%的真假判断置信度。\n",
    "\n",
    "我们首先定义会出现的标记符号：\n",
    "\n",
    "**判别器的表示**\n",
    "\n",
    "- $x$：代表图像数据；\n",
    "- $D(x)$：判别器网络，给出图像判定为真实图像的概率；\n",
    "\n",
    "在判别过程中，$D(x)$ 需要处理作为二进制文件的大小为 $1\\times 28\\times 28$ 的图像数据。当 $x$ 来自训练数据时，$D(x)$ 数值应该趋近于 $1$ ；而当 $x$ 来自生成器时，$D(x)$ 数值应该趋近于 $0$ 。因此 $D(x)$ 也可以被认为是传统的二分类器。\n",
    "\n",
    "**生成器的表示**\n",
    "\n",
    "- $z$：标准正态分布中提取出的隐码(隐向量)；\n",
    "- $G(z)$：表示将隐码(隐向量) $z$ 映射到数据空间的生成器函数；\n",
    "\n",
    "函数 $G(z)$ 的目标是**将服从高斯分布的随机噪声 $z$ 通过生成网络变换为近似于真实分布 $p_{data}(x)$ 的数据分布**，我们希望找到 $θ$ 使得 $p_{G}(x;\\theta)$ 和 $p_{data}(x)$ 尽可能的接近，其中 $\\theta$ 代表网络参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 模型结构\n",
    "\n",
    "\n",
    "$D(G(z))$ 表示生成器 $G$ 生成的假图像被判定为真实图像的概率，如[Goodfellow 的论文](https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf)中所述，`D` 和 `G` 在进行一场博弈，`D` 想要最大程度的正确分类真图像与假图像，也就是参数 $\\log D(x)$；而 `G` 试图欺骗 `D` 来最小化假图像被识别到的概率，也就是参数 $\\log(1−D(G(z)))$。\n",
    "\n",
    "因此GAN的损失函数为：\n",
    "\n",
    "$$\n",
    "\\min\\limits_{G}\\max\\limits_{D} V(D,G)=E_{x\\sim p_{data}\\;\\,(x)}[\\log D(x)]+E_{z\\sim p_{z}\\,(z)}[\\log (1-D(G(z)))]\n",
    "$$\n",
    "\n",
    "从理论上讲，此博弈游戏的平衡点是$p_{G}(x;\\theta) = p_{data}(x)$，此时判别器会随机猜测输入是真图像还是假图像。下面我们简要说明生成器和判别器的博弈过程：\n",
    "\n",
    "![gan](https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/website-images/r1.8/tutorials/application/source_zh_cn/cv/images/gan_image.png)\n",
    "\n",
    "在上图中，蓝色虚线表示判别器，黑色虚线表示真实数据分布，绿色实线表示生成器生成的虚假数据分布，$z$ 表示隐码，$x$ 表示生成的虚假图像 $G(z)$。\n",
    "\n",
    "1. 在训练刚开始的时候，生成器和判别器的质量都比较差，生成器会随机生成一个数据分布。\n",
    "2. 判别器通过求取梯度和损失函数对网络进行优化，将靠近真实数据分布的数据判定为1，将靠近生成器生成出来数据分布的数据判定为0。\n",
    "3. 生成器通过优化，生成出更加贴近真实数据分布的数据。\n",
    "4. 生成器所生成的数据和真实数据达到相同的分布，此时判别器的输出为1/2。\n",
    "\n",
    "根据原论文中的训练方法，每轮训练都是在优化 `D` 的k步和优化 `G` 的1步之间交替进行，因此只要 `G` 变化得足够慢，便可保证 `D` 在其最优解附近。详细过程如下\n",
    "\n",
    "![method](./step.png)\n",
    "\n",
    "\n",
    "\n",
    "本案例将使用MNIST手写数字数据集来训练一个生成式对抗网络，接着使用该网络模拟生成手写数字图片。\n",
    "\n",
    "此项目文件目录介绍如下：\n",
    "\n",
    "```text\n",
    "./mindspore_GAN\n",
    "├─eval.py                     # 评估模型参数效果\n",
    "├─data_loader.py              # 加载数据集\n",
    "├─GAN.ipynb                   # 可独立运行总项目的notebook\n",
    "├─structure.png               # GAN模型结构图\n",
    "├─test.py                     # 测试模型参数\n",
    "├─train.py                    # 训练\n",
    "│      \n",
    "├─data                        # 数据集\n",
    "│\n",
    "├─result\n",
    "│  ├─gan_mnist.gif           # 保存训练中动图\n",
    "│  ├─iter.png                # Loss迭代关系图\n",
    "│  ├─src_data.png            # 抽取显示原始数据集图\n",
    "│  │\n",
    "│  ├─images\\                 # 生成器生成结果图片\n",
    "│  │\n",
    "│  └─checkpoints\\            # 模型参数\n",
    "│\n",
    "└─src\n",
    "    ├─configs.py              # 超参数设置\n",
    "    ├─GAN_model.py            # 模型网络结构\n",
    "    ├─loss.py                 # 损失函数设置\n",
    "    ├─TrainOneStep.py         # 训练一步代码\n",
    "    └─utils.py                # 其他函数代码\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2 案例演示——数据准备\n",
    "\n",
    "本案例基于MindSpore-GPU版本实现，在GPU上完成模型训练。\n",
    "\n",
    "\n",
    "## 2.1 数据下载\n",
    "\n",
    "首先可从[网站](http://yann.lecun.com/exdb/mnist/)将数据集下载到指定目录下并解压（注意本项目将 `t10k-images-idx3-ubyte` 改为 `t10k-images.idx3-ubyte` ）\n",
    "\n",
    "下载后的数据集目录结构如下：\n",
    "\n",
    "```text\n",
    "./data/MNIST_DATA\n",
    "├── t10k-images.idx3-ubyte\n",
    "├── t10k-labels.idx1-ubyte\n",
    "├── train-images.idx3-ubyte\n",
    "└── train-labels.idx1-ubyte\n",
    "```\n",
    "\n",
    "其中训练数据集包含60000张28\\*28的手写数字图片的二进制文件，在GAN模型中只需要用到训练数据集。\n",
    "\n",
    "\n",
    "## 2.2 数据处理\n",
    "\n",
    "首先为执行过程定义一些输入："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "TOTLE_EPOCH = 100         # 训练周期数\n",
    "latent_size = 100         # 隐码的长度\n",
    "BATCH_SIZE = 128          # 用于训练的训练集批量大小\n",
    "lr = 0.0002               # 学习率\n",
    "b1 = 0.5                  # Adam优化器的beta1超参数\n",
    "b2 = 0.999                # Adam优化器的beta2超参数\n",
    "channels = 1              # 图像通道数\n",
    "img_size = 28             # 训练图像长宽大小\n",
    "BATCH_SIZE_VALID = 1000   # 用于评估的验证集批量大小\n",
    "BATCH_SIZE_TEST = 10      # 测试批量大小\n",
    "gap = 1                   # 判别器等待轮数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " - 定义 `decode_idx3_ubyte` 和 `decode_idx1_ubyte` 函数解析数据，并通过类 `DatasetGenerator` 调用训练数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import struct\n",
    "import random\n",
    "import numpy as np\n",
    "from mindspore import dataset\n",
    "\n",
    "DATA_PATH = 'data'\n",
    "# train set\n",
    "train_images_file = os.path.join(DATA_PATH, 'train-images.idx3-ubyte') \n",
    "# train labels set\n",
    "train_labels_file = os.path.join(DATA_PATH, 'train-labels.idx1-ubyte') \n",
    "# test set\n",
    "test_images_file = os.path.join(DATA_PATH, 't10k-images.idx3-ubyte')\n",
    "# test labels set\n",
    "test_labels_file = os.path.join(DATA_PATH, 't10k-labels.idx1-ubyte')\n",
    "\n",
    "\n",
    "\"\"\"解析图片数据\"\"\"\n",
    "def decode_idx3_ubyte(idx3_ubyte_file):\n",
    "    # 获取二进制数据\n",
    "    bin_data = open(idx3_ubyte_file, 'rb').read()\n",
    "    offset = 0\n",
    "    fmt_header = '>iiii'\n",
    "    magic_number, num_images, num_rows, num_cols = struct.unpack_from(fmt_header, bin_data, offset)\n",
    "    print('magic numbers: %d, image numbers: %d, image size: %d*%d' % (magic_number, num_images, num_rows, num_cols))  # 图片magic number 为2051\n",
    "    # 解析数据集\n",
    "    image_size = num_rows * num_cols\n",
    "    offset += struct.calcsize(fmt_header)\n",
    "    fmt_image = '>' + str(image_size) + 'B'\n",
    "    print(fmt_image, offset, struct.calcsize(fmt_image))\n",
    "    images = np.empty((num_images, num_rows, num_cols))\n",
    "    for i1 in range(num_images):\n",
    "        images[i1] = np.array(struct.unpack_from(fmt_image, bin_data, offset)).reshape((num_rows, num_cols))\n",
    "        offset += struct.calcsize(fmt_image)\n",
    "    return images\n",
    "\n",
    "\"\"\"解析标签数据\"\"\"\n",
    "def decode_idx1_ubyte(idx1_ubyte_file):\n",
    "    # 获取二进制数据\n",
    "    bin_data = open(idx1_ubyte_file, 'rb').read()\n",
    "    offset = 0\n",
    "    fmt_header = '>ii'\n",
    "    magic_number, num_images = struct.unpack_from(fmt_header, bin_data, offset)\n",
    "    print('magic numbers: %d, label numbers: %d' % (magic_number, num_images))  # 文本magic number 为2049\n",
    "    # 解析数据集\n",
    "    offset += struct.calcsize(fmt_header)\n",
    "    fmt_image = '>B'\n",
    "    labels = np.empty(num_images)\n",
    "    for i2 in range(num_images):\n",
    "        labels[i2] = struct.unpack_from(fmt_image, bin_data, offset)[0]\n",
    "        offset += struct.calcsize(fmt_image)\n",
    "    return labels\n",
    "\n",
    "class DatasetGenerator:\n",
    "    def __init__(self):\n",
    "        self.data = decode_idx3_ubyte(train_images_file).astype(\"float32\")\n",
    "        self.label = decode_idx1_ubyte(train_labels_file).astype(\"float32\")\n",
    "    def __getitem__(self, item):\n",
    "        return self.data[item], self.label[item]\n",
    "    def __len__(self):\n",
    "        return len(self.data)\n",
    "\n",
    "class DatasetGenerator_valid:\n",
    "    def __init__(self):\n",
    "        self.data =decode_idx3_ubyte(train_images_file).astype(\"float32\")\n",
    "        self.label = decode_idx1_ubyte(train_labels_file).astype(\"float32\")\n",
    "    def __getitem__(self, item):\n",
    "        return self.data[item], self.label[item]\n",
    "    def __len__(self):\n",
    "        return len(self.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " - 定义 `create_dataset_train` 函数载入训练数据，并分批整理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "magic numbers: 2051, image numbers: 60000, image size: 28*28\n",
      ">784B 16 784\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[WARNING] ME(21104:20600,MainProcess):2022-11-15-19:23:52.722.216 [mindspore\\dataset\\engine\\datasets_user_defined.py:656] Python multiprocessing is not supported on Windows platform.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "magic numbers: 2049, label numbers: 60000\n",
      "<class 'mindspore.dataset.engine.datasets.BatchDataset'>\n",
      "iter size is 468\n"
     ]
    }
   ],
   "source": [
    "# 创建训练数据集,GAN网络只需要训练数据即可\n",
    "def create_dataset_train(batch_size=128, repeat_size=1, latent_size=100):\n",
    "    dataset_generator = DatasetGenerator()\n",
    "    dataset1 = dataset.GeneratorDataset(dataset_generator, [\"image\", \"label\"], shuffle=True)\n",
    "    # 数据增强\n",
    "    mnist_ds = dataset1.map(\n",
    "        operations=lambda x: (\n",
    "            x.astype(\"float32\"), # 未归一化\n",
    "            np.random.normal(size=(latent_size)).astype(\"float32\")\n",
    "        ),\n",
    "        output_columns=[\"image\", \"latent_code\"],\n",
    "        column_order=[\"image\", \"latent_code\"]\n",
    "    )\n",
    "    # 批量操作\n",
    "    mnist_ds = mnist_ds.batch(batch_size, True)\n",
    "    # mnist_ds = mnist_ds.repeat(1)  # 数据加倍\n",
    "    return mnist_ds\n",
    "\n",
    "# 创建验证数据集\n",
    "def create_dataset_valid(batch_size=128, repeat_size=1, latent_size=100):\n",
    "    dataset_generator = DatasetGenerator_valid()\n",
    "    dataset2 = dataset.GeneratorDataset(dataset_generator, [\"image\", \"label\"], shuffle=False)\n",
    "    mnist_ds = dataset2.map(\n",
    "        operations=lambda x: (\n",
    "            x[-10000:].astype(\"float32\"),\n",
    "            np.random.normal(size=(latent_size)).astype(\"float32\")\n",
    "        ),\n",
    "        output_columns=[\"image\", \"latent_code\"],\n",
    "        column_order=[\"image\", \"latent_code\"]\n",
    "    )\n",
    "    # 批量操作\n",
    "    mnist_ds = mnist_ds.batch(batch_size, True)\n",
    "    # mnist_ds = mnist_ds.repeat(1)   # 数据加倍\n",
    "    return mnist_ds\n",
    "\n",
    "# 获取处理后的数据集\n",
    "dataset_mnist = create_dataset_train(batch_size=BATCH_SIZE, repeat_size=1, latent_size=latent_size)\n",
    "print(type(dataset_mnist))\n",
    "\n",
    "# 获取数据集大小\n",
    "iter_size = dataset_mnist.get_dataset_size()\n",
    "print('iter size is %d' % (iter_size))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 数据演示\n",
    "\n",
    " - 通过 `create_dict_iterator` 函数将数据转换成字典迭代器，然后使用 `matplotlib` 模块可视化部分训练数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASQAAAEeCAYAAAAjGGgnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAACScElEQVR4nOy9d5Rc533f/bnTe93Z3vtiK3YXvbEXsIkUqVhWtZxQcWwnduQkPnlj2fGxZTuKlLxWLMuW7Tc6aqYpUhRJkRTBBqIRxPbe+2ydnZ2dsjM79f0D515hCYAAyC2z0HzOwSE4ZfE8e+/9Pr/n9/yKkEgkSJEiRYpkQLbTA0iRIkUKkZQgpUiRImlICVKKFCmShpQgpUiRImlICVKKFCmShpQgpUiRImlQfNibgiDs2piARCIh3OxnU/NMfm52nr8Kc4Tbd54pCylFihRJQ0qQUqRIkTR86JYtGbBaraSnp/PII4+QmZnJ9PQ0XV1dvP322zs9tBQpUmwySSlIgiAgk8lQq9Wkp6dTXl7OAw88QGFhIW1tbfj9/pQg7RDitZHL5ajVauRyOQrFtW+jWCxGNBplbW2NWCy2zSPdOgRBQC6Xo9FoUCgUyOVy1tfXiUajhEKhnR7eriYpBcloNOJwODh58iQHDx7k7rvvxmg04vf7mZiYYGlpaaeH+CuJXC5Hq9WSk5NDdnY2TzzxBMXFxdTU1CAIV/spR0dH6e/v55vf/CYjIyM7MOKtwWw2k5uby5e+9CXKysooLS3lpZdeoqenhx//+MdEIpGdHuKuJakESafTYTQaqaqqIjc3l3379lFRUUFaWhpTU1PMzMzQ39/P/Pz8Tg/1VwpBEDCbzdhsNmpqasjOziY7O5v6+nqys7PJy8tDEAQSiQTRaJREIiH9v0Kh4NChQ5hMJnp6eohGo8Tj8Z2e0kdCrVaj0WhoamqipKSEqqoqzGYz4XAYhUKBUqnc6SHuepJKkDIzM6mqquKLX/wiFRUVVFZWIggCsViMU6dO0draygsvvEAgENjpof7KIAgCCoWC0tJSGhsb+cM//ENsNhsmk2nD58SqEYFAgFgshlKpJD09nezsbAwGA93d3fzH//gf8Xq9hMPhnZjKx8ZisZCVlcXv/u7vUllZSVpaGp2dnZw+fZrZ2VlWV1dJVc/4eGyaIBkMBvLz85mfn8ftdt/Sd7VaLRUVFRw/fpx77rmHqqoqrFYrMpmMkZEROjo6eOWVVxgcHCQQCBCNRjdr2Cmug7g9q66upqysjPvvv5+CggLS0tJwu92Mj4+zuLiI1+tlenqaRCJBIpHA6XSSSCTIzMzk4MGDHDt2jLy8PPx+PxqNhrW1tV0nSIIgoFKpOHDgAPfffz95eXl4PB7+7//9v4yPjzM6OkowGJTEeLcik8nQaDRkZmaSlZVFZWUl8XicixcvsrS0tC2ukk0TJLVaTVZWFl6v95YESaVSYTabqa6u5uDBg9x7773I5XISiQR+v5+xsTHOnDlDW1sb09PTmzXcFDdALpej0+koLy/n8OHD3H333ZjNZqLRKPPz8wwPDzMxMcHi4iLd3d3E43Hi8TiTk5MAlJaWYrfbOX78ODabDYfDgU6nw+fz7fDMbh2FQoHVaqW6upq7776bRCLByMgIL7zwAouLi7e8ACcrCoUCu91OUVGRdN0jkQizs7OEw+HdJUh+v5++vj5WV1dv+jsymYxHH32U2tpafu3Xfg2bzYZCocDj8TA9Pc03v/lNhoeHGRgYSG3TthmlUonNZqOoqIjS0lJefPFF5ubmeP/995mdnWVxcZFwOEwsFmN9fV36XiQSQa1Ws7y8vOGaaTQaqqurkclkt3SP7DRqtZqysjL+y3/5L1RUVGA2m/na175Ge3s74+Pjt5UDu6ioiL/8y7/EYrGg1+u5dOkSc3Nz6HQ6VCrVtoxh0wQpGo3i8/lu6gLJ5XKsVit2u53Gxkaqq6vJzs5GJpMRCoXo7u5mcHCQnp4e5ubmdtUN/FHR6/WYzWbJOvR6vayvr2942LeTeDxOKBRicnISo9HI4OAgCwsL9Pf3s7Ky8qHXRKFQSM5tEblcjt1ux2AwbMfwNwVBEHA4HOTl5VFdXU08HmdkZIShoSEmJycJh8O3lc9Io9FQUlJCNBolEAgwMjLCzMwMi4uL22YQbJogxWIx/H7/DT8nxhc1NTVx9OhRPve5z5GTkyM9hG63m29961u0trYyMzOza09kbpW8vDz27duHXq8nEonQ3t7O4uIiMzMzOzKeYDDIxMQEExMT0onZx0GpVJKdnc34+PgmjXDrkclk7Nu3j3379lFbW8vLL7/MCy+8QFtb220ZeiJeo+7ubi5dusRzzz3H1NTUto5hW0/Z5HI5GRkZPPTQQzQ1NbF3714sFgvr6+tMT0/T0dHB+++/T39/P263e8fESKfT8W/+zb/BZDKRSCQYHBxkamqKzs7OTQ98MxqNHD16lMbGRo4cOYJarSYWi3Hw4EHeeecdfvjDH27qv/dRuJ2sgJtFrVZjMBi47777qKioYHh4mI6ODi5evLgr/WC3gnhAsRPXfdsESRAEdDodWVlZ3H///ezZs4fy8nIAvF4vY2NjXLhwgZdeegmn00kwGNyuoV2FWq3mk5/8JFlZWQC89dZbtLW1MT4+Tjwev+a29GYvnhjlK5PJkMlk2Gw2jh8/zr59+zh69ChyuZx4PI7L5WJ5eXlT57WTJBIJIpHIrrF41Wo1FouF5uZmsrOz6ejoYGhoiIGBgZ0e2pay04vPtgiSWq1Gr9fz5S9/mbq6Ou666y40Gg0Ac3NzDA0N8bWvfY2pqSlmZ2d33FEoiqfFYsFgMPDII49w/PhxDh8+TG9vL88888yGz0ciEXw+H+vr69c90hYdgyaTiaqqKimoMD09nTvvvBO9Xo9CoSAWi+HxePj5z39OR0fHNsx2e/D7/bz++us7tgW9Vfbs2cP+/fux2Wy43W5+8IMf0NnZudPD2lLi8bh0H+8UWy5IMpmM3NxcioqKqK+vp7y8HKPRSCgUYnl5mba2Nvr6+hgbG8Pj8SRFjEosFmNwcBBBENizZ4906lBTU4NarcbpdG5Y6UOhEIuLi/h8PtbW1q5aZQRBwGazodfryczMpLy8nKqqKjIyMrBaraSlpSGTyUgkEiwuLjI7O0t7e/uuDnOQyWQb0klisRirq6s7avneCna7neLiYgRBwO/3Mzo6etsc71+PcDjMwsICkUgEg8GAXC7f9jFsqSCJgVaPPvoon/vc5ygrK0Or1QIwOzvL0NAQ/+W//BeGhoaSypT3+/187Wtf4/jx4/zxH/8xRqMRvV5PfX09dXV1PPHEExs+v7q6SldXF7Ozs7hcrmsKUmlpKenp6dTX10sPq/jAXvngnjlzhtbWVv7v//2/u+bh/SBidLdMtrG6TTwe3/Etwc1SVFTE4cOHJf9me3v7jlvuW43P5+PcuXNkZ2dTUlIi7WK2ky0VpPT0dB588EEOHTpEdnY2SqWSWCzG2toa3d3dvPHGGzvqvL4e8Xic+fl52tra+Pu//3tqamooLCyksrISlUp1VSKpTqejpKSEzMzM61pIVqsVnU53VWZ8NBpleXmZ7u5uLly4QEdHB06nMyksxY+K0WjkxIkTFBUVAZdP7Px+/67wIYmLqM1mIz09nf7+fkZHR5N+3JuBXC5Hr9djNBqlEJTtZssESSaTkZ6ezmOPPcaePXtIS0sDLt+cy8vL9Pb28vbbb+P1erdqCB+ZRCLB8vIy6+vrrK6ucu+999LU1ER2djZGoxH4pVUjl8ul41Lx9SsFSSzX8cGfD5eFb319HafTyTvvvMN3v/tdvF7vrl6JZTIZRqORQ4cOUVhYCFy2OL1eL9FoNOlTK8QIdYvFgt1uZ3p6msnJyWtWM7jdkMlkGAwGjEbj7bNlUyqVqFQq9u7dS3NzM8ePH5e2acvLywwODvIXf/EXjI2NMTExkdSWwNraGuPj4/zLv/wLr776Kj/60Y+kjO60tDRsNht33XXXhkRTUZDEeKusrCxJrESi0SiRSIShoSFGR0f567/+a2ZmZvB4PLt6JRYEgZqaGurr67njjjswGo3EYjF+8Ytf0NraysrKyo46TG8GlUpFdnY2ZrMZpVKJx+NhbW2NnJwcPB7PbR2kazAYaGxsRKFQ7NhzuemCpNFoMJlM1NfXU1NTg9lsljL2Z2ZmGB4epre394Y3p7jSqlQq1Go16+vrRCIRvF7vtj20ogWztLSE2+3G7/dLq4bdbsdqtWIymaToY3EVTSQSUgGv6upqtFotJpNJ+m4sFiMYDNLX1ydFpft8vqS3Hj4MMZyhsLCQ0tJSrFYrcrmcaDTKyMgIg4ODRCKRpPchiRaSSqWS4ubW1tYIhUIsLCxI90IwGLwtxUkMSdkpNl2QMjIyKC0t5fd+7/coKCgAkOrkPP/887S1td1UBLZWq6WxsZG8vDwKCwuZnJxkYWGBM2fOSNX5tgsxcfTKOkyzs7MAnD179prfEQXpk5/8JE888QSHDx/GYrEAly2vpaUl/s//+T+0tLTsaiESUSgU6HQ6HnjgAZqamqR4qvX1dd566y3OnTuX9GIEl+dhNpulapCf/exnSSQSxONxxsfHGRsb48UXX2RoaIh33nnntrh2IuFwGKfTicVi2bbctQ+yqYIkHm/n5+ej0Wiko+z29na6urp47733mJiYuEqMxPIOZrMZq9VKXV0dWVlZNDQ0YLVasVqteDwelpaWiMVijI+P73gFQvHhut4NKZ4oeb1eVlZWrvpcPB4nFovdNjd0dnY2xcXFVFRUkJubi0wmY25ujrGxMXw+364QI4D19XXm5ubw+/3E43Ha2trwer2EQiFMJhPZ2dmcPHmSxsZG0tPTpeTvYDC466/l2toavb29lJeXk5OTsyNj2FRBkslk2O12CgoKJF9LLBajra2NZ599lkuXLl0Vdi8eERuNRrKzsykqKuKpp56irKyMysrKDcfHLpeL6elpBEHYcUG6EWJk8traGqurqxssut3ycF6Lazl3E4kEWVlZNDc3U1paSlZWFolEQjqpvJkcx2QhHA5LghSLxWhpaWFqagqPx8PRo0c5evQopaWlBINB0tPTee2116TYnd0uSKFQiMHBQaxW6+4XJJ1OR25uLg888ABPPPEEFosFn8/H4OAgly5d4tKlSwQCgatOoBobGykrK+MLX/gCVqsVs9mM3W5HqVSyurqKVqvd4KNRKpU74v3/uIipIvF4nKWlJbq6unbVgypeh7y8PMn6CwQCkuju3buX+++/XyrpOjc3x6lTp/iHf/gHaXu720gkEszOztLX18fFixd5++23sdvt7Nu3j5KSEp566ilycnI4duwYf/qnf7rr00oUCgUWi0U6hNqRMWzWDxJPlHJycqQcML/fT2dnJ1NTU5IDULSIDAYDJpOJuro69uzZQ2Njo+RI9Pv9BAIB3G43DodDEqREIiHV4El2RPE0Go1SnScRscribuhQodfr0el05OTkYLVaKS0tJZFISI759fV1fD4f5eXl5Ofno1KpiMVirKysMD8/z+Tk5K64XiKivygajRKNRiVLf3FxcYOFHgwGuffeezGZTNTW1pKZmYnT6dz1ibdimIq4MxEX0e1i0wTJZrNx7NixDUfcExMT/Pf//t/xeDy//AcVCtLS0jh27BjHjh3jscceIyMjA7lcztzcHE6nk/feew+Xy8X6+jpHjx6VzMdwOMzExAQul2uzhr1lKJVKMjMzaWho4L777sNkMkkXdnJyknfeeSfpT2lkMhm1tbXU19fz2c9+Vuo2ciWRSASPxyOdNgqCwOrqKhMTEywvL+8qMYLLgiSK7OrqKg0NDSQSCV5++WVJpM6fPy8VDXzwwQd59NFHOXDgAIIg8O677+66OYtEo1FWV1eJx+Po9Xr0ej1arXZbiyNuqg9JLpdvSImIxWIbirqLEbD79u1j//797N+/H6vVSiKRYGpqivb2dt5//30pdKCiooLS0lIEQaClpYXBwUH6+vqYm5vbzGFvCVf2ltPpdMhkMqLRKIuLi8zNzUllQZMVvV5PQUEBx48f58iRIxQUFKDT6fB6vWg0GtRqNQqFQrL8roxgFwQBjUYjhWzspkJm8XiccDgsWXiZmZnk5eWh0WgIhULEYjHi8TiBQICOjg7Ky8tZXl6moqKCcDjMuXPndq0gra2t0dnZSW1tLQaDAYPBsO2CtGkBB9fKy4rH4wSDQaLRKIIgYDAYyMzM5MiRIxw+fJh9+/ZhMBgIh8OMjo5y9uxZnn32WbxeL+np6dx///1UVVWRSCR47733eOWVV+ju7t41giS2xtFoNAiCIPlWZmZmcDqdSR0kaDQaqa2t5c477+SRRx4hNzcXtVrN3NwcHo+HUCgkBYDq9foNLYDE14xGIzqdblf5/ES3gNvtxul0kpmZSX5+/lVzDAaD9PT0MDIywvz8PFVVVTQ3N++quX4Qv99PW1sby8vLUsS2Tqfb1ij1TbOQPqyok9jH63Of+xylpaVUV1djNpslwZqfn+fdd9+loKCAv/zLv6Sqqgq73Y5cLmd0dJSenh5+8Ytf0Nvbu6s7joilRVZXV1ldXU3alVSr1ZKZmSltlxOJBDMzM3R1dfH1r3+diooKKioq+OxnP0t6evpV39fpdNTX16NQKCgrK+M73/kOo6Oju8JnJtLZ2YkgCFRWVpKdnc2DDz5Ia2srPT09Gz43PT3N2bNnpfnuZjQaDYWFhVK83E6wqb/BD4qS2N4oJyeH/Px8GhsbKSwslJze4nfgl+UzGxsbsVqt0tH+wMAAbW1tUsfa3ZpaIQgCkUiEmZkZlpeXk1ZYBUHAZDJJAa6i72tubo7x8XG6u7vR6/Xk5uZuuBZi/W+xzbbJZKKoqAi5XM5bb71FKBRiYmIiaUX4g7hcLoaHh1lbW0Ov11NZWcnU1NRVuYo+nw+n00ldXR1KpVKq5LBbtqhXIl63ncjyF9nUmtofPAGrq6vjlVdeQaPRoNFo0Gq1V4Wl6/V6ioqK+IM/+APJqz8wMEB/fz9/9Ed/hNvtlhIzd6sYweUHfWlpib/+67/G6XTu9HCui0KhYN++fVLrI7lcTjgc5uWXX6a3txeNRsPhw4f53Oc+h91ul743Pj7O4OAgWq0Ws9nMvn37yMjIwOFw8J//83+mq6uLP/zDP5Sy/pMdp9OJ2+1mZGSE8vJyHnzwQWZmZrhw4QLr6+uS4Ph8PmZnZ9FqtWi1WnQ6HZFIJKn9g9dDbO6p1+t3bAybJkg+n4+2tjZKSkrIyMggLy9PaqUjOj8/KEbi3jQej+P1eqUi/x0dHQwPD7O0tEQgENiVF/eDiEfluyFsIRwOE41GpUMKuVxOTU0NaWlpVFdXc+DAASwWC3K5nEgkwvLyMh0dHbz77rtoNBocDofUcDAjI4Pc3FwikQh33HEHw8PDDA0NJX1em+hLevvtt1leXua+++6jvr6ehx56iPfee0/KZ7NYLFKKlBiXlezX93qItZ928vR30wTJ7XZz+vRpCgoKsNvtpKWlSYXNYGN0svh38b9iCY6xsTGGh4d5//33mZmZkY4gbweuLDmSzIgNOgOBAPF4HLlcjlwu59ixYwiCQFpamiRUcPkhHB8f58yZMzzzzDNoNBpycnKwWCw0NTWRnp5OTk4Oer2exx9/nLfeegun07krLKVYLMYLL7yA0+nkySef5NChQ2RmZkopTOvr6zgcDmpqaojH46yurrK+vr5rBSkYDDIwMLCjYTWb6tSORCKcOnWKnp4enn/+efLz87n77rspKSkhNzeXqakpXC4Xvb29TE5OSvWV19fXmZmZkWI/xAub7A/vh2G1Wjl58iRlZWU7PZRbIhaL0dvbi1Kp5NVXX5WK09lsNslaEgSBeDwuhWL84Ac/kHLWAoEAoVCI73//+/T19TE8PCx1r7377rvZs2cPDz/8MD/+8Y+lrXmyXudEIsHS0hLt7e389//+3zl69CgHDhzg93//95mZmaGzs5NDhw5x9OhRurq6fmUKuW0lm+7Unp2dZWlpibm5OclaWl9fJxgMMjo6KuU3jYyMMDY2BlwOrnO5XEQikaRfNW8WtVpNUVERVqt1p4dySyQSCVZXV3E6nXR3d0s1v8VYKrH/ns/nk8qntLa2EggEJEd9LBZjaGhI2qpnZWWRl5eHw+EgPT1dSlFQq9VJX/hsfX0dl8vFhQsXyM7Opq6ujuLiYtLT01Gr1RQXF6PX65mfn2dqauq2ECQx8Ruunbu4lWz6OaUoKtPT0zidTi5duiTVWBHLeIhZ7ldevNvhQl6JWq3e8SPUj8P8/Dw//OEPWVlZweVycdddd6HValldXeVnP/sZr7/+Op2dnbjdblZWVjZsycXUkffee4+WlhZOnTpFbm4ujz/+OPPz8wwNDfHuu+8yPz+/K7Y3fr+f1tZW3G43b7/9Nl/84hcpKyvjs5/9LKOjo7z11lv80z/9EwMDA7t6QRXLBPn9fmnbdmXs1XawZYETovgk6/H2VhOJRFhcXCQ/Px+4vNKYzWbuv/9+qWDZTvef+zDEwnStra2srq4yPj6OUqkkGAxKnWIWFxcJBoPXdE6LN3c0GmV+fl6qi+T1ellYWMDr9e4KMRKJxWJSn7xf/OIXdHZ2kpuby/z8PNPT08zOzrK2trbDo/z4XBm2s2/fPjQaDcvLy6ytrW1LIO/ujuRKYkKhECMjI1IrHbFP/NNPP82FCxc4ffo0r7/+etIKUiQSYWlpibfffpu33377Y/0st9uN2+1mdHR0k0a3M6ysrLCyspL0pW82A7lcLvlAW1paWFxc3BZB2rlalbc5Pp+PCxcuMD09La06CoWC3NxcDhw4wJNPPik1PkiRIhkQrdq5uTn6+vpQqVRkZGRQVVV1zYj8rSBlIW0R4XCYmZkZ3G43a2trUgVNg8FARkYGiURiRyNiU6S4FmIb96GhIZRKJYFAAJVKtW1pMSlB2iLEk8Pe3l7Onj3LgQMHMJvNwOXtnMfj+ZX1r6VIbk6dOsXp06dRKBQkEglCodC23aspQdoixEhfp9NJe3s7MplMapc0OTnJ8PBw0tdDSvGrSTgcvn3aIKW4jChIg4ODrK+vMzw8LEWtj4yM0NfXx8LCwg6PMkWK5EL4sHwiQRCSN9noBiQSiZuO6NrKeep0OnQ63YZOoGJgYSgU+tjxV8kyz63mZuf5qzBHuH3nmRIkUvPcDaQEaSO36zw/VJBSpEiRYjtJxSGlSJEiaUgJUooUKZKGlCClSJEiaUgJUooUKZKGlCClSJEiaUgJUooUKZKGlCClSJEiaUgJUooUKZKGD81lu12jQT9Iap7JTypSeyO36zxTFlKKFCmShpQgpUiRImlIlR9JkeIGWK1Wjhw5Qk1NDZWVlcRiMSYnJ/na176WKrK3yaQEKUWKG2AwGGhububEiRMcPXqUaDRKe3s7/+N//I+UIG0yKUFKkeIGWK1WHnnkEXJycpDL5USjUWQyGVqtllgstqt7sSUbWyZIMpkMvV5PUVERSqUSpVKJ1WqVGs9FIhHpYoqlXr1er9SvK1UWJUUyIJfLUSgUUisr8TWtVktBQQGzs7MsLi7u8Ci3D5lMhkqlIjMzE5/PJ/Wq2yy2TJBUKhUFBQV8+ctfxmazYbVaOXToECaTiUQigdfrxefz4ff7mZub45lnnqG3t5euri4CgcCuaiKY4vZF7BaztLSExWIhPT0dpVKJzWbjzjvv5OLFi79SgqRSqUhLS+Ohhx6iu7ubd999d1N//qYLUlZWFhkZGTz++OMUFRXR1NSEWq1GpVJJbX8EQUCr1SKXyzEYDJhMJj772c+ysLDA3Nwcr7zyCuPj4wwNDd12LbZvBxQKBWq1mjvuuAONRsPCwgJ79uyhsrKS06dP43K5CIVCuFwuFhYWCIfDu/I6yuVy7rvvPurr66mqqtq1bdE3C0EQOHDgAMXFxTQ2NuJ2uzf939h0QXI4HFRUVPD444+Tl5eHwWCQWipfq5uBKEhZWVmsr68TCARwu92oVCrGxsaIRCKp7VsSIZfLMRqNWK1WDh8+jNFoZHh4mLvuuovjx48Ti8WYmprC5/MxMTFBNBrF5XJtS9fTzUYmk1FfX09TUxPZ2dnIZL+MkhGbKu52S150p8DlnmyhUOi6nxUEgT179rBnzx7y8/O3RKA3XZAaGxt5+OGHyc3NRSaT0dvby/nz52lra9vwOaVSiUaj4XOf+5w0OaVSicVi4bd/+7fp7++nu7sbl8uF1+vd7GGm+AjodDoKCwv5whe+wBNPPIHVakUulxMOh6VmBr/5m79JJBIhHo/j8/lYWVnh3/27f0d7e/uus5IEQcBut2Oz2ST/kShEKysrnDlzhtnZ2R0e5cfjvvvu46677gJgYmKCb3/729cVWUEQOHnyJHV1dbz99tusrKxs+ng2TZA0Gg1paWnk5eWRm5uLWq3G6/XS2dlJZ2cnXV1dGz6vVCpRqVTk5+eTn59PWVkZaWlpOBwOrFYrmZmZZGZmsr6+/isnSIIgoFarMZlMUrvtcDjM+Pj4jq7ISqWStLQ0srOzKSwslLqoANIDa7VaJYvWarVit9spKipiYWEBp9O5a6xduVyOWq3GbDZjNBo3vLe2tobX62V5eZlAILBDI/x4CIKAQqEgMzOT8vJyFhcX0Wq1N/yeyWSSdj1bcS9umiClpaVx8uRJ9u3bR0lJCUqlEqfTyXe/+13Gx8dxOp3X/N4777xDWloaTzzxBHfffTcPPPAAKpUKlUrF3r17icfj1/3u7YpcLic9PZ29e/dy3333IQgCy8vL/OVf/uWOPgAajYaCggIsFssGMboeYgvmO++8E4PBwA9/+MNdc0Su0+mw2+3k5eWRlZW1wUJaXFzE6XRK/rHdiEKhwGw2k5+fT2VlJUNDQywsLNxwwUgkElu6qGyaIBkMBmpqasjOzkav10vtosfHx/F4PNf9XiKRwOfzcfbsWTweD2NjY+j1enw+H+fOnWN+fn6zhrgrMJvNZGRk8PTTT2O327FYLLz66qv09fXt+M0vCAJyuZzu7m68Xi9zc3PSmPR6PUajkaqqKtLS0iguLkYQBGQyGUePHiUrK4uVlRUGBwcZGBjY0Xl8GHK5HL1ez549e6iqqiIzM1Nq8BkIBPB4PPz4xz+mo6NjV/uP1Go1ubm5ZGRkYLFYGBkZYXh4+IZic2X4w1awaYIk+hdsNhtqtZpgMIjP52Nubk7yHYiTkcvl0sRjsRjr6+v09vaysrLCxMQEZrOZUChEd3d30l90cT5qtZpIJEI0Gv1YvhJx1XriiSdYX19nZmaGwcFBWlpadty6SCQSxGIxSVT6+vpYW1sDwGaz4XA4CIVCFBcXk5OTg0qlQiaTUVtbS2ZmJkNDQ4TD4aQWJPFIv7y8nP3792O326XTYb/fz8LCAqdOnaK3tzfp780PQ61WU1BQgMPhQK/XMzs7y8zMzE1bP/F4fEt8gtsSqX1lYKTVaqW2tpZoNMr6+jo9PT14PB5WV1dxuVysrKwgk8mkmz+ZEQSBrKws6urq+MM//EN++tOf8vOf/5zp6emPfKp0xx130NzcTFpaGqdPn+bv/u7vpHbcO43L5eLFF19EEAQSiQTr6+vSDTwxMYFcLqelpYW8vDy++MUvsnfvXpqamoDL27eioiJ6enp2cgo3JD8/n//4H/8jdXV1VFRUYDKZpPdaWlr4+c9/zujoKKurqzs4yo9PYWEhX/3qV8nJySEWi+F0Opmbm7vh98QW8VNTU7vj2B+QTHW9Xk9ZWZl0tC86rKuqqiRBMpvNeDweFhYWCAQCUvTnTm9PboRMJkOpVFJRUUFNTQ0VFRVYrdaP/PNUKhVarZaSkhKKiopYXl5mZmaGkZER/H5/UjiDY7HYDR/EYDBIPB5nZGSE/Px86XWFQkFWVhZ2ux2tVsv6+nrSnboplUqMRiPFxcVkZ2dL1zMSibC8vMz4+Dj9/f34/f6kG/utoFAoMBgMFBQUEA6HmZ6exu/3f6gFLt7vgiAQjUZZWlraEn/mlllICoWCwsJCfud3foeKigpKSkrIyspCpVJt2IMmEgnW1tYYHBxkaGiInp4eXnnlFaanp1lZWUmKB/FaKJVKzGYzTz/9NBUVFSiVStbW1lhaWvpICZc2m43CwkLuvfdeysrKeO655zh9+nRSb2+uRTQaZXV1lffff5/S0lLpdZ1OR3NzMwMDA+Tk5OB0OgkGgzs40o0IgoDVaiU9PZ2CgoINlpHH4+G1117jjTfe4N13303ae/JmMRqNmM1mzGYz7777LmfOnPlQPy8gnTgqlUrJxbIVIQ9bJkhqtZq0tDSOHDmCzWbDYrFIPoUrEQQBjUZDXl4eer2e3NxcrFYro6Oj/PSnP8Xv90t+imSiqqqK2tpaSkpKAHjuuefo7u4mFAp9pNUzNzeXu+++G71ez8rKyq4UI5FoNMr8/Dyrq6tEIhEUCgWJRIJIJCIFvu60P+yDKBQK7r77bpqamkhLS9twBB4MBhkeHsblcl1XjAoLC7FYLFitVkwmE1arVbL6XS4Xbrd7xw9oRCv8rrvuYt++fQiCwPz8PD09PR8aEAmXA57LysqQyWQEAgH8fv+W7GK21EKyWCw0NDRIryUSCeLx+AbfkEwmQyaTSTFIlZWVZGVlMTAwwMWLF5mbm0sqQRKd2FVVVdxxxx1kZWUxOzvLK6+8wtDQ0Efy9QiCQF5eHkeOHEGj0bCyssKlS5d2bY5UNBpleXkZn89HOByWDjHC4TB+vz8pLV+FQsGxY8eor6/HarVKfrJEIkEwGNxwWiyXy6X7VnRPlJWVkZ2dTX5+Pjk5OeTn5zMwMMDS0hLDw8OMjY2xtLS0o35RtVqNxWLh6NGjVFdXSwvHzdy3NpuNyspKSZDW1tZ2lyBdiShCMzMzOJ1O3njjDenC7Nu3T1pV0tPTycrKIi8vD6vVyje+8Q1eeuklvve97+Hz+ZKi9kxWVhb33HMPjz76KPv27eOVV16hu7ub06dPfyThVKvVlJWVsXfvXg4cOEBXVxf9/f0sLi7i9/u3YAZbjxjJvLS0xMLCAtnZ2dJq7PV6k06MDAaDtGUWxyoyOjpKV1cX58+fx+fzYTAYePjhhyksLKS6upr09HQyMjLQ6/VSsK/436amJslXeubMGX70ox/R1tbG0tLSjsxzz5491NfXc//99wPwj//4j7zxxhsMDw/f0EIqLy/n0UcfRRAEFhYWmJ+f310+JLgcXbyysoLP58Pn8zE8PMzMzAytra2SIMXjcex2OyaTiaqqKoxGIzqdDrPZTFVVFePj49TW1tLT07OjK6sgCBiNRrKzs2lqasLhcBCNRunr66O/v5/V1dWPNDbx9CkrKwuDwcDMzIy0YiW74/TKmJQPBsx9sIRMNBrF7XYnlciK1m5GRgb5+fnY7XYMBgPwyxQR0bcZjUbJyckhIyODffv2SQGFFosFu92O1+uVtqEymQyFQkFaWprkopieniYvL4/+/v4dm29WVhbV1dUYjUZcLhctLS1MT0/flC9Pq9Vis9nw+XwsLS0RDoeTO1L7g8RiMRYXF3n11Vdpb2+np6eH7u7uq06MTp06hUwmQ61W8+lPf5p//a//NZWVlRiNRjIzM3nooYeoqanhD/7gD3j//fd37PRNLpfT1NTE4cOH+dKXvkRXVxdvvPEGzz//PLOzsx9ZKE0mE4888ohkQr/88sucPn066U8ZgQ0+QTEGCy4/6CqVCqPRiMPhkBz+vb29O+5HuRIxWvmee+7hzjvvpLy8HLPZDFwW0EAgwHe+8x1GR0cpLi7mC1/4Ar/+67+OSqWSDmN8Ph/j4+OcO3cOt9uNVqslLy+P0tJSioqKJIGzWq1UV1dz4cKFHZtvU1MTTz31FF6vl66uLn7wgx/c0q4jkUjQ2dnJwMDAlm09N02QwuEwHo8Hl8uFWq2mr6+PsbExXnzxRZxOJy6Xi7W1tWtORFxNW1tbUSgUfOlLX6KkpASLxYLBYCAvL49Dhw6hVCo5e/bstjtEDQYDVquVkydPUlFRgcfjoa2tjbfeeovV1dWPfHHS09MpLCyktrYWlUpFb28vCwsL+Hy+pNvWXElhYSH5+fkUFRVJAaFisKTX60Wr1bJ//35KS0tRq9XIZDLC4TCjo6M7tl25FmKhtfz8fGprayVHdiKRYGlpCafTiVarpbCwkCNHjlBeXo5Go0EQBDweD6dOnWJiYoLx8XEmJycJhUIolUpKSkqYn5/HYrFIUd7AlkY4Xw8xX00c//r6Os899xydnZ1Eo9Eb3mdiiSCLxYLZbCYSiRAKhdDpdMhkMjQaDaurq4TD4U1xqWyaIIllJubn54lGo7z11lv09/fzyiuv3NTDFY1G6erqYnBwkEOHDmE2mzGZTOj1evR6PYcOHUKtVnPp0iVisdi2bmdMJhM5OTncf//9WK1WlpaWaGlp4Y033vjI+2hBEMjMzKS4uJg9e/YwMTFBR0cHi4uLSZuwKW5xSktLOXToEIcPH5bShF599VU8Hg+RSASLxcKxY8coLS1FpVIBlxessbExXC7XDs/il8jlcnQ6HXl5eVRVVW14b3FxkcHBQclSv+eee8jNzUUQBGKxGMvLy7z66qt0dHTQ0dGx4buVlZWsra1x7NgxcnNzt3FGVyPmH/6rf/WvyM3Nxe/38/zzz99Umghc/h2JAc1ms1nyB5tMJrRaLRaLhVgshtfrTS5BmpiY4P/9f/9f1Go1SqWSlZUVgsHgLa30omL/3d/9HRcuXODP/uzPMBgMKJVKTpw4QX5+PqdOnWJ6enpbyz7ce++93H333ajVarq7u/nmN7/J0NAQgUDgIwmjWHrlgQceoLGxEaVSydDQEP/8z/+cVFuaK9FoNNK2paGhQfL3iVH1+fn5fOITn8DtdkuiJVYqgMtWcCAQSKqtqNVq5dixY+Tk5EivxeNxgsEgo6OjtLW18cQTT5CTk0NpaSnxeJylpSV+9KMfSfFy1/KJlZaW8thjj5Genr6d07kKlUrFJz7xCZqamjh27BgtLS10dXUxNzd304uew+Hg3/27f8f+/fuxWCycPHmSI0eO8OlPf5p4PE4ikeDP//zP6ezs3JQxb5oghUIhJicnP/bPicVijI6OolQqmZ+fJyMjQyqBu7a2Rlpa2pbUYfkw0tPTKSoqksqZCoJw3eJUYrwN/DKkQfyjUCikSgZiAmdhYSE+n4/5+XnGx8dveNqxEwiCgM1mIy8vT6rm8MGVX61Wk5GRgc/nQxAETCYTarUaQCrQJ/4e5HL5jh5/i9cvOzv7qgj7YDDIxMQEq6uryGQy6RhfrVYzMzPD1NQUra2tDAwM4HK5NuRpKpVK0tPTycvLIz8/H41GQzwel1wZ212oTiaTkZWVJUWd63Q61Go1FRUV2Gw2vF4vCoUCheKyDMTj8Q21rQCys7Opr68nJycHhUKBw+FAq9Uik8nw+/2Sy2KzdixJ2XVkfn4eQRB46aWXOHjwIMePHwcuWxZ5eXmbXlj8Ruj1eux2uySOZWVl9PX1MTU1ddWFSCQSUkKxXq9Hq9Wi0Wiw2WxSuL5Go5Ee4PX1dd555x06OztxOp1JkbP2QRQKBcePH2ffvn0cPXr0mqVH1Gq1VMMJNvpLVlZW8Hq9OBwOKRwgEAjsWBiHUqnk/vvvp6GhgSeffBKbzSa953Q6+eu//mtKS0s5fPgweXl56HQ6lpeXee655/jRj37EyMgIa2trG669Wq0mMzOT3/qt36K+vp7CwkK0Wi2BQIBXX32VtrY2Tp06xcLCwrbNU1wYDAYDgiBw5MgRDh8+zNNPP83c3ByvvfYaaWlp0vyDwSCTk5M0NTVJ8YOi0F55Pd1uN3/7t38rFVH8uAnlV7JpgiQGiNntdoxGI1NTUx/L+ezz+Th9+jRms5n9+/dLKSdarVZaebeLixcvAnD06FFpP52bmytVubzy6Dsej0uxNmq1WupaoVQqpe2NWOJCJpMRDAbp6OiQfl/J6MyWyWTSqZFcLqe1tZXe3l4qKyux2+2UlJRIv4NrOW51Oh2ZmZk88sgjTE1NMTo6yltvvYXT6dz2MACxfdH+/fupqanBarVuuJ8ikQhLS0uUlZVhs9lQKBSEQiEGBgYYGxtjdnZ2QzS+TqfDYDDQ0NBAcXExBw8eJDs7G61Wy/j4ODMzM7z++uuMj4+zsLCwrQtOIpFgfn6egYEBKX9Nq9VKJX/Eap9iAcRIJIJSqWR5eVkKT9BqtZSXlxMKhfD5fLz44osMDAzQ1dW1JfWgNlWQxBorOTk5LC4ufixBCgQCvPXWW5SUlBAMBlEoFFLCrugo3S7Onz9Pf38/oVCIsrIyDh06RFpaGvn5+RgMBunoWyzJIN6sMplMcgKGQiHW1taYnp6WhCoWi+Hz+WhpaWFsbCwpAj+vhVwup7y8nOLiYuDy7+MHP/gBn/zkJ9mzZ48kVNdDPJh48skncblcTE9PMzMzg9frJRAIbKsIKxQKtFothw4dory8fEPOGlx+KN1ut5TbJpfL8fv99Pb2Mjk5KZ0SisJrNBrJycnhnnvuoaamhsOHDyOXy4nH4wwNDdHR0cHLL798w+TVrSAWizE9PU0gEGBiYoL09PQN1iAgtR67kqmpKaamphAEgbS0NEpLS6XuQP/8z/9Me3v7DXPfPiqbJkhFRUX8+q//Oo2NjeTl5aHRaBgcHLyqdO3NIqYaiOagaHHU1tZuewVJv99PKBTihz/8IVqtFqPRiEajkfpTXXmSFI1GiUQikqkr9p0T87pisRiPPfYYn/nMZ1hcXGRkZIS2trYtu8AfF4fDQXZ2NrW1tWRnZzM6OioV8/rud79LUVERq6urNDQ0UFNTs+G7YgkZn88HXHYia7VaHA4HiURiR/xl4kmh1Wq9qjTtlbjdbqampigsLMRsNnP33Xezvr6+wR+mUqk4ePAgJ06ckBJy5XI54+Pj9PT08E//9E/09vbi8/l2xGcWjUY5e/YsSqUSuVy+oaD/zVBWVkZ1dTWPPfYYFy5c4Mc//jEDAwPS9dwKNk2QjEYj9fX11NTUkJOTQ11dHYlEgomJCUKh0C2bduIDLZfLpYhgUaS2++LGYjFptRERt2EZGRmSU1C8Wa8lSF6vF0EQcDgc+P1+BEGQ/CkrKytJ6cyGy74Rg8GA2WxGq9UyOztLIBCQEizF8rpitK/o1F9aWmJ9fZ1gMMj6+jqCIEi5bcFgkGg0uiNxOeK2emVlhdXVVYxGo2R9i/PNycnBbDZL4xMXnvLy8g35aBqNhr1799LQ0CBtwQOBAIuLi4yNjTE6OsrU1NSOWb6JROIj1ywSBEHycYrlRvr7+6VGrlvFpgiSQqHAarVSX1+Pw+FArVbzW7/1W/T19RGJROjt7WVkZOSWfqZSqSQ3N3eDV9/r9fL973+f0dHRzRj2x0IUn5s5WRS3JA6Hg8cff5zDhw+Tm5vLu+++S3t7e9IXorsSuVwu5WuJlkZNTY10xB0Oh5mcnOSb3/ymFDBYVFSETqeTVmiVSsXKygomkwmPx7OtW7ZIJILH4+Hv//7vqa2t5ZFHHsHhcEhbt6KiIv7sz/5MOn3S6XTI5XIsFgsPPvgg991331W/D3G7GggEaGlpoaenh66uLtxud1KFOdwKMpmMxsZGGhoa8Hq9eDwePB7PlovrpgiSGLsxPz+PXq+XAhrz8/N59NFHpZw0j8dDIBBgdnYWt9t93W2K0WjEbrdz3333sWfPHhQKBZFIBL/fL5W1SBZu5WESb+z19XXGxsbo6Oigp6cnqXPWVCoVOp1OsvjS0tIoKiqivr6evLw8SkpKyM/Px2g0Eo1GOX/+PAMDA3R0dEhlN2KxmBSxLTr5xZO27XbiixZcV1cXoVCI7OxsGhsbJUESS9iKhzRXlssRx34lYvLs0NCQ5MCemZlhcnIyqfL2bgUxLKWsrIycnByGhoaYnp7ecusINlGQ/H4/IyMj2Gw2MjIyUKvV5Ofn88UvfpGZmRnm5uYYHh5mbm5OchJfr72R3W6ntLSUX/u1X6OgoAClUsnq6iperzfpBOlWEJssiu2hLly4QH9/f1JbSFqtVvKNqFQqMjIyqK6uxuVyceLECXJzc6WgwVAoxCuvvEJraystLS2S0CabfywajdLa2sri4qLUHlusayWXy6V8tmtx5eIhuhCCwSDnzp2jo6ODH/3oRwSDwaS+pjdCq9Vit9upra0lKyuL1157jdHR0W25jpvmQ5qcnOR//a//xeHDh6mrq+Ohhx7CZrOh1+tJT0+XiteHQiHuueceSWCuhcFgwGAwUFpaKuUXvfnmm7S2tuL3+5P2NOpGuFwu/uEf/kFqDzQ5OZm0aSIiYkG1Kx+wAwcOUF5ejtVqRalU4vf7aW1t5dKlS1IkfTJbfSJLS0v8/Oc/x+v10tLSIpURsdvtFBcXk5eXB1x2cJ85c0bqvPzGG2/gdrslp308Hmd6eprV1VWphO9uRqPRYDKZMBqNhMNhnn322U0Jer4ZNk2Q/H4/fX19qFQq6Xg8Ozub9PR0NBqNVFcbfmk2X+8YVMwil8vlBINBFhcX6evro7e3N2ljdW6G9fX1W/al7TRiC/Qrf+dpaWlSWkgwGGRqaor+/n7ef/99ZmZmtj2S/qMSCoWYnp6mu7ubtbU1cnJyMBqNpKenEwqFpIOGxcVF2traWF9flwJZryyeJ1pKu12IRBQKhXSKLGZObNeuZNMESawfI5YaaWlpITc3l+PHj3Pw4EEaGhowGo3SqZmYQnEj2traeOmll3j++ec/drBlis0lHo8zMzPDN77xDdra2qSo3d1EIpGgt7eX/v5+KS1IDA0QndXxeFy678TF9HYRn2uh0Wgk6zcSiUiW33aw6akjogk7NzdHKBSSKgV2dHRQXFwsOazF/DSFQiE5C2OxGJFIRNqvOp1OOjs7uXTp0q4+sdjNrK+v4/f7WVxcxGQyYbFYpGC6S5cuMTw8LCVs7tbFQgzrSHGZKwvvqVQqCgoKWFhY2BbLd0ty2RKJBMvLyywvLzMyMsJrr72GQqHg6NGj5OXlUV9fT3V1NVVVVVI2v16vJxwOEwgEOHv2LH19fbz55pu4XK5dW1v6diAYDOJ2u5mcnJTSJBYWFhgZGeGv/uqvGB0d3TVbtBQ3h1j9Mx6Po9FoqK+vlyq2bjXCh/ljBEHYNGeNmDEuntoYjUYMBoNUMF0Mt49Go8zNzeH3+3G5XJIj8VZJJBI3HXW3mfPcbrZ6nmKEb01NjVRW4/z581y6dImBgQECgcC2WEY3O8/Utfz4iAtPVVUVMpmMwcFBvF7vpoUxfNg8t02QtptkubhbzXbNUyaTkZeXx1NPPcXZs2dpaWnZVn9RSpA2crvOMyVIpOZ5s4i1z6+sn71dpARpI7frPJOyHlKK5ESMyE+RYquQ3fgjKVKkSLE9pAQpRYoUSUNKkFKkSJE0fKhTO0WKFCm2k5SFlCJFiqQhJUgpUqRIGlKClCJFiqQhJUgpUqRIGlKClCJFiqQhJUgpUqRIGlKClCJFiqQhJUgpUqRIGj40ufZ2zSj+IKl5Jj+pbP+N3K7zTFlIKVKkSBpSgpQiRYqkISVIKVKkSBpSgpQiRYqkISVIKVKkSBpSJWy3GLHnXE1NjdRt5YOIDRfHxsYIBAKpHmFJhiAIKJVKlEolCsXlR0Yul6PT6cjIyCAnJwe43DJqcHCQYDAo/UnGa6lQKNBqtWRlZaHT6dBoNCQSCZRKJTabjUgkInX19fl8LCwsbN/Ytu1f+hVFrVZjMpn4yle+wt69eykvL5ea8MEvO6H++Mc/5hvf+AYTExOb1m4mxeYgl8ulJplmsxm43N01Pz+fRx99lKeeegoAp9PJn/zJnzA9Pc3MzAzT09MEAoGdHPo10Wq15Ofn84lPfIKioiJycnKIRCJYLBaOHDnCysoKi4uLfP/736e7u5tXXnll2zr1brsgKRQKTCYTBw4cYP/+/aSlpbG2tsabb77J6Ogoo6Oj2z2kLUEmk6HVamlubmbfvn3U1NSQkZHB+vo6Ho+H5eVlFAoFarWa3NxctFotFotFWoFTJAc6nY709HTuv/9+SktLKSwsBC7fx0ajkfz8fMQih2azmc997nPMzc0xMzPDd7/7XUZGRnZw9BuxWq3Y7XY+//nPU1hYSGVlJUajEb1eTzweR6VSIQgCer2ezMxMHn30UQ4cOEBTUxPvvvsub7/99paPcVvvfrGNTlZWFgcPHuTJJ5+kqKhI6nK7trb2oYIkms7xeFz6k4wIgoBKpSItLY2amhruuususrOz0Wg0LC8vMzs7y9TUFCqVCoPBgMlkIhKJoFQqN1hPuwWFQoFKpUKlUl3z/XA4TDQa3XWt0AVBwGw2k5uby9GjR6mrq6Oqqkp6T7xW4n2o0+k4evQoLpcLp9PJiy++yOTkZNK0GNfr9WRkZPDwww9Lbe0/SCwWQ6lUolKpaG5uJhgMUlJSwurqKqdPn97yZ27bBEkQBLKzs6mpqeGrX/0qOTk5ZGRkSJPPzs6WzOFrIZPJMBqN3HvvvbhcLoaHh1leXiYUCm3XFG4ao9FIeXk5f/RHf0RxcTGFhYU4nU6Gh4f5q7/6K1wuF6urq8hkMhQKBVarFa/Xy9LSEmtrazs9/FvCYDCwZ88ennrqKR5//PGr3k8kEjz//PN0d3fzzDPPJM3DeTMoFAq+9KUv0dDQwJ133olarb6p74kWxtGjR9FoNJw+fXrb+9hdi0gkQiAQkBaID7K+vo7T6cRqtWK1WiUDwuFwkJmZSXZ2NouLi1u6sGyLIKlUKrRaLU1NTdTX11NSUgLA8vIyHo+H+fl5JicnP7R3eF5eHjk5ORw8eJCZmRmpR1iyCZIgCOTn51NZWUlFRQVWqxWA7u5uurq6GBoawufzEQwGEQQBmUyGy+UiFosRiUSS0gn6YZhMJo4dO0ZDQ4N0Xa8kkUiwd+9eZDIZb7zxBl6vd1eIbmZmJjk5OVILcaPReNPWq0KhQKfTUVlZSTQaZXBwkNXVVXw+3xaP+sNZX1/H5/PR29tLJBKhsrKS5eVl6bkLhUJMTU1hs9lwOBwUFBSg0WjQaDRYrVYyMzNZXV3d/YJks9nIzc3lD/7gD6isrMRut9PT00NPTw/nzp1jcnKSixcvXtcBKAgCJ0+eZP/+/Xz6059mYGCArKwsFhcXcbvd2zGFm0ahUHDy5EmampooLS1ldXUVp9PJ3/zN39DZ2blBdBOJBLFYLCkdnzdLfn4+X/va1667XRMEgXvvvZfS0lLefPNNhoeHd4Wf8MSJEzz66KPcc889WK1WEokEN9sQQ6VSoVareeSRR2hsbMTpdNLf309nZ+cWj/rD8Xg8+Hw+vv3tb7Nnzx6++MUv8sYbb3Du3DngsmDNzs5Kvqbf+Z3foby8nKqqKgoLCzlw4ACzs7NbKqxbKkhKpRKz2cydd97JnXfeKTkE29raeOuttzh9+jTT09N4vV58Pt81zUitVovJZGLv3r3s3buXYDDIxMQE586d+1CLaifIzc0lPz+fY8eOUV5eTjgcpqOjg3PnzjE9Pb0rLINbZWJigt/7vd+7pjO+uLiYyspKDhw4QFpaGr/1W7/FCy+8sCsEyWg0kpmZKTl6gassJJlMRiKRQCbbGM4n+pd0Oh1ms5m8vDyWlpZQKpVEo9GbFratIB6PMzU1hd/vx+fzMT09zdzcHADRaJS1tTUppEGv10tulO3ybW6pICkUCux2O83NzTz88MM4HA5WVlbo6enhwoULvPHGG4TD4eteINFvlJWVRXl5OSUlJbjdbqanp+nq6sLr9W7l8G8JmUxGdnY29fX11NXVkZOTw9raGmNjY7z77ru4XC7W19d3epibzvz8PH/7t397zfcOHz7MvffeS1VVFQUFBTz66KNMTk7ywx/+kEgksqMP5ochk8nQ6/VYrVYUCgWCIFw1VvG1eDxOLBZDJpNtcHTD5ZAPg8FAdnY2k5OTqFQq6fM7RSKRYGlpiaWlJQYHB6/5GZPJBFx20ut0OgBp3Ft9zbZUkEwmE/v376esrIy0tDTkcjkzMzP8j//xP1hYWPhQMVKr1RQVFfHggw/y67/+65SVlbG2tsZ3vvMdWlpacDqdSeEohMsXLj8/nyeeeIInnniC9PR04PJNa7FYKCgooKenB5/Pl7Qng1vB/Pw8bW1tPPnkk9JrtbW1/Jt/82948cUXmZ6e3sHRXRuDwUBFRQX5+fmYTCYEQSAej191n0ajUfx+P+Pj4wwPD1NRUYHNZiM7Oxu5XA5cfogNBgOf/vSnMRgM9PT0SMGGyYpCoaC2tpZ77rmHwsJCDAYDfr+fyclJWlpatjxGbssEyWQykZWVRV1dHRkZGcjlcpxOJxMTEzidTtbW1q4rRgqFArPZzIEDB2hoaKCsrIxwOMz8/Dz9/f04nc6kOa2RyWTSTVxYWEhWVpbkT5HL5WRkZFBdXc3o6Chms5m1tTXC4TDr6+v4/f6kEdWtQDxFvHJLo9VqSUtLQ6lU7uDINqJQKFAqlWRkZJCWlkZtba0UG3bl2K+8X0OhEP39/QwODtLb24tWqwUgKytrw+dkMhk2mw2TybRh+5eMaLVaioqKqKyspLKyEr1ej0wmIxKJ4PP5cLlcW/7cbYkgyeVyampqOHDgAP/23/5bVCoV4XCY559/ntbW1humR5jNZiorK/mf//N/YjabUSqVvPnmm3R2dnL27Nmk2qqJgY2f+cxnqKurQ6/XSzedRqPh6NGjHDx4kJKSEqamppiYmGBhYYG5uTlaW1vxeDw7O4EtJCMjg4aGBvR6/U4P5UMxGo1SwGBRURF1dXWkp6djt9uv8g+JLC4u8n/+z/+ht7eXgYEBZDIZ+/bto6qqSrKQdhvZ2dn8/u//Pk1NTdTV1SEIgpRG4nK5mJiY2PIxbJkgHThwgObmZtRqNR6Ph6WlJc6ePUtfX981TWARQRBobGyksbERjUZDMBhkfn6ed955RxKzZLEq5HI5dXV1NDQ0UFtbi8Ph2LACiua+IAiUlpZK1pLf78fj8ZCfn8/Y2Bhnz57ddcf9N4PVaqW4uFiyHgAmJyd54403kup0VPT1NDQ0UFRUhMPhQKfTXdOaicfjvPXWW/T29tLZ2cnS0lJSB+leC5VKhdlspqqqSrLs4fL12r9/PxkZGQiCQCgUYn5+np/85Ce0tbVty9g2XZDEYKrm5mZqa2uRyWS43W5GR0dpbW1lbGzsut8VBAG5XE59fT1NTU2oVCpcLheTk5O89957tLa2EgqFksYZKpPJqKqqoq6uTspRE4/yr3R4xmIxHA4HGRkZGAwGEokE0WgUjUZDe3s7LS0tBIPBXXVTfxiCIKBWq7Hb7RQUFKBWq6WcvampKemYORkQT8Psdjt79uyhoKBgw/vicX80GpVixd555x3a2toYGhoiHo9f04oS74Vk3KKJwY5Hjhzh3nvv5ciRI8Avnz8R0Rj46U9/um3+vk0XpObmZhoaGti/fz82m43JyUmeeeYZfvrTn0rHi9cjJyeH4uJiHnroIWpqaohGo7zzzjv83d/9Hb29vXi93qQRI7hsIV1p3sbjccLhMD//+c+Znp5meHiYUChEKBRibm4OtVrNwYMHOXz4MCdOnOD48eOUl5eTSCR47733eOedd3Z6SptCRUUF3/ve98jIyMBut6PT6Zibm+NrX/sara2tOz08Ca1WS0ZGBv/23/5bPvGJT5CdnX1V4vPk5CSLi4u0trbS0tJCe3s7CwsLrK2tbVhAxFM2mUy24e/i/ZoswiQIgpRJ8Fu/9VuYzeZrhmwkEgk8Hg9zc3MMDAxsW6zcpguS3W6nsLAQs9mMXC5nYWGBiYkJRkZGrnvsLQgCGo2G3Nxc9u7dS05ODkajkbm5OaamphgcHMTn8yXdtiaRSLCyssLc3JxUdiIQCNDS0sLMzAwjIyOEQiHW19dZXFxEpVKhVCoxGAxkZmaSl5dHdnY2dXV1SXni9FEoLCykurqahoaGDcGSoVCIjo4OZmZmdnB0v0Sv1+NwOGhoaKCqqorS0tINArO2tkYgEJDG3N7eTnt7O11dXR/6c5NpwbweV1ruomWu1+s3WEeioGq1WgoKClhcXGR5eXnLwzU2XZAcDgclJSWo1Wq8Xi+XLl26YUkNlUpFQUEBJ0+e5F//63+NzWYjEAhw6tQpWltbt7Uey60QDof5h3/4B2w2G0VFRTidTmZnZ1laWrrqwol/f+WVV+jp6eHUqVN8/etfZ8+ePdx77727IljwRigUCr7yla+wb9++q1bdUChEZ2dnUpRWkcvlVFVV0dzczO/93u+Rnp5+lV9zbGyM3t5evvrVrzIxMXHDSG3x++JWXSaTbRC4ZBGqRCKB2+1mcHCQ73//+xQUFJCTk0Nzc/NVybaZmZno9Xq++c1v8vLLL/OTn/yExcXFLY2n21RBEgPKbDYb8Xgct9vNhQsXbrhV02q1VFVVkZeXh9FoJJFIsLy8zKuvvkp/f/9mDnFTES2kUCjE2tqaFHEeiUSu6w+Kx+Osrq4yNTWFy+XC6/VKZv5uprCwkNLSUqqrq8nPz5fmk0gkeO2112htbU2aUA25XC7lGtrt9g1Js6JwjI2NcebMGVZXVz+SZS7+nGQRoiuJRqO4XC7efPNNzGYzZrOZvr4+SZDy8vLIy8sjLS0NlUpFcXExJ06cQKfT0dHRwdzcHF1dXVtyuLRpgiSaeAaDAYvFIgnSpUuXcLlcH/pdrVZLdXU1ubm56PV6/H4/LpeLU6dOSXtX8QZPNsevz+fD5/OxuLh4098JBALMz8/jdrvx+XxS4ua1IoKTHdERWlZWxp133klFRQVZWVnA5WsViUR45ZVXeO+995JCkMQSNhUVFZSWlmI2m69aDGKxGGNjY5w/f35X5xleD/HZPH36NHD52erv75cE6eDBgxw4cID6+nosFgu5ublSrF16ejp9fX0MDg5KBzibyaZv2T4YPn8jTCYTeXl5PPTQQ+Tm5pJIJOjo6KC9vZ1oNIpcLkelUrFv3z60Wi3vvvsu6+vrSedPuhU0Gg0Wi4X09HTS09NRKBTk5eXR1NTE0NBQUsVZ3YiCggK+8pWvUF1dTUVFBQ6HQ3rvjTfe4Pnnn+eNN95gfn4+KRaT8vJyacX/4IkawMLCAq+99pqUf7jbajh9GKIlLjrbxQUiHo/T398v+ZAGBwd5/vnn2bt3L4WFhZw8eZKsrCzy8/N56qmnmJmZYX19nYGBAfr6+lhbW9s0a2nTt2wqlQqdTifVktZqtdf04stkMuRyObm5uVIZTaPRSCQSYXZ2lsXFRXJzc1EoFBgMBurr65HL5dJKu5sFSSxPodFopO2CGAuzW4LqBEEgLy+Pqqoq9u3bR35+vmQZibWlW1pauHTpEgsLCztuaSiVSul0qa6ujqysLMxms2SVijleExMTtLa2SsnQHyaioovCbDaTkZGB1WqVFuQrrd1IJLLjSbUAFosFo9HI+vo66+vrG5LTr7w+Xq9XmoPb7SYrK4u1tTXUajVpaWkoFAqampqkSPahoSFWV1c3xQLeNEES0wSysrKoqKhAJpNhsVgoLS0lFAqxvLy84fNiFv9v/MZvUF9fT1ZWFqFQCLfbTUdHB/Pz83zlK1+RCkNlZGSwsLDAv/zLvxAKhZLC/P+oiPVlxAqR0WhUSiXZLUKrVCr5wz/8QxobG2lubt4gpENDQ9xxxx3SyrnTDyJAeno6x48f5wtf+AInTpyQkmbhsk8lGAzy7LPP0t7ezo9+9CNisdgNxUij0dDc3Mydd97Jk08+KfnOxIdZTLtwuVx4PB7W19d3zEqUyWQcPnyYffv20d/fz8zMDGfPnr3u5xOJBIODg4yMjPD+++/T0NDAgQMH+M3f/E0KCwt5+umn8Xg8LC4u8tWvflXKL/2489s0QYrH40SjUZaWlhgfHycvLw+TycShQ4cwGAzY7Xbgl+Vd8/PzKSwsZP/+/eTm5krWlV6v58SJE/h8PsmkjsfjnDlzhvHx8aSK1P4oKBQK8vPzOXLkCGazmVgshtfrZXl5mbm5uV1REaChoYH6+nrq6+vJy8tDLpezurrKysoKr7/+Ot3d3Ul3nbKysnjssccoKSm5Ko/O7/ezvLzMxYsX6evru6nFTq/X09zczNGjRzly5AgWi2WDIx9gdXUVt9vN66+/zvvvv8/y8vKObAHtdjuNjY2cOHGC+vp6pqambup7oo9obW2N8fFxotEoNpuN4uJiamtrpa4rx48fx2g08s///M8fe36bJkjinnR2dpbBwUHS09OxWCycOHECm80mmfPidqWpqYm9e/dis9mkG0RsM3PvvfdKvwyn08no6CivvfaalDG/W6yIDyJGsZeUlHDHHXdgsViIxWK43W4WFhZwOp07PcQbIpfLOXToEJ/+9Kepq6vDYDAA4Ha7GR4e5n/9r//FwMDADo/yarKzs3nyySelY/kr8fl8zM/Pc+HChRvma4nWj8lk4s477+TIkSMcO3YM+OWBi2h5ud1uxsfHef755xkfH2dpaWnzJ3YTOBwOTp48yR133EF5eTkvvfTSTX83kUgQDoeZnJxkcnISn89HcXExn/nMZ6ioqCAnJ4c777yTtLQ0nnvuueQRJLg8+L6+PrRaLTU1NTgcDqkWjlicTDyVMRgMGAyGDf6l8fFxKXva7Xbj9XqZmZlhdHSUxcXFXd2zTKVSYbFYOH78OHfddRcHDhxAp9OxvLzMj3/8Y1paWnZ6iDekoaGB//2//zc5OTmkp6ej0+lYW1tjeHiY8+fPc/78+RueqO5mSkpKKC0tpbi4mIKCAp588kksFstVn1tbW8Pv9/PjH/+Yzs5O2tvbd9SHZjabaW5ulsrifBxGRkYkI+Hxxx/ny1/+MhqNRkqNEkNZPiqbfsq2uLjI0NAQg4ODeL1edDqd1F0DfllFUlxFxDgen8/H4OAgPT09dHR04Ha78Xg8LCwsJE1070dFJpNhNpulAm7FxcVYrVb8fj9ut5vu7m5mZ2d3epjXRWzQUFFRwbFjx6Stiegb7OzspLu7m4GBAYLB4A6P9uMh3p8ajeaqsryVlZXs2bOHyspKcnJyJJ/RBy2uaDRKIBBgZGSEvr4+VlZWdvSEUalUYrFYUKvVCIKA3W7HbrejUqmIRqO3NLZAIMDa2hpra2tSTTO1Wo1er8dkMhEIBJJLkEZGRhgfH6e3t1cqA1paWkpBQQGCIJCVlcWnPvUp6WI///zz9PT08Itf/IKVlRXp4oknH8ngEP04iC2C7rzzTvbu3cvv/M7vSJ1Ch4aG6Ozs5LXXXkvq8rZqtZo//uM/pr6+fkM8WH9/P5cuXeIrX/mKFIqRDEf7H4fMzEz+1b/6V+zZs4fS0tIN71ksFumPUqm8bjCr2HtvcHCQgYGBpPqdKJVKPvWpT1FSUkJ3dzfLy8usrq7e0s9QKBQ4HA5J5MQ0sZKSEiKRyMfKrNh0QRJLMbjdbgKBAIFAAJ/Px+TkJFqtlsrKSmKxGOvr64TDYYaGhujv72dxcVEqXpYMqFQqjEYjubm5LCwsSKcktyKQcrmcgoICSkpKuOuuuygvL0er1eL3+1lZWeHMmTN0dXXt6OnLjRBzvWpqasjOzgZgamqKqakpfvGLX9Df33/D4/Fk4XrxcSaTiZycHH7jN36DRCJBU1MTWVlZpKWlSd9JJBJotVopVEMul0v1gsLhMMFgEI/Hw3vvvcfCwgKzs7PMzs4mxe/F6/XS1dUlWUppaWlUVFTwyU9+kvHxcaamppicnCQQCFwztUcsJW0ymaipqcFut5ORkUFTUxM6nU7yH4fD4Y/tUtmyipF+v186vZiYmEAQBBwOh5RWEQqF8Hg8dHd3093dzcrKStJYQ2JJipycHE6cOMH777/P6OioVILiZsYpnhqKuWqPPfYYGRkZwOX2T0NDQ7z44ov09fUl1WnUlchkMk6cOMGTTz5JfX29dPMNDg7y9ttv80//9E9Jm2d4La4nSFarFZvNxn/7b/9tw7W9Msj3g9a6+J7Y62xhYYHR0VG++c1vMjc3d0uR+1vNysoKZ8+eJSsri7KyMux2OwaDgbS0NNra2ujq6uK1115jbm7uqkqu4qm4w+GgqKiIL3zhC5SUlGCz2TCbzRgMhg3buI/blmxb2iCJk3rggQdobm6WIq5fe+01+vr6cLvdSSNGcDlO6NOf/rTUdfbhhx9mYWGBjo4OKfZiZWVFcroLgoDVapWC40ZGRkhLS+Mzn/kM5eXlUoqC2BPr3LlzvPnmm5J/IZnmLtLc3Myf//mfU1BQgMPhYGJigpmZGd544w3JX5RsXV9uxPXcAGIRvQ9aM1d2FPnge+JxeEtLC2fPnuXixYvMzc0xOjqaNFa+iGiNFxYWYrPZKC8vl0r2HjlyhLq6Ovbt28fKygqTk5NXfV+tVlNQUEBxcTE5OTnodDrpRBxgbm6OsbEx+vv7P/Y9sS2CJJfLUavVVFZWUlZWhiAILC4uSn3Kki32Ri6Xk52dTX5+PkVFRdjtdrxeLxqNBrfbzdzcHC6Xi5WVFSn/Jz09nczMTKnaoMPh4OjRozgcDmw2G263m+XlZbq6uiQn8MrKStLdvGL76KKiIu69914EQSAWi9Ha2kpfXx9nzpxhcnLyhgnTycj1BOnKqOoPWgcfrKMt1kOPRCJ4vV4GBgZoaWnh/fffT6oqmFeyvr4uiWV/fz8mkwmz2YzRaCQtLY20tDSMRiM+n4+8vLxrWkji83Dle6Ioz8/PMzMzg8fj+diHGtsiSGLtmQcffJDKykrW1tZYWVnZlqLhH4VoNMrAwABKpZLCwkIcDgfZ2dlSrt2VSYXiyimupHK5XHpPpVLh9/txOp18//vfp6enh1dffVW6oZPRMlKpVPzar/0a+/fvl7YrkUiEZ599ls7OTtra2nZt6MXHZWRkhJ6eHvr6+vB4PHg8Hvr7++nt7U3abTdcvn5ut5uf/OQnvPnmmzz44IPs2bOHJ554ArVaLbUrS0tLu2Z+35VbV9FPJJfLmZ+fZ2Jigp/+9Kf09/dvyuK6bVs2MUlWzI9ZXl7G7XYn5YUU2x+Lmfxi14jMzEy0Wi0GgwGbzYbBYCAUCqFWq8nKypJWT7fbjd/vZ25uDqfTyczMDOfPn2dmZga/35+UQiQil8vJy8sjMzMTgJ6eHvr7++nv72d2djYpr9fNMDs7y3PPPQdcXjwqKyuxWCySX++DjI6OMjk5KaXzJBIJxsfHGRsbY3x8HL/fTzAYZHFxMSkX1Q+SSCSkOL62tjYWFhbwer2SLygjI0NqbGmz2bDZbFIds0AgIAWULi0tEQ6HUSqVOJ1ORkZGaG9v37Tk6W0RpCsJh8PMzc0xPz+fVI6/K4lGo1Kg4osvvih18dy7d69UJ7q6uprCwkJcLpfU9zwQCEiOeqfTyfnz5+ns7KSnp2eHZ3TzyGQyCgsLycnJAeCtt97i2Wefpa2tLalDE27E8PAwf/EXfwFc9hH+xm/8hlROQ6x9feVC0dLSwksvvcTU1JTkqPV6vayurrK6urorhVlMqj1//jwAzz77LIWFheTn51NdXY3D4aCsrIy6ujrMZrNULXNmZkbKSezr6yMQCKDVapmammJoaGhTx7gtgiQmL7rdbsn87+vr245/elNYX18nGo3S3t6OSqVCq9Xy2muvodPpCIfDkskbiUSkHlahUEhyfO8WcnJyyMvLo7a2FrvdTk9PD93d3fT19SWdn+9WCQQCUoMJmUzGt771LQwGA1arVXrtyhV+bm6OhYWFDc0XotHorq808UFES2lyclIqr2wymdDr9SwuLhIKhSS/UDwel3IU5XL5lgTBbosgiXFHU1NTKJVKenp6dtVxsVh/eKdykbYLi8UilYGJRqP09vYyPT2dtM7aWyEajW5YHG7nfni3QjAYlIyFZED4MH+GIAib5uwQux3A5V/CVkf1JhKJm64St5nz3G42c54PPvggBw4c4PDhw/T39/PVr36VYDCYFCeBNzvP1LVMfj5sntvmQ0okErtq+/KryNzcHN3d3ayuruJ0Om85pSBFio/LtllI201qtbmaX4V5/irMEW7fee7uVhcpUqS4rUgJUooUKZKGD92ypUiRIsV2krKQUqRIkTSkBClFihRJQ0qQUqRIkTSkBClFihRJQ0qQUqRIkTSkBClFihRJQ0qQUqRIkTSkBClFihRJQ0qQUqRIkTR8aLb/7ZrA90FS80x+Usm1G7ld55mykFKkSJE0pAQpRYoUSUNKkFKkSJE0pAQpRYoUSUNKkFKkSJE0bHtfthS3H4IgYDKZMJlMVFRUIJfLpdbK1+v1Hg6HicfjW9roIcX1USgUGI1GsrKyyMjIQK/XS80kx8fHmZyc3Jlx7ci/muK2QexKXFRURH19PX/0R3+EwWDghz/8IS0tLVy8ePGq78TjcVZWVgiFQru+39tuRafTUVFRwRNPPMGDDz5ISUkJsViMkZERvv3tb/Pd7353R8a1rUX+lUolGRkZPPTQQ8DlXlkvvvjilvQ7S8V0XM1mz9NkMmGz2Whubmb//v3s27ePvXv3olQqGRkZYWlp6brdiTs7O5mcnOSll14iFArdsPlissUhWa1W7rjjDkpLSykrK8PlcjE7O8sPfvAD1tbWPlLrqO26lhaLhcrKSn73d3+XiooKCgsLCQaDLC8vc+7cOS5evEhraytwuWXZ+Pj4prZ/T4o2SHBZlXNycjh58iSJRIJgMMjp06dvmwaMgiAgk8lQqVTIZDIUCgWJRIJEIkE0GiUWiyVFj7OPg2gRyeVyHA4H+fn5HD16lEOHDtHc3Cx9rqam5kN/TnZ2Nl1dXZw6dWrXdYPVaDTY7XaOHz/OgQMHOHToEFNTU/T29vKzn/2MSCSStNdZJpORlpZGSUkJ999/P1qtFoVCwdLSEktLS0xPT6NSqaisrATA5/OxsrIiXSPxPt6qrfa2CZJCoeBTn/oUjY2NnDhxAo/Hw8LCAmq1eruGsOU4HA7S09N5+OGHKS4u5uDBg/j9fjweD2fOnGFkZITnn39+Vz18VyKXyzEYDJSVlVFWVsbjjz9OQUEBZWVlaDSaW/pZTU1N2O12vvOd7xCPx5P2Af4gMpmMJ598koaGBj7zmc9IvpfMzEy8Xi9ms5m1tTUCgcBOD/WaKJVKvvzlL1NbW4tKpWJpaYnV1VUsFgvV1dXU19cTj8elezQajbK8vMzo6Cj9/f28/fbbOJ1ORkZGtkSUtkWQBEFAoVBQWFhIYWEhBoOB0dFROjs7k/bC3Qp6vZ6CggKKi4spLCykqqqKjIwMNBoNoVAIuVxOVVUVRqORwcFB5ufnr7uVSVYsFgs2m436+nqKi4spKCigsrKS9PR0LBYL8XicaDRKIpEgHA6zsLBAIBDA5/Oh1+vR6/UUFxcjk10+2NVqteh0OmQymfTabkAQBHJycigoKMBisSCXy0kkEsjlchQKBTKZDEG46Z3XtqLX67HZbJSWlpKRkYHT6WR4eJjJyUnS0tLQarWYTCbp82q1WrKGTSYTRUVFBINBZmdnkclkLC8vs7y8vKnCtC2CpFAo0Ol0VFVVUV5ejiAI/OIXv+DrX//6ru9mK5PJyMvL43d/93fZv38/1dXVTE5O4nK5eOedd1haWsLj8fD5z3+eo0ePkpaWxiuvvMILL7yw00O/aeRyOZWVlRw8eJA//dM/Ra1Wo1BcvnXEhy8cDhMKhYhEIiwuLvLTn/6UwcFBBgYGKC4upry8nP/n//l/JItYNP/j8fim+ie2GkEQKCwspKioaIOQXrmdSdb5FBQUUFFRQUNDA4Ig8LOf/YyXX36Zd955B51Oh1KplK6PIAhkZGRgtVqpra2lqqqK2tpa7r33XkKhED/+8Y85d+6ctOXeLFHaFkHKzs6mqKiIvLw8dDodAwMDzM7Osra2tmuPfQVBQKlUcvfdd1NdXU1jYyOjo6O8/fbbDA8Ps7q6ytLSEsFgkPX1dXJycsjIyJBWnN2EXC7n+PHjNDY2SqumyOrqKq2trQwPDzM2Nsbq6io+n4+xsTFWVlbweDxkZmYSjUY3/MyBgQH6+vpYXl7eNVay6MQXH1SRRCLBm2++SWdnJ4uLi0k3H6VSicViobm5mcOHDxMMBpmZmeGFF15gdHSUaDQq3afBYFD63vr6OgsLC7hcLjo6Ojh16hTl5eXYbDZKSkqQyWRYrVbOnTsnWcQfl20RJIfDQWVlJQ6HA5VKxeTkJMvLy1cd+cpkMuRy+ZY6zTYLmUyGWq3m8OHDVFdXk5ubyzvvvCNZBoFAYMNquWfPHoqKiq5aWXcDcrmc6upqKioqJEe9aBG4XC4uXLjA+++/T0tLCy6Xi0gkAvxyqx6Px6+a8/j4OAMDA6yuru6Ko39BELBYLOTm5mK32zEajQiCIB1YXLp0iffff5+VlZWrxHenUalU2Gw2qqqq2L9/P8FgkMnJSc6fP08kEpG22R9EFJiZmRnptYqKCkpLS/n3//7fU15ejsViYWpqilAoRDAY/NjP7bYI0r59+/j85z+P3W7H7/fjdDqvuVXbs2cPzc3NnDlzhoWFBfx+/3YM7yNhMpnIyMjg/vvvJxKJ8Pu///v09fUxOjrK+vq6dLomk8kkU3i3OvDj8Tjj4+OS+T4zM8P09DQvvvgiIyMjnD9/nkAgQCgU2vAwiluEz3/+81RWVqJUKoHLFkVbWxvnz59Puof3WqhUKtLT03nqqad46qmnqKioQK/XAzA3N8fExATnzp2jq6srKQ8sdDodpaWllJeXU1paymuvvcbQ0NBH2mqNj4/jdDrp6ekhNzeX0tJS7rnnHu68807+8R//EY/Hg8/n+8hj3RZBMhqN0nYlGAwyNjaGx+OR3hdX0ry8PPbt28fCwgJKpZKhoaGktZTsdjtFRUXYbDYWFxfp6+tjbm5ug8kLl+NVHA4HxcXFZGZmSn6W3UQ8Hmd+fp6RkREuXLjAzMwMTqeTjo4OpqenWVxc3OA3USgUGAwG8vPzqa2tpaioiKysLARBYG1tDY/Hg9PpZH5+Pmn9LVeiUCiwWCxkZWVRXFyMTqdDLpcTj8dZWFigu7ub+fl5vF5v0s1HEAT0ej2lpaXY7XYUCgXj4+NMTU19pLGGw2HC4TB+v59oNIpCoaC8vByz2UxpaSnT09P4/f6P/HvYFkFSq9UYjUZisRjz8/P89Kc/3XDKJJfLsVgsHDp0iKeffpqMjAxaW1v5xje+kbTHwY2Njdxzzz3o9Xp8Ph/9/f3XXB2bm5t54IEH+NSnPkU8Hucf/uEfWF1d3YERf3RisRjvv/8+ly5d4utf/zpra2uSsH5wwZDL5RiNRhobG7n77rulKGDRopiYmOC9996jpaWFwcHBpLQoPohKpSIvL4+srCzsdjtwWaRDoRAXL17k29/+NuPj46ytre3wSK9GpVKRm5vLk08+SUFBAcFgkJdeeom+vr6PLZ5Op5PZ2VmKioqorKzki1/8ImfOnGFqako6sLhVtlSQNBoN2dnZOBwOyeLp6+vD5XJtuHgajYY9e/aQmZmJIAhSgGEyI5fL0Wg00gliSUmJFNMRj8cxGAzs2bOHAwcO0NTUxPDwMPPz81y4cIGpqamdHv4tkUgkcDqdkoWjVqvR6/UbbjiNRoNGo6G5uZns7GwaGhooKioiOzsblUol+Vr6+/v5yU9+wuzs7K4QI7hs4R88eJDc3FzptXA4zPj4ONPT08zPzyflwimm9JSVlVFYWEgkEmF6epqVlZVNE89EIkFHRwc+n48vfelLlJaWsm/fPvr6+q6bx/hhbKkgabVaiouLsdvtKJVKRkdHGRoaYmVlZcPNqNFoKC8vx+FwAJcnmaxbNREx3kQQBClYUBAE1tfXiUQimM1mmpqaqK+vp6qqip/97Gf09PTQ0dGR1L6xa5FIJFhcXEQul6PVajEajdhstg2fEV977LHHKC0tpba2dkM8TiKRIBKJMDY2xuuvv550W5sPQ6/X09DQQFZWlvRaOBxmZmaGubk5lpeXd3B010cMSSksLCQ7O5uRkRGcTic+n49QKLRp/05fXx9+vx+9Xk9+fj6NjY3Mzc3h8Xhu+TpvmSDJZDKys7P5whe+QF1dHYIg0N7eTmdn51Vio9frOXLkCIWFhSQSCSYnJxkbG0tqUXr99dfp7u4mOzubvLw8/vf//t/MzMwwMzPD+fPnMZlM/MZv/AZer5eWlhZ+8pOfMDQ0hNvtRqPRkJ6ezurq6qbGcGwVgiCQlZVFXl4eDzzwAPX19VJqgSg64gmp1WpFpVJdMzhQ3M5lZmayvLyclFbFB1GpVGg0GvR6veSU363MzMzQ0dGxqWIEEIlEcLlcfO9732P//v38h//wH/B6vchkMsbHx2/JEt4yQTIajTgcDsrKyrBYLEQiEWZnZ5mbm7vqs0qlkuzsbClKdGVlBbfbndSrqNvtJhQK0dbWRigUoqmpiby8PCwWC+vr6+j1evLy8pidncXn85GVlUU0GiU9PR2tVovBYMDr9RIIBJiZmZGimpMRQRBwOBwUFhayd+9e9uzZQ0lJifSeyA0StZHJZGRmZtLY2Eh3d/euiEEyGAyYzWYMBgMqlUp6XUypSPbxq1QqSUi9Xi+Li4ubfrKZSCQIhUIMDg5SXFyM2WwmMzOTzMxMJicnd16QxFSJvXv30tjYSCgUYmVlhdbWVnp7e6+6cTUaDRUVFVitVhKJBFNTU4yPjye15RCLxfD5fPzRH/0R5eXl/O7v/i6VlZUUFxfzmc98BqVSiUwmo6SkhOLiYo4ePSpFJYtbPfH06pvf/CYtLS289957Oz2tayKXy9m/fz/Nzc2cPHnyIwd2KpVK7rvvPini++LFi3R3dyetL0kmk1FcXExVVRVFRUUYDAbpPb/fz1tvvcXIyMgOjvDDkclk2O12LBYLgiCwsLDA4ODgllimoVCICxcu4HA46O/vJzc3l3379tHa2npLp8pbIkgymYzS0lIKCwuRyWSMjIzQ2dmJz+e77ioqCAJ+vx+32y2lW+wGwuEwc3NzvPDCC2RmZpKWlobBYJBSK0TEuCSAjIwMcnJy2LdvHzqdjoyMDMxm804M/6YQBAGz2YxOpyMQCKBQKFAoFITDYYLBIHNzc4yPjzMxMYHP59uwAguCQHFxMVlZWdx5552o1WpMJpN0IJDMyGQyqqqqKCsr2/D6xMQEg4OD9Pf3J3WlCtFtIvpmV1ZWmJ6e3rKwk0QigdfrZXJyUvId/+M//uNVoTAfxpYJUmFhIXl5eQCMjY1x9uxZgsGglIx4pamvUCgkQRLLIKyurib1lk0kHo+ztLTEz3/+c5RKpWQiXy/BUhAE6urqOHDggBS/kZaWhk6n2+aR3zyCIEi5TqIPTKVS4ff7WVlZoa2tjdOnT3PmzBnm5uY2RF4LgsDdd99NY2Mjx48fl4RIpVIlvSAJgkBZWRmlpaUAkoU7NjZGX18fw8PDSX1AIQgCmZmZGwTJ6XRuaRycz+djamqKhoYGVCqV5E+82Wd5y+6IK4/vq6urpViO5eVlZmdnsVqtUj5QZmYmRqOR9fV16ZRqN0TwfhAxufJ6TsMry3c8/PDDGAwG3G4377zzDsPDw9s82psnHA7zj//4jxgMBtLS0jCZTOj1eubn56VAR6/Xi8/nu+Z2QLQOr/yzGxAEQfL3mc1mKRjyzTffpLW1Fa/Xuyvv061kbGyMZ555hoMHD1JZWcmBAwcYHBy86a3tlgiSmBsjKrHVapVO0DwejyRINpuNRCKB2WxGqVRKKRZ6vV7aHuwmbvSwKZVKMjMzycnJITc3l0AgwOzsLJOTkx8pZmO7EOOQZDIZMzMzGI1GdDodS0tLrK+v35JPYreIkriYqtVqyZqLxWKsra1JqTNiuZVkJhgMSgukVqvFbDYTCAS2zD8bDAZZWloiEolIZUuu9L3diC0TpMXFRZaWlojH4zgcDtLS0qiqqpJuRtGCEv8uk8lwOBxotVr27t1LNBqlpaUlqR3bt4I4vy984QscOnSIvLw8nnnmGVpaWujo6NgVCabxeJxAIEAgELglM3w3olQq0ev1WCwW6fTX5XIxMzPDyMgIMzMzST//WCzG2NiYtGUrLS3l2LFjvPTSS1uWLaBSqTCZTJLb4lYPQLZEkGKxGN3d3YTDYfLz89Hr9Wi1WkpLSzdUFhQTT0V/SyQSYW1tTcptS9ZCVx8Fsezp/v37sVgsTE9Pc+HCBdrb26WM693EbhvvrWK328nJySEvL4+0tDSCwSDDw8NcvHhRsgyTHXFHIiayi0GtW5kFYbVaqaqqwmAwEI/HWV9fv6Vt7ZYIUjwep729nbm5OVQqFWlpadjtdgwGw4YIX0EQMBqNkkm8vr7O6uoqsVgs6VNHbhW9Xk96ejoHDx5kcXGRkZERzp49S2dn523/cIvW8G5CLJlTUFBAWloaXq+X/v5+Xn/9dWlLkuwkEglWVlakpF8xp3Qrny2bzUZtbS0Gg4FoNHrLyeRb5tSOxWIsLS3xyiuvoFQqUSgUPPvss9LJikKhQKPR8J/+03+ioqKC7OxsLl68yPPPP8+ZM2dYXFxM2viUW0Umk3HPPffQ1NSEQqHA7XYzODj4sbKidwPiNW5sbKS5uVmqdRWJRAgGg7d0HLxdiLWDTp48yZNPPonD4ZDSRPr7+2ltbd01vs14PM7s7CwLCwvEYjGKi4uJxWL8y7/8CysrK1viDnE4HDQ3N+NyuRgdHaWjowOXy3XT39/Sc1extrLIxMSE9HeNRoPRaMTv90vFvubn5+nq6mJ+fj6pj1NvBYVCgVqtprS0VKpJ7HK5mJycTAqzXzxJstlsyOVyBEEgFAoRCoU+diyYeCpXWFhIbm6ulOvn9/vx+XwEAoGkE2SZTIZGoyErK4vy8nI0Gg1ra2sEg0F8Pt+uiY+DX8YFeTwePB4PWq2WnJwcMjMzCQQCmxpDpVAosNvtUoT27OwsExMTrKys7Hwc0s0gds0sKSkhNzcXr9fL1NQU7e3tt40jGyAtLY3c3FzuvvtuCgsL6ezs5NSpU/zLv/xLUgTVabVa6urq+PKXv4zdbker1dLa2kp3dzc//OEPP9bPbmho4OTJkzzwwAPk5eUhk8mYm5ujq6uLrq4uRkdHk06QRP+lmL+2m4nH4zidTvr6+vjFL35Bc3MzNTU1fOlLX6KlpYXvfve7m/L7l8vl2O12fvu3f5u9e/dSXl7Oc889x7vvvnvLi+6ORqaJHSfEVVlsr3w7kZ6eTnV1NXa7HZlMxtDQkFQCYqf9EEqlkoaGBhobG6V9/5XBihkZGfj9/hs2cpTL5ajVajQaDbm5uWg0GmQymVR6RUxdCIfDjIyM8Oqrr15V1C3Z2G0+r+uRSCRwuVy8+eabUqeXiooKBEFgcHCQyclJnE7nRwphEANcm5ubKSkp4cCBAyiVSlpbWxkaGsLpdN6y22XHQ2XF4+OP2u0z2cnJyaG5uZm0tDTC4TCdnZ2Mj48nRZE2pVLJ0aNHaW5uZu/evdLrGRkZUumKubk53G63VJb3g4jNDsQ2SUeOHCEtLQ2VSsWBAwc4fPiwVFc7GAxKllcyX+vb7UBlYWGB559/nrS0NBwOB3v27CErKwu/38/rr7/OysqK5Dq5FdRqNWazmYcffpjm5mYOHjxIR0cHL774Ij09PUxPT9/yWHdckMR97rvvvsv4+PhOD2fTUKvVZGVl0djYyB133MH8/Dyjo6O89tprSbFVg8sHDwMDA9hsNqnwmlh4rr6+nm9961ssLS1JTvhrRaDrdDoKCwtxOBw4HA4sFouUWGw2m1EoFAwNDTE1NcXLL79Me3t7UlvCaWlpfOITn6C8vHynh7JpiAGdr7/+OhMTE/z2b/82mZmZPP7449TV1UnR1RMTE4yOjkrxQ1daTVcGiprNZg4ePEh9fT2HDx8mJyeHeDzO3/zN39DV1SXVxP8o7LggweX4o4WFhaQtv/FRUKlU5OTkkJOTQ1ZWFpcuXWJ0dJSZmZlNr0fzUYnFYlJJGJfLhdVqxWAwSO1t9u/fj8fjYXV1FZvNds1xa7VaysvLsdvtWK1WqWOMGKm/tLTE4OAgw8PDUrucZBUjuLyQFBYWbmiYuNsRq3VOT0+ztrZGf38/iUSCxsZGysvLyczMZGBgAK1WKzU2VSqVrK+vS9dKbIRpMpmwWq3U19dTW1tLZWUlq6urLCws8P777zM4OLjh8OpWSQpBuh2x2Wx84hOfoLa2Fq1Wy8svv0xLSwuhUChpHshoNEp7ezvBYJC0tDTuuusuamtr0el0kg/FbDZjMpnIzs6+5s8QV1Px88vLy3i9XkZHRxkfH2doaIg333yT+fl5PB5P0odyiFUxkz3x96MgBkn+yZ/8CeXl5Xz5y1+mrq5OauLp8Xjo7OxEp9Oh0+k2tHTSaDTodDqpbfrq6ipdXV38zd/8DW+88QbT09Obcn2T4rcuFiK/XVYlg8Eg7dVlMhkDAwOMjY0xOzubdI5cMV7s/PnzwGV/Q15enlTUS9yGqdVqSXTcbjerq6uo1WpisRirq6t4PB6pvIXX62VmZoalpSXm5+dZWFjYNYmo4hyvdGqLFvxuD0UR++mtrq4yNTUlbeEKCwvJysqSDpmUSqVUlUG8X6PRKF6vl87OTtbX1xkdHWV0dJSBgQFmZmbweDyb4hfcUUESJ6tWq6msrOTixYs7OZxNQRAEbDab1NJpYGCAc+fO0dfX95GcfNvB4uIir776KpOTkxQVFXH48GGpHIpo1ttsNslqmJ2dZWhoCIvFQigUkm7MgYEBRkdHd128zo0QW3clcwL0rbC+vs7U1BTf//73SUtLIy0tjRMnTpCVlUVZWRkKhQKlUrnhoMnv9+P1eunq6sLpdPLee+/dclrIzbBjguTz+ZidneXcuXMsLi5K2dS7Hblczl133UVTUxN6vV6a425YXaempnC5XIyMjEhJkWK2tuioBqSgxisz4AOBAH6/n0AgsCssoeuxuLjIT37yEwwGA5WVlXR1dTE4OMjrr7+e1NUhPyper5dQKMSrr74qbcuuTOUSn0nRLyiGgWyV62HHBCkSieD3+xkYGGB9fV3KF9rtCIJASUkJhYWFrK+vs7S0xMTERNI4sj8Mv9+P3+/f0DPvVw3R6Ts6Osrk5CS9vb309/czMjJyW1l9Ilc2fkwGhBsUZt9yh4dWq5WcouFweNPSKRKJxE1Htm3mPDUaDS+//DLZ2dmcP3+eF198kVdeeYVYLLYl/qOdmud2c7Pz3Iw5ymQytFotarWaSCRCLBaTTpy20geYupZJ4NROxgTLj0MsFuPNN9/EbDbT29vL2NjYrt7C/CpyZd2nFNvLjltIW0VqtbmaX4V5/irMEW7fed5eMfIpUqTY1aQEKUWKFEnDh27ZUqRIkWI7SVlIKVKkSBpSgpQiRYqkISVIKVKkSBpSgpQiRYqkISVIKVKkSBpSgpQiRYqkISVIKVKkSBpSgpQiRYqk4UOTa2/XfJkPkppn8pPKZdvI7TrPlIWUIkWKpCElSClSpEgaUoKUIkWKpCElSClSpEgaUoKUIkWKpCElSClSpEgaUoKUIkWKpGHLi/zLZDLKy8spLS2V+r9XVVUxMzPD9PQ0LpcLr9fL2NgYXq93V7VCkslkNDQ0kJOTw/79+zEajeh0OoLBIBMTE/x//9//t6HZ3m5Gp9ORnp7OiRMnyMvLIyMjA5VKJTVRHBwcZHBwkLm5uU3rHJNie1Gr1WRlZZGWlobZbObSpUvb/jxuqSDJ5XKpH/ihQ4fIy8sjJyeHO+64g56eHrq7u5mammJxcZF4PM7s7KzUJ2ormtBtJiqVCp1OR3V1NTU1NTzxxBOkpaVhsVjw+Xy0t7fzyiuvsLKygt/vJxaLSXOKx+Nb3lJns5DJZMjlcux2OyUlJdx3333U1NRIPd69Xi+XLl3CarVK7YK8Xq/UNijZr2OKX6JWqykoKKCgoIDs7GwGBgbw+Xzbep9uadeRgoICPve5z3Hs2DGamppQKBTI5XL0ej3r6+tSK16xb3hrayvnz5/nhRdewOl0XnUzy2QyqZ/8jayOrY56/bVf+zXuu+8+jh8/jsViwWg0IpfLkcvlxONxVlZWeP/991laWmJpaYnOzk48Hg9ra2vMzc0xPT3N2trax+7Wu9XzLC4upqioiP/6X/8rBQUFOBwO1Go1SqUSQRCIx+Osr69L3WvHxsYYGxvju9/9LjMzMzidzlv9J69JKlJ7I1sxz+LiYv7Tf/pP1NTUUFpayuc//3na29txuVyb+u/sSF82hUKB0Whkz5495OXlYbVaSSQSRKNR3G63pLoqlQq1Wo3FYpG2N2NjY+h0OoaHh4HLllZ6ejpGo5HS0lKWlpYYHR3F6/Vu+3bIZDJRWlpKQ0MDtbW1ZGVlodVqpfcTiQSCIKDX6ykvLyc9PR2Px4NWq8Xn8xEKhVhYWGBubo65uTlWV1cZHx8nEokklTWhVCoxmUzs2bOHvXv3SnNRKpX4/X7cbjcLCwsoFApKS0sxm81YrVaUSiU6nY6DBw/S0tLCwsLCljXJ3Eo0Gg1arRa73Y5KpQJ+2eU1GAwiCAI2m01qL65SqVCpVJhMJrxeLy6Xi/n5eYLB4K6Yu91uJzc3l4qKCtLT06WFVewpaLPZcDgcTE1NbWkvxS0TJJ1OR1ZWFkeOHMFqtQJIllBPT490kex2O0ajkYKCAmpra6mursZms9HR0cGf/dmfAaDX67n77rupr6/n6aef5p133uFb3/oW7e3tLC0tbdUUrklpaSl/8id/QnV1NYWFhQjCtcVerVZTXFws/f9dd90lzTkWixGJRDh//jw9PT381V/9FV6vl7W1tW2Zw81gMpnYu3cvn/vc53j44YfRaDTSXCcmJujr6+P555/HarXy3/7bf8NisaDX6yUfRF5eHt/97nfp6emRtqy7ifT0dPLz83nwwQel+9flcrG0tMTIyAgajYa77rpL6rqckZGB3W6nsbGR1tZWfvGLX/Dcc89Ji00yI5fLaWxsZP/+/Rw7doy5uTkGBgaYm5uT2ofX19fz4IMP8p3vfIexsbEtG8uWCJJMJiM9PZ2srCwsFou0zTp16hSjo6OcPn1aejhNJhN6vZ6ioiJqampoaGggEAiwvr6OWq1mz5493HnnnRw8eJC8vDzUajUVFRV84QtfIBKJ0NXVtcHi2irkcjm1tbXs27eP6upqjEYjfr8fp9NJLBZDp9Px3nvv0d7efs3vi5ZDYWEhxcXFlJaWUlJSgs1mA+Ds2bO8+uqrhEKhHbeUtFotpaWl/OZv/iZ1dXUolUrW19dxOp28/PLLDA0NMT4+Lj2Yf/Znf8aRI0c4dOgQubm5knVlt9vJyMiQrIrdgFqtxuFwcO+993L8+HEqKyslCzgUChEMBlldXUUul5Obm4sgCAiCgE6nQ6PRoFarKSkp4ZFHHqGtrQ23283y8nJSW0kymYzm5maampqQyWRMTExw+vRpvF4vcrkco9FIZWUld911F//8z/+8pWPZEkESzVmr1Yper0cmkxEOh2lra6O1tZWXX35Zeui0Wi06nY6CggIefPBBHA4Hq6urhMNhzGYztbW1PProo1RVVWEymQDIy8vDZrPx7rvvMjs7i8fj2fIVWC6XU1VVRU1NDYWFhayurrKyssLg4CCxWAy73c7rr7/Oc889d83v63Q6LBYL+/fv5/Dhw9hsNnJzc8nJySE3N5dYLMaZM2eIRqM7eionCAJWq5Xi4mJOnjyJWq0GYHV1ldHRUZ599lkmJyeZnZ0lkUggk8kYGxsjFouRk5Mj+Zj0ej0mkwmLxcLc3NyOzedWUavV5Ofnc+DAAR5++GFMJhNyuRxgg4WbSCQ2iMyVf8/KyiI7O5vMzEz0ev22LJgfFblcjkajkQ4qYrEY09PTtLa2EggEkMlk0g6moaEBg8GwpePZEkFKJBLSEX4wGEStViMIAhaLBbPZvOGzoVCIcDhMIBBgenqaZ555hubmZvLz8/nWt75FYWEhJSUlG/w0KpUKhULBgw8+SEZGBt/4xje2vA+7RqPh6aefprKyEkEQeO6553j55ZcZHBzEYDBw4sQJBgYGrrvtCgaDeDweFhYW6Orq4tSpU/z7f//vqaysxG63U1tbyyOPPMJrr722aY7gj4LBYOBb3/oWe/bsQa/XS+EZX//61xkdHWV8fJxwOCw9YPF4nGAwyNTUFG1tbVRWVmI0GolEIgSDQQKBwK7arhUXF/Pnf/7nFBUVYTabkclkku/T5/Ph8/kYHBzE7XYzMzMjLawulwulUsnx48cpKiqirKxsh2dycxw9epSjR49y6NAhdDodb7/9Nm+88QZvvvkmgUAAjUZDeno6er1+W8azZYIUCATw+/2EQiHkcjkKhYKcnBxcLhd5eXmsrKzg9XpJJBLSkbher0ehUJCVlUVRURFVVVXYbDZ0Oh2xWIxgMIjL5ZIeBo/Hs21OQ3EbKm6xFhcXGRwcZGpqCr1ez8DAAC6X60O3W+Lp4MzMDNFolL6+PhQKBWlpaRgMBrKyslAqlVs+lw9DtJD0ej0+n4/h4WG6u7vp6+tjbm7umoIr/v5lMhmCIBAOh5mammJhYWFX+Y/0ej1Wq1USo1gsxurqKmtra6ysrDA/P8/CwgKjo6N4PB7m5uYkS8ntdmMymdi/fz/hcJj19XXC4TDRaDRprSOAtLQ06VAiEokwPj7O/Pw8Pp8PuHxN9Xq9ZClvNVsmSNPT02RnZ+N2u5HL5ZjNZu6//34qKipYW1vj7NmzXLx4Ebj8ECgUCvbv389jjz3GQw89RFZW1gaHsdfrZWlpiZ/85CdEIhESiQTnz59ncnJy2wLxxGN9gJWVFWZmZgiFQqytrfHKK6/c9I3ndrtZWVnhe9/7Hg0NDTQ1NUmCpNFotnIKNyQWizE0NEQwGMRoNPK9732Pl156CZ/P96Fia7PZKCkpQaPRsLKywjPPPMO7777L5OTkNo7+oyOTySgqKqK0tBSHw0EkEmFlZYWOjg6mp6c5f/487e3tdHV1AVzzWhcVFWGz2ZDL5bhcLvx+f9L7ztLT06mqqkKj0bC8vMzp06eZmpqS3lcoFDgcjt1tIYl4PB7OnTtHTU0N5eXlqFQqHA4HJ06cQKFQoFQq6ezsRC6Xc88993DkyBEOHDiA2WyWxGh5eZnZ2VnOnDnD5OQk7733nnQUOTs7i9fr3REnsOikFm+4W10FRStSXImShXA4zM9+9jPMZjNqtZqOjo4PtUKVSiU2m428vDwKCwuJx+O4XC4uXry4a8QILl+PpaUl2tvb+Yu/+Aui0SiRSITp6WlWV1dxOp0sLCxc916zWCykp6eTnp4u/e7EeK1kRDxIqqioICcnh56eHvr7++nq6mJ+fl76nFarpaamhszMzG0Z15YKktfr5eLFi2i1WhwOB5mZmZJjV7ywMzMzKBQK7r//fvbu3UtdXR2AtG8XfS4vvvgiAwMD1wyY3C6udGRqNBoMBgMrKysfaUsik8mIxWIbHKQ7fboGEIlEePPNN6XtVyQSkRaADyIIgpRucKVz3u1209nZycrKyjaP/qOTSCRYXl7G6/UyPj4uuRH8fv8Nr4t4iCMe/Yv+TjEQOBnR6/XU1tZSVFSEw+Hg1KlTXLp0if7+/g3zVavVlJeX43A4NnxfobgsHaLLZbPYUkFyuVw899xzDA0N8d577/HFL36R3NxcrFYrx48fp7GxkU9+8pMIgkBBQYF0ISORCPPz8/zd3/0dAwMDdHV1sbS0RDAY3FExCgQCBINBDAYDhw4dAuBv//ZvpdSXm8Vut5OVlcVv//ZvU19fj0qlwuPxMDo6mhSxSFfGzVzPMhIEgYqKCqqqqvid3/kdioqKMBqNtLa20tvby/Ly8q7LaYtGoxuyAG5mkZDL5ahUKh577DGam5sxmUxMTU3R3d3N8PAwS0tLSelDSk9P51Of+hRKpZJTp07x93//94yMjNzUfWy32/n1X/91vF4vs7OztLa24na7N2VcWypIkUgEl8vF+Pg4AB0dHfj9furq6tBqtVKsCrDBd7KwsMD4+DgdHR2Mjo4yMTGx47lfom9FpVJRXV1NVlYWtbW11NTUMDU1JR2Di2NUKpWYzWaUSiUKhYJQKARcXnFycnIoLCykpqaGgoICQqGQ5CRNhof4Rr9ntVotmfJ1dXVUVVVhNBoli0Cj0ZCTkyMdXCS7Y/dKbnXFNxqNpKenU1FRQUlJCYlEgoWFBdra2lheXk7KoEilUonBYKC4uJipqSnGx8eZmZm5ZoqIQqGQDpYAysvL0Wg0NDc3MzU1xdra2qZuS7c82x8uR/ZOTU0xMjJCdXU1f/zHf0xubi7Z2dlXpV3E43Fee+01WlpaeOuttzYcMe8kwWCQr33ta9xxxx184xvfoKqqioqKCgoKCujp6eHb3/42kUhEupltNhv33HMPDocDq9XKxMQEiURCCmMQL2w0GmVsbIze3l4uXrzI6urqDs/0xmRmZlJcXMx//s//mbKyMkwmk+Tzq6ysxGq14vF4eO+99zh37hwul+u2qHhwLaqrq7n33nu57777JBE+e/Ysf/3Xf50U1u4HEbeXubm5VFdX09PTw7vvvovf77/m58WQFvE5/dM//VNisRhZWVm8+uqrkpN/s9gWQYLLYhMOh1lbW2NxcRGbzXZV2sX8/DxTU1OcO3eOrq4u6TQtGYjH48zPz9PR0cH3vvc9mpubKSsrk47qn3766Q0Z/TqdjqKiIgwGA5r/v71zf23q/v/4IyfH2rRpk5j0frW2db3baqezWsVtOuYYMvbDhjgYgzHYYGM/DMYYbP+BQ5jsIo7BmIhO1iEKIortpnP23rRZk15j2zRtmjY5bXpL8/1BzvtTNy+d317SLQ8ogjTpOTnnvPJ+v17P1/MVHU16ejpwL/lpMpmIjo5Gq9UyPT3N4OAgsixTUlJCKBRidHR0xXVV/xRZljGZTCLPV1RURHp6OlqtlsHBQbES3LhxI2azmaqqKtLS0igtLeXixYs4nU6xivw3oNfrqaioYO/evezbt4+YmBjcbjcXLlzg9u3ba5peeBRarZa9e/eyc+dO0aweFxcnhJwlJSUkJSWJnFFycrLo5QMwGAxMTk5y584drFYrTqdTrP6Xg1UJSKq8XpIkFhYWGBsbe2DlxuVy0dDQwK1bt7Db7WF186qBwmq18v333yNJkkhiJiYmUlRU9NC+tsVbOfVfjUZDKBRibm4Ot9uNTqejoqICj8cjbDzCpSlVo9Gg0+lITU3l0KFDVFVVUV5ejkajQVEUnE6n2MZlZGSIwkVZWRmKojA6OookSbhcrnWjSXoUkiRhMBjYv38/e/bsYffu3SiKQn9/P2fPnqW/v/+hhYC1RpZlqqqq2LZtG4BoIM7MzCQ+Pp5XX32V4uJiCgoK/vbaUChEVFQUPp+Pmzdv0tzcLKQvy3Z8y/ZOD0Gr1ZKRkUFmZiZvvPEGW7ZsoaCgQIgd1WoO3Iu+mZmZREVFhcWD+CB8Ph/Nzc0cP36c8+fPc+DAAZKTk9m6dSvp6emiPBoMBvH7/czMzBAIBLBarYyPjzM9PY0sy8iyTGlpKQaDgerqavH+hw8fxuVyceHCBdrb24VWa62QZZnY2Fg+/fRTCgsLKSsrQ5ZlxsbG+Oabb4RwUq/XYzQaOXr0KHl5eZSXlxMVFYXBYODYsWOUlZXR3t6Ooihh+7AuBUmSeOmllygqKuLo0aOYzWa0Wi11dXU0NTWJc1wv7N69m+LiYgKBAFqt9r580WJGRkYYHBzk1KlT2Gw27Ha7UK4vZ55sxQKSJEkicbZ582ays7MpLy8XS0C3283Y2BhpaWnExsYSFxeHTqfDYrGI7Uw4fpuqgaa3t5fR0VHi4+PJzs7GYDAIFffc3Bw+nw+bzYaiKCiKQltbGxMTEwQCAaHBmpubw2KxkJqaKrZyubm5JCUl4XQ6mZ2dxWq1EggE1uyzkGUZnU5HXl4eubm5bNq0SbST1NfXY7fbcTgc4hoWFhYyMzNDRkYGer0enU5HZmYmgUCA5ORkhoeH15UcQEWSJKKjo0W5vLS0lOTkZILBIMPDw7S3t9Pe3i6S+OsFo9GI0WgE/ndvT05O4vP50Ol04vqPj4/T3d1NY2MjHR0dK9aft2IBKTY2lu3bt/P1119jMpnQ6/XIsizEkufOnaOmpoZPPvmE8vJyqqurSUxMxGg0kpKSQnd3N16vN2xXSmqgOXfuHFu2bCEpKYmUlBRCoRAul4uWlhY+/vhjRkZG8Hq9D8wnqDd5Tk4OO3fu5LnnnqOyspKEhATeeustkpOT6evro729fdnKqv+U2NhY4dgwPz/PwMAAP/zwAz/99BMOh0NsvWdnZxkfH+fEiRNkZWURDAbZtm0b5eXlWCwW5ufnOXLkCLdu3eL69etrci5PinqdCgoKKCoq4tixY2RlZTE1NUVzczO3b9/m1KlTQr+0XpmYmODGjRuMj4/j9/spLCwkKSmJoqIibDYbZ8+epbOzE4/Hs2LHsCIBSavVsnXrVgoLC4VQTJZl7t69S09PDz///DPNzc1MTk7S09PDpk2bCIVC9/W8ZWRkMD4+HrYBSSUYDIqb1WKxEAwGuXr1Kk1NTbhcLhRFeWSFKRgMMjg4yJ07d/D5fHi9XnJzc6mqqiIxMZGysjLu3r27ZgEpEAgwNjbGmTNnMBqNaLVaGhoaGBoa+lsFNBQKMTMzw8jICJcuXWJhYYH09HSMRiN6vZ79+/czMzMjbupwkDgsBXWFWFVVRXV1tQiwXV1dNDY2cv36dZH7C3eCwSAtLS0oioLX6xXOrSMjI8LRYXp6mvn5eXJzc0V1bWRkhLa2thUvtqxYQCorK6OkpESUhIPBIF1dXdTX13P27FmxDent7RUBCe59G6WlpZGZmYnVag3LSsVf0ev1wpphfn6eS5cu0djYuCQfnGAwiNvtxu12U19fj6IolJeXs3PnTiwWCxUVFdTV1a3Smfydqakppqam+O6775b0+2rRoqamBpPJRGVlJTExMcJkz+PxcPPmTeF5Fe6o7p8lJSUcOHCAl19+WTTT2mw2bt++zdWrV9fNNi0YDNLQ0EB3dzfNzc1MTEwwMTFBW1vbfTIFtXKs1+sJhUK43W7a2tpW/PiWPSDJsoxer+e1116jsLAQjUbD9PQ0fr+fb7/9ltbWViYnJ1lYWPhbVWpqaorx8XFsNhudnZ3rIhipCWqNRiOWun19fbhcrida3d28eZOhoSFefPFFTCYTBw8eXHFTrOVGrR7+9ttvBAIB3n33XfLz8zEajRQUFPD666/z5Zdfhr3mSpIkIXP48MMPSU5OZmFhgStXrmC1Wjl9+jRut3tdCT8XFhZwOBwij6mq0xdXytRtekJCAhs2bKC2tnZFXSIXs+wBSZIkZFkmPT2dlJQU4F6+ZXh4GLvdLvbZGo1GGP6rncRTU1OMjIwwOjoa1vmjxcTExBATE4Msy0xNTYmk4JOWQr1eL3q9nunpaTZs2EBCQsJ9OpD1gqpYbmlpwel0YjabMRqNmM1mCgoKxFY+XLvh1SR9UVERJSUlPPXUU8zOzuLxeGhra6O5uZnOzs6wVGI/jsdtu+Li4khJSSE2NpZQKERPT8+qpQxWRYdktVqpra1lYGBAfBjR0dHExcWxY8cOtm3bJqwz6+rqcDgcYdsDtBitVkt+fj75+fkYDAYURWFqakpMU3kSZFkmOjqazMxMTCbTuhgJ9TA8Hg/j4+NcuXIFr9dLVlYWaWlpJCYmUllZyczMDE1NTWGZe1H9pY8ePUpKSgparZbW1lZ+//13vvrqK/r7+9dlMFoKJSUlHDx4ELPZjKIoXL9+HYfDsSp/e1UC0sTEBIODg8zNzSFJEjqdjs2bN5OTk0NBQQFZWVloNBpGR0dF4izcg5GKJElCS7VYAPqkqMZ0er1ejEwKN9X2UlH7woaHh8UWVpIkNmzYQHZ2NsPDw7S2toZVQNLpdKSlpbFt2zaeeeYZLBYLGo2G3t5eGhsbuXbtGmNjY//aVhi49xmYTCZkWRZfrqt1jVZ0lLYaVFQ/mbm5OWRZxmg0UlxczKFDh9ixYwe5ubloNBpcLheNjY1h5xG0VNRt6GKx5z99fVFREZWVlej1enw+H01NTWGfa3kcw8PDDA4OipWeRqMhNzeXwsLCsLPnUAsUe/bs4dlnn8VoNAo92I0bN6ipqVmziudqER0dLSqqq82KrpAWP5SSJImu8P3791NaWkppaSkmk4mJiQmuXbtGXV0dPT09yypFX0kWFhZwOp04nU4URcFgMJCfn09paSmhUIjOzs4lrfQW2/YeOXKEsrIyNmzYwNDQEHV1dev6AQiFQtjtdrRaLT09Peh0OkKhEDU1Nfzxxx9hs9KQJImKigpKSkp45513yMjIQKPR0NLSQltbG8ePH19TL67/CquyZYuJicFsNmMwGLBYLGzfvp0tW7aQmpqKoigMDQ3R0NBAb2/vuvJgDoVC+P1+MVMtLi4Oo9FIXl4efr8fp9P5QIMz1QpXr9eLkdybN2+msLCQrVu3kpqayujoKAMDA/T09IRV1/jjVn4PCsCqza/qhhAIBMSE23B4wKOiotDpdBQUFFBaWiqanBVFwW6309bWFra5rpVAq9WyceNGYSKoKMqq5ctWJSDt2rWL7OxsMfIoLS2Nubk5/H4/Fy5coKWlhdOnT69pi8SToigKHo+H3t5esrOzSU5O5v3338dqteL3+7l79+59lqDwvymgR44cIScnh7y8PBITE0lKSgLuGdt98cUXNDU18euvv4bNZ6LRaIiKinpkUFIdDxYfc2ZmpvAL6unpobW1VRjJh0OuMCcnh/z8fD766COysrKIjo6mo6ODjo4OTp48SVdXV9hcg9VA7fqPiYmht7eXq1evrtqX4rIHJLWzfXZ2lrm5OTE0ULU7lWVZDAHo6uqirq4Ou92+7sblLMbj8XDx4kWRc9Dr9WRlZfHKK68wNjb2txyQOrOsvLychIQEEhMTgXuBqL6+np6eHhoaGsKuazwpKYkXXniB6Ojoh5py+Xw+ZmZmmJiYEGPFi4qKyMrKIhAICPlHOBQuoqKiMJlMlJSUsGPHDiwWC7Ozs9TV1dHW1obNZsPpdIrprf8lJEliYGBA9FSu1kp2xaaOTE9PMzMzIwJSfHy8qLooikJzczOXL1/m4sWLD3SqW08MDAxw8uRJYmNjKS4uxmg0kp2dzQcffPDI16nBOxgM0tfXR0dHBydOnKC1tTUsdVjZ2dl89tlnmEwm4uLiHvg7qil+X1+fEL8WFxeLgYkOhyNsChfqVrm6uprnn38ek8lEX18fP/74I+3t7TgcDtxu97+2vP8oQqEQNpuNjo6OVb0Plz0gzc/PoygKJ0+epKKigjfffBOPx4PH4xFzrWpra+nq6qK7uxufz7fch7DqqF3SZ86coampiaNHj5KTkyOU6iqBQIA///wTp9NJb28vPp8Pv99Pe3u7mEDS3d0dNluZv9Lb28vnn39Ofn4+2dnZJCQkoNfrSUpKwmQyER8fT0JCgpjAAfdu7I6ODvr7+/nll1/o6+ujr69vza+7Orj06aefZuvWraSkpODxeHA4HNTW1jI6Osr4+HhYrVBXC5vNRk1NDZIkrXo/6Yps2WZnZ7lz5w7z8/Ps27cPl8sl3CAHBga4du0aXq933ZezVdRWic7OTgYHByksLGR6elqMEVfx+/1YrVbsdjs2mw2v14vX66WhoSEskruPQzXmGhsbw+12k5aWhsFgYGJiAovFIvzRF7OwsEBXVxft7e3U1dXh9/vDIkkfGxuL2WwmLy8Ps9mMLMv09fXhcDiEgDdcKoCrjaqw37Rp05L6MZcTzaP+mEajeeIjUTv3Y2JiWFhYED/qVIe/zkZfbkKh0JKFQP+f83zAewl3A3VUzKJjEiIzNfmr/t+TstrnqVYIVb3V4p+HJbvVSuPMzMwTB96lnudSzjEqKorDhw+za9cu3nvvPUZGRnC5XLz99tt0d3evWX5rre7Zv6Leu6rD63LLcB51nitWZVMfuvXQ0b2chEKhsFgBrBTqdV3PaDQaLBYLBoOB+fl5bDYb9fX1wi7mv878/PyabVVXzeQ/QoRwQZIkLBYLsbGxjIyMcPnyZc6fP7+uBaj/FlZsy7bWhMvyd6WJnOf9LOUctVqtGN9kNpux2+0MDAwwPT29psWEyLWMBCQgcp7rgeUMSOFK5Fo+JiBFiBAhwmqyot3+ESJEiPBPiASkCBEihA2RgBQhQoSwIRKQIkSIEDZEAlKECBHChkhAihAhQtjwfwkJ025UBbRZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x360 with 25 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "src_data = './result/src_data.png'\n",
    "\n",
    "# 可视化部分训练数据\n",
    "def visualize(dataset_mnist):\n",
    "    data_iter = next(dataset_mnist.create_dict_iterator(output_numpy=True))\n",
    "\n",
    "    figure = plt.figure(figsize=(5, 5))\n",
    "    cols, rows = 5, 5\n",
    "\n",
    "    for idx in range(1, cols * rows + 1):\n",
    "        image = data_iter['image'][idx]\n",
    "        figure.add_subplot(rows, cols, idx)\n",
    "        plt.axis(\"off\")\n",
    "        plt.imshow(image.squeeze(), cmap=\"gray\")\n",
    "    plt.savefig(src_data)\n",
    "    plt.show()\n",
    "    \n",
    "visualize(dataset_mnist)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.4 隐码(latent code)构造\n",
    "\n",
    "为了跟踪生成器的学习进度，我们在训练的过程中的每轮epoch结束后，将一组固定的遵循高斯分布的隐码 `test_noise` 输入到生成器中，通过固定隐码所生成的图像来评估生成器的好坏。\n",
    "\n",
    " - 创建隐码代码如下"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindspore import Tensor\n",
    "from mindspore.common import dtype as mstype\n",
    "\n",
    "# 利用随机种子创建一批隐码\n",
    "np.random.seed(2323)\n",
    "test_noise = Tensor(np.random.normal(size=(25, latent_size)), dtype=mstype.float32)\n",
    "random.shuffle(test_noise)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 3 案例演示——模型准备\n",
    "\n",
    "当处理完数据后，就可以来进行网络的搭建了。\n",
    "\n",
    "本案例实现中所搭建的GAN模型结构与2-14年论文中提出的GAN结构大致相同，但由于所用数据集 MNIST 为单通道小尺寸图片，可识别参数少，便于训练，因此我们只需在判别器和生成器中采用全连接网络架构和Relu激活函数即可达到令人满意的效果。省略了原论文中用于减少参数的 `dropout` 和可学习激活函数 `maxout`。\n",
    "\n",
    "本案例中采用模型图像如下所示：\n",
    "\n",
    "![structure](./structure.png)\n",
    "\n",
    "MindSpore框架构建网络的流程与PyTorch类似，在定义模型类时需要继承Cell类，并重写init和construct方法。\n",
    "\n",
    "## 3.1 生成器\n",
    "\n",
    "生成器 `Generator` 的功能是将隐码 `test_noise` 映射到数据空间。由于数据是图像，这一过程也会创建与真实图像大小相同的灰度图像(或RGB彩色图像)。在本案例演示中，该功能通过五层 `Dense` 全连接层来完成的，每层都与 `BatchNorm1d` 批归一化层和 `ReLu` 激活层配对，输出数据会经过 `Tanh` 函数，使其返回 `[-1,1]` 的数据范围内。\n",
    "\n",
    "此外，我们通过输入部分中设置的 `latent_size`、`img_size` 和 `channels` 来影响代码中的生成器结构:\n",
    "\n",
    " - `latent_size` 是隐码 `latent_code` 的长度\n",
    " \n",
    " - `img_size` 与通过生成器传播的特征图的大小有关\n",
    " \n",
    " - `channels` 表示生成特征图的通道数\n",
    "\n",
    "以下是，\n",
    "\n",
    " - 生成器的代码实现："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindspore import nn\n",
    "import mindspore.ops.operations as P\n",
    "import mindspore.ops.functional as F\n",
    "import mindspore.ops.composite as C\n",
    "\n",
    "# 重写reshape方法\n",
    "class Reshape(nn.Cell):\n",
    "    def __init__(self, shape, auto_prefix=True):\n",
    "        super().__init__(auto_prefix=auto_prefix)\n",
    "        self.shape = shape\n",
    "        self.reshape = P.Reshape()\n",
    "\n",
    "    def construct(self, x):\n",
    "        return self.reshape(x, self.shape)\n",
    "\n",
    "class Generator(nn.Cell):\n",
    "    def __init__(self, latent_size, auto_prefix=True):\n",
    "        super(Generator, self).__init__(auto_prefix=auto_prefix)\n",
    "        self.model = nn.SequentialCell()\n",
    "        # [N, 100] -> [N, 128]\n",
    "        self.model.append(nn.Dense(latent_size, 128))  #输入一个100维的0～1之间的高斯分布，然后通过第一层线性变换将其映射到256维\n",
    "        self.model.append(nn.ReLU())\n",
    "        # [N, 128] -> [N, 256]\n",
    "        self.model.append(nn.Dense(128, 256))\n",
    "        self.model.append(nn.BatchNorm1d(256))\n",
    "        self.model.append(nn.ReLU())\n",
    "        # [N, 256] -> [N, 512]\n",
    "        self.model.append(nn.Dense(256, 512))\n",
    "        self.model.append(nn.BatchNorm1d(512))\n",
    "        self.model.append(nn.ReLU())\n",
    "        # [N, 512] -> [N, 1024]\n",
    "        self.model.append(nn.Dense(512, 1024))\n",
    "        self.model.append(nn.BatchNorm1d(1024))\n",
    "        self.model.append(nn.ReLU())\n",
    "        # [N, 1024] -> [N, 784]\n",
    "        self.model.append(nn.Dense(1024, img_size * img_size))  # 经过线性变换将其变成784维\n",
    "        self.model.append(nn.Tanh())  # 经过Tanh激活函数是希望生成的假的图片数据分布能够在-1～1之间\n",
    "\n",
    "    def construct(self, x):\n",
    "        img = self.model(x)\n",
    "        return Reshape((-1, 1, 28, 28))(img)\n",
    "    \n",
    "# 实例化生成器\n",
    "netGenerator = Generator(latent_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 判别器\n",
    "\n",
    "如前所述，判别器 `Discriminator` 是一个二分类网络模型，输出判定该图像为真实图的概率。主要通过一系列的 `Dense` 和 `LeakyReLU` 层对其进行处理，最后通过 `Sigmoid` 激活函数，使其返回 `[0, 1]` 的数据范围内，得到最终概率。\n",
    "\n",
    "\n",
    " - 判别器的代码实现如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Discriminator(nn.Cell):\n",
    "    def __init__(self, auto_prefix=True):\n",
    "        super().__init__(auto_prefix=auto_prefix)\n",
    "        self.model = nn.SequentialCell()\n",
    "        # [N, 784] -> [N, 512]\n",
    "        self.model.append(nn.Dense(img_size * img_size, 512))  #输入特征数为784，输出为512\n",
    "        self.model.append(nn.LeakyReLU())  #默认斜率为0.2的非线性映射激活函数\n",
    "        # [N, 512] -> [N, 256]\n",
    "        self.model.append(nn.Dense(512, 256))  #进行一个线性映射\n",
    "        self.model.append(nn.LeakyReLU())\n",
    "        # [N, 256] -> [N, 1]\n",
    "        self.model.append(nn.Dense(256, 1))\n",
    "        self.model.append(nn.Sigmoid())  #二分类激活函数，将实数映射到[0,1]\n",
    "    \n",
    "    def construct(self, x):\n",
    "        x_flat = Reshape((-1, img_size * img_size))(x)\n",
    "        return self.model(x_flat)\n",
    "    \n",
    "# 实例化判别器\n",
    "netDiscriminator = Discriminator()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 损失函数\n",
    "\n",
    "MindSpore将损失函数、优化器等操作都封装到了Cell中，因为GAN结构上的特殊性，其损失是判别器和生成器的多输出形式，这就导致它和一般的分类网络不同。所以我们需要自定义`WithLossCell`类，将网络和Loss连接起来。\n",
    "\n",
    "\n",
    "当定义了 `Generator` 和 `Discriminator` 后，接下来将使用MindSpore中定义的二进制交叉熵损失函数[BCELoss](https://www.mindspore.cn/docs/zh-CN/r1.8/api_python/nn/mindspore.nn.BCELoss.html)\n",
    "，为 `Gen` 和 `Dis` 加上损失函数 `GenWithLossCell` 和 `DisWithLossCell`。\n",
    "\n",
    "- 定义损失函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from mindspore import nn\n",
    "\n",
    "# 定义损失函数\n",
    "adversarial_loss = nn.BCELoss(reduction='mean')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 连接生成器和损失函数，代码如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mindspore.ops.functional as F\n",
    "\n",
    "'''连接生成器和损失'''\n",
    "class GenWithLossCell(nn.Cell):\n",
    "    def __init__(self, netG, netD, loss_fn, auto_prefix=True):\n",
    "        super(GenWithLossCell, self).__init__(auto_prefix=auto_prefix)\n",
    "        self.netG = netG\n",
    "        self.netD = netD\n",
    "        self.loss_fn = loss_fn\n",
    "    '''构建生成器损失计算结构'''\n",
    "    def construct(self, latent_code2):\n",
    "        fake_data = self.netG(latent_code2)\n",
    "        fake_out = self.netD(fake_data)\n",
    "        loss_G = self.loss_fn(fake_out, F.ones_like(fake_out))\n",
    "        return loss_G\n",
    "    \n",
    "# 实例化WithLossCell\n",
    "loss_G = GenWithLossCell(netGenerator, netDiscriminator, adversarial_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 连接判别器和损失函数，代码如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "'''连接判别器和损失'''\n",
    "class DisWithLossCell(nn.Cell):\n",
    "    def __init__(self, netG, netD, loss_fn, auto_prefix=True):\n",
    "        super(DisWithLossCell, self).__init__(auto_prefix=auto_prefix)\n",
    "        self.netG = netG\n",
    "        self.netD = netD\n",
    "        self.loss_fn = loss_fn\n",
    "    '''构建判别器损失计算结构'''\n",
    "    def construct(self, real_data, latent_code1):\n",
    "        fake_data = self.netG(latent_code1)\n",
    "        real_out = self.netD(real_data)\n",
    "        real_loss = self.loss_fn(real_out, F.ones_like(real_out))\n",
    "        fake_out = self.netD(fake_data)\n",
    "        fake_loss = self.loss_fn(fake_out, F.zeros_like(fake_out))\n",
    "        loss_D = real_loss + fake_loss\n",
    "        return loss_D\n",
    "    \n",
    "# 实例化WithLossCell\n",
    "loss_D = DisWithLossCell(netGenerator, netDiscriminator, adversarial_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4 优化器\n",
    "\n",
    "这里设置了两个单独的优化器，`optimizerD` 用于判别器，另一个 `optimizerG` 用于生成器。这两个都是 `lr = 0.0002` 和 `beta1 = 0.5，beta2 = 0.999` 的 `Adam` 优化器。\n",
    "\n",
    " - 以下是优化器的代码实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 为生成器和判别器设置优化器\n",
    "optimizerD = nn.Adam(netDiscriminator.trainable_params(), learning_rate=lr, beta1=b1, beta2=b2)\n",
    "optimizerG = nn.Adam(netGenerator.trainable_params(), learning_rate=lr, beta1=b1, beta2=b2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 4 案例演示——训练模型\n",
    "\n",
    "## 4.1 训练准备\n",
    "\n",
    "训练分为两个主要部分：训练判别器和训练生成器。\n",
    "\n",
    "- 训练判别器:\n",
    "\n",
    "   训练判别器的目的是最大程度地提高判别图像真伪的概率。\n",
    "   \n",
    "   按照[GAN原论文](https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf)的方法，希望通过提高其随机梯度来更新判别器，所以我们要最大化 $log D(x) + log(1 - D(G(z))$ 的值。\n",
    "\n",
    "\n",
    "- 训练生成器:\n",
    "\n",
    "   如论文所述，我们希望通过最小化 $log(1 - D(G(z)))$ 来训练生成器，以产生更好的虚假图像。\n",
    "\n",
    "在这两个部分中，分别获取训练过程中的损失，并在每个周期结束时进行统计，将 `test_noise` 批量推送到生成器中，以直观地跟踪生成器 `Generator` 的训练进度。\n",
    "\n",
    "下面是，\n",
    "\n",
    " - 单次训练过程的函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from mindspore import nn\n",
    "from mindspore.communication import init\n",
    "\n",
    "'''TrainOneStepCell'''\n",
    "class TrainOneStepCell(nn.Cell):\n",
    "    def __init__(\n",
    "        self,\n",
    "        netG: GenWithLossCell,\n",
    "        netD: DisWithLossCell,\n",
    "        optimizerG1: nn.Optimizer,\n",
    "        optimizerD1: nn.Optimizer,\n",
    "        gap: int,\n",
    "        sens=1.0,\n",
    "        auto_prefix=True,\n",
    "    ):\n",
    "        super(TrainOneStepCell, self).__init__(auto_prefix=auto_prefix)\n",
    "        self.netG = netG\n",
    "        self.netG.set_grad()\n",
    "        self.netG.add_flags(defer_inline=True)\n",
    "\n",
    "        self.netD = netD\n",
    "        self.netD.set_grad()  # 生成需要计算梯度的反向网络\n",
    "        self.netD.add_flags(defer_inline=True)\n",
    "\n",
    "        self.weights_G = optimizerG1.parameters\n",
    "        self.optimizerG1 = optimizerG1\n",
    "        self.weights_D = optimizerD1.parameters\n",
    "        self.optimizerD1 = optimizerD1\n",
    "\n",
    "        self.grad = C.GradOperation(get_by_list=True, sens_param=True) # 灵敏度：返回的梯度乘以灵敏度\n",
    "\n",
    "        self.sens = sens\n",
    "\n",
    "        self.reducer_flag = False\n",
    "        self.grad_reducer_G = F.identity\n",
    "        self.grad_reducer_D = F.identity\n",
    "\n",
    "        self.gap = gap\n",
    "\n",
    "    # train discriminator\n",
    "    def trainD(self, real_data, latent_code, loss, loss_net, grad, optimizer, weights, grad_reducer):\n",
    "        sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens)\n",
    "        grads = grad(loss_net, weights)(real_data, latent_code, sens)\n",
    "        grads = grad_reducer(grads)\n",
    "        optimizer(grads)\n",
    "        return loss\n",
    "\n",
    "    # train generator\n",
    "    def trainG(self, latent_code4, loss, loss_net, grad, optimizer, weights, grad_reducer):\n",
    "        sens = P.Fill()(P.DType()(loss), P.Shape()(loss), self.sens)\n",
    "        grads = grad(loss_net, weights)(latent_code4, sens) # 先得到gradient function, 再带入参数; sens是grad_wrt_output\n",
    "        grads = grad_reducer(grads)\n",
    "        optimizer(grads)\n",
    "        return loss\n",
    "\n",
    "    def construct(self, real_data, latent_code5, now_iter):\n",
    "        d_loss, g_loss = -1, -1\n",
    "        if now_iter % self.gap == 0:\n",
    "            loss_D = self.netD(real_data, latent_code5)\n",
    "            d_loss = self.trainD(real_data, latent_code5, loss_D, self.netD, self.grad, self.optimizerD1, self.weights_D, self.grad_reducer_D)\n",
    "        \n",
    "        loss_G = self.netG(latent_code5)\n",
    "        g_loss = self.trainG(latent_code5, loss_G, self.netG, self.grad, self.optimizerG1, self.weights_G, self.grad_reducer_G)\n",
    "        return d_loss, g_loss\n",
    "    \n",
    "# 实例化TrainOneStepCell\n",
    "GAN_train = TrainOneStepCell(loss_G, loss_D, optimizerG, optimizerD, gap)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 正式训练\n",
    "\n",
    "在正式进行网络模型训练时，我们设置模型训练的 `epoch` 次数为100轮，并定义训练集 `batch_size` 大小为128。图像尺寸为28\\*28，按照高斯分布初始化的25个隐码 `test_noise` 向量长度为100。损失函数使用 `nn.BCELoss`。优化器使用 `nn.Adam`，并设置学习率为0.0002，beta两个超参数分别为0.5和0.999。\n",
    "\n",
    "一般情况下，由于判别器效果提高较块，为保持两者平衡制约，会使判别器等待生成器，但在本案例中，我们将判别器等待轮数 `gap` 设为1，即指示其无需等待。\n",
    "\n",
    "每次迭代，均收集当时生成器和判别器的损失，以便于后面绘制训练过程中损失函数的图像(保存至./result/iter.png)。每经过5次迭代，即保存生成器和判别器的网络模型参数(保存至./result/ipynb_checkpoints/)，以便于后面利用参数还原隐码生成的图片以及评估训练效果。\n",
    "\n",
    " - 模型训练部分的代码如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "import random\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import mindspore\n",
    "from tqdm import tqdm\n",
    "from mindspore import nn\n",
    "from mindspore import ops\n",
    "from mindspore import Tensor, context, save_checkpoint\n",
    "from mindspore.common import dtype as mstype\n",
    "from mindspore.common import set_seed\n",
    "from mindspore.communication import init\n",
    "\n",
    "# 设置参数保存路径\n",
    "checkpoints_path = \"./result/checkpoints\"\n",
    "os.makedirs(checkpoints_path, exist_ok=True)\n",
    "\n",
    "# 设置中间过程生成图片保存路径\n",
    "image_path = \"./result/images\"\n",
    "os.makedirs(image_path, exist_ok=True)\n",
    "def save_imgs(gen_imgs1, idx): # 保存生成的test图像\n",
    "    for i3 in range(gen_imgs1.shape[0]):\n",
    "        plt.subplot(5, 5, i3 + 1)\n",
    "        plt.imshow(gen_imgs1[i3, 0, :, :]/2+0.5, cmap=\"gray\")\n",
    "        plt.axis(\"off\")\n",
    "    plt.savefig(image_path+\"/{}.png\".format(idx))\n",
    "\n",
    "# 选择执行模式为图模式；指定训练使用的平台为\"GPU\"，如需使用昇腾硬件可将其替换为\"Ascend\"\n",
    "mindspore.set_context(mode=mindspore.GRAPH_MODE, device_target=\"Ascend\")\n",
    "\n",
    "# set train\n",
    "netGenerator.set_train()\n",
    "netDiscriminator.set_train()\n",
    "# 储存生成器和判别器loss\n",
    "G_losses, D_losses = [], []\n",
    "\n",
    "\n",
    "for epoch in range(TOTLE_EPOCH):\n",
    "    start = time.time()\n",
    "    train_bar = tqdm(dataset_mnist, ncols=100, total=iter_size)\n",
    "    for (iter, data) in enumerate(train_bar):\n",
    "        image, latent_code = data\n",
    "        image = (image - 127.5) / 127.5 # [0, 255] -> [-1, 1]\n",
    "        image = ops.Reshape()(image, (image.shape[0], 1, image.shape[1], image.shape[2]))\n",
    "        d_loss, g_loss = GAN_train(image, latent_code, iter)\n",
    "        if iter % 100 == 0:\n",
    "            print('[%3d/%d][%3d/%d]  Loss_D:%6.4f  Loss_G:%6.4f' % (epoch+1, TOTLE_EPOCH, iter+1, iter_size, d_loss.asnumpy(), g_loss.asnumpy()))\n",
    "    D_losses.append(d_loss.asnumpy())\n",
    "    G_losses.append(g_loss.asnumpy())\n",
    "    end = time.time()\n",
    "    print(\"time of epoch {} is {:.2f}s\".format(epoch+1, end - start))\n",
    "\n",
    "    # 每个epoch结束后，使用生成器生成一组图片\n",
    "    netGenerator.set_train(False)\n",
    "    gen_imgs = netGenerator(test_noise)\n",
    "    netGenerator.set_train(True)\n",
    "    save_imgs(gen_imgs.asnumpy(), epoch)\n",
    "\n",
    "    # 保存网络模型参数为ckpt文件\n",
    "    if(epoch % 5 == 0):\n",
    "        save_checkpoint(netGenerator, checkpoints_path+\"/Generator%d.ckpt\" % (epoch))\n",
    "        save_checkpoint(netDiscriminator, checkpoints_path+\"/Discriminator%d.ckpt\" % (epoch))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```text\n",
    "[  1/100][  1/468]  Loss_D:1.3855  Loss_G:0.6936\n",
    "[  1/100][101/468]  Loss_D:1.4229  Loss_G:0.6473\n",
    "[  1/100][201/468]  Loss_D:1.2682  Loss_G:0.7873\n",
    "[  1/100][301/468]  Loss_D:1.3058  Loss_G:0.7107\n",
    "[  1/100][401/468]  Loss_D:1.4320  Loss_G:0.7475\n",
    "time of epoch 1 is 463.44s\n",
    "[  2/100][  1/468]  Loss_D:1.2474  Loss_G:0.8244\n",
    "[  2/100][101/468]  Loss_D:1.3096  Loss_G:0.8339\n",
    "[  2/100][201/468]  Loss_D:1.4208  Loss_G:0.7158\n",
    "[  2/100][301/468]  Loss_D:1.3529  Loss_G:0.6910\n",
    "[  2/100][401/468]  Loss_D:1.3520  Loss_G:0.8308\n",
    "time of epoch 2 is 4.66s\n",
    "[  3/100][  1/468]  Loss_D:1.3420  Loss_G:0.8229\n",
    "[  3/100][101/468]  Loss_D:1.5290  Loss_G:0.7103\n",
    "[  3/100][201/468]  Loss_D:1.1767  Loss_G:0.7473\n",
    "[  3/100][301/468]  Loss_D:1.2207  Loss_G:1.2621\n",
    "[  3/100][401/468]  Loss_D:1.3237  Loss_G:0.7856\n",
    "time of epoch 3 is 4.40s\n",
    "...\n",
    "[ 99/100][  1/468]  Loss_D:1.2258  Loss_G:0.9294\n",
    "[ 99/100][101/468]  Loss_D:1.2221  Loss_G:0.8950\n",
    "[ 99/100][201/468]  Loss_D:1.2068  Loss_G:0.9534\n",
    "[ 99/100][301/468]  Loss_D:1.2346  Loss_G:1.0572\n",
    "[ 99/100][401/468]  Loss_D:1.2776  Loss_G:0.8317\n",
    "time of epoch 99 is 4.60s\n",
    "[100/100][  1/468]  Loss_D:1.2261  Loss_G:1.0504\n",
    "[100/100][101/468]  Loss_D:1.2066  Loss_G:0.9289\n",
    "[100/100][201/468]  Loss_D:1.2527  Loss_G:0.7268\n",
    "[100/100][301/468]  Loss_D:1.2531  Loss_G:0.7871\n",
    "[100/100][401/468]  Loss_D:1.2139  Loss_G:1.0260\n",
    "time of epoch 100 is 4.39s\n",
    "```\n",
    "![last_img](./result/images/99.png)\n",
    "\n",
    "# 5 案例演示——结果分析\n",
    "\n",
    "## 5.1 迭代关系图\n",
    "\n",
    "运行下面代码，描绘 `Dis` 和 `Gen` 损失与训练迭代的关系图："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 设置迭代关系图\n",
    "iter_path = './result/iter.png'\n",
    "def paintIters(D_losses, G_losses):\n",
    "    plt.figure(figsize=(10, 10))\n",
    "    plt.title(\"Generator and Discriminator Loss During Training\")\n",
    "    plt.plot(G_losses, label=\"G\", color='blue')\n",
    "    plt.plot(D_losses, label=\"D\", color='orange')\n",
    "    plt.xlabel(\"iterations\")\n",
    "    plt.ylabel(\"Loss\")\n",
    "    plt.legend()\n",
    "    plt.savefig(iter_path)\n",
    "    plt.show()\n",
    "    \n",
    "paintIters(D_losses, G_losses)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2 训练过程动图\n",
    "\n",
    "为了更加直观的观察多轮训练后生成图像结果的变化，我们通过动图可视化，展现了训练过程中随着训练轮数的增加，通过隐码 `test_noise` 生成的图像不断完善。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.animation as animation\n",
    "import cv2\n",
    "image_list = []\n",
    "for i in range(TOTLE_EPOCH):\n",
    "    image_list.append(cv2.imread(\"./result/images/{}.png\".format(i), cv2.IMREAD_GRAYSCALE))\n",
    "\n",
    "gif_path = './result/gan_mnist.gif'\n",
    "def showGif(image_list):\n",
    "    show_list = []\n",
    "    fig = plt.figure(dpi=120)\n",
    "    for epoch in range(0, len(image_list), 5):\n",
    "        plt.axis(\"off\")\n",
    "        show_list.append([plt.imshow(image_list[epoch], cmap='gray')])\n",
    "\n",
    "    ani = animation.ArtistAnimation(fig, show_list, interval=10, repeat_delay=1000, blit=True)\n",
    "    ani.save(gif_path, writer='pillow', fps=1)\n",
    "\n",
    "showGif(image_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![gan_mnist](./result/gan_mnist.gif)\n",
    "\n",
    "## 5.3 模型生成查看\n",
    "\n",
    "从上面的图像可以看出，随着训练次数的增多，图像质量也越来越好。如果增大训练周期数，当 `num_epochs` 达到100以上时，生成的手写数字图片与数据集中的较为相似，下面我们通过加载训练周期为70的生成器网络模型参数文件[Generator70.ckpt](./result/checkpoints/Generator70.ckpt)来生成图像，代码如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import mindspore\n",
    "model = Generator(latent_size=100)\n",
    "param_dict = mindspore.load_checkpoint(\"./result/checkpoints/Generator70.ckpt\")\n",
    "mindspore.load_param_into_net(model, param_dict)\n",
    "\n",
    "model.set_train(True)\n",
    "\n",
    "# 生成图片\n",
    "test_noise = Tensor(np.random.normal(0, 1, (25, 100)).astype(np.float32))\n",
    "imgs = model(test_noise).transpose(0, 2, 3, 1).asnumpy()\n",
    "\n",
    "fig = plt.figure(figsize=(5, 5), dpi=120)\n",
    "for i in range(25):\n",
    "    fig.add_subplot(5, 5, i+1)\n",
    "    plt.axis(\"off\")\n",
    "    plt.imshow(imgs[i].squeeze(), cmap=\"gray\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "## 5.4 模型检验\n",
    "\n",
    "在GAN原文中，使用了Parzen Window方法。\n",
    "\n",
    "Parzen Window是一种应用广泛的非参数估计方法，从采样的样本中估计概率密度函数。这种方法最基本的思想是给定一个特定的区域（Window）对落入其中的样本进行计数，可以得到样本落入该区域的概率。这里Window由核函数确定，根据原文，这里基于高斯分布设计核函数，其中的参数 $\\sigma$ 确定核宽度，采用交叉验证获得。\n",
    "\n",
    "对于我们的项目，实现思路大致是：产生的样本通过高斯核Parzen窗口法计算出一个概率模型 $P_g$（784维的高斯分布），然后估计测试样本的概率，从而计算出在该分布下的对数似然性，方差参数采用在验证集上交叉验证来获得。\n",
    "\n",
    "具体流程如下：\n",
    "\n",
    " 1. 利用函数 `mind_parzen` 定义Gauss Parzen Window\n",
    "\n",
    " 2. 通过 `get_nll` 函数获得似然值\n",
    "\n",
    " 3. 利用函数 `cross_validate_sigma` 在验证集上进行交叉检验\n",
    "\n",
    " 4. 计算对数最大似然估计结果\n",
    "\n",
    " - 代码实现情况如下：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import gc\n",
    "import os\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from mindspore import context\n",
    "import mindspore.numpy\n",
    "from mindspore.common import dtype as mstype\n",
    "from mindspore.common import set_seed\n",
    "from mindspore import Tensor\n",
    "\n",
    "from numpy import cov\n",
    "from numpy.random import random\n",
    "from scipy.linalg import sqrtm\n",
    "\n",
    "set_seed(1)\n",
    "reshape = mindspore.ops.Reshape()\n",
    "log = mindspore.ops.Log()\n",
    "exp = mindspore.ops.Exp()\n",
    "cat = mindspore.ops.Concat()\n",
    "cat2 = mindspore.ops.Concat(1)\n",
    "squeeze1 = mindspore.ops.Squeeze(1)\n",
    "sigma = None\n",
    "\n",
    "# log_mean_exp函数计算平均值\n",
    "def log_mean_exp(a):\n",
    "    max_ = a.max(axis=1)\n",
    "    max2 = reshape(max_, (max_.shape[0], 1))\n",
    "    return max_ + log(exp(a - max2).mean(1))\n",
    "\n",
    "# 使用parzen进行模型评估，设计窗口函数\n",
    "def mind_parzen(x, mu, sigma):\n",
    "    a = (reshape(x, (x.shape[0], 1, x.shape[-1])) - reshape(mu, (1, mu.shape[0], mu.shape[-1]))) / sigma\n",
    "    a5 = -0.5 * (a ** 2).sum(2)\n",
    "    E = log_mean_exp(a5)\n",
    "    t4 = sigma * np.sqrt(np.pi * 2)\n",
    "    t5 = np.log(t4)\n",
    "    Z = mu.shape[1] * t5\n",
    "    return E - Z\n",
    "\n",
    "# 通过get_nll函数获得似然值，其中的操作是每次输入一个batch_size,最后对各个batch_size求取平均值\n",
    "def get_nll(x, samples, sigma, batch_size=10):\n",
    "    inds = list(range(x.shape[0]))\n",
    "    n_batches = int(np.ceil(float(len(inds)) / batch_size))\n",
    "\n",
    "    nlls = Tensor(np.array([]).astype(np.float32))\n",
    "    for i in range(n_batches):\n",
    "        nll = mind_parzen(x[inds[i::n_batches]], samples, sigma)\n",
    "        nlls = cat((nlls, nll))\n",
    "    return nlls\n",
    "\n",
    "# 在验证集上进行交叉检验\n",
    "def cross_validate_sigma(samples, data, sigmas, batch_size):\n",
    "    lls = Tensor(np.array([]).astype(np.float32))\n",
    "    for sigma in sigmas:\n",
    "        print(\"sigma=\", sigma)\n",
    "        tmp = get_nll(data, samples, sigma, batch_size=batch_size)\n",
    "        tmp = reshape(tmp.mean(), (1, 1))\n",
    "        tmp = squeeze1(tmp)\n",
    "        lls = cat((lls, tmp))\n",
    "        gc.collect()\n",
    "\n",
    "    ind = lls.argmax()\n",
    "    return sigmas[ind]\n",
    "\n",
    "# 获取有效的验证图像\n",
    "def get_valid(batch_size_valid=1000):\n",
    "    dataset1 = create_dataset_valid(batch_size=batch_size_valid, repeat_size=1, latent_size=100)\n",
    "    data = []\n",
    "    for image in enumerate(dataset1):\n",
    "        data = image\n",
    "        break\n",
    "    image = data[1]\n",
    "    image = image[0]\n",
    "    image = reshape(image, (image.shape[0], 784))\n",
    "    return image\n",
    "reshape\n",
    "\n",
    "# 获取测试图像\n",
    "def get_test(limit_size):\n",
    "    dataset_ = decode_idx3_ubyte(test_images_file).astype(\"float32\")\n",
    "    image = Tensor(dataset_)\n",
    "    image = reshape(image, (image.shape[0], 784))\n",
    "    return image[0:limit_size]\n",
    "\n",
    "# 使用对数最大似然进行评估\n",
    "def parzen(samples):\n",
    "    ll, se = [1], 1\n",
    "    shape = samples.shape\n",
    "    samples = reshape(samples, (shape[0], -1))\n",
    "    \n",
    "    valid = get_valid(batch_size_valid=BATCH_SIZE_VALID).asnumpy()\n",
    "    valid = Tensor(valid / 255)\n",
    "\n",
    "    sigma_range = np.logspace(-1, 0, num=5)  # 等比数列\n",
    "    sigma = cross_validate_sigma(samples, valid, sigma_range, batch_size=BATCH_SIZE_TEST)\n",
    "\n",
    "    print(\"Using Sigma: {}\".format(sigma))\n",
    "    gc.collect()\n",
    "\n",
    "    test_data = get_test(limit_size=1000) / 255\n",
    "    ll = get_nll(test_data, samples, sigma, batch_size=BATCH_SIZE_TEST)\n",
    "    se = ll.std() / np.sqrt(test_data.shape[0])\n",
    "\n",
    "    return ll.mean(), se\n",
    "\n",
    "context.set_context(mode=context.GRAPH_MODE, device_target=\"GPU\")\n",
    "test_noise = Tensor(np.random.normal(size=(1000, latent_size)), dtype=mstype.float32)\n",
    "llh, epochs = [], []\n",
    "def eval(n):\n",
    "    # 导入生成器参数\n",
    "    model = Generator(latent_size=100)\n",
    "    param_dict = mindspore.load_checkpoint(\"./result/checkpoints/Generator\"+str(n)+\".ckpt\")\n",
    "    mindspore.load_param_into_net(model, param_dict)\n",
    "    # 生成噪音数据\n",
    "    samples1 = model(test_noise)\n",
    "    fake_img = Tensor((samples1.asnumpy() * 127.5 + 127.5) / 255)\n",
    "    fake_img = reshape(fake_img, (fake_img.shape[0], 784))\n",
    "    # 计算对数最大似然估计\n",
    "    mean_ll, se_ll = parzen(fake_img)\n",
    "    epochs.append(n)\n",
    "    llh.append(mean_ll)\n",
    "    print(\"epoch = {}, Log-Likelihood of test set = {}, se: {}\".format(n, mean_ll, se_ll))\n",
    "\n",
    "    \n",
    "for epoch in range(0, 100, 5):\n",
    "    eval(epoch)\n",
    "llh = [i.asnumpy() for i in llh]\n",
    "plt.plot(epochs, llh)\n",
    "eval_path = \"./result/eval.png\"\n",
    "plt.savefig(eval_path)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```text\n",
    "epoch = 0, Log-Likelihood of test set = -126.10223, se: 2.4169679\n",
    "epoch = 5, Log-Likelihood of test set = -75.78903, se: 1.9270847\n",
    "epoch = 10, Log-Likelihood of test set = -93.169586, se: 2.8074365\n",
    "epoch = 15, Log-Likelihood of test set = -94.35247, se: 2.8048227\n",
    "epoch = 20, Log-Likelihood of test set = -81.78036, se: 3.007094\n",
    "epoch = 25, Log-Likelihood of test set = -88.062004, se: 3.1151288\n",
    "...\n",
    "epoch = 70, Log-Likelihood of test set = -5.689421, se: 8.7551365\n",
    "epoch = 75, Log-Likelihood of test set = -3.196967, se: 9.07537\n",
    "epoch = 80, Log-Likelihood of test set = 15.992743, se: 8.775329\n",
    "epoch = 85, Log-Likelihood of test set = 11.999017, se: 8.659082\n",
    "epoch = 90, Log-Likelihood of test set = 14.268165, se: 8.967118\n",
    "epoch = 95, Log-Likelihood of test set = 7.885767, se: 8.825404\n",
    "```\n",
    "![eval](./result/eval.png)\n",
    "\n",
    "针对多轮下生成模型生成的图片分布与真实图片的分布，计算对数最大似然估计得到结果，可以发现当epoch约为75时，生成图形效果最好。\n",
    "\n",
    "# 6 总结\n",
    "\n",
    "本案例基于MindSpore框架针对MNIST数据集，完成了数据读取、数据集构建、GAN模型构建，并根据特定需求自定义了训练轮数和模型框架，进行了模型训练和可视化检验，顺利完成了生成结果的输出。通过此案例进一步加深了对GAN模型结构和特性的理解，并结合MindSpore框架提供的文档和教程，掌握了利用Mindspore框架实现特定案例的流程，以及多种API的使用方法，为以后在实际场景中应用MindSpore框架提供支持。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}