inzva x METU ImageLab Joint Program/Week 2 - Autoregressive Models, Maximum Likelihood Estimation/inzva_x_METU_ImageLab_Joint_Program_Week_2_Solutions_Notebook.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "inzva x METU ImageLab Joint Program Week 2 Solutions Notebook.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ODnWd5AOHArS"
      },
      "source": [
        "# Q1: Theoretical Questions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RYaLVuVYIYyl"
      },
      "source": [
        "1. Briefly explain the differences between following models:\n",
        "* FVSBN\n",
        "* NADE\n",
        "* RNADE\n",
        "* PixelRNN\n",
        "* PixelCNN\n",
        "\n",
        "----\n",
        "\n",
        "2. Briefly explain the relationships between following concepts:\n",
        "* Neg-log-likelihood\n",
        "* Max-likelihood-estimation\n",
        "* Cross-entropy-loss\n",
        "* KL-divergence\n",
        "\n",
        "---\n",
        "\n",
        "3. True/False questions\n",
        "* $D(p || q)$ is equal to $D(q || p)$\n",
        "* Autoregressive models tries to learn a data distribution in an explicit manner.\n",
        "* In PixelCNN masking applies only to previous pixels."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r2gIutxFZQ6h"
      },
      "source": [
        "# Q2: *Fully Visible Sigmoid Belief Networks*\n",
        "In this part, first we are going to implement the evaluation function for the FVSBN. Afterwards we are going to implement a function for sampling purposes."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Taw0m23amk1C"
      },
      "source": [
        "Suppose that we have a dataset which contains handwritten digit images. Each image consist of 28x28 black-or-white pixels (binarised MNIST). \n",
        "Our goal is learning a probabilty distribution $p(x) = p(x_1, x_2, \\dots , x_{784})$ such that when we sample an instance from that distribution it should look like a digit.\n",
        "\n",
        "In autoregressive models, we are binding each pixel (variable) conditionally and sequentially (left-to-right etc.). \n",
        "$p(x_1, x_2, \\dots , x_{784}) = p(x_1)p(x_2|x_1)p(x_3|x_1, x_2) \\dots p(x_{784}|x_1, x_2 \\dots, x_{783})$\n",
        "\n",
        "\n",
        "In Fully Visible Sigmoid Belief Networks, we assume;\n",
        "\n",
        "$p(x_i = 1 | x_1, x_2 \\dots, x_{i-1}) = \\sigma(\\alpha^{(i)}_0 + \\sum_{h=1}^{i-1} \\alpha^{(i)}_h x_h)$\n",
        "\n",
        "Where $\\sigma$ denotes the sigmoid function and $\\alpha$ denotes the learnable parameters.\n",
        "\n",
        "Every i'th pixel have parameter for all the previous pixels and also a bias term, so that the model have $1+2+\\dots+784$ parameters.\n",
        "\n",
        "In the coding part, a denotes the $\\alpha$ and it is a [lower triangular matrix](https://wikimedia.org/api/rest_v1/media/math/render/svg/f770c7fa4c212eac3d7d7f9a54f7decbc811276f)\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XI8yu5EACHoC"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "def sigmoid(x):\n",
        "  return 1 / (1 + np.exp(-x))\n",
        "\n",
        "def evaluate(a, x):\n",
        "  # args:\n",
        "  #   a -> paramaters np.array(784, 784)\n",
        "  #   x -> image sample np.array(1, 784)\n",
        "\n",
        "  # return:\n",
        "  #   a floating point number that denotes the p(x)\n",
        "\n",
        "\n",
        "  ################################# solution ################################# \n",
        "\n",
        "  # \"a\" is a lower triangular matrix:\n",
        "  #  [[1., 0., 0., 0.],\n",
        "  #   [1., 1., 0., 0.],\n",
        "  #   [1., 1., 1., 0.],\n",
        "  #   [1., 1., 1., 1.]]\n",
        "\n",
        "  # since \"Every i'th pixel have parameter for all the previous\n",
        "  #        pixels and also a bias term, so that the model have \n",
        "  #        1+2+ ... +784 parameters.\" \n",
        "  # the first column denotes the bias term, lets split the bias first\n",
        "  b = a[:,0]    # [784]\n",
        "  w = a[:,1:]   # [784, 783]\n",
        "\n",
        "  # since \"Every i'th pixel have parameter for all the previous pixels ...\"\n",
        "  # we don't have any parameter for the last pixel\n",
        "  x = x[0,:-1]  # [783]\n",
        "\n",
        "  z = np.matmul(w, x) + b\n",
        "  p_i = sigmoid(z)\n",
        "\n",
        "  # finally multiply all the i'th probabilities\n",
        "  p = np.prod(p_i)\n",
        "\n",
        "  ############################################################################\n",
        "\n",
        "  return p\n",
        "\n",
        "\n",
        "def sample(a):\n",
        "  # args:\n",
        "  #   a -> paramaters (784, 784)\n",
        "\n",
        "  # return:\n",
        "  #   a numpy array in the shape of (28x28)\n",
        "\n",
        "\n",
        "  ################################# solution ################################# \n",
        "  image = np.zeros([784])\n",
        "  \n",
        "  b = a[:,0]    # [784]\n",
        "  w = a[:,1:]   # [784, 783]\n",
        "\n",
        "  for i in range(1, 783):\n",
        "    p_i = sigmoid(b[i] + np.matmul(w[i,:], image[:-1]))\n",
        "    image[i] = np.random.choice([1,0], p=[p_i, 1-p_i])\n",
        "\n",
        "  image = np.reshape(image, [28,28])\n",
        "  ############################################################################\n",
        "\n",
        "  return image\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ItfIHOKM25ut"
      },
      "source": [
        "# Q3: *Kullback–Leibler Divergence*\n",
        "In this part we are going to implement a function for calculating the KL divergence between given two Gaussians."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4lP8sq8i6wy1"
      },
      "source": [
        "For the math part, [this](https://mr-easy.github.io/2020-04-16-kl-divergence-between-2-gaussian-distributions/) link might help."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dy0_nHwy62eh"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "def gaussian_probability_density(x, mean, std_dev):\n",
        "  return 1 / (std_dev * np.sqrt(2 * np.pi)) * np.exp(-1 * (x - mean)**2 / (2 * std_dev**2))\n",
        "\n",
        "def kl_divergence(p, q):\n",
        "  # args:\n",
        "  #   p -> gaussian-1, tupple(mean, std)\n",
        "  #   q -> gaussian-2, tupple(mean, std)\n",
        "\n",
        "  # return:\n",
        "  #   a floating point number that denotes D(p||q)\n",
        "\n",
        "  p_dist = np.random.normal(p[0], p[1], 1000)\n",
        "  q_dist = np.random.normal(q[0], q[1], 1000)\n",
        "\n",
        "\n",
        "  ##### solution #####\n",
        "  p = gaussian_pdf(p_dist, p[0], p[1])\n",
        "  q = gaussian_pdf(q_dist, q[0], q[1])\n",
        "  d = np.sum(p * np.maximum(np.log(p / q), 0))\n",
        "  ####################\n",
        "\n",
        "  return d\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "NXSqy5vaDSg_",
        "outputId": "fe95f85f-cfa0-4bba-c011-1a8f12922573"
      },
      "source": [
        "# Test\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import scipy.stats as stats\n",
        "import math\n",
        "\n",
        "p = (0, 1)\n",
        "q = (16, 3)\n",
        "y = (20, 6)\n",
        "\n",
        "pl = np.linspace(p[0] - 3*p[1], p[0] + 3*p[1], 100)\n",
        "ql = np.linspace(q[0] - 3*q[1], q[0] + 3*q[1], 100)\n",
        "yl = np.linspace(y[0] - 3*y[1], y[0] + 3*y[1], 100)\n",
        "\n",
        "plt.plot(pl, stats.norm.pdf(pl, p[0], math.sqrt(p[1])))\n",
        "plt.plot(ql, stats.norm.pdf(ql, q[0], math.sqrt(q[1])))\n",
        "plt.plot(yl, stats.norm.pdf(yl, y[0], math.sqrt(y[1])))\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYAAAAD4CAYAAADlwTGnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXzU9b34+9d7JvtKAmHfERBwQwPuttWKqD2grVY91Ys9vce2R7sc23trl2M9tp7b3fae2kfL6aG1thYRreLSUrcu1o2gCAICYTOBhATIRtaZzPv3x/c7OIQsk+Q7W+b9fDzyYOa7vhnIvL+fXVQVY4wx6ceX6ACMMcYkhiUAY4xJU5YAjDEmTVkCMMaYNGUJwBhj0lRGogPoacyYMTp9+vREh2GMMSll48aNh1W1bDDnJF0CmD59OhUVFYkOwxhjUoqI7B/sOVYFZIwxacoSgDHGpClLAMYYk6YsARhjTJqKKgGIyFIR2SEilSJyVz/HfUxEVETKI7Z91T1vh4hc4UXQxhhjhm/AXkAi4gceAC4HqoENIrJOVbf1OK4Q+ALwesS2+cCNwAJgIvC8iMxR1W7v/grGGGOGIpoSwGKgUlX3qGoXsBpY3stx3wK+C3REbFsOrFbVTlXdC1S61zPGGJNg0SSASUBVxPtqd9txInI2MEVVnxnsuclOVXn8zWqe33Yo0aEYY4ynhj0QTER8wI+AW4dxjduA2wCmTp063JA89edth7hzzdsAPP25izhtUnGCIzLGGG9EUwI4AEyJeD/Z3RZWCJwG/EVE9gHnAevchuCBzgVAVVeqarmqlpeVDWokc8w9WlFNYU4GGT7hsTerEx2OMcZ4JpoEsAGYLSIzRCQLp1F3XXinqjap6hhVna6q04HXgGWqWuEed6OIZIvIDGA28Ibnf4sYCXaH+EflYa5dOImLZ4/hrzvqEx2SMcZ4ZsAqIFUNisgdwHrAD6xS1a0ici9Qoarr+jl3q4isAbYBQeD2VOoBtPPQMdoD3ZwzrYRxRTm8tKOepvYAxbmZiQ7NGGOGLao2AFV9Fni2x7a7+zj2gz3e3wfcN8T4EmrLgUYAzpg8itL8LAC2HmjiglPGJDIsY4zxhI0E7seuQ8fIzvAxtTSPueMKnW11xxIclTHGeMMSQD8q648xs6wAv08oK8ymMCeDXXUtiQ7LGGM8YQmgH3sPtzKrLB8AEWFWWQF76lsTHJUxxnjDEkAfukPKgYZ2ppbmHd82tTSPqoa2BEZljDHesQTQh5qmdoIhPSEBTCnN5WBjB8HuUAIjM8YYb1gC6EPV0XYApkQmgJI8ukNKTVNHX6cZY0zKsATQh4ONTgKYOCr3+Lbwa0sAxpiRwBJAH2qbnS/5CcU5x7dNHOW8rmlqT0hMxhjjJUsAfahpaqckL5OcTP/xbeOLrQRgjBk5LAH0obapg3FFOSdsK8jOoDA7g5pGKwEYY1KfJYA+HGruZHxxzknbxxXnUNfSmYCIjDHGW5YA+lDX0kFZQfZJ28sKsi0BGGNGBEsAvegOKYePdTG26OQEMLYom7oWawMwxqQ+SwC9ONraRXdIGVt4chXQ2MJs6ls6UdUERGaMMd6xBNCL8BN+WWEvJYDCHDoCIVo6g/EOyxhjPGUJoBdHjnUBvSeAMYVZJxxjjDGpKqoEICJLRWSHiFSKyF297P+MiGwRkU0i8rKIzHe3TxeRdnf7JhH5udd/gVg40uo08o52F4GJNDrfSQqHj1lDsDEmtQ24IpiI+IEHgMuBamCDiKxT1W0Rhz2sqj93j18G/AhY6u7brapneRt2bIWf7kf30gtodEG4BGAJwBiT2qIpASwGKlV1j6p2AauB5ZEHqGpzxNt8IKVbSOuPdZLl91GUc3J+DHcNrbcqIGNMiosmAUwCqiLeV7vbTiAit4vIbuB7wOcjds0QkbdE5K8icnFvNxCR20SkQkQq6uvrBxF+bBw51kVpfhYictK+Erda6LCNBTDGpDjPGoFV9QFVnQV8BfiGu7kGmKqqC4E7gYdFpKiXc1eqarmqlpeVlXkV0pAdbe06XtXTU6bfR3FuJg1tVgIwxqS2aBLAAWBKxPvJ7ra+rAauAVDVTlU94r7eCOwG5gwt1Pg52uqUAPpSmp/F0VZLAMaY1BZNAtgAzBaRGSKSBdwIrIs8QERmR7y9Gtjlbi9zG5ERkZnAbGCPF4HHkiUAY0w6GLAXkKoGReQOYD3gB1ap6lYRuReoUNV1wB0i8mEgADQAK9zTLwHuFZEAEAI+o6pHY/EX8VJDaxcleX0ngJK8LKptbWBjTIobMAEAqOqzwLM9tt0d8foLfZz3GPDYcAKMt66gM8q3tzEAYaPzs9hyoDGOURljjPdsJHAPjW7j7qh+EkCJWwVk8wEZY1KZJYAejroJoLTfKqBMAt1Ka1d3vMIyxhjPWQLooaE1ADhf8n0Jtw80WEOwMSaFWQLo4XgVUD8lgFFucmhsC8QlJmOMiQVLAD00uF/q/XUDDY8GtsFgxphUZgmgh4bjJYD+qoAyTzjWGGNSkSWAHhrbusjJ9JGT6e/zmHD1kFUBGWNSmSWAHhrbAv0OAgMoznVKAE3tlgCMManLEkAPje2B41/wfcn0+yjIzrASgDEmpVkC6KGpLdBv/X9YcW4mje3WBmCMSV2WAHpobO9iVG7/VUDgNBI3WQnAGJPCLAH00BhlCWBUXiaN1gZgjElhlgAiqGpUbQAAo3KzrBuoMSalWQKI0BEI0RUMURxFCaAoN5NmKwEYY1KYJYAI4W6dUbcBtAdsRlBjTMqyBBAhnACiqQIqznVmBG2zGUGNMSkqqgQgIktFZIeIVIrIXb3s/4yIbBGRTSLysojMj9j3Vfe8HSJyhZfBe22wCSDyHGOMSTUDJgB3Td8HgCuB+cBNkV/wrodV9XRVPQv4HvAj99z5OGsILwCWAj8LrxGcjMIzgVoCMMakg2hKAIuBSlXdo6pdwGpgeeQBqtoc8TYfCFeMLwdWq2qnqu4FKt3rJaWhlABsNLAxJlVFsybwJKAq4n01cG7Pg0TkduBOIAu4NOLc13qcO6mXc28DbgOYOnVqNHHHhFUBGWPSiWeNwKr6gKrOAr4CfGOQ565U1XJVLS8rK/MqpEFr7ggiAoU5A+fFcAJo7rAEYIxJTdEkgAPAlIj3k91tfVkNXDPEcxOquT1AYXYGPp8MeGxROAFYCcAYk6KiSQAbgNkiMkNEsnAadddFHiAisyPeXg3scl+vA24UkWwRmQHMBt4Yftix0dQeOP7FPpDC7AxErArIGJO6BqzrUNWgiNwBrAf8wCpV3Soi9wIVqroOuENEPgwEgAZghXvuVhFZA2wDgsDtqpq0Heebo5wGAsDnE4pybDSwMSZ1RdMIjKo+CzzbY9vdEa+/0M+59wH3DTXAeGoaRAIAKMrNsBKAMSZl2UjgCE3tAYpyok8AxbmZlgCMMSnLEkCE5o4ARblRFYoASwDGmNRmCSBCc3twcFVAOZm0dARjGJExxsSOJQBXVzBEe6B7UAmgODfTxgEYY1KWJQBX+Is82m6g4WOtCsgYk6osAbjC3TkH0whclJNBRyBEZzBpe7YaY0yfLAG4mt26/ME0AodLC9YOYIxJRZYAXEMrAdh0EMaY1GUJwDWUNgCbEdQYk8osAbia251qnMGOBAarAjLGpCZLAK7jJYChVAFZV1BjTAqyBOBqag+Q6RdyMqP/SN6fEtpKAMaY1GMJwNXszgMkMvBaAGFWAjDGpDJLAK7mjuCgGoABcjJ9ZPrFegEZY1KSJQBXc3sgqqUgI4kIhTk2HYQxJjVZAnA1dww+AYAzGrjJ2gCMMSkoqgQgIktFZIeIVIrIXb3sv1NEtonIZhF5QUSmRezrFpFN7s+6nucmi8GsBhapKDeTFisBGGNS0ICPvCLiBx4ALgeqgQ0isk5Vt0Uc9hZQrqptIvJZ4HvADe6+dlU9y+O4PdfcERxUF9AwWxbSGJOqoikBLAYqVXWPqnYBq4HlkQeo6kuq2ua+fQ2Y7G2Ysdc8iAXhIxXlZhyfR8gYY1JJNAlgElAV8b7a3daXTwF/jHifIyIVIvKaiFzT2wkicpt7TEV9fX0UIXmrM9hNZzBE0ZDaAKwKyBiTmgb/jdcPEbkZKAc+ELF5mqoeEJGZwIsiskVVd0eep6orgZUA5eXl6mVM0Wg5PhPo0NoAbCCYMSYVRVMCOABMiXg/2d12AhH5MPB1YJmqdoa3q+oB9889wF+AhcOINybCdfhD6QVUmJ1Be6CbrmDI67CMMSamokkAG4DZIjJDRLKAG4ETevOIyELgFzhf/nUR20tEJNt9PQa4EIhsPE4Kx9cCGEojcK6NBjbGpKYBH3lVNSgidwDrAT+wSlW3isi9QIWqrgO+DxQAj7pTKbynqsuAecAvRCSEk2y+06P3UFJ4vwQwtEZgcKqRxhRkexqXMcbEUlR1Hqr6LPBsj213R7z+cB/nvQKcPpwA4yH89D6kcQC2KIwxJkXZSGAiG4GH0AvIqoCMMSnKEgBDWw4y7P0SgPUEMsakFksAOE/vfp+Ql+Uf9LnhnkNWAjDGpBpPxwGkqub2IEU5GYNaCyDs/UVhLAGklO4grP8qVD4PuaWw7L9h3PxER2VMXFkJAOfpfSiDwADys/z4xEoAKSUUgkdXwBsrYcxcaKqGX10JNW8nOjJj4soSAENbCyBMRCjKzaTJSgCpY8saePdpuPxb8M+r4VPrISMHnvoiaNwHohuTMJYAcAaCDaULaJgzH5A1AqeEQAe88C2YcBacf4ezrWQ6XHY3HHwTtv4hoeEZE0+WAHh/PeChKsrNsDaAVPHWQ9BcDUu+Bb6I//5n3gjjToOX/stKASZtWALAGQcwrARgJYDU8dZDMP4MmHHJidt9fjjvs3BkF1S9kZjYjIkzSwCEG4GH3iGqyNYFTg217zgNvQtv7n3//GsgMx82/Ta+cRmTIGmfAALdIdq6uj2oArISQNLb9DD4MuG063rfn10A85fDO3+ArrbejzFmBEn7BDCctQDCrASQAlRh2xMw+3LIH933cWfeAF0tsPvF+MVmTIKkfQIYzloAYUW5mbR1dRPotjUBktahrdB8AOZe2f9x0y6E7CLYtT4+cRmTQGmfAML994fXDdSdDsJ6AiWv8Bf6KZf3f5w/E2Z9CHY9Z72BzIiX9gkgXHUzrCqg4zOCWjtA0tr5Z6f3T9GEgY+dvQRaaqB2c+zjMiaBLAG0D301sLBimw8oubU3QPUbMOeK6I6fvcT5c9efYxeTMUkgqgQgIktFZIeIVIrIXb3sv1NEtonIZhF5QUSmRexbISK73J8VXgbvhfdLAMNrA4i8lkky+18BDcHMD0V3fMFYZ1DY3r/HNi5jEmzABCAifuAB4EpgPnCTiPScNvEtoFxVzwDWAt9zzy0FvgmcCywGvikiJd6FP3zNnrQB2JoASW3fP8CfDZPOif6caRc6A8KCXbGLy5gEi6YEsBioVNU9qtoFrAaWRx6gqi+parjj9GvAZPf1FcBzqnpUVRuA54Cl3oTujeaOABk+ITdz8GsBhIVLD1YCSFL7X4YpiyEzJ/pzpl8EwXY4+Fbs4jImwaJJAJOAqoj31e62vnwK+ONgzhWR20SkQkQq6uvrowjJO03uTKBDWQsgLFx6sBlBk1BHE9RucZ7oByN8/P6XvY/JmCThaSOwiNwMlAPfH8x5qrpSVctVtbysrMzLkAbU3D68mUABcjP9ZPjEGoGT0XuvOfX/0weZAPJHQ9k8p/rImBEqmgRwAJgS8X6yu+0EIvJh4OvAMlXtHMy5idTUHhh2AhARim1NgOT03qvgy4DJiwZ/7rQLnHaAULf3cRmTBKJJABuA2SIyQ0SygBuBdZEHiMhC4Bc4X/51EbvWA0tEpMRt/F3ibksaw1kNLFJRbqaNA0hG1RVO///M3MGfO2WxMy1E/Q7v4zImCQyYAFQ1CNyB88W9HVijqltF5F4RWeYe9n2gAHhURDaJyDr33KPAt3CSyAbgXndb0mhq9y4BWAkgyXQH4cCbQ3v6h/fPq97gXUzGJJGoOr+r6rPAsz223R3x+sP9nLsKWDXUAGPNWRDegwSQY4vCJJ367RBoHXoCKJ0JuSVOAjgn6YawGDNsaT0SWFVp9qANAJyeQJYAkkz4yX1y+dDOF3GSR3WFdzEZk0TSOgF0BEJ0dYeGNQo4zKqAklB1BeSNcdb8HarJi6D+Xac7qTEjTFonAC9mAg0L9wJSm0EyeVRXOE//wxjj4YweVqctwZgRxhIA3iWAYEhpD1iXwaTQ2QKHd8LEhcO7Tvj8mk3Dj8mYJGMJAO8SQOQ1TYLVbgF0+AkgrxRGTYODlgDMyGMJAEsAI1J4Dp8JZw3/WhPPsjmBzIhkCQCPE0CbJYCkcHATFE6EwnHDv9bEhdC4H9qSagiLMcNmCQArAYxIB99ynty9EC5FWDuAGWEsAQCFHgwECyeARksAidfZAkcqh1//HxZOJNYOYEaY9E4AbV0U5WTg9w2jm6CryJaFTB7hBuAJZ3pzvdwSpyHYSgBmhEnvBNAeoDhv+E//AIXZGfjEqoCSQo27mPv4M7y75oQz3MRizMgx/CGwKayxPcCo3CxPruXzCUW5mTRaI3Di1W6G/DIoHO/dNcefCdufgo5myCka8mXea36Pdw6/A4BPfCyesJjSnFKvojRmUNI6ATS1BxjlUQkAYJRNB5EcajY7T//DGQHc0wS3NHHoHWedgEEKhAKs2rKKX2z+BYHQ+/9HirOL+cqir/CRmR8Z1qp0xgxFeieAtgATRw1hnvg+FOdmWiNwogU7nVlAZ/c5Qe3QhKuTajYPOgFUtVTxxZe+yM6GnVwx/Qr+9fR/JcufRVNnEz+o+AFfe/lrPLv3Wb5/yfcpyCrwNm5j+pHeCcCjmUDDivOyaGrr8ux6ZgjqtkMo6G39PzjVSfllTvXSILQF2vj8i5+nrq2OH3/ox1w29bIT9j+49EF+/+7v+UHFD/j6y1/n/g/dj0/SumnOxFFU/9NEZKmI7BCRShG5q5f9l4jImyISFJHreuzrdheJOb5QTDJQVRrbA5RYFdDIEv6C9qoHUJiIk1Rqok8Aqso3/vEN9jTt4Qcf+MFJX/4Afp+fm+ffzJfKv8SLVS/yP5v/x8uojenXgAlARPzAA8CVwHzgJhGZ3+Ow94BbgYd7uUS7qp7l/izrZX9CHOsM0h1SzxqBAUblWRVQwtVugawCKJnh/bXHn+5ULwU7Bz4W+NXWX/Hc/uf497P/nfMnnt/vsTfPu5mrZ17NA5se4G/Vf/MiWmMGFE0JYDFQqap7VLULWA0sjzxAVfep6mYgFIMYYyLcW8erbqDwfgkgFLIpoROmdguMWwC+GFSjjD/dqV6KYo3gHUd38JM3f8IV069gxYKBVxMTEb55/jeZWzqXr7/8dZo6bf0BE3vR/JZMAqoi3le726KVIyIVIvKaiFwzqOhiKJwARnncBqDqLDRvEiAUgtp3nC/qWBgf0ROoH6rK9yu+T2FWIf9x3n9E3bsnNyOXb134LZo6m6wqyMRFPFqbpqlqOfDPwI9FZFbPA0TkNjdJVNTX18chJGhsdxprS/K9qwIKtyfYWIAEadwPXS2xSwCjZ0FG7oADwv5+4O+8XvM6nz3zsxRnFw/qFqeWnsryU5bz8LsPU9VcNfAJxgxDNAngADAl4v1kd1tUVPWA++ce4C/ASRO0qOpKVS1X1fKysrJoLz0ssSgBhMcUWDtAgoS/mGOVAHx+p3qpnwQQCAX4QcUPmF40nY/P/fiQbvO5hZ8jw5fB/W/eP9RIjYlKNAlgAzBbRGaISBZwIxBVbx4RKRGRbPf1GOBCYNtQg/VSo9td08s2gGK3QbnBuoImxqF3QHwwtmcfBQ+NP91JAH0s/fnYzsfY27SXfz/n38n0De3/1ti8sXxywSd5bv9zvFVn6xCY2BkwAahqELgDWA9sB9ao6lYRuVdElgGIyCIRqQauB34hIlvd0+cBFSLyNvAS8B1VTYoE0HC8BBCLKiBLAAlRuwVGz4ZM7wb3nWT8adDRCE3VJ+3q6u5i5eaVnDPuHD405UPDus2KBSsoyy3jgbceGNZ1jOlPVAPBVPVZ4Nke2+6OeL0Bp2qo53mvADEqjw9PQ1sXBdkZZGV41wxSkueWAFqtCighat+BKYtie49wQ3DtFhg15YRdz+x5hvr2er590beHPa1DXmYet8y/hR9t/BHbjmxj/ugYlmpM2krbIYeNbd7OAwTOlNAiVgJIiPYGaHrP+xHAPY2dD8hJPYFUlQe3PsickjmcP6H/Pv/Rum7OdeRn5vPg1gc9uZ4xPaVtAmho6zr+xO4Vv08YlZvJUUsA8VfrfiGPPy2298kugNKZJ00J8fKBl9ndtJtbF9zq2aRuhVmFfGz2x1i/bz01x2o8uaYxkdI4AXhfAgCnGqjBuoHG3/EeQDEuAcD7DcERHtz6IGNzx7J0+lJPb3XzvJsB+O3233p6XWMgjRNAYwxKAOB0BW1otRJA3NVugYJxUDA29vcafzo07HPWBgC2HdnG67Wv84n5nyDT7+1DxYSCCVwx/QrW7lxLc1ezp9c2Jm0TwNHWLko9HAQWVppvJYCEOLQFxsW4+icsPM7gkNPZ7eHtD5Obkct1c67r56ShW7FgBW3BNtZVJs1cimaESMsEEOgO0dIRjEkJoCQvy0oA8Rbsgrp3YzcArKfwfWq30NzVzPp967lqxlUUZQ19pbD+zB89n9PHnM7anWvRPsYfGDMUaZkAwgO1SvO9bwMozc/iaFuX/aLG0+EdEArELwEUToC80VD7Nk/vfpqO7g6un3t9TG95/Zzr2d202waGGU+lZwJw++l7OQ9QWEl+Fl3BEG1d3Z5f2/QhFovA90cExp+O1m5m7a61zB89nwWjF8T0lldMv4KCzALW7lwb0/uY9JKWCeCoW0VTGoMqoPA1j1o1UPzUboHMPGeytngZfzpvN+5mV8OumNX9R8rLzOPqmVezft96myraeCatE0CsSgCR9zBxULvZXQPAH797jj+TtfnZ5PlzuGrGVXG55fVzrqcr1MVTu5+Ky/3MyJemCcBZ0Wl0QWx6ATn3sAQQF6pOCSBe1T+u5jGzWJ+fx1XFp5KfmR+Xe84tncsZY87g0Z2PWhuT8URaJoAj4RJADKqARrsJ4IglgPho2AedzfFrAHb9qXknHT4f11EQ1/teN+c69jTtYfPhwS1Ob0xv0jIBHG3tojg3k0y/93/90oJwCSC6dWPNMB1fBD6+JYAndz/FKSE/8w/vj+t9l0xfQm5GLk9WPhnX+5qRKS0TwJHWruNP6l4rzM4g0y9WAoiX2i0g/tiuAdDDnkbnCfyagplI7Tt9rg0QC/mZ+Vw+7XL+tPdPdAQ74nZfMzKlZQI4eiw2o4DBWdx7dH42R45ZAoiLg5ugbG5s1wDo4YndT+AXP1dPuRQ6m6Bhb9zuDbB81nJaAi28+N6Lcb2vGXnSMgEcae1kTEF2zK4/uiDLGoHjQRVqNsGEs+J2y2AoyFO7n+LiyRczZuqFzsaat+N2f4Dy8eVMKpjEE5VPxPW+ZuSJKgGIyFIR2SEilSJyVy/7LxGRN0UkKCLX9di3QkR2uT8rvAp8OA4f64pJD6Cw0QXZHD5mbQAx11IDrfUwMX4J4JWDr3C4/TDXzLrGqXbyZTqlkDjyiY9ls5bxWs1r1LbWxvXeZmQZMAGIiB94ALgSmA/cJCI9K1zfA24FHu5xbinwTeBcYDHwTREpGX7YQxfsDtHQ1hXTEsCYgiyrAoqH8JP3hDPjdssnK5+kJLuESyZfAhnZMHZe3EsAAMtmLUNR1u22CeLM0EVTAlgMVKrqHlXtAlYDyyMPUNV9qroZCPU49wrgOVU9qqoNwHOAtxOmD5IzT4/zJR0rYwqyqT/WaX21Y+3gJmcR+Dh1AW3qbOKlqpe4auZV70/7PPEspxoqzv/Wkwsns2j8ItbtXmf/z8yQRZMAJgFVEe+r3W3RiOpcEblNRCpEpKK+vj7KSw9N+Ml8dCzbANz5gFo6gzG7h8H54h0zB7LiMxDrj3v/SCAUYPmsiOefCWc5y1E2vheXGCItm7WM/c37ebs+/iUQMzIkRSOwqq5U1XJVLS8rK4vpvepbnLr5ssLYJYDwtQ+3WDtATB3cFNfqn3W71zGnZA6nlp76/sZwA3RNfNsBAC6fdrkzJmC3jQkwQxNNAjgATIl4P9ndFo3hnBsT4cbZshiWAI4nAGsHiJ3mg3CsFiaeHZfb7Wncw5bDW1g2a9mJa/6OP81pCD7wZlziiGRjAsxwRZMANgCzRWSGiGQBNwLRtjytB5aISInb+LvE3ZYw8SwB1FsJIHbCX7iTzonL7Z7c/aTT93/m1SfuyMh2ksDB+CcAcKqBjgWO8VLVSwm5v0ltAyYAVQ0Cd+B8cW8H1qjqVhG5V0SWAYjIIhGpBq4HfiEiW91zjwLfwkkiG4B73W0JU9/SSV6Wn/zsjJjdY2xhjnsveyqLmQMbwZcRlwbg7lA3T+9+mosmXcSY3DEnHzDxbDjwFoR69oGIvUXjFzEhf4JNDWGGJKo2AFV9VlXnqOosVb3P3Xa3qq5zX29Q1cmqmq+qo1V1QcS5q1T1FPfnV7H5a0SvriW2g8AARuVmkuET6qwEEDsHNjpTQGfmxPxWr9W8Rl17HctPWd77AZPOga4WOLIr5rH05BMf/zTrn3i15lXq2urifn+T2pKiETie6lo6GFcU2wTg8wllhdmWAGIlFHIagONV/VP5JEVZRXxg8gd6PyAcRwLaAcCZGiKkIRsTYAYt/RJAc+fxKppYGluYzaFmqwKKiaO7nTl44tAA3NTZxAvvvcBHZn6ELH8fY0fGzIasAjhQEfN4ejO1aCpnjz2bJyuftDEBZlDSLwG0dDI2xiUAgLFFOdYIHCvVG5w/41ACeGbPM3SFurh29rV9H+Tzw8SFUJ2YBABw7exr2de8zxaNN4OSVgmgtTPIsc5gXEoA44qyqbUSQGxUvQ7ZxVB26sDHDtMTlU8wr3TeiX3/ezPlXGdq6q7WmMfUmyXTlpCXkccfKv+QkPub1JRWCSD8hTy+OPYlgHGFOTS2BegIdMf8XmmnagNMWQS+2AyWKekAABWpSURBVP733X5kO9uPbu//6T9syrmg3XAwMU/geZl5XDnjStbvW09rIDFJyKSe9EoATW4CKIr93PHji51ShrUDeKyjCeq2weTFMb/VHyr/QJYvK7pF3yeXO39WvR7boPpxzSnX0B5sZ/2+hA61MSkkLRPAhOLYVwGFE0D4nsYj1RWAwpTYJoDO7k6e2fMMl027jOLs4oFPyCt15iWqeiOmcfXnzLIzmVk8k8d3PZ6wGExqSa8EcLwKKPYJIJxkrB3AY1VvODOAxrgB+Pn9z9Pc1cy1p0RR/RM2ZbFTAkhQTxwR4dpTruXt+repbKhMSAwmtaRVAqhpamdUXiY5mf6Y32t8sVPNdLDREoCn3nvFGQCWUxTT2zyy4xGmFE7h3AnnRn/SlPOcmUHrd8QusAEsP2U5Wb4sHtnxSMJiMKkjrRLAwcYOJo2Kz9qxBdkZFOVkUNPUHpf7pYVgp1MCmHZRTG+zs2Enb9W9xQ1zb8Ang/gVme4uEbn/5dgEFoWSnBKWTF/CU3ueoi3QlrA4TGpIswTQzsQ4JQCAiaNyOdhoCcAzB96EYMf7X7QxsmbHGrJ8WSfO+x+NkhlQOBH2JS4BANww9wZaA608s/eZhMZhkl9aJYADje1MjEP9f9ikUblUN1gC8Ez4yXpa7BJAa6CVp3Y/xdIZSxmVM2pwJ4vA9Itg3z8S1g4ATmPw3JK5PPLuIzYy2PQrbRJAU3uAlo4gk0riVwKYVJLLASsBeGffyzB2gdPjJkae3v00bcE2bph7w9AuMP1CaK2DI4lrhBURbjj1BnY07LDVwky/0iYBVDc49aFTSvLids8pJXm0dARpag/E7Z4jVrj+P4bVP6rK6h2rmVc6j9PHDHGa6ekXO3/u/Zt3gQ3B1TOuJj8zn9U7Vic0DpPc0iYBVB11nsSnlMYvAUx2SxtVR60xbtiqXodAG8z8UMxu8WrNq1Q2VnLTqTeduOrXYJTOhOIpsPtFb4MbpLzMPK455RrW71vPodZDCY3FJK+0SQDhEsDkOFYBTXZLG+F7m2GofMFZAGbGxTG7xa/f+TVjcsecvOrXYIjArEudEkB30LvghuDmeTcT0hC/2/67hMZhkldUCUBElorIDhGpFJG7etmfLSKPuPtfF5Hp7vbpItIuIpvcn597G3709h9poygng1F5fUzpGwNTR+cdv7cZpt0vOvPtZBfG5PLvHn2XV2te5RPzPtH3tM/RmnUpdDYnbHrosMmFk1kybQmP7nyUY13HEhqLSU4DJgAR8QMPAFcC84GbRGR+j8M+BTSo6inA/cB3I/btVtWz3J/PeBT3oO070sr0MflxvWdxbiYleZnsswQwPMfqoXYzzIpd9c+vt/6avIw8Pj7348O/2MwPOKOVE1wNBHDrabdyLHCMtTvXJjoUk4SiKQEsBipVdY+qdgGrgZ4dpJcDD7qv1wKXyZArUWPjvaNtTI1j/X/YtNH57D9iszMOS+Xzzp+zLovJ5WuO1fCnvX/iY3M+RlGWByOMc0ucqSp2/Xn41xqmBaMXsHj8Yh7a/hCBbuuMYE4UTQKYBFRFvK92t/V6jLuIfBMw2t03Q0TeEpG/ikivFbgicpuIVIhIRX19/aD+AtHoDHZTdbSNmXEuAQDMHJPP3sOWAIZlxzNQOAEmnBWTy/9m228AuGXeLd5ddO6VztTQzQe9u+YQ3brgVura6nh277OJDsUkmVg3AtcAU1V1IXAn8LCInPSIpaorVbVcVcvLyso8D+K9I22EFGaWFXh+7YHMGJNPTVMHbV2JbRBMWYEOqHzR+UKNwfz/h1oP8ejOR7l65tVMKJjg3YXnug3JOxL/pXvRpIs4tfRUfv72zwmErBRg3hfNb9QBYErE+8nutl6PEZEMoBg4oqqdqnoEQFU3AruBOcMNerB21ztP4DPLElACcJPOnnorBQzJ3r9CoBVOHUbPnH6s3LySbu3ms2d+1tsLl82F0lnwbuITgIjwuYWfo/pYNX/YZSuGmfdFkwA2ALNFZIaIZAE3Aut6HLMOWOG+vg54UVVVRMrcRmREZCYwG9jjTejR23WoBYBZCSgBzB7n3LOyznphDMn2pyCr8P0BVh6qaq7i8V2Pc93s65hcONnbi4vAqVc53UHbG7299hBcPOliFo5dyC/e/gUdQZuh1jgGTABunf4dwHpgO7BGVbeKyL0issw97H+B0SJSiVPVE+4qegmwWUQ24TQOf0ZVj3r9lxjIzrpjTBqVS352RrxvzfTR+WT4hJ1uEjKDEOyE7eucL9IM75fx/NnbPyPDl8FtZ9zm+bUBmH8thAJOEkswEeHzCz9PXXudTRVtjovqG1FVnwWe7bHt7ojXHcD1vZz3GPDYMGMctp21LcwZF/+nf4CsDB8zy/LZUWsJYNB2PecsAXm6B10ze166YRfP7HmGT572ScryvG93AmDS2c7I4C1r4GwPG5iHqHx8ORdOvJBfbvklH539UQqzYjOmwqSOET8SuCPQze76Y8yfGNsFRPozb0IR22uaE3b/lLVlDeSNgZkf9PSyqsp9r99HYVYh/3Lav3h67ROIOMlr79+TojcQwOfP/jxNnU389K2fJjoUkwRGfAKorDtGMKTMm5C4BDB/QhEHmzpoaO1KWAwpp+0o7PgTnPZR8Htbdbdu9zo2HtrInefcGd16v8Nx+vWAwubkqHaZP3o+N556I6t3rGbr4a2JDsck2IhPAJurmwA4fVKMf9H7cZp77y0HmhIWQ8rZ9DB0d8LZ/5enl23saOSHFT9k4diFXDt7EOv9DtWYU5z1Czb+GkKh2N8vCp9b+DlKc0q597V76Q51Jzock0BpkAAaKc7NTMgo4LBwAthcnfjeICkhFIKKVc4au+OHOC1zH+5/835aulr4xnnfGNxyj8Ox6FPQsC8ppoYAKMwq5CuLvsK2I9tsuug0N+ITwJvvNXDWlFFDn97XA8W5mcwqy+fN9ywBRGXPS3B0t/PF6aG/Vf+Nx3c9zi3zb2FOSRyHo5z6T5A/Ft5YGb97DuCK6Vdw4cQL+cmbP2Fv095Eh2MSZEQngKa2ADsPHWPR9JJEh0L5tFI27m8gFLIl+gb09x85Uz/MH+SavP2oba3lay9/jbklc7l94e2eXTcqGVmw6P+GXeuh9p343rsPIsI9F9xDtj+bL/31SzY2IE2N6ATw+t4jACyaHrslBKO1eEYpTe0B3rXuoP3b/4qz9u+FX/Cs738gFODLf/0yge4AP/zgD8n2ez+mYEDn3uYMaPv7D+J/7z6Mzx/Pf130X+xq2MV33vhOosMxCTCiE8A/Kg+Tm+ln4dTElwAuPGUM4MRk+qAKL/0X5JfB2SsGPj5K92+8n7fr3+aeC+5hWtE0z647KLklThLY+kTSlAIALp58MZ867VM8tusxnqx8MtHhmDgbsQlAVXlxRx0XzBpNVkbi/5rji3OYO66Ql3bUJTqU5LX9Kdj3d/jAVyDLm0b7Ve+s4qFtD3HTqTdx5YwrPbnmkF3wOScR/OkuJ9kliTsW3sHi8Yu555V7+Ft1YtcyNvGV+G/GGHm3toWqo+1cOm9sokM57tJ5Y3lj71Ea22w8wEm6WuHPX4exC+CcT3pyyTU71nD/xvu5cvqVfGXRVzy55rDklsClX3eS3NbHEx3NcRm+DH7yoZ8wp3QOd/7lTjbUbkh0SCZORmwCeOrtg/gElswfn+hQjrvqtAkEQ8of36lNdCjJZ/3XobEKrvq+JwO/1u5cy7df+zaXTL6E+y6+D7/P70GQHjjnkzBxITzzZWhJnv8HBVkF/PzDP2dywWTueOEO3qh5I9EhmTgYkQkg2B3iD28d4JI5ZZQVJqDBrw+nTSrilLEFPFpRNfDB6WTbOtj4K6eKZPqFw7pUMBTkexu+x3+++p9cMPECfviBH5Lpy/QoUA/4/PDR/4FAO/zh0wlfOD5SSU4JK5esZHz+eD793Kd5dOejiQ7JxNiITADrtx6ipqmDf148NdGhnEBEuGnxVN58r5FNVTYmAIADG+Hx22BSOVz6jWFd6nD7Ye548Q4e2vYQn5j3CX562U/JycjxKFAPjZntlHT2/AX++P8kVXvA2Lyx/Paq33LexPO499V7+fZr36Y92J7osEyMjLgE0B1S/v8XdjFzTD6XzRuX6HBO8vHyyRTnZvLj53cmOpTEq94ID30UCsrgpt8PudtnSEM8tvMxlj2xjDdq3uDu8+/mrsV3keGL//TfUTv7Frjwi86I5/VfS5ppIsAZKfzTS3/KivkreGTHI1z75LW8cuCVRIdlYmDEJYAHX9nHjkMtfGnJXPy+pFqXHoDCnExu/9As/rKjnvVbk6cOOO62rIUH/wlyR8GKp6Bg8I31IQ3x16q/cssfb+GeV+9hbslcHlv2GNfPOWlm8uR02TfhvH+D134Ga291pr5OEn6fny8v+jKrrlhFpi+TTz//ab7w4hfYUr8l0aEZD4kmUfEToLy8XCsqKoZ0bsW+o/zz/7zOxbPH8MsV5Qmd/qE/ge4Qy3/6Dw40tvP4v12QkJXKEqbxPfjzf8C2J5y5fj7+IBQOrqH+cPthntv/HGt2rKGysZIJ+RP47Jmf5ZpTrknaf/M+qcIr/w3P3wNFE2HJt2D+Nc5U0kmiq7uLVe+s4jfbfkNLVwuLxi/io7M/ygcnf5CCrDT6v5vkRGSjqpYP6pxoEoCILAV+AviBX6rqd3rszwZ+A5wDHAFuUNV97r6vAp8CuoHPq+r6/u411ASwu/4Yy/77ZcYW5fD4Zy+gJD9r0NeIp/1HWvnoz17B5xP+/MVLkj7eYekOOF0fN/3e6f7oy4SL74SL7oyqx097sJ13Dr/DxkMbeb3mdTYe2oiizC2Zy4oFK1g6Y2lyNfQORdUGePqLcOgdGDsfyv/FmQpjCCWjWGkNtLJ251p+t/131LTWkOXL4oKJF1A+vpxzxp3D3NK5qf/vkMJikgDcNX13ApcD1ThrBN+kqtsijvk34AxV/YyI3Ahcq6o3iMh84PfAYmAi8DwwR1X7nIN2qAkgFFJ+/MIuPnHuVMYVJWHDXy92HWrhLzvq+ddLZiY6lOHrDkBnizOP/7Fa50n/8C6oeRuqXoeuY85UCGffAuffTqhoIh3BDtqCbbQH2mkONNPU2URDRwN1bXUcajtEVUsVexr3cODYARRFEGaXzOayqZexZNoSTik5JdF/a291B+GdtfDqT6HWrWoZdxpMOgfGzoOSGU4pIX8M5BRDZl5CSgohDbG5fjPr963nL1V/ofpYNQAZksHUoqnMGjWLifkTGZc/jrF5YxmVPYri7GIKswrJzcglLyOPbH926pXWklysEsD5wD2qeoX7/qsAqvr/RRyz3j3mVRHJAGqBMty1gcPHRh7X1/2GUwWUjD7+1Mfp6k6ygV9NVc4Xdg+9/0/QXt6p23FF3R4sih7fAyFAEVSg25dJyJ9Jtz+ToM9PIBQg0B0gqP13fyzILGBiwURmFs9kZvFMFoxZwJllZ8Z+AZdkcWgr7PyTs5pYzSZob+jlIHGSQEY2+LOcLqbiB58PxOfsF3H+hBNf90YEPvJjmHb+oEKta6vjzbo3effIu+xp2sPepr3UtNbQ2d3Z73mZvkyy/Fn4xU+GL4MMyUBE8IsfEcEnPgRBRJBe4g4nkN72nXDcAPuTyZySOXzvA98b0rlDSQDRdJOYBER2XK8Gzu3rGFUNikgTMNrd/lqPcyf1vIGI3AbcBjB1anJ13RyuGcUzCIRO/rJNqI5WCPQ+++OJD2XSy073l1GcX0znC8eP+DKcqh1/Fr7MXCQrH192ET6fH7/4j/+S+31+sv3ZZPmyyMnIITcjl9yMXIqzi52frGLG5o21uuVxC5yfi7/kJNnWw86aAi010HYYOpqdEleww/npDkAoCKFuQN//8/gDng7Q3dTdl5U/6FDH5o1l6fSlLJ2+9P2rqdLc1cyhtkM0dTbR3NlMc1cz7cF22oPtdHZ30tXdRWd3J93aTXeom6AGCWno+I+ihDR0PDSNeBg5/sAxwAOs9vFYk6wmFZ709RhTSdFPTlVXAivBKQEkOBxPffeS7yY6BJPqRJyusgUxWrw+BkTkeFI3ySuabqAHgCkR7ye723o9xq0CKsZpDI7mXGOMMQkQTQLYAMwWkRkikgXcCKzrccw6IDx/73XAi+qUzdYBN4pItojMAGYDNsmIMcYkgQGrgNw6/TuA9TjdQFep6lYRuReoUNV1wP8CD4lIJXAUJ0ngHrcG2AYEgdv76wFkjDEmfkbUQDBjjElXQ+kFNOKmgjDGGBMdSwDGGJOmLAEYY0yasgRgjDFpKukagUWkHtif6Dh6MQY4nOggUoB9TtGxzyl69llFZ66qFg7mhKQYCRxJVZNyuKOIVAy2hT0d2ecUHfucomefVXREZNDdJ60KyBhj0pQlAGOMSVOWAKK3MtEBpAj7nKJjn1P07LOKzqA/p6RrBDbGGBMfVgIwxpg0ZQnAGGPSlCWAAYjIUhHZISKVInJXouNJJiKySkTqROSdiG2lIvKciOxy/yxJZIzJQESmiMhLIrJNRLaKyBfc7fZZRRCRHBF5Q0Tedj+n/3S3zxCR193fwUfcaenTnoj4ReQtEXnafT/oz8kSQD9ExA88AFwJzAduche6N45fA0t7bLsLeEFVZwMvuO/TXRD4kqrOB84Dbnf/H9lndaJO4FJVPRM4C1gqIucB3wXuV9VTgAbgUwmMMZl8Adge8X7Qn5MlgP4tBipVdY+qdgGrgeUJjilpqOrfcNZ/iLQceNB9/SBwTVyDSkKqWqOqb7qvW3B+aSdhn9UJ1HHMfZvp/ihwKbDW3Z72nxOAiEwGrgZ+6b4XhvA5WQLo3/HF7l29LmpvTjBOVWvc17XAuEQGk2xEZDqwEHgd+6xO4lZrbALqgOeA3UCjqgbdQ+x30PFj4P8FQu770Qzhc7IEYGLGXRbU+hm7RKQAeAz4oqo2R+6zz8qhqt2qehbO+uGLgVMTHFLSEZGPAHWqunG410q6uYCSjC1qP3iHRGSCqtaIyAScJ7m0JyKZOF/+v1PVx93N9ln1QVUbReQl4HxglIhkuE+39jsIFwLLROQqIAcoAn7CED4nKwH0bwMw221dz8JZ63hdgmNKduuAFe7rFcCTCYwlKbj1s/8LbFfVH0Xsss8qgoiUicgo93UucDlOe8lLwHXuYWn/OanqV1V1sqpOx/lOelFVP8EQPicbCTwAN8v+GPADq1T1vgSHlDRE5PfAB3Gm6z0EfBN4AlgDTMWZ1vvjqtqzoTitiMhFwN+BLbxfZ/s1nHYA+6xcInIGTuOlH+fhdI2q3isiM3E6YJQCbwE3q2pn4iJNHiLyQeDLqvqRoXxOlgCMMSZNWRWQMcakKUsAxhiTpiwBGGNMmrIEYIwxacoSgDHGpClLAMYYk6YsARhjTJr6Pxq5yvE64dwlAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "34dXRibnDbY9"
      },
      "source": [
        "# After uncommenting below lines, please run your implementation and observe it\n",
        "\n",
        "D_pq = kl_divergence(p, q)\n",
        "D_qp = kl_divergence(q, p)\n",
        "D_py = kl_divergence(p, y)\n",
        "D_yq = kl_divergence(y, q)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yEBIeabTBSyM",
        "outputId": "ad289a11-1bdc-4bc3-e162-ca2b4d56ac31"
      },
      "source": [
        "D_pq, D_qp, D_py, D_yq"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(388.1094152035839, 7.45238393725293, 565.8057688445564, 6.381930027732736)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 122
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1vyn4A0RNCYN"
      },
      "source": [
        "# Q4: *PixelCNN*\n",
        "In this part we are going to implement the masking and the generation parts for a very simplified version of the PixelCNN."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QS0059yeNgID"
      },
      "source": [
        "The model consist of one convolutional layer and fully-connected layer. Assume that inputs are 28x28 grayscale (0 to 255) images.\n",
        "\n",
        "You should first apply a masking strategy for 7x7 kernels.\n",
        "\n",
        "Then you should autoregressively (pixel by pixel) generate the image by passing each pixel to network."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "trIi1e_ZktyJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "43707834-b2af-461f-fa81-1a3754b6078d"
      },
      "source": [
        "image_size = 28\n",
        "KERNEL_SIZE = 7\n",
        "FILTER_NUM = 64\n",
        "STRIDE = 1\n",
        "PADDING = KERNEL_SIZE//2\n",
        "OUT = 256\n",
        "\n",
        "sample_img = np.zeros([image_size, image_size])\n",
        "\n",
        "filters = np.random.randn(KERNEL_SIZE, KERNEL_SIZE, FILTER_NUM)\n",
        "bias = np.random.randn(FILTER_NUM)\n",
        "fully_connected = np.random.randn(FILTER_NUM, OUT)\n",
        "\n",
        "############## SOLUTION ##############\n",
        "\n",
        "\n",
        "########## masking ###########\n",
        "filters[KERNEL_SIZE//2:,KERNEL_SIZE//2:, :] = 0\n",
        "filters[KERNEL_SIZE//2+1:,:,:] = 0\n",
        "####### end of masking #######\n",
        "\n",
        "\n",
        "\n",
        "######### generation ##########\n",
        "\n",
        "img_padded = np.pad(sample_img, KERNEL_SIZE//2, mode='constant')\n",
        "\n",
        "for i in range(image_size):\n",
        "  for j in range(image_size):\n",
        "    img_slice = img_padded[i:i+KERNEL_SIZE, j:j+KERNEL_SIZE]\n",
        "    feats = np.tensordot(img_slice, filters) + bias # [64]\n",
        "    outs = np.matmul(feats, fully_connected) # [256]\n",
        "    probs = outs/np.sum(outs) # 256\n",
        "    img_padded[(KERNEL_SIZE//2) + i, (KERNEL_SIZE//2) + j] = np.argmax(probs)\n",
        "\n",
        "img = img_padded[KERNEL_SIZE//2:-(KERNEL_SIZE//2),KERNEL_SIZE//2:-(KERNEL_SIZE//2)]\n",
        "###### end of generation ######\n",
        "\n",
        "########### END OF SOLUTION ###########\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(28, 28)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 111
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "hW-TpEWB1uRj",
        "outputId": "a3a31d7f-1475-407f-e3f0-3ac1f2711a51"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "\n",
        "plt.imshow(img)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.image.AxesImage at 0x7f363a6fb5d0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 112
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAUjklEQVR4nO3df3CV5ZUH8O8BEmiiYmIAYwgIFVmpKdgNLE4p0lZBkA66s+Po2A7OMBu12qFdZlpx/5A/dip1RNs6tmy6MuKW2rEtLo6l9dc6Rmddx+gCIaKCbARC5IcBEcKvJGf/yMWJmueceN9733vX5/uZYZLcc5/7PnmTw725532eI6oKIvriG1LoCRBROpjsRJFgshNFgslOFAkmO1EkhqV5sJLh5Tq8rCJ8h1Hd6U2G6Avo5L4jOP1hlwwUS5TsInI1gF8AGArg31R1pXX/4WUVmDZnaTDee9vBJNMhit6WO9YGY1m/jBeRoQAeAjAfwBQAN4rIlGwfj4jyK8nf7DMA7FDVnap6CsDvASzKzbSIKNeSJHsNgN39vt6Tue0TRKRBRJpFpLn75LEEhyOiJPL+bryqNqpqvarWDxtenu/DEVFAkmRvB1Db7+uxmduIqAglSfbXAEwSkQkiUgrgBgBP5mZaRJRrWZfeVLVbRO4A8DT6Sm9rVLXVGnP+2E78032/C8bvbv2OeczFF72axUz7TBnBFx1UPH706JJgrHJbT/YPfLAkGEpUZ1fVjQA2JnkMIkoHL5cligSTnSgSTHaiSDDZiSLBZCeKBJOdKBKprmcXKEbI6WD8Z5euT3E2RGH/cufNZnzoiWS7Mk9o2hqMXfxClzl2+7XnB2NDj50KxvjMThQJJjtRJJjsRJFgshNFgslOFAkmO1EkJM3GjucMqdSZw+YF45Uvnm2Ov/zcncHY6t9eY46tauE21ZSevbOHmvHSQwPu9gwAODGm1xw7Yl/4Obptzf043rF7wAfnMztRJJjsRJFgshNFgslOFAkmO1EkmOxEkWCyE0Ui1SWu507pxjV/OBCM/2X/aHO8VUvPdx29fNmeYOy9Zy40x46f22bG8zm+kMf2xic9dj55c/OUHsp+7PTp75jx9lWTgrHdJ8Lj+MxOFAkmO1EkmOxEkWCyE0WCyU4UCSY7USSY7ESRSLXOfvBkOR7Z+XfB+M0T7ZbMD351VDBWPjdcvweAjZPtZrNfefD7ZrzcjFI2Wn/wKzPu/UyS6JoY3tIcAMqc8d51HYtWPmfGV7d8Ixh77bWLzbEXrf/vYGyIHgvGEiW7iLQB+AhAD4BuVa1P8nhElD+5eGb/pqoezMHjEFEe8W92okgkTXYF8IyIvC4iDQPdQUQaRKRZRJq7P7Tb2hBR/iR9GT9LVdtFZDSAZ0XkLVVt6n8HVW0E0AgAZZOq09vdkog+IdEzu6q2Zz7uB/AEgBm5mBQR5V7WyS4i5SJy9pnPAcwFEG5NSUQFleRl/BgAT4jImcf5nar+1RrQc2IYDm+vDMZPT7D32r617qVg7OF1V5tj5/UuNOOeD0+OCMamLtxmjm3eNc6MVzs122MtY834kpXh0+6dF69enOTYALDhziuDsSsw4Ns8H6tCHvcoaAnv2w4A5/14uxn/oGWCGV+/4iozPuK7R4KxY/iSOfaS18Npu+Wm8Lisk11VdwKYmu14IkoXS29EkWCyE0WCyU4UCSY7USSY7ESRSLVl89nnjtVps5cG49Z2zYC9vW/SraRrltullvZ7wtv3el5sbDTjf7/DLtO8/Rf72Nb3frDOLrjkewtu6/inKuzfvbH/ac9t/2UlWc0JSL5FtifJElivXGq1dN57389xchdbNhNFjclOFAkmO1EkmOxEkWCyE0WCyU4UCSY7USRS3Uq6Z1QPjt5yOBgv5HbNm5+6xIxPNpY8fnCvvdzxigZ7KadXC0+y5bJbT8aFZvypW+814/P+63YzftH52be69uroSWrlx1bZS3e95bXeVtFerfzpfVOCseM1PebYoV3Gc7Rx6QKf2YkiwWQnigSTnSgSTHaiSDDZiSLBZCeKBJOdKBKp1tk980e3mvHVRk3Yq1V7a6cnTW8z4+aa8jpzqFsPrik9bsa91sXW2umkW0EvXP1jM+5tg33u8vD3duVN9rG9WvWRh2rN+JIV9uNbrC2wAX+r6KoT9nmpWBhuhaZD7N/V3lIjbuyQzWd2okgw2YkiwWQnigSTnSgSTHaiSDDZiSLBZCeKRKp19qrhx3DzxFeD8VcOT8z6sSu2O2uAT9i1y2NNdj166vJwW+b6ke+ZY716sT3al2QPcq+e7F1D4K3rth6/HfZ++NYeAgDwze++Y8b/+JN5wdiuRfbvQ5lz3Yb3++b1Ieg1CuJlu+1jW/vGW9xndhFZIyL7RWRrv9sqReRZEdme+ViR1dGJKDWDeRn/CIBPPz3cCeB5VZ0E4PnM10RUxNxkV9UmAJ2funkRgLWZz9cCuDbH8yKiHMv2DboxqtqR+fx9AGNCdxSRBhFpFpHmo52nszwcESWV+N147esMGXy3Q1UbVbVeVevPqsy+ER8RJZNtsu8TkWoAyHzcn7spEVE+ZJvsTwJYnPl8MYANuZkOEeWL259dRB4DMAdAFYB9AO4G8B8AHgcwDn1l4utV9dNv4n2G15/9H372tDl+9W+v8Q4RtCTh2ul89jFP2hu+Z0S4ZvvyL//VHOutle+aaL/PUrazeHukW3sYSG14PTkA3Fr3khl3r09wVC/fEYy1r7J/3ntnh3/eVn9296IaVb0xEPq2N5aIigcvlyWKBJOdKBJMdqJIMNmJIsFkJ4pEUW0l7S1xnbowvMzUa7nslUq8Fr17Zw8Nxm5bYJcMvbLeQWfuSbaq9tpFu993hf0rkqSk6bVNHr+szYxfOeqtrI9d1TTcHPvgIrvY5C2BTVKq7Zwc/l3rk6clrkT0xcBkJ4oEk50oEkx2okgw2YkiwWQnigSTnSgS7hLXXKq9dKQufXxmMP7cgb8xx1tLHr2WzB6vVm7V6a0lpoC/zPTiFxeb8ZIt5Wbcqul6razzvfTXWr7rXRuRdFmx9b17vy8XNNlbRXtbaHvnrW5h+BqBlqfsPLC2kraWuPKZnSgSTHaiSDDZiSLBZCeKBJOdKBJMdqJIMNmJIpHqenaBokTC9cv5o1vN8a8sPB6MNe8aZ46tXmevX374kF0XtdoHb22/wBzrrSmvNqPAeU7r4lk3vRuMeev4vXpw6w9+Zca9awRgbIOdZA8BAJg03W52fdC4LqP0kH1thMe7JqQY8ZmdKBJMdqJIMNmJIsFkJ4oEk50oEkx2okgw2YkikWqd/fCbw/DnqaPCdxD7/57j88I12xJn3bZXq8a9E8zwBy3huFcnz/eacmtuXjvoatj15hv+91tm3Ftrv3d2eN140v32vX3nvf32kzjyUK0ZrzrhrMVfmMPJDJL7zC4ia0Rkv4hs7XfbChFpF5FNmX8L8jtNIkpqMC/jHwEw0H+xD6jqtMy/jbmdFhHlmpvsqtoEoDOFuRBRHiV5g+4OEdmSeZlfEbqTiDSISLOINJ/WkwkOR0RJZJvsvwbwZQDTAHQAWBW6o6o2qmq9qtaXiL0YhYjyJ6tkV9V9qtqjqr0AfgNgRm6nRUS5llWyi0j/atN1ALaG7ktExcGts4vIYwDmAKgSkT0A7gYwR0SmAVAAbQBuGczB9JwyHJ/9tWDcqwkfPrUnGLPWLgPAB04d3WPNrX6kva4632vK7+8M97VPuu+7Z8lK+xoB63vf0GSfF69OXr18h32Hey4Khry9/j1DTyTrU3Dk1IhE47PhJruq3jjAzQ/nYS5ElEe8XJYoEkx2okgw2YkiwWQnigSTnSgS6W4lfaQLX/rrG8H44WWjzfFWy2aPV9bzymdWCWsz7NbDU5dvM+OLnGN7y0y9Fr/msZ3Ww17Z0NtSOVnbZLss2NESLq0lPXbSraY91aUn8vr4A+EzO1EkmOxEkWCyE0WCyU4UCSY7USSY7ESRYLITRSLVOnvvyDJ0zQkvcR1batfCrzS2XPaWcu564GIzvnmSXSufPD88t1mV4ZbJgD+39pbwFtmDYdXx242WyYBfR3+xsdGMey2bS4zYBU3h9t2A37JZarvMePU6M5yId32C9zMvBD6zE0WCyU4UCSY7USSY7ESRYLITRYLJThQJJjtRJFKtsw/58DjKNm4Kxtt7ppnjN9fZtXDLvhn2/2u3LUiwJTLsbapbG+2toL1adfU6u5POK2+G13WXOe2iPVc0NJhxr111jXENgLcPgLeeHbDPi7WePWmb7PUrrjLj/y9bNhPRFwOTnSgSTHaiSDDZiSLBZCeKBJOdKBJMdqJIiGqy1rOfR+Ulo3TumuuC8cvP3WmOf3DLnKyP7dWqPda+8827xiU6tlUPBvyWzV958PtmPJ+PPX5umxlPste/t7f7bQueNuPWtRFey2avJfOBafbPbNp8u1dAL8LH9/oAnBjTG4ztve/nOLlr94AP7j6zi0itiLwgIm+KSKuILM3cXikiz4rI9szHCu+xiKhwBvMyvhvAMlWdAmAmgNtFZAqAOwE8r6qTADyf+ZqIipSb7KraoapvZD7/CMA2ADUAFgFYm7nbWgDX5muSRJTc53qDTkQuBHAZgFcBjFHVjkzofQBjAmMaRKRZRJpPHkq/vxUR9Rl0sovIWQD+BOCHqnqkf0z73uUb8B0NVW1U1XpVrR9eMSLRZIkoe4NKdhEpQV+ir1PV9Zmb94lIdSZeDWB/fqZIRLngrn8UEQHwMIBtqnp/v9CTABYDWJn5uMF7rO63e9B5xUfhO2y2x5dsKfcOEWQttQTsUggAbH4qvBzT2i4ZAMqXtZnxjvdHmXGv/FXVEl5O6ZWY7u+caMY923bUmPGSrx4Lxm6te8kc6y0z9bbBHtIdLp/V/MjettzT42xN3r7J3sK7evmOYKzybWeL7THZtZMezGLnrwP4HoAWETmzGP0u9CX54yKyBMB7AK7PagZElAo32VX1ZSD4tPft3E6HiPKFl8sSRYLJThQJJjtRJJjsRJFgshNFItUlruMuPUeX/WFGMP7K4exrvl5LZm/Josdq0bu65Rvm2KTLa70lsNa2yF6t2qrRA/bSXsC+/mAwj2/56S9Xm/FXu8JbaHu8Gn2+lx3XLXwrGCvYElci+mJgshNFgslOFAkmO1EkmOxEkWCyE0WCyU4UiVTr7OcMqdSZw+YF40ev/VtzfPu3wnMt2519LRrIb638xcZGM+6tKfdqwtaa9UOThppjrXov4Nd8PZPnh+v0syrfNcd637fHOu/eOfd+H2rWlprx3mH2mnNrPTvr7ESUCJOdKBJMdqJIMNmJIsFkJ4oEk50oEkx2okikWmcfPrFGx/40vM53+vj3sn7sfK6rBuz17J58r522asb5Xmtv1YuBZHX6qQvtvf497ffYe7cnsWuBHb98qr0PQOsfw7+v3U7jJNbZicjEZCeKBJOdKBJMdqJIMNmJIsFkJ4oEk50oEm6dXURqATwKYAwABdCoqr8QkRUA/hHAgcxd71LVjdZj1V46Upc+PjMY9/Y4t5yqsL+Pupl2PTjJ2mqvTu7Vi+tH2tcXJFnXnXQtvfczGf0/pz/3nM7w1nx7vPPeVRu+tsKrgye9bsObm7WPQPsq+/qAvbPD582qsw+mP3s3gGWq+oaInA3gdRF5NhN7QFXvG8RjEFGBDaY/eweAjsznH4nINgA1+Z4YEeXW5/qbXUQuBHAZgFczN90hIltEZI2IVATGNIhIs4g0H+08lWiyRJS9QSe7iJwF4E8AfqiqRwD8GsCXAUxD3zP/qoHGqWqjqtarav1Zlfa+XUSUP4NKdhEpQV+ir1PV9QCgqvtUtUdVewH8BkC4YyMRFZyb7CIiAB4GsE1V7+93e3W/u10HYGvup0dEuTKYd+O/DuB7AFpEZFPmtrsA3Cgi09BXjmsDcIv3QAJFifQE40mWNHotmz9ommDGN8COW62LFyUsnW2us8s8U5dnX7q7oqHBHOtpbUzWmthS3mGXSw9NscdL7TEzPs5Yvrt5t33OPd6SZ29psaVzsr39NxBe4moZzLvxLwMYqG5n1tSJqLjwCjqiSDDZiSLBZCeKBJOdKBJMdqJIMNmJIjGYOntqvGWFJmfXYK+G37xrnP0AxrbE7c7BrRo9AFQPWNns9/jOlshWnf7UbGcJc22XGffq9FWwl3paS2y9Gn3pITOM8dMPmPH36i4MxpIuO/aW/lZ7W5cvt8P5wGd2okgw2YkiwWQnigSTnSgSTHaiSDDZiSLBZCeKRKotm0XkAID+BcwqAAdTm8DnU6xzK9Z5AZxbtnI5t/GqOmqgQKrJ/pmDizSran3BJmAo1rkV67wAzi1bac2NL+OJIsFkJ4pEoZPd7k1UWMU6t2KdF8C5ZSuVuRX0b3YiSk+hn9mJKCVMdqJIFCTZReRqEXlbRHaIyJ2FmEOIiLSJSIuIbBKR5gLPZY2I7BeRrf1uqxSRZ0Vke+bjgD32CjS3FSLSnjl3m0RkQYHmVisiL4jImyLSKiJLM7cX9NwZ80rlvKX+N7uIDAXwDoCrAOwB8BqAG1X1zVQnEiAibQDqVbXgF2CIyGwARwE8qqqXZm67F0Cnqq7M/EdZoao/KZK5rQBwtNBtvDPdiqr7txkHcC2Am1HAc2fM63qkcN4K8cw+A8AOVd2pqqcA/B7AogLMo+ipahOAzk/dvAjA2szna9H3y5K6wNyKgqp2qOobmc8/AnCmzXhBz50xr1QUItlrAOzu9/UeFFe/dwXwjIi8LiLJeiflxxhV7ch8/j6AMYWczADcNt5p+lSb8aI5d9m0P0+Kb9B91ixV/RqA+QBuz7xcLUra9zdYMdVOB9XGOy0DtBn/WCHPXbbtz5MqRLK3A6jt9/XYzG1FQVXbMx/3A3gCxdeKet+ZDrqZj/sLPJ+PFVMb74HajKMIzl0h258XItlfAzBJRCaISCmAGwA8WYB5fIaIlGfeOIGIlAOYi+JrRf0kgMWZzxcD2FDAuXxCsbTxDrUZR4HPXcHbn6tq6v8ALEDfO/LvAvjnQswhMK+JADZn/rUWem4AHkPfy7rT6HtvYwmA8wA8D2A7gOcAVBbR3P4dQAuALehLrOoCzW0W+l6ibwGwKfNvQaHPnTGvVM4bL5cligTfoCOKBJOdKBJMdqJIMNmJIsFkJ4oEk50oEkx2okj8HwxV+gwz3fIjAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZzQGJzTx9YPJ"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}