Gender Detection From Bangla Handwritten Images using Logistic Regression.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "160204099_SC_Problem 2.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1TAb1RyLITwX"
      },
      "source": [
        "#**Problem 2 Male and Female Handwriting Digit**\r\n",
        " \r\n",
        "Handwriting is a unique quality of human being, every person has unique handwriting that is very hard to copy but there can be some similarities\r\n",
        "Between male and female handwriting. we will classify the difference between Male and Female Handwritting Digit using logistic regression.\r\n",
        " \r\n",
        "Differences between Male and Female Handwriting Digit can be used in various applications like Archeology, forensic department. In this assignment, our main goal will be to achieve a good result using logistic regression and tuning hyperparameters correctly to get a better result.\r\n",
        " \r\n",
        " \r\n",
        " \r\n",
        " \r\n",
        " \r\n",
        "#**Dataset**\r\n",
        " \r\n",
        " \r\n",
        "> For this experiment, we will use the dataset [Ekush](https://shahariarrabby.github.io/ekush/#download) which is available in **Github**. \r\n",
        "There will be a total of **30830** images, and it was split in a **90:10** ratio. **90%** (**27747**) of data is used in training and **10%** (**3083**) was used in testing.Whole dataset was shuffled before spliting , so that traning and testing dataset can have both label data. \r\n",
        " \r\n",
        "**Snapshot from Dataset**\r\n",
        "\r\n",
        "><div align=\"center\">\r\n",
        "<img src=\"https://drive.google.com/uc?id=1GhuJfYchOjfNoBtwkpWivPRQinegMYwy\" width=\"600\">\r\n",
        "</div>\r\n",
        " \r\n",
        " \r\n",
        "#**Experimental Setup**\r\n",
        " \r\n",
        " \r\n",
        "> All of these experiments were performed using Google Colab free GPU, Models were created in PyTorch. \r\n",
        " \r\n",
        " \r\n",
        "During the whole experiment,\r\n",
        "* The height and width of the input was **28*28 =784** \r\n",
        "* Output dimension was **(0,1)=2**\r\n",
        "* Each batch size was **256**\r\n",
        "* The number of iteration was **4000**\r\n",
        "* **Softmax** activation function was used \r\n",
        "* GPU **Tesla T4** was available \r\n",
        "* Number of Epochs **332**\r\n",
        " \r\n",
        "- **totaldata:** 30830\r\n",
        " \r\n",
        " \r\n",
        " \r\n",
        " \r\n",
        "> - **minibatch:** 256\r\n",
        "- **iterations:** 4,000\r\n",
        "- **epochs**\r\n",
        "  - $epochs = iterations \\div \\frac{totaldata}{minibatch} = 4000 \\div \\frac{30830}{256} = 332.14 == 332 $\r\n",
        " \r\n",
        "We will use different optimizer(SGD and Adam) and learning rate to achieve better performance\r\n",
        " \r\n",
        "# **Result**\r\n",
        " \r\n",
        " \r\n",
        "| Experiment Number      | Optimizer     | Learning Rate    | | Accurecy of last 500 iterations    |\r\n",
        "| :------------- | :----------: | :-----------: || :-----------: |\r\n",
        "|  1 |SGD   | 0.0001    ||51.41 |\r\n",
        "|  2 |SGD   | 0.0005    ||  50.02 |\r\n",
        "| 3   | SGD | 0.0010 || 49.92 |\r\n",
        "|  4 |Adam   | 0.0001    ||51.05 |\r\n",
        "|  5 |Adam  | 0.02   || 50.86 |\r\n",
        "|  6 |Adam   | 0.05    || 48.3 | |\r\n",
        " \r\n",
        " \r\n",
        "From the result, it is can be said that when **SGD** optimizer is used for the first setup accuracy was **51.41** and loss was **increasing**, and when we set the learning rate 0.0005 accuracy was **50.02** which was less than the last setup, and then when the learning rate was increased to **0.0010** accuracy decreased to **49.92** and loss graph was **decreasing**, so it can be assume that optimal learning rate for this setup is between **0.0001-0.0005**.\r\n",
        " \r\n",
        " \r\n",
        "And when **Adam** optimizer is used for the first setup accuracy was **51.05** and loss was **increasing in the last 500 iterations**, and when we set the learning rate **0.02** accuracy was **50.86** which was better than the last setup, and then when the learning rate was increased to **0.05** accuracy decreased to **48.3** and loss graph was increasing **in last 500 iterations**, so it can be assume that optimal learning rate for this setup is between **0.0001-0.02**.\r\n",
        "\r\n",
        "\r\n",
        "\r\n",
        "\r\n",
        " \r\n",
        " \r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4gxns8PIZakG"
      },
      "source": [
        "**Importing All Important Library**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8fKb5x6UbBVY",
        "outputId": "8f78ac1c-8ae3-4c4d-e7ec-753c82315f62"
      },
      "source": [
        "import torch\r\n",
        "import torch.nn as nn\r\n",
        "import torch.nn.functional as F\r\n",
        "import torchvision\r\n",
        "import torchvision.transforms as transforms\r\n",
        "import pandas as pd \r\n",
        "import numpy as np\r\n",
        "import matplotlib.pyplot as plt \r\n",
        "from sklearn.model_selection import train_test_split\r\n",
        "from google.colab import drive\r\n",
        "from sklearn.utils import shuffle\r\n",
        "drive.mount('/drive')\r\n",
        "print(torch.cuda.get_device_name())\r\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mounted at /drive\n",
            "Tesla T4\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vW9huJK_adv9"
      },
      "source": [
        "**Reading CSV**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "75q9YEJ2cKiQ"
      },
      "source": [
        "female_df=pd.read_csv(\"/drive/MyDrive/MaleFemaleDatsetCsv/femaleDigits.csv\")\r\n",
        "male_df=pd.read_csv(\"/drive/MyDrive/MaleFemaleDatsetCsv/maleDigits.csv\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DJPo468KBYNP"
      },
      "source": [
        "**setting label 1 for female and 0 for male**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "96UJP4GmcjLN"
      },
      "source": [
        "female_df['label']=1\r\n",
        "male_df['label']=0"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-7Xbmr88-76P"
      },
      "source": [
        "**Merging, Normalize and Reshaping  CSV**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "22HnvcGnebFV",
        "outputId": "ab36b439-4128-414e-c33c-329c470733ca"
      },
      "source": [
        "df_name=[female_df,male_df]\r\n",
        "dataset_image=pd.concat(df_name,ignore_index=True)\r\n",
        "dataset_image= shuffle(dataset_image)\r\n",
        "dataset_label=dataset_image['label']\r\n",
        "print(dataset_image.shape)\r\n",
        "dataset_image=dataset_image.drop(labels='label',axis=1)\r\n",
        "print(dataset_image.shape)\r\n",
        "dataset_image=dataset_image/255.0\r\n",
        "dataset_image=dataset_image.values.reshape(-1,28,28,1)\r\n",
        "print(dataset_image.shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(30830, 785)\n",
            "(30830, 784)\n",
            "(30830, 28, 28, 1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JOuuB6W9azKd"
      },
      "source": [
        "**Displaying Image and Label**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        },
        "id": "jeSRUqB6fBt9",
        "outputId": "5a2eeac5-3c60-483e-98ad-a7ada26ec4ae"
      },
      "source": [
        "import matplotlib.pyplot as plt \r\n",
        "import numpy as np\r\n",
        "\r\n",
        "show_img = dataset_image[0].reshape(28, 28)\r\n",
        "plt.imshow(show_img, cmap='gray')\r\n",
        "print(dataset_label[0])"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "1\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAASaklEQVR4nO3dfWxVVboG8OcFWouAkS+bKiADaQJIvGVsQCO5cjN2FDXCaELA5KYqsRMzJGMkcYzGwJ8odxyJEpLOFWFu5jKOmSEi6ly4hIRLIihoEVCrXIRABTqKgEiQFt75o5tJ1e53Hc8++0Pe55c0bc/bdc5i9zzs07P2WktUFUR06euXdweIKBsMO5ETDDuREww7kRMMO5ETA7J8MBHhW/9EKVNV6ev2RGEXkdsBLAPQH8B/quqSEtokecjUcAiyPP362S8OL1y4UPZ9J32u8Hf6bWW/jBeR/gCWA5gJYBKAeSIyqVIdI6LKSvI3+1QA+1R1v6qeA/AnALMq0y0iqrQkYb8GwKFe3x+ObvsWEWkRkR0isiPBYxFRQqm/QaeqrQBaAb5BR5SnJGf2DgCje30/KrqNiAooSdjfAVAvIj8RkWoAcwGsq0y3iKjSyn4Zr6rdIrIAwP+gZ+htparurVjP+n7M2FpomIbDMOXp37+/WT9//rxZb2pqiq1Nnz7dbLto0SKzHhr24+/82xL9za6qbwB4o0J9IaIU8XJZIicYdiInGHYiJxh2IicYdiInGHYiJyTLsUgRUWtsNM2x8tB9h6Zieh2zraqqMutdXV1m/eWXX46tzZw502w7dOhQsx4a409Tka/riJvPzjM7kRMMO5ETDDuREww7kRMMO5ETDDuRE5kuJQ0kW200TaGpnFa/L+VhudDQWui4TZoUvwbp559/brZNelyTrE4beuwf4++cZ3YiJxh2IicYdiInGHYiJxh2IicYdiInGHYiJzIdZ+/Xrx8uv/zy2PrixYvN9rW1teZ9W9ats5e0t6ZiAvZ4cp5TLZNKOlVz4MCBZn348OGxtWHDhplt6+vrzXp7e7tZt/5tRb3eI008sxM5wbATOcGwEznBsBM5wbATOcGwEznBsBM5kek4e01NjTm/eeHChak99r333mvWx40bZ9aXLl0aWyvyssJpC11jcObMmdhaXV2d2da6rgIAPv74Y7OeZD77pShR2EXkAICvAJwH0K2qjZXoFBFVXiXO7P+mqvaSI0SUO/7NTuRE0rArgA0islNEWvr6ARFpEZEdIrKju7s74cMRUbmSvoyfrqodInIVgI0i8pGqbun9A6raCqAVAAYNGnTpvlNFVHCJzuyq2hF97gSwFsDUSnSKiCqv7LCLyCARGXLxawA/B7CnUh0jospK8jK+FsDaaCxzAID/VtW/hRpZ47KhMdtdu3bF1tra2sy2U6faLzqam5vN+ooVK2JrJ06cMNteyuPwoXnh1joDoTXpQ8f1UlzbPU1lh11V9wP4lwr2hYhSxKE3IicYdiInGHYiJxh2IicYdiInMp3iWl1djdGjR8fWQ9v/rl+/Pra2aNEis21o2eLQMM3p06fNepL7zlPSvtXU1Jj1qqqq2Fro983LqyuLZ3YiJxh2IicYdiInGHYiJxh2IicYdiInGHYiJzIdZ+/u7sbx48fLbt/YGL94bWjL5iSPC6S7LHGaU2BD9x2qh6awTpw40axfffXVsbWTJ0+abb/55huzTj8Mz+xETjDsRE4w7EROMOxETjDsRE4w7EROMOxETmQ6zn7+/Png8sCWMWPGxNZCc6NDQuPNoWWuk0hzvnvovkPXJ4QMGGA/haz7//LLL822HR0dZfXpoiKvI5AHntmJnGDYiZxg2ImcYNiJnGDYiZxg2ImcYNiJnMh0nB2wx11Dc6dHjhwZWxsyZIjZNjSfPcm87yKP56a9XfTZs2fNurUt87XXXmu2bWhoMOvbtm0z69ZzLc3rJpJK63cWPLOLyEoR6RSRPb1uGyYiG0Xkk+jz0LIenYgyU8rL+FUAbv/ObY8D2KSq9QA2Rd8TUYEFw66qWwB89zXwLACro69XA5hd4X4RUYWV+zd7raoeib4+CqA27gdFpAVAC2Dv+0VE6Ur8brz2vFsQ+46BqraqaqOqNoYmTRBResoN+zERqQOA6HNn5bpERGkoN+zrADRHXzcDeLUy3SGitARfV4vIGgAzAIwQkcMAFgFYAuDPIjIfwEEAc0p9wLTWQE97L+80x9KTrklv9S3U76SPferUKbNurf0+ePBgs+3AgQPL6tNFaa71n6Yk13xY16oEw66q82JKPwu1JaLi4OWyRE4w7EROMOxETjDsRE4w7EROZHpJ29mzZ/HRRx/F1tvb283248ePj61dddVVZtvQEFFoSeUf61LSaQtdFZnk33bzzTeb9c2bN5v1PLfZTtI+NNW7XDyzEznBsBM5wbATOcGwEznBsBM5wbATOcGwEzmR6Ti7qppTHk+ePGm2r66ujq1NmDDBbLtv3z6z/mOdDplUaEw3dFxC10YcPHgwtnbdddeZba+44gqznmQqaOi6iqTXPoS2ELeu25g6darZ1rpm5MCBA7E1ntmJnGDYiZxg2ImcYNiJnGDYiZxg2ImcYNiJnCjUFi1vv/22WZ82bVpZNQBYv359WX3yLsl4MQDs3r07tjZ58mSz7ZQpU8x6aCzcuoYg7esqkmy7/Pzzz5ttretN7rvvvtgaz+xETjDsRE4w7EROMOxETjDsRE4w7EROMOxEThRqnP2LL74w69bYZWjd+JAf89rtaUoyXgwAHR0dsbWuri6z7Y033mjWGxoazHpbW1tszRqrBpKv3R76t1nXL4SeyyNGjIit1dTUxNaCZ3YRWSkinSKyp9dti0WkQ0Taoo87QvdDRPkq5WX8KgC393H771S1Ifp4o7LdIqJKC4ZdVbcAOJ5BX4goRUneoFsgIu9HL/OHxv2QiLSIyA4R2ZHgsYgooXLDvgLAeAANAI4A+G3cD6pqq6o2qmpjmY9FRBVQVthV9ZiqnlfVCwB+D8BeDpOIcldW2EWkrte3vwCwJ+5niagYguPsIrIGwAwAI0TkMIBFAGaISAMABXAAwC8r0ZnQuvGWpOPkee7PXmTd3d1mPXTcli9fHlv77LPPzLbPPvusWX/66afN+m233RZbO3funNk2qSuvvNKsW3PWx44da7Z95ZVXYmtWhoJhV9V5fdz8YqgdERULL5clcoJhJ3KCYSdygmEncoJhJ3KiUFNct2/fnncXLjlJp6iG6qGlpq0thF980R7Ueeqpp8x6U1OTWX/ttddia59++qnZNjTUOmbMGLNeV1dn1m+66abYmrWtOQCsWbMmtnb8ePw0Fp7ZiZxg2ImcYNiJnGDYiZxg2ImcYNiJnGDYiZzIfJzdGve1xmQB4MyZM7G10Pa+oamYSaZyJh2rTlPajx1acrmqqiq2durUKbPtCy+8YNZD4/B33XWXWS+q0JbNa9euLet+eWYncoJhJ3KCYSdygmEncoJhJ3KCYSdygmEnckKyHAMWEbXG2UNzr3ft2hVbmzRpktl23LhxZv3QoUNm3eqb12WmgfDvzJrvHjpuEydONOvNzc1m3RpnD10fEKqH5rMPGTLErFvLaIeuGbHmrKsqVLXPXwrP7EROMOxETjDsRE4w7EROMOxETjDsRE4w7EROZDrO3q9fPx0wIH4KfVdXl9nemue7YMECs+3SpUvN+mOPPWbWq6urY2uh7X9Dc+lDY7pJ135PU6hvSdpazxUgfNyHDx9edtvQOPrmzZvN+siRI836gw8+GFt76aWXzLbWGgHd3d24cOFCeePsIjJaRDaLyAcisldEfh3dPkxENorIJ9HnoaH7IqL8lPIyvhvAQlWdBOBGAL8SkUkAHgewSVXrAWyKvieiggqGXVWPqOq70ddfAfgQwDUAZgFYHf3YagCz0+okESX3g9agE5GxAKYA2A6gVlWPRKWjAGpj2rQAaCm/i0RUCSW/Gy8igwH8BcAjqvqtlQK15x2iPt8lUtVWVW1U1cYkb+YQUTIlhV1EqtAT9D+q6l+jm4+JSF1UrwPQmU4XiagSgkNv0nM6Xg3guKo+0uv2pQC+UNUlIvI4gGGqao5fiYgmmfJ4ww03xNbeeusts21oG9w777zTrG/ZsiW2Zg3LAeEhxdArntDQXBJFfrUV6ps1BAWEf+eWVatWmfXQ9NqtW7ea9RkzZsTWQv9uKyfWFNdS/ma/GcC/A9gtIm3RbU8AWALgzyIyH8BBAHNKuC8iykkw7Kq6FUDcfzU/q2x3iCgtvFyWyAmGncgJhp3ICYadyAmGnciJzJeStsbZQ+PJVl+3bdtmtp02bZpZ379/v1m/++67Y2t79+4129bU1Jj10Dh8ki2hk06vLeE6jLLr1nOhlMcObbNtTXF95plnzLbWFFQAOHjwoFmfPdueKmItix6a2lvC84VLSRN5xrATOcGwEznBsBM5wbATOcGwEznBsBM5Uagtm0PjrtY83qamJrPt+vXrzXpobvTOnTtja3Pm2LN7Q2P4obHwUN06pmnOhQ89dkhonDzkgQceMOuPPvpobG3y5Mlm2xMnTpj1e+65x6yHlpoOLQdtKeG6C46zE3nGsBM5wbATOcGwEznBsBM5wbATOcGwEzlRqHH2EGu8ObTm/MMPP2zWly9fbtatfre3t5ttn3zySbP++uuvm/WzZ8+a9STy3A761ltvNetz58416/Pnzy/7sU+fPm3WW1rsHcvWrFlj1kPXbVjXP4SOeQlrEHCcncgzhp3ICYadyAmGncgJhp3ICYadyAmGnciJUvZnHw3gDwBqASiAVlVdJiKLATwE4O/Rjz6hqm8E7ivRoG1oXrclNDY5c+ZMs75s2bLYWn19fVl9uii05v3KlSvNujX/+b333jPb7tmzx6xPmDDBrIfmhY8fPz62Frr+YODAgWY9pLOzM7Z2//33m23ffPNNsx4aRw9d92E9H5Ne+5Bkf/ZuAAtV9V0RGQJgp4hsjGq/U9X/KOE+iChnpezPfgTAkejrr0TkQwDXpN0xIqqsH/S6WETGApgCYHt00wIReV9EVorI0Jg2LSKyQ0R2JOopESVScthFZDCAvwB4RFVPAVgBYDyABvSc+X/bVztVbVXVRlVtrEB/iahMJYVdRKrQE/Q/qupfAUBVj6nqeVW9AOD3AKam100iSioYdul5a/BFAB+q6rO9bq/r9WO/AGC/rUtEuSpl6G06gP8DsBvAxfGCJwDMQ89LeAVwAMAvozfzrPtKbb5kkuWWgfBQyahRo2JrCxYsMNuGpmKOGDHCrCdx9OhRs75v3z6zPnbsWLNuHZekzpw5Y9a//vprs/7QQw/F1l599VWz7WWXXWbWz507Z9ZD0pw6XPbQm6puBdBXY3NMnYiKhVfQETnBsBM5wbATOcGwEznBsBM5wbATOZH5UtIJ21eqK98zYIA9CtnV1VX2fV9//fVmPbRk8i233GLWp0yZEltLOk00JHR9gmXDhg1m/bnnnjPr1hRWAGhra4uthX7fSZdzDsljnJ1ndiInGHYiJxh2IicYdiInGHYiJxh2IicYdiInsh5n/zuAg71uGgHg88w68MMUtW9F7RfAvpWrkn27VlVH9lXINOzfe3CRHUVdm66ofStqvwD2rVxZ9Y0v44mcYNiJnMg77K05P76lqH0rar8A9q1cmfQt17/ZiSg7eZ/ZiSgjDDuRE7mEXURuF5F2EdknIo/n0Yc4InJARHaLSFve+9NFe+h1isieXrcNE5GNIvJJ9LnPPfZy6ttiEemIjl2biNyRU99Gi8hmEflARPaKyK+j23M9dka/Mjlumf/NLiL9AXwMoAnAYQDvAJinqh9k2pEYInIAQKOq5n4Bhoj8K4DTAP6gqpOj254BcFxVl0T/UQ5V1d8UpG+LAZzOexvvaLeiut7bjAOYDeB+5HjsjH7NQQbHLY8z+1QA+1R1v6qeA/AnALNy6EfhqeoWAMe/c/MsAKujr1ej58mSuZi+FYKqHlHVd6OvvwJwcZvxXI+d0a9M5BH2awAc6vX9YRRrv3cFsEFEdopIS96d6UNtr222jgKozbMzfQhu452l72wzXphjV87250nxDbrvm66qPwUwE8CvoperhaQ9f4MVaey0pG28s9LHNuP/lOexK3f786TyCHsHgNG9vh8V3VYIqtoRfe4EsBbF24r62MUddKPP9qqLGSrSNt59bTOOAhy7PLc/zyPs7wCoF5GfiEg1gLkA1uXQj+8RkUHRGycQkUEAfo7ibUW9DkBz9HUzAHs70gwVZRvvuG3GkfOxy337c1XN/APAHeh5R/7/ATyZRx9i+jUOwK7oY2/efQOwBj0v67rQ897GfADDAWwC8AmA/wUwrEB9+y/0bO39PnqCVZdT36aj5yX6+wDaoo878j52Rr8yOW68XJbICb5BR+QEw07kBMNO5ATDTuQEw07kBMNO5ATDTuTEPwBIZ3y49K8AvgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IDvFaRHc_SVQ"
      },
      "source": [
        "**Train and Test Data Split**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "R02ThzBvfFGd",
        "outputId": "396646e9-0422-4262-cd1d-574ae3fb6503"
      },
      "source": [
        "train_image, test_image,train_label,test_label = train_test_split(dataset_image, dataset_label, test_size=0.1)\r\n",
        "print(len(train_image))\r\n",
        "print(len(train_label))\r\n",
        "print(len(test_image))\r\n",
        "print(len(test_label))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "27747\n",
            "27747\n",
            "3083\n",
            "3083\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "81Dt50rj_bNQ"
      },
      "source": [
        "**Ziping Train, Test Data and Label**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "biE0zczh2bwm"
      },
      "source": [
        "train_dataset = list(zip(test_image,train_label))\r\n",
        "test_dataset = list(zip(test_image,test_label))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N2Ml3daBhvJz"
      },
      "source": [
        "#**Experiment 1:**\r\n",
        "* **Optimizer =SGD**\r\n",
        "* **Learning rate =0.0001**\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bQZGJgRoa9Cj"
      },
      "source": [
        "**Hyperparameters**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "au6LGW_p26LG"
      },
      "source": [
        "#HYPERPERAMETERS\r\n",
        "batch_size = 256\r\n",
        "num_iters = 4000\r\n",
        "input_dim = 28*28 # num_features = 784\r\n",
        "output_dim = 2\r\n",
        "\r\n",
        "learning_rate = 0.0001\r\n",
        "\r\n",
        "# Device\r\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k143yc3Y_u_q"
      },
      "source": [
        "**MAKING DATASET ITERABLE**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ajD_p9JB2_pC"
      },
      "source": [
        "'''\r\n",
        "MAKING DATASET ITERABLE\r\n",
        "'''\r\n",
        "num_epochs = num_iters / (len(train_dataset) / batch_size)\r\n",
        "num_epochs = int(num_epochs)\r\n",
        "\r\n",
        "train_loader = torch.utils.data.DataLoader(dataset=train_dataset, \r\n",
        "                                           batch_size=batch_size, \r\n",
        "                                           shuffle=True)  \r\n",
        "\r\n",
        "test_loader = torch.utils.data.DataLoader(dataset=test_dataset, \r\n",
        "                                          batch_size=batch_size, \r\n",
        "                                          shuffle=False) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IlAk0YEbbAeb"
      },
      "source": [
        "**Logistic Regression Model**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "X7bpnzHU3B2X"
      },
      "source": [
        "class LogisticRegressionModel(nn.Module):\r\n",
        "    def __init__(self, input_size, num_classes):\r\n",
        "        super().__init__()\r\n",
        "        self.linear = nn.Linear(input_size, num_classes)\r\n",
        "\r\n",
        "    def forward(self, x):\r\n",
        "        logits  = self.linear(x)\r\n",
        "        probas = F.softmax(logits, dim=1)\r\n",
        "        return logits, probas"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ohbCJNV9bGG_"
      },
      "source": [
        "**INSTANTIATE MODEL CLASS**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hGX5dHZk3DUG",
        "outputId": "389701f0-92f4-4438-bfa0-9365034ea569"
      },
      "source": [
        "'''\r\n",
        "INSTANTIATE MODEL CLASS\r\n",
        "'''\r\n",
        "model = LogisticRegressionModel(input_size=input_dim,\r\n",
        "                                num_classes=output_dim)\r\n",
        "# To enable GPU\r\n",
        "model.to(device)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegressionModel(\n",
              "  (linear): Linear(in_features=784, out_features=2, bias=True)\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3CVt4G9xbIvX"
      },
      "source": [
        "**Optimizer setup**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Nz--RXQV3FUK"
      },
      "source": [
        "# INSTANTIATE OPTIMIZER CLASS\r\n",
        "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "forTL2I6bP5p"
      },
      "source": [
        "**Function TRAIN THE MODEL**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z9v5p_eY3HHb",
        "outputId": "9a76a288-f197-4c68-9e84-d4ba3e78c637"
      },
      "source": [
        "'''\r\n",
        "TRAIN THE MODEL\r\n",
        "'''\r\n",
        "iteration_loss = []\r\n",
        "iter = 0\r\n",
        "for epoch in range(num_epochs):\r\n",
        "    for i, (images, labels) in enumerate(train_loader):\r\n",
        "\r\n",
        "        \r\n",
        "        images = images.view(-1, 28*28).to(device)\r\n",
        "        labels = labels.to(device)\r\n",
        "\r\n",
        "        # Clear gradients w.r.t. parameters\r\n",
        "        optimizer.zero_grad()\r\n",
        "\r\n",
        "        # Forward pass to get output/logits\r\n",
        "        logits, probas = model(images.float()) \r\n",
        "\r\n",
        "        # Calculate Loss: PyTorch implementation of CrossEntropyLoss works with logits, not probabilities\r\n",
        "        loss = F.cross_entropy(logits, labels)\r\n",
        "\r\n",
        "        # Getting gradients w.r.t. parameters\r\n",
        "        loss.backward()\r\n",
        "\r\n",
        "        # Updating parameters\r\n",
        "        optimizer.step()\r\n",
        "\r\n",
        "        iter += 1\r\n",
        "\r\n",
        "        if iter % 500 == 0:\r\n",
        "            # Calculate Accuracy         \r\n",
        "            correct = 0\r\n",
        "            total = 0\r\n",
        "            # Iterate through test dataset\r\n",
        "            for images, labels in test_loader:\r\n",
        "               \r\n",
        "                images = images.view(-1, 28*28).to(device)\r\n",
        "\r\n",
        "                # Forward pass only to get logits/output\r\n",
        "                logits, probas = model(images.float())\r\n",
        "\r\n",
        "                # Get predictions from the maximum value\r\n",
        "                _, predicted = torch.max(probas, 1)\r\n",
        "\r\n",
        "                # Total number of labels\r\n",
        "                total += labels.size(0)\r\n",
        "\r\n",
        "\r\n",
        "                # Total correct predictions\r\n",
        "                if torch.cuda.is_available():\r\n",
        "                    correct += (predicted.cpu() == labels.cpu()).sum() \r\n",
        "                else:\r\n",
        "                    correct += (predicted == labels).sum()\r\n",
        "\r\n",
        "            accuracy = 100 * correct.item() / total\r\n",
        "\r\n",
        "            # Print Loss\r\n",
        "            iteration_loss.append(loss.item())\r\n",
        "            print('Iteration: {}. Loss: {}. Accuracy: {}'.format(iter, loss.item(), round(accuracy, 2)))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Iteration: 500. Loss: 0.6908424496650696. Accuracy: 51.99\n",
            "Iteration: 1000. Loss: 0.6893777251243591. Accuracy: 52.09\n",
            "Iteration: 1500. Loss: 0.683989942073822. Accuracy: 51.83\n",
            "Iteration: 2000. Loss: 0.6994321346282959. Accuracy: 52.19\n",
            "Iteration: 2500. Loss: 0.6977075338363647. Accuracy: 52.03\n",
            "Iteration: 3000. Loss: 0.6857379674911499. Accuracy: 51.51\n",
            "Iteration: 3500. Loss: 0.6856712102890015. Accuracy: 51.48\n",
            "Iteration: 4000. Loss: 0.6950100660324097. Accuracy: 51.41\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u_wK660SbXcW"
      },
      "source": [
        "**Loss show**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 298
        },
        "id": "Nn4CJKSp3MuX",
        "outputId": "76f8a8e1-d4dc-42d1-db75-128656aa75d9"
      },
      "source": [
        "import matplotlib\r\n",
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "print (iteration_loss)\r\n",
        "plt.plot(iteration_loss)\r\n",
        "plt.ylabel('Cross Entropy Loss')\r\n",
        "plt.xlabel('Iteration (in every 500)')\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0.6908424496650696, 0.6893777251243591, 0.683989942073822, 0.6994321346282959, 0.6977075338363647, 0.6857379674911499, 0.6856712102890015, 0.6950100660324097]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEICAYAAABI7RO5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9b34/9c7OwFCgAwIJCxCEmUH4wZqolaLdbe3Kr/aW9uqve3VVu2mrdd+q1erdrfX215atbZ1aa2itKJoK4uiKJBhCxgI2yRhC5AQ9mzv3x/nRMcxyyTkZLb38/GYBzOfOefMe3gQ3jmf5f0RVcUYY4wJV1KkAzDGGBNbLHEYY4zpEkscxhhjusQShzHGmC6xxGGMMaZLLHEYY4zpEk8Th4jMEpFyEakQkbvaeP8XIrLKfWwUkbqg974oIpvcxxeD2k8TkbXuNR8VEfHyOxhjjPk48Wodh4gkAxuBi4AqYDkwW1XXt3P8bcA0Vf2yiAwCVgBFgAIrgdNUtVZE3ge+AbwHzAceVdVXO4olJydHR48e3TNfzBhjEsTKlSv3qqovtD3Fw888A6hQ1S0AIvIccCXQZuIAZgM/dJ9/GnhDVfe7574BzBKRRUCWqi5z2/8IXAV0mDhGjx7NihUrTuzbGGNMghGR7W21e9lVNQKoDHpd5bZ9goiMAsYAb3Zy7gj3eafXNMYY441oGRy/Hvibqjb31AVF5BYRWSEiK2pqanrqssYYk/C8TBzVQF7Q61y3rS3XA8+GcW61+7zTa6rqHFUtUtUin+8TXXTGGGO6ycvEsRzIF5ExIpKGkxzmhR4kIqcAA4F3g5oXABeLyEARGQhcDCxQ1Z1AvYic5c6m+nfgZQ+/gzHGmBCeDY6rapOI3IqTBJKBJ1S1TETuA1aoamsSuR54ToOmd6nqfhG5Hyf5ANzXOlAOfB34A9AHZ1C8w4FxY4wxPcuz6bjRpKioSG1WlTHGdI2IrFTVotD2aBkcN8YYEyMscRjTiYamFv68bDs76o5GOhRjooKXCwCNiQsLynZxz0vrSEtOYvYZeXz9/HEMzcqIdFjGRIzdcRjTidJALRmpSVwzfQRPvxfg3EcWct/f17Pn4LFIh2ZMRFjiMKYT/kAdk3Ozeeizk3nzWyVcOWU4T727jfMeWcgDr6xn76HjkQ7RmF5licOYDhxvamb9jnqmjcwGYOTgTH7yuSn8885iPjNxGI+/vZVzH17IQ69+wP7DDRGO1pjeYYnDmA6U7ainobmF6SMHfqx9TE5ffn7dVF6/o5iLxg/l/5Zs5tyH3+SnC8qpO2IJxMQ3SxzGdKB0ey0A0/Ky23x/3JB+PDp7GgtuP4+SwiH8z8IKzn14Ib/850bqjzX2ZqjG9BpLHMZ0wF9Zx4jsPgzpZBZVwdD+PPb56bz6zXOZOS6HX/5zE+c89Cb/8+YmDh1v6qVojekdljiM6cCqQB3TRw3s/EDXqcOy+O0XTuMft53DGWMG89PXN3LOw2/ym0WbOWwJxMQJSxzGtGN3/TGq6462203VkYkjBvD7Lxbx8n/OZFpeNg+/9gHnPbKQOUs2c7Shx3YPMCYiLHEY0w5/wB3fGNn1xNFqSl42T37pDF78+gzGD8/iwfkfcO4jC3n87a0ca7QEYmKTJQ5j2uEP1JGWnMT44VknfK3pIwfyp6+cyfP/cTYFQ/tx/z/Wc94jC3nqnW0cb7IEYmKLJQ5j2lEaqGXiiCzSU5J77Jqnjx7EMzefxbM3n8XowX354bwySn6yiD8v205DU0uPfY4xXrLEYUwbGptbWFN1gGkjwx8Y74qzxw7mL189i6dvOpNhAzK456V1nP/TRTz3foDGZksgJrpZ4jCmDR/sPMjxppYTGt/ojIgwc1wOL3xtBk99+Qxy+qdz14trufBni/nbyiqaLIGYKGWJw5g2lLoD46Erxr0gIhQX+Hjp6zN44sYisvqk8O3nV3PRL5Yw119Fc0v8b7ZmYouniUNEZolIuYhUiMhd7RxzrYisF5EyEXkmqP1hEVnnPq4Lar9QREpFZJWIvC0i47z8DiYx+QO1DM1KZ9iA3iufLiJccMpQ/n7rOcz5wmlkpCZzx19Wc/EvFjNv9Q5aLIGYKOFZ4hCRZOAx4BJgPDBbRMaHHJMP3A3MVNUJwO1u+6XAdGAqcCbwbRFpndryG+DzqjoVeAa4x6vvYBKXv7KOaXkDEZFe/2wR4eIJJ/HKbefwm89PJzlJ+Mazfmb9agnz1+60BGIizss7jjOAClXdoqoNwHPAlSHH3Aw8pqq1AKq6x20fDyxR1SZVPQysAWa57ynQmkQGADs8/A4mAe07dJzt+44wfZR34xvhSEoSLpk0jNe+eR6/nj2N5hbl60+X8plH32JB2S5ULYGYyPAycYwAKoNeV7ltwQqAAhFZKiLLRKQ1OawGZolIpojkAOcDee57NwHzRaQK+ALwkGffwCQkf6AOwLMZVV2VlCRcPmU4r99RzC+vm8rxpha++qeVXP4/b/OvDbstgZheF+nB8RQgHygBZgO/E5FsVX0dmA+8AzwLvAu0rpK6A/iMquYCTwI/b+vCInKLiKwQkRU1NTXefgsTV/yVtaQkCZNGDIh0KB+TnCRcNW0Eb9xxHj/93BTqjzbxladWcNVjS1lUvscSiOk1XiaOaj66SwDIdduCVQHzVLVRVbcCG3ESCar6gKpOVdWLAAE2iogPmKKq77nn/wWY0daHq+ocVS1S1SKfz9dz38rEPX+gjvHDs8hI7bmFfz0pJTmJfzstl399q5iHPzuJvYcauPHJ5Xz2N+/w9qa9lkCM57xMHMuBfBEZIyJpwPXAvJBjXsK528DtkioAtohIsogMdtsnA5OB14FaYICIFLjnXwRs8PA7mATT3KKsrqzrVmHD3paanMR1p49k4bdLeODqiew8cIwbHn+P6/5vGRt21kc6PBPHUry6sKo2icitwAIgGXhCVctE5D5gharOc9+7WETW43RFfUdV94lIBvCWO6OlHrhBVZsARORm4AURacFJJF/26juYxLNx90EONzRHzfhGONJSkvj8maP4t9Ny+cvySn75z01874U1zLv1nEiHZuKUZ4kDQFXn44xVBLfdG/RcgTvdR/Axx3BmVrV1zbnA3B4P1hiCB8aj/44jVHpKMv9+9mgOHGnkZ29spObgcXz90yMdlolDkR4cNyaqlAZqGdw3jZGDMiMdSreVFA4B4K1NNinEeMMShzFB/IFapo3MjsjCv54yYXgWOf3SWFRuicN4wxKHMa4DRxrZXHM4psY32pKUJJyX7+OtTTVW58p4whKHMS5/5Ynv+Bctigt91B5pZE1VXaRDMXHIEocxLn+gjiSBybmxnzjOzfchgnVXGU9Y4jDG5a+so2Bof/qlezrZsFcM6pvGlNxsFm+0xGF6niUOY4CWFsUfqGX6qNge3whWUuhjdVUd+w83RDoUE2cscRgDbNl7iIPHmmJixXi4igt8qNq0XNPzLHEYA5RGWUXcnjA5N5uBmakstnEO08MscRiDMzA+oE8qJ+f0jXQoPSY5STivwMfijTW2+ZPpUZY4jMFZ+Dc1L5ukpNhd+NeW4gIf+w43sG7HgUiHYuKIJQ6T8A4db6J898G4WL8R6rwCZ0sB664yPckSh0l4ayrrUI2v8Y1WOf3SmZw7gEU2Ldf0IEscJuGVBpwV41PjaEZVsOICH/5ALXVHbFqu6RmWOEzC8wfqGDekHwP6pEY6FE+UFPpoUXi7Ym+kQzFxwhKHSWiqij9Gdvzrrim52WRlpFj5EdNjPE0cIjJLRMpFpEJE7mrnmGtFZL2IlInIM0HtD4vIOvdxXVC7iMgDIrJRRDaIyDe8/A4mvm3fd4T9hxviasV4qJTkJM61abmmB3lWlEdEkoHHcPYFrwKWi8g8VV0fdEw+cDcwU1VrRWSI234pMB2YCqQDi0TkVVWtB24E8oBTVLWl9RxjuiOeKuJ2pKTAxytrdrJhVz0Thg+IdDgmxnl5x3EGUKGqW1S1AXgOuDLkmJuBx1S1FkBV97jt44ElqtqkqoeBNcAs972vAfepakvIOcZ0mT9QR7/0FPKH9I90KJ4qdqflWneV6QleJo4RQGXQ6yq3LVgBUCAiS0VkmYi0JofVwCwRyRSRHOB8nLsMgLHAdSKyQkRede9ajOmW0kAtU/IGkBxnC/9CDcnKYPywLKuWa3pEpAfHU4B8oASYDfxORLJV9XVgPvAO8CzwLtDsnpMOHFPVIuB3wBNtXVhEbnGTy4qaGvthMZ90tKGZDTsPMi0vfsc3gpUU+li5vZb6Y42RDsXEOC8TRzUf3SUA5LptwaqAearaqKpbgY04iQRVfUBVp6rqRYC477We86L7fC4wua0PV9U5qlqkqkU+n69HvpCJL2urD9DconE/vtGquMBHc4uydJNNyzUnxsvEsRzIF5ExIpIGXA/MCznmJZy7DdwuqQJgi4gki8hgt30yTnJ4Peic893nxXyUUIzpktaFf/G4Yrwt00cNpH96inVXmRPm2awqVW0SkVuBBUAy8ISqlonIfcAKVZ3nvnexiKzH6Yr6jqruE5EM4C0RAagHblDVJvfSDwFPi8gdwCHgJq++g4lv/kAtowdnMqhvWqRD6RWpyUmck5/DovIaVBX358uYLvN0j0xVnY8zVhHcdm/QcwXudB/BxxzDmVnV1jXrgEt7PFiTUFSV0kAd54zLiXQovaq4wMer63ZRvvsgp5yUFelwTIyK9OC4MRGx48Axag4eT5jxjVbFhVYt15w4SxwmIZVud8Y3pifI+EarYQP6cMpJ/W09hzkhljhMQvIH6shITaLwpPhe+NeW4gIfK7bv59Dxps4PNqYNljhMQvJX1jJ5RDapyYn3I1Bc6KOxWXnHquWabur0p0ZE+opIkvu8QESuEJH4rD9tEsLxpmbKquuZNiqxxjdaFY0aRN+0ZNvcyXRbOL9uLQEyRGQEzlqKLwB/8DIoY7xUtqOehuaWhFkxHiotJYkZ43JY7E7LNaarwkkcoqpHgGuA/1XVzwETvA3LGO/4A3UATE+wGVXBSgp9VNcdZXPNoUiHYjzS0qI0Nbd4cu2wEoeInA18HnjFbUv2JBpjekFpoJYR2X0YkpUR6VAixqrlxr+lm/dy1o/fZMPO+h6/djiJ43acPTPmuiu/TwYW9ngkxvSSVYG6hFu/ESp3YCbjhvSz8iNxbG5pNcebmhmT07fHr91p4lDVxap6hao+7A6S71VV23XPxKTd9ceorjuaMPWpOlJS4OO9Lfs50mDTcuPNkYYmXivbxaWThpGR2vMdROHMqnpGRLJEpC+wDlgvIt/p8UiM6QX+QGLs+BeO4kIfDc0tvLt5X6RDMT1sQdkujjQ0c/W00C2QekY4XVXj3S1brwJeBcbgzKwyJub4A3WkJScxYbjVaTpjzCD6pCZbd1UcerG0mtyBfTh99CBPrh9O4kh1121chbt3BmBz+ExM8gfqmDAii/QUm9+RnpLMjLGDP6yWa+LD7vpjLK3Yy9XTRpDk0c6W4SSO/wO2AX2BJSIyCqfUuTExpbG5hTXVdQm7fqMtxYU+AvuPsHXv4UiHYnrIy6uqaVE866aC8AbHH1XVEar6GXVs56ONlIyJGR/sPMixxhamJ+iK8baUFAwBsO6qOPJiaTVT8rI52dfPs88IZ3B8gIj8vHX/bhH5Gc7dhzExxV+ZWDv+hWPk4ExOzulr6znixIad9Xyw6yDXeHi3AeF1VT0BHASudR/1wJNeBmWMF/yBOob0T2f4gMRd+NeW8wp8LNuyj2ONzZEOxZyguf5qUpKEy6cM9/RzwkkcY1X1h6q6xX38CDg5nIuLyCwRKReRChG5q51jrhWR9SJSJiLPBLU/LCLr3Md1bZz3qIhYvQQTttJALdNHDrQtU0OUFPo43tTCsi02LTeWNbcoL6+qpqTQ5/l2yOEkjqMick7rCxGZCRzt7CQRSQYeAy7B2QZ2toiMDzkmH2dV+kxVnYCzSh0RuRSYDkwFzgS+LSJZQecVAdbfYMK279Bxtu87Yus32nDWyYNJT0my7qoY987mveyuP87V03I9/6xwEsd/AI+JyDYR2Qb8D/DVMM47A6hw71IagOeAK0OOuRl4TFVrAVR1j9s+Hliiqk2qehhYA8yCDxPST4DvhhGDMQCsqnQKG9r4xidlpCZz1smDbYA8xs0traZ/RgoXnjrE888KZ1bValWdAkwGJqvqNOCCMK49AqgMel3ltgUrAApEZKmILBORWW77amCWiGSKSA7OLK48971bcdaT7AwjBmMAp5sqJUmYNGJApEOJSiWFPrbuPcz2fTYtNxYdPu6UGLlssjclRkKFvf2Zqta7K8gB7uyhz08B8oESYDbwOxHJVtXXgfnAO8CzwLtAs4gMBz4H/LqzC4vILa0zwWpq7DepROcP1HHqsCz6pNnCv7a0Vsu1u47Y9FGJEe+7qaD7W8eGM7pYzUd3CQC5bluwKtzV6Kq6FdiIk0hQ1QdUdaqqXuR+3kZgGjAOqHC7zTJFpKKtD1fVOapapKpFPp+vC1/NxJvmFmV1pVXE7ciYnL6MHJRp4xwxaq7fKTFSNKp3umK7mzjCqU+wHMgXkTEikgZcD8wLOeYlnLsN3C6pAmCLiCSLyGC3fTJON9nrqvqKqp6kqqNVdTRwRFXHdfM7mASxcfdBDjc0W+LogIhQUujj3c02LTfW9EaJkVDtJg4ROSgi9W08DgKdThJW1Sac8YgFwAbgr+5+HveJyBXuYQuAfSKyHmePj++o6j4gFXjLbZ8D3OBez5gu+2jHPxsY70hxgY+jjc0s37Y/0qGYLuiNEiOhUtp7Q1X7n+jFVXU+zlhFcNu9Qc8VZ7zkzpBjjuHMrOrs+t6tqTdxwx+oZVDfNEYOyox0KFHt7LGDSUtOYnF5DefmW/durOiNEiOhuttVZUzMKA3UMi0v2xb+dSIzLYUzTx7EIhsgjxm9VWIklCUOE9cOHGlkc81hpvfSoGGsKy7wUbHnEFW1RyIdiglDb5UYCWWJw8S1VVXuwr88GxgPR0mhTcuNFb1ZYiRUONVxbxMR+3XNxKTS7bUkCUy2xBGWsb5+jMjuY9NyY0BvlhgJFc4dx1BguYj81S1aaB3FJmb4K+soGNqffuntzgMxQUSE4kIf71TspaGpJdLhmA70ZomRUOGUHLkHZ1He48CNwCYReVBExnocmzEnpKVFWRWotfpUXVRS4ONwQzMrttu03Gh1+HgTr67rvRIjocIa43Cnze5yH004lWn/JiKPeBibMSdky97D1B9rsoV/XTRjXA6pycJi666KWgvKdnG0sfdKjIQKZ4zjmyKyEngEWApMUtWvAacBn/U4PmO6rTTg7PhnC/+6pl96CkWjBtk4RxTr7RIjocK54xgEXKOqn1bV51W1EUBVW4DLPI3OmBPgD9SRlZHCyTm203FXlRT6KN99kJ0HOt16x/SySJQYCRXOGMcPgcEi8g13htX0oPc2eBqdMSfAH6hl6siBEfvhimUlhc6Aq3VXRZ9IlBgJFU5X1X8BTwGDgRzgSRG5x+vAjDkRh443sXH3QVu/0U0FQ/txUlaGdVdFoUiUGAkVTlfVDcDp7r7jPwTOAr7gbVjGnJg1lXW0KLZivJtaq+UurdhLY7NNy40WkSoxEiqcxLEDyAh6nc4n99UwJqr43a1ip+baHUd3FRf4OHi8idLttZEOxbgiVWIkVDiJ4wBQJiJ/EJEngXVAnYg8KiKPehueMd1Tur2Wsb6+DMhMjXQoMWtmfg4pSWLlR6JEc4vykr+aksIhvV5iJFQ4y2nnuo9Wi7wJxZieoar4K+u48JTeX1EbT7IyUpk+aiCLymv47qxTIh1OwltasZc9B49zzfTIdlNBGIlDVZ9yd/ArcJvKW6fkGhONAvuPsP9wg60Y7wHFBT5+sqCcPfXHGJKV0fkJxjNz/U6JkQui4BeicGZVlQCbgMeA/wU2ish5HsdlTLe1LvyzFeMnzqrlRofDx5t4LYIlRkKFM8bxM+BiVS1W1fOATwO/COfiblHEchGpEJG72jnmWhFZLyJlIvJMUPvDIrLOfVwX1P60e811IvKEiFgntvkYf6COvmnJFAw94U0sE974YVn4+qfb5k4RFukSI6HCSRypqlre+kJVN+LsCd4hEUnGuUu5BGcb2NkiMj7kmHzgbmCmqk4AbnfbLwWmA1OBM4Fvi0iWe9rTwCnAJKAPcFMY38EkEH+gjil52STbwr8TJiIUF/h4a2MNTTYtN2IiXWIkVDiJY6WI/F5EStzH74AVYZx3BlChqltUtQF4Drgy5JibgcdUtRZAVfe47eOBJarapKqHgTXALPeY+eoC3geiIwWbqHC0oZkNO+utm6oHlRT6qD/WxGp3UyzTu6KhxEiocBLHfwDrgW+4j/XA18I4bwRQGfS6ym0LVgAUiMhSEVkmIrPc9tXALBHJFJEc4HwgL/hEt4vqC8BrbX24iNwiIitEZEVNjd1mJ4q11QdoalErbNiDzh3nI0mwVeQREg0lRkJ1OKvK7W5araqnAD/36PPzgRKcO4clIjJJVV8XkdOBd4Aa4F2gOeTc/8W5K3mrrQur6hxgDkBRUZF6ELuJQn53YHyqlRrpMQMyU5k20pmW+62LCyMdTsKJhhIjoTq841DVZqBcREZ249rVfPwuIZdPrjivAuapaqOqbgU24iQSVPUBVZ2qqhcB4r4HgIj8EPABd3YjLhPH/IE6Rg3OZHC/9EiHEldKCnysrT7A3kPHIx1KQomWEiOhwumqGoizcvxfIjKv9RHGecuBfBEZ464DuR4IPe8lnLsN3C6pAmCLiCSLyGC3fTIwGXjdfX0Tzsyu2W5pd2MAZ+FfaaDWCht6oLVa7hKbXdWroqXESKhwVo7/V3curKpNInIrsABIBp5Q1TIRuQ9Yoarz3PcuFpH1OF1R31HVfSKSAbzlbm9eD9ygqk3upX8LbAfedd9/UVXv606MJr7sOHCMPQePW2FDD0wYnkVOvzQWlddwzXSbj9IboqnESKhwEsdnVPV7wQ0i8jCwuLMTVXU+MD+k7d6g54rT3XRnyDHHcGZWtXXNcGI2Cah1fGNaniWOnpaUJJyX72Nh+R6aW9SmOveCaCoxEiqcrqqL2mi7pKcDMeZElW6vIyM1iVOG2cI/LxQX+qg90sgam5bbK6KpxEiodhOHiHxNRNYChSKyJuixFVjbeyEaEx5/ZS2TR2STmhzO70Omq87N9yE2LbdXRFuJkVAd/YQ9A1yOM6B9edDjNFX9fC/EZkzYjjc1U1ZtC/+8NKhvGlNys61uVS+IthIjodpNHKp6QFW3qepsnGmzjYAC/bo5PdcYz5TtqKehucUSh8eKC3ysrqpj/+GGSIcS16KtxEiocKrj3grsBt4AXnEf//A4LmO6xB9w+t2tlLq3Sgp9qMJbm+yuwyvRWGIkVDgzlG4HClV1n9fBGNNd/kAtI7L7MNT2jPDU5NxsBmamsri8hiunRt9sn3gQjSVGQoUziliJs32sMVHLH6hjqnVTeS45STg338fijTW0tFglHy+8WFrN1CgrMRIqnMSxBVgkIneLyJ2tD68DMyZcu+uPUV131Aob9pKSQh/7DjdQtqM+0qHEnfU73BIjUbh2I1g4iSOAM76RBvQPesS9vy6v5Kl3tnHoeFPnB5uI+Wh8w+44esN5Bc6ugIvK93RypOmquf4qUpKEyyZHV4mRUOHsOf6j0DYRSYjV2wvL9/Dqul38dEE5nyvK44szRjFqcN9Ih2VC+AO1pCUnMWF4VucHmxOW0y+dSSMGsGhjDbddmB/pcOJGc4vy8qodUVliJFRHCwDfDnr+p5C33/csoijymxtOY+7XZ3DhqUP407JtlPx0ETc9tZy3N+3FqZZiooE/UMf44Vmkp0TfQql4VVLowx+o5cCRxkiHEjeiucRIqI66qoJ/tZ4Y8l50zhHzwLSRA/nl9dNY+r0LuO2CfFZV1nHD4+9x8S+W8PR72znSYN1YkdTY3MKa6job3+hlJYU+WhTeqrBpuT0lmkuMhOoocWg7z9t6HfeGZGVw50UFLL3rAn72uSmkpybxg7nrOOvBf/Hg/A1U7j8S6RATUvmugxxrtIV/vW1KbjZZGSlWfqSHRHuJkVAdjVVki8jVOMklW0SucdsFGOB5ZFEqPSWZz56WyzXTR7Byey1PvrONx9/eyu/f2sJF44dy44wxnHXyINyS78Zjpa0VcS1x9KqU5CTOLXCm5aqq/Xs/QdFeYiRUR4ljMXBF0PPLg95b4llEMUJEKBo9iKLRg9hRd5Q/L9vOs+8HWFC2m1NO6s+XZo7myqkjYuK3h1jmD9QxpH86I7L7RDqUhFNS4OOVNTtZv7OeCcMT9nfJHhHtJUZCtZs4VPVLvRlILBue3YfvzjqFb1yYz8urqnly6Ta+98JaHnr1A2afMZIvnD2KYQPsPzYv+AO1TBuZbb/xRkDxh9NyayxxnIDWEiO3nj8uakuMhPK0/rSIzBKRchGpEJG72jnmWhFZLyJlIvJMUPvDIrLOfVwX1D5GRN5zr/kXd1vaqJCRmsx1p4/k1W+ey7M3n8UZYwbx28WbOefhhfznM6Ws2LbfZmP1oH2HjrNt3xGrTxUhQ7IyGD8sy6rlnqAPS4zE0M6Knq3HEJFk4DGcjaCqgOUiMk9V1wcdkw/cDcxU1VoRGeK2XwpMB6YC6Tgr119V1XrgYeAXqvqciPwW+ArwG6++R3eICGePHczZYwdTuf8If1q2nefeD/DKmp1MGjGAG2eM5rIpw2z66AlaVeks/LMZVZFTXOhjzpIt1B9rJCsjNdLhxKTWEiNjcmJnjZiXdxxnABWqukVVG4DngCtDjrkZeExVawFUtXUp6nhgiao2qephYA0wS5z+iAuAv7nHPQVc5eF3OGF5gzL5/mdOZdn3L+S/r5rI0cZmvvX8amY+9CY/f2Mje+qPRTrEmOUP1JGcJEwaYd0kkVJS4KO5RXmnYm+kQ4lJsVJiJFQ4ZdU/JyL93ef3iMiLIjI9jGuPwCmQ2KrKbQtWABSIyFIRWSYis9z21TiJIlNEcoDzgTxgMFCnqk0dXDMqZaalcMNZo3jjjvP401fOYEpuNr9+c6uI0RQAAB+kSURBVBMzH36T25/zf/jbswlfaaCWU4f1p0+a3blFyvRRA+mfbtNyuytWSoyECqer6r9U9XkROQf4FPATnK6hM3vo8/OBEiAXWCIik1T1dRE5HXgHqAHeBZq7cmERuQW4BWDkyOjZd0rEqS56br6PbXsP89S723h+RRUvrdrB1LxsvjRzNJdMHEZaim1/2pHmFmV1ZR3XxFC/cDxKTU5i5rgcFpXbtNyuiqUSI6HC+d+p9T/sS4E5qvoKTsHDzlTj3CW0ynXbglUB81S1UVW3AhtxEgmq+oCqTlXVi3DWjmwE9uGsKUnp4Jq4589R1SJVLfL5fGGE2/tG5/Tlh5dPYNn3L+T/XT6eA0cb+eZzqzjn4Td59F+b2HvoeKRDjFqb9hzkcEMz00fZ+o1IKyn0sav+GBt3H4p0KDEllkqMhAoncVSLyP8B1wHzRSQ9zPOWA/nuLKg04Hqc/cuDvYRzt4HbJVUAbBGRZBEZ7LZPBiYDr6szJWkh8G/u+V8EXg4jlqjWLz2FG2eO4V93FvPkjadzyrAsfv7GRmb8+E2+9dfVrKu27VBClW53K+Lm2cB4pBUXWrXc7oilEiOhwumquhaYBfxUVetEZBjwnc5OUtUmd9vZBUAy8ISqlonIfcAKVZ3nvnexiKzHubP5jqruE5EM4C33trceuCFoXON7wHMi8t+AH3i8K184miUlCeefMoTzTxlCxZ5DPPXONl4oreKF0ipOHz2QG2eM4dMThpKSbN1Y/kAtg/qmMWpwZqRDSXjDBvShcGh/FpXX8NXisZEOJya0lhi5atrwmFwkHE7iGAa8oqrHRaQE57f/P4ZzcVWdD8wPabs36LkCd7qP4GOO4cysauuaW3BmbMW1cUP6cf9VE/n2pwt5fkUlT727jf98ppRhAzL4wtmjmH36SAbGWL9oT/JX1jEtzxb+RYuSQh9PLN3KoeNN9EtPiF0XTkislRgJFc6vri8AzSIyDpiDM27xTMenmJ4yoE8qN517Mou+fT5zvnAaY3L68shr5Zz1439x1wtr2LAz8XZhO3CkkYo9h6w+VRQpLvTR2GzTcsM1119N3qDYKTESKpxfDVrcbqdrgF+r6q9FxO91YObjkpOEiyecxMUTTqJ810H+8M5WXiyt5rnllZx18iC+NHMMnzp1KMkxUrLgRKyqat3xLzZ/6OJR0ahB9E1LZtHGGi6ecFKkw4lquw4c4+2KvdwWQyVGQoVzx9EoIrOBfwf+4bbZEtEIKjypPz++ZjLL7r6Q7806hcC+I3z1Tys5/6eL2FIT/zNb/IFaRGBKnt1xRIu0lCRmjMthsTst17Tv5VXVaIyVGAkVTuL4EnA28ICqbhWRMUDojoAmAgb2TeNrJWNZ8t3z+c3np1N3pIG7XlxLS0t8/+D6A3UUDu1vfelRpqTQR3XdUTYnwC8vJ2KuP/ZKjITqNHG4taW+DawVkYlAlao+7HlkJmwpyUlcMmkY3//Mqby/dT/Pr6zs/KQY1dKiH1bENdEluFquaVuslhgJFU7JkRJgE07Bwv8FNorIeR7HZbrh2qI8zhg9iAfnf0DNwfhcPLhl72HqjzXZ+EYUyh2Yybgh/axabgditcRIqHC6qn4GXKyqxap6HvBp4BfehmW6IylJePCaiRxtaOb+f6zv/IQY5Hd3/JtudxxRqaTAx3tb9nOkoanzgxNMLJcYCRVO4khV1fLWF6q6ERscj1rjhvTnayVjmbd6R1yu5C0N1NE/I4WTc/pFOhTThuJCHw3NLby7eV+kQ4k6sVxiJFQ4iWOliPxeRErcx++AFV4HZrrv6+eP5WRfX+55aV3c/ebnD9QyNS87ZqcxxrvTRw+iT2qydVe1IZZLjIQKJ3H8B7Ae+Ib7WA98zcugzIlJT0nmx1dPoqr2KL/656ZIh9NjDh1vYuPug7ZxUxTLSE3m7LGDP6yWaxytJUYumxybJUZCdZg43F38Vqvqz1X1GvfxC1WNz5HXOHLmyYO5riiP37+9lbId8VEkcU1lHS2KzaiKciWFPgL7j7Bt35FIhxI1WkuMxEM3FXSSOFS1GSgXkejZ0MKE7e7PnMLAzFTufnEtzXGwtsNfaRVxY0FJgdMVE49jbN31YmlslxgJFU5X1UCgTET+JSLzWh9eB2ZOXHZmGv912XjWVB3gj+9ui3Q4J8wfqGWsry8DMm1uRjQbOTiTMTl9bT2Ha9eBYyzdvJerp46Im6KcYe0A6HkUxjNXTBnOi6XV/HRBOZ+ecBLDs/tEOqRuUVVKA3VxMbCYCIoLfDz7foBjjc1x0ad/IuKhxEiodu84RGSciMxU1cXBD5x9M6p6L0RzIkSE/75qIs2q3PtyWcwOWAb2H2H/4QYb34gRJYU+jje1sGyLTcuNhxIjoTrqqvolziZKoQ6475kYkTcokzsvKuCfG3azoGxXpMPpFn/AGd+wGVWx4ayTB5OekpTw3VXxUmIkVEeJY6iqrg1tdNtGexaR8cSXZ45h/LAs7n25jPpjjZEOp8tKA7VkpiVTMLR/pEMxYchITeaskwezJMHXc8RLiZFQHSWOjvoEwuooF5FZIlIuIhUiclc7x1wrIutFpExEnglqf8Rt2yAij4o7qiQis0VkrYisEZHX3L3KTSdSkpP48TWT2HvoOD95rbzzE6KMP1DHlNzshNhvJF6UFPrYsvcwgQSdlhtPJUZCdZQ4VojIzaGNInITsLKzC7trQB4DLsHZBna2iIwPOSYfuBuYqaoTgNvd9hnATJxtaicCpwPFIpIC/Ao4X1UnA2uAWzuLxTim5GXzxRmj+fN721m5vTbS4YTtaEMzG3bWM32UjW/Ekg+r5W5MzGm58VRiJFRHieN24EsiskhEfuY+FgNfAb4ZxrXPACpUdYuqNgDPAVeGHHMz8Jiq1gKoauu/MAUygDQgHac21m5A3Edf9w4kC9gRRizG9a2LCzkpK4Pvv7iWxuaWSIcTlnU7DtDUorZ+I8aMyenLyEGZLE7QcY65/mqy4qTESKh2E4eq7lbVGcCPgG3u40eqeraqhjPCOgII3hiiym0LVgAUiMhSEVkmIrPcz34XWAjsdB8LVHWDqjbilDtZi5MwxgOPhxGLcfVLT+H+KydSvvsgc5ZsiXQ4YSl1746m2oyqmCIilBT6eGfzPo41Nkc6nF7VWmLk0jgpMRIqnI2cFqrqr93Hmz38+SlAPlACzAZ+JyLZIjIOOBXIxUk2F4jIuSKSipM4pgHDcbqq7m7rwiJyi4isEJEVNTWJ+RtPez41fiiXTDyJX/1rE9v2Ho50OJ3yB+oYOSiTnH7pkQ7FdFFxgY+jjc0s37Y/0qH0qtfWxVeJkVDhrBzvrmogL+h1rtsWrAqYp6qNqroV2IiTSK4GlqnqIVU9BLyKs33tVABV3azOgoS/AjPa+nBVnaOqRapa5PP5evJ7xYX/d8UE0pOT+MFLa6N6bYez8K/W9t+IUWePHUxaclLCdVfN9cdXiZFQXiaO5UC+iIwRkTTgeiC0VMlLOHcbuLOjCoAtQAB3MNy9yygGNuAknvEi0poJLnLbTRcNzcrgu5ecwtKKfcz1h+bz6LHjwDH2HDxuO/7FqMy0FM48eRCLEmhabjyWGAnlWeJQ1SacGU8LcP5z/6uqlonIfSJyhXvYAmCfiKzHGdP4jqruA/4GbMYZy1iNU6H376q6A2fMZYmIrMG5A3nQq+8Q7z5/xkimj8zm/n+sZ//hhkiH06bWHf9sxXjsKi7wUbHnEFW1iTEtNx5LjITy8o4DVZ2vqgWqOlZVH3Db7lXVee5zVdU7VXW8qk5S1efc9mZV/aqqnuq+d2fQNX/rtk9W1cvdRGO6ISlJ+PE1kzl4rIkHXonOGzd/oI70lCROHZYV6VBMN5UUOh0EibK5UzyWGAnlaeIw0a/wpP58tfhkXiitYmnF3kiH8wmlgVom5w4gNdn+qcaqsb5+jMjukxDlR+K1xEgo+2k03HZBPqMHZ/KDuWujatrk8aZmyqrrbXwjxokIxYU+3qnYS0NTbKwd6q54LTESyhKHISM1mQeunsS2fUf4nzcrIh3Oh9bvqKehucVmVMWBkgIfhxuaWbE9fqfltpYYOf+U+CsxEsoShwFg5rgcrpk+gt8u3kz5roORDgeAUrcirt1xxL4Z43JITZa4npb7YYmRafHdTQWWOEyQey4dT/+MFO5+cQ0tUbDVrD9Qy/ABGQzNyoh0KOYE9UtPoWjUoLgeIH+xtMopMXJq/JUYCWWJw3xoUN807rl0PKWBOp5+PxDpcPAH6pgWpwuoElFJoY8Pdh1k54GjkQ6lxx0+3sSCst1cOnk46SnxV2IklCUO8zHXTB/BzHGDeeTVD9hdfyxiceypP0Z13VGm5dn4Rrwobp2WG4fdVfFeYiSUJQ7zMSLCA1dNoqG5hR/9vSxicdj4RvwpHNqfk7Iy4rK7Kt5LjISyxGE+YXROX75xYT7z1+7in+t3RyQGf2UtqcnChOG28C9etFbLfXvT3pgp6R+ORCgxEsoSh2nTzeeeTOHQ/tz78joOHW/q9c/3B+qYMHxAXJakTmTFBT4OHm/6sFR+PEiEEiOhLHGYNqWlJPHgNZPYWX+Mn73eu1vNNja3sKaqzupTxaGZ+TmkJElcdVclQomRUJY4TLtOGzWQG84cxVPvbGNNVV2vfW75roMca2yx8Y04lJWRyvRRA+Om/EiilBgJZYnDdOg7swrJ6ZfOXS+spamX+qVbK+LaivH4VFzgY/3OevZEcNZeT5nrryI1Of5LjISyxGE6lJWRyo+umMD6nfU8sXRrr3xmaaAOX/90RmT36ZXPM70rXqrlNjW38NKqHZQUxn+JkVCWOEynZk08iU+dOpRfvLGJyv3e76ngD9QyLS87YWaoJJrxw7Lw9U+P+c2dlm7eR02ClBgJZYnDdEpEuO/KCSQJ3PPSOk+3mt1/uIFt+44wPUHmwyciEaG4wJmW21vdn16Ym0AlRkJ5mjhEZJaIlItIhYjc1c4x14rIehEpE5Fngtofcds2iMij4v76KSJpIjJHRDaKyAci8lkvv4NxDM/uw7c/XcjijTX8fc1Ozz7nwx3/bMV4XCsp9HHgaCOre3HSRU9KtBIjoTxLHCKSDDwGXAKMB2aLyPiQY/KBu4GZqjoBuN1tnwHMBCYDE4HTcfYdB/gBsEdVC9zrLvbqO5iP+/ezRzMldwD3/b2MA0caPfkMf6CO5CRhUu4AT65vosM543JIEmJ2dlWilRgJ5eUdxxlAhapuUdUG4DngypBjbgYeU9VaAFXd47YrkAGkAelAKtC6hPnLwI/d41tUNfq2rYtTyUnCg9dMovZIIz9+1ZutZv2VtZw6rD+ZaSmeXN9Eh+zMNKaNHBizA+SJVmIklJeJYwRQGfS6ym0LVgAUiMhSEVkmIrMAVPVdYCGw030sUNUNItLaf3G/iJSKyPMiMtTD72BCTBg+gJvOGcNzyyt5f2vPbsrT3KKsCtQxLS8xfxgTTUmBjzVVB9h76HikQ+mSRCwxEirSg+MpQD5QAswGfici2SIyDjgVyMVJNheIyLnu8bnAO6o6HXgX+GlbFxaRW0RkhYisqKmJzd9qotU3P5VP7sA+3P3iGo439dxWs5v2HORwQ7OtGE8QrdVyl8TYXUcilhgJ5WXiqAbygl7num3BqoB5qtqoqluBjTiJ5GpgmaoeUtVDwKvA2cA+4Ajwonv+88D0tj5cVeeoapGqFvl8vp76TgbITEvhv6+ayOaaw/xm0eYeu67fKuImlInDB5DTLy3muqvm+quZNjKxSoyE8jJxLAfyRWSMiKQB1wPzQo55CeduAxHJwem62gIEgGIRSRGRVJyB8Q3qzAP9e+s5wIXAeg+/g2lHSeEQrpgynP9duJmKPYd65Jr+QC0DM1MZPTizR65noltSknBevo8lG2tYV32AD3bVU7HnEIF9R9hRd5Q9B49Re7iBg8caOdbYTFNzi6dTwcPxYYmRBFy7EcyzEUhVbRKRW4EFQDLwhKqWich9wApVnee+d7GIrAeage+o6j4R+RtwAbAWZ6D8NVX9u3vp7wF/EpFfAjXAl7z6DqZj/3XZeBaV7+H7c9fy3M1nkZR0Yv29pYE6po0cmLD9xonoglOH8KK/mst+/XbY56QmCylJSaQkC6nJSaQkuX8mS8jzpE6Odd//2PMkUpOcP1OShVT33Nb2ReU1CVliJJSnU1dUdT4wP6Tt3qDnCtzpPoKPaQa+2s41twPn9Xiwpst8/dP5waWn8r0X1vL8ykquO31kt6914GgjFXsOceWUxP6BTDSfmTiMp29K4/DxJppalMbmFpqanT8bW5Sm1tctzp9NQe2NzUpTS+vxwc9bPnatppYWjjZ+8v0PPyf0M1o6vqu5dNIwBiZYiZFQNufRnJBri/J4obSaB+d/wAWnDMXXP71b11ld6Yxv2IrxxJKUJMwclxPpMD5GVT9KLG0krGEDrIZapGdVmRgnIjx49SSONjRz/z+6P9xUGqhFBCbbwj8TYSJOd1aftGSyMlIZ1DeNIVkZjMjuw6jBfUlLsf827W/AnLBxQ/rx9fPHMm/1DhaV7+n8hDb4A3UUDOlP/4zUHo7OGNPTLHGYHvG1krGM9fXlnpfWcaSha1vNtrQoqyrrmD7K1m8YEwsscZgekZ6SzINXT6Kq9ii/+uemLp27Ze9hDhxttBXjxsQISxymx5x58mCuPz2P37+9lbIdB8I+78OKuLZi3JiYYInD9Ki7LzmVgZlp3P3iWpo7mdbYyl9ZR/+MFMb6+nkcnTGmJ1jiMD1qQGYq914+njVVB/jju9vCOqd0ey1T87JPeAGhMaZ3WOIwPe7yycMoKfTx0wXl7Kg72uGxh443sXH3QatPZUwMscRhepyIcP+VE2lRuPflsg7rC62pqqNFbXzDmFhiicN4Im9QJndclM8/N+xmQdmudo/7sCKubRVrTMywxGE88+WZYxg/LIt7Xy6j/ljbW836A7Wc7OtLdmZi1/4xJpZY4jCeSUlO4qHPTmLvoeP85LXyT7yvqvhtxz9jYo4lDuOpybnZ3DhjDH9+bzsrt9d+7L3K/UfZd7jBVowbE2MscRjPfeviAoZlZfD9F9fS2NzyYXtp68I/u+MwJqZY4jCe65uewn1XTqR890HmLNnyYbs/UEtmWjIFQ23hnzGxxBKH6RWfGj+Uz0w6iV/9axPb9h4GnBXjU3KzSUm2f4bGxBJPf2JFZJaIlItIhYjc1c4x14rIehEpE5Fngtofcds2iMijErKfqIjME5F1XsZvetYPL59AenISP3hpLUcbmlm/o97WbxgTgzxLHCKSDDwGXAKMB2aLyPiQY/KBu4GZqjoBuN1tnwHMBCYDE4HTgeKg864BDnkVu/HG0KwMvnfJKSyt2Md9/yijqUVtxbgxMcjLO44zgApV3aKqDcBzwJUhx9wMPKaqtQCq2roLkAIZQBqQDqQCuwFEpB/OHuX/7WHsxiP/3xkjOW3UQJ59vxKwFePGxCIvE8cIoDLodZXbFqwAKBCRpSKyTERmAajqu8BCYKf7WKCqG9xz7gd+BhzxMHbjkaQkZ6vZlCRh5KBMcvp1b49yY0zkpETB5+cDJUAusEREJgE5wKluG8AbInIucBAYq6p3iMjoji4sIrcAtwCMHDnSi9hNNxWe1J9H/m2yDYobE6O8TBzVQF7Q61y3LVgV8J6qNgJbRWQjHyWSZap6CEBEXgXOxkkcRSKyzY19iIgsUtWS0A9X1TnAHICioqLwNoYwveaa6bmdH2SMiUpe/sq3HMgXkTEikgZcD8wLOeYlnCSBiOTgdF1tAQJAsYikiEgqzsD4BlX9jaoOV9XRwDnAxraShjHGGO94ljhUtQm4FVgAbAD+qqplInKfiFzhHrYA2Cci63HGNL6jqvuAvwGbgbXAamC1qv7dq1iNMcaETzraKyFeFBUV6YoVKyIdhjHGxBQRWamqRaHtNjppjDGmSyxxGGOM6RJLHMYYY7rEEocxxpguscRhjDGmSxJiVpWI1ADbu3l6DrC3B8PxWizFa7F6J5bijaVYIbbiPdFYR6mqL7QxIRLHiRCRFW1NR4tWsRSvxeqdWIo3lmKF2IrXq1itq8oYY0yXWOIwxhjTJZY4Ojcn0gF0USzFa7F6J5bijaVYIbbi9SRWG+MwxhjTJXbHYYwxpksscXRARGaJSLmIVIjIXZGOpz0i8oSI7BGRdZGOJRwikiciC0VkvYiUicg3Ix1Te0QkQ0TeF5HVbqw/inRMnRGRZBHxi8g/Ih1LZ0Rkm4isFZFVIhLVlUhFJFtE/iYiH4jIBhE5O9IxtUdECt2/09ZHvYjc3mPXt66qtolIMrARuAhnw6nlwGxVXR/RwNogIucBh4A/qurESMfTGREZBgxT1VIR6Q+sBK6K0r9bAfqq6iF3b5i3gW+q6rIIh9YuEbkTKAKyVPWySMfTEXdTtiJVjfp1ESLyFPCWqv7e3WMoU1XrIh1XZ9z/y6qBM1W1u+vZPsbuONp3BlChqltUtQF4DrgywjG1SVWXAPsjHUe4VHWnqpa6zw/i7NcSuh99VFDHIfdlqvuI2t+2RCQXuBT4faRjiSciMgA4D3gcQFUbYiFpuC4ENvdU0gBLHB0ZAVQGva4iSv9zi2Xu3vHTgPciG0n73K6fVcAe4A1VjdpYgV8C3wVaIh1ImBR4XURWisgtkQ6mA2OAGuBJtxvw9yLSN9JBhel64NmevKAlDhMxItIPeAG4XVXrIx1Pe1S1WVWnArnAGSISld2BInIZsEdVV0Y6li44R1WnA5cA/+l2u0ajFGA68BtVnQYcBqJ23LOV26V2BfB8T17XEkf7qoG8oNe5bpvpAe54wQvA06r6YqTjCYfbNbEQmBXpWNoxE7jCHTd4DrhARP4c2ZA6pqrV7p97gLk4XcTRqAqoCrrb/BtOIol2lwClqrq7Jy9qiaN9y4F8ERnjZu3rgXkRjikuuAPOjwMbVPXnkY6nIyLiE5Fs93kfnMkSH0Q2qrap6t2qmquqo3H+vb6pqjdEOKx2iUhfd3IEbrfPxUBUzgxU1V1ApYgUuk0XAlE3maMNs+nhbipwbr9MG1S1SURuBRYAycATqloW4bDaJCLPAiVAjohUAT9U1ccjG1WHZgJfANa6YwcA31fV+RGMqT3DgKfcmSlJwF9VNeqnucaIocBc5/cIUoBnVPW1yIbUoduAp91fJLcAX4pwPB1yk/FFwFd7/No2HdcYY0xXWFeVMcaYLrHEYYwxpksscRhjjOkSSxzGGGO6xBKHMcaYLrHEYaKWiBxy/xwtIv9fD1/7+yGv3+nBa/+ydQW0W5pifE9d22si0hxUUXVeUPsYEXnPrRT9F3dKKiKS7r6ucN8f7bZPEpE/RORLGM9Z4jCxYDTQpcQhIp2tUfpY4lDVGV2Mqb3PHQyc5RaeRFVvinTV3zD+LoIdVdWp7uOKoPaHgV+o6jigFviK2/4VoNZt/4V7HKq6FsgVkZEn/g1MtLHEYWLBQ8C57m/Bd7hFB38iIstFZI2IfBVAREpE5C33N+X1bttLbgG9stYieiLyENDHvd7Tblvr3Y24117n7hNxXdC1FwXtx/C0uwI+1GeBDxexuecUtX6GiDwgzt4ey0RkaOjJ7mrqJ8TZA8QvIle67ctEZELodTs4/kYRmScibwL/EpE/ishVQec/3XpsZ9zveQFOmQ2Ap4DWa13pvsZ9/8Kgv5e/46xgN/FGVe1hj6h8AIfcP0uAfwS13wLc4z5PB1bgVC8twSk+Nybo2EHun31wylkMDr52G5/1WeANnGoBQ4EAzurxEuAATs2yJOBdnAJ9oTE/BVwe9HoRzn4T4FSCvdx9/kjrdwg5/0HgBvd5Ns6eMH2BO4Afue3DgPJOjr8Rp75S6/cvBl5ynw8AtgIpbXx+k/v3uQxnjxSAHJwtBlqPyQPWuc/XAblB720GctznM4G/R/rfkT16/mF3HCYWXQz8u1uu5D1gMJDvvve+qm4NOvYbIrIa5z/CvKDj2nMO8Kw6FXF3A4uB04OuXaWqLcAqnC60UMNwym+3pQFoLVeysp3zLwbucr/bIiADGAn8Ffg395hr+ei3//aOB6cE/H4AVV2MU3vNh1O/6AVVbWrj80epahFO1+AvRWRsO98lHHuA4SdwvolSVqvKxCIBblPVBR9rFCnBueMIfv0p4GxVPSIii3D+Y+2u40HPm2n75+doB5/RqKqtNX7aO1+Az6pq+SfeENknIpOB64D/6Oh4ETmToL8L1x+BG3C6j9qss6QfVavd4v59TcOpYpwtIilusgmuFN1aRbrKHUsZAOxz38vA+fswccbuOEwsOAj0D3q9APiaOKXZEZECaXtTnQE4A7dHROQU4Kyg9xpbzw/xFnCdO47iw9n17f0uxLoBGNeF40MtAG5rHScQkWlB7/0FZ5OmAaq6JozjQ/0BuB1A2xiwF5GBIpLuPs/B6Wpa7ya7hXx0x/NF4GX3+Tz3Ne77bwYlxwKitNqtOTGWOEwsWAM0u4PKd+Bsi7oeKBWRdcD/0fZv768BKSKyAWeAPXif8DnAmtbB8SBz3c9bDbwJfFedktrhegVnPKS77sfZnnaNiJS5r1v9Dedu4a9hHv8xbtfbBuDJdg45FVjhdu0tBB4KSjDfA+4UkQqcrsHW6suPA4Pd9jv5+OZG5+P8fZg4Y9VxjelhIvI2cJlG2Z7UIpIJrAWmq+oBjz8rHWd86Jx2xlJMDLM7DmN63rf4aIA6KojIp3DuNn7tddJwjQTusqQRn+yOwxhjTJfYHYcxxpguscRhjDGmSyxxGGOM6RJLHMYYY7rEEocxxpguscRhjDGmS/5/Jj1STtOE/nYAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P0H0e17lAobv"
      },
      "source": [
        "#**Experiment 2:**\r\n",
        "* **Optimizer =SGD**\r\n",
        "* **Learning rate =0.0005**\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xKh3qO7JAoby"
      },
      "source": [
        "**Hyperparameters**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CWFKvak0Aobz"
      },
      "source": [
        "#HYPERPERAMETERS\r\n",
        "batch_size = 256\r\n",
        "num_iters = 4000\r\n",
        "input_dim = 28*28 # num_features = 784\r\n",
        "output_dim = 2\r\n",
        "\r\n",
        "learning_rate = 0.0005\r\n",
        "\r\n",
        "# Device\r\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xNgAmVXZAobz"
      },
      "source": [
        "**MAKING DATASET ITERABLE**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iQcPBDg8Aobz"
      },
      "source": [
        "'''\r\n",
        "MAKING DATASET ITERABLE\r\n",
        "'''\r\n",
        "num_epochs = num_iters / (len(train_dataset) / batch_size)\r\n",
        "num_epochs = int(num_epochs)\r\n",
        "\r\n",
        "train_loader = torch.utils.data.DataLoader(dataset=train_dataset, \r\n",
        "                                           batch_size=batch_size, \r\n",
        "                                           shuffle=True)  \r\n",
        "\r\n",
        "test_loader = torch.utils.data.DataLoader(dataset=test_dataset, \r\n",
        "                                          batch_size=batch_size, \r\n",
        "                                          shuffle=False) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CwBUnEfkAob0"
      },
      "source": [
        "**Logistic Regression Model**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lB5XdEHXAob0"
      },
      "source": [
        "class LogisticRegressionModel(nn.Module):\r\n",
        "    def __init__(self, input_size, num_classes):\r\n",
        "        super().__init__()\r\n",
        "        self.linear = nn.Linear(input_size, num_classes)\r\n",
        "\r\n",
        "    def forward(self, x):\r\n",
        "        logits  = self.linear(x)\r\n",
        "        probas = F.softmax(logits, dim=1)\r\n",
        "        return logits, probas"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rst_MVqMAob0"
      },
      "source": [
        "**INSTANTIATE MODEL CLASS**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "k-awaiOTAob1",
        "outputId": "592df4e3-e826-4c13-d028-fe34680277d9"
      },
      "source": [
        "'''\r\n",
        "INSTANTIATE MODEL CLASS\r\n",
        "'''\r\n",
        "model = LogisticRegressionModel(input_size=input_dim,\r\n",
        "                                num_classes=output_dim)\r\n",
        "# To enable GPU\r\n",
        "model.to(device)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegressionModel(\n",
              "  (linear): Linear(in_features=784, out_features=2, bias=True)\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q6RrHHPnAob3"
      },
      "source": [
        "**Optimizer setup**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "M0qY5wODAob3"
      },
      "source": [
        "# INSTANTIATE OPTIMIZER CLASS\r\n",
        "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QfsQziAlAob3"
      },
      "source": [
        "**Function TRAIN THE MODEL**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3B-YHGS3Aob3",
        "outputId": "12f1be9d-f0ab-4d1f-d10e-67a3242494af"
      },
      "source": [
        "'''\r\n",
        "TRAIN THE MODEL\r\n",
        "'''\r\n",
        "iteration_loss = []\r\n",
        "iter = 0\r\n",
        "for epoch in range(num_epochs):\r\n",
        "    for i, (images, labels) in enumerate(train_loader):\r\n",
        "\r\n",
        "        \r\n",
        "        images = images.view(-1, 28*28).to(device)\r\n",
        "        labels = labels.to(device)\r\n",
        "\r\n",
        "        # Clear gradients w.r.t. parameters\r\n",
        "        optimizer.zero_grad()\r\n",
        "\r\n",
        "        # Forward pass to get output/logits\r\n",
        "        logits, probas = model(images.float()) \r\n",
        "\r\n",
        "        # Calculate Loss: PyTorch implementation of CrossEntropyLoss works with logits, not probabilities\r\n",
        "        loss = F.cross_entropy(logits, labels)\r\n",
        "\r\n",
        "        # Getting gradients w.r.t. parameters\r\n",
        "        loss.backward()\r\n",
        "\r\n",
        "        # Updating parameters\r\n",
        "        optimizer.step()\r\n",
        "\r\n",
        "        iter += 1\r\n",
        "\r\n",
        "        if iter % 500 == 0:\r\n",
        "            # Calculate Accuracy         \r\n",
        "            correct = 0\r\n",
        "            total = 0\r\n",
        "            # Iterate through test dataset\r\n",
        "            for images, labels in test_loader:\r\n",
        "               \r\n",
        "                images = images.view(-1, 28*28).to(device)\r\n",
        "\r\n",
        "                # Forward pass only to get logits/output\r\n",
        "                logits, probas = model(images.float())\r\n",
        "\r\n",
        "                # Get predictions from the maximum value\r\n",
        "                _, predicted = torch.max(probas, 1)\r\n",
        "\r\n",
        "                # Total number of labels\r\n",
        "                total += labels.size(0)\r\n",
        "\r\n",
        "\r\n",
        "                # Total correct predictions\r\n",
        "                if torch.cuda.is_available():\r\n",
        "                    correct += (predicted.cpu() == labels.cpu()).sum() \r\n",
        "                else:\r\n",
        "                    correct += (predicted == labels).sum()\r\n",
        "\r\n",
        "            accuracy = 100 * correct.item() / total\r\n",
        "\r\n",
        "            # Print Loss\r\n",
        "            iteration_loss.append(loss.item())\r\n",
        "            print('Iteration: {}. Loss: {}. Accuracy: {}'.format(iter, loss.item(), round(accuracy, 2)))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Iteration: 500. Loss: 0.6974784731864929. Accuracy: 52.48\n",
            "Iteration: 1000. Loss: 0.6825542449951172. Accuracy: 51.74\n",
            "Iteration: 1500. Loss: 0.7036118507385254. Accuracy: 51.09\n",
            "Iteration: 2000. Loss: 0.7018072605133057. Accuracy: 50.73\n",
            "Iteration: 2500. Loss: 0.6974782943725586. Accuracy: 50.37\n",
            "Iteration: 3000. Loss: 0.692642092704773. Accuracy: 49.4\n",
            "Iteration: 3500. Loss: 0.6929709911346436. Accuracy: 49.43\n",
            "Iteration: 4000. Loss: 0.6840822100639343. Accuracy: 50.02\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iSiMMiXIAob4"
      },
      "source": [
        "**Loss show**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        },
        "id": "9ttfwFcIAob4",
        "outputId": "c2201cc5-64a3-4765-b482-cb4406c3e959"
      },
      "source": [
        "import matplotlib\r\n",
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "print (iteration_loss)\r\n",
        "plt.plot(iteration_loss)\r\n",
        "plt.ylabel('Cross Entropy Loss')\r\n",
        "plt.xlabel('Iteration (in every 500)')\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0.6974784731864929, 0.6825542449951172, 0.7036118507385254, 0.7018072605133057, 0.6974782943725586, 0.692642092704773, 0.6929709911346436, 0.6840822100639343]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9fX4/9eZZLJBFiALIQmLQIABgSCLuCKaSDe1al2qVq17P9pWW7v9unzaT9tfN2v3VkWtWtdabW21BVxQq4CEDDuEnQkBw5YJgYSs5/vH3NRIkzCBTO5Mcp6Px304c3PvnTMxzJn7Xs5bVBVjjDEmXB63AzDGGBNbLHEYY4zpFkscxhhjusUShzHGmG6xxGGMMaZb4t0OoDdkZmbqyJEj3Q7DGGNiyooVK/aratax+/tF4hg5ciSlpaVuh2GMMTFFRHZ2tN+aqowxxnSLJQ5jjDHdYonDGGNMt1jiMMYY0y2WOIwxxnSLJQ5jjDHdYonDGGNMt1jiMK4oC1TzV38ldY3NbodijOmmfjEB0ESfb7ywho3v1zIwMZ6Lpg7jqhkFnJqXjoi4HZox5jgscZheV3u0ifKqWi6aMox4j/CXFbt4almACblpXDk9n0uK8shISXA7TGNMJyxxmF63elcNqnD5afmcU5jFdy6ayEurdvPs8gD/+/f1/PCfG/nIpKFcOaOA00cNweOxuxBjooklDtPr/IFqAKYUZACQnuzlutNHcN3pI1hbWcNzpRW86K/kbyt3M3xwClfOKODy0/LJSUtyM2xjjMMSh+l1/kCQsdkDSU/2/tfPJuWlMykvnW98dAL/XLuHZ96r4KcLyrlvYTlzx2dzxfQCzhufjTfOxnUY4xZLHKZXqSr+iiAXTMju8rgkbxyfLMrnk0X5bN9/hOdKK3h+xS5e3bCXrNRELj8tnyumFzAqc0AvRW6MaWOJw/SqnQfqOHikkaLhg8I+Z1TmAL46bzz3FBfyxsa9PFdawQNvbuX3i7cya9RgrppZwEcm5ZLkjYtg5MaYNpY4TK/yV4T6N6Z1I3G08cZ5KJk4lJKJQ3m/5ih/KdvFs8sruPvZVXz7b+v4ZFEeV0wvYFJeek+HbYxpxxKH6VVlO4MMTIxnTPbAk7rO0PQk/ue8Mdxx7miWbj/As8sreGZ5BY8v2cmkvDSunDGci6YM67AfxRhzcixxmF7lr6hmSkE6cT00xNbjEc4YnckZozP5bl0jf/VX8szyCr7117X84OX1fHRSLlfOKGDmqME2udCYHmKJw/Sa+sYWNuyp5XNzRkfk+hkpCdxw5iiuP2MkaypreGZ5BS+t3M0L/kpOyRzAFTMKuHRaHtmpNqzXmJNhicP0mtW7grS0KkXDMyL6OiLC5PwMJudn8M2PTeCVNe/z7PIAP/rnRn66oJzzx2dz1cwCzhmbRbwN6zWm2yxxmF7jrwgCMLWg+x3jJyolIZ7LT8vn8tPy2bL3MM+VVvCXFbtYuL6KoWlJ/xnWO3xISq/FZEysE1V1O4aImz59upaWlrodRr932xOllL9fy+J7z3M1jsbmVl7fWMUzyyt4a9M+WhXOHDOEK2cMp8SXY8N6jXGIyApVnX7sfrvjML1CVSkLBDl7TKbboZAQ72HepFzmTcpld7Ce51eEhvV+/mk/6clePlmUx1UzCxg/NM3tUI2JSpY4TK+oDNazr7Yh4v0b3TUsI5nPnz+WO88bwztb9/Ps8gqeWhbgj+/uYEpBBldOL+ATU3JJTbJhvca0scRheoU/EOrf6M6M8d7k8Qhnj83i7LFZHDzSyIv+Sp5dHuAbL67h//6xnoumDOPrHx1v5d6NwRKH6SVlgWqSvB7GD011O5TjGjwggZvOGsVnzxzJyoogzy6v4IWySlZWBHnipplkW5Ve08/ZWETTK/yBIJPzM2Jq+KuIUDR8ED+6bDKP3jiDiuo6Lv/DEioO1rkdmjGuip1/xSZmNTS3sH73oajr3+iOM8dk8uTNs6ipb+Ky37/Lpqpat0MyxjWWOEzErdt9iMaW1hMqbBhNioYP4rnbZgNwxQNLWOXMSzGmv7HEYSKubGeoIm5RQezecbQZNzSVP98+m9SkeD790FLe3brf7ZCM6XWWOEzE+SuC5GUk95lO5RFDBvD87WcwLCOZGx5dzqL1VW6HZEyvssRhIm5lIMi0EbHdTHWsnLQknrttNhOGpnL7n1bwon+X2yEZ02sscZiIqjp0lMpgfZ9opjrWoAEJPHnL6cwcOZi7n13F40t2uB2SMb3CEoeJKH/A6d+I4RFVXRmYGM+jN86g2JfDt/+2jt+8vpn+UP/N9G+WOExE+QNBEuI8+Ib13bpPSd44fn/NNC4tyuNnCzfxw1c2WPIwfZrNHDcRVRaoZlJeGonxfbvibHych599agqpSfE89PZ2DtU388NLT+2xlQ6NiSaWOEzENLW0snpXDdeePsLtUHqFxyP870UTSU/28qvXt1Db0MT9V07t80nT9D+WOEzEbNxTS0Nza5/t3+iIiHBPyTjSkr18/+UN1B4t5YHrTiMlwf6pmb7D+jhMxJQ5HeOxPmP8RNx89in85LLJvLNlP9c9/B419U1uh2RMj4lo4hCReSJSLiJbRORrHfz8fhFZ6WybRCTY7mfXi8hmZ7u+3f7TRGSNc81fiYg1Ikcpf6CanLREctP7xsS/7rpiRgG/+fQ0Vu8KctWDS9lX2+B2SMb0iIglDhGJA34LfATwAVeLiK/9Map6t6pOVdWpwK+BF5xzBwPfAWYBM4HviEjb19bfA7cAY51tXqTegzk5/oogRQWD6M+5/aOn5jL/+hns2H+EKx5Ywq5qq6xrYl8k7zhmAltUdZuqNgLPABd3cfzVwNPO4wuBRap6UFWrgUXAPBHJBdJUdamGxjs+DlwSubdgTtT+ww3sPFDHtBH9p3+jM+cWZvGnm2ey/3ADn/rDErbsPex2SMaclEgmjjygot3zXc6+/yIiI4BRwOvHOTfPeRzONW8VkVIRKd23b98JvQFz4lZG+Yp/ve20EYN59tbZNLW0csUDS1hbWeN2SMacsGjpHL8KeF5VW3rqgqr6oKpOV9XpWVlZPXVZEyZ/RTXxHmHSsHS3Q4kavmFp/Pn2M0j2xnH1g0tZtu2A2yEZc0IimTgqgYJ2z/OdfR25ig+aqbo6t9J5HM41jYvKdgaZkJtGcoLNYWhvVOYAnr9jNtlpiXzmkfd4Y+Net0MyptsimTiWA2NFZJSIJBBKDi8de5CIjAcGAUva7V4AlIjIIKdTvARYoKp7gEMicrozmuozwN8i+B7MCWhpVVbtCjKtH83f6I7c9GSeu202Y3MGcsvjpby0arfbIRnTLRFLHKraDNxJKAlsAJ5T1XUi8j0RuajdoVcBz2i74j6qehD4P0LJZznwPWcfwOeA+cAWYCvwz0i9B3NiNlXVUtfYYv0bXRgyMJGnbjmdaSMG8YVn/Dy5bKfbIRkTtohOZ1XVV4BXjtn37WOe/28n5z4CPNLB/lJgUs9FaXqa/z8d43bH0ZW0JC+Pf3Ymn3uyjP/vxbXU1DfxuTlj3A7LmOOKls5x04eUBaoZMiCB4YNT3A4l6iV543jgutO4aMowfvKvcn70z41WWddEPSugY3qcP1BN0fCMfj3xrzu8cR7uv3IqqUnx/OHNrdQebeJ7F0+yyromalniMD2qpq6JrfuOcOm0/OMfbP4jziN8/5JJpCV7+f3irRw62szPr5iCN84aBUz0scRhepS/om+v+BdJIsJX540nLcnLj/+1kcNHm/jdNafZkGYTdezrjOlR/kAQj8DkfEscJ+qOOaP5wScnsXjTPq5/5D0OHbXKuia6WOIwPcpfEaQwJ5WBiXYzezKumTWCX15VRFmgmk8/tJQDh62yrokeljhMj2ltVfyBaqaNsPkbPeGiKcN46DPT2Vx1mCseWMLuYL3bIRkDWOIwPWjb/sPUHm2mqMCaqXrKeeOzeeKmWew9FKqsu33/EbdDMsYSh+k5ZVYRNyJmjhrM07eeTn1TC5/6w7us333I7ZBMP2eJw/QYf6CatKR4Tskc4HYofc6kvHSeu2023jgPVz64hNIdB49/kjERYonD9Bh/IEjR8EF4bOJaRIzJHsifb59N5sBErn14GW9usnVmjDsscZgecbihmfKqWpu/EWH5g1J47rbZjMocyM2PLeeVNXvcDsn0Q5Y4TI9YVRFE1fo3ekNWaiLP3Ho6U/IzuPOpMp5dHnA7JNPPWOIwPcIfCM0Yn2ojqnpFerKXx2+ayZljMvnqX9bw0Fvb3A7J9CPHTRwiMkBEPM7jQhG5SES8kQ/NxBJ/IMiY7IGkJ9ufRm9JSYhn/vXT+eipQ/nBKxv42YJyq6xrekU4dxxvAUkikgcsBK4D/hjJoExsUVX8FUGbv+GCxPg4fn31NK6cXsBv3tjCd15aR2urJQ8TWeHUhRBVrRORm4DfqepPRGRlpAOLFg3NLSTGW5G5ruw8UMfBI402Y9wlcR7hR5edSlpyPA+9vZ3ao8385PLJVlnXRExYiUNEZgPXADc5+/rFJ+n/PFlGTX0Tf7p5ltuhRDWriOs+EeEbH51AerKXny3cRF1jM7/59DRLHiYiwvmr+iLwdeBFZ83wU4A3IhtWdCgYnMLSbQeoqbfqpF3xB4IMTIxnbHaq26H0ayLCnXPH8u2P+1iwroovPbeKFmu2MhFw3MShqm+q6kWq+mOnk3y/qn6+F2JzXbEvh+ZWZXH5XrdDiWplgWqmFKTbinVR4rNnjeIr88bx0qrdfOOFNdbnYXpcOKOqnhKRNBEZAKwF1ovIvZEPzX1FBRlkDkxk4foqt0OJWvWNLWzYU0tRgfVvRJPPzRnDXXPH8GxpBd/7x3obbWV6VDhNVT5VPQRcAvwTGEVoZFWf5/EIxb5sFm/cS0Nzi9vhRKU1lTW0tKr1b0She4oLuemsUfzx3R38xIbqmh4UTuLwOvM2LgFeUtUmoN/8BRb7cjjS2MKSrQfcDiUqldnEv6glInzzYxP49Kzh/H7xVn7z+ha3QzJ9RDiJ4wFgBzAAeEtERgD9pq7zGaMzSUmIs+aqTvgD1YwcksKQgYluh2I6ICJ8/+JJXFqUx32LNjH/bZthbk5eOJ3jv1LVPFX9qIbsBM7rhdiiQpI3jnMLs3h1fZV1Mh5DVSlzKuKa6OXxCD+5fDIfPXUo3395A39autPtkEyMC6dzPF1Efi4ipc52H6G7j36jZGIOe2sbWLUr6HYoUaUyWM++2gbr34gB8XEefnFlEXPHZ/PNv67lLyt2uR2SiWHhNFU9AtQCVzjbIeDRSAYVbc4bl02cR1hkzVUf4ndW/JtmdxwxISHew++umcaZY4Zw7/OreHm1lWQ3JyacxDFaVb+jqtuc7bvAKZEOLJpkpCQwa9Rg6+c4hj8QJMnrYdxQm/gXK5K8cTz0melMGz6ILzzj57UN9jdtui+cxFEvIme1PRGRM4H6yIUUnYp9OWzZe5jt+4+4HUrUKAtUMzkvw8paxJiUhHgeuXEGvmFp3PFkGf/evN/tkEyMCedf/O3Ab0Vkh4jsAH4D3BbRqKJQsS8HgEXr33c5kujQ0NzC+t2HKBph/RuxKC3Jy2M3zuSUzAHc8ngpy20Nc9MN4YyqWqWqU4DJwGRVLQLmRjyyKJM/KAVfbhoL19mtPcC63YdobGm1GeMxbNCABJ64aRa5GUnc+OhyVlXY4A8TnrDbGFT1kDODHOCeCMUT1Uom5rAiUM3+ww1uh+K6sp2hiX/TbERVTMtKTeTJm2cxaICXzzzyHhv29JspWuYknGjjdL+sZlfsy0EV61AE/BVB8jKSyU5LcjsUc5Jy05N56ubTSfbGcd3Dy9iy97DbIZkod6KJo1/OhPPlppGXkWzDcoGVgaDN3+hDCgan8OQtoXVnrp2/jMCBOpcjMtGs08QhIrUicqiDrRYY1osxRg0RodiXw9ub91PX2Ox2OK6pOnSUymC9zRjvY0ZnDeSJm2ZR39TCp+cvZXew3w2eNGHqNHGoaqqqpnWwpapqOCsH9kklvhwamlt5a1P/HcLoD9iKf33VhNw0nrhpJjV1TVw7fxn7aq0/z/w3G4DfTTNGDSY92cvCfjws1x8IkhDnYeKwNLdDMREwOT+DR2+cwZ6ao1w7fxnVRxrdDslEGUsc3eSN8zB3fDavb9xLc0ur2+G4wh8IMjEvjcT4frH0fL80feRg5l8/ne0HjvCZR97j0FFbPtl8IKKJQ0TmiUi5iGwRka91cswVIrJeRNaJyFPt9v9YRNY625Xt9v9RRLaLyEpnmxrJ99CREl8OwbomSp0hqf1JU0srqyuDNn+jHzhzTCZ/uHYaG/Yc4rOPLu/X/Xrmw8KpjnuXiHT7U0JE4oDfAh8BfMDVIuI75pixwNeBM1V1IvBFZ//HgGnAVGAW8GURad8ucq+qTnW2ld2N7WSdU5hFQrynX04G3LinlqNNrUyzGeP9wtzxOfzyqiLKAtXc/FgpR5tsJUwT3h1HDrBcRJ5z7iDCncMxE9jiFEZsBJ4BLj7mmFuA36pqNYCq7nX2+4C3VLVZVY8Aq4F5Yb5uxA1IjOesMZks2vB+v1uO01/R1jFudxz9xccm5/KzT01hybYD3PGnFTQ2988mWvOBcEqOfBMYCzwM3ABsFpEfisjo45yaB1S0e77L2ddeIVAoIu+IyFIRaUsOq4B5IpIiIpmEFo4qaHfeD0RktYjcLyIdLj0nIre2rSGyb9++473Nbiv25VBxsJ6N79f2+LWjWdnOarJTExmWbhP/+pNLp+Xz/Usm8Ub5Pr7wjL/f9u+ZkLD6ODT0tfp9Z2sGBgHPi8hPTvL14wklpTnA1cBDIpKhqguBV4B3gaeBJUDbPfLXgfHADGAw8NVOYn5QVaer6vSsrKyTDPO/nT8hGxH63WRAf0WQacMHEf6Np+krrpk1gm9+bAL/XPs+X3l+ta2I2Y+F08fxBRFZAfwEeAc4VVXvAE4DLuvi1Eo+fJeQ7+xrbxfwkqo2qep2YBOhRIKq/sDpwygmVOJkk7N/j7OEbQOhBaVmhvE+e1x2ahJFBRn9aljugcMN7DxQZ/M3+rGbzz6FLxUX8oK/km/+bW2/a6o1IeHccQwGLlXVC1X1z6raBKCqrcDHuzhvOTBWREaJSAJwFfDSMcf8ldDdBk6TVCGwTUTiRGSIs38yocq8C53nuc5/BbgEWBvOG42EYt9Q1lYe6jczbNtW/LP+jf7tzrljuGPOaJ5aFuD7L2+w5NEPhdPH8R1giIh83hlhNa3dzzZ0cV4zcCewANgAPKeq60TkeyJykXPYAuCAiKwH3iA0WuoA4AXedvY/CFzrXA/gSRFZA6wBMoHvd/M995iSiW1rdPSP5ip/RTXxHuHUvHS3QzEuEhG+cuE4bjhjJA//ezs/X7TJ7ZBMLztu6RAR+RahtcZfcHY9KiJ/VtXjfmCr6iuE+ira7/t2u8dKqET7Pcccc5TQyKqOrhk1a4GMzhrIKVkDWLS+iuvPGOl2OBHnDwSZkJtGcoJN/OvvRIRvf9zH0aYWfv36FpK8cfzPeWPcDsv0knBqTl0LTHE+zBGRHwErcfGbfjQp8Q1l/tvbqKlvIj3Z63Y4EdPSqqyqCHLZafluh2KihMcj/OCTp1Lf1MJPF5ST7I3js2eNcjss0wvC6ePYDbQfe5nIf3dy91vFvhyaW5XF5XuPf3AM21RVy5HGFusYNx8S5xHu+9QULpyYw/f+sZ5n3gu4HZLpBeEkjhpgnVPq41FCndFBEfmViPwqsuFFv6KCDDIHJrKwj/dztHWMT7OOcXOM+DgPv7q6iHMLs/j6i2v4q9++V/Z14TRVvehsbRZHJpTY5PEIxb5sXlq5m4bmlj5b+M8fqGbwgASGD05xOxQThRLj43jgutO44dH3+NKfV5HkjWPepKFuh2UiJJxRVY8RmoS3wtmeUtXH2rZIBxgLSnxDOdLYwpKtB9wOJWLKAtUUFWTYxD/TqSRvHPOvn8Hk/HTuerqMN/p4821/Fs4EwDnAZkIFC38HbBKRcyIcV0yZPXoIKQlxfba5qqauia37jjBthDVTma4NTIznjzfOpDAnldufWMG7W/vvgmd9WTh9HPcBJap6rqqeA1wI3B/ZsGJLkjeOcwuzeHV9VZ8sw7BylzPxr8A6xs3xpSd7eeKmWYwYksLNj5WyYudBt0MyPSycxOFV1fK2J6q6idAEPdNOycQc9tY2sMr5kO1LynZW4xGYbInDhGnwgAT+dNMsslMTueGR5aytrHE7JNODwkkcK0RkvojMcbaHgNJIBxZrzhuXTZxH+uQscn9FkMKcVAYm9tul5s0JyE5L4slbTict2ct1Dy+jvJ9Vku7LwkkctwPrgc8723rgjkgGFYsyUhKYNWpwn0scra3KykC11acyJyQvI5mnbpmFN87DNfOXsX3/EbdDMj2gy8ThrOK3SlV/rqqXOtv9TmVac4xiXw6b9x7uU/84tu0/zKGjzTbxz5ywEUMG8NQts1BVrnloKRUH69wOyZykLhOHqrYA5SIyvJfiiWnFvraih32n1HrZfyb+WeIwJ25MdipP3DSLww3NXDN/Ge/XHHU7JHMSwmm0HkRo5vh7wH++SqvqRZ2f0j/lD0rBl5vGwnVV3HrO8RZIjA3+QJC0pHhOyRzodigmxvmGpfHYZ2dy7fxlXDN/Kc/eNpvMgR0u4OmahuYWauqaCNY3EaxrIljXSLC+iZq6JqrrGqmpb+ICXw7njct2O1RXhZM4vhXxKPqQkok5/PK1zew/3BB1/yhOhD9QzdThg/B4bOKfOXlFwwfxyA0zuP7R97ju4fd4+pZZZKQk9PjrHG1qoabdh391XRM19Y2h5+2TgvO8xkkQdY0tnV4z3iPEeYR/rX2fN79yXr8eLBLOO/+oqn5oeVYR+THwZmRCim3Fvhx+8epmXttQxZUzYruF73BDM+VVtVw40UpHmJ4z65QhPHjddG5+rJTrH13On26aSWpSxyP8jza1UN32Ad/uw7+6rolgfWPo7sB5HGz3+GhT52uix3uEjJQEMlK8ZCR7yctIZuKwNDKSvWSkeElPSWBQipeM5NAx6c7+gYnxrNpVwyW/fYdH/72du84fG6lfUdQLJ3EU89/ren+kg30G8OWmkZeRzKL1sZ84VlcEUcVmjJsed05hFr/5dBF3PFnGNfOXMWFoWihBOM1CbYmgobnzBOCNcxJAspdBKQkUDE7h1LzQh/wHieHDH/6DUhJISYg74dI5UwsyKPbl8ODb2/jM7JGkp/TPKW2dJg4RuQP4HHCKiKxu96NU4N1IBxarRIRiXw5PvxegrrGZlITYvZ0tC1QDMDXfOsZNzyuZOJRfXDmV7/59HVWHjpKRnEB6ipcRQ1KYkpLOoJTQ87YP/9AdgfM4xUuy98QTwMn4UkkhH/nl2zzw1la+Mm98r79+NOjqU+0p4J/A/w98rd3+WlW1GgJdKPHl8Md3d/DWpv0xXSHUHwgyOmtAv/1WZSLvE1OG8Ykpw9wOo1vGD03joinDePSdHdxw5kiyU5OOf1If0+lwXFWtUdUdqno1sAtoAhQYaMNzuzZj1GDSk70xPRlQVfFXBG39DWM6cPcFhTS2tPK7N7a6HYorwqmOeydQBSwCXna2f0Q4rpjmjfMwd3w2r22sorml8zbaaLbzQB0HjzTajHFjOjAycwCfOi2fp5YFqAzWux1Orwun5MgXgXGqOlFVT3W2yZEOLNaV+HII1jVRurPa7VBOiL8iFLfNGDemY593RlX96tXNLkfS+8JJHBWElo813XBOYRYJ8R4WrovN5ip/IMiAhDgKc1LdDsWYqDQsI5lrTh/O82W72LbvsNvh9KpwEsc2YLGIfF1E7mnbIh1YrBuQGM9ZYzJZtOF9VGNvjQ5/IMiUggzibOKfMZ363JwxJMZ7uL+f3XWEkzgChPo3EggNxW3bzHEU+3KoOFjPxhgrJ13f2MKGPYesmcqY48hKTeTGM0fy91W7Wb/7kNvh9JrjTjJQ1e8eu09EYndyQi86f0I2IrBofRUTctPcDidsaypraG5VigqsY9yY47n17NE8vmQnP19UzvzrZ7gdTq/o9I5DRP7d7vETx/z4vYhF1IdkpyZRVJDBwhirlusPWMe4MeFKT/Fy+7mjeXXD3v9Mmu3rumqqGtDu8aRjfmYN32Eq9g1lbeUhdsfQkL2yQDUjhqQwpA8UaTSmN9xwxkgyByZw38Ly4x/cB3SVOLSTxx09N50omRhao+PVDbExukpVKQsEKbL1xY0J24DEeD43ZwzvbDnAu1v2ux1OxHWVODJE5JMicpnz+FJnuwxI76X4Yt7orIGckjUgZobl7q45yr7aBitsaEw3fXrWcHLTk/jpwvKYHEnZHV0ljjeBi4CPO48/4WwfB96KfGh9R4lvKEu3HaCmvsntUI6rzJmwaB3jxnRPkjeOz58/Fn8gyGsb9rodTkR1OjpKVW/szUD6smJfDn94cyuLy/dy8dQ8t8Ppkj8QJMnrYXyujbg2prsuPy2fB97cys8WljN3fHafXQAtnHkc5iQVFWSQOTCRhTFQ9NBfUc3kvAy8cfanYUx3eeM83F1cyMb3a3l5zR63w4kY+3ToBR6PUOzLZvHGvTQ0d740pdsamltYV2kT/4w5GZ+YPIzxQ1O5f9GmmC1yejyWOHpJiW8oRxpbWLL1gNuhdGrd7kM0trRa4jDmJHg8wj3FhWzbf4QXyirdDiciwimr/ikRSXUef1NEXhCRaZEPrW+ZPXoIKQlxUd1c5Q8EAayUujEnqdiXw5SCDH752uaobmU4UeHccXxLVWtF5CzgAuBh4PeRDavvSfLGMWdcFq+ur6K1NTqH6pUFqsnLSCYnrf+taGZMTxIR7i0ZR2WwnqeXBdwOp8eFkzja0uXHgAdV9WVCBQ9NNxX7cthb28DqyuisUr8yEGSqNVMZ0yPOHDOE008ZzG/e2EpdY7Pb4fSocBJHpYg8AFwJvCIiiWGeZ45x3rhs4jzCwnXRV7uq6tBRKoP1NmPcmB4iItx74Tj2H27gj+/ucDucHhVOArgCWABcqKpBYDBwbzgXF5F5IlIuIltE5GudHHOFiKwXkXUi8lS7/T8WkbXOdmW7/WLjHssAABjXSURBVKNEZJlzzWdFJGbufjJSEpg1anBUrkXe1r9hM8aN6TmnjRjM3PHZPPDmtpiYAByucBJHLvCyqm4WkTnApwijOq6IxAG/BT4C+ICrRcR3zDFjga8DZ6rqRELL1CIiHwOmAVOBWcCXRaStLvmPgftVdQxQDdwUxnuIGsW+HDbvPcz2/UfcDuVD/IFqEuI8TBwWO+XfjYkFXyoppKa+iflvb3M7lB4TTuL4C9AiImOAB4EC4KmuTwFgJrBFVbepaiPwDHDxMcfcAvxWVasBVLVtnr4PeEtVm1X1CLAamCciAswFnneOewy4JIxYokaxL1T0cFGUlVr3B4L4hqWRGB/ndijG9CkTh6Xzscm5PPLv7Rw43OB2OD0inMTRqqrNwKXAr1X1XkJ3IceTR2i98ja7nH3tFQKFIvKOiCwVkXnO/lWEEkWKiGQC5xFKWEOAoBNPZ9cEQERuFZFSESndt29fGOH2jvxBKfhy06Kq6GFTSyurK4NMs2G4xkTE3RcUUt/Uwu8Xb3U7lB4RTuJoEpGrgc8A/3D2eXvo9eOBscAc4GrgIRHJUNWFwCvAu8DTwBI+GN0VFlV9UFWnq+r0rKysHgq3Z5RMzGFFoJr9UfLtY+OeWo422cQ/YyJlTPZALpuWz+NLd7KnJnbW5ulMOInjRmA28ANV3S4io4BjVwTsSCWhu4Q2+c6+9nYBL6lqk6puBzYRSiSo6g9UdaqqFhNaOGoTcIBQiff4Lq4Z9Yp9OajCa1GyRoe/wlb8MybSPn/+WFSVX7++xe1QTtpxE4eqrge+DKwRkUnALlX9cRjXXg6MdUZBJQBXAS8dc8xfCd1t4DRJFQLbRCRORIY4+ycDk4GFGipy/wZwuXP+9cDfwoglqvhy08jLSI6a0VX+QJDs1ETyMpLdDsWYPqtgcApXzxzOc8sr2HkgugbHdFc4JUfmAJsJjZD6HbBJRM453nlOP8SdhIbybgCeU9V1IvI9EbnIOWwBcEBE1hNKCPeq6gFCTWFvO/sfBK5t16/xVeAeEdlCqM/j4bDfbZQQEYp9Oby9eX9UTAwqC1RTNDyD0NgDY0yk3HneGOLjhF+8utntUE5Kp+txtHMfUKKq5QAiUkio3+G0452oqq8Q6qtov+/b7R4rcI+ztT/mKKGRVR1dcxuhEVsxrcSXwx/f3cFbm/Yzb9JQ1+I4cLiBnQfquHrmcNdiMKa/yE5L4vozRvLgW9u4Y85oCnNic92bcPo4vG1JA0BVN9FzneP91oxRg0lP9rreXLWywilsaDPGjekVt58zmoEJ8dy3sPz4B0epcBLHChGZLyJznO0hoDTSgfV13jgPc8dn89rGKldr9pcFqonzCJPzLXEY0xsGDUjg5rNPYcG6KlY5X9xiTTiJ43ZgPfB5Z1sP3BHJoPqLEl8OwbomSp11vt3gDwSZkJtKcoJN/DOmt3z2rJEMSvHysxi96+gycThlQ1ap6s9V9VJnu19Vo2MCQow7pzCLhHiPa5MBW1qVVRVBigps4p8xvSk1ycvn5ozh7c37Wbotehd360yXiUNVW4ByEbGe0wgYkBjPWWMyWbThfULjBHrXpqpajjS2MG2ENVMZ09uumz2CnLREfrag3JV//ycjnKaqQcA6EXlNRF5q2yIdWH9R7Muh4mA9G9+v7fXX/s+Kf3bHYUyvS/LGcdfcsZTurGbxpugpixSOcIbjfiviUfRj50/IRgQWra9iQm7vVqb1B6oZPCCBEUNSevV1jTEhV0wv4IG3tvKzBeWcOzYLjyc25lJ1eschImNE5ExVfbP9Rqhm1K7eC7Fvy05Noqggw5VhuWWBaooKbOKfMW5JiPfwxfMLWbf7EP+KwgXeOtNVU9UvgEMd7K9xfmZ6SMnEoayprGF3sPeKn9XUNbF13xGrT2WMyy4pymNM9kDuW1hOS2ts9HV0lThyVHXNsTudfSMjFlE/1LZGx6u9WPRw5S6nf8NKqRvjqjiP8KXiQrbuO8KL/tio2dpV4ujqq6hVw+tBo7MGckrWgF4dlusPVCMCU2zGuDGumzdpKKfmpfOLVzfR2OzehOBwdZU4SkXklmN3isjNwIrIhdQ/lfiGsnTbgV5bl7gsEGRcTioDE8MZH2GMiSQR4UslheyqrufZ0orjn+CyrhLHF4EbRWSxiNznbG8SWuP7C70TXv9R7MuhuVVZXL73+AefpNZWZaVTEdcYEx3OLcxixshB/Pq1zdQ3dmvdul7XaeJQ1SpVPQP4LrDD2b6rqrNVNXa6/2NEUUEGmQMTWdgLo6u27T/CoaPNNn/DmCgiItx74Xj21jbwxNIdbofTpeO2U6jqG4TWyjAR5PEIxb5sXlq5m4bmFhLjI1c7qiwQqo1lM8aNiS4zRw3mnMIsfrd4K1fPHE5qUnQWIg9n5rjpJSW+oRxpbGHJ1sjWrvEHgqQmxXNK5sCIvo4xpvvuLRlHsK6Jh/+93e1QOmWJI4rMHj2ElIS4iE8G9AeqmVqQETOzVI3pT07NT2fexKHMf3s71Uca3Q6nQ5Y4okiSN44547JYtL6K1ghNBDrc0Mymqlqm2fwNY6LWPSWFHGls5g9vbnU7lA5Z4ogyxb4c9tY2sLqyJiLXX10RpFWxEVXGRLHCnFQ+OTWPx5bsoOrQUbfD+S+WOKLMeeOyifMICyNUt8ZfYRVxjYkFX7ygkOYW5Tevb3E7lP9iiSPKZKQkMGvU4Ij1c5TtrGZ01gDSU6JztIYxJmT4kBSunFHAM8sDVBysczucD7HEEYWKfTls3nuY7fuP9Oh1VRV/RdDqUxkTI+6aOxaPCL98bbPboXyIJY4o1Fb0cNH6nm2uChys4+CRRuvfMCZGDE1P4rrTR/BC2S627O39xd46Y4kjCuUPSsGXm9bjRQ//M/HP7jiMiRl3zBlNsjeO+xdFz12HJY4oVTIxhxWBavYfbuixa/oDQVIS4ijMSe2xaxpjImvIwERuOmsUL6/Zw9oIjbbsLkscUarYl4MqvL6h54oe+gNBpuRnEGcT/4yJKTefcwrpyV7uW1judiiAJY6o5ctNIy8jmYU91M9R39jChj2HrH/DmBiUluTl9nNH80b5Pkp3HHQ7HEsc0UpEKPbl8Pbm/dQ1Np/09dZU1tDcqta/YUyMuv6MEWQOTOSnC8pRdXeJWUscUaxkYg4Nza28tWn/SV/L73SMT7U7DmNiUkpCPHfNHcOy7Qf595aT/0w4GZY4otjMkYNJT/b2yGRAfyDI8MEpZA5M7IHIjDFuuGpmAXkZya7fdVjiiGLxcR7mjs/mtY1VNLec+DrEqkpZoJppdrdhTExLjI/jCxeMZfWuml5Z9K0zljiiXIkvh2BdE6U7q0/4GrtrjrK3tsFmjBvTB1xalMcpWQP4+cJNtESoivbxWOKIcucUZpEQ7zmpyYBt/Rs2osqY2Bcf5+HuCwopr6rl76t2uxKDJY4oNyAxnrPGZLJow/sn3KZZtjNIYryHCblpPRydMcYNHzs1lwm5adz/6iaaTqIZ+0RZ4ogBxb4cKg7WU151YrVq/BXVTM5Pxxtn/7uN6Qs8HuHLJYXsPFDHn0t39f7r9/ormm47f0I2IpxQc1VDcwvrKg9Z/4Yxfczc8dlMG57Br1/fzNGmll59bUscMSA7NYmigowTGpa7bvchGltabUSVMX2MiPDlC8exp+YoTy4L9OprW+KIESUTh7KmsobdwfpunecPOCv+2R2HMX3OGaMzOWtMJr97YwtHGk6+wkS4Ipo4RGSeiJSLyBYR+Vonx1whIutFZJ2IPNVu/0+cfRtE5FciIs7+xc41VzpbdiTfQ7RoW6Pj1Q3du+vwB6oZlp5ETlpSJMIyxrjsyxeO48CRRh59Z3uvvWbEEoeIxAG/BT4C+ICrRcR3zDFjga8DZ6rqROCLzv4zgDOBycAkYAZwbrtTr1HVqc7Wc+Vjo9jorIGMzhrQ7X4Of8BW/DOmL5takMEFE3J44K1t1NQ19cprRvKOYyawRVW3qWoj8Axw8THH3AL8VlWrAdolAQWSgAQgEfAC7k2TjBLFvqEs3XaAmvrw/jiqDh2lMlhv8zeM6eO+VFLI4YZmHnhra6+8XiQTRx5Q0e75Lmdfe4VAoYi8IyJLRWQegKouAd4A9jjbAlXd0O68R51mqm+1NWEdS0RuFZFSESndt29fT70nVxX7cmhuVRaXh3eTZf0bxvQPE3LT+MTkYTz6zg721fbc4m+dcbtzPB4YC8wBrgYeEpEMERkDTADyCSWbuSJytnPONap6KnC2s13X0YVV9UFVna6q07OysiL8NnpHUUEGmQMTw65R4w9U440TJg6ziX/G9HV3FxfS2NLKb9/YEvHXimTiqAQK2j3Pd/a1twt4SVWbVHU7sIlQIvkksFRVD6vqYeCfwGwAVa10/lsLPEWoSaxf8HiEYl82b5bvo6H5+OO2/YEgE4elk+SN64XojDFuGpU5gE+dls9TywJUdnP0ZXdFMnEsB8aKyCgRSQCuAl465pi/ErrbQEQyCTVdbQMCwLkiEi8iXkId4xuc55nO8V7g48DaCL6HqFPiG8rhhmaWbD3Q5XFNLa2srgxa/4Yx/chd548F4Fevbo7o60QscahqM3AnsADYADynqutE5HsicpFz2ALggIisJ9Snca+qHgCeB7YCa4BVwCpV/TuhjvIFIrIaWEnoDuahSL2HaDR79BBSEuKOOxmw/P1ajja1Wv+GMf1IXkYy15w+nOfLdrF9/5GIvU58xK4MqOorwCvH7Pt2u8cK3ONs7Y9pAW7r4HpHgNMiEmyMSPLGMWdcFovWV/F/F0/C4+lwbABlTkVcmzFuTP/yuTljeOa9Cu5ftIlfXV0Ukddwu3PcnIBiXw57axtYXVnT6TH+QJCs1ETyMpJ7MTJjjNuyUhP57FkjeWnVbjbsORSR17DEEYPmjsshziMsXPd+p8f4A9UUFWTQyWhlY0wfduvZo0lNiue+hZsicn1LHDEoPcXLrFGDO+3nOHC4gR0H6pg2wvo3jOmP0lO83HbOKby6oYrVu4I9fn1LHDGq2JfD5r2HO+wAW1nhTPwrsP4NY/qrG88cxc+vmMLEYek9fm1LHDGqrejhovX/3VzlDwSJ8win5vf8H4wxJjYMSIzn0mn5xHUygOZkWOKIUfmDUvDlpnXYXFUWqGb80FRSEiI6aM4Y009Z4ohhJRNzKN1Zzf7DH9SmaWlVVlUEmWbzN4wxEWKJI4YV+3JQhdc3fFD0cPPeWo40ttiMcWNMxFjiiGG+3DTyMpJZ2K6fo2ynVcQ1xkSWJY4YJiIU+3J4e/N+6hpDy0b6A9UMSvEyckiKy9EZY/oqSxwxrmRiDg3Nrby1aT8A/orQin828c8YEymWOGLczJGDSU/2smh9FTV1TWzZe9jmbxhjIsoSR4yLj/Nw/vhsXttYxYrAQQCbMW6MiShLHH1AsS+HYF0TD721HRGYbBP/jDERZImjDzinMIuEeA9Lth2gMDuV1CSv2yEZY/owSxx9wIDEeM4akwlg8zeMMRFniaOPaKtdZTPGjTGRZsWM+ohPTBnG5qrDXDhxqNuhGGP6OEscfcTAxHi+/Qmf22EYY/oBa6oyxhjTLZY4jDHGdIslDmOMMd1iicMYY0y3WOIwxhjTLZY4jDHGdIslDmOMMd1iicMYY0y3iKq6HUPEicg+YOcJnp4J7O/BcCItluK1WCMnluKNpVghtuI92VhHqGrWsTv7ReI4GSJSqqrT3Y4jXLEUr8UaObEUbyzFCrEVb6RitaYqY4wx3WKJwxhjTLdY4ji+B90OoJtiKV6LNXJiKd5YihViK96IxGp9HMYYY7rF7jiMMcZ0iyUOY4wx3WKJowsiMk9EykVki4h8ze14OiMij4jIXhFZ63Ys4RCRAhF5Q0TWi8g6EfmC2zF1RkSSROQ9EVnlxPpdt2M6HhGJExG/iPzD7ViOR0R2iMgaEVkpIqVux9MVEckQkedFZKOIbBCR2W7H1BkRGef8Ttu2QyLyxR67vvVxdExE4oBNQDGwC1gOXK2q610NrAMicg5wGHhcVSe5Hc/xiEgukKuqZSKSCqwALonS360AA1T1sIh4gX8DX1DVpS6H1ikRuQeYDqSp6sfdjqcrIrIDmK6qUT+hTkQeA95W1fkikgCkqGrQ7biOx/ksqwRmqeqJToT+ELvj6NxMYIuqblPVRuAZ4GKXY+qQqr4FHHQ7jnCp6h5VLXMe1wIbgDx3o+qYhhx2nnqdLWq/bYlIPvAxYL7bsfQlIpIOnAM8DKCqjbGQNBznA1t7KmmAJY6u5AEV7Z7vIko/3GKZiIwEioBl7kbSOafpZyWwF1ikqlEbK/AL4CtAq9uBhEmBhSKyQkRudTuYLowC9gGPOs2A80VkgNtBhekq4OmevKAlDuMaERkI/AX4oqoecjuezqhqi6pOBfKBmSISlc2BIvJxYK+qrnA7lm44S1WnAR8B/sdpdo1G8cA04PeqWgQcAaK237ON06R2EfDnnryuJY7OVQIF7Z7nO/tMD3D6C/4CPKmqL7gdTzicpok3gHlux9KJM4GLnH6DZ4C5IvInd0PqmqpWOv/dC7xIqIk4Gu0CdrW723yeUCKJdh8BylS1qicvaomjc8uBsSIyysnaVwEvuRxTn+B0OD8MbFDVn7sdT1dEJEtEMpzHyYQGS2x0N6qOqerXVTVfVUcS+nt9XVWvdTmsTonIAGdwBE6zTwkQlSMDVfV9oEJExjm7zgeibjBHB66mh5upIHT7ZTqgqs0iciewAIgDHlHVdS6H1SEReRqYA2SKyC7gO6r6sLtRdelM4DpgjdN3APANVX3FxZg6kws85oxM8QDPqWrUD3ONETnAi6HvEcQDT6nqv9wNqUt3AU86XyS3ATe6HE+XnGRcDNzW49e24bjGGGO6w5qqjDHGdIslDmOMMd1iicMYY0y3WOIwxhjTLZY4jDHGdIslDhO1ROSw89+RIvLpHr72N455/m4PXvsXbTOgndIUvp66dqSJSEu7iqovtds/SkSWOZWin3WGpCIiic7zLc7PRzr7TxWRP7ryJkzEWeIwsWAk0K3EISLHm6P0ocShqmd0M6bOXncIcLpTeBJVvdntqr9h/C7aq1fVqc52Ubv9PwbuV9UxQDVwk7P/JqDa2X+/cxyqugbIF5HhJ/8OTLSxxGFiwY+As51vwXc7RQd/KiLLRWS1iNwGICJzRORt55vyemffX50CeuvaiuiJyI+AZOd6Tzr72u5uxLn2WmediCvbXXtxu/UYnnRmwB/rMuA/k9icc6a3vYaI/EBCa3ssFZGcY092ZlM/IqE1QPwicrGzf6mITDz2ul0cf4OIvCQirwOvicjjInJJu/OfbDv2eJz3OZdQmQ2Ax4C2a13sPMf5+fntfi9/JzSD3fQ1qmqbbVG5AYed/84B/tFu/63AN53HiUApoeqlcwgVnxvV7tjBzn+TCZWzGNL+2h281mXAIkLVAnKAAKHZ43OAGkI1yzzAEkIF+o6N+THgE+2eLya03gSEKsF+wnn8k7b3cMz5PwSudR5nEFoTZgBwN/BdZ38uUH6c428gVF+p7f2fC/zVeZwObAfiO3j9Zuf3uZTQGikAmYSWGGg7pgBY6zxeC+S3+9lWINN5fCbwd7f/jmzr+c3uOEwsKgE+45QrWQYMAcY6P3tPVbe3O/bzIrKK0AdhQbvjOnMW8LSGKuJWAW8CM9pde5eqtgIrCTWhHSuXUPntjjQCbeVKVnRyfgnwNee9LQaSgOHAc8DlzjFX8MG3/86Oh1AJ+IMAqvomodprWYTqF/1FVZs7eP0RqjqdUNPgL0RkdCfvJRx7gWEncb6JUlarysQiAe5S1QUf2ikyh9AdR/vnFwCzVbVORBYT+mA9UQ3tHrfQ8b+f+i5eo0lV22r8dHa+AJepavl//UDkgIhMBq4Ebu/qeBGZRbvfheNx4FpCzUcd1lnSD6rVbnN+X0WEqhhniEi8k2zaV4puqyK9y+lLSQcOOD9LIvT7MH2M3XGYWFALpLZ7vgC4Q0Kl2RGRQul4UZ10Qh23dSIyHji93c+a2s4/xtvAlU4/ShahVd/e60asG4Ax3Tj+WAuAu9r6CUSkqN3PniW0SFO6qq4O4/hj/RH4IoB20GEvIoNEJNF5nEmoqWm9k+ze4IM7nuuBvzmPX3Ke4/z89XbJsZAorXZrTo4lDhMLVgMtTqfy3YSWRV0PlInIWuABOv72/i8gXkQ2EOpgb79O+IPA6rbO8XZedF5vFfA68BUNldQO18uE+kNO1P8RWp52tYisc563eZ7Q3cJzYR7/IU7T2wbg0U4OmQCUOk17bwA/apdgvgrcIyJbCDUNtlVffhgY4uy/hw8vbnQeod+H6WOsOq4xPUxE/g18XKNsTWoRSQHWANNUtSbCr5VIqH/orE76UkwMszsOY3rel/iggzoqiMgFhO42fh3ppOEYDnzNkkbfZHccxhhjusXuOIwxxnSLJQ5jjDHdYonDGGNMt1jiMMYY0y2WOIwxxnTL/wP9gM3kyX3rBgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sAdwlpxqAutv"
      },
      "source": [
        "#**Experiment 3:**\r\n",
        "* **Optimizer =SGD**\r\n",
        "* **Learning rate =0.0010**\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WSbolJOBAutw"
      },
      "source": [
        "**Hyperparameters**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "o0OddHuqAutw"
      },
      "source": [
        "#HYPERPERAMETERS\r\n",
        "batch_size = 256\r\n",
        "num_iters = 4000\r\n",
        "input_dim = 28*28 # num_features = 784\r\n",
        "output_dim = 2\r\n",
        "\r\n",
        "learning_rate = 0.0010\r\n",
        "\r\n",
        "# Device\r\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9Ivk0r1XAutx"
      },
      "source": [
        "**MAKING DATASET ITERABLE**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EImogF8DAutx"
      },
      "source": [
        "'''\r\n",
        "MAKING DATASET ITERABLE\r\n",
        "'''\r\n",
        "num_epochs = num_iters / (len(train_dataset) / batch_size)\r\n",
        "num_epochs = int(num_epochs)\r\n",
        "\r\n",
        "train_loader = torch.utils.data.DataLoader(dataset=train_dataset, \r\n",
        "                                           batch_size=batch_size, \r\n",
        "                                           shuffle=True)  \r\n",
        "\r\n",
        "test_loader = torch.utils.data.DataLoader(dataset=test_dataset, \r\n",
        "                                          batch_size=batch_size, \r\n",
        "                                          shuffle=False) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "P1sHoFYEAutx"
      },
      "source": [
        "**Logistic Regression Model**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uil1d0OZAutx"
      },
      "source": [
        "class LogisticRegressionModel(nn.Module):\r\n",
        "    def __init__(self, input_size, num_classes):\r\n",
        "        super().__init__()\r\n",
        "        self.linear = nn.Linear(input_size, num_classes)\r\n",
        "\r\n",
        "    def forward(self, x):\r\n",
        "        logits  = self.linear(x)\r\n",
        "        probas = F.softmax(logits, dim=1)\r\n",
        "        return logits, probas"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pQk5N5QVAuty"
      },
      "source": [
        "**INSTANTIATE MODEL CLASS**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UnKasIZ2Auty",
        "outputId": "14d59915-0d42-4079-b822-3458d2fc9299"
      },
      "source": [
        "'''\r\n",
        "INSTANTIATE MODEL CLASS\r\n",
        "'''\r\n",
        "model = LogisticRegressionModel(input_size=input_dim,\r\n",
        "                                num_classes=output_dim)\r\n",
        "# To enable GPU\r\n",
        "model.to(device)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegressionModel(\n",
              "  (linear): Linear(in_features=784, out_features=2, bias=True)\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RHAkeu2SAuty"
      },
      "source": [
        "**Optimizer setup**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WB6fhw8JAuty"
      },
      "source": [
        "# INSTANTIATE OPTIMIZER CLASS\r\n",
        "optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate)\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M66Mn4ZtAutz"
      },
      "source": [
        "**Function TRAIN THE MODEL**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CCpzGa9LAutz",
        "outputId": "fc13c735-1f2a-471c-efd7-bb6a4b8a2209"
      },
      "source": [
        "'''\r\n",
        "TRAIN THE MODEL\r\n",
        "'''\r\n",
        "iteration_loss = []\r\n",
        "iter = 0\r\n",
        "for epoch in range(num_epochs):\r\n",
        "    for i, (images, labels) in enumerate(train_loader):\r\n",
        "\r\n",
        "        \r\n",
        "        images = images.view(-1, 28*28).to(device)\r\n",
        "        labels = labels.to(device)\r\n",
        "\r\n",
        "        # Clear gradients w.r.t. parameters\r\n",
        "        optimizer.zero_grad()\r\n",
        "\r\n",
        "        # Forward pass to get output/logits\r\n",
        "        logits, probas = model(images.float()) \r\n",
        "\r\n",
        "        # Calculate Loss: PyTorch implementation of CrossEntropyLoss works with logits, not probabilities\r\n",
        "        loss = F.cross_entropy(logits, labels)\r\n",
        "\r\n",
        "        # Getting gradients w.r.t. parameters\r\n",
        "        loss.backward()\r\n",
        "\r\n",
        "        # Updating parameters\r\n",
        "        optimizer.step()\r\n",
        "\r\n",
        "        iter += 1\r\n",
        "\r\n",
        "        if iter % 500 == 0:\r\n",
        "            # Calculate Accuracy         \r\n",
        "            correct = 0\r\n",
        "            total = 0\r\n",
        "            # Iterate through test dataset\r\n",
        "            for images, labels in test_loader:\r\n",
        "               \r\n",
        "                images = images.view(-1, 28*28).to(device)\r\n",
        "\r\n",
        "                # Forward pass only to get logits/output\r\n",
        "                logits, probas = model(images.float())\r\n",
        "\r\n",
        "                # Get predictions from the maximum value\r\n",
        "                _, predicted = torch.max(probas, 1)\r\n",
        "\r\n",
        "                # Total number of labels\r\n",
        "                total += labels.size(0)\r\n",
        "\r\n",
        "\r\n",
        "                # Total correct predictions\r\n",
        "                if torch.cuda.is_available():\r\n",
        "                    correct += (predicted.cpu() == labels.cpu()).sum() \r\n",
        "                else:\r\n",
        "                    correct += (predicted == labels).sum()\r\n",
        "\r\n",
        "            accuracy = 100 * correct.item() / total\r\n",
        "\r\n",
        "            # Print Loss\r\n",
        "            iteration_loss.append(loss.item())\r\n",
        "            print('Iteration: {}. Loss: {}. Accuracy: {}'.format(iter, loss.item(), round(accuracy, 2)))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Iteration: 500. Loss: 0.6841294765472412. Accuracy: 50.92\n",
            "Iteration: 1000. Loss: 0.6837232112884521. Accuracy: 49.72\n",
            "Iteration: 1500. Loss: 0.6935368180274963. Accuracy: 49.59\n",
            "Iteration: 2000. Loss: 0.6817538738250732. Accuracy: 50.11\n",
            "Iteration: 2500. Loss: 0.6724648475646973. Accuracy: 49.43\n",
            "Iteration: 3000. Loss: 0.6770821213722229. Accuracy: 50.05\n",
            "Iteration: 3500. Loss: 0.6774747371673584. Accuracy: 50.41\n",
            "Iteration: 4000. Loss: 0.676094114780426. Accuracy: 49.92\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oe_Oo7bYAutz"
      },
      "source": [
        "**Loss show**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        },
        "id": "YFvoJcv3Autz",
        "outputId": "32bdec89-3f4d-4756-dc6d-187bbf53c1fb"
      },
      "source": [
        "import matplotlib\r\n",
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "print (iteration_loss)\r\n",
        "plt.plot(iteration_loss)\r\n",
        "plt.ylabel('Cross Entropy Loss')\r\n",
        "plt.xlabel('Iteration (in every 500)')\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0.6841294765472412, 0.6837232112884521, 0.6935368180274963, 0.6817538738250732, 0.6724648475646973, 0.6770821213722229, 0.6774747371673584, 0.676094114780426]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3wc9Z3/8ddH3SqWbRU3SZbcMSU2yAYXsDAlpoQasHwQSgjFHCGEhARyyS+X3OUucHchRwKEFiA5wKZjWkziSjNY7kUuki1syUW2XOUiWdrP748Z4UVI8sra1eyuPs/HYx7anZ2Z/awe9r71/X5n5iuqijHGGBOoGK8LMMYYE1ksOIwxxrSLBYcxxph2seAwxhjTLhYcxhhj2iXO6wI6Q2Zmpubn53tdhjHGRJTFixfvUtWs5uu7RHDk5+dTUlLidRnGGBNRROSLltZbV5Uxxph2seAwxhjTLhYcxhhj2sWCwxhjTLtYcBhjjGkXCw5jjDHtYsFhjDGmXSw4jCcWVexmUcVur8swxpwACw7T6Q7XN3L7Xxfzzy8soaHR53U5xph2suAwnW7Gos3sPlhP9YE65q7b6XU5xph2suAwnepoo4+nPtzEqLweZKUlMv3zzV6XZIxpJwsO06neWraVqr2HuevcwVxzRg5z11Wzfd8Rr8syxrSDBYfpND6f8qf55Qzvk8ak4dlMGZ2LT+GVki1el2aMaQcLDtNp/l66g7LqWqYVDUJEGJCRwrhBGcwo2YLPp16XZ4wJkAWH6RSqymPzysnrlcwlp/b9cn3xmDwq9xzmo7JdHlZnjGkPCw7TKT4tr2H5lr3cds5A4mKP/bP75sm96ZEcz4xF1l1lTKSw4DCd4rF55WSlJfLtM3K+sj4xLparRuXwwZrt1NTWeVSdMaY9LDhMyK2o3MtHZbu4ZUIBSfGxX3t96phcjjYqry2p9KA6Y0x7WXCYkHtsbjndk+K47sy8Fl8f0juNMwb0ZPqiLajaILkx4c6Cw4RUWXUts9Zs54ax+aQlxbe63ZTRuWzceZBFFXs6sTpjzImw4DAh9af55STExnDT+Pw2t7v0tL6kJcbZleTGRAALDhMyVXsP8+bSKopH55KZmtjmtskJcVw2sh/vrtzGvsNHO6lCY8yJsOAwIfP0hxsBuPWcgQFtP3VMHnUNPt5aVhXKsowxHWTBYUJi98F6pn++hctG9iOnZ3JA+5zSP52T+3Xnpc9tkNyYcGbBYULiuY83cfhoI9MmDmrXfsVj8ijdtp8VlftCVJkxpqMsOEzQ1dY18NwnFVw4ojdDeqe1a9/LR/YjKT6G6XYluTFhy4LDBN2Ln33B/iMN3Hnu4Hbv2z0pnktO7cfMZVUcrGsIQXXGmI6y4DBBVdfQyNMfbmLcoAxG5vY4oWNMHZPLwfpG3lmxNcjVGWOCwYLDBNVri6uoPlDHnUXtb200OWNATwZnp1p3lTFhyoLDBE1Do48nFpRzWk464wdnnPBxRITi0bks3byXddsPBLFCY0wwWHCYoHlv1Xa+qDnEne5ETR1x1ek5JMTG8JJdSW5M2LHgMEGhqjw+r5yBWSlcOKJPh4/XKyWBC0/uzRtLqzhytDEIFRpjgsWCwwTFvPU7Kd22nzsmDiImpmOtjSZTx+Sx7/BRZq3eHpTjGWOCw4LDBMXjc8vpm57EFSP7B+2YYwdmkNurm3VXGRNmLDhMh5VU7Obzit3cevZAEuKC908qJkYoHp3Hwo272bTrYNCOa4zpGAsO02GPzSunZ3I8xWNyg37sa87IITZGbE5yY8KIBYfpkNJt+5mztpqbxxeQnBAX9ONnd09i0vBsXl1cydFGX9CPb4xpPwsO0yGPzysnJSGWG8fmh+w9ikfnsqu2jtmlO0L2HsaYwIU0OERksoisE5EyEbm/lW2uFZE1IrJaRF70W/+giKxylyl+6wtE5DP3mDNEJCGUn8G07ouag7yzYivXnTWA9OTWp4XtqIlDs+jTPcmuJDcmTIQsOEQkFngUuAgYAUwVkRHNthkCPACMV9WTgXvc9ZcApwMjgTOBH4tId3e3B4GHVXUwsAe4JVSfwbTtiQUbiYuJ4ZYJBSF9n7jYGK4tzGH++p1U7T0c0vcyxhxfKFscY4AyVd2oqvXAdODyZtvcCjyqqnsAVLXaXT8CWKCqDap6EFgBTBbncuRJwKvuds8DV4TwM5hWVO8/wqsllVx9Rg69uyeF/P2uKXQG3l+2VocxngtlcPQH/P+XV7rr/A0FhorIxyKyUEQmu+uX4wRFsohkAucCuUAGsFdVG9o4JgAicpuIlIhIyc6dO4P0kUyTZz7eRIPPx+0BTgvbUbm9kpkwOJNXSrbQ6LPZAY3xkteD43HAEKAImAo8JSI9VPUD4D3gE+Al4FOgXfedUNUnVbVQVQuzsrKCW3UXt+/wUV5YuJmLT+1LfmZKp73v1DF5bN13hAUb7A8BY7wUyuCowmklNMlx1/mrBGaq6lFV3QSsxwkSVPU3qjpSVS8AxH2tBughInFtHNOE2F8/raC2roFpRe2bFrajzj+pNxkpCUy3K8mN8VQog2MRMMQ9CyoBKAZmNtvmTZzWBm6X1FBgo4jEikiGu/404DTgA1VVYC7wbXf/G4G3QvgZTDOH6xv588cVFA3L4uR+6Z363glxMVx9Rg6zS6upPnCkU9/bGHNMyILDHYe4C5gFlAIvq+pqEfm1iFzmbjYLqBGRNTiBcJ+q1gDxwIfu+ieB6/3GNX4K3CsiZThjHs+E6jOYr5uxaDO7D9Z3aKKmjpgyOpcGn/LaYmtoGuMVcf6Ij26FhYVaUlLidRkR72ijj6L/mkff9CRenTbOszqufeJTqvcfYe6Pizo874cxpnUislhVC5uv93pw3ESQt5ZtpWrvYe48t3PHNporHp1LRc0hPt1Y42kdxnRVFhwmID6f8qf55Qzvk8a5w7I9reXiU/vSPSnObnxojEcsOExAPlizg7LqWqYFYVrYjkqKj+XKUf15f9V29h6q97QWY7oiCw5zXM60sGXk9UrmklP7el0OAFNG51Hf4OP1JTZIbkxns+Awx/VpeQ3LK/dx+8SBxMWGxz+ZEf26842cdKYv2kxXOMHDmHASHt8CJqw9Nq+crLRErj49x+tSvqJ4TB7rd9SydMter0sxpkux4DBtWlG5l4/KdnHLhAKS4mO9LucrvvWNfiQnxNqV5MZ0MgsO06bH5pbTPSmO687M87qUr0lNjONbp/Xj7eXbOHDkqNflGNNlWHCYVpVV1zJrzXZuGJtPWlLoJmrqiOIxuRw+2sjby7d5XYoxXYYFh2nVn+aXkxgXw83j870upVUjc3swvE8a0xdZd5UxncWCw7Soau9h3lxaRfHoPDJSE70up1UiwpTRuayo3Mfqrfu8LseYLsGCw7ToqQUbAbi1kyZq6ogrR/UnIS7GriQ3ppNYcJivqamtY/qizVw+sj/9e3Tzupzj6pGcwMWn9OGNpVUcrm/XfF/GmBNgwWG+5rlPKqhr8DGtKPxbG02mjM7jwJEG3ltpg+TGhJoFh/mK2roGnv+kggtH9GZwdprX5QTsrIG9KMhMsUFyYzrBcYNDRFJEJMZ9PFRELhOR8Dw303TYi599wf4jDUzzaKKmE9U0SL6oYg9l1bVel2NMVAukxbEASBKR/sAHwHeA50JZlPFGXUMjT3+4iXGDMhiZ28Prctrt6tNziIsRZlirw5iQCiQ4RFUPAVcBj6nqNcDJoS3LeOG1xVVUH6jzbFrYjspKS+T8k3rz2pIq6hpskNyYUAkoOERkLHAd8K67LrxuWmQ6rKHRxxMLyjktJ53xgzO8LueEFY/JZffBev6xptrrUoyJWoEExz3AA8AbqrpaRAYCc0Nbluls763azhc1h7gzDCZq6oizh2TRv0c3GyQ3JoSOGxyqOl9VL1PVB91B8l2qencn1GY6iTNRUzmDslK4cEQfr8vpkNgY4ZrCHD7csIstuw95XY4xUSmQs6peFJHuIpICrALWiMh9oS/NdJZ563ZSum0/d0wcRExM5LY2mlxbmEuMwMsldiW5MaEQSFfVCFXdD1wBvA8U4JxZZaLEY/PK6JeexOUj+3tdSlD069GNiUOzeLlkCw2NPq/LMSbqBBIc8e51G1cAM1X1KGBzdUaJRRW7WVSxh1vPGUhCXPRcD1o8Jo8d++uYt26n16UYE3UC+aZ4AqgAUoAFIjIA2B/KokzneXxeOb1SEigeHX4TNXXEpOHZZKYm2iC5MSEQyOD4I6raX1UvVscXwLmdUJsJsdJt+5mztpqbx+XTLSG6zrCOj43hmsIc5qytZvu+I16XY0xUCWRwPF1EficiJe7yPzitDxPhHp9XTkpCLDeMzfe6lJCYUpiLT+HVxTZIbkwwBdJV9WfgAHCtu+wHng1lUSb0vqg5yDsrtnLdWQNIT47OW4/lZ6YwdmAGM0q24PPZsJwxwRJIcAxS1V+q6kZ3+RUQOffbNi16YsFG4mJiuGVCgdelhFTxmFy27D7MJ+U1XpdiTNQIJDgOi8iEpiciMh44HLqSTKhV7z/CqyWVXH1GDr27J3ldTkh98+Q+9EiO5yUbJDcmaOIC2OYO4C8iku4+3wPcGLqSTKg989EmGnw+7pgY/Q3HpPhYrhqVw18XVlBTWxfW86cbEykCOatquap+AzgNOE1VRwGTQl6ZCYl9h47yfwu/4JLT+jEgo2uc41A8JpejjcrrS6q8LsWYqBDwFV+qut+9ghzg3hDVY0LsL59WcLC+kWkTB3ldSqcZ2juN0/N6MH3RZlRtkNyYjjrRS4Uj/4ZGXdDh+kae/aSCc4dlMaJfd6/L6VTFY/Io33mQki/2eF2KMRHvRIPD/myLQDMWbWb3wXruPDcyJ2rqiEtP60tqYhwvfW6D5MZ0VKvBISIHRGR/C8sBoF8n1miC4Gijj6c+3MTo/J6Mzu/ldTmdLjkhjstG9uO9ldvYd/io1+UYE9FaDQ5VTVPV7i0saaoayNlYJoy8tWwrVXsPR+y0sMEwdXQeR476mLnMBsmN6YjouR2qaZXPp/xpfjnD+6RRNCzL63I8c0r/7ozo252XPt9ig+TGdEBIg0NEJovIOhEpE5H7W9nmWhFZIyKrReRFv/UPuetKReQRceczFZF57jGXuUt2KD9DNPhgzQ7KqmuZFuHTwnaUiDB1TC5rtu1nVZXd4NmYExWy4BCRWOBR4CJgBDBVREY022YIznzm41X1ZJz5zRGRccB4nGtHTgFGAxP9dr1OVUe6S3WoPkM0cKaFLSOvVzKXnNrX63I8d/mo/iTFx9iV5MZ0QCB3x/2+iPQ8gWOPAcrc+1vVA9OBy5ttcyvwqKruAfALAQWSgAQgEYgHdpxADV3eJ+U1LK/cx+0TBxIXaz2T3ZPiueTUfsxctpWDdQ1el2NMRArkm6Q3sEhEXna7ngLt6+gP+N/PutJd528oMFREPhaRhSIyGUBVPwXmAtvcZZaqlvrt96zbTfWL1uoRkduabgW/c2fXnQXusXllZKUlcvXpOV6XEjaKx+RSW9fAuyu2eV2KMREpkFuO/BwYAjwD3ARsEJH/EJFgXHoc5x67CJgKPCUiPURkMHASkIMTNpNE5Gx3n+tU9VTgbHdpcf5zVX1SVQtVtTArq2sOCC/fspePy2r43oQCkuKja6Kmjigc0JPB2ak2O6AxJyigvgt1TkHZ7i4NQE/gVRF5qI3dqoBcv+c57jp/lbjzmKvqJmA9TpBcCSxU1VpVrQXeB8a6tVS5Pw8AL+J0iZkWPD6vnO5JcVx31gCvSwkrIkLx6FyWbN7L+h0HvC7HmIgTyBjHD0RkMfAQ8DFwqqpOA84Arm5j10XAEBEpEJEEoBiY2WybN3FaG4hIJk7X1UZgMzBRROJEJB5nYLzUfZ7pbh8PXAqsCvTDdiVl1bXMWrOdG8flk5pol900d+Wo/sTHil1JbswJCKTF0Qu4SlW/qaqvqOpRAFX14Xxxt0hVG4C7gFlAKfCyqq4WkV+LyGXuZrOAGhFZgzOmcZ+q1gCvAuXASmA5sFxV38YZKJ8lIiuAZTgtmKfa/am7gD/NLycxLoabxuV7XUpYykhN5MKT+/DG0iqOHG30uhxjIooEciGUiJwOTMA52+ljVV0S6sKCqbCwUEtKSrwuo9NU7T3MxIfmcv1ZA/jXy072upyw9dGGXVz/zGf8b/FILh/Z/LwNY4yILFbVwubrA+mq+gXwPJABZOKc0fTz4JdoguWpBRsBuPWc6J+oqSPGDcogt1c3pn++5fgbG2O+FEjn9/XAN1T1CICI/Banm+jfQ1lYOFhVtY+6Bh+ZqQlkpiaSnBAb9lde19TWMX3RZi4f2Z/+Pbp5XU5Yi4kRphTm8t8frKdi10HyM7vGxFbGdFQgwbEV52K8I+7zRL5+dlRU+q9Z65i//tg1IEnxMWSmJpKRmkhmSoL7+NjPrKbXUhPokZxAbEznh8xzn1RQ1+BjWpG1NgJxTWEuD/9jAzNKtvDTycO9LseYiBBIcOwDVovI33HGOC4APheRRwBU9e4Q1uepX1x6EjftyWfXgTpqDtZTU1vHrtp6dtXWsXXfEVZW7aPmYD2Nvq+PE8UI9GoeLimJZKYlkOn+dJ4nkpGSEJTrLA4cOcrzn1Rw4YjeDM5O6/DxuoLe3ZM4d1g2r5RUcu8FQ4m3q+uNOa5AguMNd2kyLzSlhJ/B2WnH/QL2+ZR9h4+yyw2VmoN1XwbNLr+gWbp5LzW1dRysb/kMntTEODJTE75ssTg/E7/sJstIcdZlpSbSvVtci11mL362mf1HGrr0rdNPRPHoXP5RuoPZpdVMPqWP1+UYE/aOGxyq+rx7HcZQd9W6plNyjdNP3jMlgZ4pCQzpffztD9c3uoFSR40bKv4hU1Nbx6ZdBymp2MPuQ/W0dNJbfKyQkfLVbrLM1EReX1LF+MEZfCO3R/A/aBQrGpZF7+6JzFi02YLDmAAcNzhEpAjnrKoKnLnGc0XkRlVdENrSolO3hFhyeyWT2yv5uNs2NPrYc+io24pxWjM7m1ozft1nZdW17Kyto6HRx92TRnXCp4gucbExXFuYy6Nzy9i69zD97KQCY9oUSFfV/wAXquo6ABEZCryEc+W4CaG42Biy0hLJSkuE4/whrKo0+NT66E/QtYW5/HFuGS+XbOGe84cefwdjurBAvmXim0IDQFXX49zm3IQREbHQ6IDcXslMGJzJKyWVLZ7sYIw5JpBvmsUi8rSIFLnLU0DXuQzbdBnFo/Oo2nuYDzd03dvwGxOIQILjDmANcLe7rAGmhbIoY7xwwYje9EpJsCvJjTmONsc43Olfl6vqcOB3nVOSMd5IiIvh6tP78+zHFew8UOeMLRljvqbNFoeqNgLrRCSvk+oxxlNTRufR4FNeW1LpdSnGhK1Auqp64lw5PltEZjYtoS7MGC8Mzk5lTH4vZizaQiB3jjamKwrkdNxfhLwKY8LIlNG5/OiV5SzcuJuxgzK8LseYsBNIi+NiVZ3vvwAXh7owY7xy8al9SUuKY4bNSW5MiwIJjgtaWHdRsAsxJlx0S4jlylH9eW/VdvYeqve6HGPCTqvBISLTRGQlMExEVvgtm3CmdDUmahWPzqO+wccbS7vEDALGtEtbLY4XgW8BM92fTcsZqnpdJ9RmjGdG9OvOaTnpTP/cBsmNaa7V4FDVfapaoapTgUrgKM58HKl2eq7pCopH57FuxwGWbdnrdSnGhJVA5hy/C9gB/B14113eCXFdxnjuspH9SE6ItSvJjWkmkMHxe4Bhqnqyqp7qLqeFujBjvJaaGMelp/Xl7RVbqa1r8LocY8JGIMGxBWf6WGO6nOIxeRyqb+Tt5Vu9LsWYsBHIBYAbgXki8i5Q17RSVe3eVSbqjcrtwbDeaUz/fDNTx9jQnjEQWItjM874RgKQ5rcYE/VEhOIxuSyv3MfKSmt4GwOBzTn+q+brRCSQlooxUeHqM3J4+O/r+cOcDTx5Q6HX5RjjubYuAPzI7/Ffm738ecgqMibMdE+K57sTCvhgzQ5Wb7VWhzFtdVWl+D0+pdlrEoJajAlbN48vIC0pjkdmb/C6FGM811ZwaCuPW3puTFRL7xbPLRMKmLXaWh3GtBUcPUTkShG52n18lbtcDaR3Un3GhA1rdRjjaGuQez5wmd/jb/m9tiBkFRkTptK7xfPd8QX87+wNrNm6nxH9untdkjGeaDU4VPXmzizEmEjw3QkF/PnjTTwyewN/+s4ZXpdjjCcCuY7DGONqanX8bfV21mzd73U5xnjCgsOYdvruBBvrMF2bBYcx7ZTeLZ6b3VZH6TZrdZiuJ5Dbql8jImnu45+LyOsicnroSzMmfN0yvoC0RGt1mK4pkBbHL1T1gIhMAM4HngEeD21ZxoS39OR4bp5QwPurrNVhup5AgqPR/XkJ8KSqvotzw0NjujRrdZiuKpDgqBKRJ4ApwHsikhjgfojIZBFZJyJlInJ/K9tcKyJrRGS1iLzot/4hd12piDwiIuKuP0NEVrrH/HK9MZ0tPTmem8fn8/6q7azdbq0O03UEEgDXArOAb6rqXqAXcN/xdhKRWOBR4CJgBDBVREY022YI8AAwXlVPxpltEBEZB4wHTsO5T9ZoYKK72+PArcAQd5kcwGcwJiS+O8FaHabrCSQ4+gLvquoGESkCriGwu+OOAcpUdaOq1gPTgcubbXMr8Kiq7gFQ1Wp3vQJJOF1iiUA8sENE+gLdVXWhqirwF+CKAGoxJiR6JCdw8/h83ltprQ7TdQQSHK8BjSIyGHgSyAVebHsXAPrjTDvbpNJd528oMFREPhaRhSIyGUBVPwXmAtvcZZaqlrr7Vx7nmACIyG0iUiIiJTt37gygXGNOzHcnFJBqrQ7ThQQSHD5VbQCuAv6gqvfhtEKCIQ6nu6kImAo8JSI93JA6CcjBCYZJInJ2ew6sqk+qaqGqFmZlZQWpXGO+zr/VsW77Aa/LMSbkAgmOoyIyFbgBeMddFx/AflU4rZMmOe46f5XATFU9qqqbgPU4QXIlsFBVa1W1FngfGOvun3OcYxrT6W6xVofpQgIJjptxvrR/o6qbRKQAaD4jYEsWAUNEpEBEEoBiYGazbd7EaW0gIpk4XVcbceY5nygicSISjzMwXqqq24D9InKWezbVDcBbAdRiTEg1tTreXbnNWh0m6h03OFR1DfBjYKWInAJUquqDAezXANyFc0ZWKfCyqq4WkV+LSNPt2mcBNSKyBmdM4z5VrQFeBcqBlcByYLmqvu3ucyfwNFDmbvN+wJ/WmBCyVofpKsQ5OamNDZwzqZ4HKnCmjM0FblTViJmTo7CwUEtKSrwuw3QB/z1rHY/OK+NvPziHYX3SvC7HmA4RkcWqWth8fSBdVf8DXKiqE1X1HOCbwMPBLtCYaHDLhAJSEuJ4ZI61Okz0CiQ44lV1XdMTVV1PYIPjxnQ5PVMSuGlcPu+t3Mb6HTbWYaJTIMGxWESeFpEid3kKsH4fY1pxy4QCkuNjbazDRK1AguMOYA1wt7usAaaFsihjIlnPlARucs+wslaHiUZtBod7v6nlqvo7Vb3KXR5W1bpOqs+YiPS9CQOt1WGiVpvBoaqNwDoRyeukeoyJCj1TErhxnNPq2GCtDhNlAumq6gmsFpHZIjKzaQl1YcZEuu+d7bY65pR5XYoxQRUXwDa/CHkVxkShXm6r4/H55dw9aTBDett1HSY6tNriEJHBIjJeVef7LzgzAla2tp8x5hhrdZho1FZX1e+BliYY2Oe+Zow5jl4pCdwwLp93Vmy1sQ4TNdoKjt6qurL5SnddfsgqMibK3Hr2QLrFx/IHa3WYKNFWcPRo47VuwS7EmGjVNNbx9oqtlFVbq8NEvraCo0REbm2+UkS+BywOXUnGRJ+mVscjs63VYSJfW2dV3QO8ISLXcSwoCnHmAb8y1IUZE016pSRww9h8nlhQzt3nDWZwtp1hZSJXqy0OVd2hquOAX+HcUr0C+JWqjlXV7Z1TnjHR49azC2ysw0SFQCZymquqf3CXOZ1RlDHRKCM1kRvG5jNz+VbKqmu9LseYExbIlePGmCA51uqwe1iZyBXIlePGmCDJSE3kO2MH8NSCjXx/0hAGZ6d6XZJxHa5vpOZgHTW19X4/66mprXN/Ouv7dE9i0vDeTBqeTZ/0JK/L9oQFhzGd7LazB/KXT77gj3M28PviUV6XE7XqG3zsPljPrto6dh88Fga7auvZ3fT44LHHh+obWzxOYlwMmamJZKQmkJGSSOm2A/yjtBqAU/p3Z9Lw3pw3PJtT+6cTEyOd+RE9Y8FhTCfLSE3khnFOq+Mua3UErKHRx55DR50QqK1jl/vTCYdjj2vcsDhwpKHF48THCr1SnBDISE2gICOZjC+D4dj6pp/JCbGIHAsEVWX9jlpmr93BnNJq/jhnA4/M3kBWWiKThmUz6aRsJgzOJCUxer9eRVW9riHkCgsLtaTEJi004aOmto4JD87lmyf37vKtjs01h9i277DTHdTUNVRb/2Vroeag83jPoXpa+rqKEb4Mgl4pCWSkJpCZeuzxsSBIICM1ke5JcV8Jgo7afbCeeeuqmb22mgXrdnKgroGEuBjGDszgvJOymTQ8m5yeyUF7v84kIotVtfBr6y04jPHGf75XylMfbuTv905kUFbXa3WoKv/zwXr+OPfrpyf3SI6nV0oCme6Xfi/3Sz8z9VhAZKY669K7xRMbJl1ERxt9LNq0m9lrq5lduoOKmkMADO+TxqTh2Zx3UjYjc3uGTb3HY8FhwWHCzK7aOs5+cC6TT+nDw1NGel1Op1JV/vuDdTw6t5yrT8/hylH9v2wV9ExJID42Ok743Lizltml1cxeu4NFFXto9Ck9k+M5d1g2553Um7OHZtI9Kd7rMlvVWnBEbyecMWEuMzWRG8YO4KkPN3LXpMFdptWhqvzXrHU8Nq+cqWPy+M0Vp0TtoPLArFQGZqVy6zkD2Xf4KAvW72R26Q7mrKvm9aVVxMUIYwp6cd5JzgB7fmaK1yUHxFocxnioq7U6VJWHZq3j8Xnl/NOZefz75dEbGm1paPSxdMtepzVSuoMN7gWhA7NSOG+40xo5Y0BPz1te1lVlwWHC1H+8V8rTH27kH/dOZGAUtzpUlQf/trktI1QAABBASURBVI4/zS/nujPz+LcuGhot2VxziDlrdzB7bTULN9ZwtFHpnhTHxGHZnDc8m6JhWfRITuj0uiw4LDhMmNpVW8eEB+dw8Sl9+V2UtjpUld/+bS1PzN/I9Wfl8evLLDRaU1vXwEcbdjK7tJq566rZVVtPjEDhgF5MOimb80/KZlBWalDPDGuNBYcFhwlj0dzqUFV++/5anljghMa/XX5Kp3zpRQOfT1leuZc5a6uZXVrNmm3OpKx5vZK/PEvrzIIMEuJC06VlwWHBYcLYzgN1nP1Q9LU6VJX/fH8tTy7YyHfOGsCvLz/ZQqMDtu49zJy11cxZW83HZbuoa/CRmhjH2UMymTQ8m3OHZ5OZmhi097PgsOAwYe43767hmY82MftHRRREyNk1bVFV/uO9Up76cBM3jB3Ary6z0Aimw/WNfFK+i3+UVjNn7Q527K9DBEbm9uC84dlMGt6bk/qmdeh3bsFhwWHC3JetjlP78rtrI7vVoar85t1Snv5oEzeOHcC/WmiElKqyeut+t0trB8sr9wHQLz2JZ28ew7A+JzZxmF3HYUyYy0pL5PozB/Dnjzfx/UlDIrbVoar8+7ulPPPRJm4al88vvzXCQiPERIRT+qdzSv907j5vCNUHjjBv7U7mra8mr1fwb3cSHZdnGhMlbps4kIS4GP4YobMEqir/9o6Fhtey05K4dnQuj113Bt0SYoN+fAsOY8JIdloS1585gDeXVVGx66DX5bSLqvLrd9bw5483cfN4C41oZsFhTJi5beJA4mMlouYmbwqNZz+u4LvjC/h/l1poRDMLDmPCTHZaEtdFUKtDVfnV205o3DKhgF9cepKFRpSz4DAmDN0+cSBxMdLiLcfDSVNoPPdJBd+bUMDPL7HQ6AosOIwJQ9lpSVx/1gDeWBq+rQ5V5V9nrua5Tyq49ewC/sVCo8sIaXCIyGQRWSciZSJyfyvbXCsia0RktYi86K47V0SW+S1HROQK97XnRGST32uRfcK7Ma0I51aHqvLLmat5/tMvuO2cgfzsYguNriRkwSEiscCjwEXACGCqiIxots0Q4AFgvKqeDNwDoKpzVXWkqo4EJgGHgA/8dr2v6XVVXRaqz2CMl5rGOt5YWsUXNeHT6lBV/t9bq/nLp19w+zkDeeCi4RYaXUwoWxxjgDJV3aiq9cB04PJm29wKPKqqewBUtbqF43wbeF9VD4WwVmPC0h1NrY4wOcPK51N+8dYq/rrwC26fOJD7LTS6pFAGR39gi9/zSnedv6HAUBH5WEQWisjkFo5TDLzUbN1vRGSFiDwsIi3e0UtEbhOREhEp2blz54l+BmM8ld3daXW8HgatjqbQ+L+Fm7lj4iDun2yh0VV5PTgeBwwBioCpwFMi0qPpRRHpC5wKzPLb5wFgODAa6AX8tKUDq+qTqlqoqoVZWVmhqd6YThAOrQ6fT/n5W6t44bPNTCsaxE8nD7PQ6MJCGRxVQK7f8xx3nb9KYKaqHlXVTcB6nCBpci3whqoebVqhqtvUUQc8i9MlZkzUyu6exD+dmedZq8PnU/7lzVW8+Nlm7iwaxE++aaHR1YUyOBYBQ0SkQEQScLqcZjbb5k2c1gYikonTdbXR7/WpNOumclshiPMv9wpgVSiKNyacTJs4iLgY4dFOPsPKCY2VvPT5Zv753EHcZ6FhCGFwqGoDcBdON1Mp8LKqrhaRX4vIZe5ms4AaEVkDzMU5W6oGQETycVos85sd+gURWQmsBDKBfw/VZzAmXDS1Ol5bUsXmms45T8TnU372xkpe+nwLd507mB9faKFhHDYfhzERYsf+I5z90FyuGNmPh779jZC+V1NoTF+0he9PGsy9Fwy10OiCWpuPw+vBcWNMgHp3T+KfxuTxeohbHT6f8sDrTmjcbaFhWmDBYUwEmVY0iJgQjnX4fMr9r69gRokTGj+00DAtsOAwJoI0tTpeW1IZ9FaHz6f89LUVvFxSyd3nDbHQMK2y4DAmwoSi1dHoU37y2gpeWVzJD84bYt1Tpk0WHMZEGP9Wx5bdHW91NPqUn7y6glfd0PjhBUODUKWJZhYcxkSgOyYGp9XR6FPue3U5ry2p5J7zLTRMYCw4jIlAfdKdVseri0+81dHoU+57ZTmvL6nih+cP5Z7zLTRMYCw4jIlQd0wcRIycWKvjy9BYWsW9FwzlB+cPOf5OxrgsOIyJUH3Sk5g6JrfdrY5Gn/JjNzR+dMFQ7j7PQsO0jwWHMRFsWtFgYkR4bF5grY5Gn/Kjl5fxxtIqfnzhUL5voWFOgAWHMRGsT3oSxWNyeaXk+K2OhkYf9768jDeXbeW+bw7jrkkWGubEWHAYE+GmFQ06bqvDCY3lvOWGxj+fO7gTKzTRxoLDmAjXN73bl62Oyj1fb3U0NPr44cvLmbl8Kz+ZbKFhOs6Cw5go0NTqeHRu+VfWN4XG28u38tPJw7mzyELDdJwFhzFRoG96N6aMzuWVki1ftjoaGn3cM2MZby/fyv0XDWda0SCPqzTRwoLDmChxbKyjnIZGHz+YsYx3VmzjgYuGc8dECw0TPHFeF2CMCY5+PZxWx/RFm9m29zBz1+3kZxcP57ZzLDRMcFmLw5goMq1oEIIwd91O/uXikyw0TEhYi8OYKNKvRzf+86pTEYGrTs/xuhwTpSw4jIkyV59hgWFCy7qqjDHGtIsFhzHGmHax4DDGGNMuFhzGGGPaxYLDGGNMu1hwGGOMaRcLDmOMMe1iwWGMMaZdRFW9riHkRGQn8MUJ7p4J7ApiOaEWSfVaraETSfVGUq0QWfV2tNYBqprVfGWXCI6OEJESVS30uo5ARVK9VmvoRFK9kVQrRFa9oarVuqqMMca0iwWHMcaYdrHgOL4nvS6gnSKpXqs1dCKp3kiqFSKr3pDUamMcxhhj2sVaHMYYY9rFgsMYY0y7WHC0QUQmi8g6ESkTkfu9rqc1IvJnEakWkVVe1xIIEckVkbkiskZEVovID7yuqTUikiQin4vIcrfWX3ld0/GISKyILBWRd7yu5XhEpEJEVorIMhEp8bqetohIDxF5VUTWikipiIz1uqbWiMgw93fatOwXkXuCdnwb42iZiMQC64ELgEpgETBVVdd4WlgLROQcoBb4i6qe4nU9xyMifYG+qrpERNKAxcAVYfq7FSBFVWtFJB74CPiBqi70uLRWici9QCHQXVUv9bqetohIBVCoqmF/QZ2IPA98qKpPi0gCkKyqe72u63jc77Iq4ExVPdELob/CWhytGwOUqepGVa0HpgOXe1xTi1R1AbDb6zoCparbVHWJ+/gAUAr097aqlqmj1n0a7y5h+9eWiOQAlwBPe11LNBGRdOAc4BkAVa2PhNBwnQeUBys0wIKjLf2BLX7PKwnTL7dIJiL5wCjgM28raZ3b9bMMqAb+rqphWyvwe+AngM/rQgKkwAcislhEbvO6mDYUADuBZ91uwKdFJMXrogJUDLwUzANacBjPiEgq8Bpwj6ru97qe1qhqo6qOBHKAMSISlt2BInIpUK2qi72upR0mqOrpwEXAP7vdruEoDjgdeFxVRwEHgbAd92zidqldBrwSzONacLSuCsj1e57jrjNB4I4XvAa8oKqve11PINyuibnAZK9racV44DJ33GA6MElE/s/bktqmqlXuz2rgDZwu4nBUCVT6tTZfxQmScHcRsERVdwTzoBYcrVsEDBGRAje1i4GZHtcUFdwB52eAUlX9ndf1tEVEskSkh/u4G87JEmu9raplqvqAquaoaj7Ov9c5qnq9x2W1SkRS3JMjcLt9LgTC8sxAVd0ObBGRYe6q84CwO5mjBVMJcjcVOM0v0wJVbRCRu4BZQCzwZ1Vd7XFZLRKRl4AiIFNEKoFfquoz3lbVpvHAd4CV7tgBwM9U9T0Pa2pNX+B598yUGOBlVQ3701wjRG/gDefvCOKAF1X1b96W1KbvAy+4f0huBG72uJ42uWF8AXB70I9tp+MaY4xpD+uqMsYY0y4WHMYYY9rFgsMYY0y7WHAYY4xpFwsOY4wx7WLBYcKWiNS6P/NF5J+CfOyfNXv+SRCP/fumK6DdW1OMCNaxQ01EGv3uqDrTb32BiHzm3il6hntKKiKS6D4vc1/Pd9efKiLPefIhTMhZcJhIkA+0KzhE5HjXKH0lOFR1XDtrau19M4Cz3BtPoqrf8/quvwH8LvwdVtWR7nKZ3/oHgYdVdTCwB7jFXX8LsMdd/7C7Haq6EsgRkbyOfwITbiw4TCT4LXC2+1fwD92bDv6XiCwSkRUicjuAiBSJyIfuX8pr3HVvujfQW910Ez0R+S3QzT3eC+66ptaNuMde5c4TMcXv2PP85mN4wb0CvrmrgS8vYnP3KWx6DxH5jThzeywUkd7Nd3avpv6zOHOALBWRy931C0Xk5ObHbWP7m0RkpojMAWaLyF9E5Aq//V9o2vZ43M85Cec2GwDPA03Hutx9jvv6eX6/l7dxrmA30UZVbbElLBeg1v1ZBLzjt/424Ofu40SgBOfupUU4N58r8Nu2l/uzG87tLDL8j93Ce10N/B3nbgG9gc04V48XAftw7lkWA3yKc4O+5jU/D3zL7/k8nPkmwLkT7Lfcxw81fYZm+/8HcL37uAfOnDApwA+BX7nr+wLrjrP9TTj3V2r6/BOBN93H6cAmIK6F929wf58LceZIAcjEmWKgaZtcYJX7eBWQ4/daOZDpPh4PvO31vyNbgr9Yi8NEoguBG9zblXwGZABD3Nc+V9VNftveLSLLcb4Ic/22a80E4CV17oi7A5gPjPY7dqWq+oBlOF1ozfXFuf12S+qBptuVLG5l/wuB+93PNg9IAvKAl4Fvu9tcy7G//lvbHpxbwO8GUNX5OPdey8K5f9FrqtrQwvsPUNVCnK7B34vIoFY+SyCqgX4d2N+EKbtXlYlEAnxfVWd9ZaVIEU6Lw//5+cBYVT0kIvNwvlhPVJ3f40Za/v9zuI33OKqqTff4aW1/Aa5W1XVfe0GkRkROA6YAd7S1vYicid/vwvUX4Hqc7qMW77Okx+5Wu9H9fY3CuYtxDxGJc8PG/07RTXeRrnTHUtKBGve1JJzfh4ky1uIwkeAAkOb3fBYwTZxbsyMiQ6XlSXXScQZuD4nIcOAsv9eONu3fzIfAFHccJQtn1rfP21FrKTC4Hds3Nwv4ftM4gYiM8nttBs4kTemquiKA7Zt7DrgHQFsYsBeRniKS6D7OxOlqWuOG3VyOtXhuBN5yH890n+O+PscvHIcSpne7NR1jwWEiwQqg0R1U/iHOtKhrgCUisgp4gpb/ev8bECcipTgD7P7zhD8JrGgaHPfzhvt+y4E5wE/UuaV2oN7FGQ85Uf+GMz3tChFZ7T5v8ipOa+HlALf/CrfrrRR4tpVNTgJK3K69ucBv/QLmp8C9IlKG0zXYdPflZ4AMd/29fHVyo3Nxfh8mytjdcY0JMhH5CLhUw2xOahFJBlYCp6vqvhC/VyLO+NCEVsZSTASzFocxwfcjjg1QhwUROR+ntfGHUIeGKw+430IjOlmLwxhjTLtYi8MYY0y7WHAYY4xpFwsOY4wx7WLBYYwxpl0sOIwxxrTL/we1iN3cBkznVQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D4JkSE10A1jb"
      },
      "source": [
        "#**Experiment 4:**\r\n",
        "* **Optimizer =Adam**\r\n",
        "* **Learning rate =0.0001**\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JwfukhOOA1jc"
      },
      "source": [
        "**Hyperparameters**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KgSSwTa8A1jc"
      },
      "source": [
        "#HYPERPERAMETERS\r\n",
        "batch_size = 256\r\n",
        "num_iters = 4000\r\n",
        "input_dim = 28*28 # num_features = 784\r\n",
        "output_dim = 2\r\n",
        "\r\n",
        "learning_rate = 0.0001\r\n",
        "\r\n",
        "# Device\r\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "K2WY9qZPA1jc"
      },
      "source": [
        "**MAKING DATASET ITERABLE**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "F6qCWQwgA1jd"
      },
      "source": [
        "'''\r\n",
        "MAKING DATASET ITERABLE\r\n",
        "'''\r\n",
        "num_epochs = num_iters / (len(train_dataset) / batch_size)\r\n",
        "num_epochs = int(num_epochs)\r\n",
        "\r\n",
        "train_loader = torch.utils.data.DataLoader(dataset=train_dataset, \r\n",
        "                                           batch_size=batch_size, \r\n",
        "                                           shuffle=True)  \r\n",
        "\r\n",
        "test_loader = torch.utils.data.DataLoader(dataset=test_dataset, \r\n",
        "                                          batch_size=batch_size, \r\n",
        "                                          shuffle=False) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "a59vuaEWA1jd"
      },
      "source": [
        "**Logistic Regression Model**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xqofIxFPA1jd"
      },
      "source": [
        "class LogisticRegressionModel(nn.Module):\r\n",
        "    def __init__(self, input_size, num_classes):\r\n",
        "        super().__init__()\r\n",
        "        self.linear = nn.Linear(input_size, num_classes)\r\n",
        "\r\n",
        "    def forward(self, x):\r\n",
        "        logits  = self.linear(x)\r\n",
        "        probas = F.softmax(logits, dim=1)\r\n",
        "        return logits, probas"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hlTUSPeQA1je"
      },
      "source": [
        "**INSTANTIATE MODEL CLASS**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iGzTtcxUA1je",
        "outputId": "16c9c197-3460-46e0-98a0-da6fae1bdf46"
      },
      "source": [
        "'''\r\n",
        "INSTANTIATE MODEL CLASS\r\n",
        "'''\r\n",
        "model = LogisticRegressionModel(input_size=input_dim,\r\n",
        "                                num_classes=output_dim)\r\n",
        "# To enable GPU\r\n",
        "model.to(device)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegressionModel(\n",
              "  (linear): Linear(in_features=784, out_features=2, bias=True)\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XswYK3okA1je"
      },
      "source": [
        "**Optimizer setup**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0LUT1ZE5A1jf"
      },
      "source": [
        "# INSTANTIATE OPTIMIZER CLASS\r\n",
        "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dwXQ282MA1jf"
      },
      "source": [
        "**Function TRAIN THE MODEL**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IVzzmPfsA1jf",
        "outputId": "5fe6f9e5-1cdd-4a61-bd6f-e392e1234816"
      },
      "source": [
        "'''\r\n",
        "TRAIN THE MODEL\r\n",
        "'''\r\n",
        "iteration_loss = []\r\n",
        "iter = 0\r\n",
        "for epoch in range(num_epochs):\r\n",
        "    for i, (images, labels) in enumerate(train_loader):\r\n",
        "\r\n",
        "        \r\n",
        "        images = images.view(-1, 28*28).to(device)\r\n",
        "        labels = labels.to(device)\r\n",
        "\r\n",
        "        # Clear gradients w.r.t. parameters\r\n",
        "        optimizer.zero_grad()\r\n",
        "\r\n",
        "        # Forward pass to get output/logits\r\n",
        "        logits, probas = model(images.float()) \r\n",
        "\r\n",
        "        # Calculate Loss: PyTorch implementation of CrossEntropyLoss works with logits, not probabilities\r\n",
        "        loss = F.cross_entropy(logits, labels)\r\n",
        "\r\n",
        "        # Getting gradients w.r.t. parameters\r\n",
        "        loss.backward()\r\n",
        "\r\n",
        "        # Updating parameters\r\n",
        "        optimizer.step()\r\n",
        "\r\n",
        "        iter += 1\r\n",
        "\r\n",
        "        if iter % 500 == 0:\r\n",
        "            # Calculate Accuracy         \r\n",
        "            correct = 0\r\n",
        "            total = 0\r\n",
        "            # Iterate through test dataset\r\n",
        "            for images, labels in test_loader:\r\n",
        "               \r\n",
        "                images = images.view(-1, 28*28).to(device)\r\n",
        "\r\n",
        "                # Forward pass only to get logits/output\r\n",
        "                logits, probas = model(images.float())\r\n",
        "\r\n",
        "                # Get predictions from the maximum value\r\n",
        "                _, predicted = torch.max(probas, 1)\r\n",
        "\r\n",
        "                # Total number of labels\r\n",
        "                total += labels.size(0)\r\n",
        "\r\n",
        "\r\n",
        "                # Total correct predictions\r\n",
        "                if torch.cuda.is_available():\r\n",
        "                    correct += (predicted.cpu() == labels.cpu()).sum() \r\n",
        "                else:\r\n",
        "                    correct += (predicted == labels).sum()\r\n",
        "\r\n",
        "            accuracy = 100 * correct.item() / total\r\n",
        "\r\n",
        "            # Print Loss\r\n",
        "            iteration_loss.append(loss.item())\r\n",
        "            print('Iteration: {}. Loss: {}. Accuracy: {}'.format(iter, loss.item(), round(accuracy, 2)))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Iteration: 500. Loss: 0.6848145127296448. Accuracy: 51.15\n",
            "Iteration: 1000. Loss: 0.6747705936431885. Accuracy: 50.15\n",
            "Iteration: 1500. Loss: 0.6767639517784119. Accuracy: 50.15\n",
            "Iteration: 2000. Loss: 0.6497737169265747. Accuracy: 50.79\n",
            "Iteration: 2500. Loss: 0.6474394202232361. Accuracy: 50.34\n",
            "Iteration: 3000. Loss: 0.6546799540519714. Accuracy: 50.79\n",
            "Iteration: 3500. Loss: 0.6469826102256775. Accuracy: 50.08\n",
            "Iteration: 4000. Loss: 0.6632315516471863. Accuracy: 51.05\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9FigNcBiA1jg"
      },
      "source": [
        "**Loss show**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        },
        "id": "uFb8JJagA1ji",
        "outputId": "d0d426dc-1905-49f5-a5a5-b235c552a8af"
      },
      "source": [
        "import matplotlib\r\n",
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "print (iteration_loss)\r\n",
        "plt.plot(iteration_loss)\r\n",
        "plt.ylabel('Cross Entropy Loss')\r\n",
        "plt.xlabel('Iteration (in every 500)')\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0.6848145127296448, 0.6747705936431885, 0.6767639517784119, 0.6497737169265747, 0.6474394202232361, 0.6546799540519714, 0.6469826102256775, 0.6632315516471863]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEGCAYAAABy53LJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXhV1dX48e/KRAhTCAlhJgwJEAQZAoqoIBKlk2K1KHWuxenV1tJXq/119K19W9uKry1axQmtE0WltLVFFAREwARlkERCCEjCEEIgEAhkXL8/zom9xgw3kJtzb7I+z3Oe3LPvOeeuS2tW9t7nrC2qijHGGOOvMK8DMMYYE1oscRhjjGkWSxzGGGOaxRKHMcaYZrHEYYwxplkivA6gNcTHx2tSUpLXYRhjTEjZuHHjIVVNqNveLhJHUlISmZmZXodhjDEhRUQ+q6/dhqqMMcY0iyUOY4wxzWKJwxhjTLNY4jDGGNMsAU0cIjJDRLaLSK6I3N/AMbNEJEtEtonIyz7tD7tt2SLymIiI2/6ee81N7tYzkN/BGGPMFwXsrioRCQfmA+lAAZAhIktVNcvnmGTgAWCyqh6pTQIich4wGRjtHvo+MAV4z92/VlXtNiljjPFAIHscE4FcVc1T1QrgVeDyOsfMAear6hEAVT3otisQDUQBHYBIoDCAsRpjjPFTIBNHXyDfZ7/AbfOVAqSIyFoRWS8iMwBUdR2wEtjvbstUNdvnvOfcYaqf1g5h1SUit4pIpohkFhUVndYXeDe7kEUZ+U0faIwx7YjXk+MRQDIwFZgNLBCRWBEZCowA+uEkm2kicoF7zrWqOgq4wN2ur+/CqvqUqqapalpCwpcefGySqvLyhj38+M2trNtZ3OzzjTGmrQpk4tgL9PfZ7+e2+SoAlqpqparuAnJwEskVwHpVPa6qx4F/AZMAVHWv+7MUeBlnSKzFiQjzrhlDUnwn7nhpI3uKywLxMcYYE3ICmTgygGQRGSQiUcA1wNI6xyzB6W0gIvE4Q1d5wB5giohEiEgkzsR4trsf7x4fCXwd+CRQX6BrdCRP35AGwC0LMyg9VRmojzLGmJARsMShqlXAXcAyIBtYpKrbRORBEbnMPWwZUCwiWThzGveqajGwGNgJbAU2A5tV9e84E+XLRGQLsAmnB7MgUN8BICm+E49fO45dh07w/Vc3UV1jS+0aY9o3aQ9rjqelpemZFjn8y/rP+MmST7jtwsE88NURLRSZMcYELxHZqKppddvbRXXclnDduQPJKSzlydV5JCd24arx/bwOyRhjPOH1XVUh5adfT2Xy0B78+I2tbPzssNfhGGOMJyxxNENkeBjzvz2OPrHR3PbiRgqO2J1Wxpj2xxJHM8XGRPH0jRMor6phzgsbOVFe5XVIxhjTqixxnIahPTvzp2+PY/uBY8xdtIkau9PKGNOOWOI4TVNSEvjJ11JZtq2Qee/keB2OMca0Grur6gzcPDmJnMJS/rgil6E9O3P5mLqluIwxpu2xHscZEBEevPwsJibFcd/iLWzOL/E6JGOMCThLHGcoKiKMJ64bR0KXDsx5IZMDR095HZIxxgSUJY4W0KNzB56+MY0T5VXc+mImpyqrvQ7JGGMCxhJHCxneqyuPXjOWrXuPcu/iLbSHUi7GmPbJEkcLSk9N5L5Lh/P3zfv404pcr8MxxpiAsLuqWtjtUwaTU1jKH5bnkJzYmRln9fY6JGOMaVHW42hhIsL/fnMUY/rH8oPXNrNt31GvQzLGmBZliSMAoiPDeeqG8cTGRDJnYSZFpeVeh2SMMS3GEkeA9OwSzYIb0jhSVsltL2ZSXmV3Whlj2gZLHAF0Vt9u/GHW2Xy0p4QH3thqd1oZY9qEgCYOEZkhIttFJFdE7m/gmFkikiUi20TkZZ/2h922bBF5TETEbR8vIlvda37eHqy+Oqo3P5iewhsf7WXBmjyvwzHGmDMWsMQhIuHAfOArQCowW0RS6xyTDDwATFbVkcA9bvt5wGRgNHAWMAGY4p72BDAHSHa3GYH6Di3lexcP5Wuje/O///qUFZ8Weh2OMcackUD2OCYCuaqap6oVwKvA5XWOmQPMV9UjAKp60G1XIBqIAjoAkUChiPQGuqrqenXGfV4AZgbwO7QIEeH3V53NyD5d+d4rm8gpLPU6JGOMOW2BTBx9gXyf/QK3zVcKkCIia0VkvYjMAFDVdcBKYL+7LVPVbPf8giauCYCI3CoimSKSWVRU1CJf6Ex0jApnwQ1pdIwK55aFGRw+UeF1SMYYc1q8nhyPwBlumgrMBhaISKyIDAVGAP1wEsM0EbmgORdW1adUNU1V0xISElo47NPTu1tHnrp+PIXHyrnjLxupqKrxOiRjjGm2QCaOvUB/n/1+bpuvAmCpqlaq6i4gByeRXAGsV9Xjqnoc+BcwyT2/XxPXDGpjB3Tn4StHs2HXYX6+9BO708oYE3ICmTgygGQRGSQiUcA1wNI6xyzB6W0gIvE4Q1d5wB5giohEiEgkzsR4tqruB46JyLnu3VQ3AH8L4HcIiJlj+3Ln1CG88mE+Cz/Y7XU4xhjTLAFLHKpaBdwFLAOygUWquk1EHhSRy9zDlgHFIpKFM6dxr6oWA4uBncBWYDOwWVX/7p5zJ/A0kOse869AfYdA+u9LhpGemsiD/8hidY73czDGGOMvaQ9DJWlpaZqZmel1GF9yvLyKq574gL0lJ1nyX5MZktDZ65CMMeZzIrJRVdPqtns9Od6ude4QwdM3phEVHsachZkcLav0OiRjjGmSJQ6P9esew5+vH0/+kTLueuUjqqrtTitjTHCzxBEEJiTF8dDMUazZcYhf/TPb63CMMaZRtpBTkJg1oT85haU8/f4ukhM7c+05A70OyRhj6mU9jiDywFdHMCUlgZ//bRvrdhZ7HY4xxtTLEkcQCQ8T/vjtsQzsEcMdL21kT3GZ1yEZY8yXWOIIMl2jI3nmxgmowi0LMyg9ZXdaGWOCiyWOIJQU34knrh1H3qETfP/VTVTXtP1nbYwxocMSR5A6b2g8v7hsJCs+PcjD//7U63CMMeZzdldVELv+3IHkHCjlydV5JCd24arx/Zo+yRhjAsx6HEHuZ99I5bwhPfjxG1vZ+Nlhr8NpEfuPnuTvm/exbd9Rr0MxxpwGq1UVAkrKKpg5fy3Hy6tY8l+T6dc9xuuQ/Kaq7Dp0gg93HebD3YfJ2H2Y/MMnARgU34kVP5xCkC8bb0y71VCtKhuqCgGxMVE8feMErnh8LXNe2Mji2yfRqUNw/k9XXaNk7z9GhpskPtx1hEPHywGI6xTFhKTu3DgpiaLScp5cncfOouMM7dnF46iNMc0RnL99zJcM7dmZP84ey3eez2Duok08ce14wsK8/0u9vKqaLQVH+XCXkyg27j5CaXkVAH1jO3JBcjwTkuKYOKg7QxI6f9672FdykidX57E866AlDmNCjCWOEDJ1WE/+39dS+Z9/ZDHvnRx+eMmwVo/heHkVH312hIzdh9mw6zCb8ks+XwJ3aM/OfP3sPpwzKI4Jg+LoG9uxwev0ie3IWX27sjzrAHdMHdJa4RtjWoAljhDznclJ5Bwo5Y8rchnaszOXj+kb0M8rPl5Oxu4jnw89bdt3jOoaJTxMGNmnK9efO5AJSXFMSOpOj84dmnXt9BG9ePTdHIpKy0no0rxzjTHescQRYkSE/5l5FrsOneC+xVtI6tGJs/vHttj195acJGOX05vI2H2Y3IPHAYiKCGNs/1junDqECUlxjBvYnc5nOM+SnprIvHdyeDe7kGsmDmiJ8I0xrSCgiUNEZgD/B4QDT6vqb+o5ZhbwC0Bxloj9tohcBMzzOWw4cI2qLhGR53HWIK+9l/MmVd0UuG8RfKIiwnjiunFc9qe1zHkhk6V3nU+vbtHNvo6qsrPoOB/uOuJOZB9mb4lzx1OXDhGMT+rON8f1ZWJSHKP6daNDRHiLfo8RvbvQN7Yjy7MscRgTSgKWOEQkHJgPpAMFQIaILFXVLJ9jkoEHgMmqekREegKo6kpgjHtMHM764m/7XP5eVV0cqNhDQY/OHXjmpjSufPwDbn0xk0W3TSI6svFf7FXVNWTvL2XDrmIydh8mc/cRik9UABDfuQMTB3XnuxcMYuKgOIb36kp4gCffRYT01ERe+XAPZRVVxERZB9iYUBDI/1InArmqmgcgIq8ClwNZPsfMAear6hEAVT1Yz3WuAv6lqlYqto7hvbry6DVjufXFTO5dvIXHrhnzhWciTlVWszm/5POJ7I8+O8KJimoA+sd1ZOqwnkwc1J2Jg3qQ1CPGk+cpLklN5PkPdrM65xAzzurV6p9vjGm+QCaOvkC+z34BcE6dY1IARGQtznDWL1T133WOuQZ4pE7bQyLyM+Bd4H5VLW+xqENMemoi9146jIf/vZ1BPWIYN7D757fGbs4/SoW7FO2wxC5cMa4vEwf1YGJS3GkNbQXChEFxdI2OYHlWoSUOY0KE12MDEUAyMBXoB6wWkVGqWgIgIr2BUcAyn3MeAA4AUcBTwI+AB+teWERuBW4FGDCgbY+f3zFlCDkHSnlsRS4AEWHCWX27cdPkJCYmxZGW1J3YmCiPo6xfZHgY04b3ZMWnhVRV1xARblVwjAl2gUwce4H+Pvv93DZfBcAGVa0EdolIDk4iyXDfnwW86b4PgKrud1+Wi8hzwH/X9+Gq+hROYiEtLa1N11UREX5z5WjSkuIYFN+JsQNiQ2q+ID21F0s27WPjZ0c4Z3APr8MxxjShyT/vRKSTiIS5r1NE5DIRifTj2hlAsogMEpEonCGnpXWOWYLT20BE4nGGrvJ83p8NvFInnt7uTwFmAp/4EUubFx0ZznXnDmTy0PiQShoAU4YlEBUexvKsQq9DMcb4wZ9xgdVAtIj0xbmz6Xrg+aZOUtUq4C6cYaZsYJGqbhORB0XkMvewZUCxiGQBK3HulioGEJEknB7LqjqXfklEtgJbgXjgV358BxPEOneIYNKQHizPLqQ9FN00JtT586epqGqZiNwCPK6qD4uIX89NqOpbwFt12n7m81qBue5W99zdOBPsddun+fPZJrSkpybykyWfsOPgcVISrXaVMcHMnx6HiMgk4Frgn25byz4JZtq96SMSAWy4ypgQ4E/iuAfnTqY33aGmwTjDSsa0mF7dohndr5slDmNCQJOJQ1VXqeplqvpbd5L8kKp+rxViM+1M+ohENuWXcPDYKa9DMcY0wp+7ql4Wka4i0gnnDqYsEbk38KGZ9iZ9pDNc9U52fQUEjDHBwp+hqlRVPYZz6+u/gEE4d1YZ06KGJXahf1xHlmcd8DoUY0wj/Ekcke5zGzOBpe7DeHbPpGlxIkL6iF6s3VnMCXcVQWNM8PEncTwJ7AY64ZQEGQgcC2RQpv1KT02koqqG1TlFXodijGmAP5Pjj6lqX1X9qjo+Ay5qhdhMOzQhqTuxMZF2d5UxQcyfyfFuIvKIiGS62x9weh/GtLiI8DCmDevJiu0HqXIr+xpjgos/Q1XPAqU4BQdn4QxTPRfIoEz7lp6aSElZJRm7j3gdijGmHv4kjiGq+nNVzXO3XwKDAx2Yab8uTEkgKsKKHhoTrPxJHCdF5PzaHRGZDJwMXEimvevUIYLJQ3qwPPuAFT00Jgj5kzhuB+aLyG4R2Q38CbgtoFGZdi89tRf5h0+yvbDU61CMMXX4c1fVZlU9GxgNjFbVsYBVqDUBNX1ETwCWb7PhKmOCjd/rdKrqMfcJcqinDLoxLaln12jG9I9lebYlDmOCzeku8CwtGoUx9UhPTWRLwVEOHLWih8YEk9NNHDZjaQIuPbW26KH1OowJJg0mDhEpFZFj9WylQB9/Li4iM0Rku4jkisj9DRwzS0SyRGSbiLzstl0kIpt8tlMiMtN9b5CIbHCv+Zq7nrlpg5J7dmZgjxi7LdeYINNg4lDVLqratZ6ti6o2ueSsiIQD84GvAKnAbBFJrXNMMs4iUZNVdSTOolGo6kpVHaOqY3Am4stw1jsH+C0wT1WHAkeAW5r7pU1ocIoeJrJuZzHHreihMUHjdIeq/DERyHUfGqwAXgUur3PMHGC+qh4BUNX6FmK4CviXu+654CSSxe57C3Gq9po2Kj01kYrqGlZtt6KHxgSLQCaOvkC+z36B2+YrBUgRkbUisl5EZtRznWuAV9zXPYASVa3987O+a5o2ZPzA7nSPibQ1OowJIk0OObXC5ycDU4F+OGXbR6lqCYCI9AZGAcuae2ERuRW4FWDAgAEtFa9pZRHhYUwbnsjyrANUVtcQGR7Iv3WMMf7wpzru3SLS/TSuvRfo77Pfz23zVYC7OJSq7gJycBJJrVnAm+7iUQDFQKyI1Ca8+q4JgKo+pappqpqWkJBwGuGbYJGemsixU1Vk7DrsdSjGGPwbqkoEMkRkkXuXlL/PcGQAye5dUFE4Q05L6xyzBKe3gYjE4wxd5fm8P5v/DFOhTuGilTjzHgA3An/zMx4Toi5MiadDRBhv291VxgQFf0qO/ASnF/AMcBOwQ0R+LSJDmjivCrgLZ5gpG1ikqttE5EERucw9bBlQLCJZOAnhXlUtBhCRJJwey6o6l/4RMFdEcnHmPJ7x43uaEBYTFcH5Q+NZnlVoRQ+NCQJ+zXGoqorIAeAAUAV0BxaLyHJVva+R894C3qrT9jPf6+KUL/lSCRNV3U09E9+qmodzx5ZpR9JTE3n304Nk7y8ltU9Xr8Mxpl3zZ47j+yKyEXgYWAuMUtU7gPHAlQGOzxgALh6RiAj2MKAxQcCfOY444Juqeqmq/rV2olpVa4CvBzQ6Y1wJXTowtn8sy7PttlxjvObPHMfPgR4i8j33DqtxPu9lBzQ6Y3ykp/bik73H2H/U1hEzxkv+DFX9FOcJ7R5APPCciPwk0IEZU1d6qrNGxzs2XGWMp/wZqroOmOCuO/5z4Fzg+sCGZcyXDUnozKD4TnZbrjEe8ydx7AOiffY70MBDd8YEkoiQnprI+rxijp2qbPoEY0xA+JM4jgLbROR5EXkO+AQoEZHHROSxwIZnzBelpyZSWa1W9NAYD/nzHMeb7lbrvcCEYkzTxg3oTo9OUSzPKuQbZ/u1LIwxpoU1mThUdaFbMiTFbdruUzvKmFYVHiZMG96Tf2+zoofGeMWfu6qmAjtwFmV6HMgRkQsDHJcxDUpPTaT0VBUb8qzooTFe8OfPtT8Al6jqFFW9ELgUmBfYsIxp2AXJCURHhtkaHcZ4xJ/EEamq22t3VDUHiAxcSMY0rmNUOOcPTbCih8Z4xJ/EsVFEnhaRqe62AMgMdGDGNOaS1ET2HT3Ftn3HvA7FmHbHn8RxO5AFfM/dsoA7AhmUMU2ZNqKnFT00xiON3lUlIuHAZlUdDjzSOiEZ07T4zh0YP6A7y7MK+UF6StMnGGNaTKM9DlWtBraLiC3abYJOemoiWfuPUXCkzOtQjGlX/Bmq6o7z5Pi7IrK0dgt0YMY0JT01EYB3sw96HIkx7Ys/T47/NOBRGHMaBid0ZnBCJ5ZnFXLjeUleh2NMu+FPj+OrqrrKdwO+6s/FRWSGiGwXkVwRub+BY2aJSJaIbBORl33aB4jI2yKS7b6f5LY/LyK7RGSTu43xJxbTNtUWPTx60ooZGNNa/Ekc6fW0faWpk9yJ9fnusanAbBFJrXNMMvAAMFlVRwL3+Lz9AvA7VR2Bs8a473jEvao6xt02+fEdTBt1SWoiVTXKe9ttuMqY1tJg4hCRO0RkKzBMRLb4bLuArX5ceyKQq6p5qloBvApcXueYOcB8VT0CoKoH3c9OBSJUdbnbflxVbQbUfMmY/t2J7xxlt+Ua04oa63G8DHwDWOr+rN3Gq+q1fly7L5Dvs1/gtvlKAVJEZK2IrBeRGT7tJSLyhoh8LCK/c3swtR5yk9g8EelQ34eLyK0ikikimUVFVoK7rQoPEy4ensiq7UVUVNV4HY4x7UKDiUNVj6rqblWdjfNLvxJQoHML3p4bASQDU4HZwAIRiXXbLwD+G5gADAZucs95ABjutscBP2og/qdUNU1V0xISElooXBOM0lMTKS2vYn1esdehGNMu+FMd9y6gEFgO/NPd/uHHtfcC/X32+/HllQMLgKWqWqmqu4AcnERSAGxyh7mqgCXAOABV3a+OcuA5nCEx046dnxxPx8hwG64yppX4Mzl+DzBMVUeq6ih3G+3HeRlAsogMctfzuAZn2MvXEpzeBiISjzNEleeeGysitV2FaTilThCR3u5PAWbirEho2rHoyHAuSI7nnWwremhMa/AnceTjLB/bLG5P4S5gGZANLFLVbSLyoIhc5h62DCgWkSxgJc7dUsXuE+v/DbzrTtALsMA95yW3bSsQD/yqubGZtic9NZH9R0/xyV4remhMoPnzAGAe8J6I/BMor21U1SZrV6nqW8Bbddp+5vNagbnuVvfc5cCXejaqOs2PmE07c/GIRMIElmcdYFS/bl6HY0yb5k+PYw/O/EYU0MVnMyZoxHWKIm1gHG/bPIcxAefPmuO/rNsmIv70VIxpVempiTz0Vjb5h8voHxfjdTjGtFmNPQD4vs/rF+u8/WHAIjLmNNUWPbS7q4yBrH3HuOLxtewsOt7i125sqKqTz+uz6rwnLR6JMWcoKb4TyT078062JQ5j5r2TQ+7B48R3qvcZ6TPSWOLQBl7Xt29MUEhPTWTDrsMcLbOih6b92lJQwvKsQr57/mC6xUS2+PUbSxyxInKFiFzpvv6mu10J2G0rJihNT02kukZZaUUPTTv2yPIcYmMi+c75SQG5fmOT3KuAy3xef8PnvdUBicaYMzSmXywJXTqwPKuQmWPrlkYzpu3b+NkR3ttexH0zhtEluuV7G9BI4lDVmwPyicYEUFiYMH1ET5Zu2kd5VTUdIsKbPsmYNuSR5duJ7xzFTQFc3Myf5ziMCSnpqYmcqKhm3U4remjal3U7i1mbW8ztU4YQExW4pyYscZg257wh8cREWdFD076oKo8s305i1w5cd+7AgH6WJQ7T5kRHhnNhcgLvZBdSU2M3AJr2Yc2OQ2TsPsJ/XTSU6MjADtH6U1b9WyLSxX39E3dxpXEBjcqYM5SemkjhsXK27m12fU5jQo6q8oflOfTpFs3VE/o3fcIZ8qfH8VNVLRWR84HpwDPAE4ENy5gzM214T8LDxIarTLuw4tODbM4v4e6Lk1vlhhB/Eke1+/NrwFOq+k+cgofGBK3unaJIG9jdEodp85y5jRwGxMVw1fh+rfKZ/iSOvSLyJHA18Ja7xrfNjZigl56ayPbCUvYUl3kdijEBs2zbAbbtO8b3Lk4mMrx1fjX78ymzcBZculRVS3DW+b43oFEZ0wIuSe0FwNtZBzyOxJjAqKlR5i3fweD4Tswc06fVPtefxNEb+Keq7hCRqcC3sOq4JgQM6BHDsMQuVvTQtFn/2Lqf7YWl3JOeQkQr9TbAv8TxOlAtIkOBp4D+wMv+XFxEZojIdhHJFZH7Gzhmlohkicg2EXnZp32AiLwtItnu+0lu+yAR2eBe8zV3PXNj6pWemkjG7iOUlFV4HYoxLaqquoZH38lhWGIXvj6qd6t+tj+Jo8ZdP/ybwB9V9V6cXkijRCQcmA98BUgFZotIap1jkoEHgMmqOhK4x+ftF4DfqeoIYCJQW7Xut8A8VR0KHAFu8eM7mHYq3S16uOJTK3po2pYlm/aRV3SCH6QnExbWuitd+JM4KkVkNnAD8A+3zZ/KWROBXFXNU9UK4FXg8jrHzAHmq+oRAFU9COAmmAh33XFU9biqlomIANOAxe75C4GZfsRi2qlRfbvR0y16aExbUVldw2Pv7mBkn65cOrJXq3++P4njZmAS8JCq7hKRQUDdFQHr0xfI99kvcNt8pQApIrJWRNaLyAyf9hL3YcOPReR3bg+mB1Di9oAauiYAInKriGSKSGZRUZEf4Zq2KCxMmJ6ayKqcIk5VVjd9gjEhYPHGAvYcLmNuegrO39Otq8nEoapZwH8DW0XkLKBAVX/bQp8fASQDU4HZwAIRiXXbL3A/dwIwGLipORdW1adUNU1V0xISElooXBOK0lMTKbOih6aNKK+q5o/v7mBM/1imDe/pSQz+lByZCuzAma94HMgRkQv9uPZenIn0Wv3cNl8FwFJVrVTVXUAOTiIpADa5w1xVwBJgHFCMs6hURCPXNOYLzhvSg05R4bxtw1WmDXgtI599R0951tsA/4aq/gBcoqpTVPVC4FJgnh/nZQDJ7l1QUcA1wNI6xyzB6W0gIvE4Q1R57rmxIlLbVZgGZKmqAiuBq9z2G4G/+RGLacc6RIQzZZgVPTSh71RlNX9akcuEpO5ckBzvWRz+JI5IVd1eu6OqOfgxOe72FO7CeXgwG1ikqttE5EERqV1ZcBlQLCJZOAnhXlUtVtVqnGGqd0VkKyDAAvecHwFzRSQXZ87jGX++qGnf0lMTKSotZ3NBidehGHPa/rL+Mw6WljM3fZhnvQ1ofOnYWhtF5GngL+7+tUCmPxdX1beAt+q0/czntQJz3a3uucuB0fW05+HcsWWM3y4a9p+ih2MHdPc6HGOarayiij+v2snkoT2YNKSHp7H40+O4HcgCvuduWcAdgQzKmJYWGxPFxKQ4uy3XhKyFH3zGoeMVzE0f5nUojfc43FtgN6vqcOCR1gnJmMBIT03kwX9ksfvQCZLiO3kdjjF+Kz1VyZOrdzJ1WALjB3rfY260x+HONWwXkQGtFI8xAZOemghgvQ4Tcp59fzclZZXMTU/xOhTAv6Gq7sA2EXlXRJbWboEOzJiW1j8uhuG9uljiMCHlaFklT7+fR3pqIqP7xXodDuDf5PhPAx6FMa3kktRE/rQyl8MnKojrZPUxTfBbsCaP0lNVQdPbgEZ6HCIyVEQmq+oq3w1nRcCC1gvRmJaTntqLGsWKHpqQcPhEBc+t3cXXRvVmRO+uXofzucaGqh4FjtXTftR9z5iQc1bfrvTuFs1yW9zJhIAnV+2krLKae6Ynex3KFzSWOBJVdWvdRrctKWARGRNAIsL0EYmszjlkRQ9NUDtYeoqF63Yzc0xfkhO7eB3OFzSWOBqbhenY0oEY01qmpyZysrKatbmHvA7FmAY98d5OKquV718cXL0NaDxxZIrInLqNIvJdYGPgQjImsM4dHEfnDhF2d5UJWlHSrYgAABp4SURBVPuPnuSlDXu4clzfoHzmqLG7qu4B3hSRa/lPokgDooArAh2YMYHyn6KHB6mp0VZfPc2YpsxfmYuqcve04OttQCM9DlUtVNXzgF8Cu93tl6o6SVVtZtGEtEtSEzl0vJyP863ooQku+YfLeC0jn1lp/ekfF+N1OPVq8jkOVV2JU7nWmDZj6rCeRLhFD4OhhIMxtf64Ygciwl3ThnodSoP8eXLcmDanW8dIzhkcZ7flmqCy+9AJXv9oL9+eOIDe3YL3HiRLHKbdSh+RyM6iE+QVHfc6FGMA+L93dxAZLtx50RCvQ2mUJQ7Tbk23oocmiOQeLGXJpr3cMCmJnl2ivQ6nUZY4TLvVr3sMqb27WuIwQWHeOzuIiQzntgsHex1KkwKaOERkhohsF5FcEbm/gWNmiUiWiGwTkZd92qtFZJO7LfVpf15Edvm8NyaQ38G0bempiWzcc4RDx8u9DsW0Y9n7j/HPLfu5efIgenTu4HU4TQpY4nAXgZoPfAVIBWaLSGqdY5KBB4DJqjoS59mRWidVdYy7XcYX3evz3qZAfQfT9qWnJqJW9NB4bN7yHLpERzDnguDvbUBgexwTgVxVzVPVCuBV4PI6x8wB5qvqEQBVtf96Tasa2acrfWM72nCV8czWgqO8nVXId88fTLeYSK/D8UsgE0dfIN9nv8Bt85UCpIjIWhFZLyIzfN6LFpFMt31mnfMeEpEtIjJPROrt14nIre75mUVFRWf8ZUzb5BQ97MmaHUWcrLCih16oqVGvQ/DUI8u3ExsTyXfOT/I6FL95PTkeASQDU4HZwAIRqS2uOFBV04BvA4+KSO39aQ8Aw4EJQBzwo/ourKpPqWqaqqYlJCQE8CuYUJee2otTlTW8b0UPW5WqMn9lLqN+sYwlH+/1OhxPbPzsCCu3F3HrhYPpEh0avQ0IbOLYC/T32e/ntvkqAJaqaqWq7gJycBIJqrrX/ZkHvAeMdff3q6MceA5nSMyY0zZxUBxdOkTYw4CtqLK6hgfe2Mrvlm0npkME97y2iYUf7PY6rFb3yPLt9OgUxY2TkrwOpVkCmTgygGQRGSQiUcA1QN21ypfg9DYQkXicoas8EeleOwTltk8Gstz93u5PAWYCnwTwO5h2ICoijKnDe/Ju9kGq2/mwSWsoPVXJLQszeTUjn7unDWXNfRdxSWoiP1+6jUffyUG1ffxvsD6vmLW5xdwxdQidOvizinfwCFjiUNUq4C5gGZANLFLVbSLyoIjU3iW1DCgWkSycelj3qmoxMAKnrPtmt/03qprlnvOSiGwFtgLxwK8C9R1M+5GemkjxiQo+3nPE61DatP1HT/KtP6/jg9xDPHzlaH54yTCiI8N5/NpxfGt8Px59Zwe/WLqtzc97qCqPvJ1Dzy4duO7cgV6H02wBTXOq+hbwVp22n/m8VmCuu/ke8wEwqoFrTmv5SE17N3VYApHhTtHDtKQ4r8Npk7L2HeM7z2dwvLyKZ2+awIUp/5l7jAgP4+GrRhMbE8mCNbs4erKS333rbCLDvZ6GDYz3cw/x4e7DPHj5SKIjw70Op9na5v8qxjRT1+hIzh3cw27LDZBVOUXMenIdIrD4jklfSBq1RIQff3UE980YxpJN+7jtxY1t8k43VeUPb+fQp1s0V0/o3/QJQcgShzGu9NRE8g6dIPegFT1sSa9l7OE7z2fQPy6GN++czPBeXRs8VkS4c+pQfn3FKFZuP8gNz27g6MnKVow28FZuP8im/BLuvjiZDhGh19sASxzGfG76CCt62JJUld8v286PXt/K+UPj+evtk+jVzb/ifd8+ZwB/mj2OTfklXPPUeg6WngpwtK1DVXlkeQ4D4mK4anw/r8M5bZY4jHH1ie3IWX272m25LaC8qpp7XtvEn1bmMntif56+MY3Ozbxz6Guje/PMjRPYfegE3/rzOvIPlwUo2tazbFshn+w9xvcuTg7p+ZvQjdyYAEgf0YuP80soKrWih6fraFklNzzzIX/btI/7Zgzj11eMOu1fkhemJPDSnHMoKavkqj9/wPYDpS0cbeupqVHmLc9hcHwnZo7p43U4Z8QShzE+/lP00IarTkf+4TK++cRaPt5Twv9dM4Y7pw7FeeTq9I0b0J2/3j4JgFlPruOjEL1l+p9b97O9sJTvT08mIoR7G2CJw5gvGNG7ixU9PE2b80u44vG1HDpewYu3TOTyMXVL052+lMQuLL79PLrHRHLtgg2szgmt+nNV1TXMeyeHlMTOfGN0aPc2wBKHMV8gIqSnJrJmxyHKKqq8DidkvL3tAFc/tY6OUeG8fsd5nDO4R4t/Rv+4GP56+3kkxXfiloUZ/GPLvhb/jED526Z95BWd4AfTUwgLO7MeWDCwxGFMHZekJlJeVcOaHVb00B/Pr93FbX/ZyLBeXXnjjskM7dk5YJ+V0KUDr956LmP6x3L3Kx/z8oY9AfusllJZXcP/vbuDkX26cunIXl6H0yIscRhTx4RBcXSNjrDhqiZU1ygP/j2LX/w9i/QRibw651wSugR+9bpuHSN54TvncNGwnvz4za3MX5kb1PWtXt9YwJ7DZcxNbxu9DbDEYcyXRIaHcdHwnqz41IoeNuRkRTV3vrSRZ9fu4ubJSTxx3Xg6RrXew2wdo8J58vrxzBzTh98t286v38oOyuRRXlXNH1fkMqZ/LNOG9/Q6nBZjicOYeqSnJnL4RAUbPwvNO3gC6dDxcmYvWM/bWYX87Oup/PwbIwn34C/pyPAwHpk1hpvOS2LBml3cu3gLVdU1rR5HYxZl5LO35CRz01PO+O6yYBJatXyNaSVTUmqLHh5g4iArelgrr+g4Nz2XwcHSU/z5uvGej9mHhQk//0Yq3WOimPdODkdPVvLH2WODonDgqcpq/rQylwlJ3bkgOd7rcFqU9TiMqUeX6EgmDYlneVZhUA6BeOHDXYf55hMfcKK8ilfmnOt50qglInx/ejK/vGwky7MKufm5DEpPeV/f6qUNeyg8Vs7c9GFtqrcBljiMaVB6aiK7i8us6CHw9837uO7pDcR1iuLNOyczdkB3r0P6khvPS+LRq8eQsfsw316wgeLj3j39X1ZRxRPv5XLekB5MGtLytyZ7zRKHMQ1Id4sevt2O765SVZ54byd3v/IxYwbE8sYd5zGgR4zXYTVo5ti+PHXDeHIKS/nWk+vYW3LSkzheWPcZh45X8MNLUjz5/ECzxGFMA3p1i2Z0v27t9rbcquoa/t+ST/jtvz/lsrP78OItE4mNifI6rCZNG57IX757DkWl5Vz1xAet3mMsPVXJn1ftZOqwBMYPbJvzYwFNHCIyQ0S2i0iuiNzfwDGzRCRLRLaJyMs+7dUissndlvq0DxKRDe41X3PXMzcmINJHJLIpv4SDx9pGWW9/HS+v4rsvZPLyhj3cOXUIj149JqTWjpiQFMdrt06islqZ9eQ6thSUtNpnP7d2NyVllcxNb5u9DQhg4hCRcGA+8BUgFZgtIql1jkkGHgAmq+pI4B6ft0+q6hh3u8yn/bfAPFUdChwBbgnUdzAmfaQzXPVO9kGPI2k9hcdOMevP61iz4xD/+81R3DdjeEg+uJbapyuLb59ETFQ4s59azwc7A18J4GhZJQvW5JGemsjofrEB/zyvBLLHMRHIVdU8Va0AXgUur3PMHGC+qh4BUNVG/+sU59aEacBit2khMLNFozbGx7DELvSP68g72e1juOrTA8eYOX8tnxWf4Jkb05g9cYDXIZ2RpPhOvH7HefTt3pGbns1g2bbArrXy9Pt5lJ6qatO9DQhs4ugL5PvsF7htvlKAFBFZKyLrRWSGz3vRIpLpttcmhx5AiarWVp+r75oAiMit7vmZRUWhVUnTBA8RIX1EL97PPcT2A6Vt+tbc93cc4ltPrKNGlUW3T2LqsLbxpHNi12gW3TaJkX27csdfNrIoM7/pk07D4RMVPPv+Lr42qjcjeje8PG5b4PXkeASQDEwFZgMLRKS2fzdQVdOAbwOPisiQ5lxYVZ9S1TRVTUtISGjJmE07M3NsH1SVSx9dzeTfrOCBN7bw70/2cywInhVoKYsy87npuQ/p270jb945mZF9unkdUouKjYnipe+ew+Sh8dy3eAsLVue1+Gc8uXonZZXV3DM9ucWvHWwC+eT4XqC/z34/t81XAbBBVSuBXSKSg5NIMlR1L4Cq5onIe8BY4HUgVkQi3F5Hfdc0pkWN7hfLqnsvYlVOEau2F/GPzft55cN8wsOE8QO6c2FKPFNSejKyT9eQmwtQdVale2xFLhckx/P4tePoEh3pdVgBERMVwdM3pjH3tc089FY2R8oquPfSlnk4r6i0nBc++IzLz+5DcmKXFog2uAUycWQAySIyCOeX+zU4vQdfS3B6Gs+JSDzO0FWeiHQHylS13G2fDDysqioiK4GrcOZMbgT+FsDvYAzgrEc+e+IAZk8cQGV1DR/vKWFVzkFW5xzi92/n8Pu3c+jRKYoLUxKYkpLABcnx9Ogc+EqxZ6Kiqob7X9/CGx/vZVZaPx46gyVeQ0WHiHAemz2WbjGRPP7eTo6UVfKrmWedca2tJ97bSUV1Dd+f3rbnNmoFLHGoapWI3AUsA8KBZ1V1m4g8CGSq6lL3vUtEJAuoBu5V1WIROQ94UkRqcIbTfqOqWe6lfwS8KiK/Aj4GngnUdzCmPpHhYUwcFMfEQXHce6lT9G/NDqc3sjqniDc/3osInNWnG1NSEpgyLIGx/WODarnQoycruf3FjazLK+aH6SncNe3Ml3gNFeFhwkMzz6J7TCTzV+7k2MlKHrn67NO+3fjA0VP8ZcNnfHNsXwbFd2rhaIOTtOXJvlppaWmamZnpdRimHaipUbbtO8aqnIOsyinioz0lVNcoXaIjOH9oPBemJHBhSgJ9Yzt6FmPBkTJufi6D3cUnePiq0Vwxtp9nsXjt6TV5/Oqf2VyQHM+frxtPpw7N/1v6p0s+4dWMPaz44VT6xwXvU/WnQ0Q2unPNX2y3xGFM4Bw9WckHuYdY7fZI9h11HiRM7tn5897IhKS4VqvmuqWghFsWZlJeWc2T16e1yTpKzbUoM5/7X9/C6H6xPHfTBLp38v+Z4oIjZVz0+/eYldafh64YFcAovWGJwxKH8ZiqknvwuDPJnlPEhl2HqaiqIToyjHMH93ASSUoCg+I7BWTY6N3sQu56+WPiOkXx/M0T2sUkrr+WbTvA3a98zMC4GF685Rx6dYv267wfLd7Cm5v2sureqfTu5l0vMlAscVjiMEHmZEU163cVfz43knfoBAD94zpyYbKTRM4bGk/n0xg+qevFdbv5+dJtnNW3G0/fmEbPLv79YmxPPth5iFtf2EhsTCQv3nJOk/MVuw+d4OJHVnH9uQP5xWUjWynK1mWJwxKHCXL5h8s+7418kHuIExXVRIQJaUndmZLSkykpCYzo3aVZvZGaGuU3//6Up1bnMX1ETx6bPZaYKFu/rSFbC45y43MfEiaw8DsTG32eZe5rm3jrk/2svu+iNpuILXFY4jAhpKKqho2fHWFVjtMbydp/DICELh2c3siwBC4YGt/oePypymrmLtrEW1sPcOOkgfzMoyVeQ03uwePc8MwGSk9V8cxNE+pdATL34HEumbeK714wmB9/dYQHUbYOSxyWOEwIO3jsFKt3HGJVThFrdhRRUlaJCJzdL/bzZ0fG9I/9PDEUHy9nzguZfJxfwv/76ghuOX9Qu7ndtiXsLTnJ9c9sYO+Rkzxx3TimDU/8wvt3vfwRKz49yJr7Lgr653XOhCUOSxymjaiuUbbuPcqq7UWsyjnIpvwSahS6dYzk/OR4zh3cg2fW5LH/6CkevXoMXxnV2+uQQ1Lx8XJufj6DbfuO8Ydvnc3MsU5ZvE8PHGPGo2v4r4uGcO+lwz2OMrAscVjiMG1USVkFa3OLP392pPBYOXGdolhwQxrjBwbfEq+h5Hh5FXMWZrIur5hffCOVmyYP4rYXM/kgt5g1P7ooJBa2OhMNJQ6bJTMmxMXGRPG10b352ujen9/yG9+5Q7OeRzD169whgudunsD3XvmYX/w9i0/2HWPZtkJ+MD2lzSeNxgRPDQRjzBkTEZITu1jSaEHRkeE8fu04vjW+H4s3FhAbE8l3zk/yOixPWY/DGGOaEBEexsNXjWZ4764MjItpsxWE/WWJwxhj/CAi3HL+IK/DCAo2VGWMMaZZLHEYY4xpFkscxhhjmsUShzHGmGaxxGGMMaZZLHEYY4xpFkscxhhjmsUShzHGmGZpF0UORaQI+Ow0T48HDrVgOIEWSvFarIETSvGGUqwQWvGeaawDVTWhbmO7SBxnQkQy66sOGaxCKV6LNXBCKd5QihVCK95AxWpDVcYYY5rFEocxxphmscTRtKe8DqCZQileizVwQineUIoVQivegMRqcxzGGGOaxXocxhhjmsUShzHGmGaxxNEIEZkhIttFJFdE7vc6noaIyLMiclBEPvE6Fn+ISH8RWSkiWSKyTUS+73VMDRGRaBH5UEQ2u7H+0uuYmiIi4SLysYj8w+tYmiIiu0Vkq4hsEpFMr+NpjIjEishiEflURLJFZJLXMTVERIa5/6a12zERuafFrm9zHPUTkXAgB0gHCoAMYLaqZnkaWD1E5ELgOPCCqp7ldTxNEZHeQG9V/UhEugAbgZlB+m8rQCdVPS4ikcD7wPdVdb3HoTVIROYCaUBXVf261/E0RkR2A2mqGvQP1InIQmCNqj4tIlFAjKqWeB1XU9zfZXuBc1T1dB+E/gLrcTRsIpCrqnmqWgG8ClzucUz1UtXVwGGv4/CXqu5X1Y/c16VANtDX26jqp47j7m6kuwXtX1si0g/4GvC017G0JSLSDbgQeAZAVStCIWm4LgZ2tlTSAEscjekL5PvsFxCkv9xCmYgkAWOBDd5G0jB36GcTcBBYrqpBGyvwKHAfUON1IH5S4G0R2Sgit3odTCMGAUXAc+4w4NMi0snroPx0DfBKS17QEofxjIh0Bl4H7lHVY17H0xBVrVbVMUA/YKKIBOVwoIh8HTioqhu9jqUZzlfVccBXgP9yh12DUQQwDnhCVccCJ4Cgnfes5Q6pXQb8tSWva4mjYXuB/j77/dw20wLc+YLXgZdU9Q2v4/GHOzSxEpjhdSwNmAxc5s4bvApME5G/eBtS41R1r/vzIPAmzhBxMCoACnx6m4txEkmw+wrwkaoWtuRFLXE0LANIFpFBbta+BljqcUxtgjvh/AyQraqPeB1PY0QkQURi3dcdcW6W+NTbqOqnqg+oaj9VTcL5/+sKVb3O47AaJCKd3JsjcId9LgGC8s5AVT0A5IvIMLfpYiDobuaox2xaeJgKnO6XqYeqVonIXcAyIBx4VlW3eRxWvUTkFWAqEC8iBcDPVfUZb6Nq1GTgemCrO3cA8GNVfcvDmBrSG1jo3pkSBixS1aC/zTVEJAJvOn9HEAG8rKr/9jakRt0NvOT+IZkH3OxxPI1yk3E6cFuLX9tuxzXGGNMcNlRljDGmWSxxGGOMaRZLHMYYY5rFEocxxphmscRhjDGmWSxxmKAlIsfdn0ki8u0WvvaP6+x/0ILXfrT2CWi3NEVqS1070ESk2qei6lKf9kEissGtFP2ae0sqItLB3c91309y20eJyPOefAkTcJY4TChIApqVOESkqWeUvpA4VPW8ZsbU0Of2AM51C0+iqt/1uuqvH/8Wvk6q6hh3u8yn/bfAPFUdChwBbnHbbwGOuO3z3ONQ1a1APxEZcObfwAQbSxwmFPwGuMD9K/gHbtHB34lIhohsEZHbAERkqoiscf9SznLblrgF9LbVFtETkd8AHd3rveS21fZuxL32J+46EVf7XPs9n/UYXnKfgK/rSuDzh9jcc9JqP0NEHhJnbY/1IpJY92T3aepnxVkD5GMRudxtXy8iI+tet5HjbxKRpSKyAnhXRF4QkZk+579Ue2xT3O85DafMBsBCoPZal7v7uO9f7PPv8necJ9hNW6OqttkWlBtw3P05FfiHT/utwE/c1x2ATJzqpVNxis8N8jk2zv3ZEaecRQ/fa9fzWVcCy3GqBSQCe3CeHp8KHMWpWRYGrMMp0Fc35oXAN3z238NZbwKcSrDfcF8/XPsd6pz/a+A693UszpownYAfAL9023sD25s4/iac+kq1338KsMR93Q3YBUTU8/lV7r/nepw1UgDicZYYqD2mP/CJ+/oToJ/PezuBePf1ZODvXv//yLaW36zHYULRJcANbrmSDUAPINl970NV3eVz7PdEZDPOL8L+Psc15HzgFXUq4hYCq4AJPtcuUNUaYBPOEFpdvXHKb9enAqgtV7KxgfMvAe53v9t7QDQwAFgEXOUeM4v//PXf0PHglIA/DKCqq3BqryXg1C96XVWr6vn8gaqahjM0+KiIDGngu/jjINDnDM43QcpqVZlQJMDdqrrsC40iU3F6HL7704FJqlomIu/h/GI9XeU+r6up/7+fk418RqWq1tb4aeh8Aa5U1e1fekOkWERGA1cDtzd2vIicg8+/hesF4Dqc4aN66yzpf6rV5rn/XmNxqhjHikiEm2x8K0XXVpEucOdSugHF7nvROP8epo2xHocJBaVAF5/9ZcAd4pRmR0RSpP5FdbrhTNyWichw4Fyf9yprz69jDXC1O4+SgLPq24fNiDUbGNqM4+taBtxdO08gImN93nsNZ5Gmbqq6xY/j63oeuAdA65mwF5HuItLBfR2PM9SU5Sa7lfynx3Mj8Df39VJ3H/f9FT7JMYUgrXZrzowlDhMKtgDV7qTyD3CWRc0CPhKRT4Anqf+v938DESKSjTPB7rtO+FPAltrJcR9vup+3GVgB3KdOSW1//RNnPuR0/Q/O8rRbRGSbu19rMU5vYZGfx3+BO/SWDTzXwCEjgEx3aG8l8BufBPMjYK6I5OIMDdZWX34G6OG2z+WLixtdhPPvYdoYq45rTAsTkfeBr2uQrUktIjHAVmCcqh4N8Gd1wJkfOr+BuRQTwqzHYUzL+yH/maAOCiIyHae38cdAJw3XAOB+Sxptk/U4jDHGNIv1OIwxxjSLJQ5jjDHNYonDGGNMs1jiMMYY0yyWOIwxxjTL/wcwUzNOaiN04wAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SGSvLewhA-q_"
      },
      "source": [
        "#**Experiment 5:**\r\n",
        "* **Optimizer =Adam**\r\n",
        "* **Learning rate =0.02**\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Dt0mZmrNA-rC"
      },
      "source": [
        "**Hyperparameters**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Qaaol6TbA-rD"
      },
      "source": [
        "#HYPERPERAMETERS\r\n",
        "batch_size = 256\r\n",
        "num_iters = 4000\r\n",
        "input_dim = 28*28 # num_features = 784\r\n",
        "output_dim = 2\r\n",
        "\r\n",
        "learning_rate = 0.02\r\n",
        "\r\n",
        "# Device\r\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hSreTFqmA-rE"
      },
      "source": [
        "**MAKING DATASET ITERABLE**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bmJKch4hA-rE"
      },
      "source": [
        "'''\r\n",
        "MAKING DATASET ITERABLE\r\n",
        "'''\r\n",
        "num_epochs = num_iters / (len(train_dataset) / batch_size)\r\n",
        "num_epochs = int(num_epochs)\r\n",
        "\r\n",
        "train_loader = torch.utils.data.DataLoader(dataset=train_dataset, \r\n",
        "                                           batch_size=batch_size, \r\n",
        "                                           shuffle=True)  \r\n",
        "\r\n",
        "test_loader = torch.utils.data.DataLoader(dataset=test_dataset, \r\n",
        "                                          batch_size=batch_size, \r\n",
        "                                          shuffle=False) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UpOwdlAdA-rE"
      },
      "source": [
        "**Logistic Regression Model**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7M-RCdRIA-rF"
      },
      "source": [
        "class LogisticRegressionModel(nn.Module):\r\n",
        "    def __init__(self, input_size, num_classes):\r\n",
        "        super().__init__()\r\n",
        "        self.linear = nn.Linear(input_size, num_classes)\r\n",
        "\r\n",
        "    def forward(self, x):\r\n",
        "        logits  = self.linear(x)\r\n",
        "        probas = F.softmax(logits, dim=1)\r\n",
        "        return logits, probas"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "x4aMcONRA-rF"
      },
      "source": [
        "**INSTANTIATE MODEL CLASS**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "e47MBpTjA-rF",
        "outputId": "43e2158a-2a75-49b1-e3a3-5cf83186d624"
      },
      "source": [
        "'''\r\n",
        "INSTANTIATE MODEL CLASS\r\n",
        "'''\r\n",
        "model = LogisticRegressionModel(input_size=input_dim,\r\n",
        "                                num_classes=output_dim)\r\n",
        "# To enable GPU\r\n",
        "model.to(device)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegressionModel(\n",
              "  (linear): Linear(in_features=784, out_features=2, bias=True)\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N9iVV3QDA-rG"
      },
      "source": [
        "**Optimizer setup**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eVHU5EoYA-rH"
      },
      "source": [
        "# INSTANTIATE OPTIMIZER CLASS\r\n",
        "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5tfNP_sMA-rH"
      },
      "source": [
        "**Function TRAIN THE MODEL**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MrMYPuHNA-rH",
        "outputId": "4a8ec726-b17e-4ef3-d95a-128532d3411b"
      },
      "source": [
        "'''\r\n",
        "TRAIN THE MODEL\r\n",
        "'''\r\n",
        "iteration_loss = []\r\n",
        "iter = 0\r\n",
        "for epoch in range(num_epochs):\r\n",
        "    for i, (images, labels) in enumerate(train_loader):\r\n",
        "\r\n",
        "        \r\n",
        "        images = images.view(-1, 28*28).to(device)\r\n",
        "        labels = labels.to(device)\r\n",
        "\r\n",
        "        # Clear gradients w.r.t. parameters\r\n",
        "        optimizer.zero_grad()\r\n",
        "\r\n",
        "        # Forward pass to get output/logits\r\n",
        "        logits, probas = model(images.float()) \r\n",
        "\r\n",
        "        # Calculate Loss: PyTorch implementation of CrossEntropyLoss works with logits, not probabilities\r\n",
        "        loss = F.cross_entropy(logits, labels)\r\n",
        "\r\n",
        "        # Getting gradients w.r.t. parameters\r\n",
        "        loss.backward()\r\n",
        "\r\n",
        "        # Updating parameters\r\n",
        "        optimizer.step()\r\n",
        "\r\n",
        "        iter += 1\r\n",
        "\r\n",
        "        if iter % 500 == 0:\r\n",
        "            # Calculate Accuracy         \r\n",
        "            correct = 0\r\n",
        "            total = 0\r\n",
        "            # Iterate through test dataset\r\n",
        "            for images, labels in test_loader:\r\n",
        "               \r\n",
        "                images = images.view(-1, 28*28).to(device)\r\n",
        "\r\n",
        "                # Forward pass only to get logits/output\r\n",
        "                logits, probas = model(images.float())\r\n",
        "\r\n",
        "                # Get predictions from the maximum value\r\n",
        "                _, predicted = torch.max(probas, 1)\r\n",
        "\r\n",
        "                # Total number of labels\r\n",
        "                total += labels.size(0)\r\n",
        "\r\n",
        "\r\n",
        "                # Total correct predictions\r\n",
        "                if torch.cuda.is_available():\r\n",
        "                    correct += (predicted.cpu() == labels.cpu()).sum() \r\n",
        "                else:\r\n",
        "                    correct += (predicted == labels).sum()\r\n",
        "\r\n",
        "            accuracy = 100 * correct.item() / total\r\n",
        "\r\n",
        "            # Print Loss\r\n",
        "            iteration_loss.append(loss.item())\r\n",
        "            print('Iteration: {}. Loss: {}. Accuracy: {}'.format(iter, loss.item(), round(accuracy, 2)))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Iteration: 500. Loss: 0.7139192819595337. Accuracy: 50.96\n",
            "Iteration: 1000. Loss: 0.6222255825996399. Accuracy: 49.85\n",
            "Iteration: 1500. Loss: 0.658583402633667. Accuracy: 51.22\n",
            "Iteration: 2000. Loss: 0.5988099575042725. Accuracy: 50.92\n",
            "Iteration: 2500. Loss: 0.6515288352966309. Accuracy: 51.83\n",
            "Iteration: 3000. Loss: 0.743750274181366. Accuracy: 48.85\n",
            "Iteration: 3500. Loss: 0.7010840773582458. Accuracy: 50.11\n",
            "Iteration: 4000. Loss: 0.6737802028656006. Accuracy: 50.86\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gr44jrThA-rI"
      },
      "source": [
        "**Loss show**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        },
        "id": "r93viw0aA-rI",
        "outputId": "f59e7fe3-ccb9-4330-ca46-db977d767a2a"
      },
      "source": [
        "import matplotlib\r\n",
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "print (iteration_loss)\r\n",
        "plt.plot(iteration_loss)\r\n",
        "plt.ylabel('Cross Entropy Loss')\r\n",
        "plt.xlabel('Iteration (in every 500)')\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0.7139192819595337, 0.6222255825996399, 0.658583402633667, 0.5988099575042725, 0.6515288352966309, 0.743750274181366, 0.7010840773582458, 0.6737802028656006]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEGCAYAAAB/+QKOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3zV9fX48dfJhjBDwgggKwxZCRpwoBZRERSi1hbB1tbdbxXrqLba1lFtf9Va66raukdF6zYsceMECUhCwozMkAAhrECArPP7436i15hxA/fmc8d5Ph6fh/d+7mecIOTcz3uct6gqxhhjTH1RbgdgjDEmOFmCMMYY0yBLEMYYYxpkCcIYY0yDLEEYY4xpUIzbAfhLcnKy9u3b1+0wjDEmpCxZsmSHqqY09FnYJIi+ffuSk5PjdhjGGBNSRGRjY59ZE5MxxpgGWYIwxhjTIEsQxhhjGmQJwhhjTIMsQRhjjGmQJQhjjDENsgRhjDGmQZYgjDEBl1e0m49Xb3c7DNNCliCMMQH3+9eXc/lzOSzdtMvtUEwLWIIwxgRU4fZyVpbsRYFrZn7N7opKt0MyPgpoghCRiSKyWkQKReTmBj6/X0SWOdsaEdld7/MOIlIkIv8KZJzGmMDJzi0hSuDfPz+W7eUHufHVPGwly9AQsAQhItHAI8AkYCgwXUSGeh+jqteraoaqZgAPA2/Uu8xdwCeBitEYE1iqyqzcYo7v34Uzhnbj5klH8/7KbTz12Xq3QzM+COQTxBigUFXXqWol8DJwThPHTwdeqnsjIscC3YB3AxijMSaACor3sn7HfrLSUwG4dGxfzhjajXveWcWyzbubOdu4LZAJoiew2et9kbPvB0SkD9AP+NB5HwXcB9zY1A1E5EoRyRGRnNLSUr8EbYzxn+zcYmKjhYnDuwMgItz7k5F0bZ/AjJlL2VNR5XKEpinB0kk9DXhNVWuc91cBc1W1qKmTVPVxVc1U1cyUlAbLmRtjXFJbq8zOLeaUgSl0ahv37f5ObeP414Wj2LrnIDe9lmv9EUEskAliC9Db630vZ19DpuHVvAScAMwQkQ3AP4BfiMjdgQjSGBMYSzbtonjPQaY4zUveRh3Vmd9PHMK7K7bx7BcbWj8445NALhi0GBgoIv3wJIZpwIX1DxKRIUBn4Mu6far6M6/PLwYyVfUHo6CMMcFrVm4xCbFRnDG0W4OfX35yPxauK+P/zV3JsX06M7JXp1aO0DQnYE8QqloNzADmAyuBV1S1QETuFJEsr0OnAS+rPWcaEzaqa2qZu7yE04Z0IzG+4e+hIsI/fppOSrt4rp65lD0HrD8i2AS0D0JV56rqIFUdoKp/dfbdpqrZXsfc0dTTgao+q6ozAhmnMca/vlxXxo59lQ02L3nrnBjHwxeOonj3QW5+3eZHBJtg6aQ2xoSR7GXFtI+PYdzg5gePHNsniZvOHMy8/K28sLDR5ZGNCyxBGGP86lB1De8UbGXCsO4kxEb7dM6VJ/fn1MEp/GX2SvK37AlwhMZXliCMMX61YHUp5QermZLew+dzoqKE+6ZmkJQYx9Uzl1J+0PojgoElCGOMX83KKyEpMY6xacktOi/J6Y8o2nWAm99Ybv0RQcAShDHGbyoqq3l/xTYmDe9ObHTLf72M7pvEDWcMYk5eCS8u2hSACE1LWIIwxvjN+yu3c6Cq5tvaS4fj1z8awCmDUrhz9goKiq0/wk2WIIwxfpO9rJjuHRIY3TfpsK8RFSXcPzWdzm1jmTHza/YdqvZjhKYlLEEYY/xiT0UVC9ZsZ/LIHkRFyRFdq0u7eB6aNoqNZfv5g/VHuMYShDHGL+YXbKWqRpudHOer4/p34YYzBpGdW8zLizc3f4LxO0sQxhi/mJVXTJ8ubRnZq6PfrnnVuDROHpjMHdkFrCzZ67frGt9YgjDGHLHS8kN8XriDKSNTETmy5iVvUVHCP6dm0KFNLFfPXMp+649oVZYgjDFHbF5+CbUKWRn+aV7yltI+ngenZbBhx37+9Fa+9Ue0IksQxpgjlr2smMHd2jOoW/uAXP/EAclce9og3vx6C6/mNLmOmPEjSxDGmCOyZfcBcjbuCsjTg7cZ49M4cUAXbsvOZ/XW8oDey3hYgjDGHJHZucUATB7pe+2lwxEdJTwwLYN28Z7+iIpK648INEsQxpgjMiuvmPTenejTJTHg9+raPoEHp2XwTek+bn2rIOD3i3SWIIwxh21d6T7yt+xlSoCfHryNTUvmmvEDeX1pEa/m2PyIQLIEYYw5bLNySxCBySMD2/9Q37WnDeT4/knc9nYBa7dZf0SgBDRBiMhEEVktIoUi8oNlRUXkfhFZ5mxrRGS3sz9DRL4UkQIRyRORCwIZpzGm5VSV7NwtjOmbRPeOCa167+go4aFpo0iMj+bqmUs5UFnTqvePFAFLECISDTwCTAKGAtNFZKj3Map6vapmqGoG8DDwhvNRBfALVR0GTAQeEJFOgYrVGNNyK0vK+aZ0f8BHLzWma4cE7r8gg7Xb93F7dr4rMYS7QD5BjAEKVXWdqlYCLwPnNHH8dOAlAFVdo6prndfFwHag+cVtjTGtJju3mJgoYdLw1ut/qO/kgSlcPS6NV3KKeGOpzY/wt0AmiJ6Adw9SkbPvB0SkD9AP+LCBz8YAccA3AYjRGHMYVJVZucWcNDCZpMQ4V2O57vSBjOmbxJ/eyqdw+z5XYwk3wdJJPQ14TVW/15AoIj2AF4BLVLW2/kkicqWI5IhITmlpaSuFaoxZumk3W3YfYEord043JCY6ioemjyIhNpqrX7T+CH8KZILYAvT2et/L2deQaTjNS3VEpAMwB/ijqi5s6CRVfVxVM1U1MyXFWqCMaS2zcouJi4liwrBubocCQPeOCfxzajqrt5Xz51k2P8JfApkgFgMDRaSfiMThSQLZ9Q8SkSFAZ+BLr31xwJvA86r6WgBjNMa0UE2tMmd5CeMHd6V9Qqzb4Xxr3OCu/HrcAF5evJm3lzX2XdS0RMAShKpWAzOA+cBK4BVVLRCRO0Uky+vQacDL+v0SjVOBU4CLvYbBZgQqVmOM7xatK6O0/JBro5ea8tszBpHZpzN/eGM560qtP+JISbiUzs3MzNScnBy3wzAm7N38eh6zcotZcusZJMRGux3OD5TsOcBZD35K945tePOqE4MyxmAiIktUNbOhz4Klk9oYEwIqq2uZl7+VCcO6B+0v3h4d2/DPqRmsLNnLnbNXuB1OSLME4QiXJyljAunTtaXsOVDFlHT35j744tQhXfnVKf2ZuWgTs5xqs6blIj5BFO2qYNKDn/Luim1uh2JM0JuVW0yntrGclBb8owZvPHMwxxzViVveWM6GHfvdDickRXyC6N4hge17D5Jt3zKMadKByhreXbGNScO7ExcT/L86YqOjePjCY4iOEq6euZSDVTY/oqWC//9ygMVER3HWiB58sHKbLYhuTBM+XLWdisoapqQH3+ilxvTs1Ib7fppOQfFe/jpnpdvhhJyITxDgWWj9YFUt71kzkzGNys7dQtf28RzXr4vbobTI6UO7ccXJ/Xhh4UbmLi9xO5yQYgkCOPaozqR2TLBmJmMasfdgFR+tLuXskT2IjhK3w2mx300cQkbvTvz+tTw2lll/hK8sQQBRUcKU9FQ+WVPKrv2VbodjTNB5t2AbldW1IdW85C02OoqHp49CBK6euZRD1dYf4QtLEI4p6alU1yrz8re6HYoxQWdWbjG9OrdhVO/QXZald1Jb7v1pOvlb9vK3uavcDickWIJwDEvtQP+URLJzrYaLMd7K9h3is8IdTElPRST0mpe8nTmsO5eO7cezX2zgnXzrj2iOJQiHiJCVnsqi9TvZuueg2+EYEzTm5W+lplbJCtHmpfpunjSE9F4duem1PDbvrHA7nKBmCcJLVnoqqjA7zzqrjamTnVtMWtd2DOne3u1Q/CIuJop/XXgMADNmLqWy+gdLzRiHJQgv/VPaMbxnB5uab4yjZM8BFm/YSVYYNC95653Ulnt/MpLcoj3cPc/6IxpjCaKerPRUcov22NR8Y4A5eSWoErKjl5oycXgPLj6xL09/vp53C2xwSkMsQdQz2VlC0Z4ijPH8OxjRsyP9khPdDiUgbjlrCCN6duTGV3OtP6IBliDqSe3UhjF9k8jOLbYKryaibdixn9yiPUFfufVIxMdE868LR6EK17z0tfVH1NNsghCRRBGJcl4PEpEsEQmedQYDYEpGKmu372PV1nK3QzHGNXWDNeqeqsNVny6J3H3+SJZt3s29860/wpsvTxCfAAki0hN4F7gIeDaQQbntrOHdiY4SK71hIlp2bjGj+3YmtVMbt0MJuLNH9uCi4/vwxKfred9qsn3LlwQhqloB/Bh4VFV/CgwLbFju6tIunpPSkpllzUwmQq3eWs6abfvCZu6DL/549tEM7dGB376ay5bdB9wOJyj4lCBE5ATgZ8AcZ59Paw2KyEQRWS0ihSJycwOf3y8iy5xtjYjs9vrslyKy1tl+6cv9/CkrPZWiXQdYuml38wcbE2ayc7cQHSVMGhG+/Q/1JcRG88jPjqGmVrlm5lKqaqw/wpcEcR1wC/CmqhaISH/go+ZOEpFo4BFgEjAUmC4iQ72PUdXrVTVDVTOAh4E3nHOTgNuB44AxwO0i0tn3H+vITRjWjfiYKBvNZCKOqjIrt4QTB3QhuV282+G0qn7JifztxyNYumk3lz67mL0Hq9wOyVXNJghVXaCqWap6j9NZvUNVf+PDtccAhaq6TlUrgZeBc5o4fjrwkvP6TOA9Vd2pqruA94CJPtzTb9onxDJ+SFdm55VQbd8kTATJLdrDpp0VYTn3wRdT0lP5+/kj+fKbMn7y2BcRPfzVl1FMM0Wkg4gkAvnAChG5yYdr9wQ2e70vcvY1dI8+QD/gw5acKyJXikiOiOSUlpb6EFLLZKWnsmPfIRau2+n3axsTrGblFhMXHcWZw7q7HYprpo7uzXOXjqFkz0HOe/Rzlm2OzKZmX5qYhqrqXuBcYB6eX+QX+TmOacBrqtqiIu2q+riqZqpqZkqK/xdRP3VIV9rFx1iFVxMxamqV2XnF/GhwCh3bhPVo9maNTUvmzatOpE1cNBf850vmReBqdL4kiFhn3sO5QLaqVgG+DO3ZAvT2et/L2deQaXzXvNTScwMmITaaCcO6MS9/qy0wYiLC4g072bb3UESNXmpKWtf2vHnVWIalduDXLy7l3wu+iaiRjb4kiP8AG4BE4BOnOWivD+ctBgaKSD8RicOTBLLrHyQiQ4DOwJdeu+cDE0Sks9M5PcHZ1+qy0lMpP1jNgtX+b8IyJthk5xbTJjaa047u6nYoQSO5XTwzrzieySN7cPe8VdzyxvKIGeHkSyf1Q6raU1XPUo+NwKk+nFcNzMDzi30l8IozCupOEcnyOnQa8LJ6pWVV3QnchSfJLAbudPa1urFpySQlxtmkORP2qmpqmbe8hDOGdqNtXIzb4QSVhNhoHpo2iqtPHcDLizdzyTORMcKp2b8FItIRz5DTU5xdC4A7gT3Nnauqc4G59fbdVu/9HY2c+zTwdHP3CLTY6CjOGtGd15YUsf9QNYnx9g/HhKfPCnewq6IqYkcvNScqSrjpzCH06ZLIH95YzvmPfsHTF4+md1Jbt0MLGF+amJ4GyoGpzrYXeCaQQQWbrPSeHKyq5f2VNgXfhK9Zy4rpkBDDKYOS3Q4lqE3N7M3zl41h217PCKevN+1yO6SA8SVBDFDV2535DOtU9c9A/0AHFkwy+3SmR8cEspdZM5MJTweranh3xTYmDu9OfIxPhRIi2okDknnjqrG0jYth2uMLmRumI5x8SRAHROSkujciMhaIqEIlUVHClPRUPllbyu6KSrfDMcbvPlq1nX2HqslKb3CqkmlAWtd2vHnViQzv2ZGrXlzKYx+H3wgnXxLE/wGPiMgGEdkA/Av4VUCjCkJZ6alU1Sjz8m3lKRN+ZuUVk9wujuP7J7kdSkjp0i6eFy8/jinpqdzzzipufj28Rjj5MoopV1XTgZHASFUdBYwPeGRBZlhqB/onJ1ozkwk75Qer+GDlds4e0YOYaFtDrKUSYqN58IIMrhmfxv9yNnPxM1+x50B4jHDy+W+Dqu51ZlQD3BCgeIKWiKeZaeH6MrbtPeh2OMb4zfsrt3GoutZGLx2BqCjhtxMG84+fpvPV+p2cHyY1nA7364L4NYoQkZWRiirMzgvPDikTmbKXFdOzUxuOOapVCyaHpZ8c24vnLz2O0vJDnPvI5ywN8RFOh5sgwqsnxkcDUtoxLLWDTZozYWPX/ko+XbuDySN7EBUVkd/7/O6EAV1446oTaZcQw/THFzInhL9QNpogRKRcRPY2sJUDEfssmpWeSu7m3Wws2+92KMYcsXn5W6muVWte8rMBKe1486qxjOjZkatnLuWRjwpDcoRTowlCVduraocGtvaqGrHTiSc7/5BsISETDmblFtM/OZFhqR3cDiXsJCXG8d/LjyMrPZV756/m96/nUVkdWiOcbMhCC/Xs1IbRfTtbM5MJedv2HmTh+jImp6ciYs1LgZAQG82D0zL4zfg0Xskp8oxwqgidEU6WIA5DVnoqa7btY9VWX4raGhOc5uSVoApZ6ZGz7rQbRIQbJgzmvp+ms3jDTn782OdsKguNEU6WIA7DWSN6EB0lNifChLTs3GKO7tGBtK7t3Q4lIpx/bC9euOw4duyr5LxHP2fJxuAf4eTLkqPXOGsyGEeXdvGclJbMrLzikOx4MmbzzgqWbd5tCwO1suP7e41wemIhs/OC+0umL08Q3YDFIvKKiEwUa6wEPM1Mm3ce4OsIXavWhLa6PrTJI615qbXVjXBK79WRGTO/DuoRTr6U2vgTMBB4CrgYWCsi/09EBgQ4tqA2YVg34mOirJnJhKRZucUcc1SnsF7LIJjVjXA6J8Mzwul3rwXnCCef+iCc1d62Ols1niVCXxORvwcwtqDWPiGW8UO6Mmd5CTW1wZn9jWnI2m3lrNpabnMfXBYfE80DF2Rw7WkDeXVJEb98OvhGOPnSB3GtiCwB/g58DoxQ1V8DxwLnBzi+oJaVnkpp+SEWritzOxRjfDYrt5gogbOtecl1IsL1Zwzin1PTydkYfCOcfHmCSAJ+rKpnquqrqloFoKq1wOSmTnT6LFaLSKGI3NzIMVNFZIWIFIjITK/9f3f2rRSRh4Kx7+PUIV1pFx9jzUwmZKgq2bnFHN+/C13bJ7gdjnH8+Jhe/Pey4yjbX8m5j37Oko073Q4J8K0P4nagi4j8xhnRdIzXZysbO09EooFHgEnAUGC6iAytd8xA4BZgrKoOA65z9p8IjMVTYnw4MBr4UQt/toBLiI1mwrBuzMsv4VB1jdvhGNOs/C172VBWYaOXgtBx/bvw5lVj6ZAQw/QnFgXFZFxfmphuBZ4DugDJwDMi8icfrj0GKHSWKa0EXgbOqXfMFcAjqroLQFW3O/sVSADigHggFgjKBaGz0lPZe7CaT9bscDsUY5qVnbuF2Ghh4vDubodiGtAvOZE3rxpLRq9O/Oalr/nXh2tdHeHkSxPTz4HRzrrUtwPHAxf5cF5PYLPX+yJnn7dBwCAR+VxEForIRABV/RL4CChxtvkNPa2IyJUikiMiOaWlpT6E5H9j05JJSowLimxvTFNqa5XZeSWcMjCFTm3j3A7HNKJzYhwvXD6G80b15B/vruHGV90b4eRLgijG822+TjywxU/3j8EzhHYcMB14QkQ6iUgacDTQC09SGS8iJ9c/WVUfV9VMVc1MSUnxU0gtExsdxVkjuvP+im1UVFa7EoMxvsjZuIuSPQdt9FIIiI+J5p9T07nu9IG8vrSIXzy9iN0Vla0ehy8JYg9QICLPisgzQD6w2+k4fqiJ87YAvb3e9+KHiaUIyFbVKlVdD6zBkzDOAxaq6j5V3QfMA07w7UdqfVnpPTlQVcN7K4KyFcwYwDN6KSE2ijOGdnM7FOMDEeG60wfxwAUZLN24mx8/+kWrLzPgS4J4E/gDniafj4E/Am8DS5ytMYuBgSLST0TigGlAdr1j3sLz9ICIJONpcloHbAJ+JCIxIhKLp4O60Q5xt2X26UyPjglWAtwEreqaWuYuL+G0Id1IjI/Yav0h6dxRPfnv5cexq6KS8x79gpwNrTfCyZdRTM8BL/FdQpipqs/VbU2cVw3MAObj+eX+iqoWiMidIpLlHDYfKBORFXgS0E2qWga8BnwDLAdygVxVnXXYP2WARUV51qtesKbUlcdAY5rzxTdllO2vtOalEDWmXxJvXjWWjm1iufDJRby9zF+t/E3zZRTTOGAtniGrjwJrROQUXy6uqnNVdZCqDlDVvzr7blPVbOe1quoNqjpUVUeo6svO/hpV/ZWqHu18dsNh/nytJis9laoa5Z38rW6HYswPZOcW0z4+hnGD3emrM0eub3Iib/z6RDJ6d+Lal5fx8AeBH+HkSxPTfcAEVf2Rqp4CnAncH9CoQtCw1A70T0600Uwm6ByqrmF+/lYmDOtOQmy02+GYI9A5MY4XLvOMcLrvvcCPcPIlQcSq6uq6N6q6Bs+8BONFxNPM9OW6MrbtPeh2OMZ86+PVpZQfqmaKLQwUFupGOF1/+iBeX1rERU8FboSTLwliiYg8KSLjnO0JICcg0YS4rIxUVGF2XonboRjzrVm5xSQlxjE2LdntUIyfiAjXnj6QB6dl8PWm3Vz4xKKAFA31ZTjD/wFXA79x3n+Kpy/C1DMgpR3DUjuQnVvMZSf1czscY9h/qJr3V27j/GN6ERttC0iGm3MyepLaqQ27K6qIjvJ/ubomE4RTTylXVYcA//T73cNQVnoqf5u3io1l++nTJdHtcEyEe3/lNg5W1VrtpTA2um9SwK7d5FcKVa0BVovIUQGLIMxMdv4h2pwIEwxm5RbTvUNCQH+JmPDlyzNnZzwzqT8Qkey6LdCBhaqendowum9nG81kXLenoooFa0qZPLIHUQFofjDhz5c+iFsDHkWYyUpP5da3C1i1dS9DundwOxwTod4pKKGqRm1ynDlsvjxBnKWqC7w34KxABxbKzhrRg+gosYWEjKtm5ZbQp0tbRvbq6HYoJkT5kiDOaGDfJH8HEk66tItnbFoys/KKXa3lbiLX9vKDfPHNDqaMTCUIF2M0IaLRBCEivxaR5cBgEcnz2tbjqZFkmpCVnsrmnQf4evNut0MxEWje8q3UqmdujjGHq6kniJnAFDwVWKd4bceq6s9aIbaQduawbsTFRFkzk3FFdm4xg7u1Z1C39m6HYkJYowlCVfeo6gZVnY5n3YYqPEuBtrNhr81rnxDL+MFdmbO8JCAzHI1pTNGuCpZs3GVPD+aI+VLNdQae9aDfA+Y42+wAxxUWsjJSKS0/xMJ1ZW6HYiJIXamXySOt9pI5Mr4Mc70OGOys02BaYPyQrrSLjyF7WbHVwTGtZlZuMem9O9lMfnPEfBnFtBnPsqOmhRJio5kwtBvz8ks4VF3jdjgmAnxTuo+C4r1MsacH4we+PEGsAz4WkTnAobqdqmq1mXwwJSOVN77ewidrdthawCbgZuUWIwKTR1r/gzlyvjxBbMLT/xAHtPfajA9OSkumc9tYK71hAk5Vyc4tZkzfJLp3THA7HBMGmn2CUNU/198nIj6tei4iE4EHgWjgSVW9u4FjpgJ34BkhlauqFzr7jwKeBHo7n52lqht8uW8wiY2O4qwRPXhj6RYqKqtpG2cLxpvAWFGyl3Wl+63UvPGbpibKfeb1+oV6H3/V3IWdUuGP4Jl1PRSYLiJD6x0zELgFGKuqw/B0iNd5HrhXVY8GxgDbm7tnsMpKT+VAVQ3vrdjmdigmjGXnFhMTJUwabv0Pxj+aamLyHgIxvN5nvszdHwMUquo6Va0EXgbOqXfMFcAjqroLQFW3AziJJEZV33P271PVCh/uGZRG902iR8cEKwFuAkZVmZ1bwkkDk0lKjHM7HBMmmkoQ2sjrht43pCeeEVB1ipx93gYBg0TkcxFZ6DRJ1e3fLSJviMjXInKv80TyPSJypYjkiEhOaWmpDyG5IypKmDyyBwvWlAZs7VgT2ZZu2sWW3QeYYp3Txo+aShCdROQ8ETnfef1jZzsf8Fd5yBhgIDAOmA48ISKdnP0nAzcCo4H+wMX1T1bVx1U1U1UzU1JS/BRSYGSl96SqRnknf6vbobQaVWXJxl1U1dS6HUrYm5VbQlxMFBOG2Ug54z9NJYgFQBYw2XldV4tpMvCJD9fegqeDuU4vZ5+3IiBbVatUdT2wBk/CKAKWOc1T1cBbwDE+3DNoDe/ZgX7JiRE1muk/n6zj/Me+4L5317gdSlirrqlldl4J4wd3pX1CrNvhmDDS6JAaVb3kCK+9GBgoIv3wJIZpwIX1jnkLz5PDMyKSjKdpaR2wG89TS4qqlgLjgZwjjMdVIsKU9FQe/nAt2/cepGuH8B6GmJ1bzN3zVtE+PoZnPl/PxSf2taGXAbJo/U527DtktZeM3/kyD+KwON/8ZwDzgZXAK6paICJ3ikiWc9h8oExEVgAfATepapmzFvaNwAdOyXEBnghUrK0lKz0V1e9q5YSrhevKuPGVXMb0S+LNq8dSq8qDH6x1O6ywlb2smMS4aMYP6ep2KCbMBHRQvqrOBebW23eb12sFbnC2+ue+B4wMZHytLa1rO4b26EB2bjGXhulY9cLt5Vz5fA5HdWnLExdl0rFtLD87rg8vLNzI5Sf3Y0BKO7dDDCuV1bXMyy9hwrDuJMT+YByHMUckYE8QpmFZGaks27ybTWUhO2q3Udv3HuSXTy8mPjaaZy4eTce2nvbwGePTiI+J4p/WF+F3n6wpZe/Baqak29wH43++lPv+qYi0d17/yRl6GtIdxm6qW0B+Vl54dVbvP1TNpc8tZldFJU//cjS9k9p++1lyu3guP7k/c5aXkFdkK+z506y8Yjq1jeWktOAexWdCky9PELeqarmInAScDjwFPBbYsMJXz05tyOzTOaxWmquuqWXGzKWsLCnnkQuPYUSvH46CvuLkfiQlxnHPO6tciDA8Haj0zM6fNLw7cTHWGGD8z5e/VXV1qs8GHlfVOXgK95nDlJWRyupt5azeWu52KEdMVbn17Xw+Wl3KXecM59RGOkrbJ8Ry9alpfF5Yxsw7qDsAABxzSURBVGdrd7RylOHpg1XbqKis+fap1Bh/8yVBbBGR/wAXAHNFJN7H80wjzhrRg+goITu3/rSQ0PPox9/w0lebufrUAVx4XNMr0f78+KPo2akN97yzilpbhvWIZS8rpmv7eI7r18XtUEyY8uUX/VQ8w1HPVNXdQBJwU0CjCnPJ7eI5cUAXZuWW4BnIFZre/LqIe+ev5rxRPblxwuBmj4+Pieb6MwaxfMse5kXQjPJA2F1RycerSzl7pOfLhjGB4EuC6AHMUdW1IjIO+Ck+VHM1TctKT2XTzgqWbQ7NTtsvCnfwu9fyOKF/F+45fyQivv2SOm9UTwZ1a8c/3l1tJTiOwH3vrqG6tpYLRvdu/mBjDpMvCeJ1oEZE0oDH8ZTPmBnQqCLAmU7HYiiW3li9tZxf/XcJ/ZIT+fdFx7aogzQ6SrjpzCGs37GfV3OKAhhl+FpetIf/LtrIL07oy5DuHdwOx4QxX/5l1zqzon8MPKyqN+F5qjBHoENCLKcOTmF2Xgk1IdQev23vQS555ivaxkXzzCVj6Nim5bV/Tj+6K5l9OvPA+2s4UGlrdbdEba3yp7fz6ZIYzw0TBrkdjglzviSIKhGZDvwCmO3ss4pgfpCV3pPS8kMsWlfmdig+2XeomoufWcyeA1U8ffFoenZqc1jXERF+P2kI28sP8ewXG/wbZJh7efFmcjfv5o9nD6GDFeYzAeZLgrgEOAH4q6qud4rv1V9hzhyG047uSmJcdEg0M1XV1PLr/y5hzbZyHv35sQxLPbKK76P7JjF+SFce+7iQPRVVfooyvO3cX8nf56/iuH5JnJtRf2kVY/yv2QShqivwFM5bLiLDgSJVvSfgkUWAhNhoJgzrzrz8rVRWB2+HraryxzeX8+naHfztvBH8aJB/Zu3edOZgyg9V8+iCQr9cL9z9/Z1VlB+s5q5zh/s8KMCYI+FLqY1xwFo860s/CqwRkVMCHFfEyEpPZc+BKj5ZE7wr4j30QSGv5BTxm9MGMtWPo2aO7tGBczN68uznG9i656DfrhuOlm7axcuLN3Pp2L4M6tbe7XBMhPCliek+YIKq/khVTwHOBO4PbFiR46SByXRuGxu0zUyvLSni/vfXcP4xvbj+9IF+v/4NZwxyyoFbIb/G1NQqt76VT7cO8Vx7unVMm9bjS4KIVdXVdW9UdQ3WSe03sdFRTBrRg/dWbKOistrtcL7n07Wl3Px6HielJfO3H48ISLNG76S2/Oy4PrySU8Q3pfv8fv1w8N+FGyko3sutk4fSLj6gFfqN+R5fEsQSEXlSRMY52xOE+OpuwSYrPZUDVTW8v3K726F8a2XJXn7936WkdW3HYz8/JqDF4OrKgd/37urmD44wpeWH+Me7qzkpLZmzR9joctO6fPlX/3/ACuA3zrYC+HUgg4o0Y/om0b1DQtBUeC3Zc4BLnllMu/gYnrlkdMDXOa4rBz53+VZyQ3RmeaD8bd5KDlbV8OdzhlnHtGl1TSYIEYkGclX1n6r6Y2e7X1UPtVJ8ESEqSpg8sgcL1mx3fcjn3oNVXPLMYvYfquaZS0bTo+PhzXVoqbpy4H+fb+XA63y1fidvLN3Claf0t5X4jCuaTBDO2tCrRaTpMp3miGVlpFJVo7xT4N561ZXVnrkOhdv38djPj+XoHq1XxqF9QiwzrBz4t6pqarn1rXx6dmrD1aemuR2OiVC+NDF1BgpE5AMRya7bfLm4iEwUkdUiUigiNzdyzFQRWSEiBSIys95nHUSkSET+5cv9QtmInh3p26Wta6OZVJWb38jj88Iy7j5/JCcNTG71GH5m5cC/9dwXG1i9rZzbpgylbZx1TBt3+PI379bDubDTPPUIcAZQBCwWkWxn4l3dMQOBW4CxqrpLROqvNnMX8Mnh3D/UiAhZ6an866NCtpcfpGv7hFa9//3vreGNpVu44YxB/OTYXq167zrxMdHccMYgfvtqLnPzS5g8MjIXwtm65yD3v7eGUwenMGFoN7fDMRGs0ScIEUkTkbGqusB7w7PCnC9lOMcAhaq6TlUrgZeBc+odcwXwiKruAlDVb4fxiMixQDfg3Zb9SKErKyOVWoU5ea3bzPS/xZt46MNCLsjszTXj3W3OOHdUTwZ3a899766J2HLgf5mzgqpa5Y4s65g27mqqiekBYG8D+/c4nzWnJ7DZ632Rs8/bIGCQiHwuIgtFZCKAiEThmaB3Y1M3EJErRSRHRHJKS4N3JrKv0rq25+geHVq1menj1dv5w5v5nDIohb+c534JB0858MGs37GfV3I2N39CmPm8cAez80q4atwA+nRJdDscE+GaShDdVHV5/Z3Ovr5+un8MMBAYB0wHnhCRTsBVwFxVbfJJRVUfV9VMVc1MSfFPfSC3ZaWn8vWm3WzeWRHwe+Vv2cPVLy5lcLf2PPqzY4iNDo6VZE9zyoE/+P7aiCoHXlldy21v53NUUlv+70cD3A7HmCYTRKcmPvNl7OMWPIsL1enl7PNWBGSrapWqrgfW4EkYJwAzRGQD8A/gFyJytw/3DHlT0j2ToQL9FLFl9wEufXYxHdvE8swlo4Nqhq53OfBnvljvdjit5snP1vFN6X7+nDWMhNhot8MxpskEkSMiV9TfKSKXA0t8uPZiYKCI9BOROGAaUH/001t4nh4QkWQ8TU7rVPVnqnqUqvbF08z0vKo2OAoq3PTq3JZj+3RmVgATxJ4DVVz89FccqKrh2UvH0K1D63aI+6KuHPi/P/7G9bkhrWHL7gM8/EEhE4Z249Qh9cdqGOOOphLEdcAlIvKxiNznbAuAy4Brm7uwswrdDGA+sBJ4RVULROROEclyDpsPlInICuAj4CZVDY3VcwIoKz2VVVvLWbOt3O/XPlRdw69eyGFD2X7+c9GxQV0Z9HcTI6cc+J2zClCU26YMdTsUY77VaIJQ1W2qeiLwZ2CDs/1ZVU9Q1a2+XFxV56rqIFUdoKp/dfbdpqrZzmtV1RtUdaiqjlDVlxu4xrOqOqPlP1roOmtED6IEv5feUFV+91oeC9ft5N6fpHPigNaf69ASQ7p34LwIKAf+0ertzC/YxjXjB9Krc1u3wzHmW74sGPSRqj7sbB+2RlCRLqV9PGPTksnOLUbVfxPG7p2/mreXFXPTmYM5d1RorEh2fZiXAz9YVcMd2QX0T0nkipP7ux2OMd8THMNWzA9MSU9l084Kcov2+OV6Ly7ayKMff8P0MUdx1bjQGSET7uXA/7NgHRvLKrgza3hAK+Yaczjsb2SQOnNYd+Kio/zSzPThqm3c+lY+pw5O4a4QrAo6Y3waCWFYDnxTWQWPflzI2SN7uFLaxJjmWIIIUh3bxDJucAqz84qpOYK6RHlFu7n6xa8ZltqRf114DDFBMtehJcKxHLiqcsesAmKihFvPto5pE5xC77dFBMnKSGV7+SEWrT+8gV2bd1Zw6bM5JCXG8dTFmSQG0VyHlro8zMqBv79yOx+u2s51pw+ie8fgG2ZsDFiCCGqnDelGYlz0Yc2J2F1RycXPfEVVTS3PXTq61Yv/+Zt3OfBP14Z2WZUDlZ6O6UHd2nHx2L5uh2NMoyxBBLE2cdGcMbQbc5dvpbLa98J1h6pruPKFJWzeeYDHLzqWtK7BO9ehJerKgf/9ndUhXQ78kY8K2bL7AHeeMzxoypsY0xD72xnksjJS2XOgyudvzbW1ym9fyeWr9Tv5x9R0juvfJcARtp66cuDLt+xhbr57CysdiXWl+3j8k3WcN6onx4fR/xsTnixBBLmT0lLo1DbW59pM98xfxey8Em6ZNISs9PBbTyGUy4GrKrdnFxAfE8UtZw1xOxxjmmUJIsjFxUQxaXgP3luxrdnKps9/uYH/LFjHRcf34cpTwnPSVSiXA5+Xv5VP1+7gtxMGhXyfkIkMliBCQFZ6KhWVNby/clujx7y3Yht3ZBdw+tFdw36hmVAsB77/UDV3zlrB0B4d+PnxfdwOxxifWIIIAWP6JdGtQ3yjzUzLNu/mmpeWMqJnRx6aPoroqPBNDhCa5cAf+mAtW/ce5K5zh4fkXBQTmexvagiIjhImj0xlwepS9hz4funrTWUVXPbsYlLax/PUxaMjZoH70X2TOG1IVx77+Bt2V1S6HU6T1mwr56nP1jM1sxfH9unsdjjG+MwSRIjISk+lsqaW+fnfFdLdtd8z16FGlWcvGUNyu3gXI2x9N00czL5D1Ty24Bu3Q2mUqnLrW/kkxsfw+4nWMW1CiyWIEDGyV0f6dGn7bTPTwaoaLn8+h6LdB3jyF5kMSGnncoStz7sceMmeA26H06Ds3GIWrd/J7yYOpkuEJXAT+ixBhAgRISs9lS++2cH2vQe54ZVlLN20iwcuyCCzb5Lb4bmmrhz4Qx+sdTuUH9h7sIq/zFlJeq+OTBt9lNvhGNNiliBCSFZ6KrUKFz65iLnLt/LHs47mrBE93A7LVcFcDvz+99awY98h7jp3eNgPHDDhyRJECBnYrT1DurencPs+Lj6xL5ed1M/tkIJCMJYDX1G8l+e+2MCFY45iZK9ObodjzGEJaIIQkYkislpECkXk5kaOmSoiK0SkQERmOvsyRORLZ1+eiFwQyDhDyR/OOpoZp6Zx6+ShYT3XoSWCrRx4ba1y69v5dGobx01nDnY7HGMOW8AShIhEA48Ak4ChwHQRGVrvmIHALcBYVR0GXOd8VAH8wtk3EXhAROxrGHDKoBRuPHOwNVnUc8Up/UlKjOOed1b5dZnWw/H60iKWbNzFzZOG0KltnKuxGHMkAvkEMQYoVNV1qloJvAycU++YK4BHVHUXgKpud/67RlXXOq+Lge1ASgBjNSGuXXwMM05N44tvyviscIdrceypqOLueas4tk9nfnJML9fiMMYfApkgegLexXKKnH3eBgGDRORzEVkoIhPrX0RExgBxwA8Gu4vIlSKSIyI5paWhvUaAOXJ15cDveWeVa+XA7313FbsqKrnrnOFE2VOeCXFud1LHAAOBccB04AnvpiQR6QG8AFyiqj8o3amqj6tqpqpmpqTYA0akqysHnr9lryvlwPOKdvPiok384oS+DE3t0Or3N8bfApkgtgC9vd73cvZ5KwKyVbVKVdcDa/AkDESkAzAH+KOqLgxgnCaM1JUD/8f81a1aDrym1jNjOrldPDdMGNRq9zUmkAKZIBYDA0Wkn4jEAdOA7HrHvIXn6QERScbT5LTOOf5N4HlVfS2AMZowU1cOfENZRauWA//f4s3kFu3hj2cdTYeE2Fa7rzGBFLAEoarVwAxgPrASeEVVC0TkThHJcg6bD5SJyArgI+AmVS0DpgKnABeLyDJnywhUrCa8tHY58J37K/n7/FUc3z+JczLCb5EmE7nE7SGB/pKZmak5OTluh2GCxOINO/npv7/kdxMHc9W4tIDe6/ev5fH60iLmXXsyA7uFx/rfJnKIyBJVzWzoM7c7qY0JiNYqB75k4y7+l7OZy07qZ8nBhB1LECZsfVsO/OPAlAOvrqnl1rfy6d4hgd+cNjAg9zDGTZYgTNj6thz4F4EpB/7iok2sKNnLrZOHkhgfGQs1mchiCcKEtbpy4A++799y4KXlh/jHu6s5eWAyZ43o7tdrGxMsLEGYsPZdOfDNFG73Xznwv81dyaGqWv6cNcyKJpqwZQnChL0Z49NoExvtt3Lgi9aV8cbXW7jylP70j8CV/EzksARhwl5dOfB5+UdeDryqppZb386nZ6c2XH1qYIfPGuM2SxAmIvirHPhzX2xgzbZ93D5lKG3iov0YoTHBxxKEiQj+KAe+dc9B7n9vDeOHdOWMod38HKExwccShIkYPzv+KHp1Pvxy4H+Zs4LqWuWOKdYxbSKDJQgTMbzLgc9Z3rJy4J8X7mB2XglXjUvjqC5tAxShMcHFEoSJKOdkeMqB3/eu7+XAD1XXcOvb+fTp0pZf/ah/gCM0JnhYgjARJTpK+N1ETznw/y32rRz4U5+tZ13pfu7IGkZCrHVMm8hhCcJEnPFDnHLgHzRfDnzL7gM8/EEhZw7rxqmDu7ZShMYEB0sQJuKICL+fNITS8kM8/fn6Jo+9c1YBALdNGdYaoRkTVCxBmIhUVw783wsaLwf+0ertzC/YxjWnpdGzU5tWjtAY91mCMBGrqXLgB6tquCO7gP4piVx+knVMm8hkCcJErKbKgf9nwTo2llVw1znDiYuxfyYmMgX0b76ITBSR1SJSKCI3N3LMVBFZISIFIjLTa/8vRWSts/0ykHGayNVQOfBNZRU8+nEhk0f2YGxasovRGeOugCUIEYkGHgEmAUOB6SIytN4xA4FbgLGqOgy4ztmfBNwOHAeMAW4Xkc6BitVErvrlwFWVO2YVEBMl/Onsoc1fwJgwFsgniDFAoaquU9VK4GXgnHrHXAE8oqq7AFR1u7P/TOA9Vd3pfPYeMDGAsZoI5l0O/L0V2/hw1XauP2MQ3TsmuB2aMa4KZILoCXjPRCpy9nkbBAwSkc9FZKGITGzBuYjIlSKSIyI5paWlfgzdRJLkdvFccYqnHPgtbyxncLf2/PLEvm6HZYzr3O59iwEGAuOA6cATItLJ15NV9XFVzVTVzJSUlACFaCLB5Sf3p0tiHGX7K7nr3OHERrv9T8MY9wVypfUtQG+v972cfd6KgEWqWgWsF5E1eBLGFjxJw/vcjwMWqYl47eJjeGj6KNbt2M+Yfkluh2NMUAjk16TFwEAR6SciccA0ILveMW/hJAIRScbT5LQOmA9MEJHOTuf0BGefMQEzNi2Zi47v43YYxgSNgD1BqGq1iMzA84s9GnhaVQtE5E4gR1Wz+S4RrABqgJtUtQxARO7Ck2QA7lTVnYGK1RhjzA/JkSy/GEwyMzM1JyfH7TCMMSakiMgSVc1s6DPriTPGGNMgSxDGGGMaZAnCGGNMgyxBGGOMaZAlCGOMMQ2yBGGMMaZBYTPMVURKgY1HcIlkYIefwgm0UIoVQiveUIoVQiveUIoVQiveI4m1j6o2WKsobBLEkRKRnMbGAgebUIoVQiveUIoVQiveUIoVQiveQMVqTUzGGGMaZAnCGGNMgyxBfOdxtwNogVCKFUIr3lCKFUIr3lCKFUIr3oDEan0QxhhjGmRPEMYYYxpkCcIYY0yDIj5BiMhEEVktIoUicrPb8TRFRJ4Wke0iku92LM0Rkd4i8pGIrBCRAhG51u2YmiIiCSLylYjkOvH+2e2YmiMi0SLytYjMdjuW5ojIBhFZLiLLRCSo6/KLSCcReU1EVonIShE5we2YGiMig50/07ptr4hc57frR3IfhIhEA2uAM/Asf7oYmK6qK1wNrBEicgqwD3heVYe7HU9TRKQH0ENVl4pIe2AJcG4Q/9kKkKiq+0QkFvgMuFZVF7ocWqNE5AYgE+igqpPdjqcpIrIByFTVoJ94JiLPAZ+q6pPOaphtVXW323E1x/l9tgU4TlWPZNLwtyL9CWIMUKiq61S1EngZOMflmBqlqp8AIbGynqqWqOpS53U5sBLo6W5UjVOPfc7bWGcL2m9PItILOBt40u1YwomIdAROAZ4CUNXKUEgOjtOAb/yVHMASRE9gs9f7IoL4l1ioEpG+wChgkbuRNM1pslkGbAfeU9VgjvcB4HdArduB+EiBd0VkiYhc6XYwTegHlALPOM13T4pIottB+Wga8JI/LxjpCcIEmIi0A14HrlPVvW7H0xRVrVHVDKAXMEZEgrIZT0QmA9tVdYnbsbTASap6DDAJuNppLg1GMcAxwGOqOgrYDwR13ySA0xSWBbzqz+tGeoLYAvT2et/L2Wf8wGnLfx14UVXfcDseXzlNCh8BE92OpRFjgSynXf9lYLyI/NfdkJqmqluc/24H3sTTvBuMioAir6fH1/AkjGA3CViqqtv8edFITxCLgYEi0s/JwNOAbJdjCgtOp+9TwEpV/afb8TRHRFJEpJPzug2egQur3I2qYap6i6r2UtW+eP7OfqiqP3c5rEaJSKIzUAGnuWYCEJQj8VR1K7BZRAY7u04DgnJgRT3T8XPzEngepyKWqlaLyAxgPhANPK2qBS6H1SgReQkYBySLSBFwu6o+5W5UjRoLXAQsd9r1Af6gqnNdjKkpPYDnnJEgUcArqhr0w0dDRDfgTc93BmKAmar6jrshNeka4EXnS+M64BKX42mSk3TPAH7l92tH8jBXY4wxjYv0JiZjjDGNsARhjDGmQZYgjDHGNMgShDHGmAZZgjDGGNMgSxDGdSKyz/lvXxG50M/X/kO991/48doP1M0IdkoyDPXXtQNNRGq8KoBme+3vJyKLnOrG/3OGeiIi8c77Qufzvs7+ESLyrCs/hAk4SxAmmPQFWpQgRKS5uTzfSxCqemILY2rsvl2A450Ciqjq5W5XqvXhz8LbAVXNcLYsr/33APerahqwC7jM2X8ZsMvZf79zHKq6HOglIkcd+U9ggo0lCBNM7gZOdr7VXu8Uz7tXRBaLSJ6I/ApARMaJyKfON98Vzr63nEJwBXXF4ETkbqCNc70XnX11TyviXDvfWafgAq9rf+y1HsCLzqzw+s4Hvp3s5ZyTWXcPEfmreNaWWCgi3eqf7Mwuflo8a1B8LSLnOPsXisiw+tdt4viLRSRbRD4EPhCR50XkXK/zX6w7tjnOzzkeT3kJgOeAumud47zH+fw0rz+XWXhmdJtwo6q22ebqBuxz/jsOmO21/0rgT87reCAHT7XNcXiKqPXzOjbJ+W8bPGUcunhfu4F7nQ+8h2cGfTdgE57Z1OOAPXjqckUBX+IpNFc/5ueAKV7vP8az3gF4KpdOcV7/ve5nqHf+/wN+7rzuhGddkkTgeuDPzv4ewOpmjr8YT/2gup//R8BbzuuOwHogpoH7Vzt/ngvxrNMBkIyn/H3dMb2BfOd1PtDL67NvgGTn9Vhgltt/j2zz/2ZPECaYTQB+4ZTqWAR0AQY6n32lquu9jv2NiOTi+YXX2+u4xpwEvKSeCq7bgAXAaK9rF6lqLbAMT9NXfT3wlIVuSCVQV6ZjSSPnTwBudn62j4EE4CjgFeAnzjFT+e7bfGPHg6c0+U4AVV2Ap75YCp76PK+ranUD9++jqpl4mvQeEJEBjfwsvtgOpB7B+SZIRXQtJhP0BLhGVed/b6fIODxPEN7vTwdOUNUKEfkYzy/Qw3XI63UNDf87OdDEPapUta6GTWPnC3C+qq7+wQciZSIyErgA+L+mjheR4/D6s3A8D/wcT7NPg3WE9LvqquucP69ReCrvdhKRGCepeFc3rqt8XOT0dXQEypzPEvD8eZgwY08QJpiUA+293s8Hfi2esuGIyCBpePGWjng6UCtEZAhwvNdnVXXn1/MpcIHTz5GCZxWxr1oQ60ogrQXH1zcfuKauHV9ERnl99j88iwF1VNU8H46v71ngOgBtoONcRDqLSLzzOhlPE9EKJ6l9xHdPML8E3nZeZzvvcT7/0CsJDiJIq7OaI2MJwgSTPKDG6dy9Hs9ymiuApSKSD/yHhr+NvwPEiMhKPB3d3utIPw7k1XVSe3nTuV8u8CHwO/WUevbVHDz9FYfrLjzLmuaJSIHzvs5reL79v+Lj8d/jNJmtBJ5p5JCjgRynSe4j4G6vRPJ74AYRKcTTpFdXLfgpoIuz/wa+v4jOqXj+PEyYsWquxhwmEfkMmKxBtmaxiLQFlgPHqOqeAN8rHk//zUmN9HWYEGZPEMYcvt/yXUdxUBCR0/E8PTwc6OTgOAq42ZJDeLInCGOMMQ2yJwhjjDENsgRhjDGmQZYgjDHGNMgShDHGmAZZgjDGGNOg/w+ztVwfnG9OtQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HBjeBHlHBFFi"
      },
      "source": [
        "#**Experiment 6:**\r\n",
        "* **Optimizer =Adam**\r\n",
        "* **Learning rate =0.05**\r\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XVJZHXkEBFFj"
      },
      "source": [
        "**Hyperparameters**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iKu8e7WvBFFj"
      },
      "source": [
        "#HYPERPERAMETERS\r\n",
        "batch_size = 256\r\n",
        "num_iters = 4000\r\n",
        "input_dim = 28*28 # num_features = 784\r\n",
        "output_dim = 2\r\n",
        "\r\n",
        "learning_rate = 0.05\r\n",
        "\r\n",
        "# Device\r\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LI2TKn4qBFFj"
      },
      "source": [
        "**MAKING DATASET ITERABLE**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EtkSIgu9BFFk"
      },
      "source": [
        "'''\r\n",
        "MAKING DATASET ITERABLE\r\n",
        "'''\r\n",
        "num_epochs = num_iters / (len(train_dataset) / batch_size)\r\n",
        "num_epochs = int(num_epochs)\r\n",
        "\r\n",
        "train_loader = torch.utils.data.DataLoader(dataset=train_dataset, \r\n",
        "                                           batch_size=batch_size, \r\n",
        "                                           shuffle=True)  \r\n",
        "\r\n",
        "test_loader = torch.utils.data.DataLoader(dataset=test_dataset, \r\n",
        "                                          batch_size=batch_size, \r\n",
        "                                          shuffle=False) "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LW0xZT3cBFFk"
      },
      "source": [
        "**Logistic Regression Model**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-dh7ZfWsBFFk"
      },
      "source": [
        "class LogisticRegressionModel(nn.Module):\r\n",
        "    def __init__(self, input_size, num_classes):\r\n",
        "        super().__init__()\r\n",
        "        self.linear = nn.Linear(input_size, num_classes)\r\n",
        "\r\n",
        "    def forward(self, x):\r\n",
        "        logits  = self.linear(x)\r\n",
        "        probas = F.softmax(logits, dim=1)\r\n",
        "        return logits, probas"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0ZxMUgF6BFFk"
      },
      "source": [
        "**INSTANTIATE MODEL CLASS**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "D0XabXpLBFFk",
        "outputId": "f710aa59-164d-4747-e26c-e590070dd77c"
      },
      "source": [
        "'''\r\n",
        "INSTANTIATE MODEL CLASS\r\n",
        "'''\r\n",
        "model = LogisticRegressionModel(input_size=input_dim,\r\n",
        "                                num_classes=output_dim)\r\n",
        "# To enable GPU\r\n",
        "model.to(device)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LogisticRegressionModel(\n",
              "  (linear): Linear(in_features=784, out_features=2, bias=True)\n",
              ")"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WrRR6MXCBFFl"
      },
      "source": [
        "**Optimizer setup**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zRER3MpXBFFl"
      },
      "source": [
        "# INSTANTIATE OPTIMIZER CLASS\r\n",
        "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)\r\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8cQnqJviBFFl"
      },
      "source": [
        "**Function TRAIN THE MODEL**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TpdKikSwBFFl",
        "outputId": "37d304c0-3bfc-4168-8185-4265da3b6c77"
      },
      "source": [
        "'''\r\n",
        "TRAIN THE MODEL\r\n",
        "'''\r\n",
        "iteration_loss = []\r\n",
        "iter = 0\r\n",
        "for epoch in range(num_epochs):\r\n",
        "    for i, (images, labels) in enumerate(train_loader):\r\n",
        "\r\n",
        "        \r\n",
        "        images = images.view(-1, 28*28).to(device)\r\n",
        "        labels = labels.to(device)\r\n",
        "\r\n",
        "        # Clear gradients w.r.t. parameters\r\n",
        "        optimizer.zero_grad()\r\n",
        "\r\n",
        "        # Forward pass to get output/logits\r\n",
        "        logits, probas = model(images.float()) \r\n",
        "\r\n",
        "        # Calculate Loss: PyTorch implementation of CrossEntropyLoss works with logits, not probabilities\r\n",
        "        loss = F.cross_entropy(logits, labels)\r\n",
        "\r\n",
        "        # Getting gradients w.r.t. parameters\r\n",
        "        loss.backward()\r\n",
        "\r\n",
        "        # Updating parameters\r\n",
        "        optimizer.step()\r\n",
        "\r\n",
        "        iter += 1\r\n",
        "\r\n",
        "        if iter % 500 == 0:\r\n",
        "            # Calculate Accuracy         \r\n",
        "            correct = 0\r\n",
        "            total = 0\r\n",
        "            # Iterate through test dataset\r\n",
        "            for images, labels in test_loader:\r\n",
        "               \r\n",
        "                images = images.view(-1, 28*28).to(device)\r\n",
        "\r\n",
        "                # Forward pass only to get logits/output\r\n",
        "                logits, probas = model(images.float())\r\n",
        "\r\n",
        "                # Get predictions from the maximum value\r\n",
        "                _, predicted = torch.max(probas, 1)\r\n",
        "\r\n",
        "                # Total number of labels\r\n",
        "                total += labels.size(0)\r\n",
        "\r\n",
        "\r\n",
        "                # Total correct predictions\r\n",
        "                if torch.cuda.is_available():\r\n",
        "                    correct += (predicted.cpu() == labels.cpu()).sum() \r\n",
        "                else:\r\n",
        "                    correct += (predicted == labels).sum()\r\n",
        "\r\n",
        "            accuracy = 100 * correct.item() / total\r\n",
        "\r\n",
        "            # Print Loss\r\n",
        "            iteration_loss.append(loss.item())\r\n",
        "            print('Iteration: {}. Loss: {}. Accuracy: {}'.format(iter, loss.item(), round(accuracy, 2)))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Iteration: 500. Loss: 0.9300315380096436. Accuracy: 49.95\n",
            "Iteration: 1000. Loss: 1.0349936485290527. Accuracy: 52.48\n",
            "Iteration: 1500. Loss: 0.8274070620536804. Accuracy: 49.79\n",
            "Iteration: 2000. Loss: 0.707048773765564. Accuracy: 49.79\n",
            "Iteration: 2500. Loss: 0.8096100091934204. Accuracy: 52.25\n",
            "Iteration: 3000. Loss: 0.7424330711364746. Accuracy: 51.7\n",
            "Iteration: 3500. Loss: 0.698421061038971. Accuracy: 50.37\n",
            "Iteration: 4000. Loss: 0.7185676693916321. Accuracy: 48.3\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ITnefhsnBFFm"
      },
      "source": [
        "**Loss show**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 299
        },
        "id": "wEt53SAEBFFm",
        "outputId": "687fc697-fe40-4462-8066-e1086caec258"
      },
      "source": [
        "import matplotlib\r\n",
        "import matplotlib.pyplot as plt\r\n",
        "\r\n",
        "print (iteration_loss)\r\n",
        "plt.plot(iteration_loss)\r\n",
        "plt.ylabel('Cross Entropy Loss')\r\n",
        "plt.xlabel('Iteration (in every 500)')\r\n",
        "plt.show()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[0.9300315380096436, 1.0349936485290527, 0.8274070620536804, 0.707048773765564, 0.8096100091934204, 0.7424330711364746, 0.698421061038971, 0.7185676693916321]\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEJCAYAAACOr7BbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9dX48c/JAmFJIJAASdghgGFLMKKobFoRECFqVaht1ceW6s8da7VWRW1duqioT91q3WoVfFCBCoq4gFBA2cIS1oACCUvCvhOSnN8fc6MjTpIhzOTOTM779bov5u4nMebk3vu954iqYowxxpwsyu0AjDHGhCZLEMYYY3yyBGGMMcYnSxDGGGN8sgRhjDHGJ0sQxhhjfApaghCRV0WkSERWVbJeRORZEckXkRUi0sdrXZmI5DrTtGDFaIwxpnLBvIJ4HRhaxfphQLozjQVe8Fp3VFUznWlk8EI0xhhTmZhgHVhVvxSR9lVsMgp4Uz1v6i0UkaYikqKq22tyvqSkJG3fvqrTGWOMOdmSJUt2qWqyr3VBSxB+SAO2es0XOMu2A3EishgoBZ5Q1Sm+DiAiY/FcfdC2bVsWL14c3IiNMSbCiMjmytaF6kPqdqqaDfwMmCAinXxtpKovq2q2qmYnJ/tMgMYYY2rIzQRRCLTxmm/tLENVK/7dBMwGsmo7OGOMqevcTBDTgF86o5nOAfar6nYRSRSR+gAikgScB6x2MU5jjKmTgvYMQkTeAQYBSSJSAIwHYgFU9UVgBjAcyAeOANc7u54BvCQi5XgS2BOqagnCGGNqWTBHMY2pZr0CN/tYPh/oGay4jDHG+CdUH1IbY4xxmSUIY4wxPlmCCDNrdxxg1uqdbodhjKkDLEGEkf1HTnD9a4u4+d9L2X/khNvhGGMinCWIMKGq3DdlJTsOHKOkrJwZq2pUkcQYY/xmCSJMfLCskOkrtnPXRV3omNyIKcsK3Q7JGBPhLEGEga17jvDg1DzOap/ITYM6k5OZxlff7GHbvqNuh2aMiWCWIEJcWbly56RcBHjqqkyio4RRmakATFu+zd3gjDERzRJEiHthdj6LN+/lkZzutGnWEIB2zRvRp21Tu81kjAkqSxAhbPnWfUz4dAMjeqWQk5n2g3U5WWms3XGQtTsOuBSdMSbSWYIIUUdKSrlzUi7J8fV5NKcnIvKD9Zf0TCE6SpiyzG4zGWOCwxJEiPrjh2v4ZvdhnryqN00axv5offPG9RmQnsS03ELKy9WFCI0xkc4SRAiatXon73y9hbH9O3Jup6RKt8vJSmPb/mN8/e2eWozOGFNXWIIIMUUHj3HPeyvISElg3JAuVW57UUZLGtaLZmquPaw2xgSeJYgQoqr8bvIKDh8v5ZnRmdSPia5y+4b1Yri4eyumr9jO8dKyWorSGFNXWIIIIf9auJnZ64q5b/gZpLeM92ufnKw0Dhwr5Yu1xUGOzhhT11iCCBH5RQd5dPoaBnZJ5pf92vm933mdmpPUuJ7dZjLGBJwliBBQUlrO7RNzaVQ/hr9e2etHQ1qrEhMdxYheqXy2toj9R63CqzEmcCxBhIAnZ60jb9sBnri8Jy3i4055/5ysNEpKy/nYKrwaYwLIEoTLFmzczctfbmJM3zYM6d6qRsfo3boJHZIa2UtzxpiAsgThov1HTnDXu7m0b96IB0Zk1Pg4Ip4Cfgu/2c32/Vbh1RgTGEFLECLyqogUiciqStaLiDwrIvkiskJE+nitu1ZENjjTtcGK0W0PTF3FzoPHefrqTBrWizmtY+VkpqEK03LtKsIYExjBvIJ4HRhaxfphQLozjQVeABCRZsB44GygLzBeRBKDGKcrpiwrZNrybdxxYTqZbZqe9vHaJzUis01TpliCMMYESNAShKp+CVRVA2IU8KZ6LASaikgKcDEwS1X3qOpeYBZVJ5qwU7D3CA9MWcWZ7RK5aVCngB03JzOVNdsPsH7nwYAd0xhTd7n5DCIN2Oo1X+Asq2z5j4jIWBFZLCKLi4vD40WxsnJl3KTlKDDh6kxiogP3n2BE71Snwqu9E2GMOX1h/ZBaVV9W1WxVzU5OTnY7HL+8OGcjX3+7h4dHft8AKFCSGtenf3oSU3O3WYVXY8xpczNBFAJtvOZbO8sqWx72Vhbs5+lZ67mkZwqX9/F5UXTacjLTKNx3lMWb9wbl+MaYusPNBDEN+KUzmukcYL+qbgdmAkNEJNF5OD3EWRbWjpaUcfukZSQ1rs+jl/U4pbelT8VFGS1pEBvNFCu9YYw5TcEc5voOsADoKiIFInKDiNwoIjc6m8wANgH5wD+A/wegqnuAPwKLnOkRZ1lYe3TGajYVexoANW1YL2jnaVQ/hiHdWzJ9xXZKSsuDdh5jTOQ7vcH3VVDVMdWsV+DmSta9CrwajLjc8Nmanby1cAu/7t+B8zpX3gAoUHKy0piau43Z64pq/Ha2McaE9UPqcFB88Di/m7yCbq3i+e3FXWvlnP07J9G8UT2m2jsRxpjTYAkiiFSVe95bwcHjpTw7JqvaBkCB4qnwmsKsNTs5cMwqvBpjasYSRBC99dUWPl9bxO+HdaOLnw2AAuX7Cq87avW8xpjIYQkiSPKLDvHo9NX0T0/i2n7ta/38mW2a0q55Q2skZIypMUsQQVBSWs4dk5bRIDaav13Zm6io4AxprYqnwmsa8zfuZueBY7V+fmNM+LMEEQQTPl3PqsIDPH55L1omnHoDoEDJyUy1Cq/GmBqzBBFgX23azQtzNnJ1dhuG9nB3iGnH5Mb0bt3EXpozxtSIJYgAOnDsBOPeXU7bZg158NKaNwAKpFGZaeRtO8AGq/BqjDlFliAC6MEpq9hx4BgTrs6kUf2gvYN4Skb0TiFKsKsIY8wpswQRIFNzC5mSu43bLkgnq23o9DdqER/H+enJTM3dhufldWOM8Y8liAAo3HeU+6esok/bptw8OHANgAIlJzOVgr1HWWIVXo0xp8ASxGnyNADKpbxceTrADYACZUj3VsTFRvGBNRIyxpyC0PttFmb+MXcTX32zh/Eju9OueSO3w/Gpcf0YhmS0YvpKq/BqjPGfJYjTsKpwP09+so5hPVpx5Zmt3Q6nSjlZqew7coIv14dHa1ZjjPssQdTQ0ZIybp+4jGaN6vHYZT2D1gAoUPqnJ9OsUT0bzWSM8ZsliBp6/KM1bCw+zN+u7E1io+A1AAqU2OgoLumZwqzVOzloFV6NMX6wBFEDX6wt4s0Fm7nh/A70T092Oxy/5WSlcby0nJl5O90OxRgTBixBnKJdh45z9+TldGsVz9211AAoUPq0bUrbZlbh1RjjH0sQp0BVufe9FRw4VsqE0ZnExdZOA6BA8VR4TeW/+bsosgqvxphqWII4Be98vZVP1xRxz9BudGuV4HY4NTIqM41yhWnLrcKrMaZqQU0QIjJURNaJSL6I3OtjfTsR+UxEVojIbBFp7bWuTERynWlaMOP0x6biQ/zxw9Wc3zmJ689t73Y4Nda5RWN6pjWxftXGmGoFLUGISDTwd2AYkAGMEZGTS5z+DXhTVXsBjwCPe607qqqZzjQyWHH640RZOXdMyqV+bBRPXuVOA6BAGpWZysrC/eQXHXI7FGNMCAvmFURfIF9VN6lqCTARGHXSNhnA587nL3ysDwnPfLqBFQX7efyynq42AAqUkb1TiRLsYbUxpkrBTBBpwFav+QJnmbflwOXO58uAeBFp7szHichiEVkoIjm+TiAiY51tFhcXB+cN4UXf7uH52flceWZrhvVMCco5aluLhDjO65xkFV6NMVVy+yH1b4GBIrIMGAgUAmXOunaqmg38DJggIj8qk6qqL6tqtqpmJycH/n2EA8dOcMfEXFonNmT8yO4BP76bRmWmsWXPEZZu2ed2KMaYEBXMBFEItPGab+0s+46qblPVy1U1C/iDs2yf82+h8+8mYDaQFcRYfXpoah47Dhzj6aszaRwiDYAC5eLuLakfE8UUq/BqjKlEMBPEIiBdRDqISD1gNPCD0UgikiQiFTH8HnjVWZ4oIvUrtgHOA1YHMdYf+c/ybby/rJBbBnfmzHah0wAoUOLjYrkooyXTV27nRJlVeDXG/Fi1CUJEGlX8EheRLiIyUkRiq9tPVUuBW4CZwBrgXVXNE5FHRKRiVNIgYJ2IrAdaAo86y88AFovIcjwPr59Q1VpLENv2HeUPH6wks01Tbr2gc22dttblZKax53AJczdYhVdjzI/5c9/kS6C/iCQCn+C5MrgauKa6HVV1BjDjpGUPen2eDEz2sd98oKcfsQVcebly17vLKS1XJoRoA6BAGdAlmaYNY/lg2TYu6NbS7XCMMSHGn99+oqpH8Iw2el5VrwQi64mtl1fmbWLBpt2MvzSD9kmh2QAoUOrFRDGiVwqzVu/g0PFSt8MxxoQYvxKEiPTDc8Uw3VkWXkWI/JS3bT9/nbmOi7u35KrsNtXvEAFyMtM4dqKcT/J2uB2KMSbE+JMg7sDzAPkD5xlCRzzPBSLKsRNl3DExl8SG9Xj88l4h3wAoUM5sl0jrxAZMsdIbxpiTVPsMQlXnAHMAnIfVu1T1tmAHVtue+GgtG4oO8eb/9KVZGDQACpSKCq8vzN5I0cFjtIgP/zfFjTGB4c8oprdFJEFEGgGrgNUicnfwQ6s9s9cV8fr8b7n+vPYM6BI+DYACJcep8Prh8u1uh2KMCSH+3GLKUNUDQA7wEdAB+EVQo6pFuw8d5+7JK+jSsjH3DO3mdjiuSG8ZT/fUBKvNZIz5AX8SRKzz3kMOME1VTwARU8BHROjbvhkTrs4KuwZAgZSTmcbygv1sKrYKr8YYD38SxEvAt0Aj4EsRaQccCGZQtalZo3r8/Zo+ZKSGZwOgQBmZmYoI9rDaGPOdahOEqj6rqmmqOlw9NgODayE2U4taJsRxbqfmTM0ttAqvxhjAv4fUTUTkqYqy2iLyJJ6rCRNhRmWmsXn3EZZttQqvxhj/bjG9ChwErnKmA8BrwQzKuGNoj1bUi4liqlV4NcbgX4LopKrjnc5wm1T1YaBjsAMztS8hLpaLzmjJhyuswqsxxr8EcVREzq+YEZHzgKPBC8m4aVRmKrsPlzAvf5fboRhjXOZPNdcbgTdFpIkzvxe4NnghGTcN6tqCJg1imbKskMFdW7gdjjHGRf6MYlquqr2BXkAvp/vbBUGPzLiiXkwUl/RK4ZO8nRy2Cq/G1Gl+NztQ1QPOG9UA44IUjwkBOZlpHD1RxqzVO90OxRjjopp2w6kbpU7rqOx2iaQ1bcAHNprJmDqtpgnC3qSKYFFRngqv8/J3sevQcbfDMca4pNIEISIHReSAj+kgkFqLMRoX5GSlUVaufLjcSm8YU1dVmiBUNV5VE3xM8arqz+gnE8a6tIznjJQEPrDaTMbUWTW9xeQXERkqIutEJF9E7vWxvp2IfCYiK0Rktoi09lp3rYhscCYbVuuCnMxUlm/dxze7DrsdijHGBUFLECISDfwdGAZkAGNEJOOkzf4GvKmqvYBHgMedfZsB44Gzgb7AeBFJDFasxreKCq/WJ8KYuimYVxB9gXynPEcJMBEYddI2GcDnzucvvNZfDMxS1T2quheYBQwNYqzGh5QmDTinQ3Om5m6zCq/G1EH+VHO9tYZ/vacBW73mC5xl3pYDlzufLwPiRaS5n/uaWpCTlco3uw6zvGC/26EYY2qZP1cQLYFFIvKu80whkO9A/BYYKCLLgIFAIVDm784iMraiDHlxcXEAwzIVhvZIoV5MFFPsnQhj6hx/Sm3cD6QD/wSuAzaIyGMi0qmaXQuBNl7zrZ1l3sfepqqXO+U7/uAs2+fPvs62L6tqtqpmJycnV/elmBpo0iCWC7u14MMV2yi1Cq/G1Cl+PYNQzw3oHc5UCiQCk0XkL1XstghIF5EOIlIPGA1M895ARJJEpCKG3+PpPQEwExgiIonO7a0hzjLjglGZaew6ZBVejalr/HkGcbuILAH+AvwX6KmqNwFnAldUtp+qlgK34PnFvgZ4V1XzROQRERnpbDYIWCci6/HcynrU2XcP8Ec8SWYR8IizzLhgcLdkEuJimGrvRBhTp/jzwlsz4HKnF/V3VLVcREZUtaOqzgBmnLTsQa/Pk4HJlez7Kt9fURgX1Y+J5pJeKUzN3caRklIa1rP3JI2pC/x5BjEeaC4itzkjmvp4rVsT1OhMyBiVmcaREqvwakxd4s8tpgeAN4DmQBLwmojcH+zATGjp274ZqU3ibDSTMXWIPw+pfw6c5fSlHg+cA/wiuGGZUBMVJYzMTOPLDbvYbRVejakT/EkQ24A4r/n6+BhyaiJfTlYqZeXK9JXb3Q7FGFML/EkQ+4E8EXldRF4DVgH7RORZEXk2uOGZUNKtVQLdWsVbIyFj6gh/hqN84EwVZgcnFBMOcrLSeOKjtWzefZh2zRu5HY4xJoiqTRCq+obzolsXZ9E6VT0R3LBMqBrZO5U/f7yWqbnbuO3CdLfDMcYEkT+jmAYBG/CU7n4eWC8iA4IclwlRqU0b0Ld9M6YsK7QKr8ZEOH+eQTwJDFHVgao6AE8p7qeDG5YJZTlZaWzadZiVhVbh1ZhI5k+CiFXVdRUzqroeiA1eSCbUDe+RQr3oKKYss9IbxkQyfxLEEhF5RUQGOdM/gMXBDsyEriYNYxncLZn/WIVXYyKaPwniRmA1cJszrQZuCmZQJvTlZKZRfPA48zfudjsUY0yQVDmKyekrvVxVuwFP1U5IJhwM7taC+LgYpuQWMqCL9eIwJhJVeQWhqmV4ynG3raV4TJiIi41meI8UZq7awdESv5sAGmPCiD+3mBLxvEn9mYhMq5iCHZgJfaOyUjlcUsasNVbh1ZhI5M+b1A8EPQoTls7p0JyUJnFMXVbIyN6pbodjjAkwf64ghqvqHO8JGB7swEzoi4oSRvZOZc76YvYcLnE7HGNMgPmTIC7ysWxYoAMx4WlUZhql5cr0FfZOhDGRptIEISI3ichKoKuIrPCavgFW1l6IJpSdkRJPl5aNmWL9qo2JOFVdQbwNXApMc/6tmM5U1WtqITYTBkSEnKw0lmzey9Y9R9wOxxgTQJUmCFXdr6rfquoYoAA4ASjQ2Ia9Gm8VD6in5lqfCGMiiT/VXG8BdgKzgOnO9KE/BxeRoSKyTkTyReReH+vbisgXIrLMuX013FneXkSOikiuM714Sl+VqVWtExvSt30zPrAKr8ZEFH+Gud4BdFXVU6qp4LyF/Xc8D7kLgEUiMk1VV3ttdj/wrqq+ICIZwAygvbNuo6pmnso5jXtystK474OV5G07QI+0Jm6HY4wJAH9GMW3F03b0VPUF8lV1k6qWABOBUSdto0CC87kJnv7XJgwN79mK2GhhirUjNSZi+JMgNgGzReT3IjKuYvJjvzQ8yaVCgbPM20PAz0WkAM/Vw61e6zo4t57miEh/XycQkbEislhEFhcXF/sRkgmWpg3rMahrC6Yt30ZZud1mMiYS+JMgtuB5/lAPiPeaAmEM8Lqqtsbz8t2/RCQK2A60VdUsYBzwtogknLyzqr6sqtmqmp2cbAXj3JaTmUbRweMssAqvxkQEf3pSP3zyMhHx59lFIdDGa761s8zbDcBQ5zwLRCQOSFLVIuC4s3yJiGzE0xPb+lCEsAvPaEF8fU+F1/PTk9wOxxhzmqp6UW6e1+d/nbT6az+OvQhIF5EOIlIPGI3nnQpvW4ALnXOcAcQBxSKS7DzkRkQ6Aul4bnWZEBYXG83QHq34eNUOjp2wCq/GhLuqbjE18vrc46R1Ut2BVbUUuAWYCazBM1opT0QeEZGRzmZ3Ab8WkeXAO8B16hknOQBYISK5wGTgRlXd49dXZFyVk5XGoeOlfGoVXo0Je1XdKtJKPvua930A1Rl4Hj57L3vQ6/Nq4Dwf+70HvOfPOUxoOadjc1om1GfKsm2M6GUVXo0JZ1UliKYichmeq4ymInK5s1zwDEk15keinQqvr8//lr2HS0hsVM/tkIwxNVTVLaY5wEhghPO5ohbTCODL4IdmwtWozDROlCnTV253OxRjzGmo9ApCVa+vzUBM5OiemkB6i8ZMzS3k5+e0czscY0wN+fMehDGnpKLC66JvrcKrMeHMEoQJiooKr9OWW/UUY8KVJQgTFG2aNSS7XSJTrMKrMWHLn3LfV4pIvPP5fhF5X0T6BD80E+5ystLYUHSIvG0H3A7FGFMD/lxBPKCqB0XkfOAnwD+BF4IblokEI3ql0Lh+DH+duc6uIowJQ/4kiIqaCZcAL6vqdDyF+4ypUtOG9bhrSBfmrC9mxsodbodjjDlF/iSIQhF5CbgamCEi9f3czxh+cU47eqQl8MiHeRw8dsLtcIwxp8CfX/RX4amndLGq7gOaAXcHNSoTMWKio3g0pydFB4/z1Kz1bodjjDkF/iSIFGC6qm4QkUHAlfhXzdUYAHq3aco1Z7fljfnfsqqwJs0JjTFu8CdBvAeUiUhn4GU8PR7eDmpUJuLcfXE3mjWqxx+mrLKOc8aECX8SRLlTuvty4DlVvRvPVYUxfmvSIJb7L8lg+dZ9vPP1FrfDMcb4wZ8EcUJExgC/BD50lsUGLyQTqUZlpnJup+b8+eO1FB887nY4xphq+JMgrgf6AY+q6jci0gE4ucOcMdUSEf6Y04PjJ8p5bMYat8MxxlSj2gThNPX5LbBSRHoABar656BHZiJSp+TG3DiwIx8sK2R+/i63wwlrR0pK3Q7BRDh/Sm0MAjYAfweeB9aLyIAgx2Ui2P8b3Jm2zRpy/9RVHC+13tU1MW/DLno//Anjp66i3B76myDx5xbTk8AQVR2oqgOAi4GngxuWiWRxsdE8Mqo7m4oP8/KcTW6HE3Z2HzrOne/mEhcbzRsLNnPrxGWWaE1Q+JMgYlV1XcWMqq7HHlKb0zSoawsu6ZnC/36Rz5bd1jPCX6rK7yavYP+RE0wa24/7hndj+ort/M/rizh03G45mcDyJ0EsEZFXRGSQM/0DWOzPwUVkqIisE5F8EbnXx/q2IvKFiCwTkRUiMtxr3e+d/daJyMX+f0kmXDwwIoPY6CgemLrKivn56c0Fm/lsbRH3DutGRmoCYwd04skre7Nw0x5Gv7zARoeZgPInQdwIrAZuc6bVwE3V7SQi0XieWwwDMoAxIpJx0mb3A++qahYwGs8zDpztRgPdgaHA887xTARp1SSOcRd5ivl9tMqK+VVnzfYDPDpjDYO7JnP9ee2/W37Fma155ZfZ5Bcd4qcvzrcrMhMwVSYI55fyclV9SlUvd6anVdWfP1P6AvmquklVS4CJwKiTtlEgwfncBKhoPzYKmKiqx1X1GyDfOZ6JML/s147uqQk8/J88u0VShaMlZdz2zjIS4mL565W9EZEfrB/crQVv//oc9h89weUvzCdvm5U0MaevygShqmXAOhFpW4NjpwFbveYLnGXeHgJ+LiIFwAzg1lPYFxEZKyKLRWRxcXFxDUI0bouJjuLRy5xifp9YMb/KPDpjNRuKDvHUVb1Jalzf5zZ92iYy+cZ+1IsWrn5pIfM32jBic3r8ucWUCOSJyGciMq1iCtD5xwCvq2prYDjwLxHxu5S4qr6sqtmqmp2cnBygkExty3SK+b0+/xsr5ufDzLwdvLVwC2MHdGRAl6p/zju3iGfyTeeS0iSO615dxIyV22spShOJ/OooB4wAHsEz5LViqk4hnsJ+FVo7y7zdALwLoKoLgDggyc99TQSpKOZ3/xQb1+9t+/6j3PPeCnqkJfDbIV392ie1aQP+78Z+9GzdhJvfXspbCzcHOUoTqSpNECLSWUTOU9U53hOeDnMFfhx7EZAuIh1EpB6eh84nX3lsAS50zncGngRR7Gw3WkTqO6U90rES4xGtophf7tZ9vLPIivkBlJUrd07KpaS0nGdHZ1Evxv8+XU0b1uOtG85mcNcW3D9lFU/PWm8jxcwpq+onbgLgq9v8fmddlZwKsLfgaTa0Bs9opTwReURERjqb3QX8WkSWA+8A16lHHp4ri9XAx8DNzvMQE8G+K+b3kRXzA3hxzkYWbtrDQyO70zG58Snv36BeNC/94kx+emZrnvlsg5VaN6dMKvurQkQWqepZlaxbqao9gxrZKcrOztbFi/16PcOEsPyiQwx75ksu7ZXKU1dnuh2Oa5Zt2ctPX1zAsB6teG5M1o9GLZ0KVeUvM9fxwuyNDO3eigmjM4mLtVHjxkNElqhqtq91VV1BNK1iXYPTC8kY3zq3aMyNAzvx/rLCOjsK5+CxE9w2cRmtEuJ49LKep5UcwFNF956h3XhgRAYf5+3g2le/5oD1Bzd+qCpBLBaRX5+8UER+BSwJXkimrru5opjflLpZzO+BKaso3HuUZ0Zn0qRB4Kra3HB+B54ZncmSzXu5+qWFFB04FrBjm8hUVYK4A7heRGaLyJPONAfPyKPbayc8Uxd5F/P7x5d1q5jf+0sLmJK7jdsv7EJ2+2YBP/6ozDReve4sNu8+zBUvzuebXYcDfg4TOSpNEKq6U1XPBR4GvnWmh1W1n6paXQQTVIO6tmB4z1Y893ndKeb37a7DPDBlFX3bN+OWCzoH7TwDuiTz9q/P4fDxMn76wnxWFti7J8Y3fxoGfaGqzznT57URlDEAD47oTkyU8OC0yC/md6KsnNsnLiM6Snh6dCbRUaf33KE6mW2a8n839iMuNprRLy9g3oa6+bzHVM3/gdXG1LJWTeIYN6Qrs9cV83GEF/N7atZ6lhfs54krepHWtHbGgHRKbsz7/+9c2jRryPWvf81/lm+rfidTp1iCMCHt2n7tyEhJ4OH/rI7YYn7z83fx4pyNjD6rDcN7ptTquVsmxDHpN/3IapPIbROX8fp/v6nV85vQZgnChDRPMb8e7Dx4jKdnRV4xvz2HS7hjUi4dkxrx4KUnV8OvHU0axPLmDX256IyWPPSf1fx15tqIv6Vn/GMJwoS8rLaJ/KxvW16f/21ElbGu6A6378gJnh2TRcN6Ma7FEhcbzfPX9GFM3zb8/YuN3PveSkrLyl2Lx4QGSxAmLPzu4m4kNoyNqGJ+by3czKdrdnLPsG50T23idjjEREfx2GU9ue2CzkxavJUb31rKsRN17z0U8z1LECYsNGkYyx8uOYNlW/YxcdHW6ncIcet2HORP09cwsEsy15/b3u1wviMijGuNN7cAABanSURBVBvSlYdHdueztTv5xT+/Yv8Re+u6rrIEYcJGTmYa/To254mP1rDrUPgW8zt2ooxb31lKfFwsf7uyN1FBHtJaE9ee257nxmSRu3UfV720gB377a3rusgShAkbIsIfc3pw9EQZj81Y43Y4Nfbo9DWs33mIJ6/qTXK87+5woWBEr1Rev74vBXuPcMUL88kvOuR2SKaWWYIwYaVzi8b8ZkAn3l9ayIKNu90O55TNWr2Tfy3czK/O78DAarrDhYLzOicx6Tf9OF5axpUvzmfZlr1uh2RqkSUIE3ZuuaAzbZo14P4pKykpDZ+RNjv2H+N3k5fTPTWBu4f61x0uFPRIa8LkG88lPi6Wn/3jK2avK3I7JFNLLEGYsBMXG80jI3uwsfgw/5gbHsX8ysqVce/mcuxEOc+OyaJ+THj1Y2if1IjJN/WjQ1IjfvXGYj5Y5k9TSRPuLEGYsDS4WwuG9WjFs59tCItifi99uZH5G3fz0MgMOtWgO1woaBEfx6TfnMNZ7Ztx56TlvBImydnUnCUIE7YevDSDmChhfIgX88vduo+nPlnPJT1TuCq7jdvhnJb4uFheu/4shvdsxZ+mr+HxGWtC+ntvTo8lCBO2Upo04M6LuvDFumJm5oVmMb9Dx0u5feIyWibE8VgAusOFgrjYaJ4b04efn9OWl77cxG//bwUn7K3riGQJwoS1685tzxkpCTw0LTSL+T04ZRVb9xxhwuhMmjQMXHc4t0VHCX8c1YM7f9KF95YWMPbNxRwpCb3vvzk9QU0QIjJURNaJSL6I3Otj/dMikutM60Vkn9e6Mq9104IZpwlf3sX8JoRYMb8pywp5f1kht16QzllB6A7nNhHh9p+k8+hlPZizvphrXvmKvYdL3A7LBFDQEoSIRAN/B4YBGcAYEflBuUpVvVNVM1U1E3gOeN9r9dGKdao6MlhxmvDXp20iY/q25bX537J62wG3wwFgy+4j3D9lFdntErk1iN3hQsE1Z7fj+Wv6kFd4gJ++OJ/CfUfdDskESDCvIPoC+aq6SVVLgInAqCq2HwO8E8R4TAS75+JuNG0Qy/1TVrpezO9EWTm3TVyGCEwYnUlMdOTfyR3aI4U3b+hL0YHj/PSF+azfedDtkEwABPMnNw3wrqpW4Cz7ERFpB3QAvFuaxonIYhFZKCI5wQvTRIKKYn5Lt+xj0mJ3i/lN+HQ9uVv38fjlPWmd2NDVWGrTOR2bM+k3/SgtV658cQFLNu9xOyRzmkLlT5vRwGRV9a4t3E5Vs4GfARNEpNPJO4nIWCeJLC4uLq6tWE2IuiwrjXM6NuOJj9a6Vsxv/sZdPD97I1dlt2ZEr1RXYnBTRmoC7990LokNY7nmla/4bM1Ot0MypyGYCaIQ8B703dpZ5stoTrq9pKqFzr+bgNlA1sk7qerLqpqtqtnJyaFf18YEl4jwp5weHCkp5fEZa2v9/HsPlzBu0nI6NG/EQyO71/r5Q0WbZg2ZfNO5pLeIZ+y/lvB/Ll/RmZoLZoJYBKSLSAcRqYcnCfxoNJKIdAMSgQVeyxJFpL7zOQk4D1gdxFhNhOjcIp6xAzry3tICFm6qvWJ+qso9761g9+HjrneHCwVJjevzzthz6NexOXdPXsELszfaC3VhKGgJQlVLgVuAmcAa4F1VzRORR0TEe1TSaGCi/vCn5wxgsYgsB74AnlBVSxDGL7cMTneK+a2qtWJ+//5qC5+s3sk9Q7vRI8397nChoHH9GF697iwu7Z3Knz9eyx+mrOLgMWs+FE4kUrJ6dna2Ll682O0wTIj4Ym0R17++iLsv7srNg4M7zHT9zoNc+tw8zu7YnNevOyskGwC5qbxceeLjtfxj7iaaN6rPfcO7cVlWWkS8VR4JRGSJ87z3R0LlIbUxATW4WwuGdm/Fc59vYOue4BXzO3aijNveWUZ8XAx/u7KXJQcfoqKE+4afwdSbz6N1YgPGvbucK19cQN62/W6HZqphCcJErPEjM4gWYfy0vKDd/358xhrW7jjIX6/sTYv4uKCcI1L0at2U9286l79c0YtNuw5z6XPzeHDqKut5HcIsQZiIVVHM7/O1RczMC/xwy8/W7OSNBZv5n/M6MLhri4AfPxJFRQlXndWGL+4axC/7teethZsZ/ORsJn69xfUXHM2PWYIwEa2imN/D/8njcACL+RUdOMbdk1dwRkoC9wwLn+5woaJJw1geGtmdD2/tT6fkRtz7/koue/6/5G7dV/3OptZYgjARLSY6ij/l9GD7/mNM+DQwxfzKy5Vx7y7nSEkpz43JDLvucKEkIzWBd3/TjwlXZ7Jt/zEue/6/3PveCna79KKj+SFLECbindnOU8zv1f9+y5rtp1/M7+W5m5iXv4vxl3anc4v4AERYt4kIOVlpfH7XQH51fgcmLylg8N9m8+aCbym1PhOusgRh6oR7hnalaYNY/vDB6RXzW751H3+buY5hPVox+qzw7g4XauLjYvnDJRl8dHt/erZuwoNT87j0f//L4m+tppNbLEGYOqFpw3rcN/z0ivlVdIdrEV+fJy7vZeP4gyS9ZTxv3XA2z1/Th/1HSvjpiwsYNymXogPH3A6tzrEEYeqMy/ukcXYHTzG/mtzjHj81jy17jjBhdFZEdYcLRSLC8J4pfHrXQG4Z3JkPV2zngifn8MrcTdbetBZZgjB1hojw6GWeYn6PnWIxv6m5hby3tIBbBnemb4fI6w4XqhrWi+G3F3dl5p0DyG6fyJ+mr2HYM3OZn7/L7dDqBEsQpk7p3CKeX/f3FPP7ys9iflv3HOH+D1ZxZrtEbrswPcgRGl86JDXitevO4pVfZnO8tIyfvfIVN7+9lG3WvS6oLEGYOufWC9JpnehfMb9SpzscAhOurhvd4UKViPCTjJbMunMg4y7qwqerd3Lhk3P4+xf5HC8tq/4A5pTZT7upcxrUi+aRUd3ZUHSIV+ZtqnLbZz7bwLIt+3jssp60aVZ3usOFsrjYaG67MJ1Pxw1kQJck/jpzHUMnzGX2uiK3Q6tVR0vKmLO+mD99uJq/fByc/id1u2i9qbMu6NaSod1b8exnG7i0V6rPX/4LN+3mf7/I58ozW3Np77rXHS7UtWnWkJd+kc2c9cU8PC2P615bxEUZLXlwREZEJvPycmX19gPMy9/F3A3FLPpmLyVl5dSLiWJIRsugnNPKfZs6a9u+o/zkqTn069icV67N/sGw1X1HShj2zFziYqP58NbzaVTf/pYKZSWl5bz632949rMNlJUrNw7sxE2DOhEXG95vue/Yf4y5G4qZu2EX/83fxe7DJQB0axVP//Qk+qcnc1b7ZjSoV/Ovs6py3/ZTb+qs1KYNuPMnXXh0xho+Wb2Ti7u3Ar7vDrfr0HHev+k8Sw5hoF5MFDcO7MSozFQem7GWZz7bwHtLC3hgRAZDMlqGzTsrR0pK+eqbPcxd77lK2FB0CPB06BvQJZn+6Umc3zmJFgm1UznYriBMnXairJxLn5vH/qMn+HTcQBrVj+Htr7Zw3wcruW94N8YO6OR2iKYGFmzczfhpq1i/8xADuyQz/tIMOiY3djusH6m4bfTlhmLmrt/Fks2e20b1Y6Lo26HZd1cJ3VrFBy3JVXUFYQnC1HlLNu/hihcWMHZAR8/zhv+dx1ntm/HG9X2tAVAYO1FWzpsLNjNh1nqOlZbxq/4duWVwZ9evCLftO8q8DbuYm++5bbTHuW10RkqCkxCSOKt9s1q7PWYJwphq/P79Fby7uIC2zRpy4OgJPrqjvzUAihBFB4/x54/W8d7SAlKaxPGHS87gkp4ptXbb6fDxUr76Zjdfrt/FvPxd5Du3jZLj63+XEM7rnOTaz5slCGOqse9ICRc8OYc9h0t49bpsLugWnFEhxj1LNu/hwal55G07QL+OzXl4VHe6tAx8Nd6yciVv237mbtjFl+uLWbplLyfKlPoxUZzdsTkD0pM4Pz2Jri2Dd9voVFiCMMYPSzbvYcueI1yW1drtUEyQlJUrb3+9hb/NXMeh46Vcd257bv9JOglxp1dbq3DfUeZtKObLDbuYn7+LvU4b1YyUBPp3SWJAejJntksMyVFVriUIERkKPANEA6+o6hMnrX8aGOzMNgRaqGpTZ921wP3Ouj+p6htVncsShDHGX3sOl/DXmeuYuGgLzRvV5/fDunF5nzS//6I/dLyUhRt3My9/F19uKGZT8WEAWibU5/zOyQzo4rltlNS4fjC/jIBwJUGISDSwHrgIKAAWAWNUdXUl298KZKnq/4hIM2AxkA0osAQ4U1X3VnY+SxDGmFO1omAfD07NI3frPrLbJfLwqO50T23yo+3KypWVhfuZu76Yufm7WLp5L6XlSlxsFOd0bM75nZMY0CWZ9BaNQ+K20alw6z2IvkC+qm5ygpgIjAJ8JghgDDDe+XwxMEtV9zj7zgKGAu8EMV5jTB3Tq3VT3r/pXCYvKeDPH6/l0ufmcc3Z7bhrSBcOHiv97q3l/+bvZv9Rz22jHmkJ/Kp/RwakJ3Fm+8SIbjkbzASRBnh3ZikAzva1oYi0AzoAn1exb5qP/cYCYwHatm17+hEbY+qcqCjhqrPacHH3Vjz96XreXPAtkxZv/a6QY6uEOIZktOR85yW15mFw2yhQQuUV0dHAZFU9pZKMqvoy8DJ4bjEFIzBjTN3QpGEsD43szlXZbfj3V5vplNyY/ulJdA7D20aBEswEUQh4N+1t7SzzZTRw80n7Djpp39kBjM0YY3zKSE3g0ct6uh1GSAhmue9FQLqIdBCReniSwLSTNxKRbkAisMBr8UxgiIgkikgiMMRZZowxppYE7QpCVUtF5BY8v9ijgVdVNU9EHgEWq2pFshgNTFSv4VSqukdE/ognyQA8UvHA2hhjTO2wF+WMMaYOq2qYq3WUM8YY45MlCGOMMT5ZgjDGGOOTJQhjjDE+WYIwxhjjU8SMYhKRYmDzaRwiCdgVoHCCLZxihfCKN5xihfCKN5xihfCK93Ribaeqyb5WREyCOF0isriyoV6hJpxihfCKN5xihfCKN5xihfCKN1ix2i0mY4wxPlmCMMYY45MliO+97HYApyCcYoXwijecYoXwijecYoXwijcosdozCGOMMT7ZFYQxxhifLEEYY4zxqc4nCBEZKiLrRCRfRO51O56qiMirIlIkIqvcjqU6ItJGRL4QkdUikicit7sdU1VEJE5EvhaR5U68D7sdU3VEJFpElonIh27HUh0R+VZEVopIroiEdNllEWkqIpNFZK2IrBGRfm7HVBkR6ep8TyumAyJyR8COX5efQYhINLAeuAhP3+tFwBhVXe1qYJUQkQHAIeBNVe3hdjxVEZEUIEVVl4pIPLAEyAnh760AjVT1kIjEAvOA21V1ocuhVUpExgHZQIKqjnA7nqqIyLdAtqqG/ItnIvIGMFdVX3GanTVU1X1ux1Ud5/dZIXC2qp7OS8PfqetXEH2BfFXdpKolwERglMsxVUpVvwTConGSqm5X1aXO54PAGiDN3agqpx6HnNlYZwrZv55EpDVwCfCK27FEEhFpAgwA/gmgqiXhkBwcFwIbA5UcwBJEGrDVa76AEP4lFq5EpD2QBXzlbiRVc27Z5AJFwCxVDeV4JwC/A8rdDsRPCnwiIktEZKzbwVShA1AMvObcvntFRBq5HZSfRgPvBPKAdT1BmCATkcbAe8AdqnrA7XiqoqplqpoJtAb6ikhI3sYTkRFAkaoucTuWU3C+qvYBhgE3O7dLQ1EM0Ad4QVWzgMNASD+bBHBuhY0E/i+Qx63rCaIQaOM139pZZgLAuZf/HvBvVX3f7Xj85dxS+AIY6nYslTgPGOnc158IXCAib7kbUtVUtdD5twj4AM/t3VBUABR4XT1OxpMwQt0wYKmq7gzkQet6glgEpItIBycDjwamuRxTRHAe+v4TWKOqT7kdT3VEJFlEmjqfG+AZuLDW3ah8U9Xfq2prVW2P52f2c1X9ucthVUpEGjkDFXBu1wwBQnIknqruALaKSFdn0YVASA6sOMkYAnx7CTyXU3WWqpaKyC3ATCAaeFVV81wOq1Ii8g4wCEgSkQJgvKr+092oKnUe8AtgpXNfH+A+VZ3hYkxVSQHecEaCRAHvqmrIDx8NEy2BDzx/MxADvK2qH7sbUpVuBf7t/NG4Cbje5Xiq5CTdi4DfBPzYdXmYqzHGmMrV9VtMxhhjKmEJwhhjjE+WIIwxxvhkCcIYY4xPliCMMcb4ZAnCuE5EDjn/theRnwX42PedND8/gMeeUPFGsFOSISNQxw42ESnzqgA6zWt5BxH5yqluPMkZ6omI1Hfm85317Z3lPUXkdVe+CBN0liBMKGkPnFKCEJHq3uX5QYJQ1XNPMabKztscOMcpoIiq/srtSrV+fC+8HVXVTGca6bX8z8DTqtoZ2Avc4Cy/AdjrLH/a2Q5VXQm0FpG2p/8VmFBjCcKEkieA/s5ftXc6xfP+KiKLRGSFiPwGQEQGichc5y/f1c6yKU4huLyKYnAi8gTQwDnev51lFVcr4hx7ldOn4GqvY8/26gfwb+et8JNdAXz3spezT3bFOUTkUfH0llgoIi1P3tl5u/hV8fSgWCYio5zlC0Wk+8nHrWL760Rkmoh8DnwmIm+KSI7X/v+u2LY6ztd5AZ7yEgBvABXHGuXM46y/0Ov78h88b3SbSKOqNtnk6gQccv4dBHzotXwscL/zuT6wGE+1zUF4iqh18Nq2mfNvAzxlHJp7H9vHua4AZuF5g74lsAXP29SDgP146nJFAQvwFJo7OeY3gEu95mfj6XcAnsqllzqf/1LxNZy0/2PAz53PTfH0JWkE3Ak87CxPAdZVs/11eOoHVXz9A4EpzucmwDdAjI/zlzrfz4V4+nQAJOEpf1+xTRtglfN5FdDaa91GIMn5fB7wH7d/jmwK/GRXECaUDQF+6ZTq+ApoDqQ7675W1W+8tr1NRJbj+YXXxmu7ypwPvKOeCq47gTnAWV7HLlDVciAXz62vk6XgKQvtSwlQUaZjSSX7DwHudb622UAc0BZ4F/ips81VfP/XfGXbg6c0+R4AVZ2Dp75YMp76PO+paqmP87dT1Ww8t/QmiEinSr4WfxQBqaexvwlRdboWkwl5AtyqqjN/sFBkEJ4rCO/5nwD9VPWIiMzG8wu0po57fS7D9/8nR6s4xwlVrahhU9n+Alyhqut+tEJkt4j0Aq4GbqxqexE5G6/vheNN4Od4bvv4rCOk31dX3eR8v7LwVN5tKiIxTlLxrm5cUfm4wHnW0QTY7ayLw/P9MBHGriBMKDkIxHvNzwRuEk/ZcESki/hu3tIEzwPUIyLSDTjHa92Jiv1PMhe42nnOkYyni9jXpxDrGqDzKWx/spnArRX38UUky2vdJDzNgJqo6go/tj/Z68AdAOrjwbmIJIpIfedzEp5bRKudpPYF31/BXAtMdT5Pc+Zx1n/ulQS7EKLVWc3psQRhQskKoMx5uHsnnnaaq4GlIrIKeAnff41/DMSIyBo8D7q9+0i/DKyoeEjt5QPnfMuBz4HfqafUs7+m43leUVN/xNPWdIWI5DnzFSbj+ev/XT+3/wHnltka4LVKNjkDWOzckvsCeMIrkdwDjBORfDy39CqqBf8TaO4sH8cPm+gMxvP9MBHGqrkaU0MiMg8YoSHWs1hEGgIrgT6quj/I56qP5/nN+ZU86zBhzK4gjKm5u/j+QXFIEJGf4Ll6eC7YycHRFrjXkkNksisIY4wxPtkVhDHGGJ8sQRhjjPHJEoQxxhifLEEYY4zxyRKEMcYYn/4/WDGqfO9nVwQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TIEtrCrLFuXL"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}