_notebooks/2021-05-23-ajtfb-chapter-4.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Copy of Copy of 2020-11-06-ajtfb-chapter-1.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sWJL4-Qwluy4"
      },
      "source": [
        "# \"A Journey Through Fastbook (AJTFB) - Chapter 4\"\n",
        "> \"The fourth in a weekly-ish series where I revisit the fast.ai book, [\\\"Deep Learning for Coders with fastai & PyTorch\\\"](https://github.com/fastai/fastbook), and provide commentary on the bits that jumped out to me chapter by chapter.  So without further adieu, let's go!\"\n",
        "\n",
        "- toc: false\n",
        "- branch: master\n",
        "- badges: true\n",
        "- hide_binder_badge: true\n",
        "- comments: true\n",
        "- author: Wayde Gilliam\n",
        "- categories: [fastai, fastbook]\n",
        "- image: images/articles/fastbook.jpg"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WfcRNESotGLU"
      },
      "source": [
        "Other posts in this series:  \n",
        "[A Journey Through Fastbook (AJTFB) - Chapter 1](https://ohmeow.com/posts/2020/11/06/ajtfb-chapter-1.html)  \n",
        "[A Journey Through Fastbook (AJTFB) - Chapter 2](https://ohmeow.com/posts/2020/11/16/ajtfb-chapter-2.html)  \n",
        "[A Journey Through Fastbook (AJTFB) - Chapter 3](https://ohmeow.com/posts/2020/11/22/ajtfb-chapter-3.html)\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gWxCOulE-ACJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "dc283df6-32e5-4892-edcb-caf85a9e0f7f"
      },
      "source": [
        "# hide\n",
        "!pip install fastai>=2.3 --upgrade\n",
        "\n",
        "import fastai\n",
        "from fastai.vision.all import *\n",
        "print(fastai.__version__ )"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "2.3.1\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WyL36r-q-MRO"
      },
      "source": [
        "## Chapter 4"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Xn6OyriNApwH"
      },
      "source": [
        "---\n",
        "### How to visualize a grayscale image in pandas ..."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LWRBzg9Z-Lof",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "1409b4e2-9b5c-45dc-a376-0540bc964494"
      },
      "source": [
        "mnist_path = untar_data(URLs.MNIST_SAMPLE)\n",
        "mnist_path.ls()"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/html": [
              ""
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(#3) [Path('/root/.fastai/data/mnist_sample/labels.csv'),Path('/root/.fastai/data/mnist_sample/valid'),Path('/root/.fastai/data/mnist_sample/train')]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SNW-l1sL_l0l",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 45
        },
        "outputId": "3922a9d8-a6e7-4e21-d6ee-da4ed1cb2717"
      },
      "source": [
        "sample_3 = Image.open((mnist_path/'train/3').ls().sorted()[1])\n",
        "sample_3"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAA9ElEQVR4nM3Or0sDcRjH8c/pgrfBVBjCgibThiKIyTWbWF1bORhGwxARxH/AbtW0JoIGwzXRYhJhtuFY2q1ocLgbe3sGReTuuWbwkx6+r+/zQ/pncX6q+YOldSe6nG3dn8U/rTQ70L8FCGJUewvxl7NTmezNb8xIkvKugr1HSeMP6SrWOVkoTEuSyh0Gm2n3hQyObMnXnxkempRrvgD+gokzwxFAr7U7YXHZ8x4A/Dl7rbu6D2yl3etcw/F3nZgfRVI7rXM7hMUUqzzBec427x26rkmlkzEEa4nnRqnSOH2F0UUx0ePzlbuqMXAHgN6GY9if5xP8dmtHFfwjuQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<PIL.PngImagePlugin.PngImageFile image mode=L size=28x28 at 0x7FDD1EC3B190>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oaesatM_AAiI",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 271
        },
        "outputId": "acbb81b3-9f1f-422f-c014-ad52d60870ef"
      },
      "source": [
        "sample_3_t = tensor(sample_3)\n",
        "df = pd.DataFrame(sample_3_t[4:15, 4:22])\n",
        "df.style.set_properties(**{'font-size':'6pt'}).background_gradient('Greys')"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<style  type=\"text/css\" >\n",
              "#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col1,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col3,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col5,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col6,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col7,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col8,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col9,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col10,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col11,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col12,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col13,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col14,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col1,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col3,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col1,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col6,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col7,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col8,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col9,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col10,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col5,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col6,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col7,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col8,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col9,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col1,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col3,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col5,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col6,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col7,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col8,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col9,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col14,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col1,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col3,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col5,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col6,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col13,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col14,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col1,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col3,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col13,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col14,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col1,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col3,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col16,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col17,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col0,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col1,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col3,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col5,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col6,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col17{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #ffffff;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col5{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #efefef;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col6,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col13{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #7c7c7c;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col7{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #4a4a4a;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col8,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col9,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col10,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col5,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col6,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col7,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col11,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col12,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col13,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col12,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col13,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col1,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col3,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col12,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col13,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col12,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col11,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col11,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col11,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col12,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col13,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col14,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col15,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col16{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #000000;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col11{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #606060;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col12{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #4d4d4d;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col14{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #bbbbbb;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col3{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #e4e4e4;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col6{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #6b6b6b;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col8,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col14,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col14{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #171717;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col9,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col11{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #4b4b4b;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col10,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col10,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col8,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col10,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col8,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col10{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #010101;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col1{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #272727;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col2{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #0a0a0a;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col3{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #050505;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col5{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #333333;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col6{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #e6e6e6;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col7,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col10{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #fafafa;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col8{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #fbfbfb;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col9{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #fdfdfd;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col4{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #1b1b1b;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col5{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #e0e0e0;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col11{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #4e4e4e;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col14{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #767676;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col1{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #fcfcfc;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col2,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col3{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #f6f6f6;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col4,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col7{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #f8f8f8;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col10,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col7{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #e8e8e8;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col11{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #222222;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col13,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col12{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #090909;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col14{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #d0d0d0;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col10,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col11,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col6{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #060606;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col13{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #979797;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col8{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #b6b6b6;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col9{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #252525;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col12{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #999999;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col5{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #f9f9f9;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col7{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #101010;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col9,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col9{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #020202;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col11{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #545454;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col12{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #f1f1f1;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col5{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #f7f7f7;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col7{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #030303;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col12{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #181818;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col13{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #303030;\n",
              "            color:  #f1f1f1;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col14{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #a9a9a9;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col15{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #fefefe;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col8,#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col9{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #bababa;\n",
              "            color:  #000000;\n",
              "        }#T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col10{\n",
              "            font-size:  6pt;\n",
              "            background-color:  #393939;\n",
              "            color:  #f1f1f1;\n",
              "        }</style><table id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >0</th>        <th class=\"col_heading level0 col1\" >1</th>        <th class=\"col_heading level0 col2\" >2</th>        <th class=\"col_heading level0 col3\" >3</th>        <th class=\"col_heading level0 col4\" >4</th>        <th class=\"col_heading level0 col5\" >5</th>        <th class=\"col_heading level0 col6\" >6</th>        <th class=\"col_heading level0 col7\" >7</th>        <th class=\"col_heading level0 col8\" >8</th>        <th class=\"col_heading level0 col9\" >9</th>        <th class=\"col_heading level0 col10\" >10</th>        <th class=\"col_heading level0 col11\" >11</th>        <th class=\"col_heading level0 col12\" >12</th>        <th class=\"col_heading level0 col13\" >13</th>        <th class=\"col_heading level0 col14\" >14</th>        <th class=\"col_heading level0 col15\" >15</th>        <th class=\"col_heading level0 col16\" >16</th>        <th class=\"col_heading level0 col17\" >17</th>    </tr></thead><tbody>\n",
              "                <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col0\" class=\"data row0 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col1\" class=\"data row0 col1\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col2\" class=\"data row0 col2\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col3\" class=\"data row0 col3\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col4\" class=\"data row0 col4\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col5\" class=\"data row0 col5\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col6\" class=\"data row0 col6\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col7\" class=\"data row0 col7\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col8\" class=\"data row0 col8\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col9\" class=\"data row0 col9\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col10\" class=\"data row0 col10\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col11\" class=\"data row0 col11\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col12\" class=\"data row0 col12\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col13\" class=\"data row0 col13\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col14\" class=\"data row0 col14\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col15\" class=\"data row0 col15\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col16\" class=\"data row0 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row0_col17\" class=\"data row0 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col0\" class=\"data row1 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col1\" class=\"data row1 col1\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col2\" class=\"data row1 col2\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col3\" class=\"data row1 col3\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col4\" class=\"data row1 col4\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col5\" class=\"data row1 col5\" >29</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col6\" class=\"data row1 col6\" >150</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col7\" class=\"data row1 col7\" >195</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col8\" class=\"data row1 col8\" >254</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col9\" class=\"data row1 col9\" >255</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col10\" class=\"data row1 col10\" >254</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col11\" class=\"data row1 col11\" >176</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col12\" class=\"data row1 col12\" >193</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col13\" class=\"data row1 col13\" >150</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col14\" class=\"data row1 col14\" >96</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col15\" class=\"data row1 col15\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col16\" class=\"data row1 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row1_col17\" class=\"data row1 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row2\" class=\"row_heading level0 row2\" >2</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col0\" class=\"data row2 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col1\" class=\"data row2 col1\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col2\" class=\"data row2 col2\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col3\" class=\"data row2 col3\" >48</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col4\" class=\"data row2 col4\" >166</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col5\" class=\"data row2 col5\" >224</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col6\" class=\"data row2 col6\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col7\" class=\"data row2 col7\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col8\" class=\"data row2 col8\" >234</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col9\" class=\"data row2 col9\" >196</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col10\" class=\"data row2 col10\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col11\" class=\"data row2 col11\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col12\" class=\"data row2 col12\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col13\" class=\"data row2 col13\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col14\" class=\"data row2 col14\" >233</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col15\" class=\"data row2 col15\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col16\" class=\"data row2 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row2_col17\" class=\"data row2 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row3\" class=\"row_heading level0 row3\" >3</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col0\" class=\"data row3 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col1\" class=\"data row3 col1\" >93</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col2\" class=\"data row3 col2\" >244</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col3\" class=\"data row3 col3\" >249</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col4\" class=\"data row3 col4\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col5\" class=\"data row3 col5\" >187</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col6\" class=\"data row3 col6\" >46</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col7\" class=\"data row3 col7\" >10</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col8\" class=\"data row3 col8\" >8</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col9\" class=\"data row3 col9\" >4</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col10\" class=\"data row3 col10\" >10</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col11\" class=\"data row3 col11\" >194</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col12\" class=\"data row3 col12\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col13\" class=\"data row3 col13\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col14\" class=\"data row3 col14\" >233</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col15\" class=\"data row3 col15\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col16\" class=\"data row3 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row3_col17\" class=\"data row3 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row4\" class=\"row_heading level0 row4\" >4</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col0\" class=\"data row4 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col1\" class=\"data row4 col1\" >107</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col2\" class=\"data row4 col2\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col3\" class=\"data row4 col3\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col4\" class=\"data row4 col4\" >230</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col5\" class=\"data row4 col5\" >48</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col6\" class=\"data row4 col6\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col7\" class=\"data row4 col7\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col8\" class=\"data row4 col8\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col9\" class=\"data row4 col9\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col10\" class=\"data row4 col10\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col11\" class=\"data row4 col11\" >192</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col12\" class=\"data row4 col12\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col13\" class=\"data row4 col13\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col14\" class=\"data row4 col14\" >156</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col15\" class=\"data row4 col15\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col16\" class=\"data row4 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row4_col17\" class=\"data row4 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row5\" class=\"row_heading level0 row5\" >5</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col0\" class=\"data row5 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col1\" class=\"data row5 col1\" >3</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col2\" class=\"data row5 col2\" >20</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col3\" class=\"data row5 col3\" >20</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col4\" class=\"data row5 col4\" >15</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col5\" class=\"data row5 col5\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col6\" class=\"data row5 col6\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col7\" class=\"data row5 col7\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col8\" class=\"data row5 col8\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col9\" class=\"data row5 col9\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col10\" class=\"data row5 col10\" >43</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col11\" class=\"data row5 col11\" >224</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col12\" class=\"data row5 col12\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col13\" class=\"data row5 col13\" >245</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col14\" class=\"data row5 col14\" >74</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col15\" class=\"data row5 col15\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col16\" class=\"data row5 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row5_col17\" class=\"data row5 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row6\" class=\"row_heading level0 row6\" >6</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col0\" class=\"data row6 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col1\" class=\"data row6 col1\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col2\" class=\"data row6 col2\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col3\" class=\"data row6 col3\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col4\" class=\"data row6 col4\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col5\" class=\"data row6 col5\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col6\" class=\"data row6 col6\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col7\" class=\"data row6 col7\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col8\" class=\"data row6 col8\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col9\" class=\"data row6 col9\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col10\" class=\"data row6 col10\" >249</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col11\" class=\"data row6 col11\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col12\" class=\"data row6 col12\" >245</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col13\" class=\"data row6 col13\" >126</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col14\" class=\"data row6 col14\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col15\" class=\"data row6 col15\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col16\" class=\"data row6 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row6_col17\" class=\"data row6 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row7\" class=\"row_heading level0 row7\" >7</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col0\" class=\"data row7 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col1\" class=\"data row7 col1\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col2\" class=\"data row7 col2\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col3\" class=\"data row7 col3\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col4\" class=\"data row7 col4\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col5\" class=\"data row7 col5\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col6\" class=\"data row7 col6\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col7\" class=\"data row7 col7\" >14</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col8\" class=\"data row7 col8\" >101</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col9\" class=\"data row7 col9\" >223</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col10\" class=\"data row7 col10\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col11\" class=\"data row7 col11\" >248</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col12\" class=\"data row7 col12\" >124</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col13\" class=\"data row7 col13\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col14\" class=\"data row7 col14\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col15\" class=\"data row7 col15\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col16\" class=\"data row7 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row7_col17\" class=\"data row7 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row8\" class=\"row_heading level0 row8\" >8</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col0\" class=\"data row8 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col1\" class=\"data row8 col1\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col2\" class=\"data row8 col2\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col3\" class=\"data row8 col3\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col4\" class=\"data row8 col4\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col5\" class=\"data row8 col5\" >11</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col6\" class=\"data row8 col6\" >166</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col7\" class=\"data row8 col7\" >239</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col8\" class=\"data row8 col8\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col9\" class=\"data row8 col9\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col10\" class=\"data row8 col10\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col11\" class=\"data row8 col11\" >187</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col12\" class=\"data row8 col12\" >30</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col13\" class=\"data row8 col13\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col14\" class=\"data row8 col14\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col15\" class=\"data row8 col15\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col16\" class=\"data row8 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row8_col17\" class=\"data row8 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row9\" class=\"row_heading level0 row9\" >9</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col0\" class=\"data row9 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col1\" class=\"data row9 col1\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col2\" class=\"data row9 col2\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col3\" class=\"data row9 col3\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col4\" class=\"data row9 col4\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col5\" class=\"data row9 col5\" >16</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col6\" class=\"data row9 col6\" >248</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col7\" class=\"data row9 col7\" >250</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col8\" class=\"data row9 col8\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col9\" class=\"data row9 col9\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col10\" class=\"data row9 col10\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col11\" class=\"data row9 col11\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col12\" class=\"data row9 col12\" >232</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col13\" class=\"data row9 col13\" >213</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col14\" class=\"data row9 col14\" >111</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col15\" class=\"data row9 col15\" >2</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col16\" class=\"data row9 col16\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row9_col17\" class=\"data row9 col17\" >0</td>\n",
              "            </tr>\n",
              "            <tr>\n",
              "                        <th id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002level0_row10\" class=\"row_heading level0 row10\" >10</th>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col0\" class=\"data row10 col0\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col1\" class=\"data row10 col1\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col2\" class=\"data row10 col2\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col3\" class=\"data row10 col3\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col4\" class=\"data row10 col4\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col5\" class=\"data row10 col5\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col6\" class=\"data row10 col6\" >0</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col7\" class=\"data row10 col7\" >43</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col8\" class=\"data row10 col8\" >98</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col9\" class=\"data row10 col9\" >98</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col10\" class=\"data row10 col10\" >208</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col11\" class=\"data row10 col11\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col12\" class=\"data row10 col12\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col13\" class=\"data row10 col13\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col14\" class=\"data row10 col14\" >253</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col15\" class=\"data row10 col15\" >187</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col16\" class=\"data row10 col16\" >22</td>\n",
              "                        <td id=\"T_09cd021a_bc13_11eb_9210_0242ac1c0002row10_col17\" class=\"data row10 col17\" >0</td>\n",
              "            </tr>\n",
              "    </tbody></table>"
            ],
            "text/plain": [
              "<pandas.io.formats.style.Styler at 0x7fdc9b1d9910>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yoBHs0wwbqG6"
      },
      "source": [
        "---\n",
        "### What is a baseline model and why do you want one?\n",
        "\n",
        "> A simple model that you are confident should perform reasonably well. It should be\n",
        "> simple to implement and easy to test\n",
        "\n",
        "**Why do you want to start with a baseline model?**\n",
        "> ... without starting with a sensible baseline, it is difficult to know whether your super-fancy models are any good\n",
        "\n",
        "**How do you build/find one of these models?**\n",
        "\n",
        "You can search online for folks that have trained models to solve a problem similar to your's and/or you can start with one of the high-level examples in the fastai docs against your data.  There are a bunch covering core vision, text, tabuluar and colab filtering tasks right [here](https://docs.fast.ai/tutorial.html).\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3x904qZCHHcb"
      },
      "source": [
        "---\n",
        "### Tensors\n",
        "\n",
        "**What is a \"Tensor\"?**\n",
        "\n",
        "Like a numpy array, but with GPU support.  The data it contains must be of the ***same type*** and must conform in ***rectangular shape***.\n",
        "\n",
        "> Important: \"try to avoid as much as possible writing loops, and replace them by commands that work directly on arrays or tensors\"\n",
        "\n",
        "\n",
        "Let's take a look .."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4So8iq5hAd7u",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "dabab129-c5de-4d68-8e0e-410fe5beada3"
      },
      "source": [
        "threes = (mnist_path/'train/3').ls().sorted()\n",
        "len(threes), threes[0]"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(6131, Path('/root/.fastai/data/mnist_sample/train/3/10.png'))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LNHfEYsHBpSF",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "348eec91-90d7-41d9-b8d8-690f31d8932b"
      },
      "source": [
        "all_threes = [ tensor(Image.open(fp)) for fp in threes ]\n",
        "len(all_threes), all_threes[0].shape"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(6131, torch.Size([28, 28]))"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "df-v9cdXBpOE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a9d31cfe-372d-44a8-e636-5182bf17b860"
      },
      "source": [
        "stacked_threes = torch.stack(all_threes).float()/255\n",
        "stacked_threes.shape"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([6131, 28, 28])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yDvJ0vpCC01_"
      },
      "source": [
        "Important information about tensors include its `shape`, `rank`, and `type`:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YcbS_oJBCsiJ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "7dd41606-9def-4baa-c38d-07a060860728"
      },
      "source": [
        "# shape = the length of each axis\n",
        "print('shape: ', stacked_threes.shape)\n",
        "\n",
        "# rank = the total number of axes\n",
        "print('rank: ', stacked_threes.ndim)\n",
        "\n",
        "# type = the datatype of its contents\n",
        "print('type: ', stacked_threes.dtype)"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "shape:  torch.Size([6131, 28, 28])\n",
            "rank:  3\n",
            "type:  torch.float32\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mIRCwDYHejmx"
      },
      "source": [
        "Important things you can do to a tensor, `view`, `@`, `where`"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LB7N7Ewqeodl",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "23294969-8b8e-4c89-86b0-3ff0ac3f58db"
      },
      "source": [
        "# view = change the shape of a tensor without changing its contents\n",
        "stacked_threes_rank_2 = stacked_threes.view(-1, 28*28)\n",
        "print('orig. shape: ', stacked_threes.shape)\n",
        "print('make into a rank 2 tensor', stacked_threes_rank_2.shape)\n",
        "\n",
        "# @ = operator for matrix multiplication\n",
        "print('result of matrix multiplication: ', (stacked_threes @ torch.randn((1,28,28))).shape)\n",
        "\n",
        "# where = torch.where(a,b,c) => [b[i] if a[i] else c[i] for i in range(len(a))] ... see p.167\n",
        "trgs = tensor([1,0,1])\n",
        "preds = tensor([0.9, 0.4, 0.2])\n",
        "\n",
        "def mnist_loss(preds, targs):\n",
        "  return torch.where(targs == 1, 1 - preds, preds).mean()\n",
        "\n",
        "print('output of where: ', mnist_loss(preds, trgs))"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "orig. shape:  torch.Size([6131, 28, 28])\n",
            "make into a rank 2 tensor torch.Size([6131, 784])\n",
            "result of matrix multiplication:  torch.Size([6131, 28, 28])\n",
            "output of where:  tensor(0.4333)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ow5Fn9hvFrSZ"
      },
      "source": [
        "For an interactive lesson on matrix multiplication, [this is the best](http://matrixmultiplication.xyz/)!\n",
        "\n",
        "Check out pp.145-148 to learn about \"broadcasting\", a critical piece to understanding how you can and should manipulate tensors or numpy arrays!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_yYHVT5kGXFc"
      },
      "source": [
        "---\n",
        "### Stochastic Gradient Descent - How to train a model\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JpqbW9GgqrZN"
      },
      "source": [
        "Here are the steps:\n",
        "\n",
        "1. **INITIALIZE** the weights = initializing parameters to random values\n",
        "\n",
        "2. For each image, **PREDICT** whether it is a 3 or 7\n",
        " \n",
        "3. Based on the predictions, calculate how good the model is by calculating its **LOSS** (small is good)\n",
        "\n",
        "4. Calculate the **GRADIENT**, ***\"which measures for each weight how changing the weight would change the loss\"***\n",
        "\n",
        "5. **STEP**, change all the weights based on the gradient\n",
        "\n",
        "6. Starting at step 2, **REPEAT**\n",
        "\n",
        "7. **STOP** when you don't want to train any longer or the model is good enough\n",
        "\n",
        "Below, we'll delve deeper into these steps. We'll do this by getting a big more into the sample code beginning on p.150 ..."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J_c6meQugtub"
      },
      "source": [
        "### Step 1: Initializing weights\n",
        "\n",
        "One way is presented on p.164:\n",
        "```\n",
        "def init_params(size, std=1.0) return (torch.randn(size)*std).requires_grad_()\n",
        "\n",
        "weights = init_params((28*28,1)) #=> raturns a rank 2, 784x1 tensor, with random values\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZGgwwU48haT8"
      },
      "source": [
        "### Step 3: Calculating the loss\n",
        "\n",
        "> Important: \"For continuous data, it's common to use *mean squared error*\".  In order to understand how to write this, read it right-to-left (e.g., error -> square -> mean)\n",
        "\n",
        "```\n",
        "def mse(preds, targs): return ((preds-targs)**2).mean()\n",
        "\n",
        "# in PyTorch\n",
        "loss = F.mse_loss(preds, targs)\n",
        "```\n",
        "\n",
        "> Important: Accuracy is a bad loss function\n",
        "\n",
        "\n",
        "**Why is accuracy a poor loss function?**\n",
        "\n",
        "> \"The gradient of a funciton is its slope, or its steepness ... how much the value of the function goes up or down, divided by how much we changed the input `(y_new - y_old) / (x_new - x_old)` .... The problem with [accuracy] is that a small change in weights from `x_old` to `x_new` isn't likely to cause any prediction to change, so `(y_new - y_old)` will almost always be 0 ... **the gradient is 0 almost everywhere.  A very small change in the value of a weight will often not change the accuracy at all\n",
        "\n",
        "A gradient = 0 will mean that the weights aren't updated.\n",
        "\n",
        "> Important: \"We need a loss function that, when our weights result in slightly better predictions, gives us a slightly better loss\"\n",
        "\n",
        "\n",
        "**Metrics v. Loss**\n",
        "\n",
        "> Important: \"... ***the metric is to drive human understanding*** and ***the loss is to drive automated learning***.\n",
        "\n",
        "\n",
        "> Important: \"... focus on these metrics, rather than the loss, when judging the performance of a model.\"\n",
        "\n",
        "\n",
        "> Important: \"... the loss must be a function that has a meaningful derivative ... must be reasonably smooth [so] that [it] would respond to small changes in confidence level.\n",
        "\n",
        "\n",
        "The loss function is one that can be optimized using its gradient!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qv7RlblLIwnn"
      },
      "source": [
        "### Step 4: Calculating the gradients\n",
        "\n",
        "> Important: \"the gradients ***tell us how much we have to change each weight*** to make our model better ... allows us to more quickly calculate whether our loss will go up or down we we make those adjustments\"\n",
        "\n",
        "\n",
        "\n",
        "> Important: \"The gradients ***tell us only the slope of our function***; they don't tell us exactly how far to adjust the parameters. But they do give us some idea of how far\" (large slope = bigger adjustments needed whereas a small slope suggests we are close to the optimal value)\n",
        "\n",
        "\n",
        "\"The ***derivative*** of a function tells you how much a change in its parameters will change its result\"\n",
        "\n",
        "Remember: We are calculating a gradient for *EVERY* weight so we know how to adjust it to make our model better (i.e., lower the LOSS)\n",
        "\n",
        "`requires_grad` tells PyTorch \"that we want to calculate gradients with respect to that variable at that value\""
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BlKI_4ZWZWor"
      },
      "source": [
        "def plot_function(f, tx=None, ty=None, title=None, min=-2, max=2, figsize=(6,4)):\n",
        "    x = torch.linspace(min,max)\n",
        "    fig,ax = plt.subplots(figsize=figsize)\n",
        "    ax.plot(x,f(x))\n",
        "    if tx is not None: ax.set_xlabel(tx)\n",
        "    if ty is not None: ax.set_ylabel(ty)\n",
        "    if title is not None: ax.set_title(title)"
      ],
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HRRz6Z39Z8_I"
      },
      "source": [
        "Here we pretend that the below is our **loss function**.  Running a number through it, our **weight** will produce a result, an **activation** ... in this case, our **loss** (which again is a value telling us how good or bad our model is; smaller = good)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HjQCRo2eGVpA",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "29366394-f913-43f9-afb6-72333886b1aa"
      },
      "source": [
        "xt = tensor(-1.5).requires_grad_(); xt"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(-1.5000, requires_grad=True)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RpWRqpteY6dJ",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 350
        },
        "outputId": "de2424d9-7e70-4cc5-a248-6bb890499716"
      },
      "source": [
        "def f(x): return x**2\n",
        "loss = f(xt)\n",
        "\n",
        "plot_function(f, 'x', 'x**2')\n",
        "plt.scatter(xt.detach().numpy(), loss.detach().numpy(), color='red')\n",
        "print('Loss: ', loss.item())"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Loss:  2.25\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/ipykernel_launcher.py:2: UserWarning: Not providing a value for linspace's steps is deprecated and will throw a runtime error in a future release. This warning will appear only once per process. (Triggered internally at  /pytorch/aten/src/ATen/native/RangeFactories.cpp:23.)\n",
            "  \n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1f3/8dcnO0kgEJKwJIEAYZF9iewgQrW4gaKoqCAFRVyqdvl+q7W1La3fflu/tXVpaxFRREQUcUFxoaIF2cO+QyA7gYSEhOzr+f0xg78Yk5CE3NzJzOf5eMzDO3MPc98ZMJ+595x7jhhjUEop5bm87A6glFLKXloIlFLKw2khUEopD6eFQCmlPJwWAqWU8nA+dgdorLCwMBMTE2N3DKWUalV27dp1zhgTXtu+VlcIYmJiiI+PtzuGUkq1KiKSXNc+vTSklFIeTguBUkp5OC0ESinl4bQQKKWUh9NCoJRSHs7yQiAi3iKyR0Q+rmWfv4isEpEEEdkuIjFW51FKKfVdLXFG8BhwpI5984HzxphY4K/An1ogj1JKqWosLQQiEgXcACypo8l0YJlzezUwRUTEiiwJmQUsWnuYsooqK95eKaUs9fy/T7D9VLYl7231GcHfgP8G6vrtGwmkAhhjKoA8oGPNRiKyQETiRSQ+KyurSUFSc4pYujmRDUfPNunPK6WUXVKyi/jrv4+zPTHHkve3rBCIyI1ApjFm1+W+lzFmsTEmzhgTFx5e6x3SlzSxTzid2wXw9s7Uy42jlFIt6p34VLwEbhsRZcn7W3lGMA6YJiJJwNvAZBF5s0abdCAaQER8gBDAknMfby9hZlwUG49ncTq32IpDKKVUs6uorGL1rjQm9gmna/s2lhzDskJgjHnSGBNljIkB7gQ2GGPuqdHsI+Be5/ZtzjaWrZ05c0Q0VQZW70qz6hBKKdWsNp7I4syFEu68MtqyY7T4fQQiskhEpjmfvgp0FJEE4KfAE1Yeu1vHQMbFduSd+FSqqnStZqWU61u1M5WOQX5M7tfJsmO0SCEwxnxtjLnRuf20MeYj53aJMWamMSbWGDPSGHPK6iy3x0WTdr6YLSet6X1XSqnmkplfwpdHMrl1RBR+Ptb9uva4O4t/OKAzIW18eXtnit1RlFKqXmt2p1NRZbg9zrrLQuCBhSDA15tbhkXyxaGznC8sszuOUkrVyhjDOztTievegdiIYEuP5XGFAOCOK6Mpq6xizZ50u6MopVStdiTmcOpcIbdb2El8kUcWgiu6tGNodHtW7kjBwkFKSinVZCt3pNA2wIebBne1/FgeWQgAZo2MJiGzgF3J5+2OopRS35FbVMa6g2e4eWgkbfy8LT+exxaCGwd3Jdjfh7d2aKexUsq1rNmdTllFFbNGdmuR43lsIQjy92H60K58sj+DvKJyu+MopRTg6CR+e2cKQ6Lb079ruxY5pscWAoBZI7tRWlHFB3u101gp5Rp2p5zn+NkCZrVAJ/FFHl0IBkaGMCgyRDuNlVIuY+WOVIL8vLlpiPWdxBd5dCEAx1nB0TP57EnNtTuKUsrD5RWX8/H+00wfFkmQv0+LHdfjC8G0oV0J8vNm5XbtNFZK2euDPemUlFcx68qW6SS+yOMLQbC/D9OHRbJ2/2ntNFZK2cYYw4rtyQyJCmFQVEiLHtvjCwHAXSO7UVJexZo9Oj21Usoe8cmOTuK7RrXs2QBoIQAcncZDo9uzYrt2Giul7LFiWzJt/X1atJP4Ii0ETneN6kZCZgE7LFoTVCml6pJTWMa6A2eYMTySQL+W6yS+SAuB002Du9I2wIcV2mmslGphq3elUlZZxV2juttyfCsXrw8QkR0isk9EDonI72ppM1dEskRkr/Nxn1V5LqWNnze3Do/is4NnyC4otSuGUsrDVFUZVu5wTDfdt3NbWzJYeUZQCkw2xgwBhgJTRWR0Le1WGWOGOh9LLMxzSXeP6kZZZRXvxGunsVKqZWw5mU3iuUJbOokvsnLxemOMKXA+9XU+XLontnentozqEcpbO5Kp1DWNlVItYPm2JDoE+nL9oC62ZbC0j0BEvEVkL5AJrDfGbK+l2a0isl9EVotIrZNriMgCEYkXkfisrCwrIzN7THdSc4rZeNza4yilVEZeMesPn+X2K6MJ8LV+uum6WFoIjDGVxpihQBQwUkQG1miyFogxxgwG1gPL6nifxcaYOGNMXHh4uJWRubZ/Z8Lb+rN8W7Klx1FKqZXbUzDAPTZ1El/UIqOGjDG5wFfA1BqvZxtjLvbMLgFGtESe+vj5eDHrymi+OpZJak6R3XGUUm6qrKKKlTtTubpvBNGhgbZmsXLUULiItHdutwGuAY7WaFP9otg04IhVeRpj1qhueInoUFKllGW+OHyGrPxSZo+292wArD0j6AJ8JSL7gZ04+gg+FpFFIjLN2eZR59DSfcCjwFwL8zRYl5A2/OCKCN6JT6WkvNLuOEopN7R8azLRoW2Y2Mfay90NYdktbMaY/cCwWl5/utr2k8CTVmW4HLNHx/D5obN8ejCDW4ZF2R1HKeVGjp/NZ3tiDk9c1w9vL7E7jt5ZXJexvTrSMzyIZVu001gp1bze2JqEn48Xt8e13Cpk9dFCUAcvL2HO6O7sTc1lny5ao5RqJhdKylmzO51pQ7oSGuRndxxAC0G9bh0RRZCfN8u2JtkdRSnlJlbHp1FUVsncsTF2R/mWFoJ6tA3w5dYRUXy8L0PnH1JKXbaqKsPybckM79aegZEtu/hMfbQQXMKcMd0pq6zi7Z2pdkdRSrVyG09kkXiukHtd6GwAtBBcUmxEW8bHhvHmtmQqKqvsjqOUasXe2JpMWLA/1w20b16h2mghaIA5Y7qTkVfC+sNn7Y6ilGqlkrML+epYJneN6oafj2v96nWtNC5qyhWdiOrQhte2JNkdRSnVSi3bkoy3CHfbON10XbQQNIC3lzBnTHd2JOZw6HSe3XGUUq1MQWkF78ancv2gLnRqF2B3nO/RQtBAd8R1o42vN69vTrI7ilKqlXlvVxr5pRX8aFyM3VFqpYWggUICfbl1RCQf7jutQ0mVUg1WVWV4fUsSQ6PbM6xbB7vj1EoLQSPMHRtDWUUVb+mspEqpBvrPcceQUVc9GwAtBI0SG9GWCb3DWL4tmbIKHUqqlLq0pZsTiWjrekNGq9NC0EjzxvUgM7+UTw9m2B1FKeXiEjLz2XTiHLNHd3e5IaPVuW4yF3VVn3B6hgWx9JtEjNEF7pVSdXtts2OW0VkuOGS0Oi0EjeTlJfxoXAz70vLYlXze7jhKKRd1vrCM93anccvQSMKC/e2OUy8rl6oMEJEdIrLPuQrZ72pp4y8iq0QkQUS2i0iMVXma060joghp48uSTYl2R1FKuai3dqRQUl7F/Ak97I5ySVaeEZQCk40xQ4ChwFQRGV2jzXzgvDEmFvgr8CcL8zSbQD8f7hrVjS8OnyElWxe4V0p9V1lFFcu2JDGhdxh9OrW1O84lWVYIjEOB86mv81Hzovp0YJlzezUwRUTsX7etAe4dE4OXCK9t0bMCpdR3fbz/NJn5pdw3oafdURrE0j4CEfEWkb1AJo7F67fXaBIJpAIYYyqAPKBjLe+zQETiRSQ+KyvLysgN1jkkgBsHd+GdnalcKCm3O45SykUYY1iyKZHeEcFM7B1md5wGsbQQGGMqjTFDgShgpIgMbOL7LDbGxBlj4sLDw5s35GWYP74nhWWVrNqhaxUopRy2ncrhcMYF5o3vQSu5wNEyo4aMMbnAV8DUGrvSgWgAEfEBQoDslsjUHAZFhTCyRyivb0miXNcqUEoBSzadIjTIj1uGRdodpcGsHDUULiLtndttgGuAozWafQTc69y+DdhgWtng/AUTepKeW8y6A3qDmVKeLiEzny+PZjJnTHcCfL3tjtNgVp4RdAG+EpH9wE4cfQQfi8giEZnmbPMq0FFEEoCfAk9YmMcSk/tF0Cs8iMUbT+kNZkp5uCWbEvH38WL26O52R2kUH6ve2BizHxhWy+tPV9suAWZalaEleHkJ90/oyRNrDrD1ZDZjY1tH55BSqnll5pewZnc6t18ZRUcXv4GsJr2zuBncPCySsGA/Fm86ZXcUpZRN3tiSTHlVFfPHt44ho9VpIWgGAb7e3Dsmhq+PZXHsTL7dcZRSLayorILl25K5tn8neoQF2R2n0bQQNJN7RncnwNeLV/SsQCmP8258GnnF5SyY2PrOBkALQbPpEOTHHXHRfLg3nYy8YrvjKKVaSEVlFa9sOsWI7h0Y0T3U7jhNooWgGd03oSdVBpZ+o9NOKOUpPjmQQdr5YhZe1cvuKE2mhaAZRYcGcuPgLry1PYW8Ip12Qil3Z4zh5f+cIjYimCn9IuyO02RaCJrZAxN7UVhWyZvbk+2OopSy2H+OZ3Ek4wILJvbEy6t1TCdRGy0Ezax/13Zc1Sec1zYnUlJeaXccpZSFXv7PSTq3C+Dmoa1nOonaaCGwwMKrenGuoIzVu9LsjqKUssje1Fy2ncph/vgeLr0ecUO07vQuanTPUIZEt2fxxlNU6GR0Srmll78+SdsAH5dfj7ghtBBYQER4aFIvUnKK+EQno1PK7SRk5vPZoTPMHRtDsL9lM/W0GC0EFrnmik70jgjmn1+f1MnolHIz//z6FG18vfnRONdfj7ghtBBYxMtLeOjqXhw9k8+Go5l2x1FKNZPUnCI+2JvOrJHdCA3ysztOs9BCYKGbBnclqkMbXvoqQc8KlHITr2w6hZfA/RPd42wAtBBYysfbiweu6sWeFMfoAqVU65aZX8LbO1OZMSyKLiFt7I7TbKxcoSxaRL4SkcMickhEHqulzSQRyRORvc7H07W9V2s2c0QUYcH+/P2rBLujKKUu09JvkqiorGLhpNY7nURtrDwjqAB+ZozpD4wGHhaR/rW022SMGep8LLIwjy0CfL1ZMLEH3yScY0/KebvjKKWa6HxhGcu3JnH9oC6tcqrp+lhWCIwxGcaY3c7tfOAI0Lpvv2uiu0d1p0OgLy9u0LMCpVqr1zYnUlhWySOTY+2O0uxapI9ARGJwLFu5vZbdY0Rkn4h8KiID6vjzC0QkXkTis7KyLExqjSB/H+aP78GGo5kcTM+zO45SqpEulJTz2pYkpg7oTL/O7eyO0+wsLwQiEgy8BzxujLlQY/duoLsxZgjwIvBBbe9hjFlsjIkzxsSFh4dbG9gic8bG0C7Ahxc3nLA7ilKqkZZtTiK/pMItzwbA4kIgIr44isAKY8yamvuNMReMMQXO7XWAr4i45erv7QJ8mTuuB58fOsuRjJr1UCnlqgpKK3h1cyJT+kUwMDLE7jiWsHLUkACvAkeMMc/V0aazsx0iMtKZJ9uqTHabNy6GID9vXtIRREq1Gm9uSya3qJwfT+ltdxTLWDlJxjhgNnBARPY6X/sl0A3AGPMycBvwoIhUAMXAncaN77xqH+jHnLExvPyfk5w4m0/vTm3tjqSUqkdRWQWvbDzFhN5hDI1ub3ccy1hWCIwx3wD1rtRgjHkJeMmqDK7o/gk9WbYliee/PMFLdw23O45Sqh7LtyaTXVjG4z9w37MB0DuLW1xokB/3jo3hkwMZHD+bb3ccpVQdisoq+JfzbKC1LkrfUFoIbHD/hJ4E+nrzwpc6gkgpV/XG1mRyCst4/Ad97I5iOS0ENtCzAqVcW2FpBYs3nmJin3BGdO9gdxzLaSGwycWzguf1rEApl/P/zwbcu2/gIi0ENukQ5MfckELW7UvnaEQPiImBFSvsjqWUxysorWDxxpNM7BPO8G7ufzYAWgjss2IF9z/zIMFlxfx13F2QnAwLFmgxUMpmr32TyPmicn52jfv3DVykhcAuTz1F+/NZzN/5AZ/3HcuBTr2gqAieesruZEp5rLyichZvOsUPrujEEDe+b6AmLQR2SUkBYN7OD2lffIG/TJj9ndeVUi3vlU2nyC+p4KcedDYAWgjs060bAO3Kinhg+3t83SuOXZH9vn1dKdWysgtKWbo5kRsGd6F/V/ebYbQ+Wgjs8swzEBgIwL27Pyas8Dx/uepex+tKqRb3r42nKCmv5CceMlKoOi0Edrn7bli8GLp3J7CijIeO/Zst0YPYPOqHdidTyuOcySth2ZYkbh4aSWyE580BVm8hEJF2IvK9xTlFZLB1kTzI3XdDUhJUVXHXJ6/SNSSAP392FDeed08pl/TChhNUGcNPPKxv4KI6C4GI3A4cBd5zLj5/ZbXdr1sdzNME+Hrz+DV92JeWx+eHztgdRymPkXiukFU7U7lrZDeiQwPtjmOL+s4IfgmMMMYMBX4ELBeRW5z76p1VVDXNjGGR9AoP4v++OE5FZZXdcZTyCM+tP46ftxePTPa8voGL6isE3saYDABjzA7gauBXIvIooNcuLODj7cXPr+1LQmYBa/ak2x1HKbd3MD2PtftOM298DOFt/e2OY5v6CkF+9f4BZ1GYBEwHal1kXl2+qQM7MzgqhL+tP05JeaXdcZRya//3xTFC2viyYOL3ukI9Sn2F4EFqXAIyxuQDU4F5l3pjEYkWka9E5LCzj+GxWtqIiLwgIgkisl9EPH6lFhHhF1P7cTqvhDe3JdsdRym3tfVkNl8fy+LBSb0IaeNrdxxb1VkIjDH7gBgAEZlS7fVyY0xDJsSpAH5mjOkPjAYeFpH+NdpcB/R2PhYA/2xUejc1LjaMCb3DeHFDAnlF5XbHUcrtVFUZ/vjpEbqEBDB3bIzdcWx3qfsIrhKRcTguCTWKMSbDGLPbuZ0PHAEiazSbDrxhHLYB7UWkS2OP5Y6euK4fF0rK+cd/dKF7pZrbJwcy2J+Wx8+u7UuAr7fdcWxX3/DR3wD+wL8BPxF5uqkHEZEYYBiwvcauSCC12vM0vl8sEJEFIhIvIvFZWVlNjdGqDOgawi1DI3ltcxKnc4vtjqOU2yirqOLZz4/Rr3Nbbhn2vV83Hqm+S0O/w3EfwW+Bo8aYRU05gIgEA+8BjxtjLjTlPYwxi40xccaYuPDw8Ka8Rav002sdN7c8t/64zUmUch9vbU8mJaeIX1zXD28vHQkPl7401NYY8yegSfdci4gvjiKwwhizppYm6UB0tedRztcUENUhkLljY3hvdxpHMppUQ5VS1VwoKeeFDQmM6dmRSX0850vlpVyqEByq8d8GExEBXgWOGGOeq6PZR8Ac5+ih0UDexXsXlMPDk2IJaePLM58c0aknlLpM//jqJDmFZfzy+itw/IpSYGFnMTAOmA1MFpG9zsf1IrJQRBY626wDTgEJwCvAQ004jlsLCfTl0cm9+SbhHF8f84z+EaWskJpTxNLNicwYFsmgqBC747gUn7p21OgsfkFEnm5MP4Ex5hsuMRWFcXzFfbih7+mp7hndneXbknlm3REm9A7Dx1snjVWqsf78+TG8BH7+w752R3E5lncWq8vn5+PFE9f1IyGzgJU7Uy/9B5RS37En5Txr951mwYSedG3fxu44LudSXy3bAWuB4OovisgkqwKp2l3bvxMje4Tyt/XHuVCiN5kp1VDGGP7wyRHC2/rzwFWePZVEXeotBMaYvwHvAIHODt02IvIi8McWSae+JSL8+ob+ZBeW8fcNepOZUg318f4MdiWf52fX9CHIv86r4R6tIRebR+EY4rkF2AmcxtERrFrYoKgQZo6IYunmRBLPFdodRymXV1xWyf9+epQBXdsxMy760n/AQzWkEJQDxUAbIABINMboZPk2+a+pffHz9uKZT47YHUUpl7d44ynSc4v5zU0D9OaxejSkEOzEUQiuBCYAs0TkXUtTqTpFtA3gkcm9+feRs2w6ocNJlarL6dxi/vmfBG4Y3IWRPULtjuPSGlII5htjnnbOOpphjJmO40YwZZN542Po3jGQRWsP60pmStXhfz89ijHw5HX97I7i8i5ZCIwx8bW8ttyaOKoh/H28eer6KziRWcByXbNAqe/ZmZTDR/tO88BVvYjq4JnrEDeG3pnUSl3TvxMTeofx3PrjnCsotTuOUi6jorKKX39wkK4hASy8qqfdcVoFLQStlIjw22kDKCmv5E+fHrU7jlIuY8X2FI6eyefXN/Yn0E+HizaEFoJWrFd4MPPG9+DdXWnsTjlvdxylbHeuoJS/fHGM8bFhTB3Y2e44rYYWglbu0cm96dTOn998eIjKKp2dVHm2Zz87RlFZJb+d1l9nF20ELQStXJC/D0/d0J8D6Xms3JFidxylbLMn5Tyr4lOZP74HsRFNWkLFY2khcAM3De7C2F4d+fNnR7XjWHmkisoqnnr/IJ3bBfDjKb3tjtPqaCFwAyLCoukDKS6v5H/0jmPlgZZtTeZwxgV+c1N/gnU+oUbTQuAmYiOCeWBiL9bsSWfLyXN2x1GqxZzJK+G5L44xqW+4dhA3kWWFQESWikimiBysY/8kEcmrtnrZ01Zl8RSPTI6lW2ggv/7gIGUVesex8gy///gwFVWGRdMGagdxE1l5RvA6MPUSbTYZY4Y6H7rwzWUK8PXmd9MHcDKrkMUbT9odRynLfX0sk08OZPDjybF066h3EDeVZYXAGLMRyLHq/VXtru4bwQ2DuvDChgROZRXYHUcpyxSVVfCrDw4SGxHM/RP1DuLLYXcfwRgR2Scin4rIgLoaicgCEYkXkfisLJ1x81J+c1N//H28+OX7B3AsC62U+3nui+OknS/mjzMG4e/jbXecVs3OQrAb6G6MGQK8CHxQV0NjzGJjTJwxJi48PLzFArZWEe0C+OX1V7DtVA7vxqfZHUepZncgLY+lmxO5a1Q3rozRKaYvl22FwBhzwRhT4NxeB/iKSJhdedzNHXHRjIwJ5Zl1R8jK13sLlPuoqKziiTX7CQv25xdTdYrp5mBbIRCRzuLs4heRkc4s2XblcTdeXsL/zBhEcVklv117yO44SjWbJd8kcuj0BX43bQAhbXztjuMWrBw+uhLYCvQVkTQRmS8iC0VkobPJbcBBEdkHvADcafSCdrOKjQjm0SmxfLI/g88OnrE7jlKX7WRWAc+tP861/TvpPQPNyLJb8Iwxsy6x/yXgJauOrxweuKoX6w6c4dcfHmR0z1DaB/rZHUmpJqmsMvz36v208fXmDzfrPQPNye5RQ8pivt5ePDtzMOcLy1j08WG74yjVZG9sTWJX8nmevrE/Ee0C7I7jVrQQeIABXUN4cFIv1uxO56ujmXbHUarRUrKL+PNnjmkkZgyPtDuO29FC4CEemRxL74hgnlxzgLyicrvjKNVgVVWG/1q9D28v4X9uGaSXhCyghcBD+Pt485fbh5BVUKqjiFSr8tqWJLYn5vD0Tf3p2r6N3XHckhYCDzI4qj0PXx3L+3vS+exght1xlLqkhMwC/vzZUab0i2DmiCi747gtLQQe5pGrYxnQtR1PvX9QF7FRLq2isoqfvbuPNn7e/HGGXhKykhYCD+Pn48Vztw8lv6SCp3QuIuXC/vn1Sfal5vKHmwfqKCGLaSHwQH07t+Vn1/bh80NndS4i5ZL2peby/JcnuGlIV24c3NXuOG5PC4GHum9CT0b3DOW3aw+RnF1odxylvlVUVsHjq/YS0dafP0wfaHccj6CFwEN5ewnP3T4UHy/h8VV7qajUFc2Ua/j9x0dIyi7kL7cPJSRQ5xJqCVoIPFjX9m145pZB7EnJ5cUNCXbHUYr1h8+yckcKCyb2ZEyvjnbH8RhaCDzcTUO6MmNYJC9uOMGORF1QTtnnTF4J/716H/27tOOn1/SxO45H0UKgWHTzQLqFBvLY23vILSqzO47yQJVVhsfe3kNpRRUv3jVMVxxrYVoIFMH+Prw4azjnCkr5r9X7dUipanEvbUhge2IOi6YPpFd4sN1xPI4WAgXAoKgQfjG1H+sPn2X5tmS74ygPsiMxh+e/PM4twyK5VSeUs4WVC9MsFZFMETlYx34RkRdEJEFE9ovIcKuyqIaZP74Hk/tF8IePj3AgLc/uOMoDZBeU8ujKPXQLDeT3usaAbaw8I3gdmFrP/uuA3s7HAuCfFmZRDSAi/N/MIXQM9uOht3bpLKXKUpVVhsdX7SWnqIy/3z2cYH/L1slSl2BZITDGbATqG4YyHXjDOGwD2otIF6vyqIYJDfLjpbuGk5Fbws9X79P+AmWZFzecYNOJc/xu2gAGdA2xO45Hs7OPIBJIrfY8zfmastmI7h148vorWH/4LK9sOmV3HOWGvjlxjue/PMGMYZHceWW03XE8XqvoLBaRBSISLyLxWVlZdsfxCPPGxXDdwM786bNjbD2ZbXcc5UbSc4t59O09xIYH84dbtF/AFdhZCNKB6l8FopyvfY8xZrExJs4YExceHt4i4TydiPDn2wYT0zGQR97azencYrsjKTdQUl7JwuW7KK+o4uXZIwj0034BV2BnIfgImOMcPTQayDPG6GopLqRtgC//mh1HaUUVD765i5LySrsjqVbMGMOvPjjIgfQ8nrtjqN4v4EKsHD66EtgK9BWRNBGZLyILRWShs8k64BSQALwCPGRVFtV0sRHB/OX2IexLy+PpDw9q57Fqsje3JbN6VxqPTunNNf072R1HVWPZeZkxZtYl9hvgYauOr5rPDwd05seTY3lxQwL9u7Rj7rgedkdSrczWk9n8bu1hJveL4PEpve2Oo2poFZ3Fyn4/+UEfrunfiUUfH2bjce2wVw2Xkl3Egyt2ERMWxN/uHIqXl3YOuxotBKpBvLyEv94xlD6d2vLwW7s5mVVgdyTVCuSXlDN/2U4AlsyJo12Ari/girQQqAYL9vfhlTlx+Hl7cf+yeJ2pVNXLMaPoXhLPFfKPu4cTExZkdyRVBy0EqlGiQwN5efYI0s4X88DyXZRW6Egi9X3GGH639hAbjmby22kDGNsrzO5Iqh5aCFSjXRkTyrMzB7M9MYcn3jugI4nU97z6TSJvbE1mwcSe3DO6u91x1CXo3RyqSaYPjSQ1p4j/++I40aGBuqKU+tZnB8/wzLojXDewM09M7Wd3HNUAWghUkz18dSwpOUW88OUJItsHcMeV3eyOpGy2KzmHx97ew9Do9vz1Dh0h1FpoIVBNJiI8c8sgMvNLeXLNAToE+nHtgM52x1I2OX42n3mvx9O1fRuWzIkjwFeXm2wttI9AXRZfby/+cfdwBkW158cr97Ajsb6Zx5W7Ss8tZs6rO+3NDgIAAA/xSURBVPD38eKNeSPpGOxvdyTVCFoI1GUL9PPhtblXEtmhDfOX7eRIxgW7I6kWlF1QypxXt1NYWsGyeSOJDg20O5JqJC0EqlmEBvnxxryRBPv7MPvV7XrDmYfIKy5nztIdpJ0vZsm9cVzRpZ3dkVQTaCFQzSaqQyBv3jcKgHuWbCc1p8jmRMpKhaUVzHt9J8fP5vPy7BGM6tnR7kiqibQQqGbVKzyY5fNHUVRWyd1LtnMmr8TuSMoCJeWVLFgez97UXF6cNYyr+0bYHUldBi0Eqtld0aUdy+aNJKewjFmvbNNi4GZKyiu5/414tpzM5tnbBjN1oC413tppIVCWGBrdnmXzRpKVX6rFwI1cLALfJJzjz7cOZsbwKLsjqWaghUBZZkT3Dt8WgzsXbyUjT5e7bM2Kyyq5b5mjCDx72xBmxumi8+7C0kIgIlNF5JiIJIjIE7XsnysiWSKy1/m4z8o8quWN6N6BN+aPJLugjJkvbyU5u9DuSKoJ8kvKuXfpDjafdBSB20bomYA7sXKpSm/g78B1QH9gloj0r6XpKmPMUOdjiVV5lH2Gd+vAW/ePprC0gpkvb+XE2Xy7I6lGOF9Yxt1LtrM75Twv3DlMi4AbsvKMYCSQYIw5ZYwpA94Gplt4POXCBkWFsOqBMQDc/q+t7E/LtTmRaoizF0q4Y/FWjp7J51+zR3DTkK52R1IWsLIQRAKp1Z6nOV+r6VYR2S8iq0Wk1ouOIrJAROJFJD4rS5dJbK36dGrLuwvHEOTvw52Lt/H1sUy7I6l6JGTmM+MfW0g7X8zrc69kyhW64Ly7sruzeC0QY4wZDKwHltXWyBiz2BgTZ4yJCw8Pb9GAqnl17xjEmgfHEtMxiPuWxbN6V5rdkVQt4pNyuPWfWymtqGLVgjGMjdWFZdyZlYUgHaj+DT/K+dq3jDHZxphS59MlwAgL8ygXEdEugFUPjGZ0z478/N19vPDlCV3cxoV8eiCDu5dsJzTIjzUPjmVQVIjdkZTFrCwEO4HeItJDRPyAO4GPqjcQkep3okwDjliYR7mQtgG+LJ17JTOGRfLc+uM8vmovJeW67KWdjDG8tOEED67YzYCu7XjvwbF066gTyHkCy9YjMMZUiMgjwOeAN7DUGHNIRBYB8caYj4BHRWQaUAHkAHOtyqNcj5+PF3+5fQi9IoJ59vNjpOQUsXh2HOFtdQrjllZSXsmTaw7w/p50bh7alf+9dbCuJ+BBpLWdksfFxZn4+Hi7Y6hm9umBDH7yzl46BPrxj7uHM6xbB7sjeYzTucU8+OYu9qXl8fNr+/Dw1bGI6Mpi7kZEdhlj4mrbZ3dnsVIAXDeoC6sXjsXbS7jjX9tYuSPF7kgeYcvJc9z04jeczCrk5XtG8Mjk3loEPJAWAuUyBkaGsPaR8YzqGcqTaw7wX+/uo6iswu5YbqmqyvDPr09yz5LttA/05YOHxzF1oC4z6ql0zWLlUjoE+fH6j0by1/XH+fvXCexJzeWlu4bRr7MueNJcsvJL+ek7e9l04hw3DOrCn24bTLC//irwZHpGoFyOt5fw8x/2Zfm8UeQWlTP9pc0s35asQ0ybwcbjWVz3/CZ2JObwP7cM4qW7hmkRUFoIlOsa3zuMTx+bwKieHfn1Bwe597WdOp11ExWVVfCrDw4wZ+kOOgT68uEj47hrVDftD1CAFgLl4sLb+vP63Cv5/fQB7EzM4dq//of396Tp2UEj7EzK4brnN7Fiewr3je/B2h+P10tt6ju0ECiX5+UlzB4Tw7rHJtC7U1t+smof9762k5RsXRO5PnlF5Ty55gAzX95KZZVh5f2j+dWN/fX+APU9eh+BalUqqwzLtybx7OfHqDSGx6b0Yf74Hvj56Heai4wxrN2fwaK1h8kpLGX++B785Jo+BPppX4Anq+8+Ai0EqlU6nVvMbz46xPrDZ+kRFsRT11/BlCsiPP6a94G0PBZ9fIidSecZFBnCH2cMYmCkzhWktBAoN/bVsUx+//FhTmUVMqF3GL+Y2s8jf/Gl5xbzt/XHWb07jdBAP37+w77cHheNt5dnF0b1/2khUG6tvLKKN7Ym88KXJ8grLueGQV346bV96BUebHc0y50rKOUfX53kzW3JAMwZ051Hf9CbdgG+NidTrkYLgfIIF0rKWbLxFEu+SaSkvJIbBndl4VU9GdDV/c4QTucW88qmU7y9I5XSikpuGxHFYz/oQ2T7NnZHUy5KC4HyKOcKSnll0ylWbEuhoLSCSX3DmTeuB+Njw/Bq5ZdKDqbn8fqWJD7cm06VgelDu/LQpFhiI9z/7EddHi0EyiPlFZWzfFsSr29J4lxBGT3CgrhndHdmDIukQ5Cf3fEarLisks8PneGNrUnsTsmlja83t8dFcf/EnkR10PUCVMNoIVAerbSikk8PnGHZ1iT2pOTi6y1c3TeCGcMjmdQ3wiXH1VdWGXYk5vD+njTWHThDQWkFMR0DmT0mhttGRBHSRvsAVOPUVwh0YLFye/4+3tw8LJKbh0Vy+PQF1uxO44O9p/ni8FkC/byZ1Deca/t3ZmKfcEJtPFMoKqtg26lsPj94ln8fOUt2YRlBft5cN6gLM4ZFMrpnx1Z/aUu5JkvPCERkKvA8jhXKlhhj/rfGfn/gDRxrFWcDdxhjkup7Tz0jUM2horKKLSez+fzQGb44fJasfMfS2f27tGN87zDiundgSHR7OrULsCxDXlE5+9Nz2Z2cy+aT59iTcp7ySkOwvw9X94vg2v6dmHJFhN4IppqFLZeGRMQbOA5cA6ThWMN4ljHmcLU2DwGDjTELReRO4BZjzB31va8WAtXcqqoM+9Jy2Zxwjs0J2exKPk9ZZRUAndsF0K9LW2LDg+kVEUx0h0A6h/jTqV0AbRswRLOkvJKzF0o4k1dCem4xJ7MKOJlZyLGz+SSeKwRABAZ2DWFsbEfG9QpjVM9Q/H1c73KVat3sKgRjgN8aY37ofP4kgDHmj9XafO5ss1VEfIAzQLipJ5QWAmW1kvJKDmdcYG9KLvvScjl+toBTWQWUVlR9p52ftxfBAT4E+Xvj7+PNxYs25ZVVFJRWUlBaTkn5d/+Mj5fQvWMgsRHBDI5qz9Do9gyKCtFx/8pydvURRAKp1Z6nAaPqauNc7D4P6Aicq95IRBYACwC6detmVV6lAAjw9WZ4tw4Mr7ZuclWVIT23mPTc4m+/4Z8vKqegtJzC0kpKKyq/bevj5UWQvw9tA3xoF+BDp3YBdA4JoEtIG7p3DMTXW+dFUq6lVVx8NMYsBhaD44zA5jjKA3l5CdGhgUSH6nBN5X6s/GqSDkRXex7lfK3WNs5LQyE4Oo2VUkq1ECsLwU6gt4j0EBE/4E7goxptPgLudW7fBmyor39AKaVU87Ps0pDzmv8jwOc4ho8uNcYcEpFFQLwx5iPgVWC5iCQAOTiKhVJKqRZkaR+BMWYdsK7Ga09X2y4BZlqZQSmlVP10+IJSSnk4LQRKKeXhtBAopZSH00KglFIertVNQy0iWUByE/94GDXuWnYRrpoLXDeb5moczdU47piruzEmvLYdra4QXA4Ria9rrg07uWoucN1smqtxNFfjeFouvTSklFIeTguBUkp5OE8rBIvtDlAHV80FrptNczWO5mocj8rlUX0ESimlvs/TzgiUUkrVoIVAKaU8nFsXAhF5VkSOish+EXlfRNrX0W6qiBwTkQQReaIFcs0UkUMiUiUidQ4FE5EkETkgIntFxPL1ORuRq0U/L+cxQ0VkvYiccP63Qx3tKp2f114RqTnteXNlqffnFxF/EVnl3L9dRGKsyNGEXHNFJKva53NfC+VaKiKZInKwjv0iIi84c+8XkeEukmuSiORV+7yerq2dBbmiReQrETns/P/xsVraNO9nZoxx2wdwLeDj3P4T8Kda2ngDJ4GegB+wD+hvca4rgL7A10BcPe2SgLAW/LwumcuOz8t53D8DTzi3n6jt79K5r8DiHJf8+YGHgJed23cCq1rg82lIrrnASy3176nacScCw4GDdey/HvgUEGA0sN1Fck0CPrbh8+oCDHdutwWO1/J32ayfmVufERhjvjDGVDifbsOxSlpNI4EEY8wpY0wZ8DYw3eJcR4wxx6w8RlM0MFeLf15O04Flzu1lwM0tcMzaNOTnr551NTBFRARr2fX3cknGmI041hupy3TgDeOwDWgvIl1cIJctjDEZxpjdzu184AiO9d2ra9bPzK0LQQ3zcFTQmiKB1GrP0/j+h24XA3whIrtEZIHdYZzs+rw6GWMynNtngE51tAsQkXgR2SYiVhSLhvz837ZxfhHJAzpakKWxuQBudV5KWC0i0bXst4Mr/z84RkT2icinIjKgpQ/uvKw4DNheY1ezfmatYvH6+ojIv4HOtex6yhjzobPNU0AFsMKVcjXAeGNMuohEAOtF5KjzW4zduSxRX7bqT4wxRkTqGvfc3fmZ9QQ2iMgBY8zJ5s7aSq0FVhpjSkXkARxnLZNtzuTKduP491QgItcDHwC9W+rgIhIMvAc8boy5YOWxWn0hMMb8oL79IjIXuBGYYpwX12pIB6p/M4pyvmZprga+R7rzv5ki8j6O0//LKgTNkMuSzwvqzyYiZ0WkizEmw3kKnFnHe1z8zE6JyNc4vk01ZyFoyM9/sU2aiPgAIUB2M2ZoUi5jTPUMS3D0u7gCy/5NXY7qv3yNMetE5B8iEmaMsXwyOhHxxVEEVhhj1tTSpFk/M7e+NCQiU4H/BqYZY4rqaLYT6C0iPUTED0fnniWjTRpDRIJEpO3FbRwd37WObmhhdn1eHwH3OrfvBb539iIiHUTE37kdBowDDjdzjob8/NWz3gZsqONLSIvmqnENeRqOa8+u4CNgjnMkzGggr9plQNuISOeLfTsiMhLH70urCzrOY74KHDHGPFdHs+b9zFq6R7wlH0ACjutoe52PiyM5ugLrqrW7HkfP/Ekcl0isznULjmt6pcBZ4POauXCM/tjnfBxylVx2fF7OY3YEvgROAP8GQp2vxwFLnNtjgQPOz+wAMN+iLN/7+YFFOL5wAAQA7zr//e0AerbQZ3SpXH90/lvaB3wF9GuhXCuBDKDc+e9rPrAQWOjcL8DfnbkPUM9IuhbO9Ui1z2sbMLaFco3H0T+4v9rvruut/Mx0igmllPJwbn1pSCml1KVpIVBKKQ+nhUAppTycFgKllPJwWgiUUsrDaSFQSikPp4VAKaU8nBYCpS6TiFzpnMgtwHlH+CERGWh3LqUaSm8oU6oZiMgfcNxR3AZIM8b80eZISjWYFgKlmoFzfp+dQAmOqQgqbY6kVIPppSGlmkdHIBjHilIBNmdRqlH0jECpZiCO9ZHfBnoAXYwxj9gcSakGa/XrEShlNxGZA5QbY94SEW9gi4hMNsZssDubUg2hZwRKKeXhtI9AKaU8nBYCpZTycFoIlFLKw2khUEopD6eFQCmlPJwWAqWU8nBaCJRSysP9P8QfuSZ+rjEqAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rrvb8INMbEIy"
      },
      "source": [
        "So if our parameter is `-1.5` we get a loss = `2.25`. Since the direction of our slope is downward (negative), by changing its value to be a bit more positive, we get closer to achieving our goal of *minimizing our loss*"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1Zb4dLlMcJMD",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        },
        "outputId": "cdfc5b45-dbd6-448f-b1f8-1076b7db3dfc"
      },
      "source": [
        "xt = tensor(-1.).requires_grad_(); xt\n",
        "\n",
        "loss = f(xt)\n",
        "\n",
        "plot_function(f, 'x', 'x**2')\n",
        "plt.scatter(xt.detach().numpy(), loss.detach().numpy(), color='red')\n",
        "print('Loss: ', loss.item())"
      ],
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Loss:  1.0\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1f3/8dcnO0kgEJKwZAXCIvsS9kWEanEDRXEXKSjiUrXbt1pba2ntZmvr0tYiooiKKKJ1wYWKCrKHfYdAyEYgISEh+3p+f8zgL41JSEJu7mTm83w85uGduYe574yQz9x7zj1HjDEopZTyXF52B1BKKWUvLQRKKeXhtBAopZSH00KglFIeTguBUkp5OB+7AzRVWFiYiYuLszuGUkq1Kdu3bz9jjAmva1+bKwRxcXEkJibaHUMppdoUEUmpb59eGlJKKQ+nhUAppTycFgKllPJwWgiUUsrDaSFQSikPZ3khEBFvEdkpIh/Vsc9fRFaISJKIbBGROKvzKKWU+l+tcUbwMHCwnn3zgLPGmHjgb8CfWiGPUkqpGiwtBCISBVwNLK6nyQxgqXN7JTBVRMSKLElZhSz88ADlldVWvL1SSlnq2f8eZcvxHEve2+ozgr8D/wfU99s3EkgDMMZUAvlA59qNRGS+iCSKSGJ2dnazgqTlFrNkQzJrD51u1p9XSim7pOYU87f/HmFLcq4l729ZIRCRa4AsY8z2i30vY8wiY0yCMSYhPLzOO6QvaFKfcLp2COCtbWkXG0cppVrV24lpeAncOCLKkve38oxgPDBdRE4AbwFTROT1Wm0ygGgAEfEBQgBLzn28vYRZCVGsO5LNybwSKw6hlFItrrKqmpXb05nUJ5zuHdtZcgzLCoEx5jFjTJQxJg64BVhrjLmjVrMPgLuc2zc621i2duasEdFUG1i5Pd2qQyilVItadzSbU+dKuWVktGXHaPX7CERkoYhMdz59GegsIknAj4FHrTx2TOdAxsd35u3ENKqrda1mpZTrW7Etjc5Bfkzp18WyY7RKITDGfGWMuca5/YQx5gPndqkxZpYxJt4YM8oYc9zqLDclRJN+toSNx6zpfVdKqZaSVVDKFwezuGFEFH4+1v269rg7i78/oCsh7Xx5a1uq3VGUUqpBq3ZkUFltuCnBustC4IGFIMDXm+uHRfL5/tOcLSq3O45SStXJGMPb29JIiO1EfESwpcfyuEIAcPPIaMqrqlm1M8PuKEopVaetybkcP1PETRZ2Ep/nkYXgkm4dGBrdkeVbU7FwkJJSSjXb8q2ptA/w4drB3S0/lkcWAoBbR0WTlFXI9pSzdkdRSqn/kVdczup9p7huaCTt/LwtP57HFoJrBncn2N+HN7dqp7FSyrWs2pFBeWU1t46KaZXjeWwhCPL3YcbQ7ny8J5P84gq74yilFODoJH5rWypDojvSv3uHVjmmxxYCgFtHxVBWWc37u7TTWCnlGnaknuXI6UJubYVO4vM8uhAMjAxhUGSIdhorpVzG8q1pBPl5c+0Q6zuJz/PoQgCOs4JDpwrYmZZndxSllIfLL6ngoz0nmTEskiB/n1Y7rscXgulDuxPk583yLdpprJSy1/s7MyitqObWka3TSXyexxeCYH8fZgyL5MM9J7XTWCllG2MMb2xJYUhUCIOiQlr12B5fCABuGxVDaUU1q3bq9NRKKXskpjg6iW8b3bpnA6CFAHB0Gg+N7sgbW7TTWClljzc2p9De36dVO4nP00LgdNvoGJKyCtlq0ZqgSilVn9yiclbvPcXM4ZEE+rVeJ/F5Wgicrh3cnfYBPryhncZKqVa2cnsa5VXV3DY61pbjW7l4fYCIbBWR3SKyX0R+U0ebOSKSLSK7nI+7rcpzIe38vLlheBSf7jtFTmGZXTGUUh6mutqwfKtjuum+XdvbksHKM4IyYIoxZggwFJgmImPqaLfCGDPU+VhsYZ4Lun10DOVV1bydqJ3GSqnWsfFYDslnimzpJD7PysXrjTGm0PnU1/lw6Z7Y3l3aM7pHKG9uTaFK1zRWSrWCZZtP0CnQl6sGdbMtg6V9BCLiLSK7gCxgjTFmSx3NbhCRPSKyUkTqnFxDROaLSKKIJGZnZ1sZmTvHxpKWW8K6I9YeRymlMvNLWHPgNDeNjCbA1/rpputjaSEwxlQZY4YCUcAoERlYq8mHQJwxZjCwBlhaz/ssMsYkGGMSwsPDrYzMFf27Et7en2WbUyw9jlJKLd+SigHusKmT+LxWGTVkjMkDvgSm1Xo9xxhzvmd2MTCiNfI0xM/Hi1tHRvPl4SzScovtjqOUclPlldUs35bGZX0jiA4NtDWLlaOGwkWko3O7HXA5cKhWm5oXxaYDB63K0xS3jo7BS0SHkiqlLPP5gVNkF5Rx5xh7zwbA2jOCbsCXIrIH2Iajj+AjEVkoItOdbR5yDi3dDTwEzLEwT6N1C2nH9y6J4O3ENEorquyOo5RyQ8s2pRAd2o5Jfay93N0Ylt3CZozZAwyr4/Unamw/BjxmVYaLceeYOD7bf5pP9mVy/bAou+MopdzIkdMFbEnO5dEr++HtJXbH0TuL6zOuV2d6hgexdKN2GiulWtZrm07g5+PFTQmttwpZQ7QQ1MPLS5g9JpZdaXns1kVrlFIt5FxpBat2ZDB9SHdCg/zsjgNoIWjQDSOiCPLzZummE3ZHUUq5iZWJ6RSXVzFnXJzdUb6lhaAB7QN8uWFEFB/tztT5h5RSF6262rBscwrDYzoyMLJ1F59piBaCC5g9Npbyqmre2pZmdxSlVBu37mg2yWeKuMuFzgZAC8EFxUe0Z0J8GK9vTqGyqtruOEqpNuy1TSmEBftz5UD75hWqixaCRpg9NpbM/FLWHDhtdxSlVBuVklPEl4ezuG10DH4+rvWr17XSuKipl3QhqlM7Xtl4wu4oSqk2aunGFLxFuN3G6abro4WgEby9hNljY9manMv+k/l2x1FKtTGFZZW8k5jGVYO60aVDgN1xvkMLQSPdnBBDO19vXt1wwu4oSqk25t3t6RSUVfKD8XF2R6mTFoJGCgn05YYRkfxn90kdSqqUarTqasOrG08wNLojw2I62R2nTloImmDOuDjKK6t5U2clVUo10tdHHENGXfVsALQQNEl8RHsm9g5j2eYUyit1KKlS6sKWbEgmor3rDRmtSQtBE80d34OsgjI+2ZdpdxSllItLyipg/dEz3Dkm1uWGjNbkuslc1KV9wukZFsSSb5IxRhe4V0rV75UNjllGb3XBIaM1aSFoIi8v4Qfj49idns/2lLN2x1FKuaizReW8uyOd64dGEhbsb3ecBlm5VGWAiGwVkd3OVch+U0cbfxFZISJJIrJFROKsytOSbhgRRUg7XxavT7Y7ilLKRb25NZXSimrmTexhd5QLsvKMoAyYYowZAgwFponImFpt5gFnjTHxwN+AP1mYp8UE+vlw2+gYPj9witQcXeBeKfW/yiurWbrxBBN7h9GnS3u741yQZYXAOBQ6n/o6H7Uvqs8Aljq3VwJTRcT+ddsa4a6xcXiJ8MpGPStQSv2vj/acJKugjLsn9rQ7SqNY2kcgIt4isgvIwrF4/ZZaTSKBNABjTCWQD3Su433mi0iiiCRmZ2dbGbnRuoYEcM3gbry9LY1zpRV2x1FKuQhjDIvXJ9M7IphJvcPsjtMolhYCY0yVMWYoEAWMEpGBzXyfRcaYBGNMQnh4eMuGvAjzJvSkqLyKFVt1rQKllMPm47kcyDzH3Ak9aCMXOFpn1JAxJg/4EphWa1cGEA0gIj5ACJDTGplawqCoEEb1COXVjSeo0LUKlFLA4vXHCQ3y4/phkXZHaTQrRw2Fi0hH53Y74HLgUK1mHwB3ObdvBNaaNjY4f/7EnmTklbB6r95gppSnS8oq4ItDWcweG0uAr7fdcRrNyjOCbsCXIrIH2Iajj+AjEVkoItOdbV4GOotIEvBj4FEL81hiSr8IeoUHsWjdcb3BTCkPt3h9Mv4+Xtw5JtbuKE3iY9UbG2P2AMPqeP2JGtulwCyrMrQGLy/hnok9eXTVXjYdy2FcfNvoHFJKtaysglJW7cjgppFRdHbxG8hq0zuLW8B1wyIJC/Zj0frjdkdRStnktY0pVFRXM29C2xgyWpMWghYQ4OvNXWPj+OpwNodPFdgdRynVyorLK1m2OYUr+nehR1iQ3XGaTAtBC7ljTCwBvl68pGcFSnmcdxLTyS+pYP6ktnc2AFoIWkynID9uTojmP7syyMwvsTuOUqqVVFZV89L644yI7cSI2FC74zSLFoIWdPfEnlQbWPKNTjuhlKf4eG8m6WdLWHBpL7ujNJsWghYUHRrINYO78eaWVPKLddoJpdydMYYXvz5OfEQwU/tF2B2n2bQQtLB7J/WiqLyK17ek2B1FKWWxr49kczDzHPMn9cTLq21MJ1EXLQQtrH/3DlzaJ5xXNiRTWlFldxyllIVe/PoYXTsEcN3QtjOdRF20EFhgwaW9OFNYzsrt6XZHUUpZZFdaHpuP5zJvQg+XXo+4Mdp2ehc1pmcoQ6I7smjdcSp1Mjql3NKLXx2jfYCPy69H3BhaCCwgItw/uRepucV8rJPRKeV2krIK+HT/KeaMiyPY37KZelqNFgKLXH5JF3pHBPOvr47pZHRKuZl/fXWcdr7e/GC8669H3BhaCCzi5SXcf1kvDp0qYO2hLLvjKKVaSFpuMe/vyuDWUTGEBvnZHadFaCGw0LWDuxPVqR0vfJmkZwVKuYmX1h/HS+CeSe5xNgBaCCzl4+3FvZf2YmeqY3SBUqptyyoo5a1tacwcFkW3kHZ2x2kxVq5QFi0iX4rIARHZLyIP19Fmsojki8gu5+OJut6rLZs1IoqwYH/+8WWS3VGUUhdpyTcnqKyqZsHktjudRF2sPCOoBH5ijOkPjAEeEJH+dbRbb4wZ6nwstDCPLQJ8vZk/qQffJJ1hZ+pZu+MopZrpbFE5yzad4KpB3drkVNMNsawQGGMyjTE7nNsFwEGgbd9+10y3j46lU6Avz6/VswKl2qpXNiRTVF7Fg1Pi7Y7S4lqlj0BE4nAsW7mljt1jRWS3iHwiIgPq+fPzRSRRRBKzs7MtTGqNIH8f5k3owdpDWezLyLc7jlKqic6VVvDKxhNMG9CVfl072B2nxVleCEQkGHgXeMQYc67W7h1ArDFmCPA88H5d72GMWWSMSTDGJISHh1sb2CKzx8XRIcCH59cetTuKUqqJlm44QUFppVueDYDFhUBEfHEUgTeMMatq7zfGnDPGFDq3VwO+IuKWq793CPBlzvgefLb/NAcza9dDpZSrKiyr5OUNyUztF8HAyBC741jCylFDArwMHDTGPFNPm67OdojIKGeeHKsy2W3u+DiC/Lx5QUcQKdVmvL45hbziCn44tbfdUSxj5SQZ44E7gb0issv52i+AGABjzIvAjcB9IlIJlAC3GDe+86pjoB+zx8Xx4tfHOHq6gN5d2tsdSSnVgOLySl5ad5yJvcMYGt3R7jiWsawQGGO+ARpcqcEY8wLwglUZXNE9E3uydOMJnv3iKC/cNtzuOEqpBizblEJOUTmPfM99zwZA7yxudaFBftw1Lo6P92Zy5HSB3XGUUvUoLq/k386zgba6KH1jaSGwwT0TexLo681zX+gIIqVc1WubUsgtKueR7/WxO4rltBDYQM8KlHJtRWWVLFp3nEl9whkR28nuOJbTQmCT82cFz+pZgVIu5/+fDbh338B5Wghs0inIjznj41i9N5NDp/S+AqVcRWFZJYvWHWNSn3CGx7j/2QBoIbDVPRN7Euznw9/WHLE7ilLK6ZVvkjlbXMFPLnf/voHztBDYqGOgH/MmOu423puucxApZbf84goWrT/O9y7pwhA3vm+gNi0ENps7oQcdA33565rDdkdRyuO9tP44BaWV/NiDzgZAC4HtOgT4cu+kXnx1OJvtKbqKmVJ2ySksY8mGZK4e3I3+3d1vhtGGaCFwAXeNiyUs2I+/fq59BUrZ5d/rjlNaUcWPPGSkUE1aCFxAoJ8P90+OZ+OxHDYknbE7jlIe51R+KUs3nuC6oZHER3jeHGANFgIR6SAi31mcU0QGWxfJM902OobuIQH8+dNDuPG8e0q5pOfWHqXaGH7kYX0D59VbCETkJuAQ8K5z8fmRNXa/anUwTxPg680jl/dhd3o+n+0/ZXccpTxG8pkiVmxL47ZRMUSHBtodxxYNnRH8AhhhjBkK/ABYJiLXO/c1OKuoap6ZwyLpFR7EXz4/QmVVtd1xlPIIz6w5gp+3Fw9O8by+gfMaKgTexphMAGPMVuAy4Jci8hCg1y4s4OPtxU+v6EtSViGrdmbYHUcpt7cvI58Pd59k7oQ4wtv72x3HNg0VgoKa/QPOojAZmAHUuci8unjTBnZlcFQIf19zhNKKKrvjKOXW/vL5YULa+TJ/0ne6Qj1KQ4XgPmpdAjLGFADTgLkXemMRiRaRL0XkgLOP4eE62oiIPCciSSKyR0Q8fqUWEeHn0/pxMr+U1zen2B1HKbe16VgOXx3O5r7JvQhp52t3HFvVWwiMMbuBOAARmVrj9QpjzBuNeO9K4CfGmP7AGOABEelfq82VQG/nYz7wryald1Pj48OY2DuM59cmkV9cYXccpdxOdbXhD58cpFtIAHPGxdkdx3YXuo/gUhEZj+OSUJMYYzKNMTuc2wXAQSCyVrMZwGvGYTPQUUS6NfVY7ujRK/txrrSCf36tC90r1dI+3pvJnvR8fnJFXwJ8ve2OY7uGho/+GvAH/gv4icgTzT2IiMQBw4AttXZFAmk1nqfz3WKBiMwXkUQRSczOzm5ujDZlQPcQrh8aySsbTnAyr8TuOEq5jfLKap7+7DD9urbn+mHf+XXjkRq6NPQbHPcRPAkcMsYsbM4BRCQYeBd4xBjTrIn3jTGLjDEJxpiE8PDw5rxFm/TjKxw3tzyj01Qr1WLe3JJCam4xP7+yH95eOhIeLnxpqL0x5k9As+65FhFfHEXgDWPMqjqaZADRNZ5HOV9TQFSnQOaMi+PdHekczNTFa5S6WOdKK3hubRJje3Zmch/P+VJ5IRcqBPtr/bfRRESAl4GDxphn6mn2ATDbOXpoDJB//t4F5fDA5HhC2vny1McHdeoJpS7SP788Rm5ROb+46hIcv6IUWNhZDIwH7gSmiMgu5+MqEVkgIgucbVYDx4Ek4CXg/mYcx62FBPry0JTefJN0hq8Oe0b/iFJWSMstZsmGZGYOi2RQVIjdcVyKT307anUWPyciTzSln8AY8w0XmIrCOL7iPtDY9/RUd4yJZdnmFJ5afZCJvcPw8dZJY5Vqqj9/dhgvgZ9+v6/dUVyO5Z3F6uL5+Xjx6JX9SMoqZPm2tAv/AaXU/9iZepYPd59k/sSedO/Yzu44LudCXy07AB8CwTVfFJHJVgVSdbuifxdG9Qjl72uOcK5UbzJTqrGMMfzu44OEt/fn3ks9eyqJ+jRYCIwxfwfeBgKdHbrtROR54A+tkk59S0T41dX9ySkq5x9r9SYzpRrroz2ZbE85y08u70OQf71Xwz1aYy42j8YxxHMjsA04iaMjWLWyQVEhzBoRxZINySSfKbI7jlIur6S8ij9+cogB3TswKyH6wn/AQzWmEFQAJUA7IABINsboZPk2+dm0vvh5e/HUxwftjqKUy1u07jgZeSX8+toBevNYAxpTCLbhKAQjgYnArSLyjqWpVL0i2gfw4JTe/PfgadYf1eGkStXnZF4J//o6iasHd2NUj1C747i0xhSCecaYJ5yzjmYaY2bguBFM2WTuhDhiOwey8MMDupKZUvX44yeHMAYeu7Kf3VFc3gULgTEmsY7XllkTRzWGv483j191CUezClmmaxYo9R3bTuTywe6T3HtpL6I6eeY6xE2hdya1UZf378LE3mE8s+YIZwrL7I6jlMuorKrmV+/vo3tIAAsu7Wl3nDZBC0EbJSI8OX0ApRVV/OmTQ3bHUcplvLEllUOnCvjVNf0J9NPhoo2hhaAN6xUezNwJPXhnezo7Us/aHUcp250pLOOvnx9mQnwY0wZ2tTtOm6GFoI17aEpvunTw59f/2U9Vtc5Oqjzb058epri8iien99fZRZtAC0EbF+Tvw+NX92dvRj7Lt6baHUcp2+xMPcuKxDTmTehBfESzllDxWFoI3MC1g7sxrldn/vzpIe04Vh6psqqax9/bR9cOAfxwam+747Q5WgjcgIiwcMZASiqq+L3ecaw80NJNKRzIPMevr+1PsM4n1GRaCNxEfEQw907qxaqdGWw8dsbuOEq1mlP5pTzz+WEm9w3XDuJmsqwQiMgSEckSkX317J8sIvk1Vi97wqosnuLBKfHEhAbyq/f3UV6pdxwrz/Dbjw5QWW1YOH2gdhA3k5VnBK8C0y7QZr0xZqjzoQvfXKQAX29+M2MAx7KLWLTumN1xlLLcV4ez+HhvJj+cEk9MZ72DuLksKwTGmHVArlXvr+p2Wd8Irh7UjefWJnE8u9DuOEpZpri8kl++v4/4iGDumaR3EF8Mu/sIxorIbhH5REQG1NdIROaLSKKIJGZn64ybF/Lra/vj7+PFL97bi2NZaKXczzOfHyH9bAl/mDkIfx9vu+O0aXYWgh1ArDFmCPA88H59DY0xi4wxCcaYhPDw8FYL2FZFdAjgF1ddwubjubyTmG53HKVa3N70fJZsSOa20TGMjNMppi+WbYXAGHPOGFPo3F4N+IpImF153M3NCdGMigvlqdUHyS7QewuU+6isqubRVXsIC/bn59N0iumWYFshEJGu4uziF5FRziw5duVxN15ewu9nDqKkvIonP9xvdxylWszib5LZf/Icv5k+gJB2vnbHcQtWDh9dDmwC+opIuojME5EFIrLA2eRGYJ+I7AaeA24xekG7RcVHBPPQ1Hg+3pPJp/tO2R1HqYt2LLuQZ9Yc4Yr+XfSegRZk2S14xphbL7D/BeAFq46vHO69tBer957iV//Zx5ieoXQM9LM7klLNUlVt+L+Ve2jn683vrtN7BlqS3aOGlMV8vb14etZgzhaVs/CjA3bHUarZXtt0gu0pZ3nimv5EdAiwO45b0ULgAQZ0D+G+yb1YtSODLw9l2R1HqSZLzSnmz586ppGYOTzS7jhuRwuBh3hwSjy9I4J5bNVe8osr7I6jVKNVVxt+tnI33l7C768fpJeELKCFwEP4+3jz15uGkF1YpqOIVJvyysYTbEnO5Ylr+9O9Yzu747glLQQeZHBURx64LJ73dmbw6b5Mu+ModUFJWYX8+dNDTO0XwawRUXbHcVtaCDzMg5fFM6B7Bx5/b58uYqNcWmVVNT95Zzft/Lz5w0y9JGQlLQQexs/Hi2duGkpBaSWP61xEyoX966tj7E7L43fXDdRRQhbTQuCB+nZtz0+u6MNn+0/rXETKJe1Oy+PZL45y7ZDuXDO4u91x3J4WAg9198SejOkZypMf7iclp8juOEp9q7i8kkdW7CKivT+/mzHQ7jgeQQuBh/L2Ep65aSg+XsIjK3ZRWaUrminX8NuPDnIip4i/3jSUkECdS6g1aCHwYN07tuOp6wexMzWP59cm2R1HKdYcOM3yranMn9STsb062x3HY2gh8HDXDunOzGGRPL/2KFuTdUE5ZZ9T+aX838rd9O/WgR9f3sfuOB5FC4Fi4XUDiQkN5OG3dpJXXG53HOWBqqoND7+1k7LKap6/bZiuONbKtBAogv19eP7W4ZwpLONnK/fokFLV6l5Ym8SW5FwWzhhIr/Bgu+N4HC0ECoBBUSH8fFo/1hw4zbLNKXbHUR5ka3Iuz35xhOuHRXKDTihnCysXplkiIlkisq+e/SIiz4lIkojsEZHhVmVRjTNvQg+m9Ivgdx8dZG96vt1xlAfIKSzjoeU7iQkN5Le6xoBtrDwjeBWY1sD+K4Hezsd84F8WZlGNICL8ZdYQOgf7cf+/vya/9yXg5QVxcfDGG3bHU26mqtrwyIpd5BaX84/bhxPsb9k6WeoCLCsExph1QEPDUGYArxmHzUBHEelmVR7VOKFBfrzQOZvMUsNPB93g6C9ISYH587UYqBb1/NqjrD96ht9MH8CA7iF2x/FodvYRRAJpNZ6nO19TNhvx+0d57KslrOkzlpdGXe94sbgYHn/c3mDKbXxz9AzPfnGUmcMiuWVktN1xPF6b6CwWkfkikigiidnZ2XbHcX+pqcxN/IArD2/gT5fOYVP0oG9fV+piZeSV8NBbO4kPD+Z312u/gCuwsxBkADW/CkQ5X/sOY8wiY0yCMSYhPDy8VcJ5tJgYBPjz6r8Td/YkD874OSfbh0FMjN3JVBtXWlHFgmXbqais5sU7RxDop/0CrsDOQvABMNs5emgMkG+M0dVSXMFTT0FgIO3LS/j3qqco8/Hjvht+Selvn7I7mWrDjDH88v197M3I55mbh+r9Ai7EyuGjy4FNQF8RSReReSKyQEQWOJusBo4DScBLwP1WZVFNdPvtsGgRxMYSfzaDv25dxu4u8TzRbqDebKaa7fXNKazcns5DU3tzef8udsdRNUhb+4edkJBgEhMT7Y7hcf76+WGeX5vEk9f2Z874HnbHUW3MpmM53PnyFib1CWfx7AS8vLRfoLWJyHZjTEJd+9pEZ7Gy34++14fL+3dh4UcHWHdEO+xV46XmFHPfG9uJCwvi77cM1SLggrQQqEbx8hL+dvNQ+nRpzwNv7uBYdqHdkVQbUFBawbyl2wBYPDuBDgG6voAr0kKgGi3Y34eXZifg5+3FPUsTdaZS1SDHjKK7SD5TxD9vH05cWJDdkVQ9tBCoJokODeTFO0eQfraEe5dtp6yyyu5IygUZY/jNh/tZeyiLJ6cPYFyvMLsjqQZoIVBNNjIulKdnDWZLci6PvrtXRxKp73j5m2Re25TC/Ek9uWNMrN1x1AXo3RyqWWYMjSQtt5i/fH6E6NBAXVFKfevTfad4avVBrhzYlUen9bM7jmoELQSq2R64LJ7U3GKe++IokR0DuHmk3nns6ban5PLwWzsZGt2Rv92sI4TaCi0EqtlEhKeuH0RWQRmPrdpLp0A/rhjQ1e5YyiZHThcw99VEundsx+LZCQT46nKTbYX2EaiL4uvtxT9vH86gqI78cPlOtiY3NPO4clcZeSXMfnkr/j5evDZ3FJ2D/e2OpJpAC9VHb4MAAA/jSURBVIG6aIF+PrwyZySRndoxb+k2DmaeszuSakU5hWXMfnkLRWWVLJ07iujQQLsjqSbSQqBaRGiQH6/NHUWwvw93vrxFbzjzEPklFcxespX0syUsviuBS7p1sDuSagYtBKrFRHUK5PW7RwNwx+ItpOUW25xIWamorJK5r27jyOkCXrxzBKN7drY7kmomLQSqRfUKD2bZvNEUl1dx++ItnMovtTuSskBpRRXzlyWyKy2P528dxmV9I+yOpC6CFgLV4i7p1oGlc0eRW1TOrS9t1mLgZkorqrjntUQ2Hsvh6RsHM22gLjXe1mkhUJYYGt2RpXNHkV1QpsXAjZwvAt8kneHPNwxm5vAouyOpFqCFQFlmRGynb4vBLYs2kZlfYnckdRFKyqu4e6mjCDx94xBmJeii8+7C0kIgItNE5LCIJInIo3XsnyMi2SKyy/m428o8qvWNiO3Ea/NGkVNYzqwXN5GSU2R3JNUMBaUV3LVkKxuOOYrAjSP0TMCdWLlUpTfwD+BKoD9wq4j0r6PpCmPMUOdjsVV5lH2Gx3TizXvGUFRWyawXN3H0dIHdkVQTnC0q5/bFW9iRepbnbhmmRcANWXlGMApIMsYcN8aUA28BMyw8nnJhg6JCWHHvWABu+vcm9qTn2ZxINcbpc6XcvGgTh04V8O87R3DtkO52R1IWsLIQRAJpNZ6nO1+r7QYR2SMiK0WkzouOIjJfRBJFJDE7W5dJbKv6dGnPOwvGEuTvwy2LNvPV4Sy7I6kGJGUVMPOfG0k/W8Krc0Yy9RJdcN5d2d1Z/CEQZ4wZDKwBltbVyBizyBiTYIxJCA8Pb9WAqmXFdg5i1X3jiOscxN1LE1m5Pd3uSKoOiSdyueFfmyirrGbF/LGMi9eFZdyZlYUgA6j5DT/K+dq3jDE5xpgy59PFwAgL8ygXEdEhgBX3jmFMz8789J3dPPfFUV3cxoV8sjeT2xdvITTIj1X3jWNQVIjdkZTFrCwE24DeItJDRPyAW4APajYQkZp3okwHDlqYR7mQ9gG+LJkzkpnDInlmzREeWbGL0gpd9tJOxhheWHuU+97YwYDuHXj3vnHEdNYJ5DyBZesRGGMqReRB4DPAG1hijNkvIguBRGPMB8BDIjIdqARygTlW5VGux8/Hi7/eNIReEcE8/dlhUnOLWXRnAuHtdQrj1lZaUcVjq/by3s4MrhvanT/eMFjXE/Ag0tZOyRMSEkxiYqLdMVQL+2RvJj96exedAv345+3DGRbTye5IHuNkXgn3vb6d3en5/PSKPjxwWTwiurKYuxGR7caYhLr22d1ZrBQAVw7qxsoF4/D2Em7+92aWb021O5JH2HjsDNc+/w3Hsot48Y4RPDiltxYBD6SFQLmMgZEhfPjgBEb3DOWxVXv52Tu7KS6vtDuWW6quNvzrq2PcsXgLHQN9ef+B8UwbqMuMeipds1i5lE5Bfrz6g1H8bc0R/vFVEjvT8njhtmH066oLnrSU7IIyfvz2LtYfPcPVg7rxpxsHE+yvvwo8mZ4RKJfj7SX89Pt9WTZ3NHnFFcx4YQPLNqfoENMWsO5INlc+u56tybn8/vpBvHDbMC0CSguBcl0TeofxycMTGd2zM796fx93vbJNp7NupuLySn75/l5mL9lKp0Bf/vPgeG4bHaP9AQrQQqBcXHh7f16dM5LfzhjAtuRcrvjb17y3M13PDppg24lcrnx2PW9sSeXuCT348IcT9FKb+h9aCJTL8/IS7hwbx+qHJ9K7S3t+tGI3d72yjdQcXRO5IfnFFTy2ai+zXtxEVbVh+T1j+OU1/fX+APUdeh+BalOqqg3LNp3g6c8OU2UMD0/tw7wJPfDz0e805xlj+HBPJgs/PEBuURnzJvTgR5f3IdBP+wI8WUP3EWghUG3SybwSfv3BftYcOE2PsCAev+oSpl4S4fHXvPem57Pwo/1sO3GWQZEh/GHmIAZG6lxBSguBcmNfHs7itx8d4Hh2ERN7h/Hzaf088hdfRl4Jf19zhJU70gkN9OOn3+/LTQnReHt5dmFU/58WAuXWKqqqeW1TCs99cZT8kgquHtSNH1/Rh17hwXZHs9yZwjL++eUxXt+cAsDssbE89L3edAjwtTmZcjVaCJRHOFdaweJ1x1n8TTKlFVVcPbg7Cy7tyYDu7neGcDKvhJfWH+etrWmUVVZx44goHv5eHyI7trM7mnJRWgiURzlTWMZL64/zxuZUCssqmdw3nLnjezAhPgyvNn6pZF9GPq9uPMF/dmVQbWDG0O7cPzme+Aj3P/tRF0cLgfJI+cUVLNt8glc3nuBMYTk9woK4Y0wsM4dF0inIz+54jVZSXsVn+0/x2qYT7EjNo52vNzclRHHPpJ5EddL1AlTjaCFQHq2ssopP9p5i6aYT7EzNw9dbuKxvBDOHRzK5b4RLjquvqjZsTc7lvZ3prN57isKySuI6B3Ln2DhuHBFFSDvtA1BN01Ah0IHFyu35+3hz3bBIrhsWyYGT51i1I533d53k8wOnCfTzZnLfcK7o35VJfcIJtfFMobi8ks3Hc/hs32n+e/A0OUXlBPl5c+WgbswcFsmYnp3b/KUt5ZosPSMQkWnAszhWKFtsjPljrf3+wGs41irOAW42xpxo6D31jEC1hMqqajYey+Gz/af4/MBpsgscS2f379aBCb3DSIjtxJDojnTpEGBZhvziCvZk5LEjJY8Nx86wM/UsFVWGYH8fLusXwRX9uzD1kgi9EUy1CFsuDYmIN3AEuBxIx7GG8a3GmAM12twPDDbGLBCRW4DrjTE3N/S+WghUS6uuNuxOz2ND0hk2JOWwPeUs5VXVAHTtEEC/bu2JDw+mV0Qw0Z0C6RriT5cOAbRvxBDN0ooqTp8r5VR+KRl5JRzLLuRYVhGHTxeQfKYIABEY2D2EcfGdGd8rjNE9Q/H3cb3LVapts6sQjAWeNMZ83/n8MQBjzB9qtPnM2WaTiPgAp4Bw00AoLQTKaqUVVRzIPMeu1Dx2p+dx5HQhx7MLKaus/p92ft5eBAf4EOTvjb+PN+cv2lRUVVNYVkVhWQWlFf/7Z3y8hNjOgcRHBDM4qiNDozsyKCpEx/0ry9nVRxAJpNV4ng6Mrq+Nc7H7fKAzcKZmIxGZD8wHiImJsSqvUgAE+HozPKYTw2usm1xdbcjIKyEjr+Tbb/hniysoLKugqKyKssqqb9v6eHkR5O9D+wAfOgT40KVDAF1DAugW0o7YzoH4euu8SMq1tImLj8aYRcAicJwR2BxHeSAvLyE6NJDoUB2uqdyPlV9NMoDoGs+jnK/V2cZ5aSgER6exUkqpVmJlIdgG9BaRHiLiB9wCfFCrzQfAXc7tG4G1DfUPKKWUanmWXRpyXvN/EPgMx/DRJcaY/SKyEEg0xnwAvAwsE5EkIBdHsVBKKdWKLO0jMMasBlbXeu2JGtulwCwrMyillGqYDl9QSikPp4VAKaU8nBYCpZTycFoIlFLKw7W5aahFJBtIaeYfD6PWXcsuwlVzgetm01xNo7maxh1zxRpjwuva0eYKwcUQkcT65tqwk6vmAtfNprmaRnM1jafl0ktDSinl4bQQKKWUh/O0QrDI7gD1cNVc4LrZNFfTaK6m8ahcHtVHoJRS6rs87YxAKaVULVoIlFLKw7l1IRCRp0XkkIjsEZH3RKRjPe2michhEUkSkUdbIdcsEdkvItUiUu9QMBE5ISJ7RWSXiFi+PmcTcrXq5+U8ZqiIrBGRo87/dqqnXZXz89olIrWnPW+pLA3+/CLiLyIrnPu3iEicFTmakWuOiGTX+HzubqVcS0QkS0T21bNfROQ5Z+49IjLcRXJNFpH8Gp/XE3W1syBXtIh8KSIHnP8eH66jTct+ZsYYt30AVwA+zu0/AX+qo403cAzoCfgBu4H+Fue6BOgLfAUkNNDuBBDWip/XBXPZ8Xk5j/tn4FHn9qN1/b907iu0OMcFf37gfuBF5/YtwIpW+Hwak2sO8EJr/X2qcdxJwHBgXz37rwI+AQQYA2xxkVyTgY9s+Ly6AcOd2+2BI3X8v2zRz8ytzwiMMZ8bYyqdTzfjWCWttlFAkjHmuDGmHHgLmGFxroPGmMNWHqM5Gpmr1T8vpxnAUuf2UuC6VjhmXRrz89fMuhKYKiKCtez6/3JBxph1ONYbqc8M4DXjsBnoKCLdXCCXLYwxmcaYHc7tAuAgjvXda2rRz8ytC0Etc3FU0NoigbQaz9P57oduFwN8LiLbRWS+3WGc7Pq8uhhjMp3bp4Au9bQLEJFEEdksIlYUi8b8/N+2cX4RyQc6W5ClqbkAbnBeSlgpItF17LeDK/8bHCsiu0XkExEZ0NoHd15WHAZsqbWrRT+zNrF4fUNE5L9A1zp2PW6M+Y+zzeNAJfCGK+VqhAnGmAwRiQDWiMgh57cYu3NZoqFsNZ8YY4yI1DfuOdb5mfUE1orIXmPMsZbO2kZ9CCw3xpSJyL04zlqm2JzJle3A8fepUESuAt4HerfWwUUkGHgXeMQYc87KY7X5QmCM+V5D+0VkDnANMNU4L67VkgHU/GYU5XzN0lyNfI8M53+zROQ9HKf/F1UIWiCXJZ8XNJxNRE6LSDdjTKbzFDirnvc4/5kdF5GvcHybaslC0Jif/3ybdBHxAUKAnBbM0KxcxpiaGRbj6HdxBZb9nboYNX/5GmNWi8g/RSTMGGP5ZHQi4oujCLxhjFlVR5MW/czc+tKQiEwD/g+YbowprqfZNqC3iPQQET8cnXuWjDZpChEJEpH257dxdHzXObqhldn1eX0A3OXcvgv4ztmLiHQSEX/ndhgwHjjQwjka8/PXzHojsLaeLyGtmqvWNeTpOK49u4IPgNnOkTBjgPwalwFtIyJdz/ftiMgoHL8vrS7oOI/5MnDQGPNMPc1a9jNr7R7x1nwASTiuo+1yPs6P5OgOrK7R7iocPfPHcFwisTrX9Tiu6ZUBp4HPaufCMfpjt/Ox31Vy2fF5OY/ZGfgCOAr8Fwh1vp4ALHZujwP2Oj+zvcA8i7J85+cHFuL4wgEQALzj/Pu3FejZSp/RhXL9wfl3aTfwJdCvlXItBzKBCuffr3nAAmCBc78A/3Dm3ksDI+laOdeDNT6vzcC4Vso1AUf/4J4av7uusvIz0ykmlFLKw7n1pSGllFIXpoVAKaU8nBYCpZTycFoIlFLKw2khUEopD6eFQCmlPJwWAqWU8nBaCJS6SCIy0jmRW4DzjvD9IjLQ7lxKNZbeUKZUCxCR3+G4o7gdkG6M+YPNkZRqNC0ESrUA5/w+24BSHFMRVNkcSalG00tDSrWMzkAwjhWlAmzOolST6BmBUi1AHOsjvwX0ALoZYx60OZJSjdbm1yNQym4iMhuoMMa8KSLewEYRmWKMWWt3NqUaQ88IlFLKw2kfgVJKeTgtBEop5eG0ECillIfTQqCUUh5OC4FSSnk4LQRKKeXhtBAopZSH+3+7tqwzvws/sgAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5HPZrsmdcd9x"
      },
      "source": [
        "And yes, our loss has improved!  If the direction of our slope were upwards (positive), we would conversely want `x` to be smaller."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DGv43ojRdVkE"
      },
      "source": [
        "***BUT*** now ... imagine having to figure all this out for a million parameters.  Obviously, we wouldn't want to try doing this manually as we did before, and thanks to PyTorch, we don't have too :)\n",
        "\n",
        "Remember that by utilizing the `requires_grad_()` function, we have told PyTorch to keep track of how to compute the gradients based on the other calucations we perform, like running it through our loss function above.  Let's see what that looks like."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "l9YLALQ7dVR1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ac4db2fb-2de7-4ec1-c5b1-e967f1f9c14d"
      },
      "source": [
        "xt = tensor(-1.).requires_grad_(); \n",
        "print(xt)\n",
        "\n",
        "loss = f(xt)\n",
        "print(loss)"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor(-1., requires_grad=True)\n",
            "tensor(1., grad_fn=<PowBackward0>)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rIgLnDoqeXUY"
      },
      "source": [
        "That `<PowBackward0>` is the gradient function it will use to calculate the gradients when needed.  And when we need it, we call the `backward` method to do so."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NA1tgKFzeh5d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "912bbbb2-662f-4636-dbd1-ef3935c8f808"
      },
      "source": [
        "loss.backward()\n",
        "print(xt.grad)"
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor(-2.)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pyCreCbhe2EP"
      },
      "source": [
        "And the calcuated gradient is exactly what we expected given that to calculate the derivate of `x**2` is `2x` ... `2*-1 = -2`.\n",
        "\n",
        "Again, the gradient tells us ***the slope of our function***.  Here have a a negative/downward slope and so at the very least, we know what moving in that direction will get us closer to the minimum.\n",
        "\n",
        "The question is now, **How far do we move in that direction?**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "N1RePernMIPn"
      },
      "source": [
        "---\n",
        "### Step 5: Change all the weights based on the gradient using a \"Learning Rate\"\n",
        "\n",
        "The **learning rate** (or LR) is a number (usually a small number like 1e-3 or 0.1) that we multiply the gradient by to get a better parameter value.  For a given parameter/weight `w`, the calculation looks like this:\n",
        "\n",
        "` w -= w.grad * lr`\n",
        "\n",
        "Notice we take the negative of the grad * lr operation because we want to move in the opposite direction.\n",
        "\n",
        "> Important: We do this in a `with torch.no_grad()` so that we don't calculate the gradient for the gradient calculating operation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zpproXJdodrR",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9237bae3-589e-4748-f778-e19ca3460649"
      },
      "source": [
        "lr = 0.01\n",
        "\n",
        "with torch.no_grad():\n",
        " xt -= xt.grad * lr\n",
        "\n",
        " print('New value for xt: ', xt)\n",
        " print('New loss: ', f(xt))"
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "New value for xt:  tensor(-0.9800, requires_grad=True)\n",
            "New loss:  tensor(0.9604)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "etZcxIkWoerO"
      },
      "source": [
        "You can see the loss get smaller which is exactly what we want!  \"The magnitude of the gradient (i.e., the steepness of the slope) [tells] us how big a step to take.\"\n",
        "\n",
        "The above operation is also called the **optimization step**\n",
        "\n",
        "See pp.156-157 for examples of what using a too small or too large LR might look like when training.  This could help you troubleshoot things if yours looks wonky."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LTKvbUUenaom"
      },
      "source": [
        "---\n",
        "### Datasets & DataLoaders\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BN0EZ3T1nq-V"
      },
      "source": [
        "A **Dataset** contains tuples of independent and dependent variables"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-9WU7lMIns5B",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "23986db7-b9a7-4fe8-d071-bd266d826eb5"
      },
      "source": [
        "ds = L(enumerate(string.ascii_lowercase))\n",
        "ds"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(#26) [(0, 'a'),(1, 'b'),(2, 'c'),(3, 'd'),(4, 'e'),(5, 'f'),(6, 'g'),(7, 'h'),(8, 'i'),(9, 'j')...]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YmSRb0BRnyKY"
      },
      "source": [
        "A **DataLoader** receives a dataset and gives us back as many *mini-batches* are necessary based on the *batch size* we specify"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AE4FiJo0oArl",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "da2e2f86-cfd1-4e37-d1db-e17b4554e21b"
      },
      "source": [
        "dl = DataLoader(ds, bs=6, shuffle=True)\n",
        "list(dl)"
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(tensor([17,  5,  9, 22, 18, 21]), ('r', 'f', 'j', 'w', 's', 'v')),\n",
              " (tensor([15,  3, 23,  1,  0, 19]), ('p', 'd', 'x', 'b', 'a', 't')),\n",
              " (tensor([ 4, 10, 16, 25,  8,  2]), ('e', 'k', 'q', 'z', 'i', 'c')),\n",
              " (tensor([24, 14,  7, 20, 13, 12]), ('y', 'o', 'h', 'u', 'n', 'm')),\n",
              " (tensor([11,  6]), ('l', 'g'))]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h2uxHGE5IK0a"
      },
      "source": [
        "---\n",
        "### Measuring distances\n",
        "\n",
        "See pp.141-142.  There are two main ways to measure distances.\n",
        "\n",
        "**L1 norm** (or mean absolute difference): Take the mean of the absolute value of differences\n",
        "\n",
        "` l1_loss = (tensor_a - tensor_b).abs().mean()`\n",
        "\n",
        "**L2 norm** (or root mean squared error, RMSE): Take the square root of the mean of the square differences. The squaring of differences makes everything positive and the square root undoes the squaring.\n",
        "\n",
        "> Important: \"... the latter will penalize bigger mistakes more heavily than the former (and be more lenient with small mistakes)\"\n",
        "\n",
        "\n",
        "` l2_loss = ((tensor_a - tensor_b) ** 2).sqrt()`\n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r7ZF3BdAItCG"
      },
      "source": [
        "---\n",
        "### Important PyTorch Modules\n",
        "\n",
        "A **module** is a class that inherits from PyTorch's `nn.Module` class. \"Every PyTorch module knows that **parameters** it has that can be trained.\"\n",
        "\n",
        "Here are some key ones ...\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r58ZltSu7IK2"
      },
      "source": [
        "**nn.Linear**: Initializes its parameters and performs a linear operation.  It contains both the weights and biases in a single class"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hbKKnzAu7apS",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ad3a236a-c759-4142-a815-167d1a55f02c"
      },
      "source": [
        "lin1 = nn.Linear(28*28, 1)\n",
        "\n",
        "# the trainable parameters\n",
        "weights, bias = lin1.parameters()\n",
        "print(weights.shape, bias.shape)"
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "torch.Size([1, 784]) torch.Size([1])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "onTRIXF08vJm"
      },
      "source": [
        "**nn.ReLU**: Allows us to add a non-linearity between linear classifiers.  Simply put, it ensures that all activations passed to it are a positive number with every negative number replaced with a 0.  \n",
        "\n",
        "Notice below that it has no trainable parameters!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NjHqHYiT96s_",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "outputId": "03887384-7944-4094-b567-82977c05544f"
      },
      "source": [
        "plot_function(F.relu)"
      ],
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9b3/8ddHVlmUJRHZF0EBRSCkoKJ1q4pUxa6C2KrVUhFqrd2oXrU/7GbtdceF2/KwLbs7tSjiSq2FEkJYZQkgS0AS9n1J8vn9MYd7x5iQSZjJmZm8n4/HPDJzlpn3HMJnTr7nzOeYuyMiIunrpLADiIhIYqnQi4ikORV6EZE0p0IvIpLmVOhFRNJc3bADlCcjI8M7deoUdgwRkZSxYMGCbe6eWd68pCz0nTp1IicnJ+wYIiIpw8zWVzRPQzciImlOhV5EJM2p0IuIpDkVehGRNKdCLyKS5iot9GbW3szeN7PlZrbMzH5UzjJmZk+aWb6ZLTazrKh5N5vZ6uB2c7zfgIiIHF8sp1cWAz9x91wzawosMLPZ7r48apmrgW7BbQDwLDDAzFoADwLZgAfrznD3nXF9FyIiUqFK9+jdfYu75wb39wKfAG3LLDYE+KtHzAWamVlr4CpgtrvvCIr7bGBQXN+BiEga+M+6Hfzpn2tJROv4Ko3Rm1knoC8wr8ystsDGqMebgmkVTS/vuUeYWY6Z5RQVFVUllohISivce4hRk3OZNG8DB4+WxP35Yy70ZtYEeBm42933xDuIu49392x3z87MLPdbvCIiaae4pJQfTl7I3kNHefamLBrVj3/DgpgKvZnVI1LkJ7n7K+UsUgC0j3rcLphW0XQREQH++PYq5q3bwW+u70X3009JyGvEctaNAX8GPnH3RytYbAbw3eDsm/OA3e6+BZgFXGlmzc2sOXBlME1EpNabvXwrz324hmH9O/CNfu0S9jqx/I0wEPgOsMTM8oJp9wIdANz9OWAmMBjIBw4AtwbzdpjZQ8D8YL2x7r4jfvFFRFLT+u37uWd6Hue0PYUHr+2Z0NeqtNC7+0eAVbKMA6MqmDcBmFCtdCIiaejQ0RJGTszFgGeH96NhvToJfb2kbFMsIpLOHnx9Gcu37GHCLdm0b9Eo4a+nFggiIjVoes5GpuVsZPSlXbmse6saeU0VehGRGrJs827uf20pA7u25MdXnFljr6tCLyJSA3YfPMqdk3Jp3qg+TwztS52TjnvoM640Ri8ikmDuzk9fXETBzoNM+8F5ZDRpUKOvrz16EZEEe37OWmYv38q9g3vQr2OLGn99FXoRkQSau3Y7f3hrBV/t1ZpbB3YKJYMKvYhIghTuOcToyQvplNGYh795LpFGAzVPY/QiIglQXFLK6CkL2X+4mMnfH0CTBuGVWxV6EZEEeGTWSv6zbgeP3dCbM1s1DTWLhm5EROJs1rLPeH7OWoYP6MDX+iauWVmsVOhFROLo0237+en0RZzb7lQeSHCzslip0IuIxMmhoyWMnJRLnTrGM8OzaFA3sc3KYqUxehGROLn/taWs+GwPE275Eu2aJ75ZWay0Ry8iEgfT5m/gxQWb+OGlXbn0rNPCjvM5KvQiIidoacFu7n99GRd1y+BHX6m5ZmWxqnToxswmANcAhe5+TjnzfwYMj3q+HkBmcHWpT4G9QAlQ7O7Z8QouIpIMjjUra9m4Po/f0KdGm5XFKpY9+heAQRXNdPdH3L2Pu/cBfgl8WOZygZcG81XkRSStlJY6P5m+iC27DzJueBYta7hZWawqLfTuPgeI9Tqvw4ApJ5RIRCRFPD9nLe98spX7Bvcgq0PzsONUKG5j9GbWiMie/8tRkx1428wWmNmIStYfYWY5ZpZTVFQUr1giIgnx8ZptPDJrBdec25qbL+gUdpzjiufB2GuBf5UZtrnQ3bOAq4FRZvblilZ29/Hunu3u2ZmZmXGMJSISX1v3HOKuKQvpnNGYh78RXrOyWMWz0A+lzLCNuxcEPwuBV4H+cXw9EZEad7SklNGTczlwpITnbupH4xCblcUqLoXezE4FLgZej5rW2MyaHrsPXAksjcfriYiE5Q9vrWD+pzv53dd70S3kZmWxiuX0yinAJUCGmW0CHgTqAbj7c8FiXwPedvf9Uau2Al4N/qSpC0x297fiF11EpGa9tXQL//PPdXznvI4M6dM27Dgxq7TQu/uwGJZ5gchpmNHT1gK9qxtMRCSZrC3ax09fXEzv9s34r2t6hB2nSvTNWBGRShw8UsKdk3Kpl2TNymKV/EcRRERC5O7c99oSVm7dywu39qdts5PDjlRl2qMXETmOqfM38kpuAXdd1o2Lz0zNU79V6EVEKrC0YDcPzog0K7vr8m5hx6k2FXoRkXLsPnCUOyYuIKNxfZ4Y2jcpm5XFSmP0IiJllJY690zPY+ueQ0z/wfm0aFw/7EgnRHv0IiJlPPvhGt5dUch/fbUnfZO4WVmsVOhFRKL8K38b//32Sq7t3Ybvnt8x7DhxoUIvIhL4bHekWVmXzCb8/uu9kr5ZWaw0Ri8iwv81Kzt4tISpw7NSollZrNLnnYiInIDfv7mCnPU7eXJY35RpVhYrDd2ISK03c8kW/vzROm65oBPX9W4Tdpy4U6EXkVptTdE+fv7SYvp2aMa9g1OrWVmsVOhFpNY6cKSYkRMXUL/uSYy7MYv6ddOzJGqMXkRqJXfnvleXsrpwH3/9Xn/apGCzslil58eXiEglJs3bwKsLC7j78jO5qFtqNiuLVaWF3swmmFmhmZV7GUAzu8TMdptZXnB7IGreIDNbaWb5ZjYmnsFFRKpr8aZdjP37ci4+M5MfXtY17DgJF8se/QvAoEqW+ae79wluYwHMrA4wDrga6AkMM7OeJxJWRORE7dx/hJETc8ls2oDHb+jDSSncrCxWlRZ6d58D7KjGc/cH8t19rbsfAaYCQ6rxPCIicVFa6vx4eh6Few8xbngWzVO8WVms4jVGf76ZLTKzN83s7GBaW2Bj1DKbgmnlMrMRZpZjZjlFRUVxiiUi8n+efj+fD1YW8cA1PenTvlnYcWpMPAp9LtDR3XsDTwGvVedJ3H28u2e7e3ZmZnofGBGRmvfR6m089s4qru/ThpvOS49mZbE64ULv7nvcfV9wfyZQz8wygAKgfdSi7YJpIiI1avOug9w1dSFdM5vw2zRqVharEy70Zna6BVvNzPoHz7kdmA90M7POZlYfGArMONHXExGpiiPFpYyanMvhoyU8951+NKpf+74+VOk7NrMpwCVAhpltAh4E6gG4+3PAN4GRZlYMHASGursDxWY2GpgF1AEmuPuyhLwLEZEK/HbmJyzcsItxN2ZxRmaTsOOEotJC7+7DKpn/NPB0BfNmAjOrF01E5MS8sXgzL3z8KbcO7MRXz20ddpzQ6JuxIpKW8gv38YuXFpPVoRm/vDo9m5XFSoVeRNLO/sORZmUN6tVh3PD0bVYWq9p3VEJE0pq7c++rS8gv2sffvjeA1qemb7OyWNXujzkRSTsT567n9bzN3POVM7mwW0bYcZKCCr2IpI28jbsY+8ZyLjkrk1GXpn+zslip0ItIWti5/wijJuVyWtOGtaZZWaw0Ri8iKa+01Ll7Wh5Few/z0sjzadaodjQri5X26EUk5T31Xj4fririgWt7cm672tOsLFYq9CKS0uasKuLxd1fx9b5tGT6gQ9hxkpIKvYikrIJdB/nR1IWceVpTfvO12tesLFYq9CKSko4UlzJqUi5HS5xnb8ri5Pp1wo6UtHQwVkRS0m/+sZy8jbt4ZngWXWpps7JYaY9eRFLOjEWb+cu/13P7hZ0Z3Kv2NiuLlQq9iKSU1Vv3MublxWR3bM4vru4edpyUoEIvIilj/+FiRk7KpVH9SLOyenVUwmJR6VYyswlmVmhmSyuYP9zMFpvZEjP72Mx6R837NJieZ2Y58QwuIrWLuzPmlSWsLdrHk0P70uqUhmFHShmxfBy+AAw6zvx1wMXu3gt4CBhfZv6l7t7H3bOrF1FEBP767/X8fdFmfnLlWVzQVc3KqiKWK0zNMbNOx5n/cdTDuUQuAi4iEje5G3by638s5/LupzHy4jPCjpNy4j3AdRvwZtRjB942swVmNuJ4K5rZCDPLMbOcoqKiOMcSkVS1Y/8RRk/KpdUpDXn022pWVh1xO4/ezC4lUugvjJp8obsXmNlpwGwzW+Huc8pb393HEwz7ZGdne7xyiUjqKil1fjR1Idv2H+GVkRdwaqN6YUdKSXHZozezc4E/AUPcffux6e5eEPwsBF4F+sfj9USkdnjy3dX8c/U2/t91Z3NO21PDjpOyTrjQm1kH4BXgO+6+Kmp6YzNreuw+cCVQ7pk7IiJlfbCykCffW803stox9Evtw46T0iodujGzKcAlQIaZbQIeBOoBuPtzwANAS+CZoKFQcXCGTSvg1WBaXWCyu7+VgPcgImlm084D3D0tj7NaNeXX15+jZmUnKJazboZVMv924PZypq8Fen9xDRGRih0uLmHU5IWUlDjP3tRPzcriQE3NRCSp/PqNT1i0cRfP3ZRF54zGYcdJC/r+sIgkjdfzCvjb3PV8/6LODDpHzcriRYVeRJLCqq17GfPyEr7UqTk/H6RmZfGkQi8iodt3uJiRExfQuEFdnr5RzcriTVtTRELl7vzi5cWs27afp4apWVkiqNCLSKhe+PhT/rF4Cz+7qjvnn9Ey7DhpSYVeREKzYP1OfvOPT/hKj1bccXGXsOOkLRV6EQnF9n2HGT05lzbNTua/v91bX4pKIJ1HLyI1LtKsLI/tx5qVnaxmZYmkPXoRqXFPvLOKj/K38dAQNSurCSr0IlKj3l9RyJPv5fOtfu244Usdwo5TK6jQi0iN2bgj0qysR+tTeOj6c8KOU2uo0ItIjYg0K8ultNR5dngWDeupWVlN0cFYEakRY/++nMWbdvP8d/rRSc3KapT26EUk4V5duIlJ8zbwgy934aqzTw87Tq2jQi8iCbXys73c+8pS+nduwc+uOivsOLVSTIXezCaYWaGZlXspQIt40szyzWyxmWVFzbvZzFYHt5vjFVxEkt/eQ0cZOXEBTRrW5elhfamrZmWhiHWrvwAMOs78q4FuwW0E8CyAmbUgcunBAUQuDP6gmTWvblgRSR3HmpWt33GAp4f15TQ1KwtNTIXe3ecAO46zyBDgrx4xF2hmZq2Bq4DZ7r7D3XcCszn+B4aIpIkJ//qUmUs+4+dXncWALmpWFqZ4/R3VFtgY9XhTMK2i6V9gZiPMLMfMcoqKiuIUS0TCsGD9Dn438xOu7NmKEV9Ws7KwJc2AmbuPd/dsd8/OzMwMO46IVNO2fYe5c1IubZufzCPfUrOyZBCvQl8AtI963C6YVtF0EUlDkWZlC9l14CjPDu+nZmVJIl6Ffgbw3eDsm/OA3e6+BZgFXGlmzYODsFcG00QkDT02exX/yt/OQ9efQ882p4QdRwIxfTPWzKYAlwAZZraJyJk09QDc/TlgJjAYyAcOALcG83aY2UPA/OCpxrr78Q7qikiKem/FVp5+P5+hX2rPt7PbV76C1JiYCr27D6tkvgOjKpg3AZhQ9Wgikio27jjA3VPzOLvNKfzqurPDjiNlJM3BWBFJTYeOljBy0gIAnh3eT83KkpCamonICRn7xnKWFuzhf76bTYeWjcKOI+XQHr2IVNsruZuYPG8Dd1x8Blf0bBV2HKmACr2IVMuKz/Zw76tLGNC5BT+98syw48hxqNCLSJXtOXSUkRNzOaVhPZ66Uc3Kkp3G6EWkStydn7+4mA07DjDl++dxWlM1K0t2+hgWkSr580freGvZZ4wZ1J3+nVuEHUdioEIvIjGb/+kOfvfmCgadfTq3X9Q57DgSIxV6EYlJ0d7DjJqUS/vmJ/OHb52rZmUpRGP0IlKp4pJS7pqykD2HjvKX7/XnlIZqVpZKVOhFpFKPzl7Fv9du54/f6k2P1mpWlmo0dCMixzV7+Vae+WANw/q355v92oUdR6pBhV5EKrRh+wHumZ7HOW1P4cFr1awsVanQi0i5jjUrM9SsLNVpjF5EyvWrGctYtnkPf745m/Yt1KwslWmPXkS+4MWcjUydv5E7LzmDy3uoWVmqi6nQm9kgM1tpZvlmNqac+Y+ZWV5wW2Vmu6LmlUTNmxHP8CISf8s37+G/XlvK+V1acs8ValaWDiodujGzOsA44ApgEzDfzGa4+/Jjy7j7j6OW/yHQN+opDrp7n/hFFpFE2XPoKHdOWkCzRvV4cpialaWLWP4V+wP57r7W3Y8AU4Ehx1l+GDAlHuFEpOa4Oz97cRGbdh5k3I1ZZDZtEHYkiZNYCn1bYGPU403BtC8ws45AZ+C9qMkNzSzHzOaa2fUVvYiZjQiWyykqKoohlojE0//8cy2zlm1lzNXdye6kZmXpJN5/lw0FXnL3kqhpHd09G7gReNzMzihvRXcf7+7Z7p6dmZkZ51gicjzz1m7n4bdWMrjX6dx2oZqVpZtYCn0B0D7qcbtgWnmGUmbYxt0Lgp9rgQ/4/Pi9iISscO8hRk9ZSMcWjXj4G2pWlo5iKfTzgW5m1tnM6hMp5l84e8bMugPNgX9HTWtuZg2C+xnAQGB52XVFJBzFJaX8cPJC9h46yjM3ZdFUzcrSUqVn3bh7sZmNBmYBdYAJ7r7MzMYCOe5+rOgPBaa6u0et3gN43sxKiXyo/D76bB0RCdcf317FvHU7ePTbvel+upqVpauYvhnr7jOBmWWmPVDm8a/KWe9joNcJ5BORBHl72Wc89+EabhzQga9nqVlZOtNJsiK10Prt+/nJi4vo1fZUHrimZ9hxJMFU6EVqmUNHSxg5MZeTzHhmeJaaldUCamomUss88PpSlm/Zw4Rb1KysttAevUgtMn3+RqbnbGL0pV25rLualdUWKvQitcSyzbu5//WlDOzakh+rWVmtokIvUgvsPniUOyfl0rxRfZ4Y2pc6J+lLUbWJxuhF0py789MXF1Gw8yDTfnAeGU3UrKy20R69SJp7fs5aZi/fyr2De9Cvo5qV1UYq9CJpbO7a7fzhrRV89dzW3DqwU9hxJCQq9CJpqnDPIUZPXkinjMZqVlbLaYxeJA0Vl5QyespC9h8uZtLtA2jSQP/VazP964ukoUdmreQ/63bw+A19OOv0pmHHkZBp6EYkzby19DOen7OWm87rwPV9y70YnNQyKvQiaeTTbfv52YuL6N3uVO5XszIJqNCLpImDR0q4Y+IC6tQxxg3PokFdNSuTiJgKvZkNMrOVZpZvZmPKmX+LmRWZWV5wuz1q3s1mtjq43RzP8CIS4e7c//pSVm7dy2M39KFdczUrk/9T6cFYM6sDjAOuADYB881sRjlXiprm7qPLrNsCeBDIBhxYEKy7My7pRQSAafM38tKCTdx1WVcuPeu0sONIkollj74/kO/ua939CDAVGBLj818FzHb3HUFxnw0Mql5UESnP0oLdPDBjGRd1y+BHX1GzMvmiWAp9W2Bj1ONNwbSyvmFmi83sJTNrX8V1MbMRZpZjZjlFRUUxxBKR3QeOcsfEBbRsXJ/Hb+ijZmVSrngdjP070MndzyWy1/6Xqj6Bu49392x3z87MzIxTLJH0VVrq3DM9j617DjFueBYt1axMKhBLoS8A2kc9bhdM+1/uvt3dDwcP/wT0i3VdEameZz9cw7srCrlvcA+yOjQPO44ksVgK/Xygm5l1NrP6wFBgRvQCZtY66uF1wCfB/VnAlWbW3MyaA1cG00TkBHy8Zhv//fZKru3dhpsv6BR2HElylZ514+7FZjaaSIGuA0xw92VmNhbIcfcZwF1mdh1QDOwAbgnW3WFmDxH5sAAY6+47EvA+RGqNz3Yf4q4pC+mc0Zjffb2XmpVJpczdw87wBdnZ2Z6TkxN2DJGkc7SklGHj57J8yx5eHzWQbq3Ux0YizGyBu2eXN09NzURSyMNvriBn/U6eGNpHRV5iphYIIinizSVb+NNH6/ju+R0Z0kfNyiR2KvQiKWBt0T5+9tJierdvxn1f7RF2HEkxKvQiSe7gkRJGTsylXh3jGTUrk2rQGL1IEnN37nttCasK9/LCrf1p2+zksCNJCtIevUgSm/KfjbySW8Bdl3Xj4jP1jXGpHhV6kSS1eNMufhU0K7vr8m5hx5EUpkIvkoR2HTjCyIm5ZDSpzxND+6pZmZwQjdGLJJnSUufH0/Io3HuIF++4gBaN64cdSVKc9uhFkswzH+Tz/soi7r+mJ33aNws7jqQBFXqRJPKv/G08OnsV1/Vuw3fO6xh2HEkTKvQiSeJYs7IumU3UrEziSmP0IkngaEkpoybncvBoCdNuyqJxA/3XlPjRb5NIEvjdzBUsWL+Tp4b1petpalYm8aWhG5GQvbF4MxP+tY5bLujEtb3bhB1H0pAKvUiI8gv38YuXFtO3QzPuHaxmZZIYMRV6MxtkZivNLN/MxpQz/x4zW25mi83sXTPrGDWvxMzygtuMsuuK1FYHjhRz56QFNKhXh3E3ZlG/rva7JDEqHaM3szrAOOAKYBMw38xmuPvyqMUWAtnufsDMRgJ/AG4I5h109z5xzi2S0tyde19ZwurCffz1e/1po2ZlkkCx7EL0B/Ldfa27HwGmAkOiF3D39939QPBwLtAuvjFF0svEeRt4LW8zd19+Jhd1U7MySaxYCn1bYGPU403BtIrcBrwZ9bihmeWY2Vwzu76ilcxsRLBcTlFRUQyxRFLToo27eOjvy7nkrEx+eFnXsONILRDX0yvN7CYgG7g4anJHdy8wsy7Ae2a2xN3XlF3X3ccD4yFycfB45hJJFjv3H+HOSblkNm3AY9/uw0lqViY1IJY9+gKgfdTjdsG0zzGzrwD3Ade5++Fj0929IPi5FvgA6HsCeUVSVmmp8+PpeRTtPcwzw7NormZlUkNiKfTzgW5m1tnM6gNDgc+dPWNmfYHniRT5wqjpzc2sQXA/AxgIRB/EFak1nn4/nw9WFvHAtT3prWZlUoMqHbpx92IzGw3MAuoAE9x9mZmNBXLcfQbwCNAEeDHoz7HB3a8DegDPm1kpkQ+V35c5W0ekVvjn6iIee2cVX+vbluEDOoQdR2oZc0++4fDs7GzPyckJO4ZIXGzedZBrnvqIjCb1eW3UQBrVV+cRiT8zW+Du2eXN0zc0RBLoSHGkWdmR4lKevamfiryEQr91Ign025mfsHDDLsbdmMUZmU3CjiO1lPboRRJkxqLNvPDxp3xvYGe+em7rsONILaZCL5IA+YV7GfPyYvp1bM4vB3cPO47Ucir0InG2/3AxIyfmcnLQrKxeHf03k3BpjF4kjtydX76yhDVF+/jbbQM4/dSGYUcS0R69SDz9be56ZizazD1XnMnArhlhxxEBVOhF4mbhhp089MZyLut+GndeomZlkjxU6EXiYMf+I4yalEurUxry6Ld7q1mZJBWN0YucoJJS5+5peWzbd4SXR15As0ZqVibJRYVe5AQ99d5q5qwq4rdf60WvdqeGHUfkCzR0I3ICPlxVxBPvrubrWW0Z1r995SuIhECFXqSaNu86yN1TF3JWq6b85vpeBJ1bRZKOCr1INRwpLuXOSbkcLXGeGZ7FyfXrhB1JpEIaoxepht/8Yzl5G3fx3E1ZdFGzMkly2qMXqaLX8wr4y7/Xc/uFnRl0jpqVSfKLqdCb2SAzW2lm+WY2ppz5DcxsWjB/npl1ipr3y2D6SjO7Kn7RRWreW0u38MtXlvClTs35xdVqViapodKhGzOrA4wDrgA2AfPNbEaZSwLeBux0965mNhR4GLjBzHoSucbs2UAb4B0zO9PdS+L9RkQSqXDvIR58fRlvLv2Ms9ucwtNqViYpJJYx+v5AvruvBTCzqcAQPn+R7yHAr4L7LwFPW+QUhCHAVHc/DKwzs/zg+f4dn/ifd+1TH3HoqD5DJP627D7EkZJSfj7oLL5/URcVeUkpsRT6tsDGqMebgAEVLRNcTHw30DKYPrfMum3LexEzGwGMAOjQoXoXTz4jszFHSkqrta7I8fRp34wfXHwGXU/TgVdJPUlz1o27jwfGQ+Ti4NV5jseH9o1rJhGRdBDL358FQPRX/toF08pdxszqAqcC22NcV0REEiiWQj8f6GZmnc2sPpGDqzPKLDMDuDm4/03gPXf3YPrQ4KyczkA34D/xiS4iIrGodOgmGHMfDcwC6gAT3H2ZmY0Fctx9BvBn4G/BwdYdRD4MCJabTuTAbTEwSmfciIjULIvseCeX7Oxsz8nJCTuGiEjKMLMF7p5d3jydIyYikuZU6EVE0pwKvYhImlOhFxFJc0l5MNbMioD11Vw9A9gWxzjxolxVo1xVo1xVk465Orp7ZnkzkrLQnwgzy6noyHOYlKtqlKtqlKtqalsuDd2IiKQ5FXoRkTSXjoV+fNgBKqBcVaNcVaNcVVOrcqXdGL2IiHxeOu7Ri4hIFBV6EZE0l/KF3sweMbMVZrbYzF41s2YVLHfcC5wnINe3zGyZmZWaWYWnS5nZp2a2xMzyzCzhndyqkKumt1cLM5ttZquDn80rWK4k2FZ5Zla2XXY88xz3/Qett6cF8+eZWadEZalirlvMrChqG91eA5kmmFmhmS2tYL6Z2ZNB5sVmlpXoTDHmusTMdkdtqwdqKFd7M3vfzJYH/xd/VM4y8d1m7p7SN+BKoG5w/2Hg4XKWqQOsAboA9YFFQM8E5+oBnAV8AGQfZ7lPgYwa3F6V5gppe/0BGBPcH1Pev2Mwb18NbKNK3z9wJ/BccH8oMC1Jct0CPF1Tv0/Ba34ZyAKWVjB/MPAmYMB5wLwkyXUJ8EZNbqvgdVsDWcH9psCqcv4d47rNUn6P3t3fdvfi4OFcIlexKut/L3Du7keAYxc4T2SuT9x9ZSJfozpizFXj2yt4/r8E9/8CXJ/g1zueWN5/dN6XgMvNzJIgV41z9zlErkNRkSHAXz1iLtDMzFonQa5QuPsWd88N7u8FPuGL19KO6zZL+UJfxveIfAqWVd4Fzsu9SHkIHHjbzBYEF0hPBmFsr1buviW4/xnQqoLlGppZjpnNNbNEfRjE8v7/d5lgR2M30DJBeaqSC+AbwZ/7L5lZ+3Lm17Rk/v93vpktMrM3zezsmn7xYMivLzCvzKy4brOkuTj48ZjZO8Dp5cy6z91fDyKcOkgAAAJNSURBVJa5j8hVrCYlU64YXOjuBWZ2GjDbzFYEeyJh54q74+WKfuDubmYVnffbMdheXYD3zGyJu6+Jd9YU9ndgirsfNrMfEPmr47KQMyWrXCK/T/vMbDDwGpHLndYIM2sCvAzc7e57EvlaKVHo3f0rx5tvZrcA1wCXezDAVUZCLlJeWa4Yn6Mg+FloZq8S+fP8hAp9HHLV+PYys61m1trdtwR/ohZW8BzHttdaM/uAyN5QvAt9LO//2DKbzKwucCqwPc45qpzL3aMz/InIsY+wJeT36URFF1d3n2lmz5hZhrsnvNmZmdUjUuQnufsr5SwS122W8kM3ZjYI+DlwnbsfqGCxWC5wXuPMrLGZNT12n8iB5XLPEKhhYWyv6AvM3wx84S8PM2tuZg2C+xnAQCLXI463WN5/dN5vAu9VsJNRo7nKjONeR2T8N2wzgO8GZ5KcB+yOGqYLjZmdfuy4ipn1J1IPE/1hTfCafwY+cfdHK1gsvtuspo84x/sG5BMZy8oLbsfOhGgDzIxabjCRo9triAxhJDrX14iMqx0GtgKzyuYicvbEouC2LFlyhbS9WgLvAquBd4AWwfRs4E/B/QuAJcH2WgLclsA8X3j/wFgiOxQADYEXg9+//wBdEr2NYsz1u+B3aRHwPtC9BjJNAbYAR4PfrduAO4A7gvkGjAsyL+E4Z6HVcK7RUdtqLnBBDeW6kMixucVRdWtwIreZWiCIiKS5lB+6ERGR41OhFxFJcyr0IiJpToVeRCTNqdCLiKQ5FXoRkTSnQi8ikub+PzGlRgmpqkeaAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "EFwEINU48X5b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d16af357-b5a4-418a-8d1c-d895cc1134fa"
      },
      "source": [
        "non_lin1 = nn.ReLU()\n",
        "print(list(non_lin1.parameters()))\n",
        "\n",
        "print('using nn.ReLU: ', non_lin1(tensor(-1)), non_lin1(tensor(4)))\n",
        "print('using max()', tensor(-1).max(tensor(0.0)), tensor(4).max(tensor(0.0)))"
      ],
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[]\n",
            "using nn.ReLU:  tensor(0) tensor(4)\n",
            "using max() tensor(0.) tensor(4.)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qTwkZarZ-Baj"
      },
      "source": [
        "**Why do you want to have non-linearities?**\n",
        "\n",
        "> Because \"there's no point in just putting one linear layout directly after another one, because when we multiply things together and then add them up multiple times, that could be replaced by multiplying different things together and adding them up just once .... BUT if we put a non-linear between them ... this is no longer true. ***Now each linear layer is somewhat decoupled from the other ones and can do its own useful work***.\"\n",
        "\n",
        "These kind of functions are also called **\"activation functions\"**, because the only operate and produce activations ... there are no trainable parameters.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jdu_z1l-_CzS"
      },
      "source": [
        "**nn.Sequential**: A module that can be passed modules, which when called, calls each of those layers in turn."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LgU1UAUd_eni",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "9f6f0d69-49ea-4939-9cf2-6e91cc431d12"
      },
      "source": [
        "lin2 = nn.Linear(1, 10)\n",
        "seq_model = nn.Sequential(lin1, non_lin1, lin2)\n",
        "\n",
        "seq_params = list(seq_model.parameters())\n",
        "print(len(seq_params))\n",
        "\n",
        "for p in seq_params: print(p.shape)"
      ],
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "4\n",
            "torch.Size([1, 784])\n",
            "torch.Size([1])\n",
            "torch.Size([10, 1])\n",
            "torch.Size([10])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0bC4j3qRALdL"
      },
      "source": [
        "Why `4`?  Simple, remember that each `nn.Linear` above has two trainable parameters (the weights and bias), 2+2 = 4."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GZoR3dj_Tu6C"
      },
      "source": [
        "---\n",
        "### Summary\n",
        "\n",
        "This chapter walks you through creating a baseline model to a full blown training loop in PyTorch. ***Read it, and read it again and again!*** (I do and have).\n",
        "\n",
        "**Important Vocb/Concepts**\n",
        "\n",
        "*Activations*: Numbers that are calculated by both linear and non-linear layers\n",
        "\n",
        "*Parameters*: Randomly initialized parameters that can be trained.\n",
        "\n",
        "*Neural Network*: A chain of linear and non-linear functions your data runs through to produce a result.\n",
        "\n",
        "*Gradient*: \"The derivative of the loss with respect to some parameter of the model\"\n",
        "\n",
        "*Backpropagation*: The computing of the gradients \"of the loss with respect to all model parameters\"\n",
        "\n",
        "*Gradient Descent*: \"Taking a step in the direction opposite to the gradients to make the model parameters a little bit better\"\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PSwEOzDpHHBj"
      },
      "source": [
        "---\n",
        "\n",
        "## Resources\n",
        "\n",
        "1. https://book.fast.ai - The book's website; it's updated regularly with new content and recommendations from everything to GPUs to use, how to run things locally and on the cloud, etc...\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B2wEfmj5CLA8"
      },
      "source": [
        ""
      ],
      "execution_count": 22,
      "outputs": []
    }
  ]
}