01_pytorch_basics.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.6"
    },
    "colab": {
      "name": "01-pytorch-basics.ipynb",
      "provenance": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/gkdivya/PyTorch/blob/master/01_pytorch_basics.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IyUiE93KQlFs",
        "colab_type": "text"
      },
      "source": [
        "# PyTorch Basics: Tensors & Gradients\n",
        "\n",
        "#### *Part 1 of \"Pytorch: Zero to GANs\"*\n",
        "\n",
        "*This post is the first in a series of tutorials on building deep learning models with PyTorch, an open source neural networks library developed and maintained by Facebook. Check out the full series:*\n",
        "\n",
        "1. [PyTorch Basics: Tensors & Gradients](https://jovian.ml/aakashns/01-pytorch-basics)\n",
        "2. [Linear Regression & Gradient Descent](https://jovian.ml/aakashns/02-linear-regression)\n",
        "3. [Image Classfication using Logistic Regression](https://jovian.ml/aakashns/03-logistic-regression) \n",
        "4. [Training Deep Neural Networks on a GPU](https://jovian.ml/aakashns/04-feedforward-nn)\n",
        "5. [Image Classification using Convolutional Neural Networks](https://jovian.ml/aakashns/05-cifar10-cnn)\n",
        "6. [Data Augmentation, Regularization and ResNets](https://jovian.ml/aakashns/05b-cifar10-resnet)\n",
        "7. [Generating Images using Generative Adverserial Networks](https://jovian.ml/aakashns/06-mnist-gan)\n",
        "\n",
        "This series attempts to make PyTorch a bit more approachable for people starting out with deep learning and neural networks. In this notebook, we’ll cover the basic building blocks of PyTorch models: tensors and gradients."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M5Qz82BLQlFu",
        "colab_type": "text"
      },
      "source": [
        "## System setup\n",
        "\n",
        "This tutorial takes a code-first approach towards learning PyTorch, and you should try to follow along by running and experimenting with the code yourself. The easiest way to start executing this notebook is to click the **\"Run\"** button at the top of this page, and select **\"Run on Binder\"**. This will run the notebook on [mybinder.org](https://mybinder.org), a free online service for running Jupyter notebooks.\n",
        "\n",
        "**NOTE**: *If you're running this notebook on Binder, please skip ahead to the next section.*\n",
        "\n",
        "### Running on your computer locally\n",
        "\n",
        "We'll use the [Anaconda distribution](https://www.anaconda.com/distribution/) of Python to install libraries and manage virtual environments. For interactive coding and experimentation, we'll use [Jupyter notebooks](https://jupyter.org/). All the tutorials in this series are available as Jupyter notebooks hosted on [Jovian.ml](https://www.jovian.ml): a sharing and collaboration platform for Jupyter notebooks & machine learning experiments.\n",
        "\n",
        "Jovian.ml makes it easy to share Jupyter notebooks on the cloud by running a single command directly within Jupyter. It also captures the Python environment and libraries required to run your notebook, so anyone (including you) can reproduce your work.\n",
        "\n",
        "Here's what you need to do to get started:\n",
        "\n",
        "1. Install Anaconda by following the [instructions given here](https://conda.io/projects/conda/en/latest/user-guide/install/index.html). You might also need to add Anaconda binaries to your system PATH to be able to run the `conda` command line tool.\n",
        "\n",
        "\n",
        "2. Install the `jovian` Python library by the running the following command (without the `$`) on your Mac/Linux terminal or Windows command prompt:\n",
        "\n",
        "```\n",
        "$ pip install jovian --upgrade\n",
        "```\n",
        "\n",
        "3. Download the notebook for this tutorial using the `jovian clone` command:\n",
        "\n",
        "```\n",
        "$ jovian clone aakashns/01-pytorch-basics\n",
        "```\n",
        "\n",
        "(You can copy this command to clipboard by clicking the 'Clone' button at the top of this page on Jovian.ml)\n",
        "\n",
        "Running the clone command creates a directory `01-pytorch-basics` containing a Jupyter notebook and an Anaconda environment file.\n",
        "\n",
        "```\n",
        "$ ls 01-pytorch-basics\n",
        "01-pytorch-basics.ipynb  environment.yml\n",
        "```\n",
        "\n",
        "4. Now we can enter the directory and install the required Python libraries (Jupyter, PyTorch etc.) with a single command using `jovian`:\n",
        "\n",
        "```\n",
        "$ cd 01-pytorch-basics\n",
        "$ jovian install\n",
        "```\n",
        "\n",
        "`jovian install` reads the `environment.yml` file, identifies the right dependencies for your operating system, creates a virtual environment with the given name (`01-pytorch-basics` by default) and installs all the required libraries inside the environment, to avoid modifying your system-wide installation of Python. It uses `conda` internally. If you face issues with `jovian install`, try running `conda env update` instead.\n",
        "\n",
        "5. We can activate the virtual environment by running\n",
        "\n",
        "```\n",
        "$ conda activate 01-pytorch-basics\n",
        "```\n",
        "\n",
        "For older installations of `conda`, you might need to run the command: `source activate 01-pytorch-basics`.\n",
        "\n",
        "6. Once the virtual environment is active, we can start Jupyter by running\n",
        "\n",
        "```\n",
        "$ jupyter notebook\n",
        "```\n",
        "\n",
        "7. You can now access Jupyter's web interface by clicking the link that shows up on the terminal or by visiting http://localhost:8888 on your browser. At this point, you can click on the notebook `01-pytorch-basics.ipynb` to open it and run the code. If you want to type out the code yourself, you can also create a new notebook using the 'New' button."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-ey2dBoBQlFv",
        "colab_type": "text"
      },
      "source": [
        "We begin by importing PyTorch:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "avw1_XNMQlFw",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Uncomment the command below if PyTorch is not installed\n",
        "# !conda install pytorch cpuonly -c pytorch -y"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fbAuljLOQlF2",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import torch"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Z9nvtOk0QlF6",
        "colab_type": "text"
      },
      "source": [
        "## Tensors\n",
        "\n",
        "At its core, PyTorch is a library for processing tensors. A tensor is a number, vector, matrix or any n-dimensional array. Let's create a tensor with a single number:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NsUkxfE5QlF7",
        "colab_type": "code",
        "colab": {},
        "outputId": "b6f8decf-2950-4d57-f7cc-2aaf86e01d1b"
      },
      "source": [
        "# Number\n",
        "t1 = torch.tensor(4.)\n",
        "t1"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(4.)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zWgtc8ArQlGA",
        "colab_type": "text"
      },
      "source": [
        "`4.` is a shorthand for `4.0`. It is used to indicate to Python (and PyTorch) that you want to create a floating point number. We can verify this by checking the `dtype` attribute of our tensor:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "06SSnIlwQlGA",
        "colab_type": "code",
        "colab": {},
        "outputId": "d3280c9f-3fca-4525-9855-190b0699a8f5"
      },
      "source": [
        "t1.dtype"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.float32"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PFFXd0OWQlGE",
        "colab_type": "text"
      },
      "source": [
        "Let's try creating slightly more complex tensors:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ioQpXC0FQlGF",
        "colab_type": "code",
        "colab": {},
        "outputId": "ded1f83f-fb47-41f2-8c30-11d6124ca761"
      },
      "source": [
        "# Vector\n",
        "t2 = torch.tensor([1., 2, 3, 4])\n",
        "t2"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([1., 2., 3., 4.])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LZLNbWn5QlGK",
        "colab_type": "code",
        "colab": {},
        "outputId": "28b0e36e-f99b-4c16-fe33-fad964bb54e2"
      },
      "source": [
        "# Matrix\n",
        "t3 = torch.tensor([[5., 6], [7, 8], [9, 10]])\n",
        "t3"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[ 5.,  6.],\n",
              "        [ 7.,  8.],\n",
              "        [ 9., 10.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oGbrZfoeQlGP",
        "colab_type": "code",
        "colab": {},
        "outputId": "1134edc3-735f-4094-892a-37391eb13174"
      },
      "source": [
        "# 3-dimensional array\n",
        "t4 = torch.tensor([\n",
        "    [[11, 12, 13], \n",
        "     [13, 14, 15]], \n",
        "    [[15, 16, 17], \n",
        "     [17, 18, 19.]]])\n",
        "t4"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[[11., 12., 13.],\n",
              "         [13., 14., 15.]],\n",
              "\n",
              "        [[15., 16., 17.],\n",
              "         [17., 18., 19.]]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kdsBQ6MXQlGS",
        "colab_type": "text"
      },
      "source": [
        "Tensors can have any number of dimensions, and different lengths along each dimension. We can inspect the length along each dimension using the `.shape` property of a tensor."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "34nVHWQjQlGT",
        "colab_type": "code",
        "colab": {},
        "outputId": "b9ecec1f-9576-4cae-9270-316b140d608f"
      },
      "source": [
        "print(t1)\n",
        "t1.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor(4.)\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "quiQxTJKQlGW",
        "colab_type": "code",
        "colab": {},
        "outputId": "6c428521-b578-4731-93f7-36376009cff4"
      },
      "source": [
        "print(t2)\n",
        "t2.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([1., 2., 3., 4.])\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([4])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PfQ5aJTXQlGa",
        "colab_type": "code",
        "colab": {},
        "outputId": "08402c0d-a3b9-4828-9d83-0b2132452474"
      },
      "source": [
        "print(t3)\n",
        "t3.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[ 5.,  6.],\n",
            "        [ 7.,  8.],\n",
            "        [ 9., 10.]])\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([3, 2])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-wIims5RQlGf",
        "colab_type": "code",
        "colab": {},
        "outputId": "9e35caef-0884-41dc-8428-8e53b0d52f1a"
      },
      "source": [
        "print(t4)\n",
        "t4.shape"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[[11., 12., 13.],\n",
            "         [13., 14., 15.]],\n",
            "\n",
            "        [[15., 16., 17.],\n",
            "         [17., 18., 19.]]])\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "torch.Size([2, 2, 3])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "INdpg0lxQlGj",
        "colab_type": "text"
      },
      "source": [
        "## Tensor operations and gradients\n",
        "\n",
        "We can combine tensors with the usual arithmetic operations. Let's look an example:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "a5Zho26JQlGk",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Create tensors.\n",
        "x = torch.tensor(3.)\n",
        "w = torch.tensor(4., requires_grad=True)\n",
        "b = torch.tensor(5., requires_grad=True)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7EsdX0hFQlGo",
        "colab_type": "text"
      },
      "source": [
        "We've created 3 tensors `x`, `w` and `b`, all numbers. `w` and `b` have an additional parameter `requires_grad` set to `True`. We'll see what it does in just a moment. \n",
        "\n",
        "Let's create a new tensor `y` by combining these tensors:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V2RSwnZfQlGo",
        "colab_type": "code",
        "colab": {},
        "outputId": "59aea458-ae6b-44d4-eb8b-f4c74055b89d"
      },
      "source": [
        "# Arithmetic operations\n",
        "y = w * x + b\n",
        "y"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor(17., grad_fn=<AddBackward0>)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hyZrrRncQlGt",
        "colab_type": "text"
      },
      "source": [
        "As expected, `y` is a tensor with the value `3 * 4 + 5 = 17`. What makes PyTorch special is that we can automatically compute the derivative of `y` w.r.t. the tensors that have `requires_grad` set to `True` i.e. w and b. To compute the derivatives, we can call the `.backward` method on our result `y`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "djPiTEXvQlGu",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# Compute derivatives\n",
        "y.backward()"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ILaPu8AbQlGx",
        "colab_type": "text"
      },
      "source": [
        "The derivates of `y` w.r.t the input tensors are stored in the `.grad` property of the respective tensors."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CLnJQ2SeQlGy",
        "colab_type": "code",
        "colab": {},
        "outputId": "24897d11-7567-4406-cf73-bd0621eea937"
      },
      "source": [
        "# Display gradients\n",
        "print('dy/dx:', x.grad)\n",
        "print('dy/dw:', w.grad)\n",
        "print('dy/db:', b.grad)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "dy/dx: None\n",
            "dy/dw: tensor(3.)\n",
            "dy/db: tensor(1.)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vSEWUm53QlG2",
        "colab_type": "text"
      },
      "source": [
        "As expected, `dy/dw` has the same value as `x` i.e. `3`, and `dy/db` has the value `1`. Note that `x.grad` is `None`, because `x` doesn't have `requires_grad` set to `True`. \n",
        "\n",
        "The \"grad\" in `w.grad` stands for gradient, which is another term for derivative, used mainly when dealing with matrices. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jik-2wsuQlG3",
        "colab_type": "text"
      },
      "source": [
        "## Interoperability with Numpy\n",
        "\n",
        "[Numpy](http://www.numpy.org/) is a popular open source library used for mathematical and scientific computing in Python. It enables efficient operations on large multi-dimensional arrays, and has a large ecosystem of supporting libraries:\n",
        "\n",
        "* [Matplotlib](https://matplotlib.org/) for plotting and visualization\n",
        "* [OpenCV](https://opencv.org/) for image and video processing\n",
        "* [Pandas](https://pandas.pydata.org/) for file I/O and data analysis\n",
        "\n",
        "Instead of reinventing the wheel, PyTorch interoperates really well with Numpy to leverage its existing ecosystem of tools and libraries."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "G72tPnvtQlG3",
        "colab_type": "text"
      },
      "source": [
        "Here's how we create an array in Numpy:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UJQyVAktQlG4",
        "colab_type": "code",
        "colab": {},
        "outputId": "7120b9e4-7aaf-4810-b3bc-f3b5d2ce2e8d"
      },
      "source": [
        "import numpy as np\n",
        "\n",
        "x = np.array([[1, 2], [3, 4.]])\n",
        "x"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1., 2.],\n",
              "       [3., 4.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L4AL0fUrQlG8",
        "colab_type": "text"
      },
      "source": [
        "We can convert a Numpy array to a PyTorch tensor using `torch.from_numpy`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s0QOOVXrQlG9",
        "colab_type": "code",
        "colab": {},
        "outputId": "bcea5fc9-3895-41ff-8f46-0a541d38df62"
      },
      "source": [
        "# Convert the numpy array to a torch tensor.\n",
        "y = torch.from_numpy(x)\n",
        "y"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "tensor([[1., 2.],\n",
              "        [3., 4.]], dtype=torch.float64)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GjFyxC5BQlG_",
        "colab_type": "text"
      },
      "source": [
        "Let's verify that the numpy array and torch tensor have similar data types."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ueBQfeU2QlHA",
        "colab_type": "code",
        "colab": {},
        "outputId": "98e51e6e-ffeb-4b07-d3bd-4de73aed627c"
      },
      "source": [
        "x.dtype, y.dtype"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(dtype('float64'), torch.float64)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DsGWAWsCQlHD",
        "colab_type": "text"
      },
      "source": [
        "We can convert a PyTorch tensor to a Numpy array using the `.numpy` method of a tensor."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "loMCdbKUQlHD",
        "colab_type": "code",
        "colab": {},
        "outputId": "58e8b3a6-08ec-492e-90f4-261022a3da63"
      },
      "source": [
        "# Convert a torch tensor to a numpy array\n",
        "z = y.numpy()\n",
        "z"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[1., 2.],\n",
              "       [3., 4.]])"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Td1ThFYcQlHG",
        "colab_type": "text"
      },
      "source": [
        "The interoperability between PyTorch and Numpy is really important because most datasets you'll work with will likely be read and preprocessed as Numpy arrays."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BP85LYyyQlHH",
        "colab_type": "text"
      },
      "source": [
        "## Commit and upload the notebook\n",
        "\n",
        "As a final step, we can save and commit out work using the `jovian` library."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "L_G9aNBSQlHH",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "!pip install jovian --upgrade --quiet"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k_Wp1rXnQlHM",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import jovian"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NTxZ77ZBQlHQ",
        "colab_type": "code",
        "colab": {},
        "outputId": "98209244-c999-4e7e-d9bc-102659f4f143"
      },
      "source": [
        "jovian.commit()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "application/javascript": [
              "window.require && require([\"base/js/namespace\"],function(Jupyter){Jupyter.notebook.save_checkpoint()})"
            ],
            "text/plain": [
              "<IPython.core.display.Javascript object>"
            ]
          },
          "metadata": {
            "tags": []
          }
        },
        {
          "output_type": "stream",
          "text": [
            "[jovian] Attempting to save notebook..\u001b[0m\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5bintJBbQlHT",
        "colab_type": "text"
      },
      "source": [
        "`jovian.commit` uploads the notebook to your [Jovian.ml](https://www.jovian.ml) account, captures the Python environment and creates a sharable link for your notebook as shown above. You can use this link to share your work and let anyone reproduce it easily with the `jovian clone` command. Jovian also includes a powerful commenting interface, so you (and others) can discuss & comment on specific parts of your notebook:\n",
        "\n",
        "![commenting on jovian](https://cdn-images-1.medium.com/max/1600/1*b4snnr_5Ve5Nyq60iDtuuw.png)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FT6yUsYOQlHU",
        "colab_type": "text"
      },
      "source": [
        "## Further Reading\n",
        "\n",
        "Tensors in PyTorch support a variety of operations, and what we've covered here is by no means exhaustive. You can learn more about tensors and tensor operations here: https://pytorch.org/docs/stable/tensors.html\n",
        "\n",
        "You can take advantage of the interactive Jupyter environment to experiment with tensors and try different combinations of operations discussed above. Here are some things to try out:\n",
        "\n",
        "1. What if one or more `x`, `w` or `b` were matrices, instead of numbers, in the above example? What would the result `y` and the gradients `w.grad` and `b.grad` look like in this case?\n",
        "\n",
        "2. What if `y` was a matrix created using `torch.tensor`, with each element of the matrix expressed as a combination of numeric tensors `x`, `w` and `b`?\n",
        "\n",
        "3. What if we had a chain of operations instead of just one i.e. `y = x * w + b`, `z = l * y + m`, `w = c * z + d` and so on? What would calling `w.grad` do?\n",
        "\n",
        "If you're interested, you can learn more about matrix derivates on Wikipedia (although it's not necessary for following along with this series of tutorials): https://en.wikipedia.org/wiki/Matrix_calculus#Derivatives_with_matrices "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gdkRU1nsQlHU",
        "colab_type": "text"
      },
      "source": [
        "With this, we complete our discussion of tensors and gradients in PyTorch, and we're ready to move on to the next topic: *Linear regression*.\n",
        "\n",
        "## Credits\n",
        "\n",
        "The material in this series is heavily inspired by the following resources:\n",
        "\n",
        "1. [PyTorch Tutorial for Deep Learning Researchers](https://github.com/yunjey/pytorch-tutorial) by Yunjey Choi: \n",
        "\n",
        "2. [FastAI development notebooks](https://github.com/fastai/fastai_docs/tree/master/dev_nb) by Jeremy Howard: \n"
      ]
    }
  ]
}