2_gradient_descent.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6RY64HwBgDLg"
   },
   "source": [
    "[![Open in Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/lorenzobasile/DeepLearning2022/blob/main/2_gradient_descent.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f_fb8h4LgFYe"
   },
   "source": [
    "# Lab 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pYsaK1ilgQW8"
   },
   "source": [
    "# Training a Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WWo1tB98gXc5"
   },
   "source": [
    "Today we will introduce the basic tools we need to apply backpropagation and train a neural network with gradient-based methods.\n",
    "\n",
    "To start, let's import the usual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "YgxcemAAYNg9"
   },
   "outputs": [],
   "source": [
    "from google.colab import drive\n",
    "import torch\n",
    "\n",
    "drive.mount('/content/drive')\n",
    "%cd drive/MyDrive/DeepLearning2022"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "G1fMrMO4WGSr"
   },
   "source": [
    "## Automatic differentiation in torch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OP0Bo0KjPoIi"
   },
   "source": [
    "\n",
    "\n",
    "PyTorch is built with support for differentiation in mind.\n",
    "In the end, Deep Learning (for now) is all about differentiation and building cascades of differentiable function into complicated multilayer deep neural networks.\n",
    "\n",
    "Essentially, all PyTorch built-ins support differentiability (unless the function is not differentiable, of course).\n",
    "Today we will see how to compute derivatives in PyTorch.\n",
    "\n",
    "\n",
    "Under the hood, each torch `Tensor` has a boolean attribute `requires_grad`, which tells `autograd` whether it should keep trace of operations on the tensor or not. `autograd` is the automatic differentiation engine of PyTorch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "TOf6IIJjQDL-"
   },
   "outputs": [],
   "source": [
    "x = torch.rand(3,3)\n",
    "\n",
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1n0d6nrTQVA2"
   },
   "outputs": [],
   "source": [
    "x.requires_grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4CZUMdXkQYZw"
   },
   "source": [
    "We can manually set this to `True` or create directly a Tensor supporting grad."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "_Q0lDgdTQZIH"
   },
   "outputs": [],
   "source": [
    "x.requires_grad = True\n",
    "# or equivalently x=torch.rand(3, 3, requires_grad=True)\n",
    "print(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ubiI7-eeQpLu"
   },
   "source": [
    "Now suppose we are handling a function $f:\\mathbb{R}\\rightarrow\\mathbb{R}$.\n",
    "\n",
    "For instance, $f(x) = x^2$.\n",
    "\n",
    "We could apply $f$ to a singleton tensor and compute the derivative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1auVJ1_rQkRq"
   },
   "outputs": [],
   "source": [
    "x = torch.rand(1, requires_grad=True)\n",
    "\n",
    "print(\"x:\", x)\n",
    "\n",
    "y = x**2\n",
    "\n",
    "print(\"y:\", y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3iJOKajXRDQQ"
   },
   "source": [
    "To compute the gradient, we call `backward()` on the Tensor `y`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "nDA4uxykRDyj"
   },
   "outputs": [],
   "source": [
    "y.backward()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dUUnstacSuLH"
   },
   "source": [
    "We expect the derivative to be $2x$. We can check this is correct by inspecting the gradient of `x`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XI4eeG08RTaS"
   },
   "outputs": [],
   "source": [
    "x.grad==2*x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LxCs1mBaReci"
   },
   "source": [
    "Notice that when there's no gradient, `grad` is automatically set to `None` to save memory"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VSyJlGA_Riji"
   },
   "outputs": [],
   "source": [
    "torch.rand(3,3).grad is None"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ye-RkzucS5wn"
   },
   "source": [
    "The same operation can be repeated in a slightly more complicated setting, when dealing with a scalar function of more than one variable $f:\\mathbb{R}^d\\rightarrow\\mathbb{R}$\n",
    "\n",
    "This step is particularly important since the core operation of Deep Learning is computing the gradients of a scalar function (our loss function) with respect to the parameters of the network.\n",
    "\n",
    "Now `x` will be a vector (or a matrix, it doesn't really matter for our case) and we will apply to it a function which returns a single scalar.\n",
    "\n",
    "One example may be $f(\\mathbf{x})=\\sum_{i=1}^d x_i$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "RLKtYCaURqAI"
   },
   "outputs": [],
   "source": [
    "x = torch.rand([5], requires_grad=True)\n",
    "\n",
    "print(x)\n",
    "\n",
    "y = x.sum()\n",
    "\n",
    "y.backward()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "aVscFTzsRqCb"
   },
   "outputs": [],
   "source": [
    "x.grad"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Yz6ngEuRWDoY"
   },
   "source": [
    "## Composition of functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "xsHzelLHVmI6"
   },
   "source": [
    "We can use also `backward` to compute the gradient of a composition of functions. For our objective, it will be very useful to think in terms of computational graph.\n",
    "\n",
    "We can view $y=g(f(x))$ as\n",
    "\n",
    "![](images/compgra1.jpg)\n",
    "\n",
    "We might extend this and add a hidden node $z$\n",
    "between $f$ and $g$\n",
    "\n",
    "![](images/compgra2.jpg)\n",
    "\n",
    "Supposing $z=f(x)=\\log(x)$\n",
    "and $y=g(z)=z^2$, we can reproduce this example in PyTorch. \n",
    "\n",
    "To sum up, $y=\\log^2(x)$, and so we expect $\\frac{dy}{dx}=2\\frac{\\log(x)}{|x|}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "K9OtyfW1RqFC"
   },
   "outputs": [],
   "source": [
    "x = torch.rand(1, requires_grad=True)\n",
    "\n",
    "print(\"x:\", x)\n",
    "\n",
    "z = x.log()\n",
    "\n",
    "y = z**2\n",
    "\n",
    "print(\"y:\", y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "VaBT4HUOXZ7l"
   },
   "outputs": [],
   "source": [
    "#z.retain_grad()\n",
    "\n",
    "y.backward()\n",
    "\n",
    "x.grad==2*x.log()/x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fsUDjb72XnQc"
   },
   "source": [
    "Now suppose that we want to access $\\frac{dy}{dz}=2z$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gD2dqGsuRqHH"
   },
   "outputs": [],
   "source": [
    "z.grad==2*z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "TkCS8OLZYVGh"
   },
   "source": [
    "To store gradients of intermediate computations, we can call `.retain_grad()` on the intermediate node."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oFZBlPs5V_hm"
   },
   "source": [
    "## Gradient accumulation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "f0n5A6zVWrme"
   },
   "source": [
    "Let us see another feature of torch differentiation functionalities.\n",
    "\n",
    "We can call `backward()` multiple times; let's see what happens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "XaWFpl0WWHQ7"
   },
   "outputs": [],
   "source": [
    "x_1 = torch.tensor([3.0], requires_grad=True)\n",
    "\n",
    "x_2 = torch.tensor([2.0], requires_grad=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ShH8Jd69Wf97"
   },
   "outputs": [],
   "source": [
    "c = x_1.cos() * x_2.log()\n",
    "c.backward()\n",
    "print(x_1.grad, x_2.grad)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dGIukjMmWKJV"
   },
   "source": [
    "What is happening? Why the gradient is not the same?\n",
    "\n",
    "PyTorch continues to accumulate (i.e., sum) the gradients. If we want to reset the gradient, we must set it to None\n",
    "```\n",
    "x_1.grad = None\n",
    "x_2.grad = None\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2OQuH1OCRm5r"
   },
   "source": [
    "## Optimizers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SQtmcB63Zcf_"
   },
   "source": [
    "Now that we have the tools to compute gradients of any function (specifically, we will be interested in loss functions), it is time to figure out what to do with these gradients.\n",
    "\n",
    "In torch, it is straightforward to use most optimization techniques based on Gradient Descent and its variations, such as SGD, Adam, RMSProp etc.\n",
    "\n",
    "To do so, you should exploit the tools of the `torch.optim` package, that contains the most famous training algorithms in the `Optimizer` class. To cite some:\n",
    "\n",
    "*   `torch.optim.SGD`\n",
    "*   `torch.optim.Adam`\n",
    "*   `torch.optim.RMSprop`\n",
    "*   `torch.optim.Adagrad`\n",
    "\n",
    "To construct an Optimizer you have to give it an iterable containing the parameters to optimize. Then, you can specify optimizer-specific options such as the learning rate, weight decay, etc. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "KOalnCT4TxvB"
   },
   "outputs": [],
   "source": [
    "# note that the lr parameter is mandatory, there is no default\n",
    "optimizer=torch.optim.SGD(model.parameters(), lr=learning_rate, momentum=0, weight_decay=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2xOb95AWUFeC"
   },
   "source": [
    "Using an optimizer is very simple, there are three main steps:\n",
    "\n",
    "*   delete the current gradient information on the parameters: `optimizer.zero_grad()`\n",
    "*   compute the derivatives of the loss: `loss.backward()`\n",
    "*   perform a gradient descent step and update parameters (according to the algorithm in use): `optimizer.step()`\n",
    "\n",
    "Note that the first step should always be performed since, as we saw, torch accumulates gradients.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9eP1wJOE4GBa"
   },
   "source": [
    "## Adaptive learning rate"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fR85Cbvr4LTs"
   },
   "source": [
    "At times, it is useful to vary the learning rate while training, for example you may want to use a large learning rate in the initial phase of training to quickly descend the loss and then you may want to decrease it to be more precise around a minimum. To do so, the easiest way is to use schedulers that you can find in `torch.optim.lr_scheduler`.\n",
    "\n",
    "The simplest LR scheduler is `ExponentialLR`, which takes a parameter $\\gamma$ and at each step does:\n",
    "\n",
    "$$\n",
    "\\text{lr}=\\gamma \\text{lr}\n",
    "$$\n",
    "\n",
    "More info on this [here](https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate), but in a nutshell all you have to do is something like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "pQBZspgE47Mw"
   },
   "outputs": [],
   "source": [
    "optimizer = torch.optim.SGD(model.parameters(), lr=0.1)\n",
    "scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer, gamma=0.9)\n",
    "\n",
    "#and then, when you want to actually decrement the learning rate\n",
    "scheduler.step()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "taOL7GFIZxpq"
   },
   "source": [
    "## Back to our linear regression"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "w96XKlptZz-Z"
   },
   "source": [
    "Coming back to the linear regression from the previous lab, we now have all the tools to fit its weights and bias with Stochastic Gradient Descent (SGD) instead of using Least Squares.\n",
    "\n",
    "First of all, let's recover some code from the last lab:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "1y_tDV24YP00"
   },
   "outputs": [],
   "source": [
    "X=torch.load(\"./data/X_reg.pt\")\n",
    "Y=torch.load(\"./data/y_reg.pt\")\n",
    "N,P=X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "rAMzW2eNY75m"
   },
   "outputs": [],
   "source": [
    "class LinearRegressor(torch.nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.regressor = torch.nn.Linear(in_features=P, out_features=1, bias=True)\n",
    "\n",
    "    def forward(self, X):\n",
    "        return self.regressor(X)\n",
    "\n",
    "lin_reg=LinearRegressor()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0BubV5-qaSN1"
   },
   "source": [
    "We define a `batch_size` of 8. This means that at each training step the network is fed with 8 datapoints from our dataset, on which a gradient descent step is performed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "8dbNOJiMY-3d"
   },
   "outputs": [],
   "source": [
    "dataset=torch.utils.data.TensorDataset(X,Y)\n",
    "dataloader=torch.utils.data.DataLoader(dataset, batch_size=8, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "orCIyegW6zYF"
   },
   "source": [
    "We are doing regression, so our objective is the minimization of the Mean Squared Error between the targets and the output of our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "OKgQVle4ZFQE"
   },
   "outputs": [],
   "source": [
    "loss=torch.nn.MSELoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jvUhbhbnajIl"
   },
   "source": [
    "Now we can go on and define our optimizer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "AENyq2s0ao79"
   },
   "outputs": [],
   "source": [
    "optimizer=torch.optim.SGD(lin_reg.parameters(), lr=1e-3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KUBOOSP86--y"
   },
   "source": [
    "The following cell contains the nested loop that we will run to train the network. Its pseudocode is as follows:\n",
    "\n",
    "\n",
    "```\n",
    "Loop over epochs:\n",
    "    Loop over data:\n",
    "        Perform a forward pass\n",
    "        Compute the loss\n",
    "        Erase the past gradients\n",
    "        Compute gradients performing a backward pass\n",
    "        Update the parameters\n",
    "```\n",
    "\n",
    "\n",
    "\n",
    "SGD training is an iterative process, that we repeat for a (usually) fixed number of *epochs*. In each epoch we traverse the whole dataset by exploiting our `DataLoader` object, that provides us with randomly drawn mini-batches of 8 elements.\n",
    "\n",
    "We can keep track of the loss so that we can compare it with the loss of the least squares estimate we obtained during the previous lab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "jJlSMAjTawC4"
   },
   "outputs": [],
   "source": [
    "epochs=100\n",
    "losses=[]\n",
    "for epoch in range(epochs):\n",
    "    epoch_loss=0 # we want to accumulate the loss on all the mini-batches to average and obtain the overall MSE\n",
    "    for x, y in iter(dataloader):\n",
    "        out=lin_reg(x)\n",
    "        l=loss(out, y)\n",
    "        epoch_loss+=l.item()\n",
    "        optimizer.zero_grad()\n",
    "        l.backward()\n",
    "        optimizer.step()\n",
    "    losses.append(epoch_loss/len(dataloader)) # len(dataloader) contains the number of mini-batches"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jJfanVLVa3sJ"
   },
   "source": [
    "The loss rapidly decreases during training, approaching the optimal one obtained with least squares the last time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "X8WvrqCJ0GW0"
   },
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.title(\"Linear regession loss\")\n",
    "plt.axhline(0.9005, color='r', label='Least Squares')\n",
    "plt.semilogy(losses, label='SGD')\n",
    "plt.legend()\n",
    "plt.xlabel(\"Optimization epoch\")\n",
    "plt.ylabel(\"MSE Loss\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "g0meO2WLbEMJ"
   },
   "source": [
    "# A simple MLP Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j5sgb06m9EE_"
   },
   "source": [
    "We will now try to solve our first *real* problem with Deep Learning. The task is handwritten digit recognition, and we will use the MNIST dataset. It is an outstandingly popular benchmark in the Machine Learning community, and it is seen as the first and simplest real-world task one can solve with neural network (aka the *Hello World of Deep Learning*).\n",
    "\n",
    "First of all, let's download the data and create our DataLoaders.\n",
    "\n",
    "The transforms we are employing are intended to convert the images into torch Tensors (`ToTensor()`) and to normalize the images to have 0 mean and standard deviation 1 (`Normalize`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "cfX8yiJa2cAJ"
   },
   "outputs": [],
   "source": [
    "import torchvision\n",
    "\n",
    "transforms = torchvision.transforms.Compose([\n",
    "        torchvision.transforms.ToTensor(),\n",
    "        torchvision.transforms.Normalize((0.1307,), (0.3081,))\n",
    "    ])\n",
    "\n",
    "trainset = torchvision.datasets.MNIST('./data/', transform=transforms,  train=True, download=True)\n",
    "trainloader = torch.utils.data.DataLoader(trainset, batch_size=256, shuffle=True)\n",
    "\n",
    "testset = torchvision.datasets.MNIST('./data/', transform=transforms, train=False, download=True)\n",
    "testloader = torch.utils.data.DataLoader(testset, batch_size=512, shuffle=False)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vWE6i_5J_B9B"
   },
   "source": [
    "Now, we can visualize our data using `matplotlib`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "65EYWL42_GPg"
   },
   "outputs": [],
   "source": [
    "x,y=next(iter(trainloader))\n",
    "print(x.shape)\n",
    "first_img=x[0]\n",
    "first_label=y[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VovnOfwo_tY6"
   },
   "source": [
    "For grayscale images, `imshow` expects input of shape $H\\times W$, so we have to reshape the image:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "a0_9NtDC_asS"
   },
   "outputs": [],
   "source": [
    "plt.imshow(first_img.reshape(28,28), cmap='gray')\n",
    "print(\"Label: \", first_label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vqRvrjXZABq0"
   },
   "source": [
    "Now we go on and define our model: a MLP with two hidden layers of 32 and 16 neurons each, with ReLU activations. The input layer size is 784, since our images are 28x28, and the output layer has 10 neurons since we have 10 classes.\n",
    "\n",
    "ReLU is the Rectified Linear Unit, defined as:\n",
    "$$\n",
    "\\text{ReLU}(x)=\\cases{0\\hspace{0.5cm}\\text{if } x\\lt 0\\\\ x\\hspace{0.5cm}\\text{if } x\\ge 0}\n",
    "$$\n",
    "\n",
    "Note that the `Linear` layer of torch expects the input to be a vector, so in the `forward` method we have to remember to flatten the image into a 1-D vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "NYwfAmNcirct"
   },
   "outputs": [],
   "source": [
    "class Classifier(torch.nn.Module):\n",
    "    def __init__(self):\n",
    "        super().__init__()\n",
    "        self.layer1 = torch.nn.Linear(in_features=28*28, out_features=32, bias=True)\n",
    "        self.layer2 = torch.nn.Linear(in_features=32, out_features=16, bias=True)\n",
    "        self.layer3 = torch.nn.Linear(in_features=16, out_features=10, bias=True)\n",
    "        self.activation = torch.nn.ReLU()\n",
    "        # alternatively, self.activation = torch.nn.functional.relu\n",
    "\n",
    "    def forward(self, x):\n",
    "        x=x.reshape(-1,28*28)\n",
    "        x=self.activation(self.layer1(x))\n",
    "        x=self.activation(self.layer2(x))\n",
    "        x=self.layer3(x) # we don't need any nonlinearity (i.e. softmax) on the output layer. why?\n",
    "        return x\n",
    "\n",
    "model=Classifier()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eIpY--tGAmrC"
   },
   "source": [
    "Now we define our optimizer and the loss function that we want to minimize. Since we are doing 10-class classification we now switch to Cross-Entropy.\n",
    "\n",
    "You can find the details of this loss in torch [here](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html), but what is really important to point out is that it is assumed that the input to this loss are \"raw, unnormalized scores for each class\". This means that we shouldn't include any softmax in the network architecture, since it is already included by this implementation of the CE loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "z6o5fxaTjaBj"
   },
   "outputs": [],
   "source": [
    "optimizer=torch.optim.SGD(model.parameters(), lr=1e-2, momentum=0.9)\n",
    "loss=torch.nn.CrossEntropyLoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AAY0NaXqC0sh"
   },
   "source": [
    "We define a utility function to compute how accurate is our classifier.\n",
    "\n",
    "Note the `model.eval()` command and the context manager `with torch.no_grad()`.The former is telling the network that we are evaluating it, not training it: in this simple example there is no real use for this clarification, but we will see cases when this is crucial; the latter is switching off gradient operations: this allows you to make computations lighter and to avoid performing gradient descent steps by mistake."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "iLtD4F7MBTal"
   },
   "outputs": [],
   "source": [
    "def get_accuracy(model, dataloader):\n",
    "    model.eval()\n",
    "    with torch.no_grad():\n",
    "        correct=0\n",
    "        for x, y in iter(dataloader):\n",
    "            out=model(x)\n",
    "            correct+=(torch.argmax(out, axis=1)==y).sum()\n",
    "        return correct/len(dataloader.dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tlwO6tYsD1od"
   },
   "source": [
    "When training, we have to pay attention to specifying `model.train()`, the converse of `model.eval()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "SUbYx1z4jkcQ"
   },
   "outputs": [],
   "source": [
    "epochs=5\n",
    "losses=[]\n",
    "for epoch in range(epochs):\n",
    "    print(\"Test accuracy: \", get_accuracy(model, testloader))\n",
    "    model.train()\n",
    "    print(\"Epoch: \", epoch)\n",
    "    for x, y in iter(trainloader):\n",
    "        out=model(x)\n",
    "        l=loss(out, y)\n",
    "        optimizer.zero_grad()\n",
    "        l.backward()\n",
    "        optimizer.step()\n",
    "        losses.append(l.item())\n",
    "print(\"Final accuracy: \", get_accuracy(model, testloader))\n",
    "\n",
    "plt.figure()\n",
    "plt.title(\"MNIST batch loss\")\n",
    "plt.plot(losses)\n",
    "plt.xlabel(\"Optimization step\")\n",
    "plt.ylabel(\"CE Loss\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I3fCdvmGms18"
   },
   "source": [
    "# First assignment (deadline 13 November)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CuJZRcSFjbGx"
   },
   "source": [
    "\n",
    "- 1. Read carefully the paper [Learning representations by back-propagating errors](https://www.iro.umontreal.ca/~vincentp/ift3395/lectures/backprop_old.pdf)\n",
    "- 2. Reproduce in PyTorch (or any other DL library you like) experiment 1. Try to be as close as possible to the original protocol regarding network architecture, activation function, training algorithm and parameter initialization.\n",
    "    - Inspect the weights you obtained and check if they provide a solution to the problem\n",
    "    - Compare the solution to the solution reported in the paper\n",
    "- 3. Write a small report (1 page) about your experiment and what you\n",
    "learned about that. The report should be a jupyter notebook with text\n",
    "cells that describe the non-trivial parts of your work.\n",
    "\n",
    "- Tips and comments:\n",
    "    - Be careful: don't expect to be able to fully reproduce the results of the paper.\n",
    "    - Optionally, you are warmly encouraged to explore (and discuss in your report) possible workarounds to improve your results. Some of these may include:\n",
    "        - changing activation function;\n",
    "        - changing optimizer;\n",
    "        - adding a learning rate scheduler\n",
    "\n",
    "You can send the jupyter notebook in any format you prefer (`ipynb`, `pdf` or `html`) to lore.basile@outlook.com by 23.59, 13/11/2022.\n",
    "\n",
    "Also, use the same email address for any question regarding the assignment and coding.\n",
    "\n",
    "If you have more \"general\" doubts regarding the assignment itself, the paper or any other part of the course until now, please also include prof. Ansuini in the conversation (alessio.ansuini@areasciencepark.it)."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "OP0Bo0KjPoIi",
    "xsHzelLHVmI6"
   ],
   "private_outputs": true,
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}