Gradient Descent, Signs of Healthy Training: Need to detach() tensor.

[ethman]

We need to detach a tensor in the "Signs of Healthy Training" section on the Gradient Descent page.

Here's the cell:

fig, ax = plt.subplots(1, 2, figsize=(15, 5))

for j, lr in enumerate(LEARNING_RATES):
    ax[0].plot(grad_norms[j], label=f'Learning rate: {lr}')
    ax[0].legend()
    ax[0].set_xlabel('Iteration')
    ax[0].set_ylabel('grad norm')
    ax[0].set_title('Gradient norm for each learning rate')
    
for j, lr in enumerate(LEARNING_RATES):
    ax[1].plot(np.log10(losses[j]), label=f'Learning rate: {lr}')
    ax[1].legend()
    ax[1].set_xlabel('Iteration')
    ax[1].set_ylabel('log(loss)')
    ax[1].set_title('Loss for each learning rate')
plt.show()

And the cell's current output:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-18-91a4bf5517c3> in <module>
      9 
     10 for j, lr in enumerate(LEARNING_RATES):
---> 11     ax[1].plot(np.log10(losses[j]), label=f'Learning rate: {lr}')
     12     ax[1].legend()
     13     ax[1].set_xlabel('Iteration')

/opt/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/site-packages/torch/tensor.py in __array__(self, dtype)
    619             return handle_torch_function(Tensor.__array__, (self,), self, dtype=dtype)
    620         if dtype is None:
--> 621             return self.numpy()
    622         else:
    623             return self.numpy().astype(dtype, copy=False)

RuntimeError: Can't call numpy() on Tensor that requires grad. Use tensor.detach().numpy() instead.