Assigning parameter to tensor doesn't pass gradient info

[InfProbSciX]

Issue description

In the example below, assigning a parameter to tensor doesn't seem to pass gradient information, unless a list is used for indexing.

Code example

import torch
torch.__version__  # '1.11.0'
z = torch.zeros(3, 2)

z[:, 0] = torch.nn.Parameter(torch.tensor(1.0))
z

# tensor([[1., 0.],
#         [1., 0.],
#         [1., 0.]])  # <------------

z[:, [0]] = torch.nn.Parameter(torch.tensor(1.0))
z

# tensor([[1., 0.],
#         [1., 0.],
#         [1., 0.]], grad_fn=<CopySlices>)  # <------------

... is this behaviour expected?

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @lezcano @Varal7

[ezyang]

No, non-advanced indexing should propagate gradients, looks like a bug.

[albanD]

Note that it doesn't require to use Parameter, the same thing happens with plain Tensors that require gradients.
Also this only happens if we use a scalar Tensor, so I guess some convertion to plain python Number in the advanced indexing code?

Note that as a workaround, you can use torch.nn.Parameter(torch.tensor((1.0,))) that is a 1D Tensor with a single value.

[yuguo68]

Thank you @InfProbSciX for reporting the bug and the reproducing code. It seems the root cause is here

pytorch/aten/src/ATen/TensorIndexing.h

Lines 356 to 358 in b447114

	} else if (src.sizes().size() == 0 && src.device().type() == at::kCPU) {
	dst.fill_(src.item());
	return;

, indeed it is "some convertion to plain python Number" suggested by @albanD. Also, if the scalar tensor is on GPU, we have the gradient info. I will have a pr to fix.

[albanD]

Fixed in master now!