jacrev and jacfwd trigger INTERNAL ASSERT FAILED when changing the .data of input tensor

[cafffeeee]

🐛 Describe the bug

jacrev and jacfwd trigger INTERNAL ASSERT FAILED when changing the .data of input tensor

import torch
from torch.func import jacrev, jacfwd

torch.manual_seed(420)

p = torch.nn.Parameter(torch.ones(2, 3))

def func(p):
    p.data = torch.ones(1, 2, 3) * 2
    return p

jacrev(func)(p)
# RuntimeError: false INTERNAL ASSERT FAILED 
# at "/opt/conda/conda-bld/pytorch_1672906354936/work/aten/src/ATen/functorch/TensorWrapper.cpp":137, 
# please report a bug to PyTorch. NYI

For such cases, the jacobian just ignores the operation on the data attribute (so it fails the gradcheck). I am not sure which way is better.

Versions

PyTorch version: 2.0.0.dev20230105
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.1 LTS (x86_64)
GCC version: (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
Clang version: Could not collect
CMake version: version 3.22.1
Libc version: glibc-2.35

Python version: 3.9.15 (main, Nov 24 2022, 14:31:59)  [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.15.0-56-generic-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 11.7.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA GeForce RTX 3090
GPU 1: NVIDIA GeForce RTX 3090
GPU 2: NVIDIA GeForce RTX 3090

Nvidia driver version: 515.86.01
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn_train.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_infer.so.8.4.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops_train.so.8.4.1
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] torch==2.0.0.dev20230105
[pip3] torchaudio==2.0.0.dev20230105
[pip3] torchvision==0.15.0.dev20230105
[conda] blas                      1.0                         mkl
[conda] mkl                       2021.4.0           h06a4308_640
[conda] mkl-service               2.4.0            py39h7f8727e_0
[conda] mkl_fft                   1.3.1            py39hd3c417c_0
[conda] mkl_random                1.2.2            py39h51133e4_0
[conda] numpy                     1.23.5           py39h14f4228_0
[conda] numpy-base                1.23.5           py39h31eccc5_0
[conda] pytorch                   2.0.0.dev20230105 py3.9_cuda11.7_cudnn8.5.0_0    pytorch-nightly
[conda] pytorch-cuda              11.7                 h67b0de4_2    pytorch-nightly
[conda] pytorch-mutex             1.0                        cuda    pytorch-nightly
[conda] torchaudio                2.0.0.dev20230105      py39_cu117    pytorch-nightly
[conda] torchtriton               2.0.0+0d7e753227            py39    pytorch-nightly
[conda] torchvision               0.15.0.dev20230105      py39_cu117    pytorch-nightly

cc @zou3519 @Chillee @samdow @soumith @kshitij12345 @janeyx99

[cafffeeee]

Besides, LazyBatchNorm will also trigger this INTERNAL ASSERT

import torch
from torch.func import jacrev, jacfwd

torch.manual_seed(420)

x = torch.randn(2, 3, requires_grad=True)

def func(x):
    m = torch.nn.LazyBatchNorm1d(3)
    out = m(x)
    return out

jacrev(func)(x)
# RuntimeError: false INTERNAL ASSERT FAILED 
# at "/opt/conda/conda-bld/pytorch_1672906354936/work/aten/src/ATen/functorch/TensorWrapper.cpp":137, 
# please report a bug to PyTorch. NYI