Fake tensor error from `aten._cudnn_rnn`

[jansel]

cc @eellison

./tests/test_torchinductor.py -vsk test_cudnn_rnn_cuda

Traceback (most recent call last):
  File "/home/jansel/torchdynamo/torchdynamo/variables/tensor.py", line 153, in create
    example_value = cls.wrap_fake_exception(
  File "/home/jansel/torchdynamo/torchdynamo/variables/tensor.py", line 102, in wrap_fake_exception
    return fn()
  File "/home/jansel/torchdynamo/torchdynamo/variables/tensor.py", line 154, in <lambda>
    lambda: cls.run_proxy(proxy, args, kwargs, nnmodule)
  File "/home/jansel/torchdynamo/torchdynamo/variables/tensor.py", line 84, in run_proxy
    return proxy.node.target(*args, **kwargs)
  File "/home/jansel/pytorch/torch/_ops.py", line 148, in __call__
    return self._op(*args, **kwargs or {})
  File "/home/jansel/pytorch/torch/utils/_python_dispatch.py", line 74, in wrapped
    return f(self, *args, **kwargs)
  File "/home/jansel/pytorch/torch/_subclasses/fake_tensor.py", line 480, in __torch_dispatch__
    r = run_fallback_kernel(func, args, kwargs, not_implemented_error)
  File "/home/jansel/pytorch/torch/_subclasses/fake_tensor.py", line 526, in run_fallback_kernel
    raise orig_not_implemented_exception
  File "/home/jansel/pytorch/torch/_subclasses/fake_tensor.py", line 476, in __torch_dispatch__
    r = func(*args, **kwargs)
  File "/home/jansel/pytorch/torch/_ops.py", line 49, in __call__
    return self._op(*args, **kwargs or {})
NotImplementedError: Could not run 'aten::_cudnn_rnn' with arguments from the 'Meta' backend. This could be because the operator doesn't exist for this backend, or was omitted during the selective/custom build process (if using custom build). If you are a Facebook employee using PyTorch on mobile, please visit https://fburl.com/ptmfixes for possible resolutions. 'aten::_cudnn_rnn' is only available for these backends: [FPGA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseCPU, SparseCUDA, SparseHIP, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, SparseXPU, UNKNOWN_TENSOR_TYPE_ID, SparseVE, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, NestedTensorCPU, NestedTensorCUDA, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, UNKNOWN_TENSOR_TYPE_ID, AutogradCPU].

[eellison]

Hey, this is fixed on a more recent version of Pytorch.

[jansel]

Ah, thanks -- I'll pull and try again.

[jansel]

@eellison I am still getting this after pulling.

empire:~/pytorch$ git log | head -n3
commit 0468dafeaadf72e66567bf81935552e4b4bb4cef
Author: Teddy Koker <teddy.koker@gmail.com>
Date:   Mon Jun 27 19:57:53 2022 +0000

[eellison]

Fixed with pytorch/pytorch#80545

[jansel]

Thanks Elias!

[malfet]

Reverted pytorch/pytorch#80545 a min ago as breaking ROCm...