Add __torch_function__ support for generated tensor methods/property of PrivateUse1

[cdzhan]

support following case:

import torch
...
class CustomFooTensor(torch.Tensor):
  @classmethod
  def __torch_function__(cls, func, types, args=(), kwargs=None):
    ...
a = CustomFooTensor([3])
print(a.is_foo)

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/121723

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (4 Unrelated Failures)

As of commit b422679 with merge base a56e057 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• pull / linux-focal-cuda12.1-py3.10-gcc9 / test (default, 5, 5, linux.4xlarge.nvidia.gpu) (gh)
  test_foreach.py::TestForeachCUDA::test_binary_op_list_error_cases__foreach_add_cuda_complex64
• pull / linux-focal-py3.12-clang10 / test (default, 1, 3, linux.2xlarge) (gh)
  RuntimeError: profiler/test_profiler 1/1 failed
• trunk / linux-focal-cuda12.1-py3.10-gcc9 / test (nogpu_AVX512, 1, 1, linux.2xlarge) (gh)
  RuntimeError: profiler/test_profiler 1/1 failed
• trunk / linux-focal-cuda12.1-py3.10-gcc9 / test (nogpu_NO_AVX2, 1, 1, linux.2xlarge) (gh)
  RuntimeError: profiler/test_profiler 1/1 failed

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[linux-foundation-easycla]

The committers listed above are authorized under a signed CLA.

• ✅ login: cdzhan (82ed0d2, b422679)

[albanD]

I don't think we should add this attribute because:

• It moves us away from device generic code
• it doesn't make sense once the privateuse1 module is renamed.

[cdzhan]

I don't think we should add this attribute because:

• It moves us away from device generic code
• it doesn't make sense once the privateuse1 module is renamed.

Thanks for your comment! Please let me know If there is anything else that needs to be modified.

[albanD]

Sorry, but I don't think we want this at all. You can use t.device.type == some_device_type to get the same thing in a more generic way. So I'm not sure we really want to add more of these one-off attributes.

[cdzhan]

I thought about this question again, but a related more general question came to mind #123245 @albanD

[cdzhan]

Thanks to #122468 's clarification of the gpu_migration method, I can express my motivation more clearly. Just because is_cuda has this attribute, if PrivateUse1 do not have this, extra change may need to be made and users have to write more device-specific code. After all, we can't predict how users will use those APIs. None of us are fans of gpu_migration, but it may be a good short term opt-in. @albanD

[albanD]

Ok, I still think we want to remove a lot of these device-specific method/attributes.
But I agree that adding torch_function support to these existing one is an ok thing to do.

[albanD]

@pytorchbot merge -r

[pytorch-bot]

Pull workflow has not been scheduled for the PR yet. It could be because author doesn't have permissions to run those or skip-checks keywords were added to PR/commits, aborting merge. Please get/give approval for the workflows and/or remove skip ci decorators before next merge attempt. If you think this is a mistake, please contact PyTorch Dev Infra.

[seemethere]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased privateuse_zcd onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via git checkout privateuse_zcd && git pull --rebase)

[albanD]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here