Add torch._lazy_clone to create COW tensors

[kurtamohler]

Part of #109833

Stack from ghstack (oldest at bottom):

• -> Add torch._lazy_clone to create COW tensors #113397

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/113397

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (1 Unrelated Failure)

As of commit f4cd5b4 with merge base 6f8fc42 ():

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• periodic / win-vs2019-cuda11.8-py3 / test (default, 1, 4, windows.g5.4xlarge.nvidia.gpu) (gh)
  The process cannot access the file 'C:\actions-runner\_work\_actions\pytorch\pytorch\main\test\onnx_caffe2' because it is being used by another process.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[kurtamohler]

The failure in cuda12.1-py3.10-gcc9-sm86 / test (inductor, 1, 1, linux.g5.4xlarge.nvidia.gpu) is real. Running test_lazy_clone with PYTORCH_TEST_WITH_INDUCTOR=1 does not work, but for some reason, sometimes it says that the test passed even though error messages get printed out.

I just found #114012, which skips the inductor test. I'll add that change to this PR instead.

I'm not sure why the inductor tests are failing. I think it probably makes sense to fix it in a follow up

[kurtamohler]

I was able to split the test up into multiple tests to get them to pass when inductor is enabled. This note explains what went wrong and the strategy I used to fix it

[vadimkantorov]

Related idea on copy-on-write/move-semantics: #106584 (comment)

And expressing that clone is not always needed for currently always-copy functions, such as torch.cat with a single-tensor argument

[ezyang]

@pytorchbot merge

[pytorchmergebot]

Merge failed

Reason: This PR has internal changes and must be landed via Phabricator

Details for Dev Infra team

Raised by workflow job

[ezyang]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

Merge failed

Reason: 1 jobs have failed, first few of them are: .github/workflows/periodic.yml / win-vs2019-cuda11.8-py3 / test (default, 1, 4, windows.g5.4xlarge.nvidia.gpu)

Details for Dev Infra team

Raised by workflow job

[ezyang]

@pytorchbot merge -f "trunk flakiness only"

[pytorchmergebot]

Merge started

Your change will be merged immediately since you used the force (-f) flag, bypassing any CI checks (ETA: 1-5 minutes). Please use -f as last resort and instead consider -i/--ignore-current to continue the merge ignoring current failures. This will allow currently pending tests to finish and report signal before the merge.

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here