Cover caller-stream TLS semantics (#21291)

[shoumikhin]

Summary:

Add focused coverage for the caller-stream contract relied on by external CUDA
backends. Verify that an explicitly selected null/default stream remains
distinct from no guard, and that selections are isolated per execution thread.
Keep the fbcode and xplat mirrors identical.

Differential Revision: D113382078

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21291

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 6 Pending, 1 Unclassified Failure

As of commit b01f69a with merge base 8134bb2 ():

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

• Test CUDA Builds / test-models-cuda / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command docker exec -t c1980bf11e09a0ba49c701e2f457260f4b06fe6e985cd1d289b84d27eed8361f /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@shoumikhin has exported this pull request. If you are a Meta employee, you can view the originating Diff in D113382078.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[Copilot]

Pull request overview

This PR strengthens CUDA delegate correctness in coalesced / ATen-host scenarios by (1) adding focused tests for the caller-selected CUDA stream TLS contract and (2) ensuring tensor device metadata and AOTInductor shim symbol resolution are propagated consistently across runtime, backends, and build systems.

Changes:

• Add unit tests validating that CallerStreamGuard distinguishes “explicitly selected nullptr stream” from “no guard”, and that selection is thread-local.
• Propagate serialized device metadata into ATen tensor deserialization and preserve device tags when sharing tensor storage.
• Introduce/standardize an executorch_ AOTI shim symbol prefix (Buck + CMake + Python), and add an ATen-mode Buck target for the CUDA backend.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
runtime/executor/tensor_parser_aten.cpp	Parse extra_tensor_info device fields and rebuild tensors with consistent device/dispatch metadata.
runtime/core/exec_aten/util/tensor_util_aten.cpp	Preserve source tensor device when sharing storage DataPtr.
backends/cuda/runtime/utils.h	Switch helper APIs to use mode-flexible executorch::aten::* tensor types.
backends/cuda/runtime/TARGETS	Add shim-prefix flags propagation and introduce cuda_backend_aten Buck target.
backends/cuda/runtime/shims/memory.h	Declare aoti_torch_empty_strided_pinned shim entrypoint.
backends/cuda/runtime/shims/memory.cpp	Implement pinned allocator shim (currently falls back to pageable allocation).
backends/cuda/runtime/cuda_backend.cpp	Register backend in ET_RUNTIME_NAMESPACE, avoid .so temp path collisions, and tighten CUDA IO device validation.
backends/cuda/cuda_backend.py	Set aot_inductor.shim_symbol_prefix and update custom-op shim symbol names to executorch_*.
backends/cuda/CMakeLists.txt	Add AOTI_SHIM_SYMBOL_PREFIX compile definition for CUDA shims.
backends/aoti/targets.bzl	Apply shim-prefix flags to AOTI common slim shims and force retention via link_whole.
backends/aoti/shim_symbol_prefix.bzl	Centralize the shim symbol prefix and corresponding preprocessor flag list.
backends/aoti/export.h	Add macro-based symbol renaming for aoti_torch_* shims when a prefix is defined.
backends/aoti/CMakeLists.txt	Apply AOTI_SHIM_SYMBOL_PREFIX to AOTI slim shims (incl. MSVC object-lib path).
backends/aoti/slim/cuda/test/test_cuda_stream_guard.cpp	Add TLS semantics tests for caller stream selection (including cross-thread isolation).
backends/aoti/aoti_delegate_handle.h	Use executorch::aten::Tensor as the mode-flexible host-tensor handle type.
backends/cuda/runtime/shims/tensor_attribute.h	Remove CUDA-specific tensor-attribute shim (now covered by common slim shims).
backends/cuda/runtime/shims/tensor_attribute.cpp	Remove CUDA-specific tensor-attribute shim implementation.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[shoumikhin]

Fixed: changed AOTI_SHIM_SYMBOL_PREFIX on aoti_cuda_shims from PRIVATE to PUBLIC. Direct consumers that include the shim headers (the shim gtests) now compile the declarations under the same executorch_ prefix and link the DLL's prefixed exports, matching the Buck target which already exports the define to header users.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.

[shoumikhin]

This is a false positive: in FlatBuffers, schema type byte maps to signed int8_t (it's ubyte that is uint8_t), and the generated ExtraTensorInfo.device_index accessor returns int8_t. So there is no uint8->int8 reinterpretation on the cast — a value like 200 cannot be stored in this field; the wire value is already a signed int8. The validation (device_type == CPU || device_index >= 0) correctly rejects a negative accelerator index (including the -1 "current device" sentinel) from an untrusted PTE, and the error prints the true int8 value. No change needed.