fix(torch): honor handle streams in generated operators - #880
Merged
Conversation
voltjia
force-pushed
the
fix/torch-backend-stream
branch
14 times, most recently
from
August 11, 2026 05:41
5d4f0dc to
a074719
Compare
voltjia
force-pushed
the
fix/torch-backend-stream
branch
from
August 11, 2026 10:05
a074719 to
232dda7
Compare
voltjia
force-pushed
the
fix/torch-backend-stream
branch
from
August 11, 2026 10:18
232dda7 to
3f5b8b7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
src/linked/torchtosrc/torch, while updating the existing linked providers to include the new paths.GetStreamFromExternaland adding only the build wiring and tests required by that behavior.Motivation
Generated ATen backends received the InfiniOps handle stream through
stream_, but they invoked PyTorch without installing that stream in C10. PyTorch work could therefore run on a different stream from the caller-selected InfiniOps stream and violate the caller's ordering assumptions.The existing C10 adapters already map a null external stream to the platform's default stream. Using the same external-stream path for both cases keeps the call sites uniform and preserves the established C10 API instead of introducing another stream-selection abstraction.
No issue is linked.
Type of Change
feat- New feature, operator, or platform.fix- Bug fix.perf- Performance improvement without a behavioral change.refactor- Code restructuring without a behavioral change.test- Adding or fixing tests only.docs- Documentation only.build/ci- Build system or CI configuration.chore- Tooling, formatting, or other non-code changes.Platforms Affected
WITH_CPU).WITH_NVIDIA).WITH_ILUVATAR).WITH_METAX).WITH_CAMBRICON).WITH_MOORE).WITH_ASCEND).WITH_TORCH).Smoke Test Result
Test Results on Supported Platforms
Full `pytest` output (optional)
Benchmark / Performance Impact
N/A. This PR fixes stream ordering and moves existing C10 helpers without changing operator algorithms.
Notes for Reviewers
This PR intentionally contains only two logical changes. The first commit moves the C10 helpers that already existed for NVIDIA, MetaX, Cambricon, and Moore. The second commit uses those helpers around generated ATen calls and adds the build wiring and regression coverage required for that use.
No C10 specialization is added for CPU, Iluvatar, Hygon, or Ascend. Generated operators on those platforms retain the previous execution path through the discarded
if constexprbranch, and additional platform adapters can be handled in separate PRs when needed.The stream guard covers both
ToAtenTensorconversion and the ATen call. A nullstream_is passed directly to the existingGetStreamFromExternalimplementation, which selects the default C10 stream on the supported adapters, while a non-null value imports the caller-provided native stream.The CMake changes are limited to dependencies that become direct requirements outside linked providers. MetaX links its C10 CUDA bridge and runtime adapter, Cambricon makes the existing
torch_mlusetup available to generated Torch sources, and Moore reuses the existingtorch_musadiscovery while linking its stream implementation.