Skip to content

executorch: make CUDA backend and partitioner targets PUBLIC - #21411

Merged
meta-codesync[bot] merged 1 commit into
pytorch:mainfrom
shoumikhin:export-D113389646
Jul 27, 2026
Merged

executorch: make CUDA backend and partitioner targets PUBLIC#21411
meta-codesync[bot] merged 1 commit into
pytorch:mainfrom
shoumikhin:export-D113389646

Conversation

@shoumikhin

@shoumikhin shoumikhin commented Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary

Make the ExecuTorch CUDA backend's Python export targets publicly visible.

backends/cuda/BUCK now sets cuda_backend and cuda_partitioner to visibility = ["PUBLIC"] (previously ["//executorch/..."]). These are the Python export-side targets: the CudaBackend delegate and the CudaPartitioner used at .pte export time.

Motivation

A consumer outside //executorch/... (the Torch-TensorRT ExecuTorch composition test, which builds a coalesced TensorRT + CUDA .pte via CudaPartitioner) needs to depend on these targets. PUBLIC is how ExecuTorch already exposes this kind of target: the CoreML backend/partitioner, the MPS and MediaTek backends, and the CUDA runtime targets (cuda_platform, runtime_shims, cuda_allocator, and the C++ cuda_backend) are all PUBLIC. Using it here avoids ad-hoc per-consumer visibility grants that would need editing every time a new consumer appears.

Test plan

Visibility-only change; no code behavior is affected. buck2 cquery on the consuming composition test resolves its dependencies on cuda_backend and cuda_partitioner with no visibility violation.

@shoumikhin @facebook-github-bot
executorch: persist external .ptd weights and preserve lifted constan…
ff51ee2 
…t dtype/device (PR pytorch#4404)

Summary:
Forward-port of upstream pytorch/TensorRT PR pytorch#4404, re-anchored on top of the
pristine release/2.13 cut. PR pytorch#4404 is now merged upstream (merge commit
8ab76546); the vendored code here is byte-identical to the merged files (the two
new test files, _exporter.py, test_api.py additions, and docs match exactly, and
_compile.py carries the merged PR's additions verbatim). This is the FULL PR:
upstream converged on the partitioners= approach and dropped the earlier
cuda_fallback kwarg, so a coalesced TensorRT + CUDA .pte is built via
save(..., partitioners=[CudaPartitioner(...)]) appended after the
TensorRTPartitioner.

Fix 1 (py/torch_tensorrt/_compile.py): write the external weight .ptd.
torch_tensorrt.save(..., output_format="executorch") wrote only the .pte and
dropped ExecutorchProgramManager._tensor_data, so any program whose delegate
externalizes weight constants to a separate data file had no backing blob at
load. The CUDA (AOTInductor) backend externalizes its weights
(save_data_externally / package_constants_in_so=False) as <hash>_weights_blob
.ptd files; without the .ptd the delegated kernel dereferenced a null weights
pointer (illegal memory access) even though the .pte itself loaded fine. Factored
into _write_external_tensor_data(program, file_path), called after write_to_file:
it writes the .ptd data file(s) into the .pte's directory when _tensor_data is
present (a no-op otherwise), and logs where it wrote. _tensor_data is read
directly (no getattr default) so a future rename fails loudly instead of silently
reintroducing the crash. The loader must be pointed at the .ptd (e.g. via the
runtime Module's data-files argument); co-locating them with the .pte is the
convention this write follows.

Fix 2 (py/torch_tensorrt/dynamo/_exporter.py, lift): keep lifted constant device
and dtype. The meta val for a lifted parameter/buffer was built with
torch.empty_strided(shape, strides), dropping the source tensor's dtype and
device. A non-float32 or non-CPU lifted constant then got wrong meta, which
matters once the program is placed on an accelerator. The current PR fakifies the
source constant through the graph's own fake_mode
(fake_mode.from_tensor(lift_val, static_shapes=True)), preserving dtype, device,
and stride without allocating a real tensor on the source device.

Tests + docs (mirrored from pytorch#4404):
- tests/py/dynamo/executorch/test_api.py, test_edge_cases.py: CPU-only unit tests
  (no GPU/CUDA backend needed) for lift() dtype/device preservation and the .ptd
  write (present -> writes, empty -> no-op, missing attr -> fails loud).
- tests/py/dynamo/executorch/test_cuda_partitioner_composition.py: GPU-gated
  composition suite -- a CudaPartitioner catch-all routes a TRT-unsupported op to
  the CUDA backend, a weighted CUDA partition persists its external .ptd, and a
  TRT-only program writes none. Buckified here (new python_pytest_remote_gpu
  target) so the suite that upstream GitHub CI runs is also covered in fbsource;
  this needs the executorch CUDA-backend python targets (cuda_backend,
  cuda_partitioner) visible to third-party/torch_tensorrt, so their visibility is
  widened in executorch/backends/cuda/BUCK.
- docsrc/user_guide/runtime_performance/saving_models.rst: the CudaPartitioner
  coalescing recipe.

Vendored as a temporary forward-port on top of the pristine release/2.13 cut,
tracked in fb/patches/upstream/4404-executorch-write-ptd-and-exporter-device.patch
and IMPORT.md; drop it when the next 2.13 import already contains pytorch#4404. The patch
and its description are public-clean.

Reviewed By: SS-JIA

Differential Revision: D113389646
Copilot AI review requested due to automatic review settings July 27, 2026 20:15
@pytorch-bot

pytorch-bot Bot commented Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21411

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 1 Unrelated Failure

As of commit ff51ee2 with merge base 55d693b (image):

NEW FAILURES - The following jobs have failed:

  • Cadence Build & Test / hifi-build / hifi4 (gh)
    Refusing to check out fork pull request code from a 'pull_request_target' workflow. This workflow runs with the base repository's GITHUB_TOKEN, secrets, default-branch cache scope, and runner access. Fetching and executing a fork's code in that trusted context commonly leads to "pwn request" vulnerabilities. To opt in, review the risks at https://gh.io/securely-using-pull_request_target and set 'allow-unsafe-pr-checkout: true' on the actions/checkout step.
  • Cadence Build & Test / vision-build / vision (gh)
    Refusing to check out fork pull request code from a 'pull_request_target' workflow. This workflow runs with the base repository's GITHUB_TOKEN, secrets, default-branch cache scope, and runner access. Fetching and executing a fork's code in that trusted context commonly leads to "pwn request" vulnerabilities. To opt in, review the risks at https://gh.io/securely-using-pull_request_target and set 'allow-unsafe-pr-checkout: true' on the actions/checkout step.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 27, 2026
@meta-codesync

meta-codesync Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

@shoumikhin has exported this pull request. If you are a Meta employee, you can view the originating Diff in D113389646.

SS-JIA
SS-JIA approved these changes Jul 27, 2026
View reviewed changes

@SS-JIA SS-JIA left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review automatically exported from Phabricator review in Meta.

@github-actions

github-actions Bot commented Jul 27, 2026

Copy link
Copy Markdown

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

@shoumikhin shoumikhin changed the title executorch: persist external .ptd weights and preserve lifted constant dtype/device (PR #4404) executorch: make CUDA backend and partitioner targets PUBLIC Jul 27, 2026
Copilot AI reviewed Jul 27, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts ExecuTorch’s Buck build visibility for CUDA backend Python targets, making them consumable by external Buck packages (e.g., third-party integrations).

Changes:

  • Widen cuda_backend Buck target visibility to PUBLIC.
  • Widen cuda_partitioner Buck target visibility to PUBLIC.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread backends/cuda/BUCK
@meta-codesync
meta-codesync Bot merged commit 1f0b2dc into pytorch:main Jul 27, 2026
209 of 217 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
@SS-JIA SS-JIA SS-JIA approved these changes

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@shoumikhin @SS-JIA