Dedup delegate blobs in emitter #14564

lucylq · 2025-09-24T23:16:16Z

Summary:
Previously we deduplicated entire 'BackendDelegate' blobs using the preprocessed blob. If two BackendDelegate fields have different id or compile specs, it would be disregarded.

This diff:

Only deduplicates the preprocessed blob. BackendDelegate retains its own compile specs, etc.
Removes the per-method 'delegate_cache', as we have a program-wide delegate cache.
Adds a test to confirm we have one delegate segment but two BackendDelegate references pointing at it.

Differential Revision: D83162107

pytorch-bot · 2025-09-24T23:16:20Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14564

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Cancelled Job, 1 Unrelated Failure

As of commit 50fb2d9 with merge base 684b5fd ():

NEW FAILURE - The following job has failed:

pull / test-samsung-models-linux / linux-job (gh)
RuntimeError: Command docker exec -t 8e240fed8c39ebe5258caa0c383433a03db7a0763cdfa3a9087e37e46932fee8 /exec failed with exit code 1

CANCELLED JOB - The following job was cancelled. Please retry:

pull / test-binary-size-linux-gcc / linux-job (gh)

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / test-setup-linux-gcc / linux-job (gh) (trunk failure)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-09-24T23:16:25Z

@lucylq has exported this pull request. If you are a Meta employee, you can view the originating diff in D83162107.

github-actions · 2025-09-24T23:17:01Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Summary: Previously we deduplicated entire 'BackendDelegate' blobs using the preprocessed blob. If two BackendDelegate fields have different id or compile specs, it would be disregarded. This diff: 1. Only deduplicates the preprocessed blob. BackendDelegate retains its own compile specs, etc. 2. Removes the per-method 'delegate_cache', as we have a program-wide delegate cache. 3. Adds a test to confirm we have one delegate segment but two BackendDelegate references pointing at it. Differential Revision: D83162107

facebook-github-bot · 2025-09-24T23:55:52Z

@lucylq has exported this pull request. If you are a Meta employee, you can view the originating diff in D83162107.

cccclai · 2025-09-26T19:14:56Z

exir/emit/_emitter.py

        processed_bytes = lowered_module.processed_bytes
        hashed = hashlib.sha256(processed_bytes).hexdigest()
-        delegate_index = self.emitter_state.delegate_cache.get(hashed)
+        delegate_index = self.program_state.backend_delegate_data_cache.get(hashed)


what's the difference between emitter state and program state? How is backend_delegate_data_cache different than delegate_cache

emitter state is per-method, program state covers all the methods in the program

cccclai · 2025-09-26T19:17:09Z

exir/emit/_emitter.py

-                    BackendDelegateInlineData(data=processed_bytes)
-                )
-
-            backend_delegate = BackendDelegate(


I'm not quite sure why the previous logic didn't work but the new one does. It seems the logic is the same here

The previous logic refers to the BackendDelegate created for the deduplicated processed blob. We may have a different compile specs, but the same processed blob, in which case the compile specs are lost. You can take a look at the test case as well, and try it on the old code.

cccclai · 2025-09-26T19:17:45Z

exir/emit/_emitter.py

-            Instruction(DelegateCall(delegate_index=delegate_index, args=delegate_args))
+            Instruction(
+                DelegateCall(
+                    delegate_index=len(self.emitter_state.delegates) - 1,


hmm why is the delegate index defined as len(self.emitter_state.delegates) - 1?

the new logic creates a separate BackendDelegate for each call_delegate, so it corresponds to the length self.emitter_state.delegates.

cccclai · 2025-09-26T19:20:01Z

exir/emit/test/test_emit.py

+
+        plan = program.execution_plan[0]
+        # Two delegates that point to the same blob.
+        self.assertEqual(len(plan.delegates), 2)


Is it for checking the number of call_delegate instruction?

Summary: Previously we deduplicated entire 'BackendDelegate' blobs using the preprocessed blob. If two BackendDelegate fields have different id or compile specs, but the same preprocessed blob, we would take the first one and use it in the execution plan. The id/compile specs of the second would be lost. This diff: 1. Only deduplicates the preprocessed blob. BackendDelegate retains its own compile specs, etc. 2. Removes the per-method 'delegate_cache', as we have a program-wide delegate cache. 3. Adds a test to confirm we have one delegate segment but two BackendDelegate references pointing at it. Reviewed By: JacobSzwejbka Differential Revision: D83162107

facebook-github-bot · 2025-09-26T20:10:55Z

@lucylq has exported this pull request. If you are a Meta employee, you can view the originating diff in D83162107.

Summary: Previously we deduplicated entire 'BackendDelegate' blobs using the preprocessed blob. If two BackendDelegate fields have different id or compile specs, but the same preprocessed blob, we would take the first one and use it in the execution plan. The id/compile specs of the second would be lost. This diff: 1. Only deduplicates the preprocessed blob. BackendDelegate retains its own compile specs, etc. 2. Removes the per-method 'delegate_cache', as we have a program-wide delegate cache. 3. Adds a test to confirm we have one delegate segment but two BackendDelegate references pointing at it. Reviewed By: JacobSzwejbka Differential Revision: D83162107

facebook-github-bot · 2025-09-26T20:57:51Z

@lucylq has exported this pull request. If you are a Meta employee, you can view the originating diff in D83162107.

lucylq · 2025-09-29T16:32:58Z

@pytorchbot cherry-pick --onto release/1.0 -c critical

Differential Revision: D83162107 Pull Request resolved: #14564 (cherry picked from commit fd5f946)

pytorchbot · 2025-09-29T16:35:17Z

Cherry picking #14564

The cherry pick PR is at #14658 and it is recommended to link a critical cherry pick PR with an issue. The following tracker issues are updated:

[v1.0.0] Release Tracker #14288 (comment)

Details for Dev Infra team

Raised by workflow job

lucylq requested review from JacobSzwejbka and larryliu0820 as code owners September 24, 2025 23:16

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 24, 2025

facebook-github-bot added fb-exported meta-exported labels Sep 24, 2025

lucylq force-pushed the export-D83162107 branch from 2712cac to f1663df Compare September 24, 2025 23:55

lucylq requested a review from cccclai September 26, 2025 17:01

JacobSzwejbka approved these changes Sep 26, 2025

View reviewed changes

cccclai reviewed Sep 26, 2025

View reviewed changes

lucylq force-pushed the export-D83162107 branch from f1663df to b4738be Compare September 26, 2025 20:10

lucylq force-pushed the export-D83162107 branch from b4738be to 50fb2d9 Compare September 26, 2025 20:57

facebook-github-bot merged commit fd5f946 into pytorch:main Sep 27, 2025
128 of 132 checks passed

pytorchbot pushed a commit that referenced this pull request Sep 29, 2025

Dedup delegate blobs in emitter

5dca271

Differential Revision: D83162107 Pull Request resolved: #14564 (cherry picked from commit fd5f946)

pytorchbot mentioned this pull request Sep 29, 2025

[v1.0.0] Release Tracker #14288

Open

Dedup delegate blobs in emitter #14564

Dedup delegate blobs in emitter #14564

Uh oh!

Conversation

lucylq commented Sep 24, 2025

Uh oh!

pytorch-bot bot commented Sep 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/14564

❌ 1 New Failure, 1 Cancelled Job, 1 Unrelated Failure

Uh oh!

facebook-github-bot commented Sep 24, 2025

Uh oh!

github-actions bot commented Sep 24, 2025

This PR needs a release notes: label

Uh oh!

facebook-github-bot commented Sep 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Sep 26, 2025

Uh oh!

facebook-github-bot commented Sep 26, 2025

Uh oh!

Uh oh!

lucylq commented Sep 29, 2025

Uh oh!

pytorchbot commented Sep 29, 2025

Cherry picking #14564

Uh oh!

Uh oh!

pytorch-bot bot commented Sep 24, 2025 •

edited

Loading

This PR needs a `release notes:` label