[ET-VK][ez] Migrate slice/select shaders to use BufferMetadata/TextureMetadata #15796

SS-JIA · 2025-11-12T23:22:44Z

Stack from ghstack (oldest at bottom):

Title says it all!

Motivation: code simplification and allows these ops to handle high dim tensors.

Differential Revision: D86910641

As title. The current implementation of split_with_sizes uses functions from the `Copy.[h|cpp]` file in particular `add_copy_channel_offset_node`. However, the shaders dispatched by this function have a critical bug where the output tensor is passed in separately with difference access types, i.e. ```cpp graph.execute_nodes().emplace_back(new DispatchNode( graph, VK_KERNEL_FROM_STR(kernel_name), global_size, local_size, // Inputs and Outputs { {out, vkapi::kWrite}, {out, vkapi::kRead}, {in, vkapi::kRead}, }, ``` This creates many validation layer errors because the memory barriers for the resource cannot be formed properly. The shader essentially relies on undefined behaviour to work correctly To fix, this diff re-implements the operator from scratch with a dedicated compute shader. Differential Revision: [D86910642](https://our.internmc.facebook.com/intern/diff/D86910642/) [ghstack-poisoned]

As title. Make sure that ops that do not support quantized tensors do not get assigned memory layouts that are intended for quantized tensors. Differential Revision: [D86910639](https://our.internmc.facebook.com/intern/diff/D86910639/) [ghstack-poisoned]

Title says it all! Add two additional export options: 1. `skip_memory_planning` - skips the memory planning pass which can be useful for debugging. 2. `small_texture_limits` - sets the default texture limit to be (2048, 2048, 2048) which is compatible with more devices (i.e. desktop/laptop GPUs) compared to the default (16384, 16384, 2048) which is more targeted for mobile GPUs Also adds some improvements to the export script that were made while debugging the `YOLO_NAS` model (#15700) Differential Revision: [D86910640](https://our.internmc.facebook.com/intern/diff/D86910640/) [ghstack-poisoned]

…eMetadata Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/) [ghstack-poisoned]

pytorch-bot · 2025-11-12T23:22:48Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15796

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 9 New Failures, 4 Unrelated Failures

As of commit b6868b9 with merge base 7600df8 ():

NEW FAILURES - The following jobs have failed:

pull / test-moshi-linux / linux-job (gh)
RuntimeError: Could not load libtorchcodec. Likely causes:
Test CUDA Builds / test-model-cuda-e2e (google, gemma-3-4b-it, non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t 6270f1af75f393753f15bf649004df166e98df515e412905a7336c5aa8cde052 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (google, gemma-3-4b-it, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t 80e5dcbabdf3e919fe74c49eb2446b870ba49d05fbc81c9f81c1d3972e6a616f /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t 86fb05c13b4a5bd9525e500215fec2b5371dab3c61d75fe524e8c2143b436076 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t 30c905055a428b499fd8e6eff9a4d14574f13b8f0149b7ef982f1de20aad138f /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-large-v3-turbo, quantized-int4-weight-only) / linux-job (gh)
RuntimeError: Command docker exec -t eacfeae577af0c8d531b1bd4cb3e752dd054b942f0c4b1458487bf0da7fc9458 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, non-quantized) / linux-job (gh)
RuntimeError: Command docker exec -t 450c935c097c529a70f439ec0690b64ec8a37db4643ecc05f4f2f3db5780ea50 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, quantized-int4-tile-packed) / linux-job (gh)
RuntimeError: Command docker exec -t 385a5cb9de6fe690e99dbfc9e854a134e7e5850fe87ecb07bad489b179867b96 /exec failed with exit code 1
Test CUDA Builds / test-model-cuda-e2e (openai, whisper-small, quantized-int4-weight-only) / linux-job (gh)
RuntimeError: Command docker exec -t 8ebf52832d840cfcf1ccdb931c8f029ed6b503e9a1ec5c36fa3387144fbe5fad /exec failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest / linux / linux-job (gh) (trunk failure)
backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_int8_static_quant_recipe
pull / unittest / macos / macos-job (gh) (trunk failure)
backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_int8_static_quant_recipe
pull / unittest-editable / linux / linux-job (gh) (trunk failure)
backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_int8_static_quant_recipe
pull / unittest-editable / macos / macos-job (gh) (trunk failure)
backends/xnnpack/test/recipes/test_xnnpack_recipes.py::TestXnnpackRecipes::test_int8_static_quant_recipe

This comment was automatically generated by Dr. CI and updates every 15 minutes.

…eMetadata Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/) ghstack-source-id: 322864453 Pull Request resolved: #15796

github-actions · 2025-11-12T23:24:00Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

…o use BufferMetadata/TextureMetadata" Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/) [ghstack-poisoned]

…data/TextureMetadata" Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/) [ghstack-poisoned]

…eMetadata Pull Request resolved: #15796 Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. ghstack-source-id: 323042847 @exported-using-ghexport Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/)

…o use BufferMetadata/TextureMetadata" Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/) [ghstack-poisoned]

…data/TextureMetadata" Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/) [ghstack-poisoned]

…eMetadata Pull Request resolved: #15796 Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. ghstack-source-id: 323216101 @exported-using-ghexport Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/)

…o use BufferMetadata/TextureMetadata" Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/) [ghstack-poisoned]

…data/TextureMetadata" Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/) [ghstack-poisoned]

…eMetadata Pull Request resolved: #15796 Title says it all! Motivation: code simplification and allows these ops to handle high dim tensors. ghstack-source-id: 323317727 @exported-using-ghexport Differential Revision: [D86910641](https://our.internmc.facebook.com/intern/diff/D86910641/)

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #15829 * #15796 * #15795 * #15794 * __->__ #15793 As title. The current implementation of split_with_sizes uses functions from the `Copy.[h|cpp]` file in particular `add_copy_channel_offset_node`. However, the shaders dispatched by this function have a critical bug where the output tensor is passed in separately with difference access types, i.e. ```cpp graph.execute_nodes().emplace_back(new DispatchNode( graph, VK_KERNEL_FROM_STR(kernel_name), global_size, local_size, // Inputs and Outputs { {out, vkapi::kWrite}, {out, vkapi::kRead}, {in, vkapi::kRead}, }, ``` This creates many validation layer errors because the memory barriers for the resource cannot be formed properly. The shader essentially relies on undefined behaviour to work correctly To fix, this diff re-implements the operator from scratch with a dedicated compute shader. Differential Revision: [D86910642](https://our.internmc.facebook.com/intern/diff/D86910642/) --------- Co-authored-by: ssjia <ssjia@devvm26340.ftw0.facebook.com>

#15794) Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #15829 * #15796 * #15795 * __->__ #15794 * #15793 As title. Make sure that ops that do not support quantized tensors do not get assigned memory layouts that are intended for quantized tensors. Differential Revision: [D86910639](https://our.internmc.facebook.com/intern/diff/D86910639/) --------- Co-authored-by: ssjia <ssjia@devvm26340.ftw0.facebook.com>

) Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * #15829 * #15796 * __->__ #15795 * #15794 * #15793 Title says it all! Add two additional export options: 1. `skip_memory_planning` - skips the memory planning pass which can be useful for debugging. 2. `small_texture_limits` - sets the default texture limit to be (2048, 2048, 2048) which is compatible with more devices (i.e. desktop/laptop GPUs) compared to the default (16384, 16384, 2048) which is more targeted for mobile GPUs Also adds some improvements to the export script that were made while debugging the `YOLO_NAS` model (#15700) Differential Revision: [D86910640](https://our.internmc.facebook.com/intern/diff/D86910640/) --------- Co-authored-by: ssjia <ssjia@devvm26340.ftw0.facebook.com>

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #15829 * #15796 * #15795 * #15794 * #15793 Title says it all! Adds `int32` and `uint8` shader variants to a bunch of operators that don't currently have variants for these dtypes, but should. This should prevent folks from running into dtype crashes at runtime when using the Vulkan delegate. Differential Revision: [D87082724](https://our.internmc.facebook.com/intern/diff/D87082724/) Co-authored-by: ssjia <ssjia@devvm1479.ncg0.facebook.com>

Stack from [ghstack](https://github.com/ezyang/ghstack) (oldest at bottom): * __->__ #15829 * #15796 * #15795 * #15794 * #15793 Title says it all! Adds `int32` and `uint8` shader variants to a bunch of operators that don't currently have variants for these dtypes, but should. This should prevent folks from running into dtype crashes at runtime when using the Vulkan delegate. Differential Revision: [D87082724](https://our.internmc.facebook.com/intern/diff/D87082724/) Co-authored-by: ssjia <ssjia@devvm1479.ncg0.facebook.com> (cherry picked from commit a6c5921)

ssjia added 4 commits November 12, 2025 15:22

This was referenced Nov 12, 2025

[ET-VK] Re-implement split_with_sizes #15793

Merged

[ET-VK] Better separation of quantized vs non-quantized memory layouts #15794

Merged

SS-JIA mentioned this pull request Nov 12, 2025

[ET-VK] Add some utility compile options + improve export script #15795

Merged

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 12, 2025

meta-codesync bot added fb-exported meta-exported labels Nov 12, 2025

ssjia added 2 commits November 13, 2025 08:51

manuelcandales approved these changes Nov 13, 2025

View reviewed changes

ssjia added 2 commits November 13, 2025 19:33

ssjia added 2 commits November 14, 2025 07:31

SS-JIA mentioned this pull request Nov 14, 2025

[ET-VK] Add int and bool tensor support for many operators #15829

Merged

SS-JIA changed the base branch from gh/SS-JIA/372/base to main November 14, 2025 21:45

SS-JIA merged commit 053193f into main Nov 14, 2025
160 of 173 checks passed

SS-JIA deleted the gh/SS-JIA/372/head branch November 14, 2025 21:45

SS-JIA mentioned this pull request Nov 14, 2025

[ET-VK] Add int and bool tensor support for many operators #15834

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ET-VK][ez] Migrate slice/select shaders to use BufferMetadata/TextureMetadata #15796

[ET-VK][ez] Migrate slice/select shaders to use BufferMetadata/TextureMetadata #15796

SS-JIA commented Nov 12, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Nov 12, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[ET-VK][ez] Migrate slice/select shaders to use BufferMetadata/TextureMetadata #15796

[ET-VK][ez] Migrate slice/select shaders to use BufferMetadata/TextureMetadata #15796

Conversation

SS-JIA commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/15796

❌ 9 New Failures, 4 Unrelated Failures

Uh oh!

github-actions bot commented Nov 12, 2025

This PR needs a release notes: label

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

SS-JIA commented Nov 12, 2025 •

edited

Loading

pytorch-bot bot commented Nov 12, 2025 •

edited

Loading

This PR needs a `release notes:` label