Skip to content

Arm backend: Fuse canonical flow-offset grid sampling - #21775

Merged
YufengShi-dudu merged 1 commit into
pytorch:mainfrom
YufengShi-dudu:fuse-canonical-flow-offset-grid-sampling
Aug 12, 2026
Merged

Arm backend: Fuse canonical flow-offset grid sampling#21775
YufengShi-dudu merged 1 commit into
pytorch:mainfrom
YufengShi-dudu:fuse-canonical-flow-offset-grid-sampling

Conversation

@YufengShi-dudu

@YufengShi-dudu YufengShi-dudu commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Recognize grid_sample calls that add a two-channel flow slice to an align-corners base grid:

base_grid ---------\
                     add -> permute -> grid_sample(image, grid)
flow[:, i:i+2] ----/

Replace the complete expression with:

image, flow -> fused flow-offset custom shader

This avoids materializing the normalized sampling grid at runtime.

Require SNORM-compatible image and output quantization ranges. Vulkan's signed normalized 8-bit image format maps both -128 and -127 to -1.0, so only fuse tensors using the range [-127, 127]. The flow tensor keeps the full int8 range because the shader reads it directly as a tensor.

If glslc is unavailable or compilation fails, leave the graph unchanged for generic grid-sampler lowering.

Cover supported rewrites, invalid patterns, SNORM ranges, and compiler fallback behavior.

Authored with Codex.

Change-Id: Ia19b4387c169bd24f8db837a5bfee7c91d4e55ce

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218 @mansnils @Sebastian-Larsson @robell @rascani

@YufengShi-dudu YufengShi-dudu added partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm ciflow/trunk module: arm Issues related to arm backend release notes: none Do not include this in the release notes labels Aug 12, 2026
@pytorch-bot

pytorch-bot Bot commented Aug 12, 2026

Copy link
Copy Markdown

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21775

Note: Links to docs will display an error until the docs builds have been completed.

❌ 8 Pending, 1 Unclassified Failure

As of commit b34ec8b with merge base 8de0262 (image):

UNCLASSIFIED FAILURE - DrCI could not classify the following job because the workflow did not run on the merge base. The failure may be pre-existing on trunk or introduced by this PR:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 12, 2026
Recognize grid_sample calls that add a two-channel flow slice to an
align-corners base grid:

    base_grid ---------\
                         add -> permute -> grid_sample(image, grid)
    flow[:, i:i+2] ----/

Replace the complete expression with:

    image, flow -> fused flow-offset custom shader

This avoids materializing the normalized sampling grid at runtime.

Require SNORM-compatible image and output quantization ranges. Vulkan's
signed normalized 8-bit image format maps both -128 and -127 to -1.0,
so only fuse tensors using the range [-127, 127]. The flow tensor keeps
the full int8 range because the shader reads it directly as a tensor.

If glslc is unavailable or compilation fails, leave the graph unchanged
for generic grid-sampler lowering.

Cover supported rewrites, invalid patterns, SNORM ranges, and compiler
fallback behavior.

Authored with Codex.

Change-Id: Ia19b4387c169bd24f8db837a5bfee7c91d4e55ce
Signed-off-by: Yufeng Shi <yufeng.shi@arm.com>
@YufengShi-dudu
YufengShi-dudu force-pushed the fuse-canonical-flow-offset-grid-sampling branch from 6976e03 to b34ec8b Compare August 12, 2026 12:17

@robell robell left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extensive internal review.

@YufengShi-dudu

Copy link
Copy Markdown
Collaborator Author

unrelated failures

@YufengShi-dudu
YufengShi-dudu merged commit bf2103d into pytorch:main Aug 12, 2026
493 of 496 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. module: arm Issues related to arm backend partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm release notes: none Do not include this in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants