[dynamic shapes] unbacked-safe slicing #161414

pianpwk · 2025-08-25T16:43:43Z

Summary:
Generates new unbacked symbols for slice output size & storage offset, when appropriate semantics are unclear. Teaches inductor to codegen the slice with flexible semantics.

Test Plan:
contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/56218d85e2da09d9ede3809718ec989c2151632c

Rollback Plan:

Differential Revision: D80948073

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben

pytorch-bot · 2025-08-25T16:43:46Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/161414

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 08cf5ae with merge base 84b57c9 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-08-25T16:43:54Z

This pull request was exported from Phabricator. Differential Revision: D80948073

Summary: Generates new unbacked symbols for slice output size & storage offset, when appropriate semantics are unclear. Teaches inductor to codegen the slice with flexible semantics. Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/56218d85e2da09d9ede3809718ec989c2151632c Rollback Plan: Differential Revision: D80948073

facebook-github-bot · 2025-09-03T16:36:57Z

This pull request was exported from Phabricator. Differential Revision: D80948073

pianpwk · 2025-09-08T22:57:25Z

torch/_inductor/lowering.py

+
+    # realize to get strides/storage offset
+    if x.maybe_get_layout() is None:
+        x.realize()


@eellison I'm not very familiar with inductor lowering, so I wasn't sure if this was the right approach; the goal was to call as_strided with codegened sizes/strides/offset, but later figured I had to call realize(), on inputs like ir.Pointwise.

maybe this case should be lowering in a different way?

laithsakka · 2025-09-15T22:04:10Z

torch/_subclasses/fake_impls.py

+        elif guard_or_false(start_index >= end_index):
+            new_size = 0
+
+    # create unbacked if case unknown


nit: unknown -> undecided due to unbacked shapes.

laithsakka · 2025-09-15T22:09:41Z

torch/_inductor/lowering.py

 def slice_(x, dim=0, start=0, end=2**63, step=1, clamp=True):
+    """
+    Lowers a slice call, creating ExternKernels for the output size & storage offset symbols,
+    if the indices are unbacked and appropriate semantics aren't known.


nit: semantics are the same for known and unknown.

maybe:
creating ExternKernels to compute the output size & storage offset symbols dynamically if they cant be determined at compile time because indices are unbacked.

laithsakka · 2025-09-15T22:10:37Z

torch/_inductor/lowering.py

+        ):
+            return x
+    except TypeError:
+        pass


why would a type error be thrown? statically_known_leq should be resilient to unbacked.

laithsakka · 2025-09-15T22:18:24Z

torch/_inductor/lowering.py

+                sym_storage = sym
+
+    if V.graph.current_node is None or not clamp or (sym_size is None and sym_storage is None):
+        return TensorBox(


// This path is taken if no unbacked symbols has been allocated meaning that we know how to determine things statically and no need for dynamic computations.

laithsakka · 2025-09-15T22:18:32Z

torch/_inductor/lowering.py

+    except TypeError:
+        pass
+
+    # try to avoid dynamic (unbacked) slice


this does not seem the appropriate line for this comment?

laithsakka · 2025-09-15T22:18:50Z

torch/_inductor/lowering.py

+        pass
+
+    # try to avoid dynamic (unbacked) slice
+    def compute_slice_index(index, size, default=None):


this is only called once with default being 0?
sounds like we never call it with index None in this version?

laithsakka

approved but please address my comments before landing

Summary: Generates new unbacked symbols for slice output size & storage offset, when appropriate semantics are unclear. Teaches inductor to codegen the slice with flexible semantics. Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/56218d85e2da09d9ede3809718ec989c2151632c Rollback Plan: Reviewed By: laithsakka Differential Revision: D80948073

facebook-github-bot · 2025-09-16T21:13:04Z

@pianpwk has exported this pull request. If you are a Meta employee, you can view the originating diff in D80948073.

Summary: Generates new unbacked symbols for slice output size & storage offset, when appropriate semantics are unclear. Teaches inductor to codegen the slice with flexible semantics. Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/56218d85e2da09d9ede3809718ec989c2151632c Rollback Plan: Reviewed By: laithsakka Differential Revision: D80948073

facebook-github-bot · 2025-09-16T21:24:07Z

@pianpwk has exported this pull request. If you are a Meta employee, you can view the originating diff in D80948073.

Summary: Generates new unbacked symbols for slice output size & storage offset, when appropriate semantics are unclear. Teaches inductor to codegen the slice with flexible semantics. Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/56218d85e2da09d9ede3809718ec989c2151632c Reviewed By: laithsakka Differential Revision: D80948073

facebook-github-bot · 2025-09-23T04:02:57Z

@pianpwk has exported this pull request. If you are a Meta employee, you can view the originating diff in D80948073.

Summary: Generates new unbacked symbols for slice output size & storage offset, when appropriate semantics are unclear. Teaches inductor to codegen the slice with flexible semantics. Test Plan: contbuild & OSS CI, see https://hud.pytorch.org/commit/pytorch/pytorch/56218d85e2da09d9ede3809718ec989c2151632c Reviewed By: laithsakka Differential Revision: D80948073

facebook-github-bot · 2025-09-29T16:24:25Z

@pianpwk has exported this pull request. If you are a Meta employee, you can view the originating diff in D80948073.

facebook-github-bot · 2025-09-29T23:11:48Z

@pytorchbot merge

(Initiating merge automatically since Phabricator Diff has merged)

pytorchmergebot · 2025-09-29T23:13:42Z

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

pytorch-bot bot added ci-no-td Do not run TD on this PR ciflow/inductor module: inductor release notes: inductor (aoti) labels Aug 25, 2025

facebook-github-bot added the fb-exported label Aug 25, 2025

facebook-github-bot force-pushed the export-D80948073 branch from 3787eed to fa9e353 Compare September 3, 2025 16:36

pianpwk changed the title ~~[reland] Dynamic shapes: unbacked-safe slicing (#157944) (#157944)~~ [dynamic shapes] unbacked-safe slicing Sep 8, 2025

pytorch-bot bot added the ciflow/trunk Trigger trunk jobs on your pull request label Sep 8, 2025

pianpwk requested review from bobrenjc93, eellison and laithsakka September 8, 2025 22:54

pianpwk commented Sep 8, 2025

View reviewed changes

laithsakka reviewed Sep 15, 2025

View reviewed changes

laithsakka approved these changes Sep 15, 2025

View reviewed changes

pianpwk force-pushed the export-D80948073 branch from b79015a to 6267444 Compare September 16, 2025 21:12

facebook-github-bot added the meta-exported label Sep 16, 2025

facebook-github-bot force-pushed the export-D80948073 branch from 6267444 to 4830a22 Compare September 16, 2025 21:23

facebook-github-bot force-pushed the export-D80948073 branch from 4830a22 to 12b4954 Compare September 23, 2025 04:02

laithsakka mentioned this pull request Sep 27, 2025

narrow with tensor size/end and tensor_split with tensor dim are not unbacked friendly #158081

Open

laithsakka added the data dependent error label Sep 27, 2025

facebook-github-bot force-pushed the export-D80948073 branch from 12b4954 to 08cf5ae Compare September 29, 2025 16:23

pytorchmergebot added the merging label Sep 29, 2025

pytorchmergebot added the Merged label Sep 30, 2025

pytorchmergebot closed this in 474d075 Sep 30, 2025

pytorchmergebot removed the merging label Sep 30, 2025

[dynamic shapes] unbacked-safe slicing #161414

[dynamic shapes] unbacked-safe slicing #161414

Uh oh!

Conversation

pianpwk commented Aug 25, 2025 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Aug 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/161414

✅ No Failures

Uh oh!

facebook-github-bot commented Aug 25, 2025

Uh oh!

facebook-github-bot commented Sep 3, 2025

Uh oh!

pianpwk Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

laithsakka Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

laithsakka Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

laithsakka Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

laithsakka Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

laithsakka Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

laithsakka Sep 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

laithsakka left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Sep 16, 2025

Uh oh!

facebook-github-bot commented Sep 16, 2025

Uh oh!

facebook-github-bot commented Sep 23, 2025

Uh oh!

facebook-github-bot commented Sep 29, 2025

Uh oh!

facebook-github-bot commented Sep 29, 2025

Uh oh!

pytorchmergebot commented Sep 29, 2025

Merge started

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

pianpwk commented Aug 25, 2025 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Aug 25, 2025 •

edited

Loading

laithsakka Sep 15, 2025 •

edited

Loading