Don't run fallback if symbolic sizes in fake tensor

[ezyang]

Stack from ghstack (oldest at bottom):

• -> Don't run fallback if symbolic sizes in fake tensor #97148

Signed-off-by: Edward Z. Yang ezyang@meta.com

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/97148

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 026d4c4:
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[ezyang]

@pytorchbot merge

[pytorchmergebot]

Merge failed

Reason: This PR needs a label
If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

Details for Dev Infra team

Raised by workflow job

[bdhirsh]

side note: I wonder if we should avoid the entire try + except and fallback if we see an operator that's not part ATen (aka all custom ops)

There was at least one example github issue of torch.compile failing with an awful error, because we tried to plumb fake tensors and the custom op wasn't written in a way to handle them.

I would probably take the hit of having some custom ops stop working automatically (with a loud error saying they need a meta function impl, and some docs on how to add one), over some custom ops silently giving bad error messages / segfaults.

[ezyang]

@pytorchbot merge

[ezyang]

side note: I wonder if we should avoid the entire try + except and fallback if we see an operator that's not part ATen (aka all custom ops)

Yes, our coverage is good enough that I will support removing the fallback entirely. @eellison WDYT

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[eellison]

I'm not in favor of that because of all the custom operators that don't have meta funcs.
Pros: dont IMA on operators that would IMA on all zeros
Cons: lose large amount of coverage on user operators

IMO the IMA zeros case is much, much less frequent.

[ezyang]

But IMA zeros is much, much worse, whereas missing coverage is kind of meh and kind of easy to fix.