[inductor] [cpp] fix the input contiguous check in max-autotune

[chunyuan-w]

Stack from ghstack (oldest at bottom):

• -> [inductor] [cpp] fix the input contiguous check in max-autotune #134982

Description

Fixes the FP32 accuracy failure of resmlp_12_224 and BF16 accuracy failure of volo_d1_224 in timm.

In this PR, we check whether input is contiguous using the following way:
If it has FixedLayout, we know the accurate strides. For FlexibleLayout, if its data is a ComputedBuffer, we could get the fill order of the buffer to decide whether it's contiguous. For the other cases, we won't use GEMM template as we can't infer whether it's contiguous.

Additional context

The current GEMM template only supports this case: input.get_stride()[-1] == 1. In resmlp_12_224, when we run into this check, the layout of input is a FlexibleLayout. The reason is that when realizing the input which is a View IR, the convert_to_reinterpret_view call fails:

pytorch/torch/_inductor/ir.py

Lines 4712 to 4715 in d14fe3f

	try:
	return cls.convert_to_reinterpret_view(x)
	except NotImplementedError:
	pass

And it finally runs into this copy_input and returns a FlexibleLayout.

pytorch/torch/_inductor/ir.py

Line 4722 in d14fe3f

return cls.copy_input(x)

When checking its stride, this FlexibleLayout indeed satisfies input.get_stride()[-1] == 1 but it is later decided as a FixedLayout with size = (3072, 196), stride = (1, 3072), which is not supported by the GEMM template, thus causing accuracy issue in this model.
The FlexibleLayout is converted to FixedLayout during CppPackedGemmTemplate.add_choices which calls slice_nd when rendering the kernel (slice_nd(X)). When creating the SliceView IR, as_storage_and_layout invokes
decide_layout and converts it to a FixedLayout with size = (3072, 196), stride = (1, 3072).

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @ColinPeppler @amjames @desertfire @chauhang

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/134982

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 8f2b3f8 with merge base 217ba7b ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[chunyuan-w]

@pytorchbot merge

[pytorchmergebot]

Merge failed

Reason: Approvers from one of the following sets are needed:

• superuser (pytorch/metamates)
• Core Reviewers (mruberry, lezcano, Skylion007, ngimel, peterbell10, ...)
• Core Maintainers (soumith, gchanan, ezyang, dzhulgakov, malfet, ...)

Details for Dev Infra team

Raised by workflow job

Failing merge rule: Core Maintainers

[jansel]

If this is failing, should we be calling freeze_layout? Flexible layouts are allowed to change, so any check you do could become false later on. If you have a FlexibleLayout you can force the last dim to be stride=1 without a copy.

[chunyuan-w]

Thanks for the suggestion. I've updated the code. Could you help take another look?

[chunyuan-w]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here