[inductor][ez] add template/externchoice uid #161341

coconutruben · 2025-08-23T02:57:41Z

Stack from ghstack (oldest at bottom):

why

to have a central registry of templates/externkernelchoice
to match them to heuristics etc, they need unique names
mm is both the triton template name and the aten_mm name

what

add a uid() to KernelTemplate/ExternKernelChoice that returns name
override in ExternKernel to prepend "aten::"
override in TritonTemplate to prepend "triton::"

This id is just use to find template heuristics, so it has no other
impact

testing

python3 -bb -m pytest test/inductor/test_max_autotune.py -v

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov

Differential Revision: D81520579

\# why - to have a central registry of templates/externkernelchoice to match them to heuristics etc, they need unique names - mm is both the triton template name and the aten_mm name \# what - add a uid() to KernelTemplate/ExternKernelChoice that returns name - override in ExternKernel to prepend "aten::" - override in TritonTemplate to prepend "triton::" This id is just use to find template heuristics, so it has no other impact \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` [ghstack-poisoned]

pytorch-bot · 2025-08-23T02:57:44Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/161341

📄 Preview Python docs built from this PR
📄 Preview C++ docs built from this PR
❓ Need help or want to give feedback on the CI? Visit the bot commands wiki

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 5199ce1 with merge base 5da573c ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

\# why - to have a central registry of templates/externkernelchoice to match them to heuristics etc, they need unique names - mm is both the triton template name and the aten_mm name \# what - add a uid() to KernelTemplate/ExternKernelChoice that returns name - override in ExternKernel to prepend "aten::" - override in TritonTemplate to prepend "triton::" This id is just use to find template heuristics, so it has no other impact \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx ipiszy chenyang78 kadeng muchulee8 amjames chauhang aakhundov [ghstack-poisoned]

…ytorch#161344) # why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior # what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520573](https://our.internmc.facebook.com/intern/diff/D81520573) Pull Request resolved: pytorch#161344 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343

# why - remove repeat patterns - we have everything to make the choicecallers - templates - input_nodes - layouts - all the kwargs # what - yield a choicecaller directly from V.choices.get_mm_configs # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520577](https://our.internmc.facebook.com/intern/diff/D81520577) Pull Request resolved: pytorch#161345 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344

…h#161346) # why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: pytorch#161346 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: pytorch#161347 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346

…pytorch#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: pytorch#161348 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346, pytorch#161347

# why - to have a central registry of templates/externkernelchoice to match them to heuristics etc, they need unique names - mm is both the triton template name and the aten_mm name # what - add a uid() to KernelTemplate/ExternKernelChoice that returns name - override in ExternKernel to prepend "aten::" - override in TritonTemplate to prepend "triton::" This id is just use to find template heuristics, so it has no other impact # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520579](https://our.internmc.facebook.com/intern/diff/D81520579) Pull Request resolved: pytorch#161341 Approved by: https://github.com/jansel, https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340

# why - central point to analyze and override all generated choices # what - add a pseudo heuristic for aten that just yields a single, empty kwargs - add a pseudo heuristic with the bias_addmm logic for it - add an addmm specific heuristic that yields a single choice, but also expands it with alpha and beta kwargs - replace all the aten.bind calls with V.choices.get_mm_configs using the now matching API for aten # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520580](https://our.internmc.facebook.com/intern/diff/D81520580) Pull Request resolved: pytorch#161342 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341

# why - simpler interface - enables future of extracting more things out of the template e.g. a hash # what V.choices.get_mm_configs now takes the whole template rather than just the template.uid # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520576](https://our.internmc.facebook.com/intern/diff/D81520576) Pull Request resolved: pytorch#161343 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342

…ytorch#161344) # why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior # what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520573](https://our.internmc.facebook.com/intern/diff/D81520573) Pull Request resolved: pytorch#161344 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343

# why - remove repeat patterns - we have everything to make the choicecallers - templates - input_nodes - layouts - all the kwargs # what - yield a choicecaller directly from V.choices.get_mm_configs # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520577](https://our.internmc.facebook.com/intern/diff/D81520577) Pull Request resolved: pytorch#161345 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344

…h#161346) # why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: pytorch#161346 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: pytorch#161347 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346

…pytorch#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: pytorch#161348 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346, pytorch#161347

# why - to have a central registry of templates/externkernelchoice to match them to heuristics etc, they need unique names - mm is both the triton template name and the aten_mm name # what - add a uid() to KernelTemplate/ExternKernelChoice that returns name - override in ExternKernel to prepend "aten::" - override in TritonTemplate to prepend "triton::" This id is just use to find template heuristics, so it has no other impact # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520579](https://our.internmc.facebook.com/intern/diff/D81520579) Pull Request resolved: pytorch#161341 Approved by: https://github.com/jansel, https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340

# why - central point to analyze and override all generated choices # what - add a pseudo heuristic for aten that just yields a single, empty kwargs - add a pseudo heuristic with the bias_addmm logic for it - add an addmm specific heuristic that yields a single choice, but also expands it with alpha and beta kwargs - replace all the aten.bind calls with V.choices.get_mm_configs using the now matching API for aten # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520580](https://our.internmc.facebook.com/intern/diff/D81520580) Pull Request resolved: pytorch#161342 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341

# why - simpler interface - enables future of extracting more things out of the template e.g. a hash # what V.choices.get_mm_configs now takes the whole template rather than just the template.uid # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520576](https://our.internmc.facebook.com/intern/diff/D81520576) Pull Request resolved: pytorch#161343 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342

…ytorch#161344) # why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior # what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520573](https://our.internmc.facebook.com/intern/diff/D81520573) Pull Request resolved: pytorch#161344 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343

# why - remove repeat patterns - we have everything to make the choicecallers - templates - input_nodes - layouts - all the kwargs # what - yield a choicecaller directly from V.choices.get_mm_configs # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520577](https://our.internmc.facebook.com/intern/diff/D81520577) Pull Request resolved: pytorch#161345 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344

…h#161346) # why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: pytorch#161346 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: pytorch#161347 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346

…pytorch#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: pytorch#161348 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346, pytorch#161347

# why - to have a central registry of templates/externkernelchoice to match them to heuristics etc, they need unique names - mm is both the triton template name and the aten_mm name # what - add a uid() to KernelTemplate/ExternKernelChoice that returns name - override in ExternKernel to prepend "aten::" - override in TritonTemplate to prepend "triton::" This id is just use to find template heuristics, so it has no other impact # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520579](https://our.internmc.facebook.com/intern/diff/D81520579) Pull Request resolved: pytorch#161341 Approved by: https://github.com/jansel, https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340

# why - central point to analyze and override all generated choices # what - add a pseudo heuristic for aten that just yields a single, empty kwargs - add a pseudo heuristic with the bias_addmm logic for it - add an addmm specific heuristic that yields a single choice, but also expands it with alpha and beta kwargs - replace all the aten.bind calls with V.choices.get_mm_configs using the now matching API for aten # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520580](https://our.internmc.facebook.com/intern/diff/D81520580) Pull Request resolved: pytorch#161342 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341

# why - simpler interface - enables future of extracting more things out of the template e.g. a hash # what V.choices.get_mm_configs now takes the whole template rather than just the template.uid # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520576](https://our.internmc.facebook.com/intern/diff/D81520576) Pull Request resolved: pytorch#161343 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342

…ytorch#161344) # why - heuristics providers know decide whether to (or which choices to add) in the max-autotune case - enables an eventual override point to gracefully fallback to the standard behavior # what - max-autotune is determined inside V.choices.get_mm_configs because it's mm only right now, we can just do `config.max_autotune or config.max_autotune_gemm` a TODO indicates that this can change in the future when this expands to more templates # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520573](https://our.internmc.facebook.com/intern/diff/D81520573) Pull Request resolved: pytorch#161344 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343

# why - remove repeat patterns - we have everything to make the choicecallers - templates - input_nodes - layouts - all the kwargs # what - yield a choicecaller directly from V.choices.get_mm_configs # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520577](https://our.internmc.facebook.com/intern/diff/D81520577) Pull Request resolved: pytorch#161345 Approved by: https://github.com/jansel ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344

…h#161346) # why - enables us to just gather relevant templates and get all choices at once - that in turns allows us to make op wide override decisions # what - V.choice.get_mm_configs takes a stack of templates - all callsites just provide a stack of size 1 right now but do not merge everything yet (other features pending) # testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520583](https://our.internmc.facebook.com/intern/diff/D81520583) Pull Request resolved: pytorch#161346 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345

# why - gather everything up to make choices, without running potentially expensive generators - enables overrides where we toss the entire list of configs from inductor, without having to enumrate it (expensive) # what - add a holding class that just gets all the components necessary to generate a ChoiceCaller - use that class to generate ChoiceCallers - this does not (yet) add the override function, but just prepares the scene ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520569](https://our.internmc.facebook.com/intern/diff/D81520569) Pull Request resolved: pytorch#161347 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346

…pytorch#161348) \# why - every callsite just executes the generator on the spot - previous pr adds the ability to add an override before expensive generators are executed, so we don't need this generator anymore \# what - rather than yielding the ChoiceCaller, just return the list of all valid ChoiceCallers \# testing ``` python3 -bb -m pytest test/inductor/test_max_autotune.py -v ``` Differential Revision: [D81520574](https://our.internmc.facebook.com/intern/diff/D81520574) Pull Request resolved: pytorch#161348 Approved by: https://github.com/eellison ghstack dependencies: pytorch#162075, pytorch#161340, pytorch#161341, pytorch#161342, pytorch#161343, pytorch#161344, pytorch#161345, pytorch#161346, pytorch#161347

pytorch-bot bot added ciflow/inductor module: inductor labels Aug 23, 2025

coconutruben added the topic: not user facing topic category label Aug 23, 2025

coconutruben added 2 commits August 22, 2025 20:05

github-actions bot deleted the gh/coconutruben/42/head branch October 6, 2025 02:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[inductor][ez] add template/externchoice uid #161341

[inductor][ez] add template/externchoice uid #161341

Uh oh!

coconutruben commented Aug 23, 2025 •

edited

Loading

Uh oh!

pytorch-bot bot commented Aug 23, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[inductor][ez] add template/externchoice uid #161341

[inductor][ez] add template/externchoice uid #161341

Uh oh!

Conversation

coconutruben commented Aug 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

why

what

testing

Uh oh!

pytorch-bot bot commented Aug 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/161341

✅ No Failures

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

coconutruben commented Aug 23, 2025 •

edited

Loading

pytorch-bot bot commented Aug 23, 2025 •

edited

Loading