Skip kernel saving if already existed.

[muchulee8]

Stack from ghstack (oldest at bottom):

• -> Skip kernel saving if already existed. #136389

Summary:
We skip the save_gpu_kernel if kernel is being saved already.
This would give us a more accurate Triton profiling result. The following trace shows before/after the change for a benchmarking of a trivial addmm:

Before:

After:

We can see that before the change, the benchmarking includes two parts,
(1) The overhead of our triton_heuristic call, which includes the save/get, and the (expensive) hash computation.
(2) The exact computation of Triton kernel.

We see that (1) accounts >50% of time, which makes kernel selection for profiling often choose aten kernels over Triton kernels.

Test Plan:
Existing OSS CI
[Redacted, Some internal model results in D63441430]

Reviewers:

Subscribers:

Tasks:

Tags:

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @yf225 @chenyang78 @kadeng @ColinPeppler @amjames @desertfire @chauhang @aakhundov @chuanhaozhuge

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/136389

• 📄 Preview Python docs built from this PR
• 📄 Preview C++ docs built from this PR
• ❓ Need help or want to give feedback on the CI? Visit the bot commands wiki or our office hours

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit 2dd8d48 with merge base 08dba25 ():

NEW FAILURES - The following jobs have failed:

• inductor-periodic / cuda12.1-py3.10-gcc9-sm80 / test (inductor_torchbench_smoketest_perf, 1, 1, linux.gcp.a100) (gh)
  moco
• Lint / Test run_test.py is usable without boto3/rockset (gh)
  ERROR: Could not find a version that satisfies the requirement torch (from versions: none)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[muchulee8]

@muchulee8 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[muchulee8]

@pytorchbot rebase

[pytorchmergebot]

@pytorchbot started a rebase job onto refs/remotes/origin/viable/strict. Check the current status here

[pytorchmergebot]

Successfully rebased gh/muchulee8/36/orig onto refs/remotes/origin/viable/strict, please pull locally before adding more changes (for example, via ghstack checkout https://github.com/pytorch/pytorch/pull/136389)

[desertfire]

This should only matter for AOTI (or JIT+cpp_wrapper) where store_cubin is True?

[muchulee8]

This should only matter for AOTI (or JIT+cpp_wrapper) where store_cubin is True?

Yes, this diff only matters when store_cubin is True. This is intentional to keep the blast radius small. We should refactor the triton_heuristic if we want a really accurate result, there's actually one more non-trivial overhead which is the grid_fn computation, which can reach to ~15% of exact kernel runtime, but that's not as bad as this save_gpu_kernel (which is more than 100%)

[muchulee8]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[pytorchmergebot]

The merge job was canceled or timed out. This most often happen if two merge requests were issued for the same PR, or if merge job was waiting for more than 6 hours for tests to finish. In later case, please do not hesitate to reissue the merge command
For more information see pytorch-bot wiki.

[muchulee8]

@pytorchbot merge

[pytorchmergebot]

Merge started

Your change will be merged once all checks pass (ETA 0-4 Hours).

Learn more about merging in the wiki.

Questions? Feedback? Please reach out to the PyTorch DevX Team

Advanced Debugging

Check the merge workflow status
here

[muchulee8]

@pytorchbot revert -m Issue #136940

[pytorch-bot]

❌ 🤖 pytorchbot command failed:

@pytorchbot revert: error: the following arguments are required: -c/--classification

usage: @pytorchbot revert -m MESSAGE -c
                          {nosignal,ignoredsignal,landrace,weird,ghfirst}

Try @pytorchbot --help for more info.

[muchulee8]

@pytorchbot revert -m Issue #136940 -c nosignal

[pytorch-bot]

❌ 🤖 pytorchbot command failed:

@pytorchbot: error: unrecognized arguments: https://github.com/pytorch/pytorch/issues/136940

usage: @pytorchbot [-h] {merge,revert,rebase,label,drci,cherry-pick,close} ...

Try @pytorchbot --help for more info.

[muchulee8]

@pytorchbot revert -m "Issue #136940 " -c nosignal

[pytorchmergebot]

@pytorchbot successfully started a revert job. Check the current status here.
Questions? Feedback? Please reach out to the PyTorch DevX Team

[pytorchmergebot]

@muchulee8 your PR has been successfully reverted.

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

[aakhundov]

@muchulee8 are we going to land this? My understanding was that excessive save_gpu_kernel call leads to a substantial distortion in autotuning results in AOTI scenarios.