Fix misleadingly high AOT Inductor dashboard performance #153060
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/153060
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ❌ 1 Cancelled Job, 1 Unrelated FailureAs of commit 07ca959 with merge base 7243c69 ( CANCELLED JOB - The following job was cancelled. Please retry:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
benjaminglass1
force-pushed
the
benjaminglass1/fixup-aot-inductor-performance-benchmarks
branch
2 times, most recently
from
7899276 to
4657eb9
Compare
benjaminglass1
force-pushed
the
benjaminglass1/fixup-aot-inductor-performance-benchmarks
branch
from
4657eb9 to
07ca959
Compare
colesbury
added
the
triaged
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 jobs have failed, first few of them are: trunk / linux-jammy-rocm-py3.10 / test (distributed, 1, 1, linux.rocm.gpu.4) Details for Dev Infra teamRaised by workflow job |
|
Test failure was an infra flake. @pytorchbot merge -i |
Merge startedYour change will be merged while ignoring the following 2 checks: inductor / unit-test / cuda12.6-py3.10-gcc9-sm86 / test (inductor_cpp_wrapper, 1, 2, lf.linux.g5.4xlarge.nvidia.gpu), trunk / linux-jammy-rocm-py3.10 / test (distributed, 1, 1, linux.rocm.gpu.4) Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Summary: Fixes misleadingly high AOTInductor performance benchmark numbers in scenarios where a model updates internal parameters during `torch.export.export`. Since `FakeTensorMode` is enabled during export, all such parameters become `FakeTensor`s, slowing down future eager-mode runs using that model substantively. This, in turn, causes misleading performance stats, where the slowness of eager-mode makes `AOTInductor` look _very_ good. An [example benchmark](https://hud.pytorch.org/benchmark/timm_models/inductor_aot_inductor?dashboard=torchinductor&startTime=Wed%2C%2030%20Apr%202025%2015%3A54%3A04%20GMT&stopTime=Wed%2C%2007%20May%202025%2015%3A54%3A04%20GMT&granularity=hour&mode=inference&dtype=bfloat16&deviceName=cuda%20(h100)&lBranch=main&lCommit=1dd36ad2d440a4f3faf724b3a8e13925e3180c24&rBranch=main&rCommit=cc7346bf19c019255dcb4484694a75850ed74d5a&model=convit_base) with this issue. The equivalent `cpp_wrapper` benchmark run shows a 2x performance gain, not 20x. Only two benchmarks we regularly run are affected by this, both in the TIMM set. X-link: pytorch/pytorch#153060 Approved by: https://github.com/desertfire Reviewed By: jeanschmidt Differential Revision: D74729281 fbshipit-source-id: bf25cd22933d9670018d935747b0604dec4178aa
github-actions
bot
deleted the
benjaminglass1/fixup-aot-inductor-performance-benchmarks
branch
5 participants
Fixes misleadingly high AOTInductor performance benchmark numbers in scenarios where a model updates internal parameters during
torch.export.export. SinceFakeTensorModeis enabled during export, all such parameters becomeFakeTensors, slowing down future eager-mode runs using that model substantively. This, in turn, causes misleading performance stats, where the slowness of eager-mode makesAOTInductorlook very good.An example benchmark with this issue. The equivalent
cpp_wrapperbenchmark run shows a 2x performance gain, not 20x.Only two benchmarks we regularly run are affected by this, both in the TIMM set.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang @amjames