[inductor] Slightly faster memory allocation on CPU #118171
Conversation
[ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/118171
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 8f31079 with merge base d59c2d6 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
ghstack-source-id: 22c9c87d75ed07ed3288ffa9d908d281afb7e10c Pull Request resolved: #118171
torch/csrc/dynamo/guards.cpp
Outdated
|
|
||
| static PyObject* _empty_strided_cpu(PyObject* dummy, PyObject* args) { | ||
| // at::empty_strided is surprising slow. This is a lower-overhead | ||
| // version that saves ~2us on every allocation. Though it is |
There was a problem hiding this comment.
I guess the extra overhead comes from the ATen dispatching which should apply to GPU too but GPU can overlap the device and host compute.
Based on `python benchmarks/dynamo/microbenchmarks/overheads.py`: - Before `12.2us` - After `10.7us` This is inspired by a2c17a2 -- but in Python rather than C++ [ghstack-poisoned]
ghstack-source-id: 6b6f39e6a4d82437dce4d04b7a22e4ac6395fbeb Pull Request resolved: #118171
Based on `python benchmarks/dynamo/microbenchmarks/overheads.py`: - Before `12.2us` - After `10.5us` This is inspired by a2c17a2 -- but in Python rather than C++ [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 1 jobs have failed, first few of them are: trunk / macos-12-py3-arm64 / build Details for Dev Infra teamRaised by workflow job |
Based on `python benchmarks/dynamo/microbenchmarks/overheads.py`: - Before `12.2us` - After `10.5us` This is inspired by a2c17a2 -- but in Python rather than C++ cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler [ghstack-poisoned]
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Pull Request resolved: #118255 Approved by: https://github.com/peterbell10 ghstack dependencies: #118065, #118070, #118171
Stack from ghstack (oldest at bottom):
Based on
python benchmarks/dynamo/microbenchmarks/overheads.py:12.2us10.5usThis is inspired by a2c17a2 -- but in Python rather than C++
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler