Skip to content

[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for constants at inference time #112405

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 22 commits into from
Closed

[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for constants at inference time #112405

wants to merge 22 commits into from

Conversation

swolchok
Copy link
Contributor

@swolchok swolchok commented Oct 30, 2023 •
edited
Loading

Stack from ghstack (oldest at bottom):

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: D50708982

cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler

@swolchok
[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for constants at infe…
8c636ad 
…rence time

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

[ghstack-poisoned]
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 30, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/112405

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure

As of commit 5f672c7 with merge base 494cb28 (image):

NEW FAILURE - The following job has failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This was referenced Oct 30, 2023
Closed
Closed
Closed
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
caf9361 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

[ghstack-poisoned]
@swolchok swolchok added the topic: not user facing topic category label Oct 30, 2023
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
df028b3 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

[ghstack-poisoned]
swolchok added a commit that referenced this pull request Oct 30, 2023
@swolchok
[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for constants at infe…
f9ed79a 
…rence time

Pull Request resolved: #112405

Cache aoti_torch_get_data_ptr at constants update time.
ghstack-source-id: 205864682
@exported-using-ghexport

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)
chenyang78
chenyang78 approved these changes Oct 31, 2023
View reviewed changes
Copy link
Contributor

@chenyang78 chenyang78 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please fix the lint failure. Thanks!

@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
2455cf2 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

[ghstack-poisoned]
This was referenced Nov 2, 2023
Closed
Closed
Closed
Closed
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
62c565c 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
a9e31de 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
71655a7 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
c657880 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
11905f5 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
567b504 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
b33480a 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
10ebde1 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
1d29a7e 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
7923eab 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
a34789f 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
23e9d8c 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
9d452ee 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
@swolchok
Update on "[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for consta…
5f672c7 
…nts at inference time"

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

cc voznesenskym penguinwu EikanWang jgong5 Guobing-Chen XiaobingSuper zhuhaozhe blzheng wenzhe-nrv jiayisunx peterbell10 ipiszy yf225 chenyang78 kadeng muchulee8 aakhundov ColinPeppler

[ghstack-poisoned]
pytorchmergebot pushed a commit that referenced this pull request Dec 12, 2023
@swolchok @pytorchmergebot
[PyTorch] AOTI: Use static_cast, not dynamic_cast (#112798)
2b323e6 
dynamic_cast is for when we aren't certain about the type. We are certain (and will crash anyway if we're wrong).

Differential Revision: [D50812978](https://our.internmc.facebook.com/intern/diff/D50812978/)

Pull Request resolved: #112798
Approved by: https://github.com/chenyang78, https://github.com/desertfire, https://github.com/jansel, https://github.com/khabinov
ghstack dependencies: #112116, #112174, #112405
pytorchmergebot pushed a commit that referenced this pull request Dec 12, 2023
@swolchok @pytorchmergebot
[PyTorch] AOTI: add more basic aoti_torch getters (#112799)
3de2596 
Lot of simple information about tensors we couldn't get. In
particular, we didn't know the lengths of the arrays returned by sizes
and strides.

Differential Revision: [D50949929](https://our.internmc.facebook.com/intern/diff/D50949929/)
Pull Request resolved: #112799
Approved by: https://github.com/desertfire, https://github.com/aakhundov
ghstack dependencies: #112116, #112174, #112405, #112798
pytorchmergebot pushed a commit that referenced this pull request Dec 12, 2023
@swolchok @pytorchmergebot
[PyTorch] AOTI: add more basic aoti_torch getters (#112799)
de4b2e5 
Lot of simple information about tensors we couldn't get. In
particular, we didn't know the lengths of the arrays returned by sizes
and strides.

Differential Revision: [D50949929](https://our.internmc.facebook.com/intern/diff/D50949929/)
Pull Request resolved: #112799
Approved by: https://github.com/desertfire, https://github.com/aakhundov
ghstack dependencies: #112116, #112174, #112405, #112798
@facebook-github-bot facebook-github-bot deleted the gh/swolchok/597/head branch December 15, 2023 15:27
guilhermeleobas pushed a commit to guilhermeleobas/pytorch that referenced this pull request Dec 18, 2023
@swolchok @guilhermeleobas
[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for constants at infe…
06e280f 
…rence time (pytorch#112405)

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

Pull Request resolved: pytorch#112405
Approved by: https://github.com/chenyang78, https://github.com/desertfire, https://github.com/khabinov
ghstack dependencies: pytorch#112116, pytorch#112174
guilhermeleobas pushed a commit to guilhermeleobas/pytorch that referenced this pull request Dec 18, 2023
@swolchok @guilhermeleobas
[PyTorch] AOTI: Use static_cast, not dynamic_cast (pytorch#112798)
c589ad7 
dynamic_cast is for when we aren't certain about the type. We are certain (and will crash anyway if we're wrong).

Differential Revision: [D50812978](https://our.internmc.facebook.com/intern/diff/D50812978/)

Pull Request resolved: pytorch#112798
Approved by: https://github.com/chenyang78, https://github.com/desertfire, https://github.com/jansel, https://github.com/khabinov
ghstack dependencies: pytorch#112116, pytorch#112174, pytorch#112405
guilhermeleobas pushed a commit to guilhermeleobas/pytorch that referenced this pull request Dec 18, 2023
@swolchok @guilhermeleobas
[PyTorch] AOTI: add more basic aoti_torch getters (pytorch#112799)
17a5a2d 
Lot of simple information about tensors we couldn't get. In
particular, we didn't know the lengths of the arrays returned by sizes
and strides.

Differential Revision: [D50949929](https://our.internmc.facebook.com/intern/diff/D50949929/)
Pull Request resolved: pytorch#112799
Approved by: https://github.com/desertfire, https://github.com/aakhundov
ghstack dependencies: pytorch#112116, pytorch#112174, pytorch#112405, pytorch#112798
guilhermeleobas pushed a commit to guilhermeleobas/pytorch that referenced this pull request Dec 18, 2023
@swolchok @guilhermeleobas
[PyTorch] AOTI: add more basic aoti_torch getters (pytorch#112799)
dac7055 
Lot of simple information about tensors we couldn't get. In
particular, we didn't know the lengths of the arrays returned by sizes
and strides.

Differential Revision: [D50949929](https://our.internmc.facebook.com/intern/diff/D50949929/)
Pull Request resolved: pytorch#112799
Approved by: https://github.com/desertfire, https://github.com/aakhundov
ghstack dependencies: pytorch#112116, pytorch#112174, pytorch#112405, pytorch#112798
dmenig pushed a commit to dmenig/pytorch that referenced this pull request Dec 21, 2023
@swolchok @dmenig
[PyTorch] AOTI: Avoid aoti_torch_data_ptr calls for constants at infe…
ed663ea 
…rence time (pytorch#112405)

Cache aoti_torch_get_data_ptr at constants update time.

Differential Revision: [D50708982](https://our.internmc.facebook.com/intern/diff/D50708982/)

Pull Request resolved: pytorch#112405
Approved by: https://github.com/chenyang78, https://github.com/desertfire, https://github.com/khabinov
ghstack dependencies: pytorch#112116, pytorch#112174
dmenig pushed a commit to dmenig/pytorch that referenced this pull request Dec 21, 2023
@swolchok @dmenig
[PyTorch] AOTI: Use static_cast, not dynamic_cast (pytorch#112798)
0584dd2 
dynamic_cast is for when we aren't certain about the type. We are certain (and will crash anyway if we're wrong).

Differential Revision: [D50812978](https://our.internmc.facebook.com/intern/diff/D50812978/)

Pull Request resolved: pytorch#112798
Approved by: https://github.com/chenyang78, https://github.com/desertfire, https://github.com/jansel, https://github.com/khabinov
ghstack dependencies: pytorch#112116, pytorch#112174, pytorch#112405
dmenig pushed a commit to dmenig/pytorch that referenced this pull request Dec 21, 2023
@swolchok @dmenig
[PyTorch] AOTI: add more basic aoti_torch getters (pytorch#112799)
576f5fd 
Lot of simple information about tensors we couldn't get. In
particular, we didn't know the lengths of the arrays returned by sizes
and strides.

Differential Revision: [D50949929](https://our.internmc.facebook.com/intern/diff/D50949929/)
Pull Request resolved: pytorch#112799
Approved by: https://github.com/desertfire, https://github.com/aakhundov
ghstack dependencies: pytorch#112116, pytorch#112174, pytorch#112405, pytorch#112798
dmenig pushed a commit to dmenig/pytorch that referenced this pull request Dec 21, 2023
@swolchok @dmenig
[PyTorch] AOTI: add more basic aoti_torch getters (pytorch#112799)
f17732e 
Lot of simple information about tensors we couldn't get. In
particular, we didn't know the lengths of the arrays returned by sizes
and strides.

Differential Revision: [D50949929](https://our.internmc.facebook.com/intern/diff/D50949929/)
Pull Request resolved: pytorch#112799
Approved by: https://github.com/desertfire, https://github.com/aakhundov
ghstack dependencies: pytorch#112116, pytorch#112174, pytorch#112405, pytorch#112798
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chenyang78 chenyang78 chenyang78 approved these changes

@desertfire desertfire desertfire approved these changes

+1 more reviewer

@khabinov khabinov khabinov approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ciflow/inductor Merged module: inductor topic: not user facing topic category

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@swolchok @chenyang78 @khabinov @desertfire @pytorchmergebot