Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Relanding] Implemented torch.linalg.multi_dot #52859

Closed
wants to merge 26 commits into from
Closed

[Relanding] Implemented torch.linalg.multi_dot #52859

wants to merge 26 commits into from

Conversation

heitorschueroff
Copy link
Contributor

@heitorschueroff heitorschueroff commented Feb 25, 2021 •
edited

Stack from ghstack:

This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: D27402390

heitorschueroff added a commit that referenced this pull request Feb 25, 2021
@heitorschueroff
Revert "Revert D26375734: Implemented torch.linalg.multi_dot"
ee0a08b 
This reverts commit 92a4ee1.

ghstack-source-id: 5167f83fe019e35268ff457919d7babf47470fb2
Pull Request resolved: #52859
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Feb 25, 2021 •
edited

💊 CI failures summary and remediations

As of commit f637190 (more details on the Dr. CI page):

  • 1/1 failures possibly* introduced in this PR
    • 1/1 non-scanned failure(s)
This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

@heitorschueroff heitorschueroff changed the title Revert "Revert D26375734: Implemented torch.linalg.multi_dot" Implemented torch.linalg.multi_dot" Feb 25, 2021
@heitorschueroff heitorschueroff changed the title Implemented torch.linalg.multi_dot" [Relanding] Implemented torch.linalg.multi_dot" Feb 25, 2021
@heitorschueroff heitorschueroff mentioned this pull request Feb 25, 2021
Closed
mruberry
mruberry reviewed Mar 1, 2021
View reviewed changes
test/test_linalg.py Outdated Show resolved Hide resolved
mruberry
mruberry approved these changes Mar 1, 2021
View reviewed changes
Copy link
Collaborator

@mruberry mruberry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really cool; nice work!

@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot""
3c2d0ed 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D26672922](https://our.internmc.facebook.com/intern/diff/D26672922)

[ghstack-poisoned]
heitorschueroff added a commit that referenced this pull request Mar 1, 2021
@heitorschueroff
Revert "Revert D26375734: Implemented torch.linalg.multi_dot"
655d295 
This reverts commit 92a4ee1.

ghstack-source-id: f2910b1438fa23f94d147e672235af93606db1b1
Pull Request resolved: #52859
@heitorschueroff heitorschueroff mentioned this pull request Mar 5, 2021
Closed
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot""
163e8cb 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D26672922](https://our.internmc.facebook.com/intern/diff/D26672922)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot""
a81a340 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D26672922](https://our.internmc.facebook.com/intern/diff/D26672922)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot""
b9d87cb 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D26672922](https://our.internmc.facebook.com/intern/diff/D26672922)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot""
365c715 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D26672922](https://our.internmc.facebook.com/intern/diff/D26672922)

[ghstack-poisoned]
heitorschueroff added a commit that referenced this pull request Mar 5, 2021
@heitorschueroff
Revert "Revert D26375734: Implemented torch.linalg.multi_dot"
6c9466c 
This reverts commit 92a4ee1.

ghstack-source-id: 1d63b4d8965d8572f591bb95bf2bae58488e68f2
Pull Request resolved: #52859
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot""
9404c3c 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D26672922](https://our.internmc.facebook.com/intern/diff/D26672922)

[ghstack-poisoned]
@heitorschueroff heitorschueroff changed the title [Relanding] Implemented torch.linalg.multi_dot" [Relanding] Implemented torch.linalg.multi_dot Mar 25, 2021
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
cce595a 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D26672922](https://our.internmc.facebook.com/intern/diff/D26672922)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
0bab687 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D26672922](https://our.internmc.facebook.com/intern/diff/D26672922)

[ghstack-poisoned]
@heitorschueroff
Copy link
Contributor Author

Does multi_dot support batched products i.e. processing a list of matrices of shapes BxP1xP2, BxP2xP3, BxP3xP4?

Also may be interesting to have a matrix product reduction method, e.g. for a tensor of BxWxPxP and dim it would be equivalent of batched multi_dot(*input.unbind(dim)) (but maybe having more efficient computation because of known contiguity / layout)

It does not currently support broadcasting which is in line with NumPy. However, we could consider adding broadcast in a future PR.

@vadimkantorov
Copy link
Contributor

I am not even talking of broadcasting, just regular batching...

@heitorschueroff
Copy link
Contributor Author

I am not even talking of broadcasting, just regular batching...

My mistake, I meant to say batching. It does not support batching.

@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
56fcf76 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D27402390](https://our.internmc.facebook.com/intern/diff/D27402390)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
5f8925e 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D27402390](https://our.internmc.facebook.com/intern/diff/D27402390)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
fc47263 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D27402390](https://our.internmc.facebook.com/intern/diff/D27402390)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
459d937 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D27402390](https://our.internmc.facebook.com/intern/diff/D27402390)

[ghstack-poisoned]
@heitorschueroff
added wrapper function to pack inputs to multi_dot in distributed tes…
895873e 
…ting on "[Relanding] Implemented torch.linalg.multi_dot"


This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D27402390](https://our.internmc.facebook.com/intern/diff/D27402390)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
9efc5be 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D27402390](https://our.internmc.facebook.com/intern/diff/D27402390)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
7787f3d 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D27402390](https://our.internmc.facebook.com/intern/diff/D27402390)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
2958624 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D27402390](https://our.internmc.facebook.com/intern/diff/D27402390)

[ghstack-poisoned]
@heitorschueroff
Update on "[Relanding] Implemented torch.linalg.multi_dot"
f637190 
This reverts commit 92a4ee1.

Added support for bfloat16 for CUDA 11 and removed fast-path for empty input tensors that was affecting autograd graph.

Differential Revision: [D27402390](https://our.internmc.facebook.com/intern/diff/D27402390)

[ghstack-poisoned]
@heitorschueroff heitorschueroff mentioned this pull request Mar 31, 2021
Closed
@facebook-github-bot
Copy link
Contributor

@heitorschueroff merged this pull request in 5d68b36.

@facebook-github-bot facebook-github-bot deleted the gh/heitorschueroff/53/head branch April 4, 2021 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mruberry mruberry mruberry approved these changes

@malfet malfet malfet approved these changes

@albanD albanD Awaiting requested review from albanD

@glaringlee glaringlee Awaiting requested review from glaringlee

@soulitzer soulitzer Awaiting requested review from soulitzer

@IvanYashchuk IvanYashchuk Awaiting requested review from IvanYashchuk

@ezyang ezyang Awaiting requested review from ezyang

@H-Huang H-Huang Awaiting requested review from H-Huang

@jbschlosser jbschlosser Awaiting requested review from jbschlosser

@mrshenli mrshenli Awaiting requested review from mrshenli

@pritamdamania87 pritamdamania87 Awaiting requested review from pritamdamania87

@rohan-varma rohan-varma Awaiting requested review from rohan-varma

@wayi1 wayi1 Awaiting requested review from wayi1

@zhaojuanmao zhaojuanmao Awaiting requested review from zhaojuanmao

Assignees
No one assigned
Labels
cla signed Merged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@heitorschueroff @facebook-github-bot @vadimkantorov @malfet @mruberry