Skip to content

Add out-of-place all-gather coalesced #6059

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Dec 12, 2023
Merged

Add out-of-place all-gather coalesced #6059

merged 3 commits into from
Dec 12, 2023

Conversation

@jeffhataws
Copy link
Collaborator

@jeffhataws jeffhataws commented Dec 8, 2023

Similar to #6058 , we add out-of-place version for all-gather coalesced (which was introduced in #5950 without an out-of-place version).

@jeffhataws jeffhataws mentioned this pull request Dec 8, 2023
Merged
@jeffhataws jeffhataws force-pushed the jeffhataws_allgather_out branch 2 times, most recently from 3587ae2 to 2df3c56 Compare December 9, 2023 07:02
@jeffhataws jeffhataws force-pushed the jeffhataws_allgather_out branch from 2df3c56 to 0d6125c Compare December 10, 2023 17:02
@jeffhataws jeffhataws mentioned this pull request Dec 11, 2023
Open
JackCaoG
JackCaoG reviewed Dec 11, 2023
View reviewed changes
JackCaoG
JackCaoG reviewed Dec 11, 2023
View reviewed changes
JackCaoG
JackCaoG reviewed Dec 11, 2023
View reviewed changes
Copy link
Collaborator

@JackCaoG JackCaoG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly lgtm, minor comments

JackCaoG
JackCaoG approved these changes Dec 11, 2023
View reviewed changes
Copy link
Collaborator

@JackCaoG JackCaoG left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I will merge once all CI passed

@JackCaoG JackCaoG mentioned this pull request Dec 11, 2023
Merged
@JackCaoG JackCaoG merged commit e3012bf into master Dec 12, 2023
JackCaoG pushed a commit that referenced this pull request Dec 12, 2023
@jeffhataws @Arjunbala @JackCaoG
Add out-of-place all-gather coalesced (#6059)
5d8f398 
Co-authored-by: Arjun Balasubramanian <arjunbala0711@gmail.com>
ManfeiBai pushed a commit that referenced this pull request Dec 12, 2023
@JackCaoG @jeffhataws @Arjunbala
[Backport]Add out-of-place all-gather coalesced (#6059) (#6108)
9a54aba 
Co-authored-by: jeffhataws <jthuynh@amazon.com>
Co-authored-by: Arjun Balasubramanian <arjunbala0711@gmail.com>
chunnienc pushed a commit to chunnienc/xla that referenced this pull request Dec 14, 2023
@jeffhataws @Arjunbala @chunnienc
Add out-of-place all-gather coalesced (pytorch#6059)
c41eccf 
Co-authored-by: Arjun Balasubramanian <arjunbala0711@gmail.com>
golechwierowicz pushed a commit that referenced this pull request Jan 12, 2024
@jeffhataws @Arjunbala @golechwierowicz
Add out-of-place all-gather coalesced (#6059)
6d74125 
Co-authored-by: Arjun Balasubramanian <arjunbala0711@gmail.com>
bhavya01 pushed a commit that referenced this pull request Apr 22, 2024
@jeffhataws @Arjunbala @bhavya01
Add out-of-place all-gather coalesced (#6059)
a15832f 
Co-authored-by: Arjun Balasubramanian <arjunbala0711@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JackCaoG JackCaoG JackCaoG approved these changes

@alanwaketan alanwaketan Awaiting requested review from alanwaketan

Assignees

No one assigned

Labels

backport_2.2

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@jeffhataws @JackCaoG @Arjunbala