Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How much can 512 H100 ReduceScatter/AllGather-8GB_msg_size run to #1211

Open
atrium-idlers opened this issue Mar 5, 2024 · 1 comment
Open

How much can 512 H100 ReduceScatter/AllGather-8GB_msg_size run to #1211

atrium-idlers opened this issue Mar 5, 2024 · 1 comment

Comments

@atrium-idlers
Copy link

AllReduce 8GB_msg_size, how many GB/s for 1024 cards, how many GB/s for 2048 cards, and how many GB/s for 4096 cards;
Reduce Scatter/AllGather 8GB_msg_size, how many GB/s for 1024 cards, how many GB/s for 2048 cards, and how many GB/s for 4096 cards;
A server with 8 cards, using NV18 fully connected, 400Gb network card. Layer 3 IB switch.
@atrium-idlers
Copy link
Author

c5417a57a7c73faaa2c7ef892627da0
The maximum value in busbw

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@atrium-idlers