-
Notifications
You must be signed in to change notification settings - Fork 61
[Draft] DSL design for next version #519
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: feature/dsl
Are you sure you want to change the base?
Conversation
docs/design/mscclpp-dsl-next.md
Outdated
channel = Channel(dst_rank, src_rank, channel_type) | ||
sem.acquire(tb=1) | ||
channel.put(other_peer_chunk, dst_chunk, tb=1) | ||
``` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this code pipelining "copy data from input buffer to scratch buffer" with "transfer data from scratch buffer to other peers". if so, which variable is the scratch buffer?
sem.release(tb=0) and sem.acquire(tb=1) ensures that copy to dst_chunk occures before put operation, right?
please provide more detailed explanation about the code
docs/design/mscclpp-dsl-next.md
Outdated
```python | ||
sem = Rank.Semaphore(rank=rank, size=1) | ||
rank = Rank(src_rank) | ||
with Loop.iteration(unit=2**20, num_chunks=1) as iter: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please add more explanation about the code. e.g., what is unit?
docs/design/mscclpp-dsl-next.md
Outdated
|
||
|
||
Here is the example for two ranks allreduce. Which achieve non-zero copy and use nvls. We use 3 thread-blocks to do the allreduce. | ||
The first thread-block is used to copy data from input buffer to scratch buffer, the second thread-block is used to do allreduce in scratch buffer, and the third thread-block is used to copy data from scratch buffer to output buffer. The thread-blocks are synchronized by semaphores. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
please add a line by line explanation of the code
docs/design/mscclpp-dsl-next.md
Outdated
``` | ||
|
||
## All2All support | ||
For now, DSL only support static all2all algorithm. For all2allv support, we need to get the send/recv size at the runtime. It may require some placeholder at the Json execution plan and relace to the real size at the runtime. If we could make chunk size be variable, we could use the same way to support all2allv. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
--> "Currently, the DSL only supports the static all2all algorithm. To support all2allv, we need to obtain the send/receive sizes at runtime. This may require using placeholders in the JSON execution plan, which would be replaced with the actual sizes during execution. If we can make the chunk size variable, the same approach could be used to support all2allv."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR drafts the next-version DSL for MSCCL++, adding new documentation and examples for the updated DSL and its execution plan.
- Included a link in the main docs to the new “MSCCL++ DSL Next” design doc.
- Added a comprehensive execution-plan markdown (
mscclpp-execution-plan.md
). - Revised the core DSL reference (
mscclpp-dsl.md
) with updated introduction, concepts, and examples.
Reviewed Changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 2 comments.
File | Description |
---|---|
docs/index.rst | Added entry for MSCCL++ DSL Next in the Design toctree |
docs/design/mscclpp-execution-plan.md | New file detailing the JSON execution-plan schema |
docs/design/mscclpp-dsl.md | Updated DSL overview, concepts, and example code snippets |
Comments suppressed due to low confidence (5)
docs/index.rst:29
- The link to
design/mscclpp-dsl-next
is added but no corresponding file is present; please ensuredocs/design/mscclpp-dsl-next.md
(or .rst) exists.
- :doc:`MSCCL++ DSL <design/mscclpp-dsl-next>` doc for those who want to understand the MSCCL++ DSL Next version.
docs/design/mscclpp-execution-plan.md:68
- [nitpick] The word "support" is split across lines with a hyphen; please merge lines or adjust formatting for readability.
- Switch Channel: A SwitchChannel provides primitives for performing collective operations among GPUs. These operations usually require specialized hardware sup-
docs/design/mscclpp-execution-plan.md:986
- The field name
connectedTo
is inconsistent with the earlierconnected_to
naming convention; please standardize onconnected_to
.
"connectedTo": [
docs/design/mscclpp-dsl.md:29
- [nitpick] Channel type names should match the class names (
MemoryChannel
,PortChannel
,SwitchChannel
) for consistency.
User need to use channel to communicate between ranks. Now we have three types of channels: memoryChannel, portChannel and switchChannel.
docs/design/mscclpp-dsl.md:109
- Example uses
Channel.memory
, but the enum is likelyChannelType.memory
; please correct the parameter tochannel_type=ChannelType.memory
.
chan1 = Channel(dst_rank, src_rank, channel_type=Channel.memory)
], | ||
"remote_buffer_ids": [ | ||
{ | ||
"access_channel_types": "memory", |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The access_channel_types
field should be an array of strings (e.g., ["memory"]
) rather than a single string to match the schema.
"access_channel_types": "memory", | |
"access_channel_types": [ | |
"memory" | |
], |
Copilot uses AI. Check for mistakes.
No description provided.