[claude code] Documentation for optimize_sharding.py #38
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Edward Yang <ezyang@meta.com>
facebook-github-bot
added
the
CLA Signed
ezyang
changed the title
wconstab
reviewed
Jul 10, 2025
| Creates binary variables x_{i,a,o,j} for each valid combination of: | ||
| - s_i: operation index | ||
| - argi: argument index | ||
| - ss: output placement strategy index (o in math notation) |
Contributor
There was a problem hiding this comment.
can i just say that ss being an index and ssi being the value associated with that index has been driving me mad
Contributor
There was a problem hiding this comment.
Yeah we should rename those variables to more meaningful names. Those namings are there from the first version of the code I wrote, and I was just naming things almost arbitrarily to get things going. I did some cleanup some time ago but those names remained, we should improve it soon
fmassa
approved these changes
Jul 11, 2025
|
|
||
| def penalize_inefficient_collectives(self): | ||
| """ | ||
| EFFICIENCY CONSTRAINTS (Category 5): Penalize inefficient collective operations like |
Contributor
There was a problem hiding this comment.
@wconstab we discussed this the other day, ideally we should just make the comms cost computation more accurate at some point
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Signed-off-by: Edward Yang ezyang@meta.com