Skip to content

[Bugfix] Consolidate params with orig size#59

Merged
comaniac merged 2 commits intoawslabs:mainfrom
comaniac:fix_pipe_consolidate
Feb 15, 2023
Merged

[Bugfix] Consolidate params with orig size#59
comaniac merged 2 commits intoawslabs:mainfrom
comaniac:fix_pipe_consolidate

Conversation

@comaniac
Copy link
Contributor

Description

Previously we use sch.metadata.base_params to maintain the original parameter shape before sharding. However, the scope of schedule metadata is limited to the current schedule and won't be transferred if .replace is used, so sch.metadata.base_params won't be correct anymore in this case.

In this PR, we removed sch.metadata.base_params and directly set an attribute to sharded parameters to record their original shapes. In future, we should revisit the sch.metadata design.

Checklist

  • PR's title starts with a category (e.g. [Bugfix], [Model], [Tutorial], etc)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage
  • Code is well-documented

cc @szhengac @chhzh123

Copy link
Contributor

@chhzh123 chhzh123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@comaniac comaniac merged commit b1daf68 into awslabs:main Feb 15, 2023
@comaniac
Copy link
Contributor Author

Thanks @szhengac @chhzh123

@comaniac comaniac deleted the fix_pipe_consolidate branch February 15, 2023 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants