Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

VP Fixes for converter + Config management #6698

Merged
merged 2 commits into from
May 26, 2023
Merged

VP Fixes for converter + Config management #6698

merged 2 commits into from
May 26, 2023

Conversation

titu1994
Copy link
Collaborator

@titu1994 titu1994 commented May 22, 2023

What does this PR do ?

Fix an issue with VP parameter loading.
NOTE: this is not a complete fix for VP, bugs are still being fixed in later PRs.

Collection: [NLP]

Changelog

  • force config to be correctly updated for megatron amp 02
  • update VP logic when loading weights

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

Signed-off-by: smajumdar <titu1994@gmail.com>
@titu1994 titu1994 marked this pull request as draft May 22, 2023 18:11
@github-actions github-actions bot added the NLP label May 22, 2023
Signed-off-by: smajumdar <titu1994@gmail.com>
@titu1994 titu1994 marked this pull request as ready for review May 23, 2023 23:35
@titu1994 titu1994 changed the title [DO NOT MERGE] VP Fixes VP Fixes for converter + Config management May 23, 2023
@ericharper
Copy link
Collaborator

Verified with pp4, vp3, and tp8, pp 4, vp 4.

LGTM. Thanks!

Copy link
Collaborator

@ericharper ericharper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Let's make sure to send this to 1.19.0 as well.

@titu1994
Copy link
Collaborator Author

Yup will cherry pick

@titu1994 titu1994 merged commit b6f46a0 into main May 26, 2023
@titu1994 titu1994 deleted the vp_fixes branch May 26, 2023 05:21
titu1994 added a commit that referenced this pull request May 26, 2023
* [Temp] VP Fixes

Signed-off-by: smajumdar <titu1994@gmail.com>

* Revert logging

Signed-off-by: smajumdar <titu1994@gmail.com>

---------

Signed-off-by: smajumdar <titu1994@gmail.com>
(cherry picked from commit b6f46a0)
titu1994 added a commit that referenced this pull request May 26, 2023
* [Temp] VP Fixes

Signed-off-by: smajumdar <titu1994@gmail.com>

* Revert logging

Signed-off-by: smajumdar <titu1994@gmail.com>

---------

Signed-off-by: smajumdar <titu1994@gmail.com>
(cherry picked from commit b6f46a0)
hsiehjackson pushed a commit to hsiehjackson/NeMo that referenced this pull request Jun 2, 2023
* [Temp] VP Fixes

Signed-off-by: smajumdar <titu1994@gmail.com>

* Revert logging

Signed-off-by: smajumdar <titu1994@gmail.com>

---------

Signed-off-by: smajumdar <titu1994@gmail.com>
Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu>
yaoyu-33 pushed a commit that referenced this pull request Oct 16, 2023
* [Temp] VP Fixes

Signed-off-by: smajumdar <titu1994@gmail.com>

* Revert logging

Signed-off-by: smajumdar <titu1994@gmail.com>

---------

Signed-off-by: smajumdar <titu1994@gmail.com>
(cherry picked from commit b6f46a0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants