tune specific params in the base model by arendu · Pull Request #7745 · NVIDIA-NeMo/NeMo

arendu · 2023-10-18T04:01:57Z

What does this PR do ?

tune only a selected subset of the params of the base model.

Collection: [NLP]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

Signed-off-by: arendu <adithya.r@gmail.com>

for more information, see https://pre-commit.ci

bmwshop

LGTM!

ertkonuk · 2023-10-18T20:43:38Z

I think it looks good.

arendu · 2023-11-08T04:06:11Z

jenkins

github-actions · 2023-11-23T01:45:42Z

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

github-actions · 2023-11-30T01:46:03Z

This PR was closed because it has been inactive for 7 days since being marked as stale.

arendu · 2024-01-10T17:25:56Z

jenkins

cuichenx

Could you rename tunable_param_* to tunable_base_model_param_* and/or add a comment to specify it's for selective tuning? just because adapter weights are also tunable so it might be confusing

Signed-off-by: arendu <adithya.r@gmail.com>

arendu · 2024-01-10T18:11:20Z

jenkins

cuichenx

Looks great, thanks!

cuichenx · 2024-01-10T18:35:28Z

jenkins

arendu · 2024-01-10T18:44:02Z

jenkins

Signed-off-by: arendu <adithya.r@gmail.com>

for more information, see https://pre-commit.ci

cuichenx · 2024-01-16T00:27:11Z

jenkins

cuichenx

Looks good!

* tune specific params in the base model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * renamed Signed-off-by: arendu <adithya.r@gmail.com> * check for attr before calling Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* tune specific params in the base model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * renamed Signed-off-by: arendu <adithya.r@gmail.com> * check for attr before calling Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: stevehuang52 <heh@nvidia.com>

* tune specific params in the base model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * renamed Signed-off-by: arendu <adithya.r@gmail.com> * check for attr before calling Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Sasha Meister <ameister@nvidia.com>

* tune specific params in the base model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * renamed Signed-off-by: arendu <adithya.r@gmail.com> * check for attr before calling Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Pablo Garay <pagaray@nvidia.com>

* tune specific params in the base model Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * renamed Signed-off-by: arendu <adithya.r@gmail.com> * check for attr before calling Signed-off-by: arendu <adithya.r@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: arendu <adithya.r@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

tune specific params in the base model

74b22d0

Signed-off-by: arendu <adithya.r@gmail.com>

arendu requested review from ertkonuk and meatybobby October 18, 2023 04:02

github-actions bot added the NLP label Oct 18, 2023

arendu requested a review from Davood-M October 18, 2023 04:02

arendu and others added 2 commits October 17, 2023 21:02

Merge branch 'main' into adithyare/submodule

344e2a7

[pre-commit.ci] auto fixes from pre-commit.com hooks

f77f86e

for more information, see https://pre-commit.ci

bmwshop reviewed Oct 18, 2023

View reviewed changes

arendu requested a review from cuichenx October 27, 2023 20:15

Merge branch 'main' into adithyare/submodule

5226bf2

Merge branch 'main' into adithyare/submodule

4c087a4

github-actions bot added the stale label Nov 23, 2023

github-actions bot closed this Nov 30, 2023

arendu reopened this Jan 10, 2024

Merge branch 'main' into adithyare/submodule

356cc12

cuichenx requested changes Jan 10, 2024

View reviewed changes

renamed

482dde4

Signed-off-by: arendu <adithya.r@gmail.com>

arendu requested review from bmwshop and cuichenx January 10, 2024 18:11

cuichenx previously approved these changes Jan 10, 2024

View reviewed changes

github-actions bot removed the stale label Jan 11, 2024

check for attr before calling

6a3c6d4

Signed-off-by: arendu <adithya.r@gmail.com>

arendu dismissed cuichenx’s stale review via 6a3c6d4 January 13, 2024 18:14

arendu and others added 2 commits January 13, 2024 10:15

Merge branch 'main' into adithyare/submodule

778cb0d

[pre-commit.ci] auto fixes from pre-commit.com hooks

7f889b8

for more information, see https://pre-commit.ci

arendu requested a review from cuichenx January 13, 2024 19:34

cuichenx approved these changes Jan 16, 2024

View reviewed changes

arendu merged commit b404b84 into main Jan 16, 2024

arendu deleted the adithyare/submodule branch January 16, 2024 17:07

Conversation

arendu commented Oct 18, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do ?

Changelog

Usage

Before your PR is "Ready for review"

Who can review?

Additional Information

Uh oh!

bmwshop left a comment

Choose a reason for hiding this comment

Uh oh!

ertkonuk commented Oct 18, 2023

Uh oh!

arendu commented Nov 8, 2023

Uh oh!

github-actions bot commented Nov 23, 2023

Uh oh!

github-actions bot commented Nov 30, 2023

Uh oh!

arendu commented Jan 10, 2024

Uh oh!

cuichenx left a comment

Choose a reason for hiding this comment

Uh oh!

arendu commented Jan 10, 2024

Uh oh!

cuichenx left a comment

Choose a reason for hiding this comment

Uh oh!

cuichenx commented Jan 10, 2024

Uh oh!

arendu commented Jan 10, 2024

Uh oh!

cuichenx commented Jan 16, 2024

Uh oh!

cuichenx left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

arendu commented Oct 18, 2023 •

edited

Loading