New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add deprecation warnings #9266

Merged

dimapihtar merged 32 commits into r2.0.0rc0 from dpykhtar/deprecate_non_mcore

May 25, 2024

Collaborator

pablo-garay commented May 21, 2024

What does this PR do ?

Adds deprecation warnings for models that will soon be removed.
Remove deprecated tests from unit tests and ci/cd.

Collection: [Note which collection this PR will affect]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Jenkins CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

There's no need to comment jenkins on the PR to trigger Jenkins CI.
The GitHub Actions CI will run automatically when the PR is opened.
To run CI on an untrusted fork, a NeMo user with write access must click "Approve and run".

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

github-actions bot added core TTS ASR NLP CI common Multi Modal labels

github-advanced-security bot found potential problems

View reviewed changes

nemo/collections/multimodal/modules/imagen/diffusionmodules/layers.py Outdated

+                  OPT_GROUP_NORM = True
+              except Exception:
+                  print('Fused optimized group norm has not been installed.')

Check notice

Code scanning / CodeQL

Use of a print statement at module level

Print statement may execute during import.

nemo/collections/multimodal/modules/stable_diffusion/attention.py Outdated

-              from transformer_engine.pytorch.module import LayerNormLinear, LayerNormMLP
+                      OPT_GROUP_NORM = True
+                  except Exception:
+                      print('Fused optimized group norm has not been installed.')

Check notice

Code scanning / CodeQL

Use of a print statement at module level

Print statement may execute during import.

nemo/collections/multimodal/modules/stable_diffusion/diffusionmodules/model.py Outdated

+                  OPT_GROUP_NORM = True
+              except Exception:
+                  print('Fused optimized group norm has not been installed.')

Check notice

Code scanning / CodeQL

Use of a print statement at module level

Print statement may execute during import.

nemo/collections/multimodal/modules/stable_diffusion/diffusionmodules/openaimodel.py Outdated

+                  OPT_GROUP_NORM = True
+              except Exception:
+                  print('Fused optimized group norm has not been installed.')

Check notice

Code scanning / CodeQL

Use of a print statement at module level

Print statement may execute during import.

nemo/collections/multimodal/modules/stable_diffusion/diffusionmodules/util.py Outdated

+                  OPT_GROUP_NORM = True
+              except Exception:
+                  print('Fused optimized group norm has not been installed.')

Check notice

Code scanning / CodeQL

Use of a print statement at module level

Print statement may execute during import.

dimapihtar and others added 14 commits

May 21, 2024 10:42


          add deprecation warnings for non-mcore models

28addad

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          [pre-commit.ci] auto fixes from pre-commit.com hooks

78b7c62

for more information, see https://pre-commit.ci


          change warning default time

3e52f4e

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          remove unused import

27c5410

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          Apply isort and black reformatting

2f27131

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>


          remove deprecated tests

08cbd8c

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          set mcore_gpt to True

0efdcdf

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          set mcore_bert to True

c5299be

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          remove deprecated tests

3e4d8d5

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          remove deprecated unit tests

9af957a

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          add deprecation warning

3be802e

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          Apply isort and black reformatting

227a87e

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>


          remove deprecated playbook

dfebd5a

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          Apply isort and black reformatting

ff9b05b

Signed-off-by: pablo-garay <pablo-garay@users.noreply.github.com>

dimapihtar force-pushed the dpykhtar/deprecate_non_mcore branch from ba0c545 to ff9b05b Compare

May 21, 2024 17:45

github-actions bot removed core TTS ASR common labels

dimapihtar and others added 3 commits

May 21, 2024 17:46


          Apply isort and black reformatting

8899b7e

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>


          remove deprecated tutorial

b53cb18

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          Merge branch 'r2.0.0rc0' into dpykhtar/deprecate_non_mcore

8f270ac

dimapihtar added Run CICD and removed Run CICD labels

dimapihtar added 2 commits

May 24, 2024 14:10


          change precision for bert

649b6ab

Signed-off-by: dimapihtar <dpihtar@gmail.com>


          change precision for fine-tuning

c1a2b83

Signed-off-by: dimapihtar <dpihtar@gmail.com>

dimapihtar added Run CICD and removed Run CICD labels


          turn off fused attention for bert

8700c6d

Signed-off-by: dimapihtar <dpihtar@gmail.com>

dimapihtar added Run CICD and removed Run CICD labels


          Merge branch 'r2.0.0rc0' into dpykhtar/deprecate_non_mcore

b2e6b88

dimapihtar added Run CICD and removed Run CICD labels


          fix bert test

e9eedb1

Signed-off-by: dimapihtar <dpihtar@gmail.com>

dimapihtar added Run CICD and removed Run CICD labels

dimapihtar requested review from ericharper and dimapihtar

May 25, 2024 19:53

dimapihtar approved these changes

View reviewed changes

dimapihtar merged commit 525604f into r2.0.0rc0

113 of 114 checks passed

dimapihtar mentioned this pull request

cherry pick of #9266 #9411

Merged

8 tasks

ericharper pushed a commit that referenced this pull request


          cherry pick of #9266 (#9411)

27de845

* add deprecation warnings for non-mcore models

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change warning default time

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unused import

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_gpt to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_bert to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated unit tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* add deprecation warning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated playbook

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tutorial

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn of FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change mcore commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* adjustments

* update TE commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix mcore precision issue

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for fine-tuning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off fused attention for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix bert test

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* revert tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix typo

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unnecessary

Signed-off-by: dimapihtar <dpihtar@gmail.com>

---------

Signed-off-by: dimapihtar <dpihtar@gmail.com>
Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: Pablo Garay <pagaray@nvidia.com>

janekl pushed a commit that referenced this pull request


          cherry pick of #9266 (#9411)

937ba6d

* add deprecation warnings for non-mcore models

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change warning default time

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unused import

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_gpt to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_bert to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated unit tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* add deprecation warning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated playbook

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tutorial

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn of FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change mcore commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* adjustments

* update TE commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix mcore precision issue

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for fine-tuning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off fused attention for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix bert test

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* revert tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix typo

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unnecessary

Signed-off-by: dimapihtar <dpihtar@gmail.com>

---------

Signed-off-by: dimapihtar <dpihtar@gmail.com>
Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: Pablo Garay <pagaray@nvidia.com>
Signed-off-by: Jan Lasek <janek.lasek@gmail.com>

galv pushed a commit to galv/NeMo that referenced this pull request


          cherry pick of NVIDIA#9266 (NVIDIA#9411)

9359a55

* add deprecation warnings for non-mcore models

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change warning default time

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unused import

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_gpt to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_bert to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated unit tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* add deprecation warning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated playbook

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tutorial

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn of FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change mcore commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* adjustments

* update TE commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix mcore precision issue

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for fine-tuning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off fused attention for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix bert test

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* revert tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix typo

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unnecessary

Signed-off-by: dimapihtar <dpihtar@gmail.com>

---------

Signed-off-by: dimapihtar <dpihtar@gmail.com>
Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: Pablo Garay <pagaray@nvidia.com>

JesusPaz pushed a commit to JesusPaz/NeMo that referenced this pull request


          cherry pick of NVIDIA#9266 (NVIDIA#9411)

839beac

* add deprecation warnings for non-mcore models

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change warning default time

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unused import

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_gpt to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_bert to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated unit tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* add deprecation warning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated playbook

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tutorial

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn of FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change mcore commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* adjustments

* update TE commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix mcore precision issue

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for fine-tuning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off fused attention for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix bert test

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* revert tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix typo

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unnecessary

Signed-off-by: dimapihtar <dpihtar@gmail.com>

---------

Signed-off-by: dimapihtar <dpihtar@gmail.com>
Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: Pablo Garay <pagaray@nvidia.com>

rohitrango pushed a commit to rohitrango/NeMo that referenced this pull request


          cherry pick of NVIDIA#9266 (NVIDIA#9411)

8788ad1

* add deprecation warnings for non-mcore models

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* change warning default time

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unused import

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_gpt to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* set mcore_bert to True

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated unit tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* add deprecation warning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* Apply isort and black reformatting

Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>

* remove deprecated playbook

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove deprecated tutorial

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn of FA for Bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change mcore commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* adjustments

* update TE commit

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix mcore precision issue

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* change precision for fine-tuning

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* turn off fused attention for bert

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix bert test

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* revert tests

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* fix typo

Signed-off-by: dimapihtar <dpihtar@gmail.com>

* remove unnecessary

Signed-off-by: dimapihtar <dpihtar@gmail.com>

---------

Signed-off-by: dimapihtar <dpihtar@gmail.com>
Signed-off-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: dimapihtar <dimapihtar@users.noreply.github.com>
Co-authored-by: Pablo Garay <pagaray@nvidia.com>

ko3n1g mentioned this pull request

Release 2.0.0rc1 #9786

Closed

2 tasks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Multi Modal NLP Run CICD