Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load lang id any2any training mengruw #8961

Closed
wants to merge 28 commits into from
Closed

Load lang id any2any training mengruw #8961

wants to merge 28 commits into from

Conversation

mengruwNv
Copy link

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Collection: [Note which collection this PR will affect]

Changelog

  • Add specific line by line info of high level changes in this PR.

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Jenkins CI

To run Jenkins, a NeMo User with write access must comment jenkins on the PR.

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

titu1994 and others added 28 commits May 17, 2023 15:13
@titu1994
Cut branch r1.19.0
4084669 
Signed-off-by: smajumdar <titu1994@gmail.com>
@BestJuly
Fix a bug, use _ceil_to_nearest instead as _round_to_nearest is not d…
1dc8b37 
…efined (NVIDIA#6681)
@artbataev
Fix k2 installation in Docker with CUDA 12 (NVIDIA#6707)
0ca1dd3 
Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
@titu1994
Tutorial fixes (NVIDIA#6717)
db6e29b 
Signed-off-by: smajumdar <titu1994@gmail.com>
@titu1994
VP Fixes for converter + Config management (NVIDIA#6698) (NVIDIA#6738)
2e2df4a 
* [Temp] VP Fixes

Signed-off-by: smajumdar <titu1994@gmail.com>

* Revert logging

Signed-off-by: smajumdar <titu1994@gmail.com>

---------

Signed-off-by: smajumdar <titu1994@gmail.com>
(cherry picked from commit b6f46a0)
@hsiehjackson
Fix fastpitch test nightly (NVIDIA#6742)
4df8f33 
Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu>
@ericharper
check for first or last stage (NVIDIA#6708)
e806e11 
* check for first or last stage

Signed-off-by: ericharper <complex451@gmail.com>

* remove redundant check

Signed-off-by: ericharper <complex451@gmail.com>

* fix typo

Signed-off-by: ericharper <complex451@gmail.com>

* add map_location

Signed-off-by: ericharper <complex451@gmail.com>

---------

Signed-off-by: ericharper <complex451@gmail.com>
@markelsanz14 @pre-commit-ci
Bug fix to restore act ckpt (NVIDIA#6753)
dbd6a56 
* Bug fix to restore act ckpt

Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@markelsanz14
Bug fix to reset sequence parallelism (NVIDIA#6756)
a0f757e 
* Bug fix to reset sequence parallelism

Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com>

* Update seq par reset/restore

Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com>

* Add nested loop

Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com>

---------

Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com>
@aklife97
Fix checkpointed forward and add test for full activation checkpointi…
39dd654 
…ng (NVIDIA#6744)

* fix checkpointed forward and add test for full activation checkpointing

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* add method

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* add method

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

---------

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
@titu1994
Fix Links (NVIDIA#6777)
216bcab 
Signed-off-by: smajumdar <titu1994@gmail.com>
@aklife97
add call to p2p overlap (NVIDIA#6779)
4ecc769 
* add call to p2p overlap

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* update Jenkins for test

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

---------

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
@ericharper
Fix get_parameters when using main params optimizer (NVIDIA#6764)
1486b12 
* fix get param

Signed-off-by: ericharper <complex451@gmail.com>

* change name

Signed-off-by: ericharper <complex451@gmail.com>

---------

Signed-off-by: ericharper <complex451@gmail.com>
@wdykas @pre-commit-ci @ericharper
Lddl bert (NVIDIA#6761)
aff5217 
* initial POC for LDDL Bert

* Finish LDDL POC

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* address comments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix merge head

* resolving merge

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* add support for  val/test loaders

* change to new LDDL class + add winding

* fix logging level

* fix winding

* test fix

* fixes to winding

* add file system

* add prepemption optimizations

* more logging

* more prints

* better logging

* asfsf

* add barrier

* removing prints

* working with mb lddl loader

* final changes

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* update requirements file with LDDL

Signed-off-by: wdykas <wdykas@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* revert adding to requirements

---------

Signed-off-by: wdykas <wdykas@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>
@timmoon10 @aklife97
Debug Transformer Engine FP8 support with Megatron-core infrastructure (
4bbb3c6 
NVIDIA#6740)

* Construct FP8 amax reduction group

Signed-off-by: Tim Moon <tmoon@nvidia.com>

* update core for CI

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

---------

Signed-off-by: Tim Moon <tmoon@nvidia.com>
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: Abhinav Khattar <aklife97@gmail.com>
@erhoo82 @timmoon10 @aklife97
Tensor-parallel communication overlap with userbuffer backend (NVIDIA…
e4460d1 
…#6780)

* add interfaces for tp_communication overlap

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Interface to provide custom userbuffer communicator settings by yaml file

[pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Construct MPI process group for userbuffers support

Signed-off-by: Tim Moon <tmoon@nvidia.com>

---------

Signed-off-by: Tim Moon <tmoon@nvidia.com>
Co-authored-by: Tim Moon <tmoon@nvidia.com>
Co-authored-by: Abhinav Khattar <aklife97@gmail.com>
@hsiehjackson
Fix adapter tutorial r1.19.0 (NVIDIA#6776)
9bd8ecd 
* Fix TTS adapter tutorial

Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu>

* Fix version

Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu>

---------

Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu>
@MaximumEntropy
Fix check (NVIDIA#6798)
913e5e5 
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
@markelsanz14
Bug fix for reset_sequence_parallel_args (NVIDIA#6802)
a8aa8f1 
Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com>
@erhoo82
Add ub communicator initialization to validation step (NVIDIA#6807)
0e0253e
@aklife97
update core version (NVIDIA#6817)
41bb941 
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
@ericharper
Add trainer.validate example for GPT (NVIDIA#6794)
45144f5 
* add trainer.validate example

Signed-off-by: ericharper <complex451@gmail.com>

* clean up white space

Signed-off-by: ericharper <complex451@gmail.com>

* add mbs and gbs to the config

Signed-off-by: ericharper <complex451@gmail.com>

---------

Signed-off-by: ericharper <complex451@gmail.com>
@yidong72
fix notebook error (NVIDIA#6840)
dc52b94 
Signed-off-by: Yi Dong <yidong@nvidia.com>
@yidong72
fix (NVIDIA#6842)
4239b80 
Signed-off-by: Yi Dong <yidong@nvidia.com>
@ericharper
Add API docs for NeMo Megatron (NVIDIA#6850)
87e1b81 
* add model pretraining and customization classes

Signed-off-by: ericharper <complex451@gmail.com>

* fix

Signed-off-by: ericharper <complex451@gmail.com>

* test width

Signed-off-by: ericharper <complex451@gmail.com>

* increase middle pane width

Signed-off-by: ericharper <complex451@gmail.com>

* add modules and datasets

Signed-off-by: ericharper <complex451@gmail.com>

* remove global in t5 dataset s and fix formatting in megatron base model

Signed-off-by: ericharper <complex451@gmail.com>

---------

Signed-off-by: ericharper <complex451@gmail.com>
@erhoo82 @pre-commit-ci
Apply garbage collection interval to validation steps (NVIDIA#6870)
f875702 
* Apply garbage collection inverval to validation steps

Signed-off-by: Sangkug Lym <slym@nvidia.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Sangkug Lym <slym@nvidia.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@ericharper
update mcore version (NVIDIA#6875)
2331b06 
Signed-off-by: ericharper <complex451@gmail.com>
fix language id issue for any2any training. This fix loads language i…
f0fa541 
…ds from .yaml file
@github-actions github-actions bot added core Changes to NeMo Core TTS labels Apr 17, 2024
@mengruwNv
Copy link
Author

Hi, @Davood-M I suggested some fix for the language ids mis-match issue when training any2any NMT model. Could you please help to review and merge it to a proper branch? We need the code for April release. Thank you in advance!

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 2, 2024

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

@github-actions github-actions bot added the stale label May 2, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented May 9, 2024

This PR was closed because it has been inactive for 7 days since being marked as stale.

@github-actions github-actions bot closed this May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
ASR CI core Changes to NeMo Core NLP Speaker Tasks stale TTS
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet