Bert interleaved #5556

shanmugamr1992 · 2022-12-06T18:35:37Z

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Collection: [Note which collection this PR will affect]

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

Related to # (issue)

for more information, see https://pre-commit.ci

Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>

ericharper

LGTM. Thanks!

* Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * Update bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Adding tests * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * Fix bug fix sequence parallel and Interleaved * Fix bug fix sequence parallel and Interleaved Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@gmail.com>

* Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * Update bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Adding tests * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * Fix bug fix sequence parallel and Interleaved * Fix bug fix sequence parallel and Interleaved Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com>

* Merge r1.13.0 main (#5570) * update branch Signed-off-by: ericharper <complex451@gmail.com> * Rename Speech Dataset Processor to Speech Data Processor (#5378) Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Megatron Export Update (#5343) * export update for Megatron + change ORT optimization Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated export_utils to use autocast instead of manually casting >:/ Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * removed dtype from LayerNorm Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * added comment Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * reverting changes on FloatCast Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * Cherry-picked changes from megatron-norm Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * updated asr_model import to cast_utils Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * updated del onnx_model place Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * changed ort optimization to basic -> temp fix Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Boris Fomitchev <bfomitchev@nvidia.com> * Disable sync_batch_comm in validation_step for GPT (#5397) * disable sync_batch_comm in validation_step Signed-off-by: ericharper <complex451@gmail.com> * Read sync_batch_comm from config or default to False Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> * Update megatron_gpt_config to default sync_batch_comm to False to avoid CUDA error Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> * Empty Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Comment out test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Markel Sanz Ausin <markelsanz14@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Radtts 1.13 (#5451) * [TTS] Fixing RADTTS training - removing view buffer and fixing accuracy issue (#5358) * [TTS] add CI test for RADTTS training recipe. Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Support for finetuning and finetuning inference with .ckpt files & batch size refactoring (#5339) (#5478) * Initial refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Resolve config before passing to load_from_checkpoint Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for model parallel and nemo restore Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert config changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Minor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix validation reconfiguration Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove old comment Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes for test_ds Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * export_utils bugfix (#5480) * updated export_utils Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Export fixes for Riva (#5496) * Export fixes for Riva Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Cleaning up training_utils Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * added set_start_method + function param bugfix (#5539) * added set_start_method + function param bugfix Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * upper bound torchmetrics Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ericharper <complex451@gmail.com> * remove notebook (#5548) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update branch Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Markel Sanz Ausin <markelsanz14@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Optimized loop and bugfix in SDE (#5573) - Fixed bug with loading custom data attributes from JSON in Speech Data Explorer Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update torchmetrics (#5566) * add task arg Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update state Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove useless files. (#5580) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add initial NFA code Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make use of the specified device during viterbi decoding Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix CodeQL notes Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix CodeQL warning Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add an option to defer data setup from ``__init__`` to ``setup`` (#5569) * Add an option to defer dataloader setup from __init__ to setup Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Updated doc Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make utt_id specified by number of parts of audio_filepath user wishes to use Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove audio_sr TODO - reduce risk of silent bugs Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add check that model is CTC Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove unused import Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Text generation improvement (UI client, data parallel support) (#5437) * Squashed commit of the following: commit a5e124f34be31bd6eafe5e5fdf5bedcd0d50915c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 13 15:07:42 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 35b424044fe80c3081e7756ab21244f701716f7e Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 08:04:49 2022 -0700 get rid of base Signed-off-by: Yi Dong <yidong@nvidia.com> commit 2955210e2311791543538cfbb5ad26b79414c954 Merge: d52edef8c eaf6757ca Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:17:02 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit d52edef8cd7b36593838fb270047e80f8ccb652e Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:16:24 2022 +0000 align with main Signed-off-by: Yi Dong <yidong@nvidia.com> commit eaf6757ca5be8e099492f57c81d984429b0ad49c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 13 13:12:11 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit c4b86d97626ea0721bf8fb4c0a45dec5becc94c9 Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:10:58 2022 +0000 same as main Signed-off-by: Yi Dong <yidong@nvidia.com> commit e335de51bcc0d681c58b568c3d8c238bc5687c3b Merge: c231086e0 4463a9fe9 Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:08:09 2022 +0000 Merge branch 'main' into universal_prompt commit c231086e057f1efaa915f691d84664cb3d5aad85 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Oct 12 19:59:12 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 6a821a4b49a23dd3408a706a2a3dd393149b0bb1 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:56:17 2022 +0000 default to pad Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9d908e39fef1beed9ba2da4d1a6806161eb7ef25 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:55:44 2022 +0000 add the option to pad the tokens Signed-off-by: Yi Dong <yidong@nvidia.com> commit 876dc395b43fdeeaa2bcbbe13c76523633764c33 Merge: fbb0f4035 fe3c77ee9 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:20:47 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit fe3c77ee93ab6cf3ea152db68cb6beefcac2a392 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 18:59:49 2022 +0000 fix import again Signed-off-by: Yi Dong <yidong@nvidia.com> commit fbb0f4035c6cd6bfefed50a20605503de8c1dccb Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Oct 12 16:00:24 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 372ca8c0d7988f2339b15888dc72aa21f4fb6937 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 15:58:32 2022 +0000 enable server Signed-off-by: Yi Dong <yidong@nvidia.com> commit cbe05d9fbc978f812cfbb671f45f147f300713c4 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 13:07:28 2022 +0000 fix comment error Signed-off-by: Yi Dong <yidong@nvidia.com> commit 1948048922e726ec6131e44b1a745389f18d4ef2 Merge: 232c2cce3 984f5c09a Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 13:05:30 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit 232c2cce34d7a8b902da406706f3dd9b39475091 Merge: 34c8a68df 658243fb6 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 12:50:00 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit 984f5c09a6dbf1d1fb5aa30ed9b0df188e66a50f Merge: 658243fb6 3fda5de46 Author: Yi Dong <43824965+yidong72@users.noreply.github.com> Date: Wed Oct 12 08:42:11 2022 -0400 Merge branch 'main' into fix_global_init commit 658243fb6580191b5d60edd30cde16dcc23cbb85 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 12 12:40:57 2022 +0000 fix import error Signed-off-by: Yi Dong <doyend@gmail.com> commit 8e0fe1cad05ec288ec122b3cd0e139a96872e08c Author: Yi Dong <doyend@gmail.com> Date: Tue Oct 11 22:44:12 2022 +0000 update the fused kernel Signed-off-by: Yi Dong <doyend@gmail.com> commit 536cf6bef9447b75843fad630729c47a2fba35f3 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:44:52 2022 -0700 add the missing file Signed-off-by: Yi Dong <yidong@nvidia.com> commit 1b437ec41dc5e354453ce0a089bca0171cbcb6c2 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:43:14 2022 -0700 fix fused softmax Signed-off-by: Yi Dong <yidong@nvidia.com> commit 7813f60e05f9783af61f8c14ec1cb0c6c4f1f263 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:16:48 2022 -0700 move global step to base Signed-off-by: Yi Dong <yidong@nvidia.com> commit 34c8a68df084b18d377e84415d9f07b2cd6673dd Author: Yi Dong <doyend@gmail.com> Date: Thu Oct 6 13:50:11 2022 +0000 fix pipeline for eval Signed-off-by: Yi Dong <doyend@gmail.com> commit eee5d38218f26660c3ffebe9f615c850c80a1f0d Author: Yi Dong <doyend@gmail.com> Date: Thu Oct 6 13:48:22 2022 +0000 fix for pipleline parallel Signed-off-by: Yi Dong <doyend@gmail.com> commit 323bca73e7ef6099ee79c0a2fffac7b709ed6c5d Merge: 125e49947 e3b4c4d1f Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 19:29:13 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit 125e4994760448ff75dd9328395813eda1c87547 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 19:29:04 2022 +0000 add share option Signed-off-by: Yi Dong <doyend@gmail.com> commit e3b4c4d1f7346c9fa596f3cca6d4df0a9e05c368 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 5 11:43:48 2022 -0700 make sure consolidation works Signed-off-by: Yi Dong <yidong@nvidia.com> commit a5c833964ecf05dc460ca1da69275c4019742150 Merge: 2a07ab52d abcb74be2 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:40:29 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit 2a07ab52d95f15ba666823028c69e23825666c05 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:40:23 2022 +0000 added requirement Signed-off-by: Yi Dong <doyend@gmail.com> commit 3abecd9dd1611993a87c537636abe7f7e6a9b04c Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:39:42 2022 +0000 added a simple web server Signed-off-by: Yi Dong <doyend@gmail.com> commit abcb74be2caf1cdec40eb9ba2be4dde4d45a3b4b Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 5 06:54:12 2022 -0700 fix empty val loss Signed-off-by: Yi Dong <yidong@nvidia.com> commit b8eb92ac4a0d665570af75e34c9ba3c2e2420c26 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 19:25:30 2022 -0700 text gen working Signed-off-by: Yi Dong <yidong@nvidia.com> commit d59f3e3f3a6fd19736d1c5706fed65a3dd4049ba Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 16:08:40 2022 -0700 first change Signed-off-by: Yi Dong <yidong@nvidia.com> commit 59d077585e6962a669b824af58f64e8a0bea6547 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 15:00:40 2022 -0700 revert Signed-off-by: Yi Dong <yidong@nvidia.com> commit 12a0f3902d99e9179403644bd951c045df716ca7 Author: Yi Dong <doyend@gmail.com> Date: Tue Oct 4 21:26:23 2022 +0000 init imp Signed-off-by: Yi Dong <doyend@gmail.com> commit 62a15dfd943cc48be495ac61b9f2f00995775c5f Merge: 82c90d2cd e0cc6b767 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 11:58:26 2022 -0700 Merge branch 'main' into universal_prompt commit 82c90d2cd0fd156f16a4b899f8c741d598f33990 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 11:17:13 2022 -0700 add sync Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9819b703eef877d90cd1257bf3610c69de9b4d7e Author: Yi Dong <yidong@nvidia.com> Date: Sun Oct 2 17:52:34 2022 -0700 fix save model Signed-off-by: root <root@luna-0197.selene.nvidia.com> commit e4937e2fc5fb7d70754c97668416e4a69c3079fe Author: Yi Dong <yidong@nvidia.com> Date: Sat Oct 1 18:56:09 2022 +0000 working Signed-off-by: Yi Dong <yidong@nvidia.com> commit b73b06d1c7cf5417a6d87cb33d8ed83a57e38b7b Author: Yi Dong <yidong@nvidia.com> Date: Sat Oct 1 17:34:03 2022 +0000 calcuate the mask Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9db3bc13eb65a94a475b837603351da68e3745bc Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 23:26:32 2022 +0000 fix bug in datasets Signed-off-by: Yi Dong <yidong@nvidia.com> commit f289900375d4412f53f8110be00fec6587627550 Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 22:29:40 2022 +0000 update the code Signed-off-by: Yi Dong <yidong@nvidia.com> commit 8e28a1f208aabaab72dbe769e72756baada04d99 Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 21:52:52 2022 +0000 added new ds Signed-off-by: Yi Dong <yidong@nvidia.com> commit 8d41315bab7ce90e200a8a7d1023c34f8e046897 Author: Yi Dong <doyend@gmail.com> Date: Fri Sep 30 18:57:09 2022 +0000 added new files Signed-off-by: Yi Dong <doyend@gmail.com> commit 984e0e94e15e16323c1ba1ca2efeabd84f69463f Merge: cbe8b7ab1 fa6cd8588 Author: Yi Dong <doyend@gmail.com> Date: Thu Sep 29 21:43:29 2022 +0000 Merge branch 'llm-prompt-learning-improvements' into universal_prompt commit fa6cd858839277939446afe7275976078d54c512 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Sep 29 16:47:30 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 78ba46e5d6fde1be53c08e1e30a54cce59824be0 Merge: 7d6d46742 8d670bc77 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Thu Sep 29 09:43:27 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 7d6d46742170a66758287a207d67e1b1bfd15613 Author: Virginia Adams <vadams@nvidia.com> Date: Thu Sep 29 16:42:43 2022 +0000 Removed inference step and added sentence peice check to predict step Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 20fd265acd6f7f9912cf52155fe66ccfa6b201a2 Author: Virginia Adams <vadams@nvidia.com> Date: Thu Sep 29 15:26:32 2022 +0000 fixed first stage check for pipeline parallel T5 pt Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 3637be2b258c8d9028856f9971edb7da4a8121f0 Merge: a3ea722fd 986a76612 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Wed Sep 28 10:23:30 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit a3ea722fdc12fbcc5989b76ef5643a574b763bc4 Merge: 770967a52 971485ce7 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Mon Sep 26 13:35:52 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 770967a5251a474b6dcc2d44bf9a2076adbcb604 Merge: d23bf6c30 e3ac280a8 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Mon Sep 26 10:17:03 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit d23bf6c30acc0e3f6af9b4e24547669866a34d62 Merge: de6a31651 333d2b749 Author: Virginia Adams <vadams@nvidia.com> Date: Mon Sep 26 10:05:16 2022 -0700 Merge branch 'llm-prompt-learning-improvements' of https://github.com/NVIDIA/NeMo into llm-prompt-learning-improvements commit de6a31651e63d88a42b971794d93f18ff5a3cdff Author: Virginia Adams <vadams@nvidia.com> Date: Mon Sep 26 17:00:53 2022 +0000 Updated PP check to be on first stage pipeline only Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 333d2b7498e6742ce66436f733c980a74616900c Merge: 592c0986a a39fc925a Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Fri Sep 23 16:11:21 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 592c0986a476a91b57b8605d7b70830d7acfa021 Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 23:08:41 2022 +0000 Fixed unused import and CI test bug Signed-off-by: Virginia Adams <vadams@nvidia.com> commit ea9cd82d85638bc60ae4ad7ef105db931c8e3455 Merge: ce4b72c8c b566c2d0e Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 18:57:25 2022 +0000 Merge branch 'llm-prompt-learning-improvements' of https://github.com/NVIDIA/NeMo into llm-prompt-learning-improvements commit ce4b72c8c52f32be336e323dd78a38089edc3e7c Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 18:57:16 2022 +0000 Switch to import from base class Signed-off-by: Virginia Adams <vadams@nvidia.com> commit b566c2d0e35a068f758fd1310bc620a47be4590b Merge: 6621f2854 e872061ac Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Fri Sep 23 10:09:03 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 6621f28543828a48484a5637f6c9f3ccb23a5b02 Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 20:47:35 2022 +0000 python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 8deafc8987b6af5f7b99a250310f57a40198c37f Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 20:28:02 2022 +0000 Save .nemo on new best val score Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 761bd36969cb465d6a129e9eee6ce1f883d3cf41 Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 18:03:19 2022 +0000 Added automatic checkpoint to nemo file method Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 3be4ed57b6cd3ddfe4876d78650dfe8fe794598b Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 02:11:56 2022 +0000 Make GPT use base prompt learning model class: Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> * fix LGTM Signed-off-by: Yi Dong <yidong@nvidia.com> * fix validation Signed-off-by: Yi Dong <yidong@nvidia.com> * change for the lm eval Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * make text generation work in data parallel environment Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * implement the service with rest service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * surpress log Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore function needed for NMT Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * handles no answer only Signed-off-by: Yi Dong <yidong@nvidia.com> * Fix config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added knn to web Signed-off-by: Yi Dong <yidong@nvidia.com> * fix lgtm.com comments Signed-off-by: Yi Dong <yidong@nvidia.com> * output the retrieved context Signed-off-by: Yi Dong <yidong@nvidia.com> * allow no neighbor query Signed-off-by: Yi Dong <yidong@nvidia.com> * remove the imports Signed-off-by: Yi Dong <yidong@nvidia.com> * warn only once Signed-off-by: Yi Dong <yidong@nvidia.com> * Change output file format from JSON to JSONL Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * new t0 dataset Signed-off-by: Yi Dong <yidong@nvidia.com> * Add T0 data preproc scripts Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Merge and multiprocessing Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix for is_correct Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * fix epoch > 2 Signed-off-by: Yi Dong <yidong@nvidia.com> * handles multiple dataloader Signed-off-by: Yi Dong <yidong@nvidia.com> * remove template Signed-off-by: Yi Dong <yidong@nvidia.com> * Refactor T0 dataset Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add script to merge train folder into individual training files to minimize number of blends Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added on the fly service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add combo instance Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added combo service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * send weights back to server Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix index store Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Minor changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add reset button Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add add eos Signed-off-by: Yi Dong <yidong@nvidia.com> * use a seperate bert service Signed-off-by: Yi Dong <yidong@nvidia.com> * no loss of accuracy Signed-off-by: Yi Dong <yidong@nvidia.com> * pin the gradio version Signed-off-by: Yi Dong <yidong@nvidia.com> * Remove bin compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix header lines Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * evaluate based on text generation Signed-off-by: Yi Dong <yidong@nvidia.com> * exact match result aggregation Signed-off-by: Yi Dong <yidong@nvidia.com> * working SP and SA Signed-off-by: Yi Dong <yidong@nvidia.com> * sync Signed-off-by: Yi Dong <yidong@nvidia.com> * fix checkpoint Signed-off-by: Yi Dong <yidong@nvidia.com> * fix eval Signed-off-by: Yi Dong <yidong@nvidia.com> * backup states Signed-off-by: Yi Dong <yidong@nvidia.com> * backup states reset Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the bug Signed-off-by: Yi Dong <yidong@nvidia.com> * fix evaluation for sentence piece Signed-off-by: Yi Dong <yidong@nvidia.com> * fix a bug Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * potential fix in the future Signed-off-by: Yi Dong <yidong@nvidia.com> * remove the universal codes Signed-off-by: Yi Dong <yidong@nvidia.com> * remove universal strategy Signed-off-by: Yi Dong <yidong@nvidia.com> * address reviewer comment Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add align function docstrings and make most args optional Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove redundant returns of viterbi and log probs matrices Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Rename h# to <initial_silence> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update manifest format description in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * always remove any spaces from utt_id Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Patch the hanging of threads on very large stderr (#5589) (#5590) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * O2 style amp for gpt3 ptuning (#5246) * enable amp o2 plugin Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * only create master param if param requires gradient Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * remove pytorch autocast Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Update optimizer_with_main_params.py Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> * create master grad only if param group requires grad Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix grad scaler for pp > 1 Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Better patch hydra (#5591) (#5592) * Readd buffereing and thread drain to Hydra Launcher Signed-off-by: smajumdar <titu1994@gmail.com> * Readd buffereing and thread drain to Hydra Launcher Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Yet another fix with hydra multirun (#5594) (#5595) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add RETRO model documentation (#5578) * added retro doc Signed-off-by: Yi Dong <yidong@nvidia.com> * finish data part Signed-off-by: Yi Dong <yidong@nvidia.com> * added the data format Signed-off-by: Yi Dong <yidong@nvidia.com> * added training script Signed-off-by: Yi Dong <yidong@nvidia.com> * added training and evaluation steps Signed-off-by: Yi Dong <yidong@nvidia.com> * edit the text Signed-off-by: Yi Dong <yidong@nvidia.com> * added the images Signed-off-by: Yi Dong <yidong@nvidia.com> * fix beginning Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the grammar Signed-off-by: Yi Dong <yidong@nvidia.com> * trim it down Signed-off-by: Yi Dong <yidong@nvidia.com> * add wandb option Signed-off-by: Yi Dong <yidong@nvidia.com> * add reference Signed-off-by: Yi Dong <yidong@nvidia.com> * fix path Signed-off-by: Yi Dong <yidong@nvidia.com> * added the parameters table Signed-off-by: Yi Dong <yidong@nvidia.com> * fix section Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix: setup_multiple validation/test data (#5585) Fix: setup_multiple validation/test data (#5585) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Move to optimizer based EMA implementation (#5169) * Move to optimizer Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix replacing weights Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Allow swapping of weights be optional Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Save 2 models Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Use different hook Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Expose cpu device Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add clause to see if this fixes issue with O2 optimizer Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Try to get O2 working Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * WIP Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fixes Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fixes to tests Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add guard Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Remove import Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add guard Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add comment Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Remove overwrite Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add BatchNorm, currently tests fail Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix tests/functionality for batch norm Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Get rid of NLP changes Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * AIStore for ASR datasets (#5462) AIStore for ASR datasets Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add support for MHA adapters to ASR (#5396) * Convert AbstractAdapterModule to AbstractAdapterMixin Signed-off-by: smajumdar <titu1994@gmail.com> * Temporary fixes to new signature of mixin Signed-off-by: smajumdar <titu1994@gmail.com> * Add adapter util for constants, add all mha adapters. Signed-off-by: smajumdar <titu1994@gmail.com> * Update name of function Signed-off-by: smajumdar <titu1994@gmail.com> * Roll back changes to convASR Signed-off-by: smajumdar <titu1994@gmail.com> * Convert AbstractAdapterModule to AbstractAdapterMixin Signed-off-by: smajumdar <titu1994@gmail.com> * First draft of Conformer support for MHA attention Signed-off-by: smajumdar <titu1994@gmail.com> * Add some preliminary tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for projection of the hidden dimension for attention Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for squeezeformer Signed-off-by: smajumdar <titu1994@gmail.com> * Update train adapter config Signed-off-by: smajumdar <titu1994@gmail.com> * Add tests for squeezeformer and unit tests for new modules Signed-off-by: smajumdar <titu1994@gmail.com> * Update config for hp search,set limits on modules for conformer and squeezeformer, update adapter mixin, add cache to import_from_class_path Signed-off-by: smajumdar <titu1994@gmail.com> * Update location of adapters Signed-off-by: smajumdar <titu1994@gmail.com> * Add pre_norm for proper attention learning, Fix the issue with nan/inf in pos_bias_u and pos_bias_v Signed-off-by: smajumdar <titu1994@gmail.com> * Update expmanager to clean up checkpoints Signed-off-by: smajumdar <titu1994@gmail.com> * Fix style Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update training scripts Signed-off-by: smajumdar <titu1994@gmail.com> * Update config and docs Signed-off-by: smajumdar <titu1994@gmail.com> * Expose nemo delete function Signed-off-by: smajumdar <titu1994@gmail.com> * Correct adapter partial state saving Signed-off-by: smajumdar <titu1994@gmail.com> * Correct a bug with state management of adapter tokens Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pull down EMA test Signed-off-by: smajumdar <titu1994@gmail.com> * Correct name of adapter module utility class Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove unused TTS eval functions w/ pesq and pystoi dependencies (#5605) (#5606) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Create separator parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Call align function with hydra config Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update usage example Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update Dockerfile (#5614) (#5616) Pinned to use `numba==0.53.1` to avoid crashing in training with `num_workers > 0`. This is just a temporary workaround, still need to fix it in the future. Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make separate pretrained_name and model_path parameters Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * make "optional" tags bold in markdown Signed-off-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Move non-main functions to utils dir Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Temp workaround: Disable test with cache_audio=True since it is failing in CI (#5607) (#5615) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] fix ranges of char set for accented letters. (#5607) * [TTS] fix ranges of char set for accented letters. * remove digits pattern and added unit tests for math operators. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Change success message to reduce confusion (#5621) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update documentation and tutorials for Adapters (#5610) * Improve docs for adapter and tests Signed-off-by: smajumdar <titu1994@gmail.com> * Improve docs for adapter and tests Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update test Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename test file Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] add type hints and change varialbe names for tokenizers and g2p (#5602) * [TTS] add type hints and change variable names for tokenizers and g2p Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * 1. Added missing import for gather_objects. (#5627) Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] add fastpitch and hifigan model NGC urls and update NeMo docs. (#5596) (#5625) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fixed RadTTS unit test (#5572) Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove tests (#5633) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][DOC] add notes about automatic conversion to target sampling rates. (#5624) (#5634) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Conformer local attention (#5525) * local attn and merge Signed-off-by: sam1373 <samuelkriman@gmail.com> * optional Signed-off-by: sam1373 <samuelkriman@gmail.com> * override Signed-off-by: sam1373 <samuelkriman@gmail.com> * incorporate comments Signed-off-by: sam1373 <samuelkriman@gmail.com> * update Signed-off-by: sam1373 <samuelkriman@gmail.com> * fix Signed-off-by: sam1373 <samuelkriman@gmail.com> * comment Signed-off-by: sam1373 <samuelkriman@gmail.com> * changes, test Signed-off-by: sam1373 <samuelkriman@gmail.com> * changes Signed-off-by: sam1373 <samuelkriman@gmail.com> * check att context Signed-off-by: sam1373 <samuelkriman@gmail.com> * readme link Signed-off-by: sam1373 <samuelkriman@gmail.com> * utils Signed-off-by: sam1373 <samuelkriman@gmail.com> * update Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add core classes and functions for online clustering diarizer part 1 (#5526) * Add core classes and functions for online clustering diarizer Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add audio to labels code Signed-off-by: Taejin Park <tango4j@gmail.com> * resolve type errors Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added unit=tests for very short audio Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Filled all missing docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * resolved conflict and added missing docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed unit-test errors Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix the wrongly added file - megatron_gpt_model.py Signed-off-by: Taejin Park <tango4j@gmail.com> * Fix wrongly included file - megatron_gpt_model.py Signed-off-by: Taejin Park <tango4j@gmail.com> * resolve code quality issue Signed-off-by: Taejin Park <tango4j@gmail.com> * Fixed unit-test errors and bugs Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * changed total_sec for offline_clustering toy_data in unit-tests Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed merging index offset bug Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * only including part 1 files Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed unused function Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed unused imports Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * divided nmesc_clustering.py into two and reflected first-pass comments Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adding offline/online_clustering.py Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix code QL autocomment Signed-off-by: Taejin Park <tango4j@gmail.com> * Removed unused imports Signed-off-by: Taejin Park <tango4j@gmail.com> * Update nemo/collections/asr/parts/utils/online_clustering.py Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> * Reflected comments Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * resolved code scanning issue Signed-off-by: Taejin Park <tango4j@gmail.com> * Update nemo/collections/asr/parts/utils/offline_clustering.py Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [STT] Add Esperanto (Eo) ASR Conformer-CTC and Conformer-Transducer models (#5639) (#5641) * add stt_eo_conformer_ctc_large model * stt_eo_conformer_transducer_large Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Removed unused import Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Specify that filepaths need to be absolute Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * replaces any spaces in utt_id with dashes Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make hydra script callable by another script Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * do not specify default model or model_downsample_factor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [Dockerfile] Remove AIS archive from docker image (#5629) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Measure audio_sr from audio instead of needing to specify Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] Disambiguate polyphones with augmented dict and Jieba segmenter for Chinese FastPitch (#5541) * Chinese TTS replaces default pypinyin dict * Add jieba word segmenter as an option Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make separate parameters for device of transcription and viterbi steps Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add mention of gecko Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [workflow] add exclude labels option to ignore cherry-picks in release changelog. (#5645) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] bugfix for the tutorial and add NGC CLI installation guide. (#5643) (#5647) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [Add] ASR+VAD Inference Pipeline (#5575) Added offline ASR+VAD inference pipeline that matches with what's in RIVA, along with some feature-based ASR and classification datasets. Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * rename separator to ctm_grouping_separator and refactor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Bert interleaved (#5556) * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * Update bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Adding tests * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * Fix bug fix sequence parallel and Interleaved * Fix bug fix sequence parallel and Interleaved Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add duration padding support for RADTTS inference (#5650) * Added duration padding support for RADTTS inference * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: Kevin Shih <kshih@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add remove_blank_tokens_from_ctm parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Dont save initial_silence line in CTM Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add DLLogger support to exp_manager (#5658) * Add DLLogger support to exp_manager Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> * Move dllogger to separate file and check import Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add minimum_timestamp_duration parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add suggestion about removing blanks to README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * reorder args Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * clarify description of ctm_grouping_separator in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update docstring Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] bugfix for ngc cli installation. (#5652) (#5664) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Port stateless timer to exp manager (#5584) * Port stateless timer to exp manager Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes and remove from all megatron code Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change message Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix EMA restart by allowing device to be set by the class init (#5668) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove SDP (moved to separate repo) - merge to main (#5630) * Remove sdp files from tools folder Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add page to docs with new SDP location Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add interface for making amax reduction optional for FP8 (#5447) * add TE interface for making amax reduction optional Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] add tts dict cust notebook (#5662) * add tts dict cust notebook Signed-off-by: ekmb <ebakhturina@nvidia.com> * review Signed-off-by: ekmb <ebakhturina@nvidia.com> * fixed audio links Signed-off-by: ekmb <ebakhturina@nvidia.com> * remove old notebook Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix typo Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [ASR] Audio processing base, multi-channel enhancement models (#5356) * Audio processing base model, enc-mask-dec enhancement, tests and modules Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed review comments Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fixed CodeQL warnings Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed PR comments Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed PR comments: - renamed AudioProcessingModel to AudioToAudioModel - various small modifications - updated unit tests Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed comments - Moved spectrogram to audio_preprocessing - Renamed MultichannelFeatures - Updated config and unit tests Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Expose ClusteringDiarizer device (#5681) * Expose device for users to set Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Expose device for users to set Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add Beam Search support to ASR transcribe() (#5443) * Add support for beam decoding via high level API. Signed-off-by: smajumdar <titu1994@gmail.com> * Add ctc decoding section Signed-off-by: smajumdar <titu1994@gmail.com> * Update ctc transcribe API to return results from beam search Signed-off-by: smajumdar <titu1994@gmail.com> * Add argument to preserve arpa file Signed-off-by: smajumdar <titu1994@gmail.com> * Update script to use hydra config, add some support for future compute timesteps, add doc for ctc decoding Signed-off-by: smajumdar <titu1994@gmail.com> * Update eval script and doc to use new API Signed-off-by: smajumdar <titu1994@gmail.com> * Add tests for ctc greedy decoding Signed-off-by: smajumdar <titu1994@gmail.com> * Address reviewer comments and add docstrings Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix changes and address comments Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Propagate attention_dropout flag for GPT-3 (#5669) * Propagate attention_dropout flag for GPT-3 Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> * Add default to megatron_gpt_config Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail…

* Flashlight Decoder for Nemo (#5790) * Added initial flashlight decoding files Signed-off-by: Daniel Egert <degert@nvidia.com> * Fixed some minor bugs Signed-off-by: Daniel Egert <degert@nvidia.com> * Added fixes from auto style thingamajig Signed-off-by: Daniel Egert <degert@nvidia.com> * Ran pre-commit and fixed script file formatting Signed-off-by: Daniel Egert <degert@nvidia.com> * Minor fixes brought up during PR review Signed-off-by: Daniel Egert <degert@nvidia.com> * Added Windows-compatible code to eval_beamsearch_ngram.py Signed-off-by: Daniel Egert <degert@nvidia.com> * Added initial flashlight decoding files Signed-off-by: Daniel Egert <degert@nvidia.com> * Fixed some minor bugs Signed-off-by: Daniel Egert <degert@nvidia.com> * Added fixes from auto style thingamajig Signed-off-by: Daniel Egert <degert@nvidia.com> * Ran pre-commit and fixed script file formatting Signed-off-by: Daniel Egert <degert@nvidia.com> * Minor fixes brought up during PR review Signed-off-by: Daniel Egert <degert@nvidia.com> * Added Windows-compatible code to eval_beamsearch_ngram.py Signed-off-by: Daniel Egert <degert@nvidia.com> Signed-off-by: Daniel Egert <degert@nvidia.com> Co-authored-by: Daniel Egert <degert@nvidia.com> * update torchmetrics to latest version (#5801) (#5802) * update torchmetrics to latest version Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * pin min version for metrics Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> * Update radtts' infer path (#5788) * Update radtts' infer path Signed-off-by: Jason <jasoli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * update to newer code Signed-off-by: Jason <jasoli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add import Signed-off-by: Jason <jasoli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * move code to model Signed-off-by: Jason <jasoli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add input and output types property Signed-off-by: Jason <jasoli@nvidia.com> * remove comment Signed-off-by: Jason <jasoli@nvidia.com> * update volume Signed-off-by: Jason <jasoli@nvidia.com> * align volume Signed-off-by: Jason <jasoli@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jason <jasoli@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> * add constraint info on batch size for tar dataset (#5812) * add constraint info on batch size for tar dataset Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * style fix Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * NeMo Forced Aligner (#5571) * Merge r1.13.0 main (#5570) * update branch Signed-off-by: ericharper <complex451@gmail.com> * Rename Speech Dataset Processor to Speech Data Processor (#5378) Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Megatron Export Update (#5343) * export update for Megatron + change ORT optimization Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated export_utils to use autocast instead of manually casting >:/ Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * removed dtype from LayerNorm Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * added comment Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * reverting changes on FloatCast Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * Cherry-picked changes from megatron-norm Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * updated asr_model import to cast_utils Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * updated del onnx_model place Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * changed ort optimization to basic -> temp fix Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Boris Fomitchev <bfomitchev@nvidia.com> * Disable sync_batch_comm in validation_step for GPT (#5397) * disable sync_batch_comm in validation_step Signed-off-by: ericharper <complex451@gmail.com> * Read sync_batch_comm from config or default to False Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> * Update megatron_gpt_config to default sync_batch_comm to False to avoid CUDA error Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> * Empty Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Comment out test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Markel Sanz Ausin <markelsanz14@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Radtts 1.13 (#5451) * [TTS] Fixing RADTTS training - removing view buffer and fixing accuracy issue (#5358) * [TTS] add CI test for RADTTS training recipe. Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Support for finetuning and finetuning inference with .ckpt files & batch size refactoring (#5339) (#5478) * Initial refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Resolve config before passing to load_from_checkpoint Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for model parallel and nemo restore Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert config changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Minor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix validation reconfiguration Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove old comment Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes for test_ds Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * export_utils bugfix (#5480) * updated export_utils Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Export fixes for Riva (#5496) * Export fixes for Riva Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Cleaning up training_utils Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * added set_start_method + function param bugfix (#5539) * added set_start_method + function param bugfix Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * upper bound torchmetrics Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ericharper <complex451@gmail.com> * remove notebook (#5548) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update branch Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Markel Sanz Ausin <markelsanz14@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Optimized loop and bugfix in SDE (#5573) - Fixed bug with loading custom data attributes from JSON in Speech Data Explorer Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update torchmetrics (#5566) * add task arg Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update state Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove useless files. (#5580) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add initial NFA code Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make use of the specified device during viterbi decoding Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix CodeQL notes Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix CodeQL warning Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add an option to defer data setup from ``__init__`` to ``setup`` (#5569) * Add an option to defer dataloader setup from __init__ to setup Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Updated doc Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make utt_id specified by number of parts of audio_filepath user wishes to use Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove audio_sr TODO - reduce risk of silent bugs Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add check that model is CTC Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove unused import Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Text generation improvement (UI client, data parallel support) (#5437) * Squashed commit of the following: commit a5e124f34be31bd6eafe5e5fdf5bedcd0d50915c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 13 15:07:42 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 35b424044fe80c3081e7756ab21244f701716f7e Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 08:04:49 2022 -0700 get rid of base Signed-off-by: Yi Dong <yidong@nvidia.com> commit 2955210e2311791543538cfbb5ad26b79414c954 Merge: d52edef8c eaf6757ca Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:17:02 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit d52edef8cd7b36593838fb270047e80f8ccb652e Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:16:24 2022 +0000 align with main Signed-off-by: Yi Dong <yidong@nvidia.com> commit eaf6757ca5be8e099492f57c81d984429b0ad49c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 13 13:12:11 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit c4b86d97626ea0721bf8fb4c0a45dec5becc94c9 Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:10:58 2022 +0000 same as main Signed-off-by: Yi Dong <yidong@nvidia.com> commit e335de51bcc0d681c58b568c3d8c238bc5687c3b Merge: c231086e0 4463a9fe9 Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:08:09 2022 +0000 Merge branch 'main' into universal_prompt commit c231086e057f1efaa915f691d84664cb3d5aad85 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Oct 12 19:59:12 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 6a821a4b49a23dd3408a706a2a3dd393149b0bb1 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:56:17 2022 +0000 default to pad Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9d908e39fef1beed9ba2da4d1a6806161eb7ef25 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:55:44 2022 +0000 add the option to pad the tokens Signed-off-by: Yi Dong <yidong@nvidia.com> commit 876dc395b43fdeeaa2bcbbe13c76523633764c33 Merge: fbb0f4035 fe3c77ee9 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:20:47 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit fe3c77ee93ab6cf3ea152db68cb6beefcac2a392 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 18:59:49 2022 +0000 fix import again Signed-off-by: Yi Dong <yidong@nvidia.com> commit fbb0f4035c6cd6bfefed50a20605503de8c1dccb Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Oct 12 16:00:24 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 372ca8c0d7988f2339b15888dc72aa21f4fb6937 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 15:58:32 2022 +0000 enable server Signed-off-by: Yi Dong <yidong@nvidia.com> commit cbe05d9fbc978f812cfbb671f45f147f300713c4 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 13:07:28 2022 +0000 fix comment error Signed-off-by: Yi Dong <yidong@nvidia.com> commit 1948048922e726ec6131e44b1a745389f18d4ef2 Merge: 232c2cce3 984f5c09a Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 13:05:30 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit 232c2cce34d7a8b902da406706f3dd9b39475091 Merge: 34c8a68df 658243fb6 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 12:50:00 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit 984f5c09a6dbf1d1fb5aa30ed9b0df188e66a50f Merge: 658243fb6 3fda5de46 Author: Yi Dong <43824965+yidong72@users.noreply.github.com> Date: Wed Oct 12 08:42:11 2022 -0400 Merge branch 'main' into fix_global_init commit 658243fb6580191b5d60edd30cde16dcc23cbb85 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 12 12:40:57 2022 +0000 fix import error Signed-off-by: Yi Dong <doyend@gmail.com> commit 8e0fe1cad05ec288ec122b3cd0e139a96872e08c Author: Yi Dong <doyend@gmail.com> Date: Tue Oct 11 22:44:12 2022 +0000 update the fused kernel Signed-off-by: Yi Dong <doyend@gmail.com> commit 536cf6bef9447b75843fad630729c47a2fba35f3 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:44:52 2022 -0700 add the missing file Signed-off-by: Yi Dong <yidong@nvidia.com> commit 1b437ec41dc5e354453ce0a089bca0171cbcb6c2 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:43:14 2022 -0700 fix fused softmax Signed-off-by: Yi Dong <yidong@nvidia.com> commit 7813f60e05f9783af61f8c14ec1cb0c6c4f1f263 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:16:48 2022 -0700 move global step to base Signed-off-by: Yi Dong <yidong@nvidia.com> commit 34c8a68df084b18d377e84415d9f07b2cd6673dd Author: Yi Dong <doyend@gmail.com> Date: Thu Oct 6 13:50:11 2022 +0000 fix pipeline for eval Signed-off-by: Yi Dong <doyend@gmail.com> commit eee5d38218f26660c3ffebe9f615c850c80a1f0d Author: Yi Dong <doyend@gmail.com> Date: Thu Oct 6 13:48:22 2022 +0000 fix for pipleline parallel Signed-off-by: Yi Dong <doyend@gmail.com> commit 323bca73e7ef6099ee79c0a2fffac7b709ed6c5d Merge: 125e49947 e3b4c4d1f Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 19:29:13 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit 125e4994760448ff75dd9328395813eda1c87547 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 19:29:04 2022 +0000 add share option Signed-off-by: Yi Dong <doyend@gmail.com> commit e3b4c4d1f7346c9fa596f3cca6d4df0a9e05c368 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 5 11:43:48 2022 -0700 make sure consolidation works Signed-off-by: Yi Dong <yidong@nvidia.com> commit a5c833964ecf05dc460ca1da69275c4019742150 Merge: 2a07ab52d abcb74be2 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:40:29 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit 2a07ab52d95f15ba666823028c69e23825666c05 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:40:23 2022 +0000 added requirement Signed-off-by: Yi Dong <doyend@gmail.com> commit 3abecd9dd1611993a87c537636abe7f7e6a9b04c Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:39:42 2022 +0000 added a simple web server Signed-off-by: Yi Dong <doyend@gmail.com> commit abcb74be2caf1cdec40eb9ba2be4dde4d45a3b4b Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 5 06:54:12 2022 -0700 fix empty val loss Signed-off-by: Yi Dong <yidong@nvidia.com> commit b8eb92ac4a0d665570af75e34c9ba3c2e2420c26 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 19:25:30 2022 -0700 text gen working Signed-off-by: Yi Dong <yidong@nvidia.com> commit d59f3e3f3a6fd19736d1c5706fed65a3dd4049ba Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 16:08:40 2022 -0700 first change Signed-off-by: Yi Dong <yidong@nvidia.com> commit 59d077585e6962a669b824af58f64e8a0bea6547 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 15:00:40 2022 -0700 revert Signed-off-by: Yi Dong <yidong@nvidia.com> commit 12a0f3902d99e9179403644bd951c045df716ca7 Author: Yi Dong <doyend@gmail.com> Date: Tue Oct 4 21:26:23 2022 +0000 init imp Signed-off-by: Yi Dong <doyend@gmail.com> commit 62a15dfd943cc48be495ac61b9f2f00995775c5f Merge: 82c90d2cd e0cc6b767 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 11:58:26 2022 -0700 Merge branch 'main' into universal_prompt commit 82c90d2cd0fd156f16a4b899f8c741d598f33990 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 11:17:13 2022 -0700 add sync Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9819b703eef877d90cd1257bf3610c69de9b4d7e Author: Yi Dong <yidong@nvidia.com> Date: Sun Oct 2 17:52:34 2022 -0700 fix save model Signed-off-by: root <root@luna-0197.selene.nvidia.com> commit e4937e2fc5fb7d70754c97668416e4a69c3079fe Author: Yi Dong <yidong@nvidia.com> Date: Sat Oct 1 18:56:09 2022 +0000 working Signed-off-by: Yi Dong <yidong@nvidia.com> commit b73b06d1c7cf5417a6d87cb33d8ed83a57e38b7b Author: Yi Dong <yidong@nvidia.com> Date: Sat Oct 1 17:34:03 2022 +0000 calcuate the mask Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9db3bc13eb65a94a475b837603351da68e3745bc Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 23:26:32 2022 +0000 fix bug in datasets Signed-off-by: Yi Dong <yidong@nvidia.com> commit f289900375d4412f53f8110be00fec6587627550 Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 22:29:40 2022 +0000 update the code Signed-off-by: Yi Dong <yidong@nvidia.com> commit 8e28a1f208aabaab72dbe769e72756baada04d99 Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 21:52:52 2022 +0000 added new ds Signed-off-by: Yi Dong <yidong@nvidia.com> commit 8d41315bab7ce90e200a8a7d1023c34f8e046897 Author: Yi Dong <doyend@gmail.com> Date: Fri Sep 30 18:57:09 2022 +0000 added new files Signed-off-by: Yi Dong <doyend@gmail.com> commit 984e0e94e15e16323c1ba1ca2efeabd84f69463f Merge: cbe8b7ab1 fa6cd8588 Author: Yi Dong <doyend@gmail.com> Date: Thu Sep 29 21:43:29 2022 +0000 Merge branch 'llm-prompt-learning-improvements' into universal_prompt commit fa6cd858839277939446afe7275976078d54c512 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Sep 29 16:47:30 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 78ba46e5d6fde1be53c08e1e30a54cce59824be0 Merge: 7d6d46742 8d670bc77 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Thu Sep 29 09:43:27 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 7d6d46742170a66758287a207d67e1b1bfd15613 Author: Virginia Adams <vadams@nvidia.com> Date: Thu Sep 29 16:42:43 2022 +0000 Removed inference step and added sentence peice check to predict step Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 20fd265acd6f7f9912cf52155fe66ccfa6b201a2 Author: Virginia Adams <vadams@nvidia.com> Date: Thu Sep 29 15:26:32 2022 +0000 fixed first stage check for pipeline parallel T5 pt Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 3637be2b258c8d9028856f9971edb7da4a8121f0 Merge: a3ea722fd 986a76612 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Wed Sep 28 10:23:30 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit a3ea722fdc12fbcc5989b76ef5643a574b763bc4 Merge: 770967a52 971485ce7 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Mon Sep 26 13:35:52 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 770967a5251a474b6dcc2d44bf9a2076adbcb604 Merge: d23bf6c30 e3ac280a8 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Mon Sep 26 10:17:03 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit d23bf6c30acc0e3f6af9b4e24547669866a34d62 Merge: de6a31651 333d2b749 Author: Virginia Adams <vadams@nvidia.com> Date: Mon Sep 26 10:05:16 2022 -0700 Merge branch 'llm-prompt-learning-improvements' of https://github.com/NVIDIA/NeMo into llm-prompt-learning-improvements commit de6a31651e63d88a42b971794d93f18ff5a3cdff Author: Virginia Adams <vadams@nvidia.com> Date: Mon Sep 26 17:00:53 2022 +0000 Updated PP check to be on first stage pipeline only Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 333d2b7498e6742ce66436f733c980a74616900c Merge: 592c0986a a39fc925a Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Fri Sep 23 16:11:21 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 592c0986a476a91b57b8605d7b70830d7acfa021 Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 23:08:41 2022 +0000 Fixed unused import and CI test bug Signed-off-by: Virginia Adams <vadams@nvidia.com> commit ea9cd82d85638bc60ae4ad7ef105db931c8e3455 Merge: ce4b72c8c b566c2d0e Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 18:57:25 2022 +0000 Merge branch 'llm-prompt-learning-improvements' of https://github.com/NVIDIA/NeMo into llm-prompt-learning-improvements commit ce4b72c8c52f32be336e323dd78a38089edc3e7c Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 18:57:16 2022 +0000 Switch to import from base class Signed-off-by: Virginia Adams <vadams@nvidia.com> commit b566c2d0e35a068f758fd1310bc620a47be4590b Merge: 6621f2854 e872061ac Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Fri Sep 23 10:09:03 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 6621f28543828a48484a5637f6c9f3ccb23a5b02 Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 20:47:35 2022 +0000 python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 8deafc8987b6af5f7b99a250310f57a40198c37f Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 20:28:02 2022 +0000 Save .nemo on new best val score Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 761bd36969cb465d6a129e9eee6ce1f883d3cf41 Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 18:03:19 2022 +0000 Added automatic checkpoint to nemo file method Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 3be4ed57b6cd3ddfe4876d78650dfe8fe794598b Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 02:11:56 2022 +0000 Make GPT use base prompt learning model class: Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> * fix LGTM Signed-off-by: Yi Dong <yidong@nvidia.com> * fix validation Signed-off-by: Yi Dong <yidong@nvidia.com> * change for the lm eval Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * make text generation work in data parallel environment Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * implement the service with rest service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * surpress log Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore function needed for NMT Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * handles no answer only Signed-off-by: Yi Dong <yidong@nvidia.com> * Fix config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added knn to web Signed-off-by: Yi Dong <yidong@nvidia.com> * fix lgtm.com comments Signed-off-by: Yi Dong <yidong@nvidia.com> * output the retrieved context Signed-off-by: Yi Dong <yidong@nvidia.com> * allow no neighbor query Signed-off-by: Yi Dong <yidong@nvidia.com> * remove the imports Signed-off-by: Yi Dong <yidong@nvidia.com> * warn only once Signed-off-by: Yi Dong <yidong@nvidia.com> * Change output file format from JSON to JSONL Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * new t0 dataset Signed-off-by: Yi Dong <yidong@nvidia.com> * Add T0 data preproc scripts Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Merge and multiprocessing Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix for is_correct Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * fix epoch > 2 Signed-off-by: Yi Dong <yidong@nvidia.com> * handles multiple dataloader Signed-off-by: Yi Dong <yidong@nvidia.com> * remove template Signed-off-by: Yi Dong <yidong@nvidia.com> * Refactor T0 dataset Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add script to merge train folder into individual training files to minimize number of blends Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added on the fly service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add combo instance Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added combo service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * send weights back to server Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix index store Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Minor changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add reset button Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add add eos Signed-off-by: Yi Dong <yidong@nvidia.com> * use a seperate bert service Signed-off-by: Yi Dong <yidong@nvidia.com> * no loss of accuracy Signed-off-by: Yi Dong <yidong@nvidia.com> * pin the gradio version Signed-off-by: Yi Dong <yidong@nvidia.com> * Remove bin compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix header lines Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * evaluate based on text generation Signed-off-by: Yi Dong <yidong@nvidia.com> * exact match result aggregation Signed-off-by: Yi Dong <yidong@nvidia.com> * working SP and SA Signed-off-by: Yi Dong <yidong@nvidia.com> * sync Signed-off-by: Yi Dong <yidong@nvidia.com> * fix checkpoint Signed-off-by: Yi Dong <yidong@nvidia.com> * fix eval Signed-off-by: Yi Dong <yidong@nvidia.com> * backup states Signed-off-by: Yi Dong <yidong@nvidia.com> * backup states reset Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the bug Signed-off-by: Yi Dong <yidong@nvidia.com> * fix evaluation for sentence piece Signed-off-by: Yi Dong <yidong@nvidia.com> * fix a bug Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * potential fix in the future Signed-off-by: Yi Dong <yidong@nvidia.com> * remove the universal codes Signed-off-by: Yi Dong <yidong@nvidia.com> * remove universal strategy Signed-off-by: Yi Dong <yidong@nvidia.com> * address reviewer comment Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add align function docstrings and make most args optional Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove redundant returns of viterbi and log probs matrices Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Rename h# to <initial_silence> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update manifest format description in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * always remove any spaces from utt_id Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Patch the hanging of threads on very large stderr (#5589) (#5590) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * O2 style amp for gpt3 ptuning (#5246) * enable amp o2 plugin Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * only create master param if param requires gradient Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * remove pytorch autocast Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Update optimizer_with_main_params.py Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> * create master grad only if param group requires grad Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix grad scaler for pp > 1 Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Better patch hydra (#5591) (#5592) * Readd buffereing and thread drain to Hydra Launcher Signed-off-by: smajumdar <titu1994@gmail.com> * Readd buffereing and thread drain to Hydra Launcher Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Yet another fix with hydra multirun (#5594) (#5595) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add RETRO model documentation (#5578) * added retro doc Signed-off-by: Yi Dong <yidong@nvidia.com> * finish data part Signed-off-by: Yi Dong <yidong@nvidia.com> * added the data format Signed-off-by: Yi Dong <yidong@nvidia.com> * added training script Signed-off-by: Yi Dong <yidong@nvidia.com> * added training and evaluation steps Signed-off-by: Yi Dong <yidong@nvidia.com> * edit the text Signed-off-by: Yi Dong <yidong@nvidia.com> * added the images Signed-off-by: Yi Dong <yidong@nvidia.com> * fix beginning Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the grammar Signed-off-by: Yi Dong <yidong@nvidia.com> * trim it down Signed-off-by: Yi Dong <yidong@nvidia.com> * add wandb option Signed-off-by: Yi Dong <yidong@nvidia.com> * add reference Signed-off-by: Yi Dong <yidong@nvidia.com> * fix path Signed-off-by: Yi Dong <yidong@nvidia.com> * added the parameters table Signed-off-by: Yi Dong <yidong@nvidia.com> * fix section Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix: setup_multiple validation/test data (#5585) Fix: setup_multiple validation/test data (#5585) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Move to optimizer based EMA implementation (#5169) * Move to optimizer Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix replacing weights Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Allow swapping of weights be optional Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Save 2 models Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Use different hook Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Expose cpu device Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add clause to see if this fixes issue with O2 optimizer Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Try to get O2 working Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * WIP Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fixes Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fixes to tests Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add guard Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Remove import Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add guard Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add comment Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Remove overwrite Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add BatchNorm, currently tests fail Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix tests/functionality for batch norm Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Get rid of NLP changes Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * AIStore for ASR datasets (#5462) AIStore for ASR datasets Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add support for MHA adapters to ASR (#5396) * Convert AbstractAdapterModule to AbstractAdapterMixin Signed-off-by: smajumdar <titu1994@gmail.com> * Temporary fixes to new signature of mixin Signed-off-by: smajumdar <titu1994@gmail.com> * Add adapter util for constants, add all mha adapters. Signed-off-by: smajumdar <titu1994@gmail.com> * Update name of function Signed-off-by: smajumdar <titu1994@gmail.com> * Roll back changes to convASR Signed-off-by: smajumdar <titu1994@gmail.com> * Convert AbstractAdapterModule to AbstractAdapterMixin Signed-off-by: smajumdar <titu1994@gmail.com> * First draft of Conformer support for MHA attention Signed-off-by: smajumdar <titu1994@gmail.com> * Add some preliminary tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for projection of the hidden dimension for attention Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for squeezeformer Signed-off-by: smajumdar <titu1994@gmail.com> * Update train adapter config Signed-off-by: smajumdar <titu1994@gmail.com> * Add tests for squeezeformer and unit tests for new modules Signed-off-by: smajumdar <titu1994@gmail.com> * Update config for hp search,set limits on modules for conformer and squeezeformer, update adapter mixin, add cache to import_from_class_path Signed-off-by: smajumdar <titu1994@gmail.com> * Update location of adapters Signed-off-by: smajumdar <titu1994@gmail.com> * Add pre_norm for proper attention learning, Fix the issue with nan/inf in pos_bias_u and pos_bias_v Signed-off-by: smajumdar <titu1994@gmail.com> * Update expmanager to clean up checkpoints Signed-off-by: smajumdar <titu1994@gmail.com> * Fix style Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update training scripts Signed-off-by: smajumdar <titu1994@gmail.com> * Update config and docs Signed-off-by: smajumdar <titu1994@gmail.com> * Expose nemo delete function Signed-off-by: smajumdar <titu1994@gmail.com> * Correct adapter partial state saving Signed-off-by: smajumdar <titu1994@gmail.com> * Correct a bug with state management of adapter tokens Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pull down EMA test Signed-off-by: smajumdar <titu1994@gmail.com> * Correct name of adapter module utility class Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove unused TTS eval functions w/ pesq and pystoi dependencies (#5605) (#5606) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Create separator parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Call align function with hydra config Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update usage example Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update Dockerfile (#5614) (#5616) Pinned to use `numba==0.53.1` to avoid crashing in training with `num_workers > 0`. This is just a temporary workaround, still need to fix it in the future. Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make separate pretrained_name and model_path parameters Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * make "optional" tags bold in markdown Signed-off-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Move non-main functions to utils dir Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Temp workaround: Disable test with cache_audio=True since it is failing in CI (#5607) (#5615) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] fix ranges of char set for accented letters. (#5607) * [TTS] fix ranges of char set for accented letters. * remove digits pattern and added unit tests for math operators. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Change success message to reduce confusion (#5621) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update documentation and tutorials for Adapters (#5610) * Improve docs for adapter and tests Signed-off-by: smajumdar <titu1994@gmail.com> * Improve docs for adapter and tests Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update test Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename test file Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] add type hints and change varialbe names for tokenizers and g2p (#5602) * [TTS] add type hints and change variable names for tokenizers and g2p Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * 1. Added missing import for gather_objects. (#5627) Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] add fastpitch and hifigan model NGC urls and update NeMo docs. (#5596) (#5625) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fixed RadTTS unit test (#5572) Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove tests (#5633) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][DOC] add notes about automatic conversion to target sampling rates. (#5624) (#5634) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Conformer local attention (#5525) * local attn and merge Signed-off-by: sam1373 <samuelkriman@gmail.com> * optional Signed-off-by: sam1373 <samuelkriman@gmail.com> * override Signed-off-by: sam1373 <samuelkriman@gmail.com> * incorporate comments Signed-off-by: sam1373 <samuelkriman@gmail.com> * update Signed-off-by: sam1373 <samuelkriman@gmail.com> * fix Signed-off-by: sam1373 <samuelkriman@gmail.com> * comment Signed-off-by: sam1373 <samuelkriman@gmail.com> * changes, test Signed-off-by: sam1373 <samuelkriman@gmail.com> * changes Signed-off-by: sam1373 <samuelkriman@gmail.com> * check att context Signed-off-by: sam1373 <samuelkriman@gmail.com> * readme link Signed-off-by: sam1373 <samuelkriman@gmail.com> * utils Signed-off-by: sam1373 <samuelkriman@gmail.com> * update Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add core classes and functions for online clustering diarizer part 1 (#5526) * Add core classes and functions for online clustering diarizer Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add audio to labels code Signed-off-by: Taejin Park <tango4j@gmail.com> * resolve type errors Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added unit=tests for very short audio Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Filled all missing docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * resolved conflict and added missing docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed unit-test errors Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix the wrongly added file - megatron_gpt_model.py Signed-off-by: Taejin Park <tango4j@gmail.com> * Fix wrongly included file - megatron_gpt_model.py Signed-off-by: Taejin Park <tango4j@gmail.com> * resolve code quality issue Signed-off-by: Taejin Park <tango4j@gmail.com> * Fixed unit-test errors and bugs Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * changed total_sec for offline_clustering toy_data in unit-tests Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed merging index offset bug Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * only including part 1 files Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed unused function Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed unused imports Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * divided nmesc_clustering.py into two and reflected first-pass comments Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adding offline/online_clustering.py Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix code QL autocomment Signed-off-by: Taejin Park <tango4j@gmail.com> * Removed unused imports Signed-off-by: Taejin Park <tango4j@gmail.com> * Update nemo/collections/asr/parts/utils/online_clustering.py Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> * Reflected comments Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * resolved code scanning issue Signed-off-by: Taejin Park <tango4j@gmail.com> * Update nemo/collections/asr/parts/utils/offline_clustering.py Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [STT] Add Esperanto (Eo) ASR Conformer-CTC and Conformer-Transducer models (#5639) (#5641) * add stt_eo_conformer_ctc_large model * stt_eo_conformer_transducer_large Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Removed unused import Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Specify that filepaths need to be absolute Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * replaces any spaces in utt_id with dashes Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make hydra script callable by another script Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * do not specify default model or model_downsample_factor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [Dockerfile] Remove AIS archive from docker image (#5629) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Measure audio_sr from audio instead of needing to specify Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] Disambiguate polyphones with augmented dict and Jieba segmenter for Chinese FastPitch (#5541) * Chinese TTS replaces default pypinyin dict * Add jieba word segmenter as an option Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make separate parameters for device of transcription and viterbi steps Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add mention of gecko Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [workflow] add exclude labels option to ignore cherry-picks in release changelog. (#5645) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] bugfix for the tutorial and add NGC CLI installation guide. (#5643) (#5647) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [Add] ASR+VAD Inference Pipeline (#5575) Added offline ASR+VAD inference pipeline that matches with what's in RIVA, along with some feature-based ASR and classification datasets. Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * rename separator to ctm_grouping_separator and refactor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Bert interleaved (#5556) * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * Update bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Adding tests * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * Fix bug fix sequence parallel and Interleaved * Fix bug fix sequence parallel and Interleaved Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add duration padding support for RADTTS inference (#5650) * Added duration padding support for RADTTS inference * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: Kevin Shih <kshih@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add remove_blank_tokens_from_ctm parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Dont save initial_silence line in CTM Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add DLLogger support to exp_manager (#5658) * Add DLLogger support to exp_manager Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> * Move dllogger to separate file and check import Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add minimum_timestamp_duration parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add suggestion about removing blanks to README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * reorder args Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * clarify description of ctm_grouping_separator in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update docstring Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] bugfix for ngc cli installation. (#5652) (#5664) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Port stateless timer to exp manager (#5584) * Port stateless timer to exp manager Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes and remove from all megatron code Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change message Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix EMA restart by allowing device to be set by the class init (#5668) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove SDP (moved to separate repo) - merge to main (#5630) * Remove sdp files from tools folder Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add page to docs with new SDP location Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add interface for making amax reduction optional for FP8 (#5447) * add TE interface for making amax reduction optional Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] add tts …

* Merge r1.13.0 main (#5570) * update branch Signed-off-by: ericharper <complex451@gmail.com> * Rename Speech Dataset Processor to Speech Data Processor (#5378) Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Megatron Export Update (#5343) * export update for Megatron + change ORT optimization Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated export_utils to use autocast instead of manually casting >:/ Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * removed dtype from LayerNorm Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * added comment Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * reverting changes on FloatCast Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * Cherry-picked changes from megatron-norm Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * updated asr_model import to cast_utils Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * updated del onnx_model place Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * changed ort optimization to basic -> temp fix Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Boris Fomitchev <bfomitchev@nvidia.com> * Disable sync_batch_comm in validation_step for GPT (#5397) * disable sync_batch_comm in validation_step Signed-off-by: ericharper <complex451@gmail.com> * Read sync_batch_comm from config or default to False Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> * Update megatron_gpt_config to default sync_batch_comm to False to avoid CUDA error Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> * Empty Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Comment out test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Markel Sanz Ausin <markelsanz14@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Radtts 1.13 (#5451) * [TTS] Fixing RADTTS training - removing view buffer and fixing accuracy issue (#5358) * [TTS] add CI test for RADTTS training recipe. Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Support for finetuning and finetuning inference with .ckpt files & batch size refactoring (#5339) (#5478) * Initial refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Resolve config before passing to load_from_checkpoint Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for model parallel and nemo restore Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert config changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Minor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix validation reconfiguration Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove old comment Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes for test_ds Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * export_utils bugfix (#5480) * updated export_utils Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Export fixes for Riva (#5496) * Export fixes for Riva Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Cleaning up training_utils Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * added set_start_method + function param bugfix (#5539) * added set_start_method + function param bugfix Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * upper bound torchmetrics Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ericharper <complex451@gmail.com> * remove notebook (#5548) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update branch Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Markel Sanz Ausin <markelsanz14@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Optimized loop and bugfix in SDE (#5573) - Fixed bug with loading custom data attributes from JSON in Speech Data Explorer Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update torchmetrics (#5566) * add task arg Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update state Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove useless files. (#5580) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add initial NFA code Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make use of the specified device during viterbi decoding Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix CodeQL notes Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix CodeQL warning Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add an option to defer data setup from ``__init__`` to ``setup`` (#5569) * Add an option to defer dataloader setup from __init__ to setup Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Updated doc Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make utt_id specified by number of parts of audio_filepath user wishes to use Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove audio_sr TODO - reduce risk of silent bugs Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add check that model is CTC Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove unused import Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Text generation improvement (UI client, data parallel support) (#5437) * Squashed commit of the following: commit a5e124f34be31bd6eafe5e5fdf5bedcd0d50915c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 13 15:07:42 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 35b424044fe80c3081e7756ab21244f701716f7e Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 08:04:49 2022 -0700 get rid of base Signed-off-by: Yi Dong <yidong@nvidia.com> commit 2955210e2311791543538cfbb5ad26b79414c954 Merge: d52edef8c eaf6757ca Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:17:02 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit d52edef8cd7b36593838fb270047e80f8ccb652e Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:16:24 2022 +0000 align with main Signed-off-by: Yi Dong <yidong@nvidia.com> commit eaf6757ca5be8e099492f57c81d984429b0ad49c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 13 13:12:11 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit c4b86d97626ea0721bf8fb4c0a45dec5becc94c9 Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:10:58 2022 +0000 same as main Signed-off-by: Yi Dong <yidong@nvidia.com> commit e335de51bcc0d681c58b568c3d8c238bc5687c3b Merge: c231086e0 4463a9fe9 Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:08:09 2022 +0000 Merge branch 'main' into universal_prompt commit c231086e057f1efaa915f691d84664cb3d5aad85 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Oct 12 19:59:12 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 6a821a4b49a23dd3408a706a2a3dd393149b0bb1 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:56:17 2022 +0000 default to pad Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9d908e39fef1beed9ba2da4d1a6806161eb7ef25 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:55:44 2022 +0000 add the option to pad the tokens Signed-off-by: Yi Dong <yidong@nvidia.com> commit 876dc395b43fdeeaa2bcbbe13c76523633764c33 Merge: fbb0f4035 fe3c77ee9 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:20:47 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit fe3c77ee93ab6cf3ea152db68cb6beefcac2a392 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 18:59:49 2022 +0000 fix import again Signed-off-by: Yi Dong <yidong@nvidia.com> commit fbb0f4035c6cd6bfefed50a20605503de8c1dccb Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Oct 12 16:00:24 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 372ca8c0d7988f2339b15888dc72aa21f4fb6937 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 15:58:32 2022 +0000 enable server Signed-off-by: Yi Dong <yidong@nvidia.com> commit cbe05d9fbc978f812cfbb671f45f147f300713c4 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 13:07:28 2022 +0000 fix comment error Signed-off-by: Yi Dong <yidong@nvidia.com> commit 1948048922e726ec6131e44b1a745389f18d4ef2 Merge: 232c2cce3 984f5c09a Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 13:05:30 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit 232c2cce34d7a8b902da406706f3dd9b39475091 Merge: 34c8a68df 658243fb6 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 12:50:00 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit 984f5c09a6dbf1d1fb5aa30ed9b0df188e66a50f Merge: 658243fb6 3fda5de46 Author: Yi Dong <43824965+yidong72@users.noreply.github.com> Date: Wed Oct 12 08:42:11 2022 -0400 Merge branch 'main' into fix_global_init commit 658243fb6580191b5d60edd30cde16dcc23cbb85 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 12 12:40:57 2022 +0000 fix import error Signed-off-by: Yi Dong <doyend@gmail.com> commit 8e0fe1cad05ec288ec122b3cd0e139a96872e08c Author: Yi Dong <doyend@gmail.com> Date: Tue Oct 11 22:44:12 2022 +0000 update the fused kernel Signed-off-by: Yi Dong <doyend@gmail.com> commit 536cf6bef9447b75843fad630729c47a2fba35f3 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:44:52 2022 -0700 add the missing file Signed-off-by: Yi Dong <yidong@nvidia.com> commit 1b437ec41dc5e354453ce0a089bca0171cbcb6c2 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:43:14 2022 -0700 fix fused softmax Signed-off-by: Yi Dong <yidong@nvidia.com> commit 7813f60e05f9783af61f8c14ec1cb0c6c4f1f263 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:16:48 2022 -0700 move global step to base Signed-off-by: Yi Dong <yidong@nvidia.com> commit 34c8a68df084b18d377e84415d9f07b2cd6673dd Author: Yi Dong <doyend@gmail.com> Date: Thu Oct 6 13:50:11 2022 +0000 fix pipeline for eval Signed-off-by: Yi Dong <doyend@gmail.com> commit eee5d38218f26660c3ffebe9f615c850c80a1f0d Author: Yi Dong <doyend@gmail.com> Date: Thu Oct 6 13:48:22 2022 +0000 fix for pipleline parallel Signed-off-by: Yi Dong <doyend@gmail.com> commit 323bca73e7ef6099ee79c0a2fffac7b709ed6c5d Merge: 125e49947 e3b4c4d1f Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 19:29:13 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit 125e4994760448ff75dd9328395813eda1c87547 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 19:29:04 2022 +0000 add share option Signed-off-by: Yi Dong <doyend@gmail.com> commit e3b4c4d1f7346c9fa596f3cca6d4df0a9e05c368 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 5 11:43:48 2022 -0700 make sure consolidation works Signed-off-by: Yi Dong <yidong@nvidia.com> commit a5c833964ecf05dc460ca1da69275c4019742150 Merge: 2a07ab52d abcb74be2 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:40:29 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit 2a07ab52d95f15ba666823028c69e23825666c05 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:40:23 2022 +0000 added requirement Signed-off-by: Yi Dong <doyend@gmail.com> commit 3abecd9dd1611993a87c537636abe7f7e6a9b04c Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:39:42 2022 +0000 added a simple web server Signed-off-by: Yi Dong <doyend@gmail.com> commit abcb74be2caf1cdec40eb9ba2be4dde4d45a3b4b Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 5 06:54:12 2022 -0700 fix empty val loss Signed-off-by: Yi Dong <yidong@nvidia.com> commit b8eb92ac4a0d665570af75e34c9ba3c2e2420c26 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 19:25:30 2022 -0700 text gen working Signed-off-by: Yi Dong <yidong@nvidia.com> commit d59f3e3f3a6fd19736d1c5706fed65a3dd4049ba Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 16:08:40 2022 -0700 first change Signed-off-by: Yi Dong <yidong@nvidia.com> commit 59d077585e6962a669b824af58f64e8a0bea6547 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 15:00:40 2022 -0700 revert Signed-off-by: Yi Dong <yidong@nvidia.com> commit 12a0f3902d99e9179403644bd951c045df716ca7 Author: Yi Dong <doyend@gmail.com> Date: Tue Oct 4 21:26:23 2022 +0000 init imp Signed-off-by: Yi Dong <doyend@gmail.com> commit 62a15dfd943cc48be495ac61b9f2f00995775c5f Merge: 82c90d2cd e0cc6b767 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 11:58:26 2022 -0700 Merge branch 'main' into universal_prompt commit 82c90d2cd0fd156f16a4b899f8c741d598f33990 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 11:17:13 2022 -0700 add sync Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9819b703eef877d90cd1257bf3610c69de9b4d7e Author: Yi Dong <yidong@nvidia.com> Date: Sun Oct 2 17:52:34 2022 -0700 fix save model Signed-off-by: root <root@luna-0197.selene.nvidia.com> commit e4937e2fc5fb7d70754c97668416e4a69c3079fe Author: Yi Dong <yidong@nvidia.com> Date: Sat Oct 1 18:56:09 2022 +0000 working Signed-off-by: Yi Dong <yidong@nvidia.com> commit b73b06d1c7cf5417a6d87cb33d8ed83a57e38b7b Author: Yi Dong <yidong@nvidia.com> Date: Sat Oct 1 17:34:03 2022 +0000 calcuate the mask Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9db3bc13eb65a94a475b837603351da68e3745bc Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 23:26:32 2022 +0000 fix bug in datasets Signed-off-by: Yi Dong <yidong@nvidia.com> commit f289900375d4412f53f8110be00fec6587627550 Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 22:29:40 2022 +0000 update the code Signed-off-by: Yi Dong <yidong@nvidia.com> commit 8e28a1f208aabaab72dbe769e72756baada04d99 Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 21:52:52 2022 +0000 added new ds Signed-off-by: Yi Dong <yidong@nvidia.com> commit 8d41315bab7ce90e200a8a7d1023c34f8e046897 Author: Yi Dong <doyend@gmail.com> Date: Fri Sep 30 18:57:09 2022 +0000 added new files Signed-off-by: Yi Dong <doyend@gmail.com> commit 984e0e94e15e16323c1ba1ca2efeabd84f69463f Merge: cbe8b7ab1 fa6cd8588 Author: Yi Dong <doyend@gmail.com> Date: Thu Sep 29 21:43:29 2022 +0000 Merge branch 'llm-prompt-learning-improvements' into universal_prompt commit fa6cd858839277939446afe7275976078d54c512 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Sep 29 16:47:30 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 78ba46e5d6fde1be53c08e1e30a54cce59824be0 Merge: 7d6d46742 8d670bc77 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Thu Sep 29 09:43:27 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 7d6d46742170a66758287a207d67e1b1bfd15613 Author: Virginia Adams <vadams@nvidia.com> Date: Thu Sep 29 16:42:43 2022 +0000 Removed inference step and added sentence peice check to predict step Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 20fd265acd6f7f9912cf52155fe66ccfa6b201a2 Author: Virginia Adams <vadams@nvidia.com> Date: Thu Sep 29 15:26:32 2022 +0000 fixed first stage check for pipeline parallel T5 pt Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 3637be2b258c8d9028856f9971edb7da4a8121f0 Merge: a3ea722fd 986a76612 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Wed Sep 28 10:23:30 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit a3ea722fdc12fbcc5989b76ef5643a574b763bc4 Merge: 770967a52 971485ce7 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Mon Sep 26 13:35:52 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 770967a5251a474b6dcc2d44bf9a2076adbcb604 Merge: d23bf6c30 e3ac280a8 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Mon Sep 26 10:17:03 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit d23bf6c30acc0e3f6af9b4e24547669866a34d62 Merge: de6a31651 333d2b749 Author: Virginia Adams <vadams@nvidia.com> Date: Mon Sep 26 10:05:16 2022 -0700 Merge branch 'llm-prompt-learning-improvements' of https://github.com/NVIDIA/NeMo into llm-prompt-learning-improvements commit de6a31651e63d88a42b971794d93f18ff5a3cdff Author: Virginia Adams <vadams@nvidia.com> Date: Mon Sep 26 17:00:53 2022 +0000 Updated PP check to be on first stage pipeline only Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 333d2b7498e6742ce66436f733c980a74616900c Merge: 592c0986a a39fc925a Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Fri Sep 23 16:11:21 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 592c0986a476a91b57b8605d7b70830d7acfa021 Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 23:08:41 2022 +0000 Fixed unused import and CI test bug Signed-off-by: Virginia Adams <vadams@nvidia.com> commit ea9cd82d85638bc60ae4ad7ef105db931c8e3455 Merge: ce4b72c8c b566c2d0e Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 18:57:25 2022 +0000 Merge branch 'llm-prompt-learning-improvements' of https://github.com/NVIDIA/NeMo into llm-prompt-learning-improvements commit ce4b72c8c52f32be336e323dd78a38089edc3e7c Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 18:57:16 2022 +0000 Switch to import from base class Signed-off-by: Virginia Adams <vadams@nvidia.com> commit b566c2d0e35a068f758fd1310bc620a47be4590b Merge: 6621f2854 e872061ac Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Fri Sep 23 10:09:03 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 6621f28543828a48484a5637f6c9f3ccb23a5b02 Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 20:47:35 2022 +0000 python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 8deafc8987b6af5f7b99a250310f57a40198c37f Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 20:28:02 2022 +0000 Save .nemo on new best val score Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 761bd36969cb465d6a129e9eee6ce1f883d3cf41 Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 18:03:19 2022 +0000 Added automatic checkpoint to nemo file method Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 3be4ed57b6cd3ddfe4876d78650dfe8fe794598b Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 02:11:56 2022 +0000 Make GPT use base prompt learning model class: Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> * fix LGTM Signed-off-by: Yi Dong <yidong@nvidia.com> * fix validation Signed-off-by: Yi Dong <yidong@nvidia.com> * change for the lm eval Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * make text generation work in data parallel environment Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * implement the service with rest service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * surpress log Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore function needed for NMT Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * handles no answer only Signed-off-by: Yi Dong <yidong@nvidia.com> * Fix config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added knn to web Signed-off-by: Yi Dong <yidong@nvidia.com> * fix lgtm.com comments Signed-off-by: Yi Dong <yidong@nvidia.com> * output the retrieved context Signed-off-by: Yi Dong <yidong@nvidia.com> * allow no neighbor query Signed-off-by: Yi Dong <yidong@nvidia.com> * remove the imports Signed-off-by: Yi Dong <yidong@nvidia.com> * warn only once Signed-off-by: Yi Dong <yidong@nvidia.com> * Change output file format from JSON to JSONL Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * new t0 dataset Signed-off-by: Yi Dong <yidong@nvidia.com> * Add T0 data preproc scripts Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Merge and multiprocessing Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix for is_correct Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * fix epoch > 2 Signed-off-by: Yi Dong <yidong@nvidia.com> * handles multiple dataloader Signed-off-by: Yi Dong <yidong@nvidia.com> * remove template Signed-off-by: Yi Dong <yidong@nvidia.com> * Refactor T0 dataset Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add script to merge train folder into individual training files to minimize number of blends Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added on the fly service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add combo instance Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added combo service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * send weights back to server Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix index store Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Minor changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add reset button Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add add eos Signed-off-by: Yi Dong <yidong@nvidia.com> * use a seperate bert service Signed-off-by: Yi Dong <yidong@nvidia.com> * no loss of accuracy Signed-off-by: Yi Dong <yidong@nvidia.com> * pin the gradio version Signed-off-by: Yi Dong <yidong@nvidia.com> * Remove bin compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix header lines Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * evaluate based on text generation Signed-off-by: Yi Dong <yidong@nvidia.com> * exact match result aggregation Signed-off-by: Yi Dong <yidong@nvidia.com> * working SP and SA Signed-off-by: Yi Dong <yidong@nvidia.com> * sync Signed-off-by: Yi Dong <yidong@nvidia.com> * fix checkpoint Signed-off-by: Yi Dong <yidong@nvidia.com> * fix eval Signed-off-by: Yi Dong <yidong@nvidia.com> * backup states Signed-off-by: Yi Dong <yidong@nvidia.com> * backup states reset Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the bug Signed-off-by: Yi Dong <yidong@nvidia.com> * fix evaluation for sentence piece Signed-off-by: Yi Dong <yidong@nvidia.com> * fix a bug Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * potential fix in the future Signed-off-by: Yi Dong <yidong@nvidia.com> * remove the universal codes Signed-off-by: Yi Dong <yidong@nvidia.com> * remove universal strategy Signed-off-by: Yi Dong <yidong@nvidia.com> * address reviewer comment Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add align function docstrings and make most args optional Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove redundant returns of viterbi and log probs matrices Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Rename h# to <initial_silence> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update manifest format description in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * always remove any spaces from utt_id Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Patch the hanging of threads on very large stderr (#5589) (#5590) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * O2 style amp for gpt3 ptuning (#5246) * enable amp o2 plugin Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * only create master param if param requires gradient Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * remove pytorch autocast Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Update optimizer_with_main_params.py Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> * create master grad only if param group requires grad Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix grad scaler for pp > 1 Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Better patch hydra (#5591) (#5592) * Readd buffereing and thread drain to Hydra Launcher Signed-off-by: smajumdar <titu1994@gmail.com> * Readd buffereing and thread drain to Hydra Launcher Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Yet another fix with hydra multirun (#5594) (#5595) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add RETRO model documentation (#5578) * added retro doc Signed-off-by: Yi Dong <yidong@nvidia.com> * finish data part Signed-off-by: Yi Dong <yidong@nvidia.com> * added the data format Signed-off-by: Yi Dong <yidong@nvidia.com> * added training script Signed-off-by: Yi Dong <yidong@nvidia.com> * added training and evaluation steps Signed-off-by: Yi Dong <yidong@nvidia.com> * edit the text Signed-off-by: Yi Dong <yidong@nvidia.com> * added the images Signed-off-by: Yi Dong <yidong@nvidia.com> * fix beginning Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the grammar Signed-off-by: Yi Dong <yidong@nvidia.com> * trim it down Signed-off-by: Yi Dong <yidong@nvidia.com> * add wandb option Signed-off-by: Yi Dong <yidong@nvidia.com> * add reference Signed-off-by: Yi Dong <yidong@nvidia.com> * fix path Signed-off-by: Yi Dong <yidong@nvidia.com> * added the parameters table Signed-off-by: Yi Dong <yidong@nvidia.com> * fix section Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix: setup_multiple validation/test data (#5585) Fix: setup_multiple validation/test data (#5585) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Move to optimizer based EMA implementation (#5169) * Move to optimizer Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix replacing weights Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Allow swapping of weights be optional Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Save 2 models Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Use different hook Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Expose cpu device Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add clause to see if this fixes issue with O2 optimizer Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Try to get O2 working Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * WIP Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fixes Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fixes to tests Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add guard Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Remove import Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add guard Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add comment Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Remove overwrite Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add BatchNorm, currently tests fail Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix tests/functionality for batch norm Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Get rid of NLP changes Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * AIStore for ASR datasets (#5462) AIStore for ASR datasets Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add support for MHA adapters to ASR (#5396) * Convert AbstractAdapterModule to AbstractAdapterMixin Signed-off-by: smajumdar <titu1994@gmail.com> * Temporary fixes to new signature of mixin Signed-off-by: smajumdar <titu1994@gmail.com> * Add adapter util for constants, add all mha adapters. Signed-off-by: smajumdar <titu1994@gmail.com> * Update name of function Signed-off-by: smajumdar <titu1994@gmail.com> * Roll back changes to convASR Signed-off-by: smajumdar <titu1994@gmail.com> * Convert AbstractAdapterModule to AbstractAdapterMixin Signed-off-by: smajumdar <titu1994@gmail.com> * First draft of Conformer support for MHA attention Signed-off-by: smajumdar <titu1994@gmail.com> * Add some preliminary tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for projection of the hidden dimension for attention Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for squeezeformer Signed-off-by: smajumdar <titu1994@gmail.com> * Update train adapter config Signed-off-by: smajumdar <titu1994@gmail.com> * Add tests for squeezeformer and unit tests for new modules Signed-off-by: smajumdar <titu1994@gmail.com> * Update config for hp search,set limits on modules for conformer and squeezeformer, update adapter mixin, add cache to import_from_class_path Signed-off-by: smajumdar <titu1994@gmail.com> * Update location of adapters Signed-off-by: smajumdar <titu1994@gmail.com> * Add pre_norm for proper attention learning, Fix the issue with nan/inf in pos_bias_u and pos_bias_v Signed-off-by: smajumdar <titu1994@gmail.com> * Update expmanager to clean up checkpoints Signed-off-by: smajumdar <titu1994@gmail.com> * Fix style Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update training scripts Signed-off-by: smajumdar <titu1994@gmail.com> * Update config and docs Signed-off-by: smajumdar <titu1994@gmail.com> * Expose nemo delete function Signed-off-by: smajumdar <titu1994@gmail.com> * Correct adapter partial state saving Signed-off-by: smajumdar <titu1994@gmail.com> * Correct a bug with state management of adapter tokens Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pull down EMA test Signed-off-by: smajumdar <titu1994@gmail.com> * Correct name of adapter module utility class Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove unused TTS eval functions w/ pesq and pystoi dependencies (#5605) (#5606) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Create separator parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Call align function with hydra config Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update usage example Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update Dockerfile (#5614) (#5616) Pinned to use `numba==0.53.1` to avoid crashing in training with `num_workers > 0`. This is just a temporary workaround, still need to fix it in the future. Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make separate pretrained_name and model_path parameters Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * make "optional" tags bold in markdown Signed-off-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Move non-main functions to utils dir Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Temp workaround: Disable test with cache_audio=True since it is failing in CI (#5607) (#5615) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] fix ranges of char set for accented letters. (#5607) * [TTS] fix ranges of char set for accented letters. * remove digits pattern and added unit tests for math operators. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Change success message to reduce confusion (#5621) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update documentation and tutorials for Adapters (#5610) * Improve docs for adapter and tests Signed-off-by: smajumdar <titu1994@gmail.com> * Improve docs for adapter and tests Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update test Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename test file Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] add type hints and change varialbe names for tokenizers and g2p (#5602) * [TTS] add type hints and change variable names for tokenizers and g2p Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * 1. Added missing import for gather_objects. (#5627) Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] add fastpitch and hifigan model NGC urls and update NeMo docs. (#5596) (#5625) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fixed RadTTS unit test (#5572) Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove tests (#5633) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][DOC] add notes about automatic conversion to target sampling rates. (#5624) (#5634) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Conformer local attention (#5525) * local attn and merge Signed-off-by: sam1373 <samuelkriman@gmail.com> * optional Signed-off-by: sam1373 <samuelkriman@gmail.com> * override Signed-off-by: sam1373 <samuelkriman@gmail.com> * incorporate comments Signed-off-by: sam1373 <samuelkriman@gmail.com> * update Signed-off-by: sam1373 <samuelkriman@gmail.com> * fix Signed-off-by: sam1373 <samuelkriman@gmail.com> * comment Signed-off-by: sam1373 <samuelkriman@gmail.com> * changes, test Signed-off-by: sam1373 <samuelkriman@gmail.com> * changes Signed-off-by: sam1373 <samuelkriman@gmail.com> * check att context Signed-off-by: sam1373 <samuelkriman@gmail.com> * readme link Signed-off-by: sam1373 <samuelkriman@gmail.com> * utils Signed-off-by: sam1373 <samuelkriman@gmail.com> * update Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add core classes and functions for online clustering diarizer part 1 (#5526) * Add core classes and functions for online clustering diarizer Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add audio to labels code Signed-off-by: Taejin Park <tango4j@gmail.com> * resolve type errors Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added unit=tests for very short audio Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Filled all missing docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * resolved conflict and added missing docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed unit-test errors Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix the wrongly added file - megatron_gpt_model.py Signed-off-by: Taejin Park <tango4j@gmail.com> * Fix wrongly included file - megatron_gpt_model.py Signed-off-by: Taejin Park <tango4j@gmail.com> * resolve code quality issue Signed-off-by: Taejin Park <tango4j@gmail.com> * Fixed unit-test errors and bugs Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * changed total_sec for offline_clustering toy_data in unit-tests Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed merging index offset bug Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * only including part 1 files Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed unused function Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed unused imports Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * divided nmesc_clustering.py into two and reflected first-pass comments Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adding offline/online_clustering.py Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix code QL autocomment Signed-off-by: Taejin Park <tango4j@gmail.com> * Removed unused imports Signed-off-by: Taejin Park <tango4j@gmail.com> * Update nemo/collections/asr/parts/utils/online_clustering.py Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> * Reflected comments Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * resolved code scanning issue Signed-off-by: Taejin Park <tango4j@gmail.com> * Update nemo/collections/asr/parts/utils/offline_clustering.py Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [STT] Add Esperanto (Eo) ASR Conformer-CTC and Conformer-Transducer models (#5639) (#5641) * add stt_eo_conformer_ctc_large model * stt_eo_conformer_transducer_large Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Removed unused import Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Specify that filepaths need to be absolute Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * replaces any spaces in utt_id with dashes Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make hydra script callable by another script Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * do not specify default model or model_downsample_factor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [Dockerfile] Remove AIS archive from docker image (#5629) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Measure audio_sr from audio instead of needing to specify Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] Disambiguate polyphones with augmented dict and Jieba segmenter for Chinese FastPitch (#5541) * Chinese TTS replaces default pypinyin dict * Add jieba word segmenter as an option Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make separate parameters for device of transcription and viterbi steps Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add mention of gecko Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [workflow] add exclude labels option to ignore cherry-picks in release changelog. (#5645) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] bugfix for the tutorial and add NGC CLI installation guide. (#5643) (#5647) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [Add] ASR+VAD Inference Pipeline (#5575) Added offline ASR+VAD inference pipeline that matches with what's in RIVA, along with some feature-based ASR and classification datasets. Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * rename separator to ctm_grouping_separator and refactor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Bert interleaved (#5556) * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * Update bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Adding tests * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * Fix bug fix sequence parallel and Interleaved * Fix bug fix sequence parallel and Interleaved Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add duration padding support for RADTTS inference (#5650) * Added duration padding support for RADTTS inference * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: Kevin Shih <kshih@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add remove_blank_tokens_from_ctm parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Dont save initial_silence line in CTM Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add DLLogger support to exp_manager (#5658) * Add DLLogger support to exp_manager Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> * Move dllogger to separate file and check import Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add minimum_timestamp_duration parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add suggestion about removing blanks to README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * reorder args Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * clarify description of ctm_grouping_separator in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update docstring Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] bugfix for ngc cli installation. (#5652) (#5664) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Port stateless timer to exp manager (#5584) * Port stateless timer to exp manager Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes and remove from all megatron code Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change message Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix EMA restart by allowing device to be set by the class init (#5668) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove SDP (moved to separate repo) - merge to main (#5630) * Remove sdp files from tools folder Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add page to docs with new SDP location Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add interface for making amax reduction optional for FP8 (#5447) * add TE interface for making amax reduction optional Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] add tts dict cust notebook (#5662) * add tts dict cust notebook Signed-off-by: ekmb <ebakhturina@nvidia.com> * review Signed-off-by: ekmb <ebakhturina@nvidia.com> * fixed audio links Signed-off-by: ekmb <ebakhturina@nvidia.com> * remove old notebook Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix typo Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [ASR] Audio processing base, multi-channel enhancement models (#5356) * Audio processing base model, enc-mask-dec enhancement, tests and modules Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed review comments Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fixed CodeQL warnings Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed PR comments Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed PR comments: - renamed AudioProcessingModel to AudioToAudioModel - various small modifications - updated unit tests Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed comments - Moved spectrogram to audio_preprocessing - Renamed MultichannelFeatures - Updated config and unit tests Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Expose ClusteringDiarizer device (#5681) * Expose device for users to set Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Expose device for users to set Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add Beam Search support to ASR transcribe() (#5443) * Add support for beam decoding via high level API. Signed-off-by: smajumdar <titu1994@gmail.com> * Add ctc decoding section Signed-off-by: smajumdar <titu1994@gmail.com> * Update ctc transcribe API to return results from beam search Signed-off-by: smajumdar <titu1994@gmail.com> * Add argument to preserve arpa file Signed-off-by: smajumdar <titu1994@gmail.com> * Update script to use hydra config, add some support for future compute timesteps, add doc for ctc decoding Signed-off-by: smajumdar <titu1994@gmail.com> * Update eval script and doc to use new API Signed-off-by: smajumdar <titu1994@gmail.com> * Add tests for ctc greedy decoding Signed-off-by: smajumdar <titu1994@gmail.com> * Address reviewer comments and add docstrings Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix changes and address comments Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Propagate attention_dropout flag for GPT-3 (#5669) * Propagate attention_dropout flag for GPT-3 Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> * Add default to megatron_gpt_config Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail…

* Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * Update bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Adding tests * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * Fix bug fix sequence parallel and Interleaved * Fix bug fix sequence parallel and Interleaved Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com>

* Merge r1.13.0 main (#5570) * update branch Signed-off-by: ericharper <complex451@gmail.com> * Rename Speech Dataset Processor to Speech Data Processor (#5378) Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Megatron Export Update (#5343) * export update for Megatron + change ORT optimization Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * updated export_utils to use autocast instead of manually casting >:/ Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * removed dtype from LayerNorm Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * added comment Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * reverting changes on FloatCast Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * Cherry-picked changes from megatron-norm Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * updated asr_model import to cast_utils Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * updated del onnx_model place Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * changed ort optimization to basic -> temp fix Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Boris Fomitchev <bfomitchev@nvidia.com> * Disable sync_batch_comm in validation_step for GPT (#5397) * disable sync_batch_comm in validation_step Signed-off-by: ericharper <complex451@gmail.com> * Read sync_batch_comm from config or default to False Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> * Update megatron_gpt_config to default sync_batch_comm to False to avoid CUDA error Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> * Empty Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Comment out test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Markel Sanz Ausin <markelsanz14@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Radtts 1.13 (#5451) * [TTS] Fixing RADTTS training - removing view buffer and fixing accuracy issue (#5358) * [TTS] add CI test for RADTTS training recipe. Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Support for finetuning and finetuning inference with .ckpt files & batch size refactoring (#5339) (#5478) * Initial refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Resolve config before passing to load_from_checkpoint Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for model parallel and nemo restore Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert config changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Minor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix validation reconfiguration Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove old comment Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes for test_ds Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * export_utils bugfix (#5480) * updated export_utils Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Export fixes for Riva (#5496) * Export fixes for Riva Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Cleaning up training_utils Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * added set_start_method + function param bugfix (#5539) * added set_start_method + function param bugfix Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * upper bound torchmetrics Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: ericharper <complex451@gmail.com> * remove notebook (#5548) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update branch Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> * revert Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Markel Sanz Ausin <markelsanz14@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Boris Fomitchev <bfomitchev@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Markel Sanz Ausin <markelsanz14@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Optimized loop and bugfix in SDE (#5573) - Fixed bug with loading custom data attributes from JSON in Speech Data Explorer Signed-off-by: George Zelenfroynd <gzelenfroind@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update torchmetrics (#5566) * add task arg Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> * update state Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove useless files. (#5580) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add initial NFA code Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make use of the specified device during viterbi decoding Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix CodeQL notes Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix CodeQL warning Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add an option to defer data setup from ``__init__`` to ``setup`` (#5569) * Add an option to defer dataloader setup from __init__ to setup Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Updated doc Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make utt_id specified by number of parts of audio_filepath user wishes to use Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove audio_sr TODO - reduce risk of silent bugs Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add check that model is CTC Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove unused import Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Text generation improvement (UI client, data parallel support) (#5437) * Squashed commit of the following: commit a5e124f34be31bd6eafe5e5fdf5bedcd0d50915c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 13 15:07:42 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 35b424044fe80c3081e7756ab21244f701716f7e Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 08:04:49 2022 -0700 get rid of base Signed-off-by: Yi Dong <yidong@nvidia.com> commit 2955210e2311791543538cfbb5ad26b79414c954 Merge: d52edef8c eaf6757ca Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:17:02 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit d52edef8cd7b36593838fb270047e80f8ccb652e Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:16:24 2022 +0000 align with main Signed-off-by: Yi Dong <yidong@nvidia.com> commit eaf6757ca5be8e099492f57c81d984429b0ad49c Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Oct 13 13:12:11 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit c4b86d97626ea0721bf8fb4c0a45dec5becc94c9 Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:10:58 2022 +0000 same as main Signed-off-by: Yi Dong <yidong@nvidia.com> commit e335de51bcc0d681c58b568c3d8c238bc5687c3b Merge: c231086e0 4463a9fe9 Author: Yi Dong <yidong@nvidia.com> Date: Thu Oct 13 13:08:09 2022 +0000 Merge branch 'main' into universal_prompt commit c231086e057f1efaa915f691d84664cb3d5aad85 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Oct 12 19:59:12 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 6a821a4b49a23dd3408a706a2a3dd393149b0bb1 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:56:17 2022 +0000 default to pad Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9d908e39fef1beed9ba2da4d1a6806161eb7ef25 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:55:44 2022 +0000 add the option to pad the tokens Signed-off-by: Yi Dong <yidong@nvidia.com> commit 876dc395b43fdeeaa2bcbbe13c76523633764c33 Merge: fbb0f4035 fe3c77ee9 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 19:20:47 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit fe3c77ee93ab6cf3ea152db68cb6beefcac2a392 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 18:59:49 2022 +0000 fix import again Signed-off-by: Yi Dong <yidong@nvidia.com> commit fbb0f4035c6cd6bfefed50a20605503de8c1dccb Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Wed Oct 12 16:00:24 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 372ca8c0d7988f2339b15888dc72aa21f4fb6937 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 15:58:32 2022 +0000 enable server Signed-off-by: Yi Dong <yidong@nvidia.com> commit cbe05d9fbc978f812cfbb671f45f147f300713c4 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 13:07:28 2022 +0000 fix comment error Signed-off-by: Yi Dong <yidong@nvidia.com> commit 1948048922e726ec6131e44b1a745389f18d4ef2 Merge: 232c2cce3 984f5c09a Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 13:05:30 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit 232c2cce34d7a8b902da406706f3dd9b39475091 Merge: 34c8a68df 658243fb6 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 12 12:50:00 2022 +0000 Merge branch 'fix_global_init' into universal_prompt commit 984f5c09a6dbf1d1fb5aa30ed9b0df188e66a50f Merge: 658243fb6 3fda5de46 Author: Yi Dong <43824965+yidong72@users.noreply.github.com> Date: Wed Oct 12 08:42:11 2022 -0400 Merge branch 'main' into fix_global_init commit 658243fb6580191b5d60edd30cde16dcc23cbb85 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 12 12:40:57 2022 +0000 fix import error Signed-off-by: Yi Dong <doyend@gmail.com> commit 8e0fe1cad05ec288ec122b3cd0e139a96872e08c Author: Yi Dong <doyend@gmail.com> Date: Tue Oct 11 22:44:12 2022 +0000 update the fused kernel Signed-off-by: Yi Dong <doyend@gmail.com> commit 536cf6bef9447b75843fad630729c47a2fba35f3 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:44:52 2022 -0700 add the missing file Signed-off-by: Yi Dong <yidong@nvidia.com> commit 1b437ec41dc5e354453ce0a089bca0171cbcb6c2 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:43:14 2022 -0700 fix fused softmax Signed-off-by: Yi Dong <yidong@nvidia.com> commit 7813f60e05f9783af61f8c14ec1cb0c6c4f1f263 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 11 14:16:48 2022 -0700 move global step to base Signed-off-by: Yi Dong <yidong@nvidia.com> commit 34c8a68df084b18d377e84415d9f07b2cd6673dd Author: Yi Dong <doyend@gmail.com> Date: Thu Oct 6 13:50:11 2022 +0000 fix pipeline for eval Signed-off-by: Yi Dong <doyend@gmail.com> commit eee5d38218f26660c3ffebe9f615c850c80a1f0d Author: Yi Dong <doyend@gmail.com> Date: Thu Oct 6 13:48:22 2022 +0000 fix for pipleline parallel Signed-off-by: Yi Dong <doyend@gmail.com> commit 323bca73e7ef6099ee79c0a2fffac7b709ed6c5d Merge: 125e49947 e3b4c4d1f Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 19:29:13 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit 125e4994760448ff75dd9328395813eda1c87547 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 19:29:04 2022 +0000 add share option Signed-off-by: Yi Dong <doyend@gmail.com> commit e3b4c4d1f7346c9fa596f3cca6d4df0a9e05c368 Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 5 11:43:48 2022 -0700 make sure consolidation works Signed-off-by: Yi Dong <yidong@nvidia.com> commit a5c833964ecf05dc460ca1da69275c4019742150 Merge: 2a07ab52d abcb74be2 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:40:29 2022 +0000 Merge branch 'universal_prompt' of github.com:NVIDIA/NeMo into universal_prompt commit 2a07ab52d95f15ba666823028c69e23825666c05 Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:40:23 2022 +0000 added requirement Signed-off-by: Yi Dong <doyend@gmail.com> commit 3abecd9dd1611993a87c537636abe7f7e6a9b04c Author: Yi Dong <doyend@gmail.com> Date: Wed Oct 5 18:39:42 2022 +0000 added a simple web server Signed-off-by: Yi Dong <doyend@gmail.com> commit abcb74be2caf1cdec40eb9ba2be4dde4d45a3b4b Author: Yi Dong <yidong@nvidia.com> Date: Wed Oct 5 06:54:12 2022 -0700 fix empty val loss Signed-off-by: Yi Dong <yidong@nvidia.com> commit b8eb92ac4a0d665570af75e34c9ba3c2e2420c26 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 19:25:30 2022 -0700 text gen working Signed-off-by: Yi Dong <yidong@nvidia.com> commit d59f3e3f3a6fd19736d1c5706fed65a3dd4049ba Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 16:08:40 2022 -0700 first change Signed-off-by: Yi Dong <yidong@nvidia.com> commit 59d077585e6962a669b824af58f64e8a0bea6547 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 15:00:40 2022 -0700 revert Signed-off-by: Yi Dong <yidong@nvidia.com> commit 12a0f3902d99e9179403644bd951c045df716ca7 Author: Yi Dong <doyend@gmail.com> Date: Tue Oct 4 21:26:23 2022 +0000 init imp Signed-off-by: Yi Dong <doyend@gmail.com> commit 62a15dfd943cc48be495ac61b9f2f00995775c5f Merge: 82c90d2cd e0cc6b767 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 11:58:26 2022 -0700 Merge branch 'main' into universal_prompt commit 82c90d2cd0fd156f16a4b899f8c741d598f33990 Author: Yi Dong <yidong@nvidia.com> Date: Tue Oct 4 11:17:13 2022 -0700 add sync Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9819b703eef877d90cd1257bf3610c69de9b4d7e Author: Yi Dong <yidong@nvidia.com> Date: Sun Oct 2 17:52:34 2022 -0700 fix save model Signed-off-by: root <root@luna-0197.selene.nvidia.com> commit e4937e2fc5fb7d70754c97668416e4a69c3079fe Author: Yi Dong <yidong@nvidia.com> Date: Sat Oct 1 18:56:09 2022 +0000 working Signed-off-by: Yi Dong <yidong@nvidia.com> commit b73b06d1c7cf5417a6d87cb33d8ed83a57e38b7b Author: Yi Dong <yidong@nvidia.com> Date: Sat Oct 1 17:34:03 2022 +0000 calcuate the mask Signed-off-by: Yi Dong <yidong@nvidia.com> commit 9db3bc13eb65a94a475b837603351da68e3745bc Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 23:26:32 2022 +0000 fix bug in datasets Signed-off-by: Yi Dong <yidong@nvidia.com> commit f289900375d4412f53f8110be00fec6587627550 Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 22:29:40 2022 +0000 update the code Signed-off-by: Yi Dong <yidong@nvidia.com> commit 8e28a1f208aabaab72dbe769e72756baada04d99 Author: Yi Dong <yidong@nvidia.com> Date: Fri Sep 30 21:52:52 2022 +0000 added new ds Signed-off-by: Yi Dong <yidong@nvidia.com> commit 8d41315bab7ce90e200a8a7d1023c34f8e046897 Author: Yi Dong <doyend@gmail.com> Date: Fri Sep 30 18:57:09 2022 +0000 added new files Signed-off-by: Yi Dong <doyend@gmail.com> commit 984e0e94e15e16323c1ba1ca2efeabd84f69463f Merge: cbe8b7ab1 fa6cd8588 Author: Yi Dong <doyend@gmail.com> Date: Thu Sep 29 21:43:29 2022 +0000 Merge branch 'llm-prompt-learning-improvements' into universal_prompt commit fa6cd858839277939446afe7275976078d54c512 Author: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Date: Thu Sep 29 16:47:30 2022 +0000 [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci commit 78ba46e5d6fde1be53c08e1e30a54cce59824be0 Merge: 7d6d46742 8d670bc77 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Thu Sep 29 09:43:27 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 7d6d46742170a66758287a207d67e1b1bfd15613 Author: Virginia Adams <vadams@nvidia.com> Date: Thu Sep 29 16:42:43 2022 +0000 Removed inference step and added sentence peice check to predict step Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 20fd265acd6f7f9912cf52155fe66ccfa6b201a2 Author: Virginia Adams <vadams@nvidia.com> Date: Thu Sep 29 15:26:32 2022 +0000 fixed first stage check for pipeline parallel T5 pt Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 3637be2b258c8d9028856f9971edb7da4a8121f0 Merge: a3ea722fd 986a76612 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Wed Sep 28 10:23:30 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit a3ea722fdc12fbcc5989b76ef5643a574b763bc4 Merge: 770967a52 971485ce7 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Mon Sep 26 13:35:52 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 770967a5251a474b6dcc2d44bf9a2076adbcb604 Merge: d23bf6c30 e3ac280a8 Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Mon Sep 26 10:17:03 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit d23bf6c30acc0e3f6af9b4e24547669866a34d62 Merge: de6a31651 333d2b749 Author: Virginia Adams <vadams@nvidia.com> Date: Mon Sep 26 10:05:16 2022 -0700 Merge branch 'llm-prompt-learning-improvements' of https://github.com/NVIDIA/NeMo into llm-prompt-learning-improvements commit de6a31651e63d88a42b971794d93f18ff5a3cdff Author: Virginia Adams <vadams@nvidia.com> Date: Mon Sep 26 17:00:53 2022 +0000 Updated PP check to be on first stage pipeline only Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 333d2b7498e6742ce66436f733c980a74616900c Merge: 592c0986a a39fc925a Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Fri Sep 23 16:11:21 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 592c0986a476a91b57b8605d7b70830d7acfa021 Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 23:08:41 2022 +0000 Fixed unused import and CI test bug Signed-off-by: Virginia Adams <vadams@nvidia.com> commit ea9cd82d85638bc60ae4ad7ef105db931c8e3455 Merge: ce4b72c8c b566c2d0e Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 18:57:25 2022 +0000 Merge branch 'llm-prompt-learning-improvements' of https://github.com/NVIDIA/NeMo into llm-prompt-learning-improvements commit ce4b72c8c52f32be336e323dd78a38089edc3e7c Author: Virginia Adams <vadams@nvidia.com> Date: Fri Sep 23 18:57:16 2022 +0000 Switch to import from base class Signed-off-by: Virginia Adams <vadams@nvidia.com> commit b566c2d0e35a068f758fd1310bc620a47be4590b Merge: 6621f2854 e872061ac Author: Virginia Adams <78445382+vadam5@users.noreply.github.com> Date: Fri Sep 23 10:09:03 2022 -0700 Merge branch 'main' into llm-prompt-learning-improvements commit 6621f28543828a48484a5637f6c9f3ccb23a5b02 Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 20:47:35 2022 +0000 python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 8deafc8987b6af5f7b99a250310f57a40198c37f Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 20:28:02 2022 +0000 Save .nemo on new best val score Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 761bd36969cb465d6a129e9eee6ce1f883d3cf41 Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 18:03:19 2022 +0000 Added automatic checkpoint to nemo file method Signed-off-by: Virginia Adams <vadams@nvidia.com> commit 3be4ed57b6cd3ddfe4876d78650dfe8fe794598b Author: Virginia Adams <vadams@nvidia.com> Date: Wed Sep 14 02:11:56 2022 +0000 Make GPT use base prompt learning model class: Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> * fix LGTM Signed-off-by: Yi Dong <yidong@nvidia.com> * fix validation Signed-off-by: Yi Dong <yidong@nvidia.com> * change for the lm eval Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * make text generation work in data parallel environment Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * implement the service with rest service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * surpress log Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore function needed for NMT Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * handles no answer only Signed-off-by: Yi Dong <yidong@nvidia.com> * Fix config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added knn to web Signed-off-by: Yi Dong <yidong@nvidia.com> * fix lgtm.com comments Signed-off-by: Yi Dong <yidong@nvidia.com> * output the retrieved context Signed-off-by: Yi Dong <yidong@nvidia.com> * allow no neighbor query Signed-off-by: Yi Dong <yidong@nvidia.com> * remove the imports Signed-off-by: Yi Dong <yidong@nvidia.com> * warn only once Signed-off-by: Yi Dong <yidong@nvidia.com> * Change output file format from JSON to JSONL Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * new t0 dataset Signed-off-by: Yi Dong <yidong@nvidia.com> * Add T0 data preproc scripts Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Merge and multiprocessing Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix for is_correct Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * fix epoch > 2 Signed-off-by: Yi Dong <yidong@nvidia.com> * handles multiple dataloader Signed-off-by: Yi Dong <yidong@nvidia.com> * remove template Signed-off-by: Yi Dong <yidong@nvidia.com> * Refactor T0 dataset Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add script to merge train folder into individual training files to minimize number of blends Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added on the fly service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add combo instance Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added combo service Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * send weights back to server Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix index store Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Minor changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add reset button Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add add eos Signed-off-by: Yi Dong <yidong@nvidia.com> * use a seperate bert service Signed-off-by: Yi Dong <yidong@nvidia.com> * no loss of accuracy Signed-off-by: Yi Dong <yidong@nvidia.com> * pin the gradio version Signed-off-by: Yi Dong <yidong@nvidia.com> * Remove bin compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix header lines Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * evaluate based on text generation Signed-off-by: Yi Dong <yidong@nvidia.com> * exact match result aggregation Signed-off-by: Yi Dong <yidong@nvidia.com> * working SP and SA Signed-off-by: Yi Dong <yidong@nvidia.com> * sync Signed-off-by: Yi Dong <yidong@nvidia.com> * fix checkpoint Signed-off-by: Yi Dong <yidong@nvidia.com> * fix eval Signed-off-by: Yi Dong <yidong@nvidia.com> * backup states Signed-off-by: Yi Dong <yidong@nvidia.com> * backup states reset Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the bug Signed-off-by: Yi Dong <yidong@nvidia.com> * fix evaluation for sentence piece Signed-off-by: Yi Dong <yidong@nvidia.com> * fix a bug Signed-off-by: Yi Dong <yidong@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * potential fix in the future Signed-off-by: Yi Dong <yidong@nvidia.com> * remove the universal codes Signed-off-by: Yi Dong <yidong@nvidia.com> * remove universal strategy Signed-off-by: Yi Dong <yidong@nvidia.com> * address reviewer comment Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add align function docstrings and make most args optional Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove redundant returns of viterbi and log probs matrices Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Rename h# to <initial_silence> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update manifest format description in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * always remove any spaces from utt_id Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Patch the hanging of threads on very large stderr (#5589) (#5590) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * O2 style amp for gpt3 ptuning (#5246) * enable amp o2 plugin Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * only create master param if param requires gradient Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * remove pytorch autocast Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * Update optimizer_with_main_params.py Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> * create master grad only if param group requires grad Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> * fix grad scaler for pp > 1 Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: Jimmy Zhang <jiemingz@nvidia.com> Signed-off-by: JimmyZhang12 <67203904+JimmyZhang12@users.noreply.github.com> Co-authored-by: Jimmy Zhang <jiemingz@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Better patch hydra (#5591) (#5592) * Readd buffereing and thread drain to Hydra Launcher Signed-off-by: smajumdar <titu1994@gmail.com> * Readd buffereing and thread drain to Hydra Launcher Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Yet another fix with hydra multirun (#5594) (#5595) Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add RETRO model documentation (#5578) * added retro doc Signed-off-by: Yi Dong <yidong@nvidia.com> * finish data part Signed-off-by: Yi Dong <yidong@nvidia.com> * added the data format Signed-off-by: Yi Dong <yidong@nvidia.com> * added training script Signed-off-by: Yi Dong <yidong@nvidia.com> * added training and evaluation steps Signed-off-by: Yi Dong <yidong@nvidia.com> * edit the text Signed-off-by: Yi Dong <yidong@nvidia.com> * added the images Signed-off-by: Yi Dong <yidong@nvidia.com> * fix beginning Signed-off-by: Yi Dong <yidong@nvidia.com> * fix the grammar Signed-off-by: Yi Dong <yidong@nvidia.com> * trim it down Signed-off-by: Yi Dong <yidong@nvidia.com> * add wandb option Signed-off-by: Yi Dong <yidong@nvidia.com> * add reference Signed-off-by: Yi Dong <yidong@nvidia.com> * fix path Signed-off-by: Yi Dong <yidong@nvidia.com> * added the parameters table Signed-off-by: Yi Dong <yidong@nvidia.com> * fix section Signed-off-by: Yi Dong <yidong@nvidia.com> Signed-off-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix: setup_multiple validation/test data (#5585) Fix: setup_multiple validation/test data (#5585) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Move to optimizer based EMA implementation (#5169) * Move to optimizer Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix replacing weights Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Allow swapping of weights be optional Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Save 2 models Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Use different hook Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Expose cpu device Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add clause to see if this fixes issue with O2 optimizer Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Try to get O2 working Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * WIP Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fixes Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fixes to tests Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add guard Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Remove import Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add guard Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add comment Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Remove overwrite Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Add BatchNorm, currently tests fail Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Fix tests/functionality for batch norm Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Get rid of NLP changes Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * AIStore for ASR datasets (#5462) AIStore for ASR datasets Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add support for MHA adapters to ASR (#5396) * Convert AbstractAdapterModule to AbstractAdapterMixin Signed-off-by: smajumdar <titu1994@gmail.com> * Temporary fixes to new signature of mixin Signed-off-by: smajumdar <titu1994@gmail.com> * Add adapter util for constants, add all mha adapters. Signed-off-by: smajumdar <titu1994@gmail.com> * Update name of function Signed-off-by: smajumdar <titu1994@gmail.com> * Roll back changes to convASR Signed-off-by: smajumdar <titu1994@gmail.com> * Convert AbstractAdapterModule to AbstractAdapterMixin Signed-off-by: smajumdar <titu1994@gmail.com> * First draft of Conformer support for MHA attention Signed-off-by: smajumdar <titu1994@gmail.com> * Add some preliminary tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for projection of the hidden dimension for attention Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for squeezeformer Signed-off-by: smajumdar <titu1994@gmail.com> * Update train adapter config Signed-off-by: smajumdar <titu1994@gmail.com> * Add tests for squeezeformer and unit tests for new modules Signed-off-by: smajumdar <titu1994@gmail.com> * Update config for hp search,set limits on modules for conformer and squeezeformer, update adapter mixin, add cache to import_from_class_path Signed-off-by: smajumdar <titu1994@gmail.com> * Update location of adapters Signed-off-by: smajumdar <titu1994@gmail.com> * Add pre_norm for proper attention learning, Fix the issue with nan/inf in pos_bias_u and pos_bias_v Signed-off-by: smajumdar <titu1994@gmail.com> * Update expmanager to clean up checkpoints Signed-off-by: smajumdar <titu1994@gmail.com> * Fix style Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * Add docstrings and update tests Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update training scripts Signed-off-by: smajumdar <titu1994@gmail.com> * Update config and docs Signed-off-by: smajumdar <titu1994@gmail.com> * Expose nemo delete function Signed-off-by: smajumdar <titu1994@gmail.com> * Correct adapter partial state saving Signed-off-by: smajumdar <titu1994@gmail.com> * Correct a bug with state management of adapter tokens Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Pull down EMA test Signed-off-by: smajumdar <titu1994@gmail.com> * Correct name of adapter module utility class Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove unused TTS eval functions w/ pesq and pystoi dependencies (#5605) (#5606) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Create separator parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Call align function with hydra config Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update usage example Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update Dockerfile (#5614) (#5616) Pinned to use `numba==0.53.1` to avoid crashing in training with `num_workers > 0`. This is just a temporary workaround, still need to fix it in the future. Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make separate pretrained_name and model_path parameters Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * make "optional" tags bold in markdown Signed-off-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Move non-main functions to utils dir Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Temp workaround: Disable test with cache_audio=True since it is failing in CI (#5607) (#5615) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] fix ranges of char set for accented letters. (#5607) * [TTS] fix ranges of char set for accented letters. * remove digits pattern and added unit tests for math operators. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Change success message to reduce confusion (#5621) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Update documentation and tutorials for Adapters (#5610) * Improve docs for adapter and tests Signed-off-by: smajumdar <titu1994@gmail.com> * Improve docs for adapter and tests Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update test Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename test file Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] add type hints and change varialbe names for tokenizers and g2p (#5602) * [TTS] add type hints and change variable names for tokenizers and g2p Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * 1. Added missing import for gather_objects. (#5627) Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] add fastpitch and hifigan model NGC urls and update NeMo docs. (#5596) (#5625) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fixed RadTTS unit test (#5572) Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * remove tests (#5633) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][DOC] add notes about automatic conversion to target sampling rates. (#5624) (#5634) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Conformer local attention (#5525) * local attn and merge Signed-off-by: sam1373 <samuelkriman@gmail.com> * optional Signed-off-by: sam1373 <samuelkriman@gmail.com> * override Signed-off-by: sam1373 <samuelkriman@gmail.com> * incorporate comments Signed-off-by: sam1373 <samuelkriman@gmail.com> * update Signed-off-by: sam1373 <samuelkriman@gmail.com> * fix Signed-off-by: sam1373 <samuelkriman@gmail.com> * comment Signed-off-by: sam1373 <samuelkriman@gmail.com> * changes, test Signed-off-by: sam1373 <samuelkriman@gmail.com> * changes Signed-off-by: sam1373 <samuelkriman@gmail.com> * check att context Signed-off-by: sam1373 <samuelkriman@gmail.com> * readme link Signed-off-by: sam1373 <samuelkriman@gmail.com> * utils Signed-off-by: sam1373 <samuelkriman@gmail.com> * update Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add core classes and functions for online clustering diarizer part 1 (#5526) * Add core classes and functions for online clustering diarizer Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add audio to labels code Signed-off-by: Taejin Park <tango4j@gmail.com> * resolve type errors Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * added unit=tests for very short audio Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Filled all missing docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * resolved conflict and added missing docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed unit-test errors Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix the wrongly added file - megatron_gpt_model.py Signed-off-by: Taejin Park <tango4j@gmail.com> * Fix wrongly included file - megatron_gpt_model.py Signed-off-by: Taejin Park <tango4j@gmail.com> * resolve code quality issue Signed-off-by: Taejin Park <tango4j@gmail.com> * Fixed unit-test errors and bugs Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * changed total_sec for offline_clustering toy_data in unit-tests Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed merging index offset bug Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * only including part 1 files Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * removed unused function Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed unused imports Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * divided nmesc_clustering.py into two and reflected first-pass comments Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * adding offline/online_clustering.py Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix code QL autocomment Signed-off-by: Taejin Park <tango4j@gmail.com> * Removed unused imports Signed-off-by: Taejin Park <tango4j@gmail.com> * Update nemo/collections/asr/parts/utils/online_clustering.py Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> * Reflected comments Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * resolved code scanning issue Signed-off-by: Taejin Park <tango4j@gmail.com> * Update nemo/collections/asr/parts/utils/offline_clustering.py Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [STT] Add Esperanto (Eo) ASR Conformer-CTC and Conformer-Transducer models (#5639) (#5641) * add stt_eo_conformer_ctc_large model * stt_eo_conformer_transducer_large Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Removed unused import Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Specify that filepaths need to be absolute Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * replaces any spaces in utt_id with dashes Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make hydra script callable by another script Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * do not specify default model or model_downsample_factor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [Dockerfile] Remove AIS archive from docker image (#5629) Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Measure audio_sr from audio instead of needing to specify Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] Disambiguate polyphones with augmented dict and Jieba segmenter for Chinese FastPitch (#5541) * Chinese TTS replaces default pypinyin dict * Add jieba word segmenter as an option Signed-off-by: Yuekai Zhang <zhangyuekai@foxmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Make separate parameters for device of transcription and viterbi steps Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add mention of gecko Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [workflow] add exclude labels option to ignore cherry-picks in release changelog. (#5645) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] bugfix for the tutorial and add NGC CLI installation guide. (#5643) (#5647) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [Add] ASR+VAD Inference Pipeline (#5575) Added offline ASR+VAD inference pipeline that matches with what's in RIVA, along with some feature-based ASR and classification datasets. Signed-off-by: stevehuang52 <heh@nvidia.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * rename separator to ctm_grouping_separator and refactor Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Bert interleaved (#5556) * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding SP and SAR support Bert * Adding SP and SAR support Bert * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adding Sequence parallel support to Bert * Update bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Adding tests * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * Adding interleaved pipeline parallelism * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Addressing Eric's comments * Fix bug fix sequence parallel and Interleaved * Fix bug fix sequence parallel and Interleaved Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add duration padding support for RADTTS inference (#5650) * Added duration padding support for RADTTS inference * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: Kevin Shih <kshih@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add remove_blank_tokens_from_ctm parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Dont save initial_silence line in CTM Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add DLLogger support to exp_manager (#5658) * Add DLLogger support to exp_manager Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> * Move dllogger to separate file and check import Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove unused import Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> Signed-off-by: Alexandre Milesi <milesial@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add minimum_timestamp_duration parameter Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add suggestion about removing blanks to README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * reorder args Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * clarify description of ctm_grouping_separator in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * update docstring Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS][ZH] bugfix for ngc cli installation. (#5652) (#5664) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Port stateless timer to exp manager (#5584) * Port stateless timer to exp manager Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes and remove from all megatron code Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change message Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix EMA restart by allowing device to be set by the class init (#5668) Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Remove SDP (moved to separate repo) - merge to main (#5630) * Remove sdp files from tools folder Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add page to docs with new SDP location Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add interface for making amax reduction optional for FP8 (#5447) * add TE interface for making amax reduction optional Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: Kirthi Shankar Sivamani <ksivamani@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [TTS] add tts dict cust notebook (#5662) * add tts dict cust notebook Signed-off-by: ekmb <ebakhturina@nvidia.com> * review Signed-off-by: ekmb <ebakhturina@nvidia.com> * fixed audio links Signed-off-by: ekmb <ebakhturina@nvidia.com> * remove old notebook Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix typo Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * [ASR] Audio processing base, multi-channel enhancement models (#5356) * Audio processing base model, enc-mask-dec enhancement, tests and modules Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed review comments Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fixed CodeQL warnings Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed PR comments Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed PR comments: - renamed AudioProcessingModel to AudioToAudioModel - various small modifications - updated unit tests Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Addressed comments - Moved spectrogram to audio_preprocessing - Renamed MultichannelFeatures - Updated config and unit tests Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Expose ClusteringDiarizer device (#5681) * Expose device for users to set Signed-off-by: SeanNaren <snarenthiran@nvidia.com> * Expose device for users to set Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: SeanNaren <snarenthiran@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Add Beam Search support to ASR transcribe() (#5443) * Add support for beam decoding via high level API. Signed-off-by: smajumdar <titu1994@gmail.com> * Add ctc decoding section Signed-off-by: smajumdar <titu1994@gmail.com> * Update ctc transcribe API to return results from beam search Signed-off-by: smajumdar <titu1994@gmail.com> * Add argument to preserve arpa file Signed-off-by: smajumdar <titu1994@gmail.com> * Update script to use hydra config, add some support for future compute timesteps, add doc for ctc decoding Signed-off-by: smajumdar <titu1994@gmail.com> * Update eval script and doc to use new API Signed-off-by: smajumdar <titu1994@gmail.com> * Add tests for ctc greedy decoding Signed-off-by: smajumdar <titu1994@gmail.com> * Address reviewer comments and add docstrings Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix changes and address comments Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Propagate attention_dropout flag for GPT-3 (#5669) * Propagate attention_dropout flag for GPT-3 Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> * Add default to megatron_gpt_config Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Signed-off-by: Mikołaj Błaż <mblaz@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail…

shanmugamr1992 and others added 30 commits November 21, 2022 14:51

Adding SP and SAR support Bert

2401d93

Merge branch 'main' into BertSP

9306d51

Adding Sequence parallel support to Bert

c70bfef

[pre-commit.ci] auto fixes from pre-commit.com hooks

14fa019

for more information, see https://pre-commit.ci

Adding Sequence parallel support to Bert

9ec628d

Merge branch 'BertSP' of github.com:NVIDIA/NeMo into BertSP

3650f74

[pre-commit.ci] auto fixes from pre-commit.com hooks

0c4d1e9

for more information, see https://pre-commit.ci

Adding SP and SAR support Bert

cd41f6b

Merge branch 'BertSP' of github.com:NVIDIA/NeMo into BertSP

256d011

[pre-commit.ci] auto fixes from pre-commit.com hooks

570bf26

for more information, see https://pre-commit.ci

Adding SP and SAR support Bert

9aa025d

Merge branch 'BertSP' of github.com:NVIDIA/NeMo into BertSP

eae5833

Adding SP and SAR support Bert

d42cddf

Merge branch 'main' into BertSP

e97c9a2

Merge branch 'main' into BertSP

38556a5

Merge branch 'main' into BertSP

2c0cd03

Merge branch 'main' into BertSP

7f5dd87

Adding Sequence parallel support to Bert

40714e2

[pre-commit.ci] auto fixes from pre-commit.com hooks

2cac221

for more information, see https://pre-commit.ci

Adding Sequence parallel support to Bert

6482e53

Merge branch 'BertSP' of github.com:NVIDIA/NeMo into BertSP

53f1aaa

Merge branch 'main' into BertSP

a176205

[pre-commit.ci] auto fixes from pre-commit.com hooks

659dd08

for more information, see https://pre-commit.ci

Adding Sequence parallel support to Bert

cf2d84b

Adding Sequence parallel support to Bert

0eca03f

Merge branch 'main' into BertSP

e118b9e

Merge branch 'main' into BertSP

e0aa816

Update bert_model.py

a684150

Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com>

Adding tests

0f19f9e

Adding interleaved pipeline parallelism

bfc73f3

shanmugamr1992 and others added 13 commits December 12, 2022 16:29

Merge branch 'main' into BertInterleaved

f831d0c

Merge branch 'main' into BertInterleaved

5fce8d0

Merge branch 'main' into BertInterleaved

6e18263

Merge branch 'main' into BertInterleaved

5062472

Merge branch 'main' into BertInterleaved

27d1c91

Merge branch 'main' into BertInterleaved

59c5ad4

Merge branch 'main' into BertInterleaved

327d5ae

Merge branch 'main' into BertInterleaved

065953b

Fix bug fix sequence parallel and Interleaved

78230a3

Merge branch 'main' into BertInterleaved

c04bc0f

Merge branch 'main' into BertInterleaved

cc09878

Fix bug fix sequence parallel and Interleaved

da4655f

Merge branch 'main' into BertInterleaved

c87def0

shanmugamr1992 requested a review from ericharper December 15, 2022 20:49

shanmugamr1992 and others added 3 commits December 15, 2022 14:01

Merge branch 'main' into BertInterleaved

7b7c110

Merge branch 'main' into BertInterleaved

f5fea9f

Merge branch 'main' into BertInterleaved

7938c0d

ericharper approved these changes Dec 16, 2022

View reviewed changes

shanmugamr1992 merged commit e53e098 into main Dec 16, 2022

shanmugamr1992 deleted the BertInterleaved branch December 16, 2022 19:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bert interleaved #5556

Bert interleaved #5556

shanmugamr1992 commented Dec 6, 2022

ericharper left a comment

Bert interleaved #5556

Bert interleaved #5556

Conversation

shanmugamr1992 commented Dec 6, 2022

What does this PR do ?

Changelog

Usage

Before your PR is "Ready for review"

Who can review?

Additional Information

ericharper left a comment

Choose a reason for hiding this comment