Fix initializing weights from ptl ckpt with exclude #4807

sam1373 · 2022-08-25T05:00:41Z

Signed-off-by: sam1373 samuelkriman@gmail.com

What does this PR do ?

Changelog

fix typo "model_path" to "ckpt_path"
also makes the "load_from_string" parameter optional

PR Type:

New Feature
Bugfix
Documentation

Additional Information

Related to #4805

Signed-off-by: sam1373 <samuelkriman@gmail.com>

Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com>

* update branch Signed-off-by: ericharper <complex451@gmail.com> * update package info and dockerfile Signed-off-by: ericharper <complex451@gmail.com> * [TTS] bugfix for missing configs. (#4725) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix pynini install in TTS tutorials (#4729) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * [TTS] updated config with a German IPA phoneme tokenizer (#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update r1.11 to new heteronyms list (#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix tutorial formatting (#4778) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * update branch and typos (#4788) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * Adding support for models trained with full context for cache-aware streaming. (#4687) * added support for models trained with full context. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped seq_range Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed indexing in caching methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated docs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> * Update megatron encoder decoder model to support py37 for colab (#4791) * [ASR] Add pretrained ASR models for Croatian (#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fix style for import Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * added/fixed export for Megatron models (#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed style Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * included comments Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch for qa notebook Signed-off-by: ericharper <complex451@gmail.com> * Fix initializing weights from ptl ckpt with exclude (#4807) Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> * Fix index error from addition of voiced_mask and p_voiced (#4811) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * T5 prompt learning fixes (#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update to reflect new config names Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Sentencepiece fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix finetuning Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add finetune eval script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix name Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update Jenkinsfile Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update check Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Backward compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * G2P docs (#4841) * g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix providing glue in seq2seq eval (#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Updated inference code and squad scripts (#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <vadams@nvidia.com> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <vadams@nvidia.com> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added padded vocab size Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed val check interval value Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back dp=1 check Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Set the number of workers to 0 for validation and test sets in all enc-dec models (#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Revert pin memory Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added utf8 encoding (#4892) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * update readme with apex commit Signed-off-by: ericharper <complex451@gmail.com> * Add support for Apex distributed Adam optimizer with GPT-3 (#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug with DDP initialization Signed-off-by: Tim Moon <tmoon@nvidia.com> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix code formatting Signed-off-by: Tim Moon <tmoon@nvidia.com> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * fixed styles Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed unsued import. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed duplicated func defintion. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace 'r1.11.0' with 'main' in Jenkinsfile and all tutorials. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: PRE_RELEASE = 'rc0' Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace branch name to main for asr_with_adapters.ipynb. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix Fastpitch mixertts tutorial format to align with main to distingshuish diff Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: correct path for tokenizers. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: ericharper <complex451@gmail.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: anteju <108555623+anteju@users.noreply.github.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com>

* update branch Signed-off-by: ericharper <complex451@gmail.com> * update package info and dockerfile Signed-off-by: ericharper <complex451@gmail.com> * [TTS] bugfix for missing configs. (NVIDIA#4725) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix pynini install in TTS tutorials (NVIDIA#4729) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * [TTS] updated config with a German IPA phoneme tokenizer (NVIDIA#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update r1.11 to new heteronyms list (NVIDIA#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix tutorial formatting (NVIDIA#4778) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * update branch and typos (NVIDIA#4788) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * Adding support for models trained with full context for cache-aware streaming. (NVIDIA#4687) * added support for models trained with full context. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped seq_range Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed indexing in caching methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated docs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> * Update megatron encoder decoder model to support py37 for colab (NVIDIA#4791) * [ASR] Add pretrained ASR models for Croatian (NVIDIA#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fix style for import Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * added/fixed export for Megatron models (NVIDIA#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed style Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * included comments Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch for qa notebook Signed-off-by: ericharper <complex451@gmail.com> * Fix initializing weights from ptl ckpt with exclude (NVIDIA#4807) Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> * Fix index error from addition of voiced_mask and p_voiced (NVIDIA#4811) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * T5 prompt learning fixes (NVIDIA#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update to reflect new config names Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Sentencepiece fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix finetuning Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add finetune eval script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix name Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update Jenkinsfile Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update check Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Backward compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * G2P docs (NVIDIA#4841) * g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix providing glue in seq2seq eval (NVIDIA#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Updated inference code and squad scripts (NVIDIA#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <vadams@nvidia.com> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <vadams@nvidia.com> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added padded vocab size Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed val check interval value Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back dp=1 check Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Set the number of workers to 0 for validation and test sets in all enc-dec models (NVIDIA#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Revert pin memory Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (NVIDIA#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added utf8 encoding (NVIDIA#4892) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * update readme with apex commit Signed-off-by: ericharper <complex451@gmail.com> * Add support for Apex distributed Adam optimizer with GPT-3 (NVIDIA#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug with DDP initialization Signed-off-by: Tim Moon <tmoon@nvidia.com> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix code formatting Signed-off-by: Tim Moon <tmoon@nvidia.com> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * fixed styles Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed unsued import. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed duplicated func defintion. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace 'r1.11.0' with 'main' in Jenkinsfile and all tutorials. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: PRE_RELEASE = 'rc0' Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace branch name to main for asr_with_adapters.ipynb. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix Fastpitch mixertts tutorial format to align with main to distingshuish diff Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: correct path for tokenizers. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: ericharper <complex451@gmail.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: anteju <108555623+anteju@users.noreply.github.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>

* update branch Signed-off-by: ericharper <complex451@gmail.com> * update package info and dockerfile Signed-off-by: ericharper <complex451@gmail.com> * [TTS] bugfix for missing configs. (NVIDIA#4725) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix pynini install in TTS tutorials (NVIDIA#4729) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * [TTS] updated config with a German IPA phoneme tokenizer (NVIDIA#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update r1.11 to new heteronyms list (NVIDIA#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix tutorial formatting (NVIDIA#4778) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * update branch and typos (NVIDIA#4788) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * Adding support for models trained with full context for cache-aware streaming. (NVIDIA#4687) * added support for models trained with full context. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped seq_range Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed indexing in caching methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated docs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> * Update megatron encoder decoder model to support py37 for colab (NVIDIA#4791) * [ASR] Add pretrained ASR models for Croatian (NVIDIA#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fix style for import Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * added/fixed export for Megatron models (NVIDIA#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed style Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * included comments Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch for qa notebook Signed-off-by: ericharper <complex451@gmail.com> * Fix initializing weights from ptl ckpt with exclude (NVIDIA#4807) Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> * Fix index error from addition of voiced_mask and p_voiced (NVIDIA#4811) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * T5 prompt learning fixes (NVIDIA#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update to reflect new config names Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Sentencepiece fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix finetuning Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add finetune eval script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix name Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update Jenkinsfile Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update check Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Backward compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * G2P docs (NVIDIA#4841) * g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix providing glue in seq2seq eval (NVIDIA#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Updated inference code and squad scripts (NVIDIA#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <vadams@nvidia.com> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <vadams@nvidia.com> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added padded vocab size Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed val check interval value Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back dp=1 check Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Set the number of workers to 0 for validation and test sets in all enc-dec models (NVIDIA#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Revert pin memory Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (NVIDIA#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added utf8 encoding (NVIDIA#4892) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * update readme with apex commit Signed-off-by: ericharper <complex451@gmail.com> * Add support for Apex distributed Adam optimizer with GPT-3 (NVIDIA#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug with DDP initialization Signed-off-by: Tim Moon <tmoon@nvidia.com> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix code formatting Signed-off-by: Tim Moon <tmoon@nvidia.com> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * fixed styles Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed unsued import. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * removed duplicated func defintion. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace 'r1.11.0' with 'main' in Jenkinsfile and all tutorials. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: PRE_RELEASE = 'rc0' Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * replace branch name to main for asr_with_adapters.ipynb. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix Fastpitch mixertts tutorial format to align with main to distingshuish diff Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * fix: correct path for tokenizers. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: ericharper <complex451@gmail.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: anteju <108555623+anteju@users.noreply.github.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com>

* [TTS] bugfix for missing configs. (#4725) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * docs typo fix Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Fix pynini install in TTS tutorials (#4729) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * Fix ASR notebooks (#4738) Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> * Multilingual VAD model (#4734) * add ngc link Signed-off-by: fayejf <fayejf07@gmail.com> * add tuned VAD config on ASR data Signed-off-by: fayejf <fayejf07@gmail.com> * yaml note Signed-off-by: fayejf <fayejf07@gmail.com> * update vad asr notebook with mVAD Signed-off-by: fayejf <fayejf07@gmail.com> * update vad infer config comment Signed-off-by: fayejf <fayejf07@gmail.com> * fix Signed-off-by: fayejf <fayejf07@gmail.com> * mvad sd config for ch109 Signed-off-by: fayejf <fayejf07@gmail.com> * update sd readme Signed-off-by: fayejf <fayejf07@gmail.com> * add new mVAD model to doc Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * update sd tutorial with mVAD Signed-off-by: fayejf <fayejf07@gmail.com> * typo fix Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: fayejf <fayejf07@gmail.com> * publish pretrained itn t5 model for English (#4748) Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com> * Updated docs and doc paths (#4754) * Updated docs and doc paths Signed-off-by: Virginia Adams <vadams@nvidia.com> * Update Multitask_Prompt_and_PTuning.ipynb * Update README.rst * Changed branch name to use single quotes Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * fix bug relating to ddp strategy in joint intent slot classification tutorial (#4762) * [TTS] updated config with a German IPA phoneme tokenizer (#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update r1.11 to new heteronyms list (#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (#4763) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (#4770) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update CMUdict with more recent 0.7b entries (#4768) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Install pynini in docker container (#4733) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Fix tutorial formatting (#4778) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * [TTS] deprecated old scripts for ljspeech. (#4780) * deprecated old scripts for ljspeech. * removed relevent function calls in TTS docs. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * update branch and typos (#4788) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * Adding support for models trained with full context for cache-aware streaming. (#4687) * added support for models trained with full context. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped seq_range Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed indexing in caching methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated docs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> * Update megatron encoder decoder model to support py37 for colab (#4791) * [ASR] Add pretrained ASR models for Croatian (#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fix style for import Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * added/fixed export for Megatron models (#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed style Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * included comments Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch for qa notebook Signed-off-by: ericharper <complex451@gmail.com> * Fix initializing weights from ptl ckpt with exclude (#4807) Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> * Fix index error from addition of voiced_mask and p_voiced (#4811) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * T5 prompt learning fixes (#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update to reflect new config names Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Sentencepiece fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix finetuning Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add finetune eval script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix name Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update Jenkinsfile Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update check Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Backward compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * G2P docs (#4841) * g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix providing glue in seq2seq eval (#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Updated inference code and squad scripts (#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <vadams@nvidia.com> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <vadams@nvidia.com> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added padded vocab size Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed val check interval value Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back dp=1 check Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Update README.rst * Fix uppercasing mismatch for IPA heteronyms (#4860) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * Set the number of workers to 0 for validation and test sets in all enc-dec models (#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Revert pin memory Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> * Fix mha (#4866) * fix bug in mha forward function related to cache update return type Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * fix lgtm Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> * ipa bug fix (#4871) Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added utf8 encoding (#4892) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * 1. Applying the same patch to r1.11.0 (#4894) Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> * Update tutorials.rst (#4897) * update readme with apex commit Signed-off-by: ericharper <complex451@gmail.com> * Add support for Apex distributed Adam optimizer with GPT-3 (#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug with DDP initialization Signed-off-by: Tim Moon <tmoon@nvidia.com> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix code formatting Signed-off-by: Tim Moon <tmoon@nvidia.com> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * latent model support * 1. Debugging. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. * update branch Signed-off-by: ericharper <complex451@gmail.com> * fix replace_bos_with_pad not found (#6443) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Support Swiglu in TP PP Conversion (#6437) * Support Swiglu in TP PP Conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Guard activation Signed-off-by: smajumdar <titu1994@gmail.com> * Guard activation Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> * BERT pre-training mp fork to spawn (#6442) * change bert fork to spawn Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * num_workers=0 fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Meagtron encoder decoder fix for empty validation outputs (#6459) * 1. Meagtron encoder decoder fix for empty validation outputs. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. --------- Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> * Added/updated new Conformer configs (#6426) * updated conf files. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * moved longconformer confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated readme. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated readme. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated batch sizes and added fastconformer ctc streaming configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated batch sizes. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added hybrid support. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added hybrid support. Signed-off-by: Vahid <vnoroozi@nvidia.com> --------- Signed-off-by: Vahid <vnoroozi@nvidia.com> * reduce workers on NMT CI (#6472) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * move to nvidia megatron repo (#6465) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Megatron KERPLE positional embeddings (#6478) * [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] whitelist broken path fix. (#6412) * [TTS] whitelist broken path fix. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] FastPitch speaker encoder (#6417) * Add initial codes Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove wemb Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix import Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore aligner loss Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add ConditionalInput Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and support pre-trained config Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change copyright and random weight test Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add initial codes Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix import error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add initial codes Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix dataset error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove reference speaker embedding Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove SV encoder Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix length type Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix append Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Move error msg Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add look-up into speaker encoder Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add valueerror msg Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Move lookup Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove unused Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Rebase and Fix error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix spk encoder Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Rename n_speakers Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix n_speakers None error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Sharded manifests for tarred datasets (#6395) * testing sharded manifests Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * compatibility Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * proper fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * adding flag tot convert_to_tarred_audio_dataset Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * shard_manifests conf param Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * distributed checks Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typo Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typo Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes based on PR comments and tests Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes to convert_to_tarred_audio_dataset.py Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * reversing manifest shards flag Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * excluding manifests from webdataset url expansion Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * expand manifest paths before attempting to cache from datastore Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * explicit use of UTF-8 for manifest i/o Signed-off-by: Dima Rekesh <bmwshop@gmail.com> --------- Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update wfst_text_normalization.rst (#6374) Add Hungarian (incoming in NeMo-text-processing) Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> * Support Swiglu in TP PP Conversion (#6437) (#6451) * Support Swiglu in TP PP Conversion * Guard activation * Guard activation --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * Update NeMo_TTS_Primer.ipynb (#6436) * Update NeMo_TTS_Primer.ipynb Changed a mistake in line 782. Instead of frequency band (ie. pitch) we should write frequency bin. Note that frequency bins in FFT are not related to pitch. Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> * Update NeMo_TTS_Primer.ipynb Corrected the description of spectrogram and mel spectrogram calculations in lines 782 & 783 and added a fourth point to the description and added a reference for more mathematical details at the end of this point. Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> --------- Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> * add rampup batch size support for Megatron GPT (#6424) * added rampup batch size support Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * added tests for rampup batch size Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * fixed the typos Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * added assertions Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * changed assertion rules Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * deleted unused imports Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * changed tests for rampup batch size Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * updated rampup batch size tests Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed styling Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * rampup batch size tests changes Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> --------- Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Meagtron encoder decoder fix for empty validation outputs (#6459) (#6461) * 1. Meagtron encoder decoder fix for empty validation outputs. * 1. Debugging. --------- Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <michalivne@users.noreply.github.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> * Code-Switching dataset creation - upgrading to aggregate tokenizer manifest format (#6448) * added functionality to create agg tokenizer compatible manifest for CS, flag to use this mode by default Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> * updated README with the new agg_tokenizer_manifest flag Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> * fixed typo in scripts/speech_recognition/code_switching/README.md Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> * changed agg_tokenizer_manifest to is_lid_manifest Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> --------- Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> * Added/updated new Conformer configs (#6426) (#6467) * Update script for ngram rnnt and hat beam search decoding (#6370) * add rnnt ngram beamsearch script Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add return encoding embedding option Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * update script Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add rnnt and hat ngram decoding script Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add some parameters Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add return_encoder_embeddings parameter to RNNTDecodingConfig Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * replace return_encoder_embeddings parameter Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * generalization of scipt behavior Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add manual encoder_embeddings calculation Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix beam_width value to 8 Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> * fix rescoring description Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> --------- Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * BERT pre-training mp fork to spawn (#6442) (#6454) * change bert fork to spawn * num_workers=0 fix --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * fix replace_bos_with_pad not found (#6443) (#6450) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * reduce workers on NMT CI (#6472) (#6474) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * 1. Added KERPLE positional embeddings to encoder-decoder. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Added a missing file. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixing commits. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. * 1. Debugging. * 1. Debugging. * 1. Debugging. --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Cheng-Ping Hsieh <37269846+hsiehjackson@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: Jim O’Regan <jaoregan@tcd.ie> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * 1. Added external index sample. (#6462) Signed-off-by: Micha Livne <mlivne@nvidia.com> * Fix cache aware hybrid bugs (#6466) * Update README to add core installation (#6488) * update README for megatron-core Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Fix typos (#6494) Signed-off-by: smajumdar <titu1994@gmail.com> * fix broken links r1.18.0 (#6501) * fix broken links Signed-off-by: Evelina <ebakhturina@nvidia.com> * fix broken links Signed-off-by: Evelina <ebakhturina@nvidia.com> --------- Signed-off-by: Evelina <ebakhturina@nvidia.com> * 1. Fixed gaussian hidden transform. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Finished updating hidden loss for MIM. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix custom forward_torch_softmax (#6512) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamp… (#6500) * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamps generator r1.18.0 Signed-off-by: Taejin Park <tango4j@gmail.com> * ignore keep_logits in FrameBatchASRLogits Signed-off-by: Taejin Park <tango4j@gmail.com> --------- Signed-off-by: Taejin Park <tango4j@gmail.com> * [TTS] fixed broken path. (#6514) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * 1. Added a hiddens module. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typos (#6523) (#6539) * Fix typos Signed-off-by: smajumdar <titu1994@gmail.com> * Fix typos Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> (cherry picked from commit 5468077f5127be1a4c88065de2544f4268b9a6e4) * added back the fast emit section to the configs. (#6540) * added back the fast emit section to the configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added back the fast emit section to the configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> --------- Signed-off-by: Vahid <vnoroozi@nvidia.com> * Fix fp16 (#6543) Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * fix (#6529) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * pass .scale instead of scaler object to core (#6545) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Change Megatron Enc Dec model to use persistent_workers (#6548) * persistent workers Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY (#6549) * Added fastconfomer hybrid asr models for en, es, it, de, pl, hr, ua, by Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * updated ASR docs with the fastconformer hybrid checkpoints Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * added the fastconformer RNNT and CTC models Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> --------- Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * Add scores for FastConformer models (#6557) Signed-off-by: smajumdar <titu1994@gmail.com> * Patch transcribe and support offline transcribe for hybrid model (#6550) Signed-off-by: fayejf <fayejf07@gmail.com> * Not doing CastToFloat by default (#6524) * Not doing CastToFloat by default Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Added docustring Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Dummy commit Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> --------- Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * temp rtd fix (#6568) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py for speedup (#6565) * Update manifest.py Re-order the checks for faster processing audio filepaths that are already absolute paths Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update manifest.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> * Turn autocast off when precision is fp32 (#6554) * Turn autocast off when precision is fp32 Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * address review Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * merge Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * More streaming conformer export fixes (#6567) Signed-off-by: Greg Clark <grclark@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> * Fix batch size reconf for T5 FT for multi-validation (#6582) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Updated Megatron LM encoder/decoder to use cfg for hiddens. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Added support to register externalhidden loss / transforms. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make tensor split contiguous (#6580) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Patches from main to r1.18.0 for Virtual Parallel (#6592) * Add interleaved pp support (#6498) * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Switch to megatron core Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 892987169ef277f328e15b71a5a0c9bd961c8ee7) * Add patches for Virtual Parallel conversion (#6589) * Add patches for Virtual Parllel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 1d813a372ab51688e3af6395d905a4c0366ffd23) * Documentation for ASR-TTS models (#6594) * Add docs about hybrid ASR-TTS models Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add docs about text-only datasets Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add docs about ASR-TTS checkpoints Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add docs about ASR-TTS configs and training Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * ASR-TTS docs: add to api, fix imports Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Wrap optional import Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Revert general ASR import Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Update SDP docs (#6485) * add info about SDP e.g. processor classes in docs Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add link to SDP docs in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * address code review comments and add SDP overview diagram Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix spelling typo Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Create dummy iters to satisy len checks (#6600) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * Restore GPT support for interleaved pipeline parallelism (#6528) * Restore logic for data-parallel communication with pipeline parallelism in GPT Signed-off-by: Tim Moon <tmoon@nvidia.com> * Support dynamic attention masks in GPT Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Debug typos Signed-off-by: Tim Moon <tmoon@nvidia.com> * Debug data iterator caching with interleaved pipeline parallelism Each model chunk accesses the data iterator multiple times, so we need to cache multiple samples. Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update Megatron-LM commit Signed-off-by: Tim Moon <tmoon@nvidia.com> * Distinguish between list of data iterators and data iterator that is a list Signed-off-by: Tim Moon <tmoon@nvidia.com> * Create dummy iters to satisy len checks Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Kludge while waiting for Megatron-LM update Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set transformers offline to avoid rate limiting Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Eric Harper <complex451@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * Patch transcribe_util for steaming mode and add wer calculation back to inference scripts (#6601) * fix write Signed-off-by: fayejf <fayejf07@gmail.com> * decoding ctc Signed-off-by: fayejf <fayejf07@gmail.com> * temp set rnnt decoding return_best_hypothesis to true Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to transcribe_speech as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_rnnt as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_ctc as requested Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * reflect change in asr_evaluator Signed-off-by: fayejf <fayejf07@gmail.com> * reflect som and vahid comment Signed-off-by: fayejf <fayejf07@gmail.com> * remove return_best_hy=true in transcribe_speech Signed-off-by: fayejf <fayejf07@gmail.com> * no text skip Signed-off-by: fayejf <fayejf07@gmail.com> --------- Signed-off-by: fayejf <fayejf07@gmail.com> * 1. Added example conf YAML. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support in tensor_parallel. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add hat image to docs (#6619) Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * update core commit hash in readme (#6622) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Patch decoding for PC models (#6630) * Patch decoding logic for PC models Signed-off-by: smajumdar <titu1994@gmail.com> * Patch decoding logic for PC models Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> * Fix wer.py where 'errors' variable was not set (#6633) Fix wer.py where 'errors' variable was not set when both reference and hypothesis are empty strings Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * fix att_context_size bug for older models. (#6635) Signed-off-by: Vahid <vnoroozi@nvidia.com> * Add megatron_core to requirements (#6639) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * remove from jenkins Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> * remove dup (#6643) Signed-off-by: ericharper <complex451@gmail.com> * 1. Fixed config to use names, and added better error messages. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support to pass extra data to hiddens for loss computation. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Working on passing extra data to hiddnes. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Fixed support in loading .nemo without hiddnes module. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Improved and fixed logging of validation and testing. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed training logging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed logging of hidden loss. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Fixed logging names. 2. Added logging to hiddens and tokens loss. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Fixed conflicts. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Evelina <ebakhturina@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: Greg Clark <grclark@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: bene-ges <61418381+bene-ges@users.noreply.github.com> Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Co-authored-by: anteju <108555623+anteju@users.noreply.github.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Co-authored-by: Neha Tadimeti <ntadimeti@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Cheng-Ping Hsieh <37269846+hsiehjackson@users.noreply.github.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: Jim O’Regan <jaoregan@tcd.ie> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Greg Clark <grclark@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com>

* [TTS] bugfix for missing configs. (#4725) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * docs typo fix Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Fix pynini install in TTS tutorials (#4729) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * Fix ASR notebooks (#4738) Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> * Multilingual VAD model (#4734) * add ngc link Signed-off-by: fayejf <fayejf07@gmail.com> * add tuned VAD config on ASR data Signed-off-by: fayejf <fayejf07@gmail.com> * yaml note Signed-off-by: fayejf <fayejf07@gmail.com> * update vad asr notebook with mVAD Signed-off-by: fayejf <fayejf07@gmail.com> * update vad infer config comment Signed-off-by: fayejf <fayejf07@gmail.com> * fix Signed-off-by: fayejf <fayejf07@gmail.com> * mvad sd config for ch109 Signed-off-by: fayejf <fayejf07@gmail.com> * update sd readme Signed-off-by: fayejf <fayejf07@gmail.com> * add new mVAD model to doc Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * update sd tutorial with mVAD Signed-off-by: fayejf <fayejf07@gmail.com> * typo fix Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: fayejf <fayejf07@gmail.com> * publish pretrained itn t5 model for English (#4748) Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com> * Updated docs and doc paths (#4754) * Updated docs and doc paths Signed-off-by: Virginia Adams <vadams@nvidia.com> * Update Multitask_Prompt_and_PTuning.ipynb * Update README.rst * Changed branch name to use single quotes Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * fix bug relating to ddp strategy in joint intent slot classification tutorial (#4762) * [TTS] updated config with a German IPA phoneme tokenizer (#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update r1.11 to new heteronyms list (#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (#4763) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (#4770) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update CMUdict with more recent 0.7b entries (#4768) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Install pynini in docker container (#4733) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Fix tutorial formatting (#4778) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * [TTS] deprecated old scripts for ljspeech. (#4780) * deprecated old scripts for ljspeech. * removed relevent function calls in TTS docs. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * update branch and typos (#4788) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * Adding support for models trained with full context for cache-aware streaming. (#4687) * added support for models trained with full context. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped seq_range Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed indexing in caching methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated docs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> * Update megatron encoder decoder model to support py37 for colab (#4791) * [ASR] Add pretrained ASR models for Croatian (#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fix style for import Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * added/fixed export for Megatron models (#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed style Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * included comments Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch for qa notebook Signed-off-by: ericharper <complex451@gmail.com> * Fix initializing weights from ptl ckpt with exclude (#4807) Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> * Fix index error from addition of voiced_mask and p_voiced (#4811) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * T5 prompt learning fixes (#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update to reflect new config names Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Sentencepiece fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix finetuning Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add finetune eval script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix name Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update Jenkinsfile Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update check Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Backward compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * G2P docs (#4841) * g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix providing glue in seq2seq eval (#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Updated inference code and squad scripts (#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <vadams@nvidia.com> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <vadams@nvidia.com> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added padded vocab size Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed val check interval value Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back dp=1 check Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Update README.rst * Fix uppercasing mismatch for IPA heteronyms (#4860) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * Set the number of workers to 0 for validation and test sets in all enc-dec models (#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Revert pin memory Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> * Fix mha (#4866) * fix bug in mha forward function related to cache update return type Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * fix lgtm Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> * ipa bug fix (#4871) Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added utf8 encoding (#4892) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * 1. Applying the same patch to r1.11.0 (#4894) Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> * Update tutorials.rst (#4897) * update readme with apex commit Signed-off-by: ericharper <complex451@gmail.com> * Add support for Apex distributed Adam optimizer with GPT-3 (#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug with DDP initialization Signed-off-by: Tim Moon <tmoon@nvidia.com> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix code formatting Signed-off-by: Tim Moon <tmoon@nvidia.com> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * latent model support * 1. Debugging. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. * update branch Signed-off-by: ericharper <complex451@gmail.com> * fix replace_bos_with_pad not found (#6443) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Support Swiglu in TP PP Conversion (#6437) * Support Swiglu in TP PP Conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Guard activation Signed-off-by: smajumdar <titu1994@gmail.com> * Guard activation Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> * BERT pre-training mp fork to spawn (#6442) * change bert fork to spawn Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * num_workers=0 fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Meagtron encoder decoder fix for empty validation outputs (#6459) * 1. Meagtron encoder decoder fix for empty validation outputs. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. --------- Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> * Added/updated new Conformer configs (#6426) * updated conf files. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * moved longconformer confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated readme. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated readme. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated batch sizes and added fastconformer ctc streaming configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated batch sizes. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added hybrid support. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added hybrid support. Signed-off-by: Vahid <vnoroozi@nvidia.com> --------- Signed-off-by: Vahid <vnoroozi@nvidia.com> * reduce workers on NMT CI (#6472) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * move to nvidia megatron repo (#6465) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Megatron KERPLE positional embeddings (#6478) * [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] whitelist broken path fix. (#6412) * [TTS] whitelist broken path fix. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] FastPitch speaker encoder (#6417) * Add initial codes Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove wemb Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix import Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore aligner loss Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add ConditionalInput Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and support pre-trained config Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change copyright and random weight test Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add initial codes Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix import error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add initial codes Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix dataset error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove reference speaker embedding Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove SV encoder Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix length type Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix append Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Move error msg Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add look-up into speaker encoder Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add valueerror msg Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Move lookup Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove unused Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Rebase and Fix error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix spk encoder Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Rename n_speakers Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix n_speakers None error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Sharded manifests for tarred datasets (#6395) * testing sharded manifests Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * compatibility Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * proper fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * adding flag tot convert_to_tarred_audio_dataset Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * shard_manifests conf param Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * distributed checks Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typo Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typo Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes based on PR comments and tests Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes to convert_to_tarred_audio_dataset.py Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * reversing manifest shards flag Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * excluding manifests from webdataset url expansion Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * expand manifest paths before attempting to cache from datastore Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * explicit use of UTF-8 for manifest i/o Signed-off-by: Dima Rekesh <bmwshop@gmail.com> --------- Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update wfst_text_normalization.rst (#6374) Add Hungarian (incoming in NeMo-text-processing) Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> * Support Swiglu in TP PP Conversion (#6437) (#6451) * Support Swiglu in TP PP Conversion * Guard activation * Guard activation --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * Update NeMo_TTS_Primer.ipynb (#6436) * Update NeMo_TTS_Primer.ipynb Changed a mistake in line 782. Instead of frequency band (ie. pitch) we should write frequency bin. Note that frequency bins in FFT are not related to pitch. Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> * Update NeMo_TTS_Primer.ipynb Corrected the description of spectrogram and mel spectrogram calculations in lines 782 & 783 and added a fourth point to the description and added a reference for more mathematical details at the end of this point. Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> --------- Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> * add rampup batch size support for Megatron GPT (#6424) * added rampup batch size support Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * added tests for rampup batch size Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * fixed the typos Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * added assertions Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * changed assertion rules Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * deleted unused imports Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * changed tests for rampup batch size Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * updated rampup batch size tests Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed styling Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * rampup batch size tests changes Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> --------- Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Meagtron encoder decoder fix for empty validation outputs (#6459) (#6461) * 1. Meagtron encoder decoder fix for empty validation outputs. * 1. Debugging. --------- Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <michalivne@users.noreply.github.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> * Code-Switching dataset creation - upgrading to aggregate tokenizer manifest format (#6448) * added functionality to create agg tokenizer compatible manifest for CS, flag to use this mode by default Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> * updated README with the new agg_tokenizer_manifest flag Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> * fixed typo in scripts/speech_recognition/code_switching/README.md Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> * changed agg_tokenizer_manifest to is_lid_manifest Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> --------- Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> * Added/updated new Conformer configs (#6426) (#6467) * Update script for ngram rnnt and hat beam search decoding (#6370) * add rnnt ngram beamsearch script Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add return encoding embedding option Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * update script Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add rnnt and hat ngram decoding script Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add some parameters Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add return_encoder_embeddings parameter to RNNTDecodingConfig Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * replace return_encoder_embeddings parameter Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * generalization of scipt behavior Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add manual encoder_embeddings calculation Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix beam_width value to 8 Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> * fix rescoring description Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> --------- Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * BERT pre-training mp fork to spawn (#6442) (#6454) * change bert fork to spawn * num_workers=0 fix --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * fix replace_bos_with_pad not found (#6443) (#6450) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * reduce workers on NMT CI (#6472) (#6474) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * 1. Added KERPLE positional embeddings to encoder-decoder. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Added a missing file. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixing commits. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. * 1. Debugging. * 1. Debugging. * 1. Debugging. --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Cheng-Ping Hsieh <37269846+hsiehjackson@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: Jim O’Regan <jaoregan@tcd.ie> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * 1. Added external index sample. (#6462) Signed-off-by: Micha Livne <mlivne@nvidia.com> * Fix cache aware hybrid bugs (#6466) * Update README to add core installation (#6488) * update README for megatron-core Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Fix typos (#6494) Signed-off-by: smajumdar <titu1994@gmail.com> * fix broken links r1.18.0 (#6501) * fix broken links Signed-off-by: Evelina <ebakhturina@nvidia.com> * fix broken links Signed-off-by: Evelina <ebakhturina@nvidia.com> --------- Signed-off-by: Evelina <ebakhturina@nvidia.com> * 1. Fixed gaussian hidden transform. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Finished updating hidden loss for MIM. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix custom forward_torch_softmax (#6512) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamp… (#6500) * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamps generator r1.18.0 Signed-off-by: Taejin Park <tango4j@gmail.com> * ignore keep_logits in FrameBatchASRLogits Signed-off-by: Taejin Park <tango4j@gmail.com> --------- Signed-off-by: Taejin Park <tango4j@gmail.com> * [TTS] fixed broken path. (#6514) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * 1. Added a hiddens module. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typos (#6523) (#6539) * Fix typos Signed-off-by: smajumdar <titu1994@gmail.com> * Fix typos Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> (cherry picked from commit 5468077f5127be1a4c88065de2544f4268b9a6e4) * added back the fast emit section to the configs. (#6540) * added back the fast emit section to the configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added back the fast emit section to the configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> --------- Signed-off-by: Vahid <vnoroozi@nvidia.com> * Fix fp16 (#6543) Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * fix (#6529) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * pass .scale instead of scaler object to core (#6545) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Change Megatron Enc Dec model to use persistent_workers (#6548) * persistent workers Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY (#6549) * Added fastconfomer hybrid asr models for en, es, it, de, pl, hr, ua, by Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * updated ASR docs with the fastconformer hybrid checkpoints Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * added the fastconformer RNNT and CTC models Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> --------- Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * Add scores for FastConformer models (#6557) Signed-off-by: smajumdar <titu1994@gmail.com> * Patch transcribe and support offline transcribe for hybrid model (#6550) Signed-off-by: fayejf <fayejf07@gmail.com> * Not doing CastToFloat by default (#6524) * Not doing CastToFloat by default Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Added docustring Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Dummy commit Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> --------- Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * temp rtd fix (#6568) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py for speedup (#6565) * Update manifest.py Re-order the checks for faster processing audio filepaths that are already absolute paths Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update manifest.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> * Turn autocast off when precision is fp32 (#6554) * Turn autocast off when precision is fp32 Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * address review Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * merge Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * More streaming conformer export fixes (#6567) Signed-off-by: Greg Clark <grclark@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> * Fix batch size reconf for T5 FT for multi-validation (#6582) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Updated Megatron LM encoder/decoder to use cfg for hiddens. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Added support to register externalhidden loss / transforms. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make tensor split contiguous (#6580) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Patches from main to r1.18.0 for Virtual Parallel (#6592) * Add interleaved pp support (#6498) * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Switch to megatron core Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 892987169ef277f328e15b71a5a0c9bd961c8ee7) * Add patches for Virtual Parallel conversion (#6589) * Add patches for Virtual Parllel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 1d813a372ab51688e3af6395d905a4c0366ffd23) * Documentation for ASR-TTS models (#6594) * Add docs about hybrid ASR-TTS models Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add docs about text-only datasets Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add docs about ASR-TTS checkpoints Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add docs about ASR-TTS configs and training Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * ASR-TTS docs: add to api, fix imports Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Wrap optional import Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Revert general ASR import Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Update SDP docs (#6485) * add info about SDP e.g. processor classes in docs Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add link to SDP docs in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * address code review comments and add SDP overview diagram Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix spelling typo Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Create dummy iters to satisy len checks (#6600) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * Restore GPT support for interleaved pipeline parallelism (#6528) * Restore logic for data-parallel communication with pipeline parallelism in GPT Signed-off-by: Tim Moon <tmoon@nvidia.com> * Support dynamic attention masks in GPT Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Debug typos Signed-off-by: Tim Moon <tmoon@nvidia.com> * Debug data iterator caching with interleaved pipeline parallelism Each model chunk accesses the data iterator multiple times, so we need to cache multiple samples. Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update Megatron-LM commit Signed-off-by: Tim Moon <tmoon@nvidia.com> * Distinguish between list of data iterators and data iterator that is a list Signed-off-by: Tim Moon <tmoon@nvidia.com> * Create dummy iters to satisy len checks Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Kludge while waiting for Megatron-LM update Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set transformers offline to avoid rate limiting Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Eric Harper <complex451@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * Patch transcribe_util for steaming mode and add wer calculation back to inference scripts (#6601) * fix write Signed-off-by: fayejf <fayejf07@gmail.com> * decoding ctc Signed-off-by: fayejf <fayejf07@gmail.com> * temp set rnnt decoding return_best_hypothesis to true Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to transcribe_speech as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_rnnt as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_ctc as requested Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * reflect change in asr_evaluator Signed-off-by: fayejf <fayejf07@gmail.com> * reflect som and vahid comment Signed-off-by: fayejf <fayejf07@gmail.com> * remove return_best_hy=true in transcribe_speech Signed-off-by: fayejf <fayejf07@gmail.com> * no text skip Signed-off-by: fayejf <fayejf07@gmail.com> --------- Signed-off-by: fayejf <fayejf07@gmail.com> * 1. Added example conf YAML. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support in tensor_parallel. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add hat image to docs (#6619) Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * update core commit hash in readme (#6622) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Patch decoding for PC models (#6630) * Patch decoding logic for PC models Signed-off-by: smajumdar <titu1994@gmail.com> * Patch decoding logic for PC models Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> * Fix wer.py where 'errors' variable was not set (#6633) Fix wer.py where 'errors' variable was not set when both reference and hypothesis are empty strings Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * fix att_context_size bug for older models. (#6635) Signed-off-by: Vahid <vnoroozi@nvidia.com> * Add megatron_core to requirements (#6639) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * remove from jenkins Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> * remove dup (#6643) Signed-off-by: ericharper <complex451@gmail.com> * 1. Fixed config to use names, and added better error messages. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support to pass extra data to hiddens for loss computation. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Working on passing extra data to hiddnes. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Fixed support in loading .nemo without hiddnes module. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Improved and fixed logging of validation and testing. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed training logging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed logging of hidden loss. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Fixed logging names. 2. Added logging to hiddens and tokens loss. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Fixed conflicts. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Evelina <ebakhturina@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: Greg Clark <grclark@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: bene-ges <61418381+bene-ges@users.noreply.github.com> Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Co-authored-by: anteju <108555623+anteju@users.noreply.github.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Co-authored-by: Neha Tadimeti <ntadimeti@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Cheng-Ping Hsieh <37269846+hsiehjackson@users.noreply.github.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: Jim O’Regan <jaoregan@tcd.ie> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Greg Clark <grclark@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com> Signed-off-by: dorotat <dorotat@nvidia.com>

* [TTS] bugfix for missing configs. (#4725) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * docs typo fix Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> * Fix pynini install in TTS tutorials (#4729) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * Fix ASR notebooks (#4738) Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> * Multilingual VAD model (#4734) * add ngc link Signed-off-by: fayejf <fayejf07@gmail.com> * add tuned VAD config on ASR data Signed-off-by: fayejf <fayejf07@gmail.com> * yaml note Signed-off-by: fayejf <fayejf07@gmail.com> * update vad asr notebook with mVAD Signed-off-by: fayejf <fayejf07@gmail.com> * update vad infer config comment Signed-off-by: fayejf <fayejf07@gmail.com> * fix Signed-off-by: fayejf <fayejf07@gmail.com> * mvad sd config for ch109 Signed-off-by: fayejf <fayejf07@gmail.com> * update sd readme Signed-off-by: fayejf <fayejf07@gmail.com> * add new mVAD model to doc Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * update sd tutorial with mVAD Signed-off-by: fayejf <fayejf07@gmail.com> * typo fix Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: fayejf <fayejf07@gmail.com> * publish pretrained itn t5 model for English (#4748) Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com> * Updated docs and doc paths (#4754) * Updated docs and doc paths Signed-off-by: Virginia Adams <vadams@nvidia.com> * Update Multitask_Prompt_and_PTuning.ipynb * Update README.rst * Changed branch name to use single quotes Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * fix bug relating to ddp strategy in joint intent slot classification tutorial (#4762) * [TTS] updated config with a German IPA phoneme tokenizer (#4756) * [TTS] added a German IPA phoneme tokenizer * [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence. * [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer. * [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update r1.11 to new heteronyms list (#4745) * Update configs to new heteronyms list * Remove old heteronyms list, add alt 'merchandise' pron to CMUdict * Update remaining references to old heteronyms list Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (#4763) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (#4770) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Update CMUdict with more recent 0.7b entries (#4768) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Install pynini in docker container (#4733) Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Fix tutorial formatting (#4778) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * [TTS] deprecated old scripts for ljspeech. (#4780) * deprecated old scripts for ljspeech. * removed relevent function calls in TTS docs. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * update branch and typos (#4788) Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> * Adding support for models trained with full context for cache-aware streaming. (#4687) * added support for models trained with full context. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * dropped seq_range Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed indexing in caching methods. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated docs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * addressed comments. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * change frame-wise to cache-aware. Signed-off-by: Vahid <vnoroozi@nvidia.com> * fixed code style. Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> * Update megatron encoder decoder model to support py37 for colab (#4791) * [ASR] Add pretrained ASR models for Croatian (#4682) * [ASR] Add pretrained ASR models for Croatian Signed-off-by: Ante Jukić <ajukic@nvidia.com> * Fix style for import Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * added/fixed export for Megatron models (#4712) * added/fixed export for Megatron models Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed style Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * fixed FusedScaleMaskSoftmax in BioMegatron Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> * included comments Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch for qa notebook Signed-off-by: ericharper <complex451@gmail.com> * Fix initializing weights from ptl ckpt with exclude (#4807) Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> * Fix index error from addition of voiced_mask and p_voiced (#4811) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * T5 prompt learning fixes (#4771) * RPE, hidden size and config fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update to reflect new config names Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Sentencepiece fixes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix finetuning Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add encoder seq len to gpt Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Add finetune eval script Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix name Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update Jenkinsfile Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update check Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Backward compat Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Update CI test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Split rank for Enc-Dec models Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Address comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * G2P docs (#4841) * g2p docs added Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix references Signed-off-by: ekmb <ebakhturina@nvidia.com> * address review feedback Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix providing glue in seq2seq eval (#4843) * Fix providing glue in seq2seq eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Updated inference code and squad scripts (#4835) * Updated inference code and squad scripts Signed-off-by: Virginia Adams <vadams@nvidia.com> * Reverted GPT & T5 inference files back to use NLPDDPlugin Signed-off-by: Virginia Adams <vadams@nvidia.com> * Overwrite frozen LM to use fused adam Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added padded vocab size Signed-off-by: Virginia Adams <vadams@nvidia.com> * Fixed val check interval value Signed-off-by: Virginia Adams <vadams@nvidia.com> * Python format fix Signed-off-by: Virginia Adams <vadams@nvidia.com> * Make t5 prompt learning preds write to file Signed-off-by: Virginia Adams <vadams@nvidia.com> * Added back dp=1 check Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Update README.rst * Fix uppercasing mismatch for IPA heteronyms (#4860) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * Set the number of workers to 0 for validation and test sets in all enc-dec models (#4790) * Set workers to 0 for validation and test Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Revert pin memory Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> * Fix mha (#4866) * fix bug in mha forward function related to cache update return type Signed-off-by: Yang Zhang <yangzhang@nvidia.com> * fix lgtm Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> * ipa bug fix (#4871) Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * Fix Megatron NMT consumed samples and ckpt_to_nemo split rank (#4884) * Fix nmt and ckpt_to_nemo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * added utf8 encoding (#4892) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> * 1. Applying the same patch to r1.11.0 (#4894) Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> * Update tutorials.rst (#4897) * update readme with apex commit Signed-off-by: ericharper <complex451@gmail.com> * Add support for Apex distributed Adam optimizer with GPT-3 (#4487) * Add support for Apex distributed Adam optimizer with GPT-3 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug in grad clipping with dist Adam Grad norm was computed over all params, not respecting model parallelism. Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix bug with DDP initialization Signed-off-by: Tim Moon <tmoon@nvidia.com> * Make distopt dependent on megatron_amp_o2 Signed-off-by: Tim Moon <tmoon@nvidia.com> * Fix code formatting Signed-off-by: Tim Moon <tmoon@nvidia.com> * Handle dist Adam in optimizer unit tests Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * update readme Signed-off-by: ericharper <complex451@gmail.com> * latent model support * 1. Debugging. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. * update branch Signed-off-by: ericharper <complex451@gmail.com> * fix replace_bos_with_pad not found (#6443) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Support Swiglu in TP PP Conversion (#6437) * Support Swiglu in TP PP Conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Guard activation Signed-off-by: smajumdar <titu1994@gmail.com> * Guard activation Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> * BERT pre-training mp fork to spawn (#6442) * change bert fork to spawn Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * num_workers=0 fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Meagtron encoder decoder fix for empty validation outputs (#6459) * 1. Meagtron encoder decoder fix for empty validation outputs. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. --------- Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> * Added/updated new Conformer configs (#6426) * updated conf files. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * moved longconformer confs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated readme. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated readme. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated batch sizes and added fastconformer ctc streaming configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * updated batch sizes. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added hybrid support. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added hybrid support. Signed-off-by: Vahid <vnoroozi@nvidia.com> --------- Signed-off-by: Vahid <vnoroozi@nvidia.com> * reduce workers on NMT CI (#6472) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * move to nvidia megatron repo (#6465) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Megatron KERPLE positional embeddings (#6478) * [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) [TTS] FastPitch adapter fine-tune and conditional layer normalization (#6416) --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] whitelist broken path fix. (#6412) * [TTS] whitelist broken path fix. Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * [TTS] FastPitch speaker encoder (#6417) * Add initial codes Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove wemb Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix import Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Restore aligner loss Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add ConditionalInput Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix error and support pre-trained config Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Rename config Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Change copyright and random weight test Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add initial codes Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix import error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add initial codes Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix dataset error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove reference speaker embedding Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove SV encoder Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix length type Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix append Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Move error msg Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add look-up into speaker encoder Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Add valueerror msg Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Move lookup Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Remove unused Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Rebase and Fix error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Fix spk encoder Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Rename n_speakers Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * Follow comments Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix n_speakers None error Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Sharded manifests for tarred datasets (#6395) * testing sharded manifests Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * compatibility Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * proper fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * adding flag tot convert_to_tarred_audio_dataset Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * shard_manifests conf param Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * propagating the shard_manifests param Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * distributed checks Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typo Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * typo Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * fixes Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes based on PR comments and tests Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes to convert_to_tarred_audio_dataset.py Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * reversing manifest shards flag Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * excluding manifests from webdataset url expansion Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * expand manifest paths before attempting to cache from datastore Signed-off-by: Dima Rekesh <bmwshop@gmail.com> * explicit use of UTF-8 for manifest i/o Signed-off-by: Dima Rekesh <bmwshop@gmail.com> --------- Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Update wfst_text_normalization.rst (#6374) Add Hungarian (incoming in NeMo-text-processing) Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> * Support Swiglu in TP PP Conversion (#6437) (#6451) * Support Swiglu in TP PP Conversion * Guard activation * Guard activation --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * Update NeMo_TTS_Primer.ipynb (#6436) * Update NeMo_TTS_Primer.ipynb Changed a mistake in line 782. Instead of frequency band (ie. pitch) we should write frequency bin. Note that frequency bins in FFT are not related to pitch. Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> * Update NeMo_TTS_Primer.ipynb Corrected the description of spectrogram and mel spectrogram calculations in lines 782 & 783 and added a fourth point to the description and added a reference for more mathematical details at the end of this point. Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> --------- Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> * add rampup batch size support for Megatron GPT (#6424) * added rampup batch size support Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * added tests for rampup batch size Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * fixed the typos Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * added assertions Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * changed assertion rules Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * deleted unused imports Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * changed tests for rampup batch size Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * updated rampup batch size tests Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixed styling Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> * rampup batch size tests changes Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> --------- Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Meagtron encoder decoder fix for empty validation outputs (#6459) (#6461) * 1. Meagtron encoder decoder fix for empty validation outputs. * 1. Debugging. --------- Signed-off-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Micha Livne <michalivne@users.noreply.github.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> * Code-Switching dataset creation - upgrading to aggregate tokenizer manifest format (#6448) * added functionality to create agg tokenizer compatible manifest for CS, flag to use this mode by default Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> * updated README with the new agg_tokenizer_manifest flag Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> * fixed typo in scripts/speech_recognition/code_switching/README.md Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> * changed agg_tokenizer_manifest to is_lid_manifest Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> --------- Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> * Added/updated new Conformer configs (#6426) (#6467) * Update script for ngram rnnt and hat beam search decoding (#6370) * add rnnt ngram beamsearch script Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add return encoding embedding option Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * update script Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add rnnt and hat ngram decoding script Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add some parameters Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add return_encoder_embeddings parameter to RNNTDecodingConfig Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * replace return_encoder_embeddings parameter Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * generalization of scipt behavior Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * remove return_encoder_embeddings parameter Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * add manual encoder_embeddings calculation Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix beam_width value to 8 Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> * fix rescoring description Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> --------- Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> * BERT pre-training mp fork to spawn (#6442) (#6454) * change bert fork to spawn * num_workers=0 fix --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * fix replace_bos_with_pad not found (#6443) (#6450) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * reduce workers on NMT CI (#6472) (#6474) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * 1. Added KERPLE positional embeddings to encoder-decoder. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Added a missing file. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixing commits. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. * 1. Debugging. * 1. Debugging. * 1. Debugging. --------- Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Cheng-Ping Hsieh <37269846+hsiehjackson@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: Jim O’Regan <jaoregan@tcd.ie> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Micha Livne <mlivne@nvidia.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * 1. Added external index sample. (#6462) Signed-off-by: Micha Livne <mlivne@nvidia.com> * Fix cache aware hybrid bugs (#6466) * Update README to add core installation (#6488) * update README for megatron-core Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Fix typos (#6494) Signed-off-by: smajumdar <titu1994@gmail.com> * fix broken links r1.18.0 (#6501) * fix broken links Signed-off-by: Evelina <ebakhturina@nvidia.com> * fix broken links Signed-off-by: Evelina <ebakhturina@nvidia.com> --------- Signed-off-by: Evelina <ebakhturina@nvidia.com> * 1. Fixed gaussian hidden transform. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Finished updating hidden loss for MIM. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix custom forward_torch_softmax (#6512) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamp… (#6500) * [BugFix] Force _get_batch_preds() to keep logits in decoder timestamps generator r1.18.0 Signed-off-by: Taejin Park <tango4j@gmail.com> * ignore keep_logits in FrameBatchASRLogits Signed-off-by: Taejin Park <tango4j@gmail.com> --------- Signed-off-by: Taejin Park <tango4j@gmail.com> * [TTS] fixed broken path. (#6514) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * 1. Added a hiddens module. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typos (#6523) (#6539) * Fix typos Signed-off-by: smajumdar <titu1994@gmail.com> * Fix typos Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> (cherry picked from commit 7fa47bfcd359f1ea80ad056d8bb2d78f97c6206a) * added back the fast emit section to the configs. (#6540) * added back the fast emit section to the configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> * added back the fast emit section to the configs. Signed-off-by: Vahid <vnoroozi@nvidia.com> --------- Signed-off-by: Vahid <vnoroozi@nvidia.com> * Fix fp16 (#6543) Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * fix (#6529) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * pass .scale instead of scaler object to core (#6545) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Change Megatron Enc Dec model to use persistent_workers (#6548) * persistent workers Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Add FastConformer Hybrid ASR models for EN, ES, IT, DE, PL, HR, UA, BY (#6549) * Added fastconfomer hybrid asr models for en, es, it, de, pl, hr, ua, by Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * updated ASR docs with the fastconformer hybrid checkpoints Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * added the fastconformer RNNT and CTC models Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> --------- Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> * Add scores for FastConformer models (#6557) Signed-off-by: smajumdar <titu1994@gmail.com> * Patch transcribe and support offline transcribe for hybrid model (#6550) Signed-off-by: fayejf <fayejf07@gmail.com> * Not doing CastToFloat by default (#6524) * Not doing CastToFloat by default Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Added docustring Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * Dummy commit Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> --------- Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> * temp rtd fix (#6568) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update manifest.py for speedup (#6565) * Update manifest.py Re-order the checks for faster processing audio filepaths that are already absolute paths Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * Update manifest.py Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> --------- Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> * Turn autocast off when precision is fp32 (#6554) * Turn autocast off when precision is fp32 Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * address review Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fixes Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * merge Signed-off-by: Abhinav Khattar <aklife97@gmail.com> --------- Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> * More streaming conformer export fixes (#6567) Signed-off-by: Greg Clark <grclark@nvidia.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> * Fix batch size reconf for T5 FT for multi-validation (#6582) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Updated Megatron LM encoder/decoder to use cfg for hiddens. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Added support to register externalhidden loss / transforms. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Make tensor split contiguous (#6580) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Patches from main to r1.18.0 for Virtual Parallel (#6592) * Add interleaved pp support (#6498) * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Add support for Virtual Pipeline Parallel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * Switch to megatron core Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit fa9ab5b41301c9d92c97bf37894f9c94de6b91b5) * Add patches for Virtual Parallel conversion (#6589) * Add patches for Virtual Parllel conversion Signed-off-by: smajumdar <titu1994@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: smajumdar <titu1994@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit 4d639ef36b10d8536db02bc47ea30f4acbc061bb) * Documentation for ASR-TTS models (#6594) * Add docs about hybrid ASR-TTS models Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add docs about text-only datasets Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add docs about ASR-TTS checkpoints Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Add docs about ASR-TTS configs and training Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * ASR-TTS docs: add to api, fix imports Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Clean up Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Wrap optional import Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Revert general ASR import Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> --------- Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> * Update SDP docs (#6485) * add info about SDP e.g. processor classes in docs Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * add link to SDP docs in README Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * address code review comments and add SDP overview diagram Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Fix spelling typo Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> --------- Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> * Create dummy iters to satisy len checks (#6600) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * Restore GPT support for interleaved pipeline parallelism (#6528) * Restore logic for data-parallel communication with pipeline parallelism in GPT Signed-off-by: Tim Moon <tmoon@nvidia.com> * Support dynamic attention masks in GPT Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Debug typos Signed-off-by: Tim Moon <tmoon@nvidia.com> * Debug data iterator caching with interleaved pipeline parallelism Each model chunk accesses the data iterator multiple times, so we need to cache multiple samples. Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update Megatron-LM commit Signed-off-by: Tim Moon <tmoon@nvidia.com> * Distinguish between list of data iterators and data iterator that is a list Signed-off-by: Tim Moon <tmoon@nvidia.com> * Create dummy iters to satisy len checks Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Kludge while waiting for Megatron-LM update Signed-off-by: Tim Moon <tmoon@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * set transformers offline to avoid rate limiting Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Eric Harper <complex451@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> * Patch transcribe_util for steaming mode and add wer calculation back to inference scripts (#6601) * fix write Signed-off-by: fayejf <fayejf07@gmail.com> * decoding ctc Signed-off-by: fayejf <fayejf07@gmail.com> * temp set rnnt decoding return_best_hypothesis to true Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to transcribe_speech as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_rnnt as requested Signed-off-by: fayejf <fayejf07@gmail.com> * add wer cal back to speech_to_text_buffered_infer_ctc as requested Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * reflect change in asr_evaluator Signed-off-by: fayejf <fayejf07@gmail.com> * reflect som and vahid comment Signed-off-by: fayejf <fayejf07@gmail.com> * remove return_best_hy=true in transcribe_speech Signed-off-by: fayejf <fayejf07@gmail.com> * no text skip Signed-off-by: fayejf <fayejf07@gmail.com> --------- Signed-off-by: fayejf <fayejf07@gmail.com> * 1. Added example conf YAML. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support in tensor_parallel. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * add hat image to docs (#6619) Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> * update core commit hash in readme (#6622) Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * Patch decoding for PC models (#6630) * Patch decoding logic for PC models Signed-off-by: smajumdar <titu1994@gmail.com> * Patch decoding logic for PC models Signed-off-by: smajumdar <titu1994@gmail.com> --------- Signed-off-by: smajumdar <titu1994@gmail.com> * Fix wer.py where 'errors' variable was not set (#6633) Fix wer.py where 'errors' variable was not set when both reference and hypothesis are empty strings Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> * fix att_context_size bug for older models. (#6635) Signed-off-by: Vahid <vnoroozi@nvidia.com> * Add megatron_core to requirements (#6639) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Remove from jenkins (#6641) * add megatron_core to requirements Signed-off-by: ericharper <complex451@gmail.com> * remove from jenkins Signed-off-by: ericharper <complex451@gmail.com> --------- Signed-off-by: ericharper <complex451@gmail.com> * remove dup (#6643) Signed-off-by: ericharper <complex451@gmail.com> * 1. Fixed config to use names, and added better error messages. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Added support to pass extra data to hiddens for loss computation. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Working on passing extra data to hiddnes. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Fixed support in loading .nemo without hiddnes module. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Improved and fixed logging of validation and testing. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed training logging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Fixed logging of hidden loss. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Fixed logging names. 2. Added logging to hiddens and tokens loss. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Fixed conflicts. Signed-off-by: Micha Livne <mlivne@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> * 1. Debugging. Signed-off-by: Micha Livne <mlivne@nvidia.com> --------- Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Signed-off-by: smajumdar <smajumdar@nvidia.com> Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Vladimir Bataev <vbataev@nvidia.com> Signed-off-by: ericharper <complex451@gmail.com> Signed-off-by: Vahid <vnoroozi@nvidia.com> Signed-off-by: Ante Jukić <ajukic@nvidia.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com> Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: Yang Zhang <yangzhang@nvidia.com> Signed-off-by: Micha Livne <mlivne@cs.toronto.edu> Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: smajumdar <titu1994@gmail.com> Signed-off-by: Micha Livne <mlivne@nvidia.com> Signed-off-by: hsiehjackson <c2hsieh@ucsd.edu> Signed-off-by: Dima Rekesh <bmwshop@gmail.com> Signed-off-by: Jim O’Regan <jaoregan@tcd.ie> Signed-off-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Signed-off-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Signed-off-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Signed-off-by: Kunal Dhawan <kunaldhawan97@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@gmail.com> Signed-off-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Signed-off-by: Evelina <ebakhturina@nvidia.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: KunalDhawan <kunaldhawan97@gmail.com> Signed-off-by: Boris Fomitchev <bfomitchev@nvidia.com> Signed-off-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Signed-off-by: Greg Clark <grclark@nvidia.com> Signed-off-by: Elena Rastorgueva <erastorgueva@nvidia.com> Signed-off-by: Eric Harper <complex451@gmail.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: bene-ges <61418381+bene-ges@users.noreply.github.com> Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: Vladimir Bataev <vbataev@nvidia.com> Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com> Co-authored-by: Eric Harper <complex451@gmail.com> Co-authored-by: Vahid Noroozi <VahidooX@users.noreply.github.com> Co-authored-by: anteju <108555623+anteju@users.noreply.github.com> Co-authored-by: Ante Jukić <ajukic@nvidia.com> Co-authored-by: David <amosalla@asu.edu> Co-authored-by: David Mosallanezhad <dmosallanezh@nvidia.com> Co-authored-by: Samuel Kriman <samuelkriman@gmail.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Sean Naren <sean.narenthiran@gmail.com> Co-authored-by: Yang Zhang <yzhang123@users.noreply.github.com> Co-authored-by: Sean Naren <snarenthiran@nvidia.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Co-authored-by: Neha Tadimeti <ntadimeti@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: Cheng-Ping Hsieh <37269846+hsiehjackson@users.noreply.github.com> Co-authored-by: Dima Rekesh <bmwshop@gmail.com> Co-authored-by: Jim O’Regan <jaoregan@tcd.ie> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Mostafa Ghorbandoost <mos.ghorbandoost@gmail.com> Co-authored-by: Dmytro Pykhtar <37850217+dimapihtar@users.noreply.github.com> Co-authored-by: Dmytro Pykhtar <dpykhtar@nvidia.com> Co-authored-by: Kunal Dhawan <kunaldhawan97@gmail.com> Co-authored-by: Andrei Andrusenko <52885736+andrusenkoau@users.noreply.github.com> Co-authored-by: Taejin Park <tango4j@gmail.com> Co-authored-by: Boris Fomitchev <borisfom@users.noreply.github.com> Co-authored-by: He Huang (Steve) <105218074+stevehuang52@users.noreply.github.com> Co-authored-by: Greg Clark <grclark@nvidia.com> Co-authored-by: Elena Rastorgueva <80532067+erastorgueva-nv@users.noreply.github.com>

fix

3df9f68

Signed-off-by: sam1373 <samuelkriman@gmail.com>

sam1373 marked this pull request as ready for review August 25, 2022 05:02

titu1994 approved these changes Aug 25, 2022

View reviewed changes

sam1373 merged commit 09b744a into NVIDIA:r1.11.0 Aug 25, 2022

ericharper pushed a commit that referenced this pull request Sep 9, 2022

Fix initializing weights from ptl ckpt with exclude (#4807)

def5b9e

Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com>

XuesongYang pushed a commit that referenced this pull request Sep 10, 2022

Fix initializing weights from ptl ckpt with exclude (#4807)

faa9038

Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com>

XuesongYang pushed a commit that referenced this pull request Sep 12, 2022

Fix initializing weights from ptl ckpt with exclude (#4807)

212ede2

Signed-off-by: sam1373 <samuelkriman@gmail.com> Signed-off-by: sam1373 <samuelkriman@gmail.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix initializing weights from ptl ckpt with exclude #4807

Fix initializing weights from ptl ckpt with exclude #4807

sam1373 commented Aug 25, 2022

Fix initializing weights from ptl ckpt with exclude #4807

Fix initializing weights from ptl ckpt with exclude #4807

Conversation

sam1373 commented Aug 25, 2022

What does this PR do ?

Changelog

Additional Information