Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TTS] fixed wrong pronunciations for r1.11. #4677

Merged
merged 5 commits into from
Aug 4, 2022
Merged

[TTS] fixed wrong pronunciations for r1.11. #4677

merged 5 commits into from
Aug 4, 2022

Conversation

XuesongYang
Copy link
Collaborator

What does this PR do ?

Mirror the bugfix from #4669

Collection: [Note which collection this PR will affect]

Changelog

  • Add specific line by line info of high level changes in this PR.

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you add or update any necessary documentation?
  • Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
    • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

  • New Feature
  • Bugfix
  • Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

  • Related to # (issue)

@XuesongYang
[TTS] fixed wrong pronunciations.
68a181a 
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
@XuesongYang
incremented the version number to 22.08 as @blisc suggested.
824a18e 
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
@XuesongYang
correct cmudict versions in world-wide places.
a616c7c 
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
@XuesongYang
Merge branch 'main' into fix-pron
c8021b4
@XuesongYang XuesongYang changed the title [TTS] fixed wrong pronunciations. [TTS] fixed wrong pronunciations for r1.11. Aug 4, 2022
@XuesongYang XuesongYang added the fix label Aug 4, 2022
@blisc blisc merged commit 02b1518 into r1.11.0 Aug 4, 2022
@blisc blisc mentioned this pull request Aug 4, 2022
Closed
8 tasks
@blisc blisc deleted the fix-pron branch August 4, 2022 20:49
MaximumEntropy pushed a commit that referenced this pull request Aug 5, 2022
@XuesongYang @MaximumEntropy
[TTS] fixed wrong pronunciations for r1.11. (#4677)
7a7cb39 
* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Davood-M pushed a commit to Davood-M/NeMo that referenced this pull request Aug 9, 2022
@XuesongYang @Davood-M
[TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)
ddf2ee7 
* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>
ericharper pushed a commit that referenced this pull request Aug 22, 2022
@XuesongYang @ericharper
[TTS] fixed wrong pronunciations for r1.11. (#4677)
f74f491 
* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
ericharper added a commit that referenced this pull request Aug 24, 2022
@ericharper @okuchaiev
@blisc @XuesongYang @rilango @michalivne @MaximumEntropy @nithinraok @titu1994 @vadam5 @redoctopus @fayejf @bene-ges @Zhilin123 @artbataev
Merge r1.11.0 main (#4787)
af7bea4 
* NeMo Megatron doc updates

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix notebook for buffered inference (#4703)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Prompt Learning Notebook Bug Fix (#4689)

* Added back dataset class list of dict input for generation in tutorial notebook

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* updated argument name for build dataset

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* add psutils to mock imports (#4728)

Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: ericharper <complex451@gmail.com>

* Update Aligner model and tutorial to add NGC checkpoint loading (#4714)

* Update Aligner model and tutorial to add NGC checkpoint loading

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix pynini install for Aligner notebook, minor formatting fix for model

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Aligner notebook formatting consistency

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] bugfix for missing configs. (#4725)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* docs typo fix

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* Fix pynini install in TTS tutorials (#4729)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix ASR notebooks (#4738)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Multilingual VAD model (#4734)

* add ngc link

Signed-off-by: fayejf <fayejf07@gmail.com>

* add tuned VAD config on ASR data

Signed-off-by: fayejf <fayejf07@gmail.com>

* yaml note

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad asr notebook with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad infer config comment

Signed-off-by: fayejf <fayejf07@gmail.com>

* fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* mvad sd config for ch109

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd readme

Signed-off-by: fayejf <fayejf07@gmail.com>

* add new mVAD model to doc

Signed-off-by: fayejf <fayejf07@gmail.com>

* style fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd tutorial with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* typo fix

Signed-off-by: fayejf <fayejf07@gmail.com>

Signed-off-by: fayejf <fayejf07@gmail.com>

* publish pretrained itn t5 model for English (#4748)

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>

* Updated docs and doc paths (#4754)

* Updated docs and doc paths

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update Multitask_Prompt_and_PTuning.ipynb

* Update README.rst

* Changed branch name to use single quotes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* fix bug relating to ddp strategy in joint intent slot classification tutorial (#4762)

* [TTS] updated config with a German IPA phoneme tokenizer (#4756)

* [TTS] added a German IPA phoneme tokenizer
* [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence.
* [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer.
* [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update r1.11 to new heteronyms list (#4745)

* Update configs to new heteronyms list
* Remove old heteronyms list, add alt 'merchandise' pron to CMUdict
* Update remaining references to old heteronyms list

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (#4763)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (#4770)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update CMUdict with more recent 0.7b entries (#4768)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Install pynini in docker container (#4733)

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Fix tutorial formatting (#4778)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* [TTS] deprecated old scripts for ljspeech. (#4780)

* deprecated old scripts for ljspeech.
* removed relevent function calls in TTS docs.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and requirements

Signed-off-by: ericharper <complex451@gmail.com>

* update container

Signed-off-by: ericharper <complex451@gmail.com>

* Update stragglers to new cmudict and heteronyms paths

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: smajumdar <smajumdar@nvidia.com>
Signed-off-by: Virginia Adams <vadams@nvidia.com>
Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Signed-off-by: fayejf <fayejf07@gmail.com>
Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com>
Co-authored-by: Jocelyn <jocelynh@nvidia.com>
Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>
Co-authored-by: bene-ges <61418381+bene-ges@users.noreply.github.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com>
Co-authored-by: Vladimir Bataev <vbataev@nvidia.com>
piraka9011 pushed a commit to piraka9011/NeMo that referenced this pull request Aug 25, 2022
@ericharper @okuchaiev
@blisc @XuesongYang @rilango @michalivne @MaximumEntropy @nithinraok @titu1994 @vadam5 @redoctopus @fayejf @bene-ges @Zhilin123 @artbataev @piraka9011
Merge r1.11.0 main (NVIDIA#4787)
45e1823 
* NeMo Megatron doc updates

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (NVIDIA#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (NVIDIA#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (NVIDIA#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix notebook for buffered inference (NVIDIA#4703)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Prompt Learning Notebook Bug Fix (NVIDIA#4689)

* Added back dataset class list of dict input for generation in tutorial notebook

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* updated argument name for build dataset

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* add psutils to mock imports (NVIDIA#4728)

Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: ericharper <complex451@gmail.com>

* Update Aligner model and tutorial to add NGC checkpoint loading (NVIDIA#4714)

* Update Aligner model and tutorial to add NGC checkpoint loading

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix pynini install for Aligner notebook, minor formatting fix for model

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Aligner notebook formatting consistency

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] bugfix for missing configs. (NVIDIA#4725)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* docs typo fix

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* Fix pynini install in TTS tutorials (NVIDIA#4729)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix ASR notebooks (NVIDIA#4738)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Multilingual VAD model (NVIDIA#4734)

* add ngc link

Signed-off-by: fayejf <fayejf07@gmail.com>

* add tuned VAD config on ASR data

Signed-off-by: fayejf <fayejf07@gmail.com>

* yaml note

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad asr notebook with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad infer config comment

Signed-off-by: fayejf <fayejf07@gmail.com>

* fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* mvad sd config for ch109

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd readme

Signed-off-by: fayejf <fayejf07@gmail.com>

* add new mVAD model to doc

Signed-off-by: fayejf <fayejf07@gmail.com>

* style fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd tutorial with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* typo fix

Signed-off-by: fayejf <fayejf07@gmail.com>

Signed-off-by: fayejf <fayejf07@gmail.com>

* publish pretrained itn t5 model for English (NVIDIA#4748)

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>

* Updated docs and doc paths (NVIDIA#4754)

* Updated docs and doc paths

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update Multitask_Prompt_and_PTuning.ipynb

* Update README.rst

* Changed branch name to use single quotes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* fix bug relating to ddp strategy in joint intent slot classification tutorial (NVIDIA#4762)

* [TTS] updated config with a German IPA phoneme tokenizer (NVIDIA#4756)

* [TTS] added a German IPA phoneme tokenizer
* [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence.
* [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer.
* [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update r1.11 to new heteronyms list (NVIDIA#4745)

* Update configs to new heteronyms list
* Remove old heteronyms list, add alt 'merchandise' pron to CMUdict
* Update remaining references to old heteronyms list

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (NVIDIA#4763)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (NVIDIA#4770)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update CMUdict with more recent 0.7b entries (NVIDIA#4768)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Install pynini in docker container (NVIDIA#4733)

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Fix tutorial formatting (NVIDIA#4778)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* [TTS] deprecated old scripts for ljspeech. (NVIDIA#4780)

* deprecated old scripts for ljspeech.
* removed relevent function calls in TTS docs.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and requirements

Signed-off-by: ericharper <complex451@gmail.com>

* update container

Signed-off-by: ericharper <complex451@gmail.com>

* Update stragglers to new cmudict and heteronyms paths

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: smajumdar <smajumdar@nvidia.com>
Signed-off-by: Virginia Adams <vadams@nvidia.com>
Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Signed-off-by: fayejf <fayejf07@gmail.com>
Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com>
Co-authored-by: Jocelyn <jocelynh@nvidia.com>
Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>
Co-authored-by: bene-ges <61418381+bene-ges@users.noreply.github.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com>
Co-authored-by: Vladimir Bataev <vbataev@nvidia.com>
Signed-off-by: Anas Abou Allaban <aabouallaban@pm.me>
khcs added a commit that referenced this pull request Aug 26, 2022
@MaximumEntropy @ericharper
@blisc @XuesongYang @rilango @michalivne @nithinraok @khcs
Megatron Encoder Decoder models with RPE and PP > 2 (#4663)
4d3e335 
* Enc-Dec RPE PP > 2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Syle

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix rpe check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert to main

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert more files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Additional changes to make PP>2 with RPE work.

* Fixes for O2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix cross attention sync check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix zero init

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Modify .data instead of the weight

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add worker changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Set workers to 0 for validation and test

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove unused imports and allreduce func

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert workers change

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Co-authored-by: ericharper <complex451@gmail.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Hoo Chang Shin <hshin@nvidia.com>
Co-authored-by: khcs <khcs@users.noreply.github.com>
yaoyu-33 pushed a commit that referenced this pull request Aug 26, 2022
@MaximumEntropy @ericharper
@blisc @XuesongYang @rilango @michalivne @nithinraok @khcs
Megatron Encoder Decoder models with RPE and PP > 2 (#4663)
6925ece 
* Enc-Dec RPE PP > 2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Syle

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix rpe check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert to main

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert more files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Additional changes to make PP>2 with RPE work.

* Fixes for O2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix cross attention sync check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix zero init

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Modify .data instead of the weight

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add worker changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Set workers to 0 for validation and test

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove unused imports and allreduce func

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert workers change

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Co-authored-by: ericharper <complex451@gmail.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Hoo Chang Shin <hshin@nvidia.com>
Co-authored-by: khcs <khcs@users.noreply.github.com>
ericharper pushed a commit that referenced this pull request Sep 9, 2022
@XuesongYang @ericharper
[TTS] fixed wrong pronunciations for r1.11. (#4677)
ed03095 
* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
XuesongYang added a commit that referenced this pull request Sep 10, 2022
@XuesongYang
[TTS] fixed wrong pronunciations for r1.11. (#4677)
fec1fb5 
* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
jubick1337 pushed a commit to jubick1337/NeMo that referenced this pull request Oct 3, 2022
@ericharper @okuchaiev
@blisc @XuesongYang @rilango @michalivne @MaximumEntropy @nithinraok @titu1994 @vadam5 @redoctopus @fayejf @bene-ges @Zhilin123 @artbataev @jubick1337
Merge r1.11.0 main (NVIDIA#4787)
f9577b8 
* NeMo Megatron doc updates

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (NVIDIA#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (NVIDIA#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (NVIDIA#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix notebook for buffered inference (NVIDIA#4703)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Prompt Learning Notebook Bug Fix (NVIDIA#4689)

* Added back dataset class list of dict input for generation in tutorial notebook

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* updated argument name for build dataset

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* add psutils to mock imports (NVIDIA#4728)

Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: ericharper <complex451@gmail.com>

* Update Aligner model and tutorial to add NGC checkpoint loading (NVIDIA#4714)

* Update Aligner model and tutorial to add NGC checkpoint loading

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix pynini install for Aligner notebook, minor formatting fix for model

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Aligner notebook formatting consistency

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] bugfix for missing configs. (NVIDIA#4725)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* docs typo fix

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* Fix pynini install in TTS tutorials (NVIDIA#4729)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix ASR notebooks (NVIDIA#4738)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Multilingual VAD model (NVIDIA#4734)

* add ngc link

Signed-off-by: fayejf <fayejf07@gmail.com>

* add tuned VAD config on ASR data

Signed-off-by: fayejf <fayejf07@gmail.com>

* yaml note

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad asr notebook with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad infer config comment

Signed-off-by: fayejf <fayejf07@gmail.com>

* fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* mvad sd config for ch109

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd readme

Signed-off-by: fayejf <fayejf07@gmail.com>

* add new mVAD model to doc

Signed-off-by: fayejf <fayejf07@gmail.com>

* style fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd tutorial with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* typo fix

Signed-off-by: fayejf <fayejf07@gmail.com>

Signed-off-by: fayejf <fayejf07@gmail.com>

* publish pretrained itn t5 model for English (NVIDIA#4748)

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>

* Updated docs and doc paths (NVIDIA#4754)

* Updated docs and doc paths

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update Multitask_Prompt_and_PTuning.ipynb

* Update README.rst

* Changed branch name to use single quotes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* fix bug relating to ddp strategy in joint intent slot classification tutorial (NVIDIA#4762)

* [TTS] updated config with a German IPA phoneme tokenizer (NVIDIA#4756)

* [TTS] added a German IPA phoneme tokenizer
* [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence.
* [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer.
* [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update r1.11 to new heteronyms list (NVIDIA#4745)

* Update configs to new heteronyms list
* Remove old heteronyms list, add alt 'merchandise' pron to CMUdict
* Update remaining references to old heteronyms list

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (NVIDIA#4763)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (NVIDIA#4770)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update CMUdict with more recent 0.7b entries (NVIDIA#4768)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Install pynini in docker container (NVIDIA#4733)

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Fix tutorial formatting (NVIDIA#4778)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* [TTS] deprecated old scripts for ljspeech. (NVIDIA#4780)

* deprecated old scripts for ljspeech.
* removed relevent function calls in TTS docs.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and requirements

Signed-off-by: ericharper <complex451@gmail.com>

* update container

Signed-off-by: ericharper <complex451@gmail.com>

* Update stragglers to new cmudict and heteronyms paths

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: smajumdar <smajumdar@nvidia.com>
Signed-off-by: Virginia Adams <vadams@nvidia.com>
Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Signed-off-by: fayejf <fayejf07@gmail.com>
Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com>
Co-authored-by: Jocelyn <jocelynh@nvidia.com>
Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>
Co-authored-by: bene-ges <61418381+bene-ges@users.noreply.github.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com>
Co-authored-by: Vladimir Bataev <vbataev@nvidia.com>
Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>
jubick1337 pushed a commit to jubick1337/NeMo that referenced this pull request Oct 3, 2022
@MaximumEntropy @ericharper
@blisc @XuesongYang @rilango @michalivne @nithinraok @khcs @jubick1337
Megatron Encoder Decoder models with RPE and PP > 2 (NVIDIA#4663)
7a6479b 
* Enc-Dec RPE PP > 2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Syle

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (NVIDIA#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (NVIDIA#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (NVIDIA#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix rpe check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert to main

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert more files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Additional changes to make PP>2 with RPE work.

* Fixes for O2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix cross attention sync check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix zero init

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Modify .data instead of the weight

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add worker changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Set workers to 0 for validation and test

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove unused imports and allreduce func

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert workers change

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Co-authored-by: ericharper <complex451@gmail.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Hoo Chang Shin <hshin@nvidia.com>
Co-authored-by: khcs <khcs@users.noreply.github.com>
Signed-off-by: Matvei Novikov <mattyson.so@gmail.com>
MaximumEntropy added a commit that referenced this pull request Oct 15, 2022
@MaximumEntropy @ericharper
@blisc @XuesongYang @rilango @michalivne @aklife97 @okuchaiev @pre-commit-ci
Prompt learning of Huggingface T5v1.1 converted checkpoints (#4746)
bc48db3 
* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Initial

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Make megatron legacy configurable

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Enc-Dec checksum matching

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add conversion script

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset docker and jenkinsfile

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset README

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove tts scripts files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update finetuning script

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* add cloning

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* map to cpu

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* Fix TP change for HF exported models

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update conversion script and style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add arg

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Change partition comment update

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor fix for prompt learning

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix default

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix to latest ptl

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add arg to perceiver

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Temporarily add

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Restore

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Move tokens head bias to cfg population

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Empty

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes to get decode to work.

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* More changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Test

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update config to 0 dropout

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset file

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove scheduler

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Support generic bos id

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add embedding dropout

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Changes for ul2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix for pad id

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update models that can be converted

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix inference

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove ipdb

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix typo

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Load ul2 in bf16

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add amp o2 arg

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Tmp

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix rmsnorm

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix eval for converted models

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update predict step for adapters

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: ericharper <complex451@gmail.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
1-800-BAD-CODE pushed a commit to 1-800-BAD-CODE/NeMo that referenced this pull request Nov 13, 2022
@MaximumEntropy @ericharper
@blisc @XuesongYang @rilango @michalivne @aklife97 @okuchaiev @pre-commit-ci @1-800-BAD-CODE
Prompt learning of Huggingface T5v1.1 converted checkpoints (NVIDIA#4746
7144d19 
)

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Initial

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Make megatron legacy configurable

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Enc-Dec checksum matching

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add conversion script

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset docker and jenkinsfile

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset README

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove tts scripts files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update finetuning script

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* add cloning

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* map to cpu

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* Fix TP change for HF exported models

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update conversion script and style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add arg

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Change partition comment update

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor fix for prompt learning

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix default

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix to latest ptl

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add arg to perceiver

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Temporarily add

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Restore

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Move tokens head bias to cfg population

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Empty

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes to get decode to work.

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* More changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Test

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update config to 0 dropout

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset file

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove scheduler

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Support generic bos id

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add embedding dropout

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Changes for ul2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix for pad id

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update models that can be converted

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix inference

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove ipdb

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix typo

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Load ul2 in bf16

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add amp o2 arg

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Tmp

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix rmsnorm

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix eval for converted models

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update predict step for adapters

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: ericharper <complex451@gmail.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: 1-800-bad-code <shane.carroll@utsa.edu>
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
@ericharper @okuchaiev
@blisc @XuesongYang @rilango @michalivne @MaximumEntropy @nithinraok @titu1994 @vadam5 @redoctopus @fayejf @bene-ges @Zhilin123 @artbataev
Merge r1.11.0 main (NVIDIA#4787)
24f4222 
* NeMo Megatron doc updates

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (NVIDIA#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (NVIDIA#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (NVIDIA#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix notebook for buffered inference (NVIDIA#4703)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Prompt Learning Notebook Bug Fix (NVIDIA#4689)

* Added back dataset class list of dict input for generation in tutorial notebook

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* updated argument name for build dataset

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* add psutils to mock imports (NVIDIA#4728)

Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: ericharper <complex451@gmail.com>

* Update Aligner model and tutorial to add NGC checkpoint loading (NVIDIA#4714)

* Update Aligner model and tutorial to add NGC checkpoint loading

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix pynini install for Aligner notebook, minor formatting fix for model

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Aligner notebook formatting consistency

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] bugfix for missing configs. (NVIDIA#4725)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* docs typo fix

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* Fix pynini install in TTS tutorials (NVIDIA#4729)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix ASR notebooks (NVIDIA#4738)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Multilingual VAD model (NVIDIA#4734)

* add ngc link

Signed-off-by: fayejf <fayejf07@gmail.com>

* add tuned VAD config on ASR data

Signed-off-by: fayejf <fayejf07@gmail.com>

* yaml note

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad asr notebook with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad infer config comment

Signed-off-by: fayejf <fayejf07@gmail.com>

* fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* mvad sd config for ch109

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd readme

Signed-off-by: fayejf <fayejf07@gmail.com>

* add new mVAD model to doc

Signed-off-by: fayejf <fayejf07@gmail.com>

* style fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd tutorial with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* typo fix

Signed-off-by: fayejf <fayejf07@gmail.com>

Signed-off-by: fayejf <fayejf07@gmail.com>

* publish pretrained itn t5 model for English (NVIDIA#4748)

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>

* Updated docs and doc paths (NVIDIA#4754)

* Updated docs and doc paths

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update Multitask_Prompt_and_PTuning.ipynb

* Update README.rst

* Changed branch name to use single quotes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* fix bug relating to ddp strategy in joint intent slot classification tutorial (NVIDIA#4762)

* [TTS] updated config with a German IPA phoneme tokenizer (NVIDIA#4756)

* [TTS] added a German IPA phoneme tokenizer
* [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence.
* [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer.
* [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update r1.11 to new heteronyms list (NVIDIA#4745)

* Update configs to new heteronyms list
* Remove old heteronyms list, add alt 'merchandise' pron to CMUdict
* Update remaining references to old heteronyms list

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (NVIDIA#4763)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (NVIDIA#4770)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update CMUdict with more recent 0.7b entries (NVIDIA#4768)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Install pynini in docker container (NVIDIA#4733)

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Fix tutorial formatting (NVIDIA#4778)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* [TTS] deprecated old scripts for ljspeech. (NVIDIA#4780)

* deprecated old scripts for ljspeech.
* removed relevent function calls in TTS docs.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and requirements

Signed-off-by: ericharper <complex451@gmail.com>

* update container

Signed-off-by: ericharper <complex451@gmail.com>

* Update stragglers to new cmudict and heteronyms paths

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: smajumdar <smajumdar@nvidia.com>
Signed-off-by: Virginia Adams <vadams@nvidia.com>
Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Signed-off-by: fayejf <fayejf07@gmail.com>
Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com>
Co-authored-by: Jocelyn <jocelynh@nvidia.com>
Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>
Co-authored-by: bene-ges <61418381+bene-ges@users.noreply.github.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com>
Co-authored-by: Vladimir Bataev <vbataev@nvidia.com>
Signed-off-by: Hainan Xu <hainanx@nvidia.com>
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
@MaximumEntropy @ericharper
@blisc @XuesongYang @rilango @michalivne @nithinraok @khcs
Megatron Encoder Decoder models with RPE and PP > 2 (NVIDIA#4663)
53d6310 
* Enc-Dec RPE PP > 2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Syle

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (NVIDIA#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (NVIDIA#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (NVIDIA#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix rpe check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert to main

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert more files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Additional changes to make PP>2 with RPE work.

* Fixes for O2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix cross attention sync check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix zero init

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Modify .data instead of the weight

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add worker changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Set workers to 0 for validation and test

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove unused imports and allreduce func

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert workers change

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Co-authored-by: ericharper <complex451@gmail.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Hoo Chang Shin <hshin@nvidia.com>
Co-authored-by: khcs <khcs@users.noreply.github.com>
Signed-off-by: Hainan Xu <hainanx@nvidia.com>
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
@MaximumEntropy @ericharper
@blisc @XuesongYang @rilango @michalivne @aklife97 @okuchaiev @pre-commit-ci
Prompt learning of Huggingface T5v1.1 converted checkpoints (NVIDIA#4746
d13c77e 
)

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Initial

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Make megatron legacy configurable

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Enc-Dec checksum matching

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add conversion script

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset docker and jenkinsfile

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset README

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove tts scripts files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update finetuning script

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* add cloning

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* map to cpu

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* Fix TP change for HF exported models

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update conversion script and style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add arg

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Change partition comment update

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor fix for prompt learning

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix default

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix to latest ptl

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add arg to perceiver

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Temporarily add

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Restore

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Move tokens head bias to cfg population

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Empty

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes to get decode to work.

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* More changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Test

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update config to 0 dropout

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset file

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove scheduler

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Support generic bos id

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add embedding dropout

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Changes for ul2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix for pad id

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update models that can be converted

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix inference

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove ipdb

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix typo

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Load ul2 in bf16

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add amp o2 arg

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Tmp

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix rmsnorm

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix eval for converted models

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update predict step for adapters

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: ericharper <complex451@gmail.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Hainan Xu <hainanx@nvidia.com>
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
@ericharper @okuchaiev
@blisc @XuesongYang @rilango @michalivne @MaximumEntropy @nithinraok @titu1994 @vadam5 @redoctopus @fayejf @bene-ges @Zhilin123 @artbataev
Merge r1.11.0 main (NVIDIA#4787)
d167953 
* NeMo Megatron doc updates

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (NVIDIA#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (NVIDIA#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (NVIDIA#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix notebook for buffered inference (NVIDIA#4703)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Prompt Learning Notebook Bug Fix (NVIDIA#4689)

* Added back dataset class list of dict input for generation in tutorial notebook

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* updated argument name for build dataset

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* add psutils to mock imports (NVIDIA#4728)

Signed-off-by: ericharper <complex451@gmail.com>

Signed-off-by: ericharper <complex451@gmail.com>

* Update Aligner model and tutorial to add NGC checkpoint loading (NVIDIA#4714)

* Update Aligner model and tutorial to add NGC checkpoint loading

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix pynini install for Aligner notebook, minor formatting fix for model

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Aligner notebook formatting consistency

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] bugfix for missing configs. (NVIDIA#4725)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* docs typo fix

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>

* Fix pynini install in TTS tutorials (NVIDIA#4729)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* Fix ASR notebooks (NVIDIA#4738)

Signed-off-by: smajumdar <smajumdar@nvidia.com>

Signed-off-by: smajumdar <smajumdar@nvidia.com>

* Multilingual VAD model (NVIDIA#4734)

* add ngc link

Signed-off-by: fayejf <fayejf07@gmail.com>

* add tuned VAD config on ASR data

Signed-off-by: fayejf <fayejf07@gmail.com>

* yaml note

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad asr notebook with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* update vad infer config comment

Signed-off-by: fayejf <fayejf07@gmail.com>

* fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* mvad sd config for ch109

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd readme

Signed-off-by: fayejf <fayejf07@gmail.com>

* add new mVAD model to doc

Signed-off-by: fayejf <fayejf07@gmail.com>

* style fix

Signed-off-by: fayejf <fayejf07@gmail.com>

* update sd tutorial with mVAD

Signed-off-by: fayejf <fayejf07@gmail.com>

* typo fix

Signed-off-by: fayejf <fayejf07@gmail.com>

Signed-off-by: fayejf <fayejf07@gmail.com>

* publish pretrained itn t5 model for English (NVIDIA#4748)

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>

Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>

* Updated docs and doc paths (NVIDIA#4754)

* Updated docs and doc paths

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* Update Multitask_Prompt_and_PTuning.ipynb

* Update README.rst

* Changed branch name to use single quotes

Signed-off-by: Virginia Adams <vadams@nvidia.com>

Signed-off-by: Virginia Adams <vadams@nvidia.com>

* fix bug relating to ddp strategy in joint intent slot classification tutorial (NVIDIA#4762)

* [TTS] updated config with a German IPA phoneme tokenizer (NVIDIA#4756)

* [TTS] added a German IPA phoneme tokenizer
* [TTS][ASR] enabled customized arguments for trimming the leading and trailing silence.
* [TTS] disabled spline interpolation for beta-binomial distribution. Let it generate align prior and save to disks. Use a new phoneme tokenizer.
* [TTS] use consistent spline interpolation with fastpitch checkpoint when generating mel-spectrograms for hifigan finetune.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update r1.11 to new heteronyms list (NVIDIA#4745)

* Update configs to new heteronyms list
* Remove old heteronyms list, add alt 'merchandise' pron to CMUdict
* Update remaining references to old heteronyms list

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add multi-speaker German FastPitch and HiFiGAN NGC checkpoints (NVIDIA#4763)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* [TTS] Add single male speaker German FastPitch and HiFiGAN NGC checkpoints (NVIDIA#4770)

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Update CMUdict with more recent 0.7b entries (NVIDIA#4768)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Install pynini in docker container (NVIDIA#4733)

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>

Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Fix tutorial formatting (NVIDIA#4778)

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

* [TTS] deprecated old scripts for ljspeech. (NVIDIA#4780)

* deprecated old scripts for ljspeech.
* removed relevent function calls in TTS docs.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and requirements

Signed-off-by: ericharper <complex451@gmail.com>

* update container

Signed-off-by: ericharper <complex451@gmail.com>

* Update stragglers to new cmudict and heteronyms paths

Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>

Signed-off-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Signed-off-by: smajumdar <smajumdar@nvidia.com>
Signed-off-by: Virginia Adams <vadams@nvidia.com>
Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Signed-off-by: fayejf <fayejf07@gmail.com>
Signed-off-by: Alexandra Antonova <aleksandraa@nvidia.com>
Signed-off-by: Vladimir Bataev <vbataev@nvidia.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@nvidia.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Somshubra Majumdar <titu1994@gmail.com>
Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com>
Co-authored-by: Jocelyn <jocelynh@nvidia.com>
Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com>
Co-authored-by: bene-ges <61418381+bene-ges@users.noreply.github.com>
Co-authored-by: Alexandra Antonova <aleksandraa@nvidia.com>
Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com>
Co-authored-by: Vladimir Bataev <vbataev@nvidia.com>
Signed-off-by: Hainan Xu <hainanx@nvidia.com>
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
@MaximumEntropy @ericharper
@blisc @XuesongYang @rilango @michalivne @nithinraok @khcs
Megatron Encoder Decoder models with RPE and PP > 2 (NVIDIA#4663)
7feee89 
* Enc-Dec RPE PP > 2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Syle

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* [TTS] incremented the version number to 22.08 in tutorials. (NVIDIA#4684)

* [TTS] incremented the version number to 22.08 in tutorials.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Megatron encode function with RPE fix (NVIDIA#4692)

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* fix to fetch config file (NVIDIA#4699)

Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix rpe check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert to main

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert more files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Additional changes to make PP>2 with RPE work.

* Fixes for O2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix cross attention sync check

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix zero init

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Modify .data instead of the weight

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add worker changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Set workers to 0 for validation and test

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove unused imports and allreduce func

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Revert workers change

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: nithinraok <nithinrao.koluguri@gmail.com>
Co-authored-by: ericharper <complex451@gmail.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Nithin Rao <nithinrao.koluguri@gmail.com>
Co-authored-by: Hoo Chang Shin <hshin@nvidia.com>
Co-authored-by: khcs <khcs@users.noreply.github.com>
Signed-off-by: Hainan Xu <hainanx@nvidia.com>
hainan-xv pushed a commit to hainan-xv/NeMo that referenced this pull request Nov 29, 2022
@MaximumEntropy @ericharper
@blisc @XuesongYang @rilango @michalivne @aklife97 @okuchaiev @pre-commit-ci
Prompt learning of Huggingface T5v1.1 converted checkpoints (NVIDIA#4746
5b46fe6 
)

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Initial

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix for RPE

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* update branch

Signed-off-by: ericharper <complex451@gmail.com>

* update package info and dockerfile

Signed-off-by: ericharper <complex451@gmail.com>

* fix fastpitch export (NVIDIA#4676)

Signed-off-by: Jason <jasoli@nvidia.com>

* [TTS] fixed wrong pronunciations for r1.11. (NVIDIA#4677)

* [TTS] fixed wrong pronunciations.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* incremented the version number to 22.08 as @blisc suggested.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* correct cmudict versions in world-wide places.

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix for incorrect batch size issue while decoding (NVIDIA#4675)

Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Eric Harper <complex451@gmail.com>

* Make megatron legacy configurable

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Enc-Dec checksum matching

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add conversion script

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset docker and jenkinsfile

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset README

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove tts scripts files

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update finetuning script

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* add cloning

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* map to cpu

Signed-off-by: Abhinav Khattar <aklife97@gmail.com>

* Fix TP change for HF exported models

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update conversion script and style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add arg

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Change partition comment update

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor fix for prompt learning

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix default

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix to latest ptl

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add arg to perceiver

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Temporarily add

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Restore

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Move tokens head bias to cfg population

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Empty

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes to get decode to work.

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* More changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update base config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Test

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update config to 0 dropout

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset file

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove scheduler

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Support generic bos id

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Minor changes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add embedding dropout

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Changes for ul2

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix for pad id

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update models that can be converted

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix inference

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Remove ipdb

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix typo

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Load ul2 in bf16

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Add amp o2 arg

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Tmp

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix rmsnorm

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Reset config

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix eval for converted models

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fix

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Style

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Update predict step for adapters

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Minor

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* Fixes

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

Signed-off-by: ericharper <complex451@gmail.com>
Signed-off-by: Jason <jasoli@nvidia.com>
Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: ericharper <complex451@gmail.com>
Co-authored-by: Jason <jasoli@nvidia.com>
Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>
Co-authored-by: Rajesh Ilango <rilango@gmail.com>
Co-authored-by: Micha Livne <michalivne@users.noreply.github.com>
Co-authored-by: Abhinav Khattar <aklife97@gmail.com>
Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: Hainan Xu <hainanx@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@blisc blisc blisc approved these changes

@redoctopus redoctopus Awaiting requested review from redoctopus

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@XuesongYang @blisc