NVIDIA Neural Modules 1.10.0

Container

For additional information regarding NeMo containers, please visit: https://catalog.ngc.nvidia.com/orgs/nvidia/containers/nemo

docker pull nvcr.io/nvidia/nemo:22.05

Known Issues

Issues

• Tutorial: Fastpitch_Training_GermanTTS.ipynb is experimental and still being tested.

ASR

Changelog

• Multilang asr tutorial by @bmwshop :: PR: #3931
• Add ASR with Adapters Tutorial by @titu1994 :: PR: #4149
• Add support for Decoder + Joint Adapters for ASR by @titu1994 :: PR: #4189
• updating PretrainedModelInfo and benchmark sheet for ASR models by @krishnacpuvvada :: PR: #4259
• Remove verbose flag from Dali Index Creator by @titu1994 :: PR: #4309
• updating PretrainedModelInfo for ASR SSL models by @krishnacpuvvada :: PR: #4292
• Adding docs for ASR SSL by @krishnacpuvvada :: PR: #4303
• Add ASR Scores to Docs by @titu1994 :: PR: #4412
• [ASR] Replace all paths with /content/ by @titu1994 :: PR: #4427
• added conformer mandarin model. by @VahidooX :: PR: #4201
• Runtime audio segment sampling for SSL by @krishnacpuvvada :: PR: #4126

TTS

Changelog

• [TTS] Add volume passthrough to fp for riva by @blisc :: PR: #4167
• Update TTS Configs from LAMB to AdamW by @redoctopus :: PR: #4233
• Add benchmark=false to all TTS configs by @redoctopus :: PR: #4263
• [TTS] add staticmethod decoration for BetaBinomialInterpolator by @XuesongYang :: PR: #4319
• [TTS] capture exception of non-supported windows. by @XuesongYang :: PR: #4320
• [TTS] enforced pin_memory = True by @XuesongYang :: PR: #4341
• [TTS] Training Fastpitch on German text and phonemes and finetuning HiFi-GAN on predicted mels by @aroraakshit :: PR: #4266
• IPA support for TTS by @redoctopus :: PR: #4310
• Bits of RADTTS support by @borisfom :: PR: #4343

NLP / NMT

Changelog

• Megatron NMT Restore from T5/BART and finetune by @MaximumEntropy :: PR: #3977
• Binarized memmap dataloader for Megatron NMT, Inference and checkpoint -> nemo by @MaximumEntropy :: PR: #4137
• Use unique names for temporary directories in punctuation and capitalization tests by @PeganovAnton :: PR: #4298
• Removes debug logging statements in Megatron NMT by @MaximumEntropy :: PR: #4312
• Raise error if trainer object is None for MegatronBaseModel by @MaximumEntropy :: PR: #4356
• Punctuation and capitalization tests race condition by @PeganovAnton :: PR: #4399
• unify intent slot dataset util functions in tutorials by @Zhilin123 :: PR: #4445
• Fix for TP=2,PP=2 decoding with megatron encoder-decoder models by @MaximumEntropy :: PR: #4484
• Add RETRO model for pretraining by @yidong72 :: PR: #4121
• Add async grad allreduce and chunk optimization by @xrennvidia :: PR: #4084
• Implements the UL2 Dataset and config by @MaximumEntropy :: PR: #4184
• Add RETRO indexed dataset and inference by @yidong72 :: PR: #4220
• Finetune T5 on the prefix-lm objective by @MaximumEntropy :: PR: #4328
• Fuse bias with geglu in ParallelMLP by @xrennvidia :: PR: #4213
• Support larger datasets for question answering by @Zhilin123 :: PR: #4205
• Refactor bias act fusion by @MaximumEntropy :: PR: #4376
• Prompt Learning Pipeline Parallel by @vadam5 :: PR: #4291
• Text memmap dataset by @michalivne :: PR: #4068
• Fuse grad division into async grad allreduce by @xrennvidia :: PR: #4327

Text Normalization / Inverse Text Normalization

Changelog

• [TN] WFST to normalize punctuation by @ekmb :: PR: #4108
• [TN/TTS] Add graph to tag IPA words/sentences in square brackets and leave them unchanged by @ekmb :: PR: #4323
• Tn tutorial by @yzhang123 :: PR: #4090
• [TN] WFST to normalize punctuation by @ekmb :: PR: #4108
• Tn add rules by @yzhang123 :: PR: #4302
• [TN/TTS] Add graph to tag IPA words/sentences in square brackets and leave them unchanged by @ekmb :: PR: #4323
• Tn install by @yzhang123 :: PR: #4055
• Fix electronic bug, new time ITN rule by @ekmb :: PR: #4355
• [TN] Bug fix: expand serial coverage of unknown symbol, remove constraints from word graph by @ekmb :: PR: #4463
• Configure T5 finetuning metrics by @MaximumEntropy :: PR: #4122

Export

Changelog

• Added support for subnet export by @borisfom :: PR: #4299

Core

Changelog

• Add Module-level Adapters, Save-Restore and tests by @titu1994 :: PR: #4114
• Add NeMo Adapters tutorial to Core by @titu1994 :: PR: #4311
• NeMo Model to HF Hub Upload Tutorial by @titu1994 :: PR: #4322

General Improvements and Fixes

Changelog

• Update container to 22.05 by @ericharper :: PR: #4329
• Fix PTL step calculation by @titu1994 :: PR: #4307
• [NLP] P&C Fix multi node cache issue, add pynini guard by @ekmb :: PR: #4410
• NeMo Megatron GPT Unit Tests by @ericharper :: PR: #4099
• Add the PP2 GPT eval CI test by @yidong72 :: PR: #4168
• BigNLP perf regression fix by @MaximumEntropy :: PR: #4267
• Fixes for Megatron Base Model Artifacts by @MaximumEntropy :: PR: #4248
• Fix a wrong description in offline_diarization_with_asr.yaml by @tango4j :: PR: #4141
• bugfix for import error in Offline_ASR_with_VAD_for_CTC_models by @fayejf :: PR: #4424
• [Fix] ASR RNNT Tutorial by @stevehuang52 :: PR: #4352
• [TTS] Fix Hifigan finetune tutorial by @subhankar-ghosh :: PR: #4182
• [Bugfix][TTS] wrong order of returned tuple for general_collate_fn. by @XuesongYang :: PR: #4432
• [bugfix][TTS] pitch, voiced_mask, prob_voiced have the same values. by @XuesongYang :: PR: #4435
• [TTS] [bugfix] German FastPitch HiFi-GAN tutorial and lr by @aroraakshit :: PR: #4459
• [TTS] [bugfix] update indentation by @aroraakshit :: PR: #4468
• Fix some 's' cases for IPA G2P by @redoctopus :: PR: #4460
• Fix ASR Typos in tutorials by @titu1994 :: PR: #4384
• Use unique names for temporary directories in punctuation and capitalization tests by @PeganovAnton :: PR: #4298
• Punctuation and capitalization tests race condition by @PeganovAnton :: PR: #4399
• Dialogue tasks unit test by @Zhilin123 :: PR: #4112
• fix error by @yzhang123 :: PR: #4120
• fix typo by @stevehuang52 :: PR: #4134
• Fix cmudict typo: phoneme YI1 -> IY1 in NVME by @redoctopus :: PR: #4139
• transcribe: scan directories recursively by @virajkarandikar :: PR: #4159
• Add 44KHz yaml file for Fastpitch training by @subhankar-ghosh :: PR: #4161
• [bugfix] consistent highfreq to both fastpitch and hifigan in their 44100 configs. by @XuesongYang :: PR: #4177
• Upperbound OmegaConf by @titu1994 :: PR: #4191
• Prompt tokenization bugfix by @vadam5 :: PR: #4197
• Updated to Prompt Learning Model to Use Distributed Sampler by @vadam5 :: PR: #4208
• Freesound fixes by @virajkarandikar :: PR: #4155
• Patch Hydra by @titu1994 :: PR: #4202
• Prompt Learning Model Saving Changes by @vadam5 :: PR: #4212
• Speakertasks manifest by @yzhang123 :: PR: #4185
• SSL Multi-loss Update by @sam1373 :: PR: #4186
• Support load_adapters with just adapter_name by @titu1994 :: PR: #4255
• Add special tokens to existing (trained) SentencePiece models by @aklife97 :: PR: #4203
• Fixing the speed slow-down for speech models. by @VahidooX :: PR: #4260
• Fix and add functions in speaker utils by @tango4j :: PR: #4138
• pt container 1.10->1.11.0 by @ekmb :: PR: #4273
• ssl fixes by @sam1373 :: PR: #4268
• Save Virtual Prompt Weights Only by @vadam5 :: PR: #4237
• add 'relative positional embedding (RPE)' feature - re-creating after… by @khcs :: PR: #4256
• Docs CSS: Update h4 tag style for the right side bar by @nickolyamba :: PR: #4284
• Fix Docs CSS: align docs left and increase width for large screens by @nickolyamba :: PR: #4154
• remove redundant condition for fastpitch. by @XuesongYang :: PR: #4281
• [Add] automaticly resolving relative audio path by @stevehuang52 :: PR: #4277
• forcing conv subsampling to 32 bit by @bmwshop :: PR: #4293
• Add library name and version when downloading from the Hugging Face Hub by @osanseviero :: PR: #4304
• clear access registry when adding if not empty by @sam1373 :: PR: #4306
• [collections] bugfix for capturing NotImplementedError of non-supported sup data types. by @XuesongYang :: PR: #4297
• Adjust lr for AdamW from LAMB default by @redoctopus :: PR: #4308
• Fix bugs in indexed dataset exam script by @yidong72 :: PR: #4325
• Torchaudio installation fix by @GNroy :: PR: #4330
• Speedup the speech commands dataset processing script by @shan18 :: PR: #4347
• fix wrong requirement by @yzhang123 :: PR: #4349
• Refactored path to manifest by @treacker :: PR: #4251
• Fix the post LN bug by @yidong72 :: PR: #4350
• [Fix] Hanging for Fully Randomized Bucketing by @stevehuang52 :: PR: #4348
• Auto-switch the input dimensions in the conformer encoder adapter to correct value by @shan18 :: PR: #4354
• Set headscale false by @MaximumEntropy :: PR: #4364
• Add wandb as dependency by @titu1994 :: PR: #4365
• Fix trainer.global_steps in WandB logging by @titu1994 :: PR: #4366
• Finetuning changes for BART by @MaximumEntropy :: PR: #4003
• Make position embedding expansion specific to a batch to avoid checkpoint size mismatches by @MaximumEntropy :: PR: #4357
• Correct support for dataclasses in default module dim by @titu1994 :: PR: #4372
• Fix no attribute 'pad_id' bug when pre-processing by @yidong72 :: PR: #4377
• Question answering bug fix by @Zhilin123 :: PR: #4381
• Docs for NeMo Adapters by @titu1994 :: PR: #4369
• Update NeMo docs by @titu1994 :: PR: #4397
• Fixing import error in some cases by @borisfom :: PR: #4402
• Fix tutorial typos and docs by @titu1994 :: PR: #4415
• Add reconfigure on validation epoch start by @MaximumEntropy :: PR: #4393
• Re-apply fixes from r1.9.0 by @redoctopus :: PR: #4425
• Fix hanging issue by multiprocessing in SD tutorial and add ETA for VAD processing by @fayejf :: PR: #4405
• Fix notebook text by @yidong72 :: PR: #4438
• Update dialogue tutorial version by @Zhilin123 :: PR: #4437
• Docs: Add table overflow handling by @nickolyamba :: PR: #4441
• Docs: Decrease Font Size on Tables by @nickolyamba :: PR: #4444
• Notebook bug fix: add subfolder by @ekmb :: PR: #4442
• Fix typo in HiFi-GAN config's max steps by @redoctopus :: PR: #4446
• Updated notebook to fix batch configuration and precision bugs by @vadam5 :: PR: #4447
• fix branch in link by @ekmb :: PR: #4454
• t5-rpe-fix targeting r1.10.0; raise exception for PP>2. by @khcs :: PR: #4469
• Add kwargs to exact string match by @MaximumEntropy :: PR: #4479