Standalone diarization+ASR evaluation script #5439

tango4j · 2022-11-16T21:28:18Z

What does this PR do ?

Add a standalone diarization-ASR evaluation transcript.
This script was build at the request for testing ASR+diarization with already extracted files.

Adding diar_infer_general.yaml (VAD optimized for multilingual ASR dataset, diarization optimized on DIHARD dev set)

Collection: [Note which collection this PR will affect]

ASR

Changelog

Add specific line by line info of high level changes in this PR.

Usage

You can potentially add a usage example below

python eval_diar_with_asr.py \
 --hyp_rttm_list="/path/to/hypothesis_rttm_filepaths.list" \
 --ref_rttm_list="/path/to/reference_rttm_filepaths.list" \
 --ref_ctm_list="/path/to/reference_ctm_filepaths.list" \
 --hyp_ctm_list="/path/to/hypothesis_ctm_filepaths.list" \
 --root_path="/path/to/output/directory"

Before your PR is "Ready for review"

Pre checks:

Make sure you read and followed Contributor guidelines
Did you write any new necessary tests?
Did you add or update any necessary documentation?
Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
- Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

New Feature
Bugfix
Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone belongs to ASR

Additional Information

Related to # (issue)

Signed-off-by: Taejin Park <tango4j@gmail.com>

for more information, see https://pre-commit.ci

lgtm-com · 2022-11-16T21:43:26Z

This pull request introduces 1 alert when merging aea2e6c into 2ecfb7a - view on LGTM.com

new alerts:

1 for Wrong number of arguments in a call

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. Please enable GitHub code scanning, which uses the same CodeQL engine ⚙️ that powers LGTM.com. For more information, please check out our post on the GitHub blog.

Signed-off-by: Taejin Park <tango4j@gmail.com>

for more information, see https://pre-commit.ci

Signed-off-by: Taejin Park <tango4j@gmail.com>

… into mulspk_asr_eval_script

Signed-off-by: Taejin Park <tango4j@gmail.com>

… into mulspk_asr_eval_script

Signed-off-by: Taejin Park <tango4j@gmail.com>

for more information, see https://pre-commit.ci

* clean Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * kwarg ref Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * extra args Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * rm prints Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * style Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>

* change for concat map dataset * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Exhaust longest dataset * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: 1-800-BAD-CODE <> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>

Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com>

* Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com>

* Fixing bug when loss mask is fully zero Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update megatron_bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>

…5236) Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com>

…n MegatronGPT (#5363) * support to disable sequence length + 1 input tokens for MegatronGPT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: Anmol Gupta <anmolg@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca>

* Create script for processing TTS training audio * Update VAD trimming logic * Remove unused import Signed-off-by: Ryan <rlangman@nvidia.com>

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

* Fix tests Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Add accidentally lost changes Signed-off-by: PeganovAnton <peganoff2@mail.ru> Signed-off-by: PeganovAnton <peganoff2@mail.ru>

Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com>

…rred dataset (#5442) Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com>

adding diar_infer_general.yaml Signed-off-by: Taejin Park <tango4j@gmail.com> Update codeql.yml Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Update codeql.yml Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> fix msdd_model in general yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> fixed errors in yaml file Signed-off-by: Taejin Park <tango4j@gmail.com>

Signed-off-by: Taejin Park <tango4j@gmail.com>

nithinraok · 2022-11-18T02:21:45Z

examples/speaker_tasks/diarization/conf/inference/diar_infer_general.yaml

+      filter_speech_first: True 
+
+  speaker_embeddings:
+    model_path: titanet_large # .nemo local model path or pretrained model name (titanet_large, ecapa_tdnn or speakerverification_speakernet)


Is it titanet_large or titanet_small ?

If titanet_small is published, we should change this right away

nithinraok · 2022-11-18T02:23:09Z

nemo/collections/asr/parts/utils/diarization_utils.py

+    return total_result_jsons
+
+
+def get_audacity_label(word: str, stt_sec: float, end_sec: float, speaker: str) -> str:


Is this moved from speaker_utils? Why?

synced offline (because of @staticmethod functions we cannot put it in the class)

nithinraok · 2022-11-18T02:43:36Z

scripts/speaker_tasks/eval_diar_with_asr.py

+def evaluate_der(audio_rttm_map_dict, all_reference, all_hypothesis, diar_eval_mode):
+    """Evaluate with a selected diarization evaluation scheme
+    """
+    eval_settings = []
+    if diar_eval_mode == "full":
+        eval_settings = [(0.0, False)]
+    elif diar_eval_mode == "fair":
+        eval_settings = [(0.25, False)]
+    elif diar_eval_mode == "forgiving":
+        eval_settings = [(0.25, True)]
+    elif diar_eval_mode == "all":
+        eval_settings = [(0.0, False), (0.25, False), (0.25, True)]
+    else:
+        raise ValueError("`diar_eval_mode` variable contains an unsupported value")
+
+    for collar, ignore_overlap in eval_settings:
+        diar_score = score_labels(
+            AUDIO_RTTM_MAP=audio_rttm_map_dict,
+            all_reference=all_reference,
+            all_hypothesis=all_hypothesis,
+            collar=collar,
+            ignore_overlap=ignore_overlap,
+        )
+    return diar_score


shall we keep this part of speaker_utils?

moved to der.py

Signed-off-by: Taejin Park <tango4j@gmail.com>

… into mulspk_asr_eval_script

for more information, see https://pre-commit.ci

Signed-off-by: Taejin Park <tango4j@gmail.com>

… into mulspk_asr_eval_script

fayejf · 2022-11-18T22:38:25Z

examples/speaker_tasks/diarization/conf/inference/diar_infer_general.yaml

+  collar: 0.25 # Collar value for scoring
+  ignore_overlap: True # Consider or ignore overlap segments while scoring
+
+  vad:


This part looks good to me.

thanks for checking

lgtm-com · 2022-11-18T22:46:40Z

This pull request introduces 1 alert when merging e2d519d into bccf6d5 - view on LGTM.com

new alerts:

1 for Unused import

Heads-up: LGTM.com's PR analysis will be disabled on the 5th of December, and LGTM.com will be shut down ⏻ completely on the 16th of December 2022. It looks like GitHub code scanning with CodeQL is already set up for this repo, so no further action is needed 🚀. For more information, please check out our post on the GitHub blog.

scripts/speaker_tasks/eval_diar_with_asr.py

Signed-off-by: Taejin Park <tango4j@gmail.com>

* first commit on eval_diar_with_asr.py Signed-off-by: Taejin Park <tango4j@gmail.com> * Add a standalone diarization-ASR evaluation transcript Signed-off-by: Taejin Park <tango4j@gmail.com> * Fixed examples in docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed staticmethod error Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added description on eval modes Signed-off-by: Taejin Park <tango4j@gmail.com> * adding diar_infer_general.yaml Signed-off-by: Taejin Park <tango4j@gmail.com> * fix msdd_model in general yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed errors in yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> * combine into 1 commit Signed-off-by: Taejin Park <tango4j@gmail.com> * Added description on eval modes Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add MoE support for T5 model (w/o expert parallel) (NVIDIA#5409) * clean Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * kwarg ref Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * extra args Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * rm prints Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * style Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix args (NVIDIA#5410) (NVIDIA#5416) Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Fix for concat map dataset (NVIDIA#5133) * change for concat map dataset * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Exhaust longest dataset * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: 1-800-BAD-CODE <> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Add temporary fix for CUDA issue in Dockerfile (NVIDIA#5421) (NVIDIA#5422) Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * Fix GPT generation when using sentencepiece tokenizer (NVIDIA#5413) (NVIDIA#5428) * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Support for finetuning and finetuning inference with .ckpt files & batch size refactoring (NVIDIA#5339) * Initial refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Resolve config before passing to load_from_checkpoint Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for model parallel and nemo restore Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert config changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Minor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix validation reconfiguration Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove old comment Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes for test_ds Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Revert "Add temporary fix for CUDA issue in Dockerfile (NVIDIA#5421)" (NVIDIA#5431) (NVIDIA#5432) This reverts commit 0718b17. Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * [ITN] fix year date graph, cardinals extension for hundreds (NVIDIA#5435) * wip Signed-off-by: ekmb <ebakhturina@nvidia.com> * add lociko's hundreds extension for cardinals Signed-off-by: ekmb <ebakhturina@nvidia.com> * add optional end Signed-off-by: ekmb <ebakhturina@nvidia.com> * restart ci Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * update doc in terms of get_label for lang id model (NVIDIA#5366) * reflect PR 5278 ion doc Signed-off-by: fayejf <fayejf07@gmail.com> * reflect comment Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: fayejf <fayejf07@gmail.com> * Revert workaround for T5 that sets number of workers to 0 & sync_batch_comm=False (NVIDIA#5420) (NVIDIA#5433) * Revert workers workaround Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix in config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Fixed bug in notebook (NVIDIA#5382) (NVIDIA#5394) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * Fixing bug in Megatron BERT when loss mask is all zeros (NVIDIA#5424) * Fixing bug when loss mask is fully zero Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update megatron_bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Use updated API for overlapping grad sync with pipeline parallelism (NVIDIA#5236) Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> * support to disable sequence length + 1 input tokens for each sample in MegatronGPT (NVIDIA#5363) * support to disable sequence length + 1 input tokens for MegatronGPT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: Anmol Gupta <anmolg@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * [TTS] Create script for processing TTS training audio (NVIDIA#5262) * Create script for processing TTS training audio * Update VAD trimming logic * Remove unused import Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] remove useless logic for set_tokenizer. (NVIDIA#5430) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix setting up of `ReduceLROnPlateau` learning rate scheduler (NVIDIA#5444) * Fix tests Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Add accidentally lost changes Signed-off-by: PeganovAnton <peganoff2@mail.ru> Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Create codeql.yml (NVIDIA#5445) Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> * Fix for getting tokenizer in character-based ASR models when using tarred dataset (NVIDIA#5442) Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> * Combine 5 commits adding diar_infer_general.yaml Signed-off-by: Taejin Park <tango4j@gmail.com> Update codeql.yml Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Update codeql.yml Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> fix msdd_model in general yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> fixed errors in yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> * moved eval_der function and fixed tqdm options Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Changed minor error in docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * removed score_labels and changed leave=True Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: PeganovAnton <peganoff2@mail.ru> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Shane Carroll <50530592+1-800-BAD-CODE@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Co-authored-by: anmolgupt <14880251+anmolgupt@users.noreply.github.com> Co-authored-by: Anmol Gupta <anmolg@nvidia.com> Co-authored-by: Ryan Langman <rlangman@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: PeganovAnton <peganoff2@mail.ru> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Jonghwan Hyeon <jonghwanhyeon93@gmail.com> Signed-off-by: shane carroll <shane.carroll@utsa.edu>

* first commit on eval_diar_with_asr.py Signed-off-by: Taejin Park <tango4j@gmail.com> * Add a standalone diarization-ASR evaluation transcript Signed-off-by: Taejin Park <tango4j@gmail.com> * Fixed examples in docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed staticmethod error Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added description on eval modes Signed-off-by: Taejin Park <tango4j@gmail.com> * adding diar_infer_general.yaml Signed-off-by: Taejin Park <tango4j@gmail.com> * fix msdd_model in general yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed errors in yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> * combine into 1 commit Signed-off-by: Taejin Park <tango4j@gmail.com> * Added description on eval modes Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add MoE support for T5 model (w/o expert parallel) (NVIDIA#5409) * clean Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * kwarg ref Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * extra args Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * rm prints Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * style Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix args (NVIDIA#5410) (NVIDIA#5416) Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Fix for concat map dataset (NVIDIA#5133) * change for concat map dataset * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Exhaust longest dataset * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: 1-800-BAD-CODE <> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Add temporary fix for CUDA issue in Dockerfile (NVIDIA#5421) (NVIDIA#5422) Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * Fix GPT generation when using sentencepiece tokenizer (NVIDIA#5413) (NVIDIA#5428) * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Support for finetuning and finetuning inference with .ckpt files & batch size refactoring (NVIDIA#5339) * Initial refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Resolve config before passing to load_from_checkpoint Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for model parallel and nemo restore Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert config changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Minor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix validation reconfiguration Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove old comment Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes for test_ds Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Revert "Add temporary fix for CUDA issue in Dockerfile (NVIDIA#5421)" (NVIDIA#5431) (NVIDIA#5432) This reverts commit 0718b17. Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * [ITN] fix year date graph, cardinals extension for hundreds (NVIDIA#5435) * wip Signed-off-by: ekmb <ebakhturina@nvidia.com> * add lociko's hundreds extension for cardinals Signed-off-by: ekmb <ebakhturina@nvidia.com> * add optional end Signed-off-by: ekmb <ebakhturina@nvidia.com> * restart ci Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * update doc in terms of get_label for lang id model (NVIDIA#5366) * reflect PR 5278 ion doc Signed-off-by: fayejf <fayejf07@gmail.com> * reflect comment Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: fayejf <fayejf07@gmail.com> * Revert workaround for T5 that sets number of workers to 0 & sync_batch_comm=False (NVIDIA#5420) (NVIDIA#5433) * Revert workers workaround Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix in config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Fixed bug in notebook (NVIDIA#5382) (NVIDIA#5394) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * Fixing bug in Megatron BERT when loss mask is all zeros (NVIDIA#5424) * Fixing bug when loss mask is fully zero Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update megatron_bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Use updated API for overlapping grad sync with pipeline parallelism (NVIDIA#5236) Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> * support to disable sequence length + 1 input tokens for each sample in MegatronGPT (NVIDIA#5363) * support to disable sequence length + 1 input tokens for MegatronGPT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: Anmol Gupta <anmolg@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * [TTS] Create script for processing TTS training audio (NVIDIA#5262) * Create script for processing TTS training audio * Update VAD trimming logic * Remove unused import Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] remove useless logic for set_tokenizer. (NVIDIA#5430) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix setting up of `ReduceLROnPlateau` learning rate scheduler (NVIDIA#5444) * Fix tests Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Add accidentally lost changes Signed-off-by: PeganovAnton <peganoff2@mail.ru> Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Create codeql.yml (NVIDIA#5445) Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> * Fix for getting tokenizer in character-based ASR models when using tarred dataset (NVIDIA#5442) Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> * Combine 5 commits adding diar_infer_general.yaml Signed-off-by: Taejin Park <tango4j@gmail.com> Update codeql.yml Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Update codeql.yml Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> fix msdd_model in general yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> fixed errors in yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> * moved eval_der function and fixed tqdm options Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Changed minor error in docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * removed score_labels and changed leave=True Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: PeganovAnton <peganoff2@mail.ru> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Shane Carroll <50530592+1-800-BAD-CODE@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Co-authored-by: anmolgupt <14880251+anmolgupt@users.noreply.github.com> Co-authored-by: Anmol Gupta <anmolg@nvidia.com> Co-authored-by: Ryan Langman <rlangman@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: PeganovAnton <peganoff2@mail.ru> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Jonghwan Hyeon <jonghwanhyeon93@gmail.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com>

* first commit on eval_diar_with_asr.py Signed-off-by: Taejin Park <tango4j@gmail.com> * Add a standalone diarization-ASR evaluation transcript Signed-off-by: Taejin Park <tango4j@gmail.com> * Fixed examples in docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixed staticmethod error Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Added description on eval modes Signed-off-by: Taejin Park <tango4j@gmail.com> * adding diar_infer_general.yaml Signed-off-by: Taejin Park <tango4j@gmail.com> * fix msdd_model in general yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> * fixed errors in yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> * combine into 1 commit Signed-off-by: Taejin Park <tango4j@gmail.com> * Added description on eval modes Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Add MoE support for T5 model (w/o expert parallel) (NVIDIA#5409) * clean Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * kwarg ref Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * extra args Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * test Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * rm prints Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * style Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * review comments Signed-off-by: Abhinav Khattar <aklife97@gmail.com> * fix Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Fix args (NVIDIA#5410) (NVIDIA#5416) Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Fix for concat map dataset (NVIDIA#5133) * change for concat map dataset * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Exhaust longest dataset * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: 1-800-BAD-CODE <> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Add temporary fix for CUDA issue in Dockerfile (NVIDIA#5421) (NVIDIA#5422) Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * Fix GPT generation when using sentencepiece tokenizer (NVIDIA#5413) (NVIDIA#5428) * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Support for finetuning and finetuning inference with .ckpt files & batch size refactoring (NVIDIA#5339) * Initial refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Resolve config before passing to load_from_checkpoint Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for model parallel and nemo restore Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fixes for eval Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Revert config changes Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Refactor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix typo Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove comments Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Minor Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix validation reconfiguration Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove old comment Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fixes for test_ds Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> * Revert "Add temporary fix for CUDA issue in Dockerfile (NVIDIA#5421)" (NVIDIA#5431) (NVIDIA#5432) This reverts commit 0718b17. Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> * [ITN] fix year date graph, cardinals extension for hundreds (NVIDIA#5435) * wip Signed-off-by: ekmb <ebakhturina@nvidia.com> * add lociko's hundreds extension for cardinals Signed-off-by: ekmb <ebakhturina@nvidia.com> * add optional end Signed-off-by: ekmb <ebakhturina@nvidia.com> * restart ci Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> * update doc in terms of get_label for lang id model (NVIDIA#5366) * reflect PR 5278 ion doc Signed-off-by: fayejf <fayejf07@gmail.com> * reflect comment Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: fayejf <fayejf07@gmail.com> * Revert workaround for T5 that sets number of workers to 0 & sync_batch_comm=False (NVIDIA#5420) (NVIDIA#5433) * Revert workers workaround Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix in config Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Fix Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> * Fixed bug in notebook (NVIDIA#5382) (NVIDIA#5394) Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> * Fixing bug in Megatron BERT when loss mask is all zeros (NVIDIA#5424) * Fixing bug when loss mask is fully zero Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update megatron_bert_model.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> * Update dataset_utils.py Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * Use updated API for overlapping grad sync with pipeline parallelism (NVIDIA#5236) Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> * support to disable sequence length + 1 input tokens for each sample in MegatronGPT (NVIDIA#5363) * support to disable sequence length + 1 input tokens for MegatronGPT * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci Co-authored-by: Anmol Gupta <anmolg@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> * [TTS] Create script for processing TTS training audio (NVIDIA#5262) * Create script for processing TTS training audio * Update VAD trimming logic * Remove unused import Signed-off-by: Ryan <rlangman@nvidia.com> * [TTS] remove useless logic for set_tokenizer. (NVIDIA#5430) Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> * Fix setting up of `ReduceLROnPlateau` learning rate scheduler (NVIDIA#5444) * Fix tests Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Add accidentally lost changes Signed-off-by: PeganovAnton <peganoff2@mail.ru> Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Create codeql.yml (NVIDIA#5445) Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> * Fix for getting tokenizer in character-based ASR models when using tarred dataset (NVIDIA#5442) Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> * Combine 5 commits adding diar_infer_general.yaml Signed-off-by: Taejin Park <tango4j@gmail.com> Update codeql.yml Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Update codeql.yml Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> fix msdd_model in general yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> fixed errors in yaml file Signed-off-by: Taejin Park <tango4j@gmail.com> * moved eval_der function and fixed tqdm options Signed-off-by: Taejin Park <tango4j@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Changed minor error in docstrings Signed-off-by: Taejin Park <tango4j@gmail.com> * removed score_labels and changed leave=True Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Taejin Park <tango4j@gmail.com> Signed-off-by: Abhinav Khattar <aklife97@gmail.com> Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: ekmb <ebakhturina@nvidia.com> Signed-off-by: fayejf <fayejf07@gmail.com> Signed-off-by: Virginia Adams <vadams@nvidia.com> Signed-off-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Ryan <rlangman@nvidia.com> Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Signed-off-by: PeganovAnton <peganoff2@mail.ru> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Abhinav Khattar <aklife97@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: Shane Carroll <50530592+1-800-BAD-CODE@users.noreply.github.com> Co-authored-by: Oleksii Kuchaiev <okuchaiev@users.noreply.github.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com> Co-authored-by: Yi Dong <yidong@nvidia.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Co-authored-by: Shanmugam Ramasamy <111910568+shanmugamr1992@users.noreply.github.com> Co-authored-by: Tim Moon <4406448+timmoon10@users.noreply.github.com> Co-authored-by: anmolgupt <14880251+anmolgupt@users.noreply.github.com> Co-authored-by: Anmol Gupta <anmolg@nvidia.com> Co-authored-by: Ryan Langman <rlangman@nvidia.com> Co-authored-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com> Co-authored-by: PeganovAnton <peganoff2@mail.ru> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Jonghwan Hyeon <jonghwanhyeon93@gmail.com> Signed-off-by: andrusenkoau <andrusenkoau@gmail.com>

tango4j and others added 4 commits November 15, 2022 20:31

first commit on eval_diar_with_asr.py

19cbe6f

Signed-off-by: Taejin Park <tango4j@gmail.com>

Add a standalone diarization-ASR evaluation transcript

d85efa2

Signed-off-by: Taejin Park <tango4j@gmail.com>

Fixed examples in docstrings

c1e7cf4

Signed-off-by: Taejin Park <tango4j@gmail.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

aea2e6c

for more information, see https://pre-commit.ci

tango4j and others added 4 commits November 16, 2022 14:54

Fixed staticmethod error

c48bf07

Signed-off-by: Taejin Park <tango4j@gmail.com>

merged main

cdc3235

Signed-off-by: Taejin Park <tango4j@gmail.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

b9feb6b

for more information, see https://pre-commit.ci

Merge branch 'main' into mulspk_asr_eval_script

af94a2e

tango4j marked this pull request as ready for review November 17, 2022 01:05

tango4j added 2 commits November 16, 2022 21:21

Merge branch 'main' into mulspk_asr_eval_script

902c088

Merge branch 'main' into mulspk_asr_eval_script

11a7188

SeanNaren added feature request/PR for a new feature ASR Speaker Tasks labels Nov 17, 2022

tango4j and others added 15 commits November 17, 2022 11:11

Added description on eval modes

979ec60

Signed-off-by: Taejin Park <tango4j@gmail.com>

Merge branch 'mulspk_asr_eval_script' of https://github.com/NVIDIA/NeMo…

2731413

… into mulspk_asr_eval_script

Merge branch 'main' into mulspk_asr_eval_script

9a48743

adding diar_infer_general.yaml

e449287

Signed-off-by: Taejin Park <tango4j@gmail.com>

Merge branch 'mulspk_asr_eval_script' of https://github.com/NVIDIA/NeMo…

a4cbd7a

… into mulspk_asr_eval_script

fix msdd_model in general yaml file

e35629d

Signed-off-by: Taejin Park <tango4j@gmail.com>

fixed errors in yaml file

db31725

Signed-off-by: Taejin Park <tango4j@gmail.com>

combine into 1 commit

50f8417

Signed-off-by: Taejin Park <tango4j@gmail.com>

Added description on eval modes

61b4809

Signed-off-by: Taejin Park <tango4j@gmail.com>

[pre-commit.ci] auto fixes from pre-commit.com hooks

250468c

for more information, see https://pre-commit.ci

Add temporary fix for CUDA issue in Dockerfile (#5421) (#5422)

dd3f3aa

Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Signed-off-by: Yu Yao <yuya@nvidia.com> Co-authored-by: yaoyu-33 <54727607+yaoyu-33@users.noreply.github.com>

shanmugamr1992 and others added 11 commits November 17, 2022 14:10

Use updated API for overlapping grad sync with pipeline parallelism (#…

51250e9

…5236) Signed-off-by: Tim Moon <tmoon@nvidia.com> Signed-off-by: Tim Moon <tmoon@nvidia.com>

[TTS] Create script for processing TTS training audio (#5262)

9de694a

* Create script for processing TTS training audio * Update VAD trimming logic * Remove unused import Signed-off-by: Ryan <rlangman@nvidia.com>

[TTS] remove useless logic for set_tokenizer. (#5430)

b75fc1a

Signed-off-by: Xuesong Yang <1646669+XuesongYang@users.noreply.github.com>

Fix setting up of ReduceLROnPlateau learning rate scheduler (#5444)

40e2fdf

* Fix tests Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Add accidentally lost changes Signed-off-by: PeganovAnton <peganoff2@mail.ru> Signed-off-by: PeganovAnton <peganoff2@mail.ru>

Create codeql.yml (#5445)

841ed52

Signed-off-by: Somshubra Majumdar <titu1994@gmail.com> Signed-off-by: Somshubra Majumdar <titu1994@gmail.com>

Fix for getting tokenizer in character-based ASR models when using ta…

db6c136

…rred dataset (#5442) Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com> Signed-off-by: Jonghwan Hyeon <hyeon0145@gmail.com>

resolved conflict

8f60d87

Signed-off-by: Taejin Park <tango4j@gmail.com>

Merge branch 'main' into mulspk_asr_eval_script

8f37ecb

nithinraok reviewed Nov 18, 2022

View reviewed changes

tango4j and others added 6 commits November 18, 2022 14:24

moved eval_der function and fixed tqdm options

1402c97

Signed-off-by: Taejin Park <tango4j@gmail.com>

Merge branch 'mulspk_asr_eval_script' of https://github.com/NVIDIA/NeMo…

5390636

… into mulspk_asr_eval_script

Merge branch 'main' into mulspk_asr_eval_script

f00250a

[pre-commit.ci] auto fixes from pre-commit.com hooks

92f4159

for more information, see https://pre-commit.ci

Changed minor error in docstrings

1fa70e8

Signed-off-by: Taejin Park <tango4j@gmail.com>

Merge branch 'mulspk_asr_eval_script' of https://github.com/NVIDIA/NeMo…

e2d519d

… into mulspk_asr_eval_script

fayejf reviewed Nov 18, 2022

View reviewed changes

github-advanced-security bot found potential problems Nov 18, 2022

View reviewed changes

scripts/speaker_tasks/eval_diar_with_asr.py Fixed Show resolved Hide resolved

tango4j added 2 commits November 18, 2022 17:48

removed score_labels and changed leave=True

037b61c

Signed-off-by: Taejin Park <tango4j@gmail.com>

Merge branch 'main' into mulspk_asr_eval_script

2b838af

nithinraok approved these changes Nov 21, 2022

View reviewed changes

nithinraok merged commit 9e76e3f into main Nov 21, 2022

nithinraok deleted the mulspk_asr_eval_script branch November 21, 2022 18:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Standalone diarization+ASR evaluation script #5439

Standalone diarization+ASR evaluation script #5439

tango4j commented Nov 16, 2022 •

edited

Loading

lgtm-com bot commented Nov 16, 2022

nithinraok Nov 18, 2022

tango4j Nov 18, 2022

nithinraok Nov 18, 2022

tango4j Nov 18, 2022 •

edited

Loading

nithinraok Nov 18, 2022

tango4j Nov 18, 2022

fayejf Nov 18, 2022

tango4j Nov 18, 2022

lgtm-com bot commented Nov 18, 2022

		return total_result_jsons


		def get_audacity_label(word: str, stt_sec: float, end_sec: float, speaker: str) -> str:

Standalone diarization+ASR evaluation script #5439

Standalone diarization+ASR evaluation script #5439

Conversation

tango4j commented Nov 16, 2022 • edited Loading

What does this PR do ?

Changelog

Usage

Before your PR is "Ready for review"

Who can review?

Additional Information

lgtm-com bot commented Nov 16, 2022

nithinraok Nov 18, 2022

Choose a reason for hiding this comment

tango4j Nov 18, 2022

Choose a reason for hiding this comment

nithinraok Nov 18, 2022

Choose a reason for hiding this comment

tango4j Nov 18, 2022 • edited Loading

Choose a reason for hiding this comment

nithinraok Nov 18, 2022

Choose a reason for hiding this comment

tango4j Nov 18, 2022

Choose a reason for hiding this comment

fayejf Nov 18, 2022

Choose a reason for hiding this comment

tango4j Nov 18, 2022

Choose a reason for hiding this comment

lgtm-com bot commented Nov 18, 2022

tango4j commented Nov 16, 2022 •

edited

Loading

tango4j Nov 18, 2022 •

edited

Loading