Merge r1.10.0 main #4448
Merged
Merge r1.10.0 main #4448
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: ericharper <complex451@gmail.com>
* Fix typos Signed-off-by: smajumdar <smajumdar@nvidia.com> * Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (#4383) Signed-off-by: tbartley94 <tbartley@nvidia.com> Co-authored-by: tbartley94 <tbartley@nvidia.com> (cherry picked from commit 0322b15) Co-authored-by: Travis Bartley <Travismbartley@gmail.com>
* Add draft of race condition fixes Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Minor improvements Signed-off-by: PeganovAnton <peganoff2@mail.ru> * More race condition fixes Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Improve error message Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Improve error message Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Improve error message Signed-off-by: PeganovAnton <peganoff2@mail.ru>
* Fix typos Signed-off-by: smajumdar <smajumdar@nvidia.com> * Fix typos Signed-off-by: smajumdar <smajumdar@nvidia.com>
* Add reconfigure on validation epoch start Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove pdb Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca>
Signed-off-by: fayejf <fayejf07@gmail.com>
* Fix link Signed-off-by: smajumdar <smajumdar@nvidia.com> * Correct model card Signed-off-by: smajumdar <smajumdar@nvidia.com> * Add ASR Results to Docs Signed-off-by: smajumdar <smajumdar@nvidia.com> * Update info Signed-off-by: smajumdar <smajumdar@nvidia.com> * Update info Signed-off-by: smajumdar <smajumdar@nvidia.com>
Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com>
Signed-off-by: smajumdar <smajumdar@nvidia.com>
…AD processing (#4405) * cherry-pick pr 4317 and avoid signoff issue Signed-off-by: fayejf <fayejf07@gmail.com> * workaround for mp nb issue Signed-off-by: fayejf <fayejf07@gmail.com> * tdqm for mp functions in vad_utils Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * reflect comment Signed-off-by: fayejf <fayejf07@gmail.com> * remove Signed-off-by: fayejf <fayejf07@gmail.com>
* add sleep to fix multi node cache issue, add pynini guard Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix lgtm Signed-off-by: ekmb <ebakhturina@nvidia.com> * add tempfile Signed-off-by: ekmb <ebakhturina@nvidia.com> * savfe tmp file to the same dir Signed-off-by: ekmb <ebakhturina@nvidia.com> Co-authored-by: PeganovAnton <peganoff2@mail.ru>
Signed-off-by: Yi Dong <yidong@nvidia.com>
Co-authored-by: Nick Goncharenko <ngoncharenko@nvidia.com>
* docs: add table overflow handling for nested sections * docs: set table font-size to small Co-authored-by: Nick Goncharenko <ngoncharenko@nvidia.com>
* add subfolder Signed-off-by: ekmb <ebakhturina@nvidia.com> * exp_dir update Signed-off-by: ekmb <ebakhturina@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com>
Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com>
* Updated notebook to fix batch configuration and precision bugs Signed-off-by: Virginia Adams <vadams@nvidia.com> * Deleted cell outputs Signed-off-by: Virginia Adams <vadams@nvidia.com> * Set datasets back to full dataset Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com>
Signed-off-by: ericharper <complex451@gmail.com>
titu1994
approved these changes
Jun 24, 2022
| @@ -266,12 +267,21 @@ def __init__( | |||
| ignore_start_end=ignore_start_end, | |||
| ) | |||
|
|
|||
| pickle.dump(features, open(features_pkl, "wb")) | |||
| # save features to a temp file first to make sure that non-master processes don't start reading the file | |||
There was a problem hiding this comment.
This dosnt have any rank check - very easy to create hundreds of temp dirs and dump loads of files into them. Whoever is author, please rethink doing this.
There was a problem hiding this comment.
@ekmb or @PeganovAnton ^
|
This pull request introduces 1 alert and fixes 1 when merging 66eb5fa into bf906e0 - view on LGTM.com new alerts:
fixed alerts:
|
|
This pull request introduces 1 alert and fixes 1 when merging ac9ab72 into 09be885 - view on LGTM.com new alerts:
fixed alerts:
|
Davood-M
pushed a commit
to Davood-M/NeMo
that referenced
this pull request
Aug 9, 2022
* update branch Signed-off-by: ericharper <complex451@gmail.com> * Fix ASR Typos in tutorials (NVIDIA#4384) * Fix typos Signed-off-by: smajumdar <smajumdar@nvidia.com> * Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (NVIDIA#4383) Signed-off-by: tbartley94 <tbartley@nvidia.com> Co-authored-by: tbartley94 <tbartley@nvidia.com> (cherry picked from commit 0322b15) Co-authored-by: Travis Bartley <Travismbartley@gmail.com> * Punctuation and capitalization tests race condition (NVIDIA#4399) * Add draft of race condition fixes Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Minor improvements Signed-off-by: PeganovAnton <peganoff2@mail.ru> * More race condition fixes Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Improve error message Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Improve error message Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Improve error message Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Fix tutorial typos and docs (NVIDIA#4415) * Fix typos Signed-off-by: smajumdar <smajumdar@nvidia.com> * Fix typos Signed-off-by: smajumdar <smajumdar@nvidia.com> * Add reconfigure on validation epoch start (NVIDIA#4393) * Add reconfigure on validation epoch start Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove pdb Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * switch branch (NVIDIA#4424) Signed-off-by: fayejf <fayejf07@gmail.com> * Add ASR Scores to Docs (NVIDIA#4412) * Fix link Signed-off-by: smajumdar <smajumdar@nvidia.com> * Correct model card Signed-off-by: smajumdar <smajumdar@nvidia.com> * Add ASR Results to Docs Signed-off-by: smajumdar <smajumdar@nvidia.com> * Update info Signed-off-by: smajumdar <smajumdar@nvidia.com> * Update info Signed-off-by: smajumdar <smajumdar@nvidia.com> * Re-apply fixes from r1.9.0 (NVIDIA#4425) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * Replace all with /content/ (NVIDIA#4427) Signed-off-by: smajumdar <smajumdar@nvidia.com> * Fix hanging issue by multiprocessing in SD tutorial and add ETA for VAD processing (NVIDIA#4405) * cherry-pick pr 4317 and avoid signoff issue Signed-off-by: fayejf <fayejf07@gmail.com> * workaround for mp nb issue Signed-off-by: fayejf <fayejf07@gmail.com> * tdqm for mp functions in vad_utils Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * reflect comment Signed-off-by: fayejf <fayejf07@gmail.com> * remove Signed-off-by: fayejf <fayejf07@gmail.com> * [NLP] P&C Fix multi node cache issue, add pynini guard (NVIDIA#4410) * add sleep to fix multi node cache issue, add pynini guard Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix lgtm Signed-off-by: ekmb <ebakhturina@nvidia.com> * add tempfile Signed-off-by: ekmb <ebakhturina@nvidia.com> * savfe tmp file to the same dir Signed-off-by: ekmb <ebakhturina@nvidia.com> Co-authored-by: PeganovAnton <peganoff2@mail.ru> * fix the notebook (NVIDIA#4438) Signed-off-by: Yi Dong <yidong@nvidia.com> * update nemo version dialogue tutorial (NVIDIA#4437) * docs: add table overflow handling for nested sections (NVIDIA#4441) Co-authored-by: Nick Goncharenko <ngoncharenko@nvidia.com> * Docs: Decrease Font Size on Tables (NVIDIA#4444) * docs: add table overflow handling for nested sections * docs: set table font-size to small Co-authored-by: Nick Goncharenko <ngoncharenko@nvidia.com> * unify intent slot dataset util functions in tutorials (NVIDIA#4445) * Notebook bug fix: add subfolder (NVIDIA#4442) * add subfolder Signed-off-by: ekmb <ebakhturina@nvidia.com> * exp_dir update Signed-off-by: ekmb <ebakhturina@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Fix typo in HiFi-GAN config's max steps (NVIDIA#4446) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Updated notebook to fix batch configuration and precision bugs (NVIDIA#4447) * Updated notebook to fix batch configuration and precision bugs Signed-off-by: Virginia Adams <vadams@nvidia.com> * Deleted cell outputs Signed-off-by: Virginia Adams <vadams@nvidia.com> * Set datasets back to full dataset Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Travis Bartley <Travismbartley@gmail.com> Co-authored-by: PeganovAnton <peganoff2@mail.ru> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Yi Dong <43824965+yidong72@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: Nick Goncharenko <8766167+nickolyamba@users.noreply.github.com> Co-authored-by: Nick Goncharenko <ngoncharenko@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Signed-off-by: David Mosallanezhad <dmosallanezh@nvidia.com>
hainan-xv
pushed a commit
to hainan-xv/NeMo
that referenced
this pull request
Nov 29, 2022
* update branch Signed-off-by: ericharper <complex451@gmail.com> * Fix ASR Typos in tutorials (NVIDIA#4384) * Fix typos Signed-off-by: smajumdar <smajumdar@nvidia.com> * Quick wav2vec fix. In-place operation adding convolutional positions to encoder was overwriting leaf history. Wasn't caught on previous torch versions. (NVIDIA#4383) Signed-off-by: tbartley94 <tbartley@nvidia.com> Co-authored-by: tbartley94 <tbartley@nvidia.com> (cherry picked from commit 0322b15) Co-authored-by: Travis Bartley <Travismbartley@gmail.com> * Punctuation and capitalization tests race condition (NVIDIA#4399) * Add draft of race condition fixes Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Minor improvements Signed-off-by: PeganovAnton <peganoff2@mail.ru> * More race condition fixes Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Improve error message Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Improve error message Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Improve error message Signed-off-by: PeganovAnton <peganoff2@mail.ru> * Fix tutorial typos and docs (NVIDIA#4415) * Fix typos Signed-off-by: smajumdar <smajumdar@nvidia.com> * Fix typos Signed-off-by: smajumdar <smajumdar@nvidia.com> * Add reconfigure on validation epoch start (NVIDIA#4393) * Add reconfigure on validation epoch start Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Style Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * Remove pdb Signed-off-by: MaximumEntropy <sandeep.subramanian.1@umontreal.ca> * switch branch (NVIDIA#4424) Signed-off-by: fayejf <fayejf07@gmail.com> * Add ASR Scores to Docs (NVIDIA#4412) * Fix link Signed-off-by: smajumdar <smajumdar@nvidia.com> * Correct model card Signed-off-by: smajumdar <smajumdar@nvidia.com> * Add ASR Results to Docs Signed-off-by: smajumdar <smajumdar@nvidia.com> * Update info Signed-off-by: smajumdar <smajumdar@nvidia.com> * Update info Signed-off-by: smajumdar <smajumdar@nvidia.com> * Re-apply fixes from r1.9.0 (NVIDIA#4425) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> * Replace all with /content/ (NVIDIA#4427) Signed-off-by: smajumdar <smajumdar@nvidia.com> * Fix hanging issue by multiprocessing in SD tutorial and add ETA for VAD processing (NVIDIA#4405) * cherry-pick pr 4317 and avoid signoff issue Signed-off-by: fayejf <fayejf07@gmail.com> * workaround for mp nb issue Signed-off-by: fayejf <fayejf07@gmail.com> * tdqm for mp functions in vad_utils Signed-off-by: fayejf <fayejf07@gmail.com> * style fix Signed-off-by: fayejf <fayejf07@gmail.com> * reflect comment Signed-off-by: fayejf <fayejf07@gmail.com> * remove Signed-off-by: fayejf <fayejf07@gmail.com> * [NLP] P&C Fix multi node cache issue, add pynini guard (NVIDIA#4410) * add sleep to fix multi node cache issue, add pynini guard Signed-off-by: ekmb <ebakhturina@nvidia.com> * fix lgtm Signed-off-by: ekmb <ebakhturina@nvidia.com> * add tempfile Signed-off-by: ekmb <ebakhturina@nvidia.com> * savfe tmp file to the same dir Signed-off-by: ekmb <ebakhturina@nvidia.com> Co-authored-by: PeganovAnton <peganoff2@mail.ru> * fix the notebook (NVIDIA#4438) Signed-off-by: Yi Dong <yidong@nvidia.com> * update nemo version dialogue tutorial (NVIDIA#4437) * docs: add table overflow handling for nested sections (NVIDIA#4441) Co-authored-by: Nick Goncharenko <ngoncharenko@nvidia.com> * Docs: Decrease Font Size on Tables (NVIDIA#4444) * docs: add table overflow handling for nested sections * docs: set table font-size to small Co-authored-by: Nick Goncharenko <ngoncharenko@nvidia.com> * unify intent slot dataset util functions in tutorials (NVIDIA#4445) * Notebook bug fix: add subfolder (NVIDIA#4442) * add subfolder Signed-off-by: ekmb <ebakhturina@nvidia.com> * exp_dir update Signed-off-by: ekmb <ebakhturina@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Fix typo in HiFi-GAN config's max steps (NVIDIA#4446) Signed-off-by: Jocelyn Huang <jocelynh@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * Updated notebook to fix batch configuration and precision bugs (NVIDIA#4447) * Updated notebook to fix batch configuration and precision bugs Signed-off-by: Virginia Adams <vadams@nvidia.com> * Deleted cell outputs Signed-off-by: Virginia Adams <vadams@nvidia.com> * Set datasets back to full dataset Signed-off-by: Virginia Adams <vadams@nvidia.com> Co-authored-by: Eric Harper <complex451@gmail.com> * update branch Signed-off-by: ericharper <complex451@gmail.com> Co-authored-by: Somshubra Majumdar <titu1994@gmail.com> Co-authored-by: Travis Bartley <Travismbartley@gmail.com> Co-authored-by: PeganovAnton <peganoff2@mail.ru> Co-authored-by: Sandeep Subramanian <sandeep.subramanian.1@umontreal.ca> Co-authored-by: fayejf <36722593+fayejf@users.noreply.github.com> Co-authored-by: Jocelyn <jocelynh@nvidia.com> Co-authored-by: Evelina <10428420+ekmb@users.noreply.github.com> Co-authored-by: Yi Dong <43824965+yidong72@users.noreply.github.com> Co-authored-by: Zhilin Wang <wangzhilin12061996@hotmail.com> Co-authored-by: Nick Goncharenko <8766167+nickolyamba@users.noreply.github.com> Co-authored-by: Nick Goncharenko <ngoncharenko@nvidia.com> Co-authored-by: Virginia Adams <78445382+vadam5@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Add a one line overview of what this PR aims to accomplish.
Collection: [Note which collection this PR will affect]
Changelog
Usage
# Add a code snippet demonstrating how to use thisBefore your PR is "Ready for review"
Pre checks:
PR Type:
If you haven't finished some of the above items you can still open "Draft" PR.
Who can review?
Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.
Additional Information