Scoring and Analysis of Adapter Experiments #5187
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ng updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com>
for more information, see https://pre-commit.ci
titu1994
reviewed
Oct 18, 2022
| Usage: | ||
| python scoring_and_analysis.py \ | ||
| --csv <path to cleaned result csv file> \ | ||
| --dataset_type_column <column in csv with the dataset types> |
There was a problem hiding this comment.
Move the basic usage documentation inside this script.
examples/asr/transcribe_speech.py
Outdated
| @@ -172,8 +172,9 @@ def main(cfg: TranscriptionConfig) -> TranscriptionConfig: | |||
| if hasattr(asr_model, 'change_decoding_strategy'): | |||
| # Check if ctc or rnnt model | |||
| if hasattr(asr_model, 'joint'): # RNNT model | |||
| rnnt_decoding = RNNTDecodingConfig(fused_batch_size=-1, compute_langs=cfg.compute_langs) | |||
| asr_model.change_decoding_strategy(rnnt_decoding) | |||
| cfg.rnnt_decoding.fused_batch_size = -1 | |||
There was a problem hiding this comment.
Rebase your pr, I've merged the other pr with these changes.
titu1994
previously approved these changes
Oct 20, 2022
|
Rebase, then force push |
…id of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: Shantanu Acharya <shantanua@nvidia.com>
titu1994
approved these changes
Oct 20, 2022
XuesongYang
pushed a commit
that referenced
this pull request
Oct 20, 2022
* Fixed bug in transcribe_speech.py where decoding strategy was not being updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Created the script to calculate scores and perform analysis on the grid of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Include the basic usage doc inside the scoring script * Update the docstrings in the adapters scoring script Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> update
1-800-BAD-CODE
pushed a commit
to 1-800-BAD-CODE/NeMo
that referenced
this pull request
Nov 13, 2022
* Fixed bug in transcribe_speech.py where decoding strategy was not being updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Created the script to calculate scores and perform analysis on the grid of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Include the basic usage doc inside the scoring script * Update the docstrings in the adapters scoring script Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: 1-800-bad-code <shane.carroll@utsa.edu>
hainan-xv
pushed a commit
to hainan-xv/NeMo
that referenced
this pull request
Nov 29, 2022
* Fixed bug in transcribe_speech.py where decoding strategy was not being updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Created the script to calculate scores and perform analysis on the grid of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Include the basic usage doc inside the scoring script * Update the docstrings in the adapters scoring script Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com>
hainan-xv
pushed a commit
to hainan-xv/NeMo
that referenced
this pull request
Nov 29, 2022
* Fixed bug in transcribe_speech.py where decoding strategy was not being updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Created the script to calculate scores and perform analysis on the grid of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Include the basic usage doc inside the scoring script * Update the docstrings in the adapters scoring script Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Added the script for scoring and analysis of the adapter experiments.
Collection: asr
Changelog
examples/asr/asr_adapters/scoring_and_analysis.pyexamples/asr/asr_adapters/README.mdBefore your PR is "Ready for review"
Pre checks:
PR Type:
Who can review?
Anyone in the NeMo ASR Team