-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Scoring and Analysis of Adapter Experiments #5187
Conversation
…ng updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com>
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall looks good, requires a bit of shuffling of the docstring + more comments on which args users must change and what should be format of the csv file c
Usage: | ||
python scoring_and_analysis.py \ | ||
--csv <path to cleaned result csv file> \ | ||
--dataset_type_column <column in csv with the dataset types> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Move the basic usage documentation inside this script.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
examples/asr/transcribe_speech.py
Outdated
@@ -172,8 +172,9 @@ def main(cfg: TranscriptionConfig) -> TranscriptionConfig: | |||
if hasattr(asr_model, 'change_decoding_strategy'): | |||
# Check if ctc or rnnt model | |||
if hasattr(asr_model, 'joint'): # RNNT model | |||
rnnt_decoding = RNNTDecodingConfig(fused_batch_size=-1, compute_langs=cfg.compute_langs) | |||
asr_model.change_decoding_strategy(rnnt_decoding) | |||
cfg.rnnt_decoding.fused_batch_size = -1 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rebase your pr, I've merged the other pr with these changes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is done
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great ! Thanks !
Rebase, then force push |
…id of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com>
for more information, see https://pre-commit.ci
Signed-off-by: Shantanu Acharya <shantanua@nvidia.com>
* Fixed bug in transcribe_speech.py where decoding strategy was not being updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Created the script to calculate scores and perform analysis on the grid of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Include the basic usage doc inside the scoring script * Update the docstrings in the adapters scoring script Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> update
* Fixed bug in transcribe_speech.py where decoding strategy was not being updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Created the script to calculate scores and perform analysis on the grid of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Include the basic usage doc inside the scoring script * Update the docstrings in the adapters scoring script Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: 1-800-bad-code <shane.carroll@utsa.edu>
* Fixed bug in transcribe_speech.py where decoding strategy was not being updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Created the script to calculate scores and perform analysis on the grid of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Include the basic usage doc inside the scoring script * Update the docstrings in the adapters scoring script Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com>
* Fixed bug in transcribe_speech.py where decoding strategy was not being updated. Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Created the script to calculate scores and perform analysis on the grid of experiments Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Include the basic usage doc inside the scoring script * Update the docstrings in the adapters scoring script Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Signed-off-by: Shantanu Acharya <shantanua@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Hainan Xu <hainanx@nvidia.com>
What does this PR do ?
Added the script for scoring and analysis of the adapter experiments.
Collection: asr
Changelog
examples/asr/asr_adapters/scoring_and_analysis.py
examples/asr/asr_adapters/README.md
Before your PR is "Ready for review"
Pre checks:
PR Type:
Who can review?
Anyone in the NeMo ASR Team