Patch transcribe_util for steaming mode and add wer calculation back to inference scripts

[fayejf]

What does this PR do ?

Patch transcribe_util for steaming mode and add wer calculation back to the inference scripts as requested

Collection: ASR

Changelog

• Patch decoding in write_transcription
• Temporarily set decoding.beam.return_best_hypothesis = True to true to be written to manifest to avoidassert, see in discussion @karpnv. May need to come up a better way for ngram/beam
• Add wer calculation back to
  transcribe_speech.py,
  speech_to_text_buffered_infer_ctc,
  speech_to_text_buffered_infer_rnnt as requested, but I still recommand to use asr evaluator and simply those inference script if possible.
• Reflect changed in ASR evaluator

Usage

python transcribe_speech.py \
    model_path=xxxx \
    dataset_manifest=xxx\
    clean_groundtruth_text=True \
    langid='zh' \
    use_cer=True \
    calculate_wer=True

[NeMo I 2023-05-08 15:43:25 transcribe_speech:360] Writing prediction and error rate of each sample to xxxx
[NeMo I 2023-05-08 15:43:25 transcribe_speech:361] {'samples': 2, 'tokens': 164, 'wer': 0.04878048780487805, 'ins_rate': 0.0, 'del_rate': 0.04878048780487805, 'sub_rate': 0.0}

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation?
• Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc)
  • Reviewer: Does the PR have correct import guards for all optional libraries?

PR Type:

• New Feature
• Bugfix
• Documentation

If you haven't finished some of the above items you can still open "Draft" PR.

Who can review?

Anyone in the community is free to review the PR once the checks have passed.
Contributor guidelines contains specific people who can review PRs to various areas.

Additional Information

• Related toPR#6559

[titu1994]

It looks ok but this is too many changes before release. @stevehuang52 for final review

[titu1994]

This all will be replaced by the models normalizer. Such global regex for language normalization Is just error prone, so it will have to be updated to support the new decoding strategy method

[fayejf]

Gotcha. I can update this part once the normalizer is ready.

[titu1994]

Is not present

[VahidooX]

Thanks! just left a minor comment.

[VahidooX]

Would you please add some comments on the top for users how and when to use these two arguments: clean_groundtruth_text and langid?