Closed CHiME-7 DASR adding evaluation inference + adding support to use diarization baseline "pre-computed" JSONs (new PR) #5228
Conversation
Codecov Report
@@ Coverage Diff @@
## master #5228 +/- ##
==========================================
- Coverage 74.43% 74.43% -0.01%
==========================================
Files 642 642
Lines 57611 57605 -6
==========================================
- Hits 42885 42880 -5
+ Misses 14726 14725 -1
Flags with carried forward coverage won't be shown. Click here to find out more. see 5 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
@simpleoier can you take a look ? This is important for the challenge to merge ASAP. |
egs2/chime7_task1/asr1/run.sh
Outdated
| asr_tt_set="kaldi/chime6/dev/gss kaldi/dipco/dev/gss/ kaldi/mixer6/dev/gss/" | ||
| elif | ||
| [ $decode_only == "eval" ]; then | ||
| # apply gss only on dev |
There was a problem hiding this comment.
The comment should be updated as eval, right?
| @@ -80,9 +80,15 @@ asr_batch_size=$(calc_int 128*$ngpu) # reduce 128 bsz if you get OOMs errors | |||
| asr_max_lr=$(calc_float $ngpu/10000.0) | |||
| asr_warmup=$(calc_int 40000.0/$ngpu) | |||
|
|
|||
| if [ $decode_only -eq 1 ]; then | |||
| if [ $decode_only == "dev" ]; then | |||
There was a problem hiding this comment.
Will you add a check if decode_only is neither dev nor eval?
I feel decode_only is a bit confusing. How about using skip_train (default: false) to control whether to process the training set? And using test_sets to let users define the gss and asr_tt sets? For example
test_sets="chime6_dev dipco_dev mixer6_dev"
gss_dsets+="${test_sets}"
test_sets_list=(${test_sets// / })
asr_tt_sets=
for dset in ${test_sets_list[@]}; do
asr_tt_sets+=$(echo "kaldi/${dset}/gss " | tr "_" "/")
There was a problem hiding this comment.
This is nice but will require too much change in the codebase at this stage.
There was a problem hiding this comment.
yeah, indeed. We may not need to update this at the moment.
| @@ -26,7 +26,7 @@ background_snrs="20:10:15:5:0" | |||
| gss_dsets=$(echo $gss_dsets | tr "," " ") # split by commas | |||
|
|
|||
|
|
|||
| if [ $decode_only == 1 ]; then | |||
| if [ -n "$decode_only" ]; then | |||
There was a problem hiding this comment.
we can use ${skip_train} and ${test_sets} instead of decode_only.
There was a problem hiding this comment.
as before, we cannot change extensively the recipe at this point. I added the check for the argument.
Note that this will also require to change all two README.md and notify participants.
| you can run: | ||
| ```bash | ||
| ./run.sh --chime7-root YOUR_PATH_TO_CHiME7_ROOT --stage 2 --ngpu YOUR_NUMBER_OF_GPUs \ | ||
| --use-pretrained popcornell/chime7_task1_asr1_baseline \ | ||
| --decode-only 1 --gss-max-batch-dur 30-360-DEPENDING_ON_GPU_MEM \ | ||
| --decode-only dev --gss-max-batch-dur 30-360-DEPENDING_ON_GPU_MEM \ |
There was a problem hiding this comment.
skip_train and test_sets instead of decode-only.
|
I added the check for checking the |
new PR from #5183 which had problems due to error of applying black.
@simpleoier I made another PR and closed the other one.