CHiME-7 DASR fixes from participants feedback #4999
Conversation
Codecov Report
@@ Coverage Diff @@
## master #4999 +/- ##
==========================================
- Coverage 77.03% 76.65% -0.39%
==========================================
Files 606 606
Lines 53755 53721 -34
==========================================
- Hits 41409 41178 -231
- Misses 12346 12543 +197
Flags with carried forward coverage won't be shown. Click here to find out more. see 21 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
This pull request is now in conflict :( |
…to chime7task1
egs2/chime7_task1/asr1/README.md
Outdated
| either the "style" of the baseline GSS ones or the ones belonging to close-talk mics. | ||
|
|
||
| To evaluate the new enhanced data, e.g. `data/chime6/dev/my_enhanced`, you can `run ./asr.sh --feats_type raw_copy --skip_train true --test_sets chime6/dev/enhanced`. | ||
|
|
There was a problem hiding this comment.
Sorry, I wrote them really roughly.
- Actually we can't run asr.sh directly, because it requires train_set etc.
- I think it's better to create a wrapping script for this purpose e.g. name as
eval.shto run asr.sh with required options. - It's enough to select one from my 1 and 2 ideas.
- It's better to write an example to run the eval.sh
e.g.
utils/copy_data_dir.sh data/kaldi/chime6/dev/gss data/kaldi/chime6/dev/my_enhanced
# rewrite wav.scp
./eval.sh --data_sets "kaldi/chime6/dev/my_enhanced"I think these should be done in the next PR.
There was a problem hiding this comment.
let me delete the last bit then.
There was a problem hiding this comment.
I think diarization part has a priority, so please just keep this comment in mind for now
There was a problem hiding this comment.
I think that
run.sh --stage 3 --asr-tt-set kaldi/chime6/dev/gss --decode-only 1 --use-pretrained popcornell/chime7_task1_asr1_baseline --asr-dprep-stage 4
could do it actually.
There was a problem hiding this comment.
Yeah, I think so, it's also okay, but it seems a little bit complicated,
There was a problem hiding this comment.
Also note that an option for --feats_type raw_copy is not prepared in run.sh
There was a problem hiding this comment.
Another idea is copying the dumpdir.
utils/copy_data_dir.sh dump/kaldi/chime6/dev/gss dump/kaldi/chime6/dev/my_enhanced
echo raw > dump/kaldi/chime6/dev/my_enhanced/feats_type
# Rewrite dump/kaldi/chime6/dev/my_enhanced/wav.scp
# --skip_data_prep true --skip_train true --test_sets kaldi/chime6/dev/my_enhanced
There was a problem hiding this comment.
So assuming that there is no ffmpeg or sox piping I could use raw_copy to avoid the copying right ?
There was a problem hiding this comment.
So assuming that there is no ffmpeg or sox piping I could use raw_copy to avoid the copying right ?
Yes, that's right.
for more information, see https://pre-commit.ci
# Conflicts: # egs2/chime7_task1/asr1/README.md
|
Can we merge ? |
|
By the way, could you also put the number of utterances in the README for each set at the final state? Especially, I'd like to know the number of training set since this recipe is complicated and I wonder I could generate all correctly. |
|
Please let me know if you think ready, then I'll merge quickly |
for more information, see https://pre-commit.ci
|
I have added the details of the number of utterances for each dataset. |
|
Thanks many! |
|
Well, thank you actually ! |
|
Tests fail because some link is not reachable |
| for dset in chime6 dipco; do | ||
| lhotse kaldi export -p ${manifests_root}/${dset}/${dset_part}/${dset}-${mic}_recordings_${dset_part}.jsonl.gz ${manifests_root}/${dset}/${dset_part}/${dset}-${mic}_supervisions_${dset_part}.jsonl.gz data/kaldi/${dset}/${dset_part}/${mic} | ||
| for dset in chime6 dipco mixer6; do | ||
| lhotse kaldi export ${manifests_root}/${dset}/${dset_part}/${dset}-${mic}_recordings_${dset_part}.jsonl.gz ${manifests_root}/${dset}/${dset_part}/${dset}-${mic}_supervisions_${dset_part}.jsonl.gz data/kaldi/${dset}/${dset_part}/${mic} |
There was a problem hiding this comment.
I had to add the --prefix-spk-id flag in lhotse kaldi export otherwise the created data dir was not sorted by spk id.
These fixes address mainly @kamo-naoyuki comments.
I have added mixer6 dev ihm.
Re-formatted the kaldi utterance-id so that it is easier to debug utterances e.g. all follow
{speaker}-{record_id}-{start}_{end}-{mic-channel}now as suggested.Other small fixes too.
Added evaluation script.
I also added memory figures from @boeddeker in the README