Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CHiME-7 DASR fixes from participants feedback #4999

Merged
merged 123 commits into from
Mar 23, 2023
Merged

CHiME-7 DASR fixes from participants feedback #4999

Show file tree
Hide file tree
Changes from 100 commits
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
dfecbbf
add common error about sox
popcornell Mar 9, 2023
df4009f
add double quotes for cmd_gss
popcornell Mar 9, 2023
3097e0a
memory consumption figures by boedekker
popcornell Mar 9, 2023
0e20b29
add quotes on cmd
popcornell Mar 9, 2023
3aa5301
changed to round
popcornell Mar 9, 2023
0e181cf
exit if wav-rev not found
popcornell Mar 10, 2023
d42b9f2
implement fixes suggested by naoyuki kamo
popcornell Mar 12, 2023
122447e
implement fixes suggested by naoyuki kamo
popcornell Mar 12, 2023
a2c4ccb
Merge branch 'master' of https://github.com/espnet/espnet
popcornell Mar 12, 2023
9457648
applied black, updated README
popcornell Mar 12, 2023
f24a197
changed error log for selection. now more clear
popcornell Mar 12, 2023
f13d49c
addressed linters errors
popcornell Mar 13, 2023
02983f4
added results.
popcornell Mar 13, 2023
c3b301b
added remarks about GPU in shared mode
popcornell Mar 13, 2023
d51b3e7
do not include by default close-talk
popcornell Mar 13, 2023
51fef16
parsing2json for scoring
popcornell Mar 13, 2023
b9ee140
added uem generation for mixer6 dev
popcornell Mar 13, 2023
686a4c0
added uem generation for dipco too for consistency
popcornell Mar 13, 2023
affaefc
more informative
popcornell Mar 13, 2023
d9d1b6b
sort to make it consistent
popcornell Mar 13, 2023
e400953
results updated
popcornell Mar 13, 2023
e7bdb86
added RESULTS.md
popcornell Mar 13, 2023
8ba4856
Merge branch 'master' into chime7task1
popcornell Mar 13, 2023
01d9b3d
added inference.log
popcornell Mar 13, 2023
e5ee10f
updated README.md
popcornell Mar 13, 2023
34fceb5
scoring tool
popcornell Mar 15, 2023
cc8af97
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 15, 2023
5d04329
added scoring
popcornell Mar 15, 2023
6d5802f
updated readme
popcornell Mar 15, 2023
e678d0b
Merge branch 'master' into chime7task1
popcornell Mar 15, 2023
2f037fe
updated readme
popcornell Mar 15, 2023
9868175
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 15, 2023
edab00f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 15, 2023
35cb977
update help in argparse
popcornell Mar 16, 2023
5624ddc
renamed to da_wer
popcornell Mar 16, 2023
bcc6f65
README.md
popcornell Mar 16, 2023
122265f
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 16, 2023
56b389c
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 16, 2023
252a771
generate uem for training set too
popcornell Mar 16, 2023
d73c395
address linter
popcornell Mar 16, 2023
7da6071
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 16, 2023
7ebbf74
need to upload figure to the website. added ref to the logs
popcornell Mar 16, 2023
d0b2c5c
changed dummy tr data
popcornell Mar 16, 2023
ccbea76
changed dummy tr data
popcornell Mar 16, 2023
94e7e40
changed dummy tr data
popcornell Mar 16, 2023
43b127d
da-wer
popcornell Mar 16, 2023
a3f69f7
regex suitable also for eval
popcornell Mar 16, 2023
c2c36ff
addressed Christoph feedback, very useful.
popcornell Mar 16, 2023
064be34
addressed Shinji comment
popcornell Mar 16, 2023
bc43727
python3 install instead of pip
popcornell Mar 16, 2023
d1d73d6
updated README.md
popcornell Mar 16, 2023
b0e102b
updated md5 checksum with .uem
popcornell Mar 16, 2023
72e60c8
keep only the last part
popcornell Mar 16, 2023
df8577c
added the remark about the dipco warning in README.md
popcornell Mar 16, 2023
f4ffcf0
christoph suggestions
popcornell Mar 16, 2023
c07bcd8
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 16, 2023
2b2fe73
removed inf tagas we are not copying anymore
popcornell Mar 17, 2023
48b69aa
Merge branch 'master' into chime7task1
popcornell Mar 17, 2023
dccdf1e
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 17, 2023
8644a54
minor fixes, added logging as suggested by Christoph.
popcornell Mar 17, 2023
abe519c
always use selection
popcornell Mar 17, 2023
e99299c
kamo readme latex removal suggestion
popcornell Mar 17, 2023
e9da607
fixed concatenation futurewarning. Removed from the log too
popcornell Mar 17, 2023
d6033a9
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 17, 2023
49ec492
updated images
popcornell Mar 17, 2023
a0f430a
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 17, 2023
093e144
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 17, 2023
6f82bbc
Merge branch 'master' into chime7task1
popcornell Mar 17, 2023
4ab1b8e
updated jiwer to version 3
popcornell Mar 18, 2023
b35c6d0
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 18, 2023
1f43d48
corrected typos in README
popcornell Mar 18, 2023
077372b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 18, 2023
bffbf70
corrected typos in README
popcornell Mar 18, 2023
9b141b2
rendering issues
popcornell Mar 18, 2023
371649e
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 18, 2023
378baa2
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 18, 2023
addcfde
Merge branch 'master' into chime7task1
popcornell Mar 18, 2023
1a03623
force re-rendering the new image
popcornell Mar 18, 2023
646ccb3
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 18, 2023
f6d36ae
adding parent in mkdir
popcornell Mar 18, 2023
477bfda
Merge branch 'master' into chime7task1
popcornell Mar 19, 2023
6121849
added remark about exclusive mode
popcornell Mar 19, 2023
32f66ee
Merge branch 'master' into chime7task1
popcornell Mar 19, 2023
ecfd7a3
updated permissions
popcornell Mar 19, 2023
a359e8a
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 19, 2023
653f358
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 19, 2023
eb2691b
Merge branch 'master' into chime7task1
popcornell Mar 20, 2023
5fbd267
Insert space
kamo-naoyuki Mar 20, 2023
7073fbf
Merge branch 'master' into chime7task1
popcornell Mar 21, 2023
fd79b34
lhotse must be installed from github.
popcornell Mar 21, 2023
1b63e6b
Merge branch 'master' into chime7task1
popcornell Mar 21, 2023
3ea2acb
Merge branch 'chime7task1' of https://github.com/popcornell/espnet
popcornell Mar 21, 2023
18f3fa0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 21, 2023
124dc75
Merge branch 'master' into chime7task1
popcornell Mar 21, 2023
41d81ad
Merge branch 'master' into chime7task1
popcornell Mar 21, 2023
ae8f053
change installation order
popcornell Mar 21, 2023
e4f0aa1
Merge branch 'master' into chime7task1
popcornell Mar 21, 2023
021f04f
checking consistency with meeteval
popcornell Mar 21, 2023
f6343a6
Merge branch 'chime7task1' of https://github.com/popcornell/espnet in…
popcornell Mar 21, 2023
45f76d4
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 21, 2023
311bf16
no empty strings
popcornell Mar 21, 2023
05ee199
Merge branch 'chime7task1' of https://github.com/popcornell/espnet in…
popcornell Mar 21, 2023
184e7bc
added naoyuki suggestions on how using your enhanced files
popcornell Mar 21, 2023
3a06669
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 21, 2023
f00c110
fixing issue about validation set discarded utts
popcornell Mar 21, 2023
0d6cc58
Merge branch 'chime7task1' of https://github.com/popcornell/espnet in…
popcornell Mar 21, 2023
8b542c9
fixing the command to evaluate
popcornell Mar 21, 2023
51a6c58
need to be string
popcornell Mar 21, 2023
bacb17b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 21, 2023
ede3556
Merge remote-tracking branch 'samco/chime7task1' into chime7task1
popcornell Mar 21, 2023
26cdf47
if diarization file has no words use a placeholder
popcornell Mar 22, 2023
e7539cf
added option for using forced alignment
popcornell Mar 22, 2023
f60bf51
not hasattr
popcornell Mar 22, 2023
484e8f2
Merge branch 'master' into chime7task1
popcornell Mar 22, 2023
b1c37ff
Merge branch 'master' into chime7task1
popcornell Mar 22, 2023
7cdcb51
mentioning that all datasets can be used
popcornell Mar 22, 2023
f3ed9c2
Merge branch 'chime7task1' of https://github.com/popcornell/espnet in…
popcornell Mar 22, 2023
5f32093
implemented naoyuki fix
popcornell Mar 22, 2023
8c6019d
added report on the utterances
popcornell Mar 22, 2023
1a9f35e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Mar 22, 2023
1cec2c4
Merge branch 'master' into chime7task1
popcornell Mar 23, 2023
abfa62f
fi missing
popcornell Mar 23, 2023
656038f
Merge branch 'chime7task1' of https://github.com/popcornell/espnet in…
popcornell Mar 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
201 changes: 167 additions & 34 deletions egs2/chime7_task1/asr1/README.md
View file
Open in desktop

Large diffs are not rendered by default.

66 changes: 66 additions & 0 deletions egs2/chime7_task1/asr1/baseline_logs/RESULTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
<!-- Generated by scripts/utils/show_asr_result.sh -->
# RESULTS
## Environments
- date: `Mon Mar 13 13:43:21 UTC 2023`
- python version: `3.9.2 (default, Mar 3 2021, 20:02:32) [GCC 7.3.0]`
- espnet version: `espnet 202301`
- pytorch version: `pytorch 1.13.1`
- Git hash: `89ebca463c544dfaa19e5f76ad5f615f473f6957`
- Commit date: `Tue Mar 7 04:02:43 2023 +0000`

## exp/asr_train_asr_transformer_wavlm_lr1e-4_specaugm_accum1_preenc128_warmup20k_raw_en_bpe500_batch_size640_scheduler_confwarmup_steps8000_max_epoch8_optim_conflr0.000500000000_sp/decode_asr_transformer_asr_model_valid.acc.ave/kaldi/mixer6
### WER

|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
|dev/gss|14804|148981|84.4|10.4|5.2|6.2|21.8|60.6|

### CER

|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
|dev/gss|14804|731649|91.0|3.6|5.4|6.5|15.5|60.6|

### TER

|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
|dev/gss|14804|251635|87.0|7.4|5.6|6.9|19.9|60.6|

## exp/asr_train_asr_transformer_wavlm_lr1e-4_specaugm_accum1_preenc128_warmup20k_raw_en_bpe500_batch_size640_scheduler_confwarmup_steps8000_max_epoch8_optim_conflr0.000500000000_sp/decode_asr_transformer_asr_model_valid.acc.ave/kaldi/chime6
### WER

|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
|dev/gss_inf|6644|58881|73.0|18.2|8.8|6.7|33.6|71.2|

### CER

|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
|dev/gss_inf|6644|281489|83.2|7.0|9.8|7.3|24.1|71.2|

### TER

|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
|dev/gss_inf|6644|98596|75.0|14.2|10.9|7.1|32.2|71.2|

## exp/asr_train_asr_transformer_wavlm_lr1e-4_specaugm_accum1_preenc128_warmup20k_raw_en_bpe500_batch_size640_scheduler_confwarmup_steps8000_max_epoch8_optim_conflr0.000500000000_sp/decode_asr_transformer_asr_model_valid.acc.ave/kaldi/dipco
### WER

|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
|dev/gss|3673|29966|74.0|17.8|8.2|8.5|34.5|72.6|

### CER

|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
|dev/gss|3673|146438|84.5|6.3|9.3|8.9|24.4|72.6|

### TER

|dataset|Snt|Wrd|Corr|Sub|Del|Ins|Err|S.Err|
|---|---|---|---|---|---|---|---|---|
|dev/gss|3673|51347|77.1|13.5|9.4|9.2|32.1|72.6|
1 change: 1 addition & 0 deletions egs2/chime7_task1/asr1/baseline_logs/inference.log
View file
Open in desktop
kamo-naoyuki marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
###################################################### Scoring mixer6 Scenario ##############################################################################+----+----------------------------+------------+---------------+---------------+----------------------+----------------------+--------+-----------------+-------------+--------------+----------+| | session id | scenario | num spk hyp | num spk ref | tot utterances hyp | tot utterances ref | hits | substitutions | deletions | insertions | wer ||----+----------------------------+------------+---------------+---------------+----------------------+----------------------+--------+-----------------+-------------+--------------+----------|| 0 | 20091106_161134_LDC_120795 | mixer6 | 2 | 2 | 289 | 289 | 2228 | 175 | 93 | 128 | 0.158654 || 1 | 20091005_150244_LDC_120534 | mixer6 | 2 | 2 | 302 | 302 | 1931 | 186 | 73 | 87 | 0.157991 || 2 | 20090722_154451_LDC_120225 | mixer6 | 2 | 2 | 218 | 218 | 2296 | 225 | 132 | 156 | 0.193366 || 3 | 20090729_155715_LDC_120311 | mixer6 | 2 | 2 | 228 | 228 | 1920 | 408 | 122 | 166 | 0.284082 || 4 | 20090811_150119_LDC_120354 | mixer6 | 2 | 2 | 163 | 163 | 1849 | 99 | 51 | 221 | 0.185593 || 5 | 20091110_141534_LDC_120278 | mixer6 | 2 | 2 | 284 | 284 | 2330 | 365 | 151 | 154 | 0.235418 || 6 | 20091002_120908_LDC_120562 | mixer6 | 2 | 2 | 316 | 316 | 2788 | 282 | 113 | 168 | 0.176877 || 7 | 20091208_175612_LDC_120788 | mixer6 | 2 | 2 | 323 | 323 | 2299 | 243 | 131 | 105 | 0.179199 || 8 | 20090908_150729_LDC_120338 | mixer6 | 2 | 2 | 303 | 303 | 1919 | 252 | 106 | 119 | 0.209486 || 9 | 20090804_165853_LDC_120269 | mixer6 | 2 | 2 | 267 | 267 | 2156 | 220 | 92 | 156 | 0.189627 || 10 | 20090805_115917_LDC_120290 | mixer6 | 2 | 2 | 191 | 191 | 1745 | 652 | 92 | 131 | 0.351547 || 11 | 20091002_170608_LDC_120488 | mixer6 | 2 | 2 | 256 | 256 | 2645 | 301 | 130 | 189 | 0.20156 || 12 | 20090716_155120_LDC_120269 | mixer6 | 2 | 2 | 193 | 193 | 2079 | 365 | 197 | 136 | 0.264294 || 13 | 20090714_134807_LDC_120290 | mixer6 | 2 | 2 | 146 | 146 | 1828 | 369 | 100 | 127 | 0.259469 || 14 | 20090826_150144_LDC_120338 | mixer6 | 2 | 2 | 285 | 285 | 1823 | 258 | 157 | 81 | 0.221626 || 15 | 20090803_120934_LDC_120271 | mixer6 | 2 | 2 | 222 | 222 | 1906 | 265 | 89 | 122 | 0.210619 || 16 | 20091006_153417_LDC_120590 | mixer6 | 2 | 2 | 336 | 336 | 2695 | 343 | 118 | 212 | 0.213245 || 17 | 20090925_160859_LDC_120503 | mixer6 | 2 | 2 | 246 | 246 | 2550 | 401 | 114 | 229 | 0.242741 || 18 | 20091123_150807_LDC_120795 | mixer6 | 2 | 2 | 263 | 263 | 1915 | 135 | 79 | 51 | 0.124472 || 19 | 20091027_153531_LDC_120651 | mixer6 | 2 | 2 | 127 | 127 | 1776 | 503 | 88 | 91 | 0.288128 || 20 | 20090930_165847_LDC_120488 | mixer6 | 2 | 2 | 283 | 283 | 2314 | 557 | 157 | 211 | 0.305482 || 21 | 20090723_111806_LDC_120290 | mixer6 | 2 | 2 | 154 | 154 | 1942 | 263 | 90 | 105 | 0.199564 || 22 | 20090915_141204_LDC_120302 | mixer6 | 2 | 2 | 256 | 256 | 1803 | 129 | 97 | 53 | 0.137506 || 23 | 20090717_133033_LDC_120311 | mixer6 | 2 | 2 | 192 | 192 | 1607 | 375 | 110 | 118 | 0.288241 || 24 | 20091019_164856_LDC_120707 | mixer6 | 2 | 2 | 313 | 313 | 2173 | 292 | 85 | 114 | 0.192549 || 25 | 20090812_145955_LDC_120271 | mixer6 | 2 | 2 | 224 | 224 | 2008 | 189 | 71 | 95 | 0.156526 || 26 | 20090807_143559_LDC_120338 | mixer6 | 2 | 2 | 334 | 334 | 2141 | 351 | 121 | 181 | 0.249904 || 27 | 20090921_185254_LDC_120562 | mixer6 | 2 | 2 | 322 | 322 | 2233 | 242 | 87 | 136 | 0.181499 || 28 | 20091007_114741_LDC_120601 | mixer6 | 2 | 2 | 252 | 252 | 2671 | 228 | 80 | 158 | 0.156428 || 29 | 20090901_104119_LDC_120454 | mixer6 | 2 | 2 | 254 | 254 | 1689 | 234 | 131 | 72 | 0.212756 || 30 | 20090909_091127_LDC_120454 | mixer6 | 2 | 2 | 209 | 209 | 1594 | 139 | 106 | 77 | 0.175095 || 31 | 20090904_090947_LDC_120217 | mixer6 | 2 | 2 | 281 | 281 | 2068 | 195 | 113 | 120 | 0.180135 || 32 | 20091211_152444_LDC_120618 | mixer6 | 2 | 2 | 297 | 297 | 2635 | 251 | 127 | 110 | 0.161965 || 33 | 20090803_111429_LDC_120225 | mixer6 | 2 | 2 | 235 | 235 | 2297 | 266 | 121 | 162 | 0.204545 || 34 | 20091026_160525_LDC_120718 | mixer6 | 2 | 2 | 236 | 236 | 2410 | 248 | 108 | 131 | 0.176067 || 35 | 20090828_090157_LDC_120217 | mixer6 | 2 | 2 | 300 | 300 | 2132 | 216 | 87 | 116 | 0.172074 || 36 | 20091104_101455_LDC_120782 | mixer6 | 2 | 2 | 238 | 238 | 2195 | 143 | 88 | 99 | 0.136026 || 37 | 20091208_150037_LDC_120795 | mixer6 | 2 | 2 | 219 | 219 | 1644 | 229 | 92 | 93 | 0.210687 || 38 | 20091022_185919_LDC_120718 | mixer6 | 2 | 2 | 265 | 265 | 2647 | 260 | 63 | 213 | 0.180471 || 39 | 20090901_144456_LDC_120302 | mixer6 | 2 | 2 | 211 | 211 | 1547 | 137 | 57 | 63 | 0.147616 || 40 | 20091012_100152_LDC_120651 | mixer6 | 2 | 2 | 242 | 242 | 2540 | 191 | 62 | 119 | 0.13319 || 41 | 20091111_112455_LDC_120707 | mixer6 | 2 | 2 | 245 | 245 | 1585 | 212 | 78 | 101 | 0.208533 || 42 | 20091029_175951_LDC_120651 | mixer6 | 2 | 2 | 230 | 230 | 2488 | 518 | 117 | 150 | 0.251361 || 43 | 20090923_175046_LDC_120534 | mixer6 | 2 | 2 | 299 | 299 | 1995 | 157 | 87 | 88 | 0.14828 || 44 | 20090914_125705_LDC_120354 | mixer6 | 2 | 2 | 177 | 177 | 1895 | 306 | 117 | 197 | 0.267472 || 45 | 20090911_110038_LDC_120454 | mixer6 | 2 | 2 | 361 | 361 | 2551 | 271 | 158 | 115 | 0.18255 || 46 | 20090910_090618_LDC_120302 | mixer6 | 2 | 2 | 230 | 230 | 1667 | 112 | 73 | 68 | 0.136609 || 47 | 20090918_165654_LDC_120488 | mixer6 | 2 | 2 | 233 | 233 | 2477 | 334 | 105 | 129 | 0.194787 || 48 | 20090917_114202_LDC_120503 | mixer6 | 2 | 2 | 289 | 289 | 2385 | 302 | 126 | 193 | 0.220761 || 49 | 20090811_170402_LDC_120269 | mixer6 | 2 | 2 | 307 | 307 | 2210 | 268 | 131 | 130 | 0.20276 || 50 | 20090826_100144_LDC_120354 | mixer6 | 2 | 2 | 184 | 184 | 1890 | 142 | 95 | 59 | 0.139163 || 51 | 20090805_110532_LDC_120225 | mixer6 | 2 | 2 | 198 | 198 | 1959 | 234 | 97 | 133 | 0.20262 || 52 | 20091028_131357_LDC_120718 | mixer6 | 2 | 2 | 220 | 220 | 2123 | 306 | 127 | 260 | 0.271127 || 53 | 20090717_113617_LDC_120278 | mixer6 | 2 | 2 | 158 | 158 | 1712 | 322 | 123 | 118 | 0.261011 || 54 | 20090722_115429_LDC_120271 | mixer6 | 2 | 2 | 194 | 194 | 2328 | 210 | 109 | 78 | 0.149981 || 55 | 20100127_140351_LDC_120855 | mixer6 | 2 | 2 | 261 | 261 | 2602 | 323 | 154 | 196 | 0.218577 || 56 | 20090814_111319_LDC_120278 | mixer6 | 2 | 2 | 287 | 287 | 2145 | 300 | 108 | 176 | 0.22875 || 57 | 20091027_190406_LDC_120732 | mixer6 | 2 | 2 | 354 | 354 | 2723 | 238 | 134 | 134 | 0.163489 || 58 | 20091022_175644_LDC_120601 | mixer6 | 2 | 2 | 302 | 302 | 2914 | 275 | 132 | 118 | 0.158085 |+----+----------------------------+------------+---------------+---------------+----------------------+----------------------+--------+-----------------+-------------+--------------+----------+/raid/users/popcornell/chime7espnet/egs2/chime7_task1/asr1/local/chime7dasr_score.py:589: FutureWarning: In a future version of pandas all arguments of concat except for the argument 'objs' will be keyword-only.###################################################### Metrics for all Scenarios ######################################################+----+------------+---------------+---------------+----------------------+----------------------+--------+-----------------+-------------+--------------+----------+| | scenario | num spk hyp | num spk ref | tot utterances hyp | tot utterances ref | hits | substitutions | deletions | insertions | wer ||----+------------+---------------+---------------+----------------------+----------------------+--------+-----------------+-------------+--------------+----------|| 0 | chime6 | 8 | 8 | 6644 | 6644 | 42748 | 11836 | 4297 | 3090 | 0.326472 || 0 | dipco | 20 | 20 | 3673 | 3673 | 22125 | 5859 | 1982 | 2212 | 0.33548 || 0 | mixer6 | 118 | 118 | 14804 | 14804 | 126617 | 16012 | 6352 | 7818 | 0.20259 |+----+------------+---------------+---------------+----------------------+----------------------+--------+-----------------+-------------+--------------+----------+####################################################################### Macro-Averaged Metrics across all Scenarios (Ranking Metric) #######################################################################+----+---------------+---------------+---------------+----------------------+----------------------+--------+-----------------+-------------+--------------+----------+| | scenario | num spk hyp | num spk ref | tot utterances hyp | tot utterances ref | hits | substitutions | deletions | insertions | wer ||----+---------------+---------------+---------------+----------------------+----------------------+--------+-----------------+-------------+--------------+----------|| 0 | macro-average | 48.6667 | 48.6667 | 8373.67 | 8373.67 | 63830 | 11235.7 | 4210.33 | 4373.33 | 0.288181 |+----+---------------+---------------+---------------+----------------------+----------------------+--------+-----------------+-------------+--------------+----------+
Expand Down