Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ConferencingSpeech 2021 recipe to egs2 #4192

Merged
merged 9 commits into from
Apr 12, 2022

Conversation

Emrys365
Copy link
Collaborator

ConferencingSpeech 2021 is a far-field multi-channel speech enhancement challenge.

The training data includes several English and Mandarin corpora, while test data are semi-real and real recordings in Mandarin.

This PR adds the enh recipe for task2 (Multi-channel speech enhancement with multiple distributed microphone arrays).

@Emrys365 Emrys365 added Recipe ESPnet2 SE Speech enhancement labels Mar 23, 2022
@sw005320 sw005320 added this to the v.0.10.7 milestone Mar 23, 2022
Copy link
Contributor

@sw005320 sw005320 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some minor comments

egs2/README.md Outdated
@@ -19,6 +19,7 @@ See: https://espnet.github.io/espnet/espnet2_tutorial.html#recipes-using-espnet2
| chime4 | The 4th CHiME Speech Separation and Recognition Challenge | ASR/Multichannel ASR | ENG | http://spandh.dcs.shef.ac.uk/chime_challenge/chime2016/ | |
| cmu_indic | CMU INDIC | TTS | 7 languages | http://festvox.org/cmu_indic/ | |
| commonvoice | The Mozilla Common Voice | ASR | 13 languages | https://voice.mozilla.org/datasets | |
| conferencingspeech21 | Far-field Multi-channel Speech Enhancement Challenge for Video Conferencing (ConferencingSpeech 2021) | SE | 2 languages | https://tea-lab.qq.com/conferencingspeech-2021 | |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 languages -> ENG, CMN
?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK. This is more clear.

egs2/conferencingspeech21/enh1/local/data.sh Outdated Show resolved Hide resolved
@sw005320
Copy link
Contributor

@LiChenda, can I ask you to review this PR?

@mergify mergify bot added the README label Mar 23, 2022
@LiChenda
Copy link
Contributor

@sw005320 Sure, I can review this PR.

path_noise = noise_data[path_noise]
out.write(
f"{path_clean} {start_time} {path_noise} "
f"/path/{args.tag}/{path_rir}.wav {snr} {scale}\n"
Copy link
Contributor

@LiChenda LiChenda Mar 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does "/path/" exist on filesystem?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a placeholder. We don't actually use this path for simulation. Because this python script is intended for generating the simulation configuration from an already generated data.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see.

Copy link
Contributor

@LiChenda LiChenda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@codecov
Copy link

codecov bot commented Apr 2, 2022

Codecov Report

Merging #4192 (bc486fd) into master (56c1c0d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #4192   +/-   ##
=======================================
  Coverage   80.71%   80.71%           
=======================================
  Files         453      453           
  Lines       39575    39575           
=======================================
  Hits        31944    31944           
  Misses       7631     7631           
Flag Coverage Δ
test_integration_espnet1 67.13% <ø> (ø)
test_integration_espnet2 50.05% <ø> (ø)
test_python 67.16% <ø> (ø)
test_utils 24.45% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@Emrys365
Copy link
Collaborator Author

Emrys365 commented Apr 2, 2022

The error in https://github.com/espnet/espnet/runs/5798659465?check_suite_focus=true#step:8:6228 seems not related to this PR.
Now it is all clear.

@Emrys365
Copy link
Collaborator Author

Emrys365 commented Apr 11, 2022

The error in https://github.com/espnet/espnet/runs/5962161616?check_suite_focus=true#step:7:762 seems not related to this PR either.

# /__w/espnet/espnet/test_utils/test_update_json_sh.bats: line 580: jsondiff: command not found

@kan-bayashi kan-bayashi modified the milestones: v.0.10.7, v.202205 Apr 11, 2022
@sw005320
Copy link
Contributor

Thanks, @Emrys365!
Can you also prepare the README file with its result and upload the model link?

@sw005320 sw005320 merged commit d263372 into espnet:master Apr 12, 2022
@Emrys365
Copy link
Collaborator Author

Thanks, @Emrys365! Can you also prepare the README file with its result and upload the model link?

I was not sure if it is appropriate to upload my local model. Because it was trained based on a local branch of espnet, where a customized Preprocessor was defined and used in espnet2/train/preprocessor.py.

This customized Preprocessor was intended for on-the-fly data simulation, which cannot be achieved with current main branch.

@sw005320
Copy link
Contributor

I see.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants