ESPnet recipe for the Kinect-WSJ dataset #5711

atharva253 · 2024-03-20T18:45:12Z

What?

This is a new recipe for preparing the Kinect-WSJ dataset and training speech enhancement/separation models on it.

Details about the dataset:

Kinect-WSJ is a multichannel, reverberated and noisy extension to the WSJ0-2mix dataset. It is designed to simulate challenging acoustic environments through strong reverberation and noise conditions along with the Kinect-like microphone array geometry used in CHiME-5.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 76.60%. Comparing base (dbd73dd) to head (51d28ca).
Report is 5 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #5711   +/-   ##
=======================================
  Coverage   76.60%   76.60%           
=======================================
  Files         761      761           
  Lines       69880    69880           
=======================================
  Hits        53534    53534           
  Misses      16346    16346

Flag	Coverage Δ
test_configuration_espnet2	`∅ <ø> (∅)`
test_integration_espnet1	`62.92% <ø> (ø)`
test_integration_espnet2	`48.84% <ø> (ø)`
test_integration_espnetez	`27.98% <ø> (ø)`
test_python_espnet1	`18.20% <ø> (ø)`
test_python_espnet2	`52.41% <ø> (ø)`
test_python_espnetez	`13.95% <ø> (ø)`
test_utils	`20.91% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sw005320 · 2024-03-22T00:23:46Z

@atharva253, please fix the CI errors https://github.com/espnet/espnet/actions/runs/8364289940/job/22899257857?pr=5711#step:8:605

for more information, see https://pre-commit.ci

atharva253 · 2024-03-22T00:43:32Z

@sw005320 Sure, I'll fix the CI errors and get back

for more information, see https://pre-commit.ci

Emrys365

LGTM! Could you fix the minor comments I just left?

Emrys365 · 2024-03-22T15:31:27Z

egs2/wsj_kinect/enh1/local/create_corrupted_speech_parallel.sh

+            # Put an & at the end of next line if you want run the wav creation in parallel using a single machine. It will spawn 50, 13 and 7 jobs for train dev and test
+            # You can also use cluster managers such as SLURM/SGE/OAR with this command
+            # python create_corrupted_speech.py $wsj_mix_list $start $file_end $wsj2_mix_base $noise_list $src_count $rir_yaml_list $SNR $dest_base  || exit 1;
+            python create_corrupted_speech.py $wsj_mix_list $start $file_end $wsj2_mix_base $noise_list $src_count $rir_yaml_list $SNR $dest_base  || exit 1 &


If you want to use parallel background jobs here, could you follow the coding style in https://github.com/espnet/espnet/blob/master/egs/wsj/asr1/run.sh#L308-L312 to add some checks to make sure all background jobs are finished without error?

Added checks to make sure all background jobs are finished without error.

Emrys365 · 2024-03-22T15:33:08Z

egs2/wsj_kinect/enh1/local/data.sh

+wget --continue -O $wdir/mixture_scripts.zip ${url}
+unzip $wdir/mixture_scripts.zip -d $wdir


How about just doing git clone ... to fetch the repository into $wdir?

Replaced wget with git clone in data.sh

Emrys365 · 2024-03-22T15:33:27Z

egs2/wsj_kinect/enh1/local/data.sh

+
+wget --continue -O $wdir/mixture_scripts.zip ${url}
+unzip $wdir/mixture_scripts.zip -d $wdir
+#chmod 700 local/create_corrupted_speech_parallel.sh $wdir/Reverberated_WSJ_2MIX-master/create_corrupted_speech.sh


Remove this line?

The chmod line is now removed.

Emrys365 · 2024-03-22T15:35:03Z

egs2/wsj_kinect/enh1/local/data.sh

+}
+
+help_message=$(cat << EOF
+Usage: $0


Could you add the documentation of the arguments below in this help message?

Sure, I'll add the documentation for the arguments. Since the min_or_max and sample_rate are fixed for Kinect WSJ, data.sh only has parallel and use_dereverb as arguments.

Added the documentation for the parallel and use_dereverb arguments.

egs2/wsj_kinect/enh1/local/wsj_kinect_data_prep.sh

Emrys365 · 2024-03-22T15:36:19Z

egs2/wsj_kinect/enh1/local/wsj_kinect_data_prep.sh

+ # elif [ ! -d f ]; then
+ #   echo "Error: $f is not a directory."
+ #   exit 1;


Removed the commented portion.

Emrys365 · 2024-03-22T15:36:29Z

egs2/wsj_kinect/enh1/local/wsj_kinect_data_prep.sh

+
+# Ensure that the wav dir exists
+for f in "$wavdir/$tr" "$wavdir/$cv" "$wavdir/$tt"; do
+ # echo "$f"


Removed the commented line.

egs2/wsj_kinect/enh1/run.sh

atharva253 · 2024-03-22T19:08:05Z

Thanks for your comments @Emrys365! The portion for the parallel background jobs might need some time for testing after the modifications. I'll complete all the fixes during the weekend and update on the same.

Co-authored-by: Wangyou Zhang <C0me_On@163.com>

…ground jobs

sw005320 · 2024-03-24T22:10:25Z

Thanks, @atharva253!
After the CI is passed, I’ll merge this PR.

sw005320 · 2024-03-25T02:57:56Z

It seems that the error is reproducible.
https://github.com/espnet/espnet/actions/runs/8409685020/job/23033292927?pr=5711
We should not change egs2/wsj_kinect/enh1/conf/slurm.conf from the original one, and this would be a reason.
Can you check it?

Atharva Anand Joshi and others added 11 commits January 31, 2024 19:51

create recipe wsj_kinect and add code for WSJ mixture

0c7034b

Merge branch 'espnet:master' into master

67f221c

Completed data prepration scripts

96ea940

fixed wsj_kinect_data_prep.sh

460b0d0

Added option for reverb/dereverb references and completed final test …

25e3496

…for data.sh

Add results for train_enh_tfgridnetv2_tf_lr-patience3_patience5_I_1_J…

adac3cc

…_1_D_128_batch_8.yaml

Merge branch 'espnet:master' into master

9adc575

Preparing recipe for PR

b38672c

Updated db.sh and egs2/README.md

41610a7

Fix nj in run.sh

2bcd462

Updated results after epoch 39

80c3555

mergify bot added ESPnet2 README labels Mar 20, 2024

sw005320 added this to the v.202405 milestone Mar 20, 2024

sw005320 added Recipe SE Speech enhancement labels Mar 20, 2024

sw005320 requested a review from Emrys365 March 20, 2024 18:48

atharva253 and others added 2 commits March 21, 2024 20:37

Merge branch 'master' into master

266f76b

[pre-commit.ci] auto fixes from pre-commit.com hooks

c9bd69d

for more information, see https://pre-commit.ci

atharva253 and others added 2 commits March 21, 2024 22:52

Fixed test_shell CI errors

37828ea

[pre-commit.ci] auto fixes from pre-commit.com hooks

91182ed

for more information, see https://pre-commit.ci

Emrys365 approved these changes Mar 22, 2024

View reviewed changes

atharva253 and others added 4 commits March 23, 2024 11:00

Minor fixes based on code review

3b7bba5

Co-authored-by: Wangyou Zhang <C0me_On@163.com>

Cleaned wsj_kinect_data_prep.sh and added local_data_opts in run.sh

adca288

Cleaned data.sh and added documentation for arguments

cf7a34f

Added checks to ensure the successful completion of all parallel back…

cf1a9c4

…ground jobs

Emrys365 approved these changes Mar 24, 2024

View reviewed changes

sw005320 added the auto-merge Enable auto-merge label Mar 24, 2024

Fix slurm.conf

51d28ca

mergify bot merged commit eed7751 into espnet:master Mar 25, 2024
35 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ESPnet recipe for the Kinect-WSJ dataset #5711

ESPnet recipe for the Kinect-WSJ dataset #5711

atharva253 commented Mar 20, 2024

codecov bot commented Mar 20, 2024 •

edited

sw005320 commented Mar 22, 2024

atharva253 commented Mar 22, 2024

Emrys365 left a comment

Emrys365 Mar 22, 2024

atharva253 Mar 24, 2024

Emrys365 Mar 22, 2024

atharva253 Mar 23, 2024

Emrys365 Mar 22, 2024

atharva253 Mar 23, 2024

Emrys365 Mar 22, 2024

atharva253 Mar 23, 2024 •

edited

atharva253 Mar 23, 2024

Emrys365 Mar 22, 2024

atharva253 Mar 23, 2024

Emrys365 Mar 22, 2024

atharva253 Mar 23, 2024

atharva253 commented Mar 22, 2024

sw005320 commented Mar 24, 2024

sw005320 commented Mar 25, 2024

		wget --continue -O $wdir/mixture_scripts.zip ${url}
		unzip $wdir/mixture_scripts.zip -d $wdir

ESPnet recipe for the Kinect-WSJ dataset #5711

ESPnet recipe for the Kinect-WSJ dataset #5711

Conversation

atharva253 commented Mar 20, 2024

What?

Details about the dataset:

Related Links

codecov bot commented Mar 20, 2024 • edited

Codecov Report

sw005320 commented Mar 22, 2024

atharva253 commented Mar 22, 2024

Emrys365 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atharva253 Mar 23, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atharva253 commented Mar 22, 2024

sw005320 commented Mar 24, 2024

sw005320 commented Mar 25, 2024

codecov bot commented Mar 20, 2024 •

edited

atharva253 Mar 23, 2024 •

edited