Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DCCRN separator #4097

Merged
merged 17 commits into from
Mar 3, 2022
Merged

Add DCCRN separator #4097

merged 17 commits into from
Mar 3, 2022

Conversation

Johnson-Lsx
Copy link
Contributor

Add espnet2/enh/layers/complexnn.py and espnet2/enh/separator/dccrn_separator.py and modified espnet2/tasks/enh.py to implement DCCRN separator, add a configuration file of dccrn for egs2/dns_ins20.

@mergify mergify bot added the ESPnet2 label Feb 21, 2022
@sw005320 sw005320 added New Features SE Speech enhancement labels Feb 21, 2022
@sw005320 sw005320 added this to the v.0.10.7 milestone Feb 21, 2022
Copy link
Collaborator

@Emrys365 Emrys365 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Could you add a unit test for DCCRN in test/espnet2/enh? e.g. test/espnet2/enh/test_espnet_model.py and test/espnet2/enh/separator

espnet2/enh/layers/complexnn.py Outdated Show resolved Hide resolved
espnet2/enh/layers/complexnn.py Outdated Show resolved Hide resolved
espnet2/enh/layers/complexnn.py Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Feb 26, 2022

Codecov Report

Merging #4097 (f587f4d) into master (812ab14) will decrease coverage by 0.50%.
The diff coverage is 90.10%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4097      +/-   ##
==========================================
- Coverage   80.94%   80.43%   -0.51%     
==========================================
  Files         435      442       +7     
  Lines       37651    38557     +906     
==========================================
+ Hits        30477    31015     +538     
- Misses       7174     7542     +368     
Flag Coverage Δ
test_integration_espnet1 67.13% <ø> (ø)
test_integration_espnet2 51.14% <12.56%> (-0.94%) ⬇️
test_python 66.51% <90.10%> (-0.18%) ⬇️
test_utils 24.45% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
espnet2/enh/layers/complexnn.py 84.58% <84.58%> (ø)
espnet2/enh/separator/dccrn_separator.py 100.00% <100.00%> (ø)
espnet2/tasks/enh.py 99.07% <100.00%> (+0.01%) ⬆️
espnet2/enh/layers/tcn.py 95.90% <0.00%> (-0.37%) ⬇️
espnet2/enh/layers/dprnn.py 100.00% <0.00%> (ø)
espnet2/bin/mt_train.py 0.00% <0.00%> (ø)
espnet2/enh/separator/skim_separator.py 97.05% <0.00%> (ø)
espnet2/tasks/mt.py 0.00% <0.00%> (ø)
espnet2/enh/layers/skim.py 99.17% <0.00%> (ø)
espnet2/bin/mt_inference.py 0.00% <0.00%> (ø)
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 812ab14...f587f4d. Read the comment docs.

@mergify mergify bot added the README label Feb 27, 2022
Copy link
Contributor

@sw005320 sw005320 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks!
This is a great job!
I have some minor comments.

Also, you may need to merge the latest PR to pass the CI.

@@ -0,0 +1,436 @@
import torch
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I missed this function.
This is really cool!
Please add the description about dccrn and this new function in https://github.com/espnet/espnet#se-speech-enhancement-and-separation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have added the DCCRN and the link of the original paper in the README

- pytorch version: `pytorch 1.9.1`
- Git hash: `6f66283b9eed7b0d5e5643feb18d8f60118a4afc`
- Commit date: `Mon Dec 13 15:30:29 2021 +0800`

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you upload a model to the HF hub and add a link here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I have uploaded the model to the HF hub, and the link is https://huggingface.co/Johnson-Lsx/Shaoxiong_Lin_dns_ins20_enh_enh_train_enh_dccrn_raw

@sw005320 sw005320 merged commit a04a98c into espnet:master Mar 3, 2022
@sw005320
Copy link
Contributor

sw005320 commented Mar 3, 2022

Thanks a lot!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants