[Recipe PR] MELD: Multimodal EmotionLines Dataset

[realzza]

About

Added baseline recipe for MELD: Multimodal EmotionLines Dataset

Checklist

• (meld) be careful about the name for the recipe. It is recommended to follow naming conventions of the other recipes
• common/shared files are linked with soft link (see Section 1.3.3)
• modified or new python scripts should be passed through latest black formating (by using python package black). The command to be executed could be black espnet espnet2 test utils setup.py egs*/*/*/local egs2/TEMPLATE/*/pyscripts tools/*.py ci/*.py
• modified or new python scripts should be passed through latest isort formating (by using python package isort). The command to be executed could be isort espnet espnet2 test utils setup.py egs*/*/*/local egs2/TEMPLATE/*/pyscripts tools/*.py ci/*.py
• cluster settings should be set as default (e.g., cmd.sh conf/slurm.conf conf/queue.conf conf/pbs.conf)
• update egs/README.md or egs2/README.md with corresponding recipes
• add corresponding entry in egs2/TEMPLATE/db.sh for a new corpus
• try to simplify the model configurations. We recommend to have only the best configuration for the start of a recipe. Please also follow the default rule defined in Section 1.3.3
• large meta-information for a corpus should be maintained elsewhere other than in the recipe itself
• recommend to also include results and pre-trained model with the recipe
  • results
  • (TODO) model

Todo

• submit model through huggingface

[sw005320]

@siddhu001, it would be great if you also review this PR.

[ftshijt]

Just to confirm: it seems that this is only an unimodal model. Is it intentional or you will keep update it.

I also recommend you to check the task of SLU, which could be a better place to hold your recipe as well (we can keep on this).

[siddhu001]

I would suggest incorporating all of Jiatong's feedback. I added a comment and clarification, rest looks good to me.

[realzza]

Just to confirm: it seems that this is only an unimodal model. Is it intentional or you will keep update it.

Yes, this recipe is designed to use unimodal to do multitask learning is ASR and SLU. We may not update MELD to involve multimodals.

I also recommend you to check the task of SLU, which could be a better place to hold your recipe as well (we can keep on this).

Thanks for mentioning, I categorized this recipe as SLU in egs2/README.md.

[sw005320]

@ftshijt, can you review it again?
I think this PR is almost ready.

[ftshijt]

Looks cool! After it passed the CI tests, I will merge it

[realzza]

Seems it fails the CI test, as well as centos7 and debian9 tests due to the overlength lines in data_prep.py, let me fix this format issue shortly!

update: should be fixed now!

[realzza]

Hi @ftshijt, thank you for your review! I fixed the flake8 error W605 invalid escape sequence '\-', we can rerun the CI tests.

[codecov]

Codecov Report

Merging #4771 (fbfe277) into master (ca2193d) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #4771   +/-   ##
=======================================
  Coverage   80.32%   80.32%           
=======================================
  Files         530      530           
  Lines       46527    46527           
=======================================
  Hits        37372    37372           
  Misses       9155     9155           

Flag	Coverage Δ
test_integration_espnet1	66.37% <ø> (ø)
test_integration_espnet2	48.87% <ø> (ø)
test_python	68.62% <ø> (ø)
test_utils	23.30% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[realzza]

The failed check may be caused by this issue reported in the latest version of numpy.

[sw005320]

Thanks, @realzza!