Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tuning] iwslt22 low-resource ST decode configuration tuning #5019

Merged
merged 3 commits into from Mar 16, 2023

Conversation

freddy5566
Copy link
Contributor

Summary

Tune decode penalty for better performance in terms of BLUE score.

Results

clean_set: from 2.6 --> 2.7 (BLUE score) / 0.2 --> 0.4 (penalty)
full_set: from 3.6 --> 3.7 (BLUE score) / 0.2 --> 0.3 (penalty)

dataset score verbose_score
decode_st_transformer_st_model_valid.acc.ave/test 2.7 22.1/4.6/1.9/0.8 (BP = 0.756 ratio = 0.782 hyp_len = 17593 ref_len = 22504)

st_full_wav2vec-transformer-warmup-15k

dataset score verbose_score
decode_st_full_transformer_st_model_valid.acc.ave/test 3.7 24.4/5.2/2.0/0.9 (BP = 0.928 ratio = 0.931 hyp_len = 20945 ref_len = 22504)

@mergify mergify bot added the ESPnet2 label Mar 16, 2023
@sw005320 sw005320 added Recipe ST Speech translation labels Mar 16, 2023
@sw005320 sw005320 added this to the v.202303 milestone Mar 16, 2023
@sw005320 sw005320 added the auto-merge Enable auto-merge label Mar 16, 2023
@codecov
Copy link

codecov bot commented Mar 16, 2023

Codecov Report

Merging #5019 (84f8055) into master (b7885c2) will decrease coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5019      +/-   ##
==========================================
- Coverage   77.02%   77.00%   -0.02%     
==========================================
  Files         606      606              
  Lines       53748    53743       -5     
==========================================
- Hits        41399    41386      -13     
- Misses      12349    12357       +8     
Flag Coverage Δ
test_integration_espnet1 66.15% <ø> (-0.15%) ⬇️
test_integration_espnet2 47.96% <ø> (ø)
test_python 66.84% <ø> (-0.01%) ⬇️
test_utils 23.28% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 8 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@mergify mergify bot merged commit 52160d6 into espnet:master Mar 16, 2023
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-merge Enable auto-merge ESPnet2 Recipe ST Speech translation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants