-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Obtaining unusual alignment results while using the ESPnet2 Branchformer model. #30
Comments
Hey Tien-Hong, I assume that your screenshots include the alignments with the corresponding token score?
Inspection of the Phone-level tokens:
I recommend to re-check the following parameters:
Also other issues may cause such misalignments that may be model-related; Give me a few days to find the time to investigate Branchformer alignments with an English language model. |
Thank you for your kind response. I have evaluated our trained model's CTC performance and verified the audio's sampling rate. All configs I used are listed: Both the CTC-ATT and CTC decoding performance (w/ the ctc suffix) of the model (Additionally, I attempted using the conformer-type encoder, but unfortunately, the alignment results remained as poor as the branchformer-type encoder.): BPE-level tokensexp/asr_train_asr_branchformer_raw_en_bpe735_spWER
Phone-level tokensexp/asr_train_asr_branchformer_raw_en_word_spWER
Do you have any further suggestions? Thank you in advance. |
Firstly, I want to express my admiration for the exceptional work accomplished here!
Recently, I've been facing the issue while using the ESPnet2 Branchformer model.
Despite following the instructions on here, I encountered poor alignment results.
This results occurred when I trained the model with phone-level transcriptions.
To understand this issue further, I experimented with two different token types, the details of which are as follows:
The accuracy of the two models is 95+%.
BPE-level tokens:
Phone-level tokens:
I would appreciate your guidance and insights to help me resolve these alignment issues.
Thank you in advance.
Tien-Hong
The text was updated successfully, but these errors were encountered: