-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[TTS] [黑客松]Add JETS #3109
Merged
Merged
[TTS] [黑客松]Add JETS #3109
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Thanks for your contribution! |
This pull request is now in conflict :( |
lym0302
reviewed
Apr 12, 2023
yt605155624
previously approved these changes
Apr 18, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This pull request is now in conflict :( |
yt605155624
approved these changes
Apr 19, 2023
ctc_loss的输入可以是log_softmax的,可以验证下。 |
luotao1
pushed a commit
to luotao1/PaddleSpeech
that referenced
this pull request
Jun 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR types
New features
PR changes
新增了 jets 模型
Describe
新增 jets 模型,以及 example 的 csmcs ,包括 文档,动态图训练推理,动态图转静态图,推理。
复现过程中存在问题:
Paddle 的 ctcloss 和 torch 的 ctcloss 的接口是不一样的。
torch 的 ctcloss
paddle 的 ctcloss:
虽然 paddle 的 nn.nn.functional.ctc_loss 文档中说明输入是 log_probs, 但似乎不是这样。源码中也是调用 warpctc,而 warpctc 输入是 logits。
综上所述:torch 输入是 log_probs,但是 paddle 输入是 logits。
模型中需要先对 log_probs 做 β伯努利,但是 paddle 的 ctc 由于输入是 logits,不能实现这个步骤,所以不能和原版模型对齐。这导致了自动对齐模块不能实现,因此最终模型采用了 mfa 的对齐结果进行训练。
fix #2773