Maybe style_prototype can instead of ref_mel？

[forwiat]

hello @keonlee9420 , thanks for your contribution on StyleSpeech.
When I read your paper and source code, I think that the style_prototype (which is an embedding matrix) maybe can instread of the ref_mel, because there is a CE-loss between style_prototype and style_vector, which can control this embedding matrix close to style. In short, we can give a speaker id to synthesize this speaker's wave. Is it right?

[forwiat]

I have another question: When training in meta-learn-2 by loss2, G is also training. Maybe G should be set required_grad=False when meta-learn-2, D should be set required_grad=False when meta-learn-1?

[keonlee9420]

Hi @forwiat , thank you for your attention. I'd like to note that I'm not the author of the paper; I implemented and published this project before they released the source, so you may refer to it just in case. Anyway, here are my answers to your questions on this repo:

1. Please note that the speaker_id of the naive version is just an addition of my decision, and the original paper does not introduce this. Technically, however, you can replace the style vector solely by speaker id, but the model will lose the controllability from inputting reference audio. This is not a StyleSpeech anymore since one of the purpose of StyleSpeech is to refer to the reference audio to get a target style.
2. I implemented it by filtering out each learnable parameter in the optimizer (please check ScheduledOptimMain and ScheduledOptimDisc).

I hope this helps.

[forwiat]

Hi @keonlee9420 , thanks for the information!
Maybe I can understand that now :) I will continue to set improved experiments.
Last, thanks for your assistance.