Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问具体怎么微调模型 #97

Open
huangmeimao opened this issue Jun 22, 2023 · 4 comments
Open

请问具体怎么微调模型 #97

huangmeimao opened this issue Jun 22, 2023 · 4 comments

Comments

@huangmeimao
Copy link

it is recommended to use Infer Loss fine-tune model after base model trained, and freeze PosteriorEncoder during fine-tuni

就是这句话提到的微调,有具体的步骤吗?盼望回复

@MaxMax2016
Copy link
Collaborator

初步训练的时候不加这个loss,模型训练好后,再加上这个loss继续训练~稍微训练一下把kl loss拉下来就好了

@huangmeimao
Copy link
Author

非常感谢您的回复,是不是loss_kl_r = kl_loss(z_r, logs_p, m_q, logs_q, z_mask) * hps.train.c_kl 这个我可以直接注释掉训练?
我在其他问题中看到还需要冻结PosteriorEncoder,是在第二次训练的时候,加上这个loss,并冻结吗?有没有代码可以参考一下

@MaxMax2016
Copy link
Collaborator

数据质量高得话,冻结和不冻结差差不多,数据比较差,冻结保险点。Screenshot_2023-06-24-15-37-49-60_320a9a695de7cdce83ed5281148d6f19.jpg

@tanggang1997
Copy link

数据质量高得话,冻结和不冻结差差不多,数据比较差,冻结保险点。Screenshot_2023-06-24-15-37-49-60_320a9a695de7cdce83ed5281148d6f19.jpg

您好,请问最开始训练的时候都不需要冻结吗,然后在微调时才需要吧,我看您发的图片解码器是冻结的

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants