Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

在大模型上有没有什么样的办法同样加入隐向量？ #7

Open
Dreamlikec opened this issue Jun 19, 2023 · 1 comment
Open

在大模型上有没有什么样的办法同样加入隐向量？ #7

Dreamlikec opened this issue Jun 19, 2023 · 1 comment

Comments

@Dreamlikec
Copy link

看到了你们的工作真的给了我很大的启发，我现在想结合你们的代码来做大语言模型的尝试。
VAE的decoder我选择了现在比较实用的chatglm-6B，但是训练的过程中发现loss虽然在下降，但是eval_bleu却始终是0，不知道你们是否在大语言模型中有过类似的尝试，希望可以得到一些经验，谢谢。
@guangyliu
Copy link
Owner

目前还没有在大模型进行尝试

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Dreamlikec @guangyliu