Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

这个是instruct tunning还是prompt tuning呢？有改变所有模型权重层吗？ #6

Closed
JetHong opened this issue Mar 27, 2023 · 3 comments
Closed

这个是instruct tunning还是prompt tuning呢？有改变所有模型权重层吗？ #6

JetHong opened this issue Mar 27, 2023 · 3 comments

Comments

@JetHong
Copy link

JetHong commented Mar 27, 2023

No description provided.

@PhoebusSi
Copy link
Owner

instruction tuning，基于LoRA实现的，训练期间不会改变大多数模型权重

@JetHong
Copy link
Author

JetHong commented Mar 27, 2023

感谢回复～，训练好的模型可以开源吗？想尝试一下效果

@PhoebusSi
Copy link
Owner

已经开源。文件夹saved+{instruction类型}+{模型大小}就是保存的LoRA权重。可直接运行generate.py尝试效果。（该脚本中会自动下载和加载full model的权重，其与LoRA权重自动结合后以复现出我们微调后的性能。 ）

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@JetHong @PhoebusSi