Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

训练之后存在灾难性遗忘问题 #19

Closed
uloveqian2021 opened this issue Jun 7, 2023 · 2 comments
Closed

训练之后存在灾难性遗忘问题 #19

uloveqian2021 opened this issue Jun 7, 2023 · 2 comments
Labels
solved This problem has been already solved.

Comments

@uloveqian2021
Copy link

大佬好，模型微调之后，之前的功能全部丢失了，只会做微调训练数据中的任务，这个怎么解决。采用的是默认配置，模型是ziya-llama, 1W条数据，3 epoch
@hiyouga
Copy link
Owner

hiyouga commented Jun 7, 2023

微调发生灾难性遗忘是正常现象，考虑减少 epoch 数可以缓解遗忘问题。

@hiyouga hiyouga added the pending This problem is yet to be addressed. label Jun 7, 2023
@uloveqian2021
Copy link
Author

微调发生灾难性遗忘是正常现象，考虑减少 epoch 数可以缓解遗忘问题。

好的，感谢！

@hiyouga hiyouga added solved This problem has been already solved. and removed pending This problem is yet to be addressed. labels Jun 7, 2023
@hiyouga hiyouga closed this as completed Jun 7, 2023
@godfly godfly mentioned this issue Aug 17, 2023
Closed
@YananSunn YananSunn mentioned this issue Aug 31, 2023
Closed
@liwenju0 liwenju0 mentioned this issue Sep 18, 2023
Closed
@xiechengmude xiechengmude mentioned this issue Feb 15, 2024
Closed
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
solved This problem has been already solved.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hiyouga @uloveqian2021