New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

如何实现基于Iter的Finetune？ #136

Closed

Hades-Z opened this issue Feb 23, 2024 · 1 comment

Hades-Z commented Feb 23, 2024 •

edited

Loading

如下图所示，咱框架给出的微调示例是基于epoch实现的，这不支持梯度累积功能；请问咱框架中，如何实现基于Iter的Finetune训练呢？

Hades-Z changed the title ~~何时支持梯度累积？~~ 如何实现基于Iter的Finetune？

Collaborator

lwmlyy commented Mar 13, 2024

参考https://github.com/alibaba/Pai-Megatron-Patch/blob/main/examples/qwen1.5/run_finetune_megatron_qwen_withGA.sh

jerryli1981 closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment