hi，when will the pre-train related codes&scripts be released?

[XL2248]

I see you have released the mmpretrain file. When will you plan to release all the code and scripts? I believe this will benefit the community a lot and gain much researchers' interest in your great work like MOSS. Besides, how many A100 GPUs were used in the pre-training stage, and how much time did it cost?

[JunZhan2000]

In fact, you only need to use the code for training LLM. I will sort out the code and release it in the next two weeks.
The pre-training took 96 80G A100 for a week, and the instruction fine-tuning was much less.

[XL2248]

Hi, are the codes and scripts ready to release?

[JunZhan2000]

Hi, are the codes and scripts ready to release?

We have uploaded the relevant scripts for reference

[JunZhan2000]

Hello, we provide some training data samples and related descriptions, please refer to https://github.com/OpenMOSS/AnyGPT?tab=readme-ov-file#pretraining-and-sft