some questions about premodel

[ld0714]

Thanks for your model ,I want to ask you about the specific details of training premodel.
Q1:What's the actual size of the dataset used to trained in premodel?
Q2:How many the number of used GPUS when training and how long is the training?
Q3:How many the steps of pre-training

[Shun14]

Sorry for the late answer.

1. I used the synth80k to train the pre-model
2. 1 gpu with batch size 16(384 scale). I don't remember the precise time , maybe 24 hours.
3. I choose this model around 12w steps.

[novioleo]

Sorry for the late answer.

1. I used the synth80k to train the pre-model
2. 1 gpu with batch size 16(384 scale). I don't remember the precise time , maybe 24 hours.
3. I choose this model around 12w steps.

老外就不会懂12w了。。。。233333.