Question about pretraining

[PipiZong]

Thanks for sharing your great work.

Just wondering which model do you select for the downstream task since you save pertaining models every 40 epochs.

Thank you.

[BestJuly]

Thank you for your interest.

For the pretraining part, I am just using the last epoch.
I have tried using different checkpoints for downstream task, the differences are limited, and there exist a better checkpoint. However, that is tricky and based on the testing performance. Therefore, only the last checkpoint will be used.