Reproduce paper's scores

[bradduy]

Hi everyone,

Is there anyone can reproduce accuracy scores in the paper?
I have tried to use 2 Nvidia Geforce 3090 but the scores were dropped by 5%.

How to achieve paper's accuracy?

Thanks.

[lxtGH]

Hi! Please follow our README strictly by first pre-training and then fine-tune.

Moreover, we use 8 A100 card for both stage. Batchsize is an important factor for the final performance.

[bradduy]

Thank you.