casual reasoning performance

[phoenixsecularbird]

Hello！It is really an excellent work. I rerun your code for casual reasoning, and obtain accuracy about 66% on the development set, with BERT_base encoder. This is far lower than the performance on the test set, which is more than 75% according to your report. What may be the matter? Or your result? Thanks a lot!

[Waste-Wood]

Sorry to miss your messages and thanks for following our work.
I have rerun the code on my device, and the performance of bert-base-cased can achieve similar accuracy (more than 75%) on the test and dev sets. If you can provide the training log, it may be helpful to find the problem.

[phoenixsecularbird]

I have got it! Which version of AlBert do you use?

[Waste-Wood]

We use albert-base-v2 for the ALBERT implementation.