How to reproduce the result on QNLI?

[zdou0830]

Hi, I am having trouble reproducing the result on the QNLI dataset. How did you choose the hyper-parameters?

[namisan]

The result on QNLI reported in our paper was on the GLUE old version (please refer GLUE official homepage for more details). As in the paper, we treat QNLI as a ranking task instead of a classification task, which gave a great improvement. However, this is not a case in the current version. If you are able to find old QNLI datasets, you should be able get the results as reported in the paper, e.g., use qnnli tasks as our codes.