Some results issue

[viyjy]

Hi Junnan, I have the following questions about the result. Hope that you can help to clarify them, thanks.

1. VQA:
   I get the result folder after fine-tuning on VQA dataset. Which json file should I use to get test-dev and test-std?
   

2. SNLI-VE:
   This is the log file after fine-tuning on SNLI-VE dataset. You didn't update the best-epoch, so it is always 0. Should I pick the row which has the best val accuracy as the final result?
   

3. Grouding
   This is the log file after fine-tuning on Ref-COCO. Should I pick the row which has the best val_d as the final result?
   

4. NLVR2
   This is the log file after fine-tuning on NLVR2, but I did't find dev and test-P as shown in your paper, any idea?
   

[LiJunnan1992]

Hi, thanks for your interest. Here are my answers.

1. VQA: vqa_result_epoch7.json is the final result which collects results from all ranks.
2. SNLI-VE: thanks for spotting my mistake. Yes you should use the row with best val_acc.
3. Grounding: if I remembered correctly, in the paper I just reported the last epoch's result.
4. NLVR: dev is val, test-P is test.

[viyjy]

Thanks very much!

[haoshuai714]

Hello! Can NLVR2 datasets be shared?

[viyjy]

You can find it here: https://github.com/lil-lab/nlvr/tree/master/nlvr2