Some results of the VQA-T model are not reproducible

[fake-warrior8]

Hi, I ran the VQA-T model from scratch using the command

python main_videoqa.py --checkpoint_dir=ft<dataset> --dataset=<dataset> --lr=0.00001 \ 
--pretrain_path=<CKPT_PATH>

On MSRVTT-QA, MSVD-QA, Anet-QA, How2QA and iVQA, I got the following results: 40.2, 41.5, 33.8, 71.4 and 15.7, while the paper showed 39.6, 41.2, 36.8, 80.8, 23.0. Do the settings of Anet-QA, How2QA and iVQA use some different hyperparameters?

[antoyang]

Hi, all the hyperparameters I can remember of are provided in the paper. You may try neighboring learning rates to see if that helps. For reproducing the main results of the paper, you may use the pretrained checkpoints.

[fake-warrior8]

Hi, all the hyperparameters I can remember of are provided in the paper. You may try neighboring learning rates to see if that helps. For reproducing the main results of the paper, you may use the pretrained checkpoints.

It works, thank you!