About total data performance

[bugczw]

Hi, I am glad to read this article. This essay is the first work that focuses on efficiently adapting large pretrained language
models for image captioning, which inspires me a lot!
In the result display section, it mainly shows the results of training using some data sets at different sampling rates. Therefore, I would like to ask, have you tested the results on all data sets without sampling? How does the performance compare to M2Transformer?

[junchen14]

Hi, I am glad to read this article. This essay is the first work that focuses on efficiently adapting large pretrained language
models for image captioning, which inspires me a lot!
In the result display section, it mainly shows the results of training using some data sets at different sampling rates. Therefore, I would like to ask, have you tested the results on all data sets without sampling? How does the performance compare to M2Transformer?

hi thanks for the interest of our work.
we also tested on training with full COCO training data, but it shows our model's performance is slightly less than M2Transformer, and we have already put this result in the supplementary. We hypotheses the performance drop is due to the domain gap of GPT-2 knowledge and COCO captions.

[qizhust]

Hi, do you have the performance under cross-entropy training only? @junchen14

[junchen14]

Hi, do you have the performance under cross-entropy training only? @junchen14

We did not report the performance under cross-entropy training only, because all the SOTA results are gained under reinforcement setting, we therefore also report the performance after reinforcement.
but through our observation, our visualgpt is always better than the baselines with or without reinforcement.

[qizhust]

Thanks!