Vocabulary-critic sequence training for image captioning

This repository includes the implementation for Vocabulary-Wide Credit Assignment for Training Image Captioning Models.

Requirements

*python 2.7
*pytorch 1.0
*gensim
*scipy
*cider
*coco-caption

Training

Prepare Data

See details in data/README.md

Start Training

bash train.sh

See opts.py for the options. (You can download the pretrained Up-Down model with VCST from here.)

Evaluation

bash test.sh

Reference

If you find this repo helpful, please consider citing:

@article{liu2021vocabulary,
  title={Vocabulary-Wide Credit Assignment for Training Image Captioning Models},
  author={Liu, Han and Zhang, Shifeng and Lin, Ke and Wen, Jing and Li, Jianmin and Hu, Xiaolin},
  journal={IEEE Transactions on Image Processing},
  volume={30},
  pages={2450--2460},
  year={2021},
  publisher={IEEE}
}