This is the implementation for TCIC: Theme Concepts Learning Cross Language and Vision for Image Captioning in IJCAI2021.
Our implementation follows Fairseq-v0.9.0 and fairseq-image-captioning.
sudo apt-get install -y openjdk-8-jdk
pip install --user fairseq==0.9.0
pip install --user pandas h5py scikit-learn matplotlib scikit-image sacremoses subword-nmt
export PYTHONIOENCODING=utf-8
You can refer to the fairseq-image-captioning for data downloading and preprocessing.
We store the directory of cider, coco-caption and data-bin in BaiduDisk, code is 3eje. You should download them to TCIC.
You need run the following code in the directory of TCIC.
# Training with Cross-Entropy
bash src/scripts/train/train.sh
# Train with Reinforcement Learning
bash src/scripts/train/train.sh