Skip to content
PyTorch code for Fast Parameter Adaptation for Few-shot Image Captioning and Visual Question Answering
Branch: master
Clone or download
Latest commit 5011c74 Feb 3, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
cocoapi
data X Aug 9, 2018
lib update README Jul 11, 2018
other-projects
.gitignore update README Jul 11, 2018
README.md
show_data.py X Aug 9, 2018

README.md

Fast Parameter Adaptation for Few-shot Image Captioning and Visual Question Answering

By Xuanyi Dong, Linchao Zhu, De Zhang, Yi Yang, Fei Wu

Data Preparation

Download

Make directory at ~/datasets/MS-COCO.

  • download the ms-coco train, val, and test.
  • download the trainval2014-annotation, and the test info.
  • organize the data as follows, where trainval2014 contains all the trainval images and test2014 contains all the test images.
- ~/datasets/MS-COCO
-- annotations
--- captions_train2014.json captions_val2014.json image_info_test2014.json instances_train2014.json instances_val2014.json
-- test2014
-- trainval2014

Compile coco api

cd cocoapi
make

Few-shot Image Caption

In the directory data, run:

python Generate_Caption.py

After run the above command, you can obtain data/COCO-Caption/few-shot-coco.pth for few-shot image caption.

Few-shot Visual Question Answering

In the directory data, run:

python Generate_VQA.py

After run the above command, you can obtain data/Toronto-COCO-QA/object.pth for few-shot visual question answering.

Show Samples

We give an example to show how to read the pre-processed data

python show_data.py

Citation

If you find this project help your research, please cite:

@inproceedings{dong2018fpait,
  title     = {Fast Parameter Adaptation for Few-shot Image Captioning and Visual Question Answering},
  author    = {Dong, Xuanyi and Zhu, Linchao and Zhang, De and Yang, Yi and Wu, Fei},
  booktitle = {Proceedings of the 2018 ACM on Multimedia Conference},
  year      = {2018}
}
You can’t perform that action at this time.