Skip to content
Branch: master
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
cocoapi
data
lib
other-projects
.gitignore
README.md
show_data.py

README.md

Fast Parameter Adaptation for Few-shot Image Captioning and Visual Question Answering

By Xuanyi Dong, Linchao Zhu, De Zhang, Yi Yang, Fei Wu

Data Preparation

Download

Make directory at ~/datasets/MS-COCO.

- ~/datasets/MS-COCO
-- annotations
--- captions_train2014.json captions_val2014.json image_info_test2014.json instances_train2014.json instances_val2014.json
-- test2014
-- trainval2014

Compile coco api

cd cocoapi
make

Few-shot Image Caption

In the directory data, run:

python Generate_Caption.py

After run the above command, you can obtain data/COCO-Caption/few-shot-coco.pth for few-shot image caption.

Few-shot Visual Question Answering

In the directory data, run:

python Generate_VQA.py

After run the above command, you can obtain data/Toronto-COCO-QA/object.pth for few-shot visual question answering.

Show Samples

We give an example to show how to read the pre-processed data

python show_data.py

Citation

If you find this project help your research, please cite:

@inproceedings{dong2018fpait,
  title     = {Fast Parameter Adaptation for Few-shot Image Captioning and Visual Question Answering},
  author    = {Dong, Xuanyi and Zhu, Linchao and Zhang, De and Yang, Yi and Wu, Fei},
  booktitle = {Proceedings of the 2018 ACM on Multimedia Conference},
  year      = {2018}
}
You can’t perform that action at this time.