Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
caffe
example
README.md

README.md

Ask_Attend_and_Answer

Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering

Code

Instructions for training and testing the "SMem-VQA Two-Hop" model:

  1. Download the provided caffe folder and install caffe following the instructions in http://caffe.berkeleyvision.org/installation.html .

  2. Download MSCOCO images, and VQA annotations and questions:

    cd example/data/

    ./get_image.sh

  3. Generate the hdf5 data for training and testing:

    cd example/

    python ./data/generate_h5_data/generate_h5_data.py

  4. Train the model:

    cd example/

    run ./train/train_mm.sh

  5. Model trained on VQA dataset: SMem-VQA

  6. Predict the answers for the images and questions in VQA test-dev dataset:

    cd example/

    python ./prediction/predict_json.py

Citation

@article{xu2015ask,
    Author = {Xu, Huijuan and Saenko, Kate},
    Title = {Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering},
    Journal = {arXiv preprint arXiv:1511.05234},
    Year = {2015}
}