Skip to content
Implementation of Query-guided Regression Network with Context Policy for Phrase Grounding in Tensorflow
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
annotation
feature
log
model add log and model folders Mar 22, 2018
util
LICENSE
README.md
dataprovider.py
evaluate.py
flickr30k_test.lst
flickr30k_train_val.lst
model.py
train.py

README.md

QRC-Net

This repository contains the implementation of Query-guided Regression Network with Context Policy for Phrase Grounding in ICCV 2017.

Setup

Note: Please read the feature representation files in feature and annotation directories before using the code.

Platform: Tensorflow-1.0.1 (python 2.7)
Visual features: We use Faster-RCNN fine-tuned on Flickr30K Entities. Afer fine-tuning, please put visual features in the feature directory (More details and download links are provided in the README.md in this directory).
Sentence features: We encode one-hot vector for each query, as well as the annotation for each query and image pair. Please put the encoded features in the annotation directory (More details are provided in the README.md in this directory).
File list: We generate a file list for each image in the Flickr30K Entities. If you would like to train and test on other dataset (e.g. Referit Game), please follow the similar format in the flickr_train_val.lst and flickr_test.lst.
Hyper parameters: Please check the Config class in the train.py.

Training & Test

For training, please enter the root folder of QRC-Net, then type

$ python train.py -m [Model Name] -g [GPU ID]

For testing, please enter the root folder of QRC-Net, then type

$ python evaluate.py -m [Model Name] -g [GPU ID] --restore_id [Restore epoch ID]

Make sure the model name entered for evaluation is the same as the model name in training, and the epoch id exists.

Reference

If you find the repository is useful for your research, please consider citing the following work:

@InProceedings{Chen_2017_ICCV,
author = {Chen, Kan* and Kovvuri, Rama* and Nevatia, Ram}, 
title = {Query-guided Regression Network with Context Policy for Phrase Grounding},
booktitle = {The IEEE International Conference on Computer Vision (ICCV)},
year = {2017} 
}
You can’t perform that action at this time.