Skip to content
No description, website, or topics provided.
Python Shell
Branch: master
Clone or download
Latest commit 03f106f May 4, 2018
Type Name Latest commit message Commit time
Failed to load latest commit information. Update May 3, 2018 Create Dec 13, 2017 Create Dec 13, 2017 Create Dec 13, 2017 Create Dec 13, 2017 Create Dec 13, 2017

Two-Branch Neural Networks


  • Train a model from scratch: Set the path to the training dataset and checkpoint save directory in Run sh --train.
  • Evaluate a model on the validation set while training: Set the path to validation dataset and checkpoint save directory. Additionally, adjust the time interval between each evaluation in By default, the script will evaluate once per minute on the newest checkpoint in the given checkpoint directory. Run sh --val.
  • Evaluate a model on a specific checkpoint: Set the path to the test dataset and checkpoint MetaGraph (.meta file). Run sh --test.
  • Use a pre-trained model: Download checkpoints from the URLs below. Follow the instruction for evaluating model on a specific checkpoint.

Any questions, feel free to contact me,


Due to the size of the features (~17G for MSCOCO and 7G for Flickr30K), only the test split is available for download.

Data set splits:

  • Train, and test splits (the left is used for validation), please check readme.txt download

To extract images and sentences features,

  • For image features, we use the VGG 19 network to extract features.
  • For sentence features, we are using the code from "B. Klein, G. Lev, G. Sadeh, L. Wolf. Associating Neural Word Embeddings With Deep Image Representations Using Fisher Vectors (CVPR 2015)". Their code can be downloaded here.

Pre-trained models:

If you find our code helpful, please cite our Two-Branch Network Papers:


title={Learning deep structure-preserving image-text embeddings},
author={Wang, Liwei and Li, Yin and Lazebnik, Svetlana},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},

title={Learning Two-Branch Neural Networks for Image-Text Matching Tasks},
author={Wang, Liwei and Li, Yin and Lazebnik, Svetlana},
journal={arXiv preprint arXiv:1704.03470},
You can’t perform that action at this time.