BISON: Binary Image SelectiON
Branch: master
Clone or download
Latest commit 24154f3 Jan 23, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
annotations Initial commit Jan 18, 2019
predictions Initial commit Jan 18, 2019 Initial commit Jan 18, 2019 Initial commit Jan 18, 2019
LICENSE Initial commit Jan 18, 2019 Update Jan 23, 2019 Initial commit Jan 18, 2019

Binary Image SelectiON (BISON)

This repository implements an evaluation script for measuring the mean BISON accuracy on the val2014 split of the COCO-caption dataset. If you use BISON to analyze your image-captioning system, please cite:

  • Hexiang Hu, Ishan Misra, and Laurens van der Maaten. Binary Image Selection (BISON): Interpretable Evaluation of Visual Grounding. _arXiv:1901.06595, 2019.


  • Python 3+
  • Numpy 1.10.0+


Please put your prediction file (format as shown in later section) in the folder predictions/, and specify the current annotation filepath, as well as the prediction filepath. The usage is listed as following:

python [-h] [--anno_path ANNO_PATH] [--pred_path PRED_PATH]

optional arguments:
  -h, --help            show this help message and exit
  --anno_path ANNO_PATH
                        Path to the annotation file
  --pred_path PRED_PATH
                        Path to the prediction file

File format for predictions

The model predictions used as input into the BISON evaluation script should be in the following file format:

		"bison_id": 0,
		"predicted_image_id": 50965,



BISON is CC-BY-NC 4.0 licensed, as found in the LICENSE file.