Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 

Binary Image SelectiON (BISON)

This repository implements an evaluation script for measuring the mean BISON accuracy on the val2014 split of the COCO-caption dataset. If you use BISON to analyze your image-captioning system, please cite:

  • Hexiang Hu, Ishan Misra, and Laurens van der Maaten. Binary Image Selection (BISON): Interpretable Evaluation of Visual Grounding. _arXiv:1901.06595, 2019.

Requirements

  • Python 3+
  • Numpy 1.10.0+

Usage

Please put your prediction file (format as shown in later section) in the folder predictions/, and specify the current annotation filepath, as well as the prediction filepath. The usage is listed as following:

python bison_eval.py [-h] [--anno_path ANNO_PATH] [--pred_path PRED_PATH]

optional arguments:
  -h, --help            show this help message and exit
  --anno_path ANNO_PATH
                        Path to the annotation file
  --pred_path PRED_PATH
                        Path to the prediction file

File format for predictions

The model predictions used as input into the BISON evaluation script should be in the following file format:

[
	{
		"bison_id": 0,
		"predicted_image_id": 50965,
	},
	...
]

References

License

BISON is CC-BY-NC 4.0 licensed, as found in the LICENSE file.

Terms of Use

Privacy Policy

About

BISON: Binary Image SelectiON

Resources

License

Releases

No releases published

Packages

No packages published

Languages