Skip to content
Official code for paper Context-aware Zero-shot Recognition ( to appear at AAAI2020)
Python MATLAB C++ Cuda C Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Update May 24, 2018
.vscode Add evaluation script Apr 20, 2018
data/vg add data Nov 6, 2019
demo Add May 10, 2018
lib squashed. Apr 7, 2019
.gitignore squashed. Apr 7, 2019
LICENSE Update Apr 29, 2019


This repository includes the code for paper Context-aware zero-shot recognition.

The code is highly based on Detectron.pytorch. Thanks for the effort from original author.


  • Pytorch 1.0
  • Python 2 and python 3
  • torchtext
  • nltk
  • pycocotools


Download dataset

bash data/vg/

Convert from bansal train test split.

(In python 2)
python lib/datasets/vg/

Build necessary libs

cd lib
cd ..

Download pretrained imagenet-model

python tools/

To reproduce numbers in the paper

Download pretrained model from link.

run scripts in scripts/reproduce


Train without unseen category annotations

bash scripts/

In this case, --cfg defines the general training test configuration, like roi align size, backbone bone etc.

--auto_resume enables it to resume from the latest snapshot, this has to be used with --id together.

--set overrides the items in the configuration files. Here we manually set NUM_CLASSES BASE_LR and MODEL.TAGGING.

--id allow us to identify where we save our model much easier.

iter_reason_new.yaml is borrowed from xinlei's iterative reasoning paper.

Train with word embedding as last layer

bash scripts/

This append MODEL.WORD_EMBEDDING_REGU True in '--set'.

Normally, the last fc layer of fast rcnn head is 2048x601, where here this layer is replaced by two linear layer 2048x300 and 300x601. The weight of 300x601 layer is the word embedding of all the classes.

Train with relation inference model.

bash (bash

Test in region classification setting

Test with WE

bash scripts/test/
bash scripts/test/

Test with conse

bash scripts/test/
bash scripts/test/

Test with gcn

Download zsl-pth-gcn to ./externals.

(In python 2)
python lib/zsl-gcn/ --weight_folder ./Outputs/rel_ft/ft_gt_relt_geo_sc
bash scripts/test/
bash scripts/test/

Test with sync

(In python 2)
python lib/zsl-gcn/ --weight_folder ./Outputs/iter_reason_new/irn_vg_gt_sc_step --dataset vgbansal --to_weight Outputs/rel_ft/ft_gt_relt_geo_sc_step/ckpt/model_final.pth

Test in detection setting:

Get edgebox proposals

cd ./external/edges
cd ../../data/vg/bansal/

or download from link, and put it under ./data/vg/bansal


bash scripts/test/
bash scripts/test/
You can’t perform that action at this time.