Skip to content
Zero-Shot Visual Recognition using Semantic-Preserving Adversarial Embedding Networks
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data add missing CUB_embedding.mat file Jan 10, 2019
model init commit May 31, 2018
utils
README.md
cfg.py init commit May 31, 2018
convert_data.py
run.py
sp_aen.py init commit May 31, 2018

README.md

Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks

This repository contains the code for the following paper:

  • Long Chen, Hanwang Zhang, Jun Xiao, Wei Liu, Shih-Fu Chang, Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks. In CVPR, 2018. [PDF]

Note: This repository is adaptived from DeepSim, tensorflow-resnet, Thanks a lot to Jiaxin Shi from Tsinghua University, and Yongqin Xian to release his dataset split and code.

Requirements and Dependencies

  • Python 2.7 (Anaconda recommended)
  • TensorFlow
  • Caffe (Optional, In my original experiment, I load pretrained ImageNet CNN model weigths in caffemodel format and save it to npy format for tensorflow. But we can direct download the npy format CNN pretrained weights)

Get Started

Dataset and Pretrain Model Download

  • Download the dataset (CUB/AWA/aPY/SUN), and save it into directory /SP-AEN-HOME/data/, and change the corresponding path of dataset and split file in /SP-AEN-HOME/cfg.py.
  • Download pretrained CNN model alexnet, caffenet, resnet, and save the pretrained model in /SP-AEN-HOME/model/pretrained/imagenet. And download pretrained model caffenet/fc6 and save to /SP-AEN-HOME/model/pretrained/generator/caffenet/fc6. Use /SP-AEN-HOME/model/caffe2tf.py to convert the caffemodel format weights to npy format.

Data Preprocessing

Extract the resnet-101 CNN featuer and preprocessing the dataset directly use:

python /SP-AEN-HOME/convert_data.py -dataset cub (for CUB)
python /SP-AEN-HOME/convert_data.py -dataset apy (for aPY)
python /SP-AEN-HOME/convert_data.py -dataset awa2 (for AWA2)
python /SP-AEN-HOME/convert_data.py -dataset sun (for SUN)

Train:

dataset can select from ['cub', 'apy', 'awa2', 'sun'], for dataset CUB:

step1:
python run.py --dataset=cub --mode=trainf --summary_path=.. --alpha_map_rank=.. --alpha_rank_dis=.. --alpha_rank_gen=..
step2:
python run.py --dataset=cub --mode=traine --retrain_model=True --train_checkpoint=.. --summary_path=..

Citiation

If you find this code useful, please cite the following paper:

@inproceedings{chen2018zero,
  title={Zero-Shot Visual Recognition using Semantics-Preserving Adversarial Embedding Networks},
  author={Chen, Long and Zhang, Hanwang and Xiao, Jun and Liu, Wei and Chang, Shih-Fu},
  booktitle={CVPR},
  year={2018}
}
You can’t perform that action at this time.