Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
net
 
 
 
 
 
 
 
 
 
 
 
 

Generating-Expensive-Relationship-Features-from-Cheap-Objects

This is the tensorflow implementation for the paper "Generating Expensive Relationship Features from Cheap Objects"

Getting Started

python version: python-3.5; cuda version: cuda-10; tensorflow version: tensorflow-1.13

Datasets

Directly download processed VG features: Dataset(code:ueef), unzip and put it in the root directory.

Evaluate the models

Our pre-trained models can be downloded here: Models, unzip and put them in the root directory.

  1. To evaluate the zero-shot and low-shot cases: python eval_lowshot.py
    --model_path: the checkpoint path of the corresponding model; --lowshot_num: 0 and 1, 5, 10, 20 for zero-shot and different low-shot cases.

  2. To evaluate the all classes case: python eval_wholedata.py
    --model_path: the checkpoint path of the corresponding model.

Train the generation model

  1. Train the generation model for all classes or low-shot classes: python main.py --training True
    --out_dir: name of the directory to save training models;
    --test_setting: wholedata or lowshot (generate new features for all classes or low-shot classes);
    --L1_weight: the weight of L1 distance;
    --gpu: GPU id;
    --num_predicates: the number of predicate classes: VRD: 70 and VG: 100;
    --ac_weight: the weight of relation classifier loss;
    --training: True for training and False for testing;
    --max_epoch: the maximum training epochs;
    --batch_size: the training batch size;

  2. Generate the new features for all classes or low-shot classes: python main.py --test_setting wholedata(lowshot)
    We train the generation model for 300 epochs and synthesize 15 times of new data for low-shot features, and same number of original data for all classes features.

  3. Train the generation model for zeroshot classes: python main_w2v.py --training True
    --train_file: the training features for generation model;
    --random_file: the random features used for generating new zero-shot features;

  4. Generate the new features for zeroshot classes: python main_w2v.py
    We train the generation model for 150 epochs and synthesize 20 times of new data for zero-shot features.

Train the recognition model

  1. Obtain a new relationship recognition model: python train_feature_cls.py
    --mode: wholedata(train the all classes classifier); or lowshot(train the low-shot and zero-shot classifiers);
    --lowshot_num: 0 for zero-shot; 1,5,10,20 for low-shot; only used when --mode=lowshot;
    --lowshot_path: the low-shot and zero-shot indexes fies;
    --train_path: the new features used for training the recognition model, (generated by step 2 or 4 of last operation);
    --test_path: the test features used for the recognition model;

Performance

VG (%) ZShot LShot n=1 LShot n=5 LShot n=10 LShot n=20 ALL @50 ALL @100
Paper 19.0 20.9 24.0 27.1 30.0 63.7 64.0
This repo 19.1 21.9 25.1 27.9 30.5 64.1 64.4

Citation

@inproceedings{wang2019generating,
     author = {Wang, Xiaogang and Sun, Qianru and ANG, Marcelo and CHUA, Tat-Seng},
     title = {Generating expensive relationship features from cheap objects},
     booktitle = {BMVC},
     year = {2019},
}

Acknowledgements

Our implementations use the source code from the following repository:
vtranse

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages