Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md

Common Feature Learning

Official implementation of Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning (IJCAI 2019) in pytorch.

Results

Teacher Performance

Teacher Model Dataset num_classes Acc
ResNet18 CUB200 200 0.7411
ResNet34 StanfordDogs 120 0.8663

Student Performance (CUB200+StanfordDogs)

Target Model KD CFL
ResNet34 0.7684 0.7721
ResNet50 0.7965 0.7997
DenseNet121 0.7769 0.7815

see logs for more information

Accuracy Curve

TSNE Visualization of 20 Classes

Feature Space: space constructed with intermediate outputs.
Common Space: common feature space in CFL Blocks.

Some Feature spaces are None because of different feature dimensions (e.g. 2048 for ResNet50 but 512 for ResNet34)

Target Model Common Space Feature Space
ResNet34 cfl-feature-space cfl-feature-space
ResNet50 cfl-feature-space None
DenseNet121 cfl-feature-space None

Quick Start

1. Download Datasets

python download_data.py

2. Get Trained Teacher Models

ResNet18 & ResNet34, 242.9 MB
Google Drive
BaiDu Yun

3. Train

python amal.py --model resnet34 --gpu_id 0 --lr 1e-4 --cfl_lr 5e-4
python kd.py --model resnet34 --gpu_id 0 --lr 1e-4

or

bash run_all.sh

4. Draw Accuracy Curve

cd logs/
python draw_acc_curve.py

5. TSNE

TSNE results will be saved at tsne_results/MODEL_NAME/

# ResNet34
python tsne_common_space.py --ckpt checkpoints/amal_resnet34_best.pth --t1_ckpt checkpoints/cub200_resnet18_best.pth --t2_ckpt checkpoints/dogs_resnet34_best.pth --gpu_id 0

# ResNet50
python tsne_common_space.py --ckpt checkpoints/amal_resnet50_best.pth --t1_ckpt checkpoints/cub200_resnet18_best.pth --t2_ckpt checkpoints/dogs_resnet34_best.pth --gpu_id 0

# DenseNet121
python tsne_common_space.py --ckpt checkpoints/amal_densenet121_best.pth --t1_ckpt checkpoints/cub200_resnet18_best.pth --t2_ckpt checkpoints/dogs_resnet34_best.pth --gpu_id 0

Citation

@inproceedings{luo2019knowledge,
  title={Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning},
  author={Luo, Sihui and Wang, Xinchao and Fang, Gongfan and Hu, Yao and Tao, Dapeng and Song, Mingli},
  booktitle={Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI)},
  year={2019},
}

About

(IJCAI 2019) Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning

Topics

Resources

Releases

No releases published

Packages

No packages published