Skip to content
Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons (AAAI 2019)
Python
Branch: master
Clone or download
Latest commit f895b14 May 1, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data message Nov 21, 2018
models bug & memory issues are fixed Feb 17, 2019
results Initial commit Nov 21, 2018
.gitignore Initial commit Nov 21, 2018
MITscenes_AB_distillation.py bug & memory issues are fixed Feb 17, 2019
README.md Update README.md Apr 30, 2019
cifar10_AB_distillation.py Typo Mar 5, 2019

README.md

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons

Official Pytorch implementation of paper:

Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons (AAAI 2019).

Slides and poster are available on homepage

Environment

Python 3.6, Pytorch 0.4.1, Torchvision

Knowledge distillation (CIFAR-10)

cifar10_AB_distillation.py


Distillation from WRN 22-4 (teacher) to WRN 16-2 (student) on CIFAR-10 dataset.

Pre-trained teacher network (WRN 22-4) is included. Just run the code.

Transfer learning (MIT_scenes)

MITscenes_AB_distillation.py


Transfer learning from ImageNet pre-trained model (teacher) to randomly initialized model (student).

Teacher : ImageNet pre-trained ResNet 50

Student : MobileNet or MobileNetV2 (randomly initialized model)

Please change base learning rate to 0.1 for MobileNetV2.


MIT_scenes dataset should be arranged for Torchvision ImageFolder function.

Train set : $dataset_path / train / $class_name / $image_name

Test set : $dataset_path / test / $class_name / $image name

and run with dataset path.

MobileNet

python MITscenes_AB_distillation.py --data_root $dataset_path

MobileNet V2

python MITscenes_AB_distillation.py --data_root $dataset_path --network mobilenetV2

Other implementations

Tensorflow: https://github.com/sseung0703/Knowledge_distillation_methods_wtih_Tensorflow

Citation

@inproceedings{ABdistill,
	title = {Knowledge Transfer via Distillation of Activation Boundaries Formed by Hidden Neurons},
	author = {Byeongho Heo, Minsik Lee, Sangdoo Yun, Jin Young Choi},
	booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
	year = {2019}
}
You can’t perform that action at this time.