This is the PyTorch source code for Distilling Knowledge by Mimicking Features. We provide all codes for three tasks.
- single-label image classification: LSHFM.singleclassification
- multi-label image classification: LSHFM.multiclassification
- object detection: LSHFM.detection
- python3
- pytorch 1.7.1
- torchvision 0.8.2
Please prepare the COCO and VOC datasets by youself. Then you need to check and edit the get_data_path function in src/dataset/coco_utils.py and src/dataset/voc_utils.py.
Teacher:
- wrn_40_2
- resnet56
- resnet110
- resnet32x4
- vgg13
- ResNet50
Student:
- wrn_16_2
- wrn_40_1
- resnet20
- resnet32
- resnet8x4
- vgg8
- MobileNetV2
- ShffleNetV1
- ShffleNetV2
Train the teacher:
python train_vanilla.py --model [teacher network] --gpus 0
e.g.
python train_vanilla.py --model resnet56 --gpus 0
Train the student:
python train_vanilla.py --model [student network] --gpus 0
e.g.
python train_vanilla.py --model wrn_16_2 --gpus 0
Please use the below command to run experiments:
python train_student.py --model_s [student network] --path_t [path to the teacher] --gpus 0
e.g.
python train_student.py --model_s wrn_16_2 --path_t save/models/wrn_40_2_vanilla/ckpt_epoch_240.pth --gpus 0
Please use the below command to run experiments:
python imagenet_lsh.py /mnt/ramdisk/ImageNet --gpus 0,1 -a ResNet18 --teacher-arch ResNet34
If you find this code useful in your research, please consider citing us:
@article{LSHFM,
title={Distilling knowledge by mimicking features},
author={Wang, Guo-Hua and Ge, Yifan and Wu, Jianxin},
journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
year={2021},
}