The repository was created to test the idea of ensembles of image augmentations for improving classification.
The ensemble can be used for improving model inference quality and for training using partially unsupervised data as suggested (for detection and pose estimation) in the article :
https://arxiv.org/pdf/1712.04440.pdf
- Inference using existing network, translating ImageNet results to desired set of classes
- Inference using ensembles by augmenting evaluation images
- Transfer learning training by changing last layer(s)
- Training with partially unsupervised data as suggested in the article (using an auxiliary trained network for tagging the unserperised data)
- continue training
- start training from scratch
A bit of experimentation was conducted with the following techniques and their combination:
- resize & rescale (keep aspect ratio, when changing size)
- flip
- crops
- color augmentations
From some test it seems that:
- For a single net it seems that rescaling image is better then resizing to fit network input.
- For an ensemble A combination of flip and crops using Pytorch's built in TenCrop from a rescaled resolution close to target resolution (e.g. rescale to 240x240 and apply TenCrop for 224x224 )
- Majority vote: find most common index of argmax of each result
- Average: argmax of average of all classification results after softmax
No clear winner between the two.
From the Cat vs Dogs train dataset a few very small subsets were created for training, unsupervised training and evaluation. https://www.kaggle.com/c/dogs-vs-cats
The smaller dataset were created as even the smallest classification networks trained on ImageNet get very high scores (>95% accuracy) even withou transfer learning, which didn't give a lot of margin to test improvements.
Again, very light network for lower initial accuracy.
- Pytroch 0.4.0
- TensorboardX
- tqdm
- PIL
- numpy
Enables running a simple or augmentations ensemble inference for pretrained and transfer learning trained networks
Run inference for pre-trained squeezent network (with majority ensemble)
python3 eval.py --pretrained_net squeezenet1_0 --ensemble majority --eval_folder_path <path to images folder>
Run inference for transfer learning trained network(with average ensemble):
python3 eval.py --trained_net_path <path to Pytorch net file> --ensemble average --eval_folder_path <path to images folder>
Enables running simple training for transfer learning and combined training with partial unsupervised data which is tagged using a data augmentation ensemble using an auxiliary net which could be the same or different net.
Start training from pre-trained net (without auxiliary net)
python3 class_trainer.py --backup_folder <backup and tensorboardX path> --batch_size 16 --pretrained_net squeezenet1_0 --num_classes 2 --train_data_path <path to train image folder> --eval_data_path <path to evaluation image folder>
Start training from pre-trained net (with auxiliary net)
python3 class_trainer.py --backup_folder <backup and tensorboardX path> --batch_size 16 --pretrained_net squeezenet1_0 --num_classes 2 --train_data_path <path to train image folder> --auxiliary_net_path <path to trained net for tagging> --ensemble majority --auxiliary_data_path <path to another train image folder> --eval_data_path <path to evaluation image folder>
Continue training existing net
python3 class_trainer.py --backup_folder <backup and tensorboardX path> --batch_size 16 --trained_net_path <trained pytorch net path> --train_data_path <path to train image folder> --auxiliary_net_path <path to trained net for tagging> --ensemble majority --auxiliary_data_path <path to another train image folder> --eval_data_path <path to evaluation image folder>
(2500 images evaluation sub set)
- ImageNet inference 84.36%
- ImageNet inference With TenCrop augmentations (rescale to 240 then ten crop to 224) and majority vote: 84.4%
- Transfer Learning training on 100 images: 91.04 %
- Transfer Learning training on 100 images inference using TneCrop and majority: 91.76%
- Continue training with additional untagged 100 images: 91.88%
- Train from scrath with additional untagged 100 images: 91.36%