Code implementing the basic results ofthe paper On the Power of Curriculum Learning in Training Deep Networks. This project demonstrate how to train simple keras model using curriculum by transfer, and fixed exponential pacing, reproducing the results depicted in the paper.
All prerequiesites are listed in the requirements.txt, and can be installed by running:
pip3 install -r requirements.txt
The project assumes python 3.5 or higher.
the file main_train_networks.py controls the entire pipeline of the project. It can train the model used in the paper, on various test cases, using the following flags:
- --dataset - any superclass of CIFAR-100. Options are: cifar100_subset_0/cifar100_subset_1/.../cifar100_subset_20. defualt is cifar100_subset_16, which is the "small-mammals" dataset which was used in the paper
- --curriculum - Which test case (as defined in the paper) to use. Can choose from: curriculum, vanilla, random and anti (corresponding to anti-curriculum)
- --output_path - location to save the output
- --repeats - number of times to repeat the experiment
- --batch_size - detemine the batchsize
- --num_epochs - number of epochs to train the model
- --learning_rate - initial learning
- --lr_decay_rate - factor by which we drop learning rate exponentially
- --minimal_lr - min learning rate we allow
- --lr_batch_size - interval of batches in which we drop the learning rate
- --batch_increase - interval of batches to increase the amount of data we sample from
- --increase_amount - factor by which we increase the amount of data we sample from
- --starting_percent - percent of data to sample from in the inital batch
- --order - determine the network from which we do transfer learning. options: inception, vgg16, vgg19, xception, resnet
An example of running each test case, including the resulting graphs, can be seen by running: main_reproduce_paper.py
- Guy Hacohen - Initial work - GuyHacohen
This project is licensed under the GNU general public License - see the LICENSE.md file for details