Abstract: Progressive Neural Network Learning is a class of algorithms that incrementally construct the network's topology and optimize its parameters based on the training data. While this approach exempts the users from the manual task of designing and validating multiple network topologies, it often requires an enormous number of computations. In this paper, we propose to speed up this process by exploiting subsets of training data at each incremental training step. Three different sampling strategies for selecting the training samples according to different criteria are proposed and evaluated. We also propose to perform online hyperparameter selection during the network progression, which further reduces the overall training time. Experimental results in object, scene and face recognition problems demonstrate that the proposed approach speeds up the optimization procedure considerably while operating on par with the baseline approach exploiting the entire training set throughout the training process.
The code is written in Python 3.x with the following dependencies:
- dill
- keras 2.2.4
- tensorflow 1.13.1
- scikit-learn 0.23.1
- joblib 0.15.1
The data should be downloaded from this, and put under the main directory. Basically the directory structure should be like this:
SIPL/
│ README.md
└───data/
│ caltech256_x_test.npy
│ caltech256_x_train.npy
│ ...
└───code/
│ exp_configurations.py
│ ...
Any inquiry can be sent to thanh.tran@tuni.fi or viebboy@gmail.com
This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 871449 (OpenDR). This publication reflects the authors’ views only. The European Commission is not responsible for any use that may be made of the information it contains.
To find more information about OpenDR project, please visit https://opendr.eu/