Skip to content
(IJCAI 2019) Submodular Batch Selection for Training Deep Neural Networks
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config Updates Jun 7, 2019
datasets ResNet 20 + CIFAR 10 + SubModular Objective changes Nov 17, 2018
final_plots
lib Updates Jun 6, 2019
models ImageNet_modified_resnet Apr 19, 2019
LICENSE
README.md
plot.py Updates Jun 6, 2019
plot_cifar10.py
plot_cifar100.py update refresh rate graphs Feb 24, 2019
plot_svhn.py
requirements.txt
smdl.py

README.md

Submodular Batch Selection for Training Deep Neural Networks

IJCAI 2019

Mini-batch gradient descent based methods are the de facto algorithms for training neural network architectures today. We introduce a mini-batch selection strategy based on submodular function maximization. Our novel submodular formulation captures the informativeness of each sample and diversity of the whole subset. We design an efficient, greedy algorithm which can give high-quality solutions to this NP-hard combinatorial optimization problem. Our extensive experiments on standard datasets show that the deep models trained using the proposed batch selection strategy provide better generalization than Stochastic Gradient Descent as well as a popular baseline sampling strategy across different learning rates, batch sizes, and distance metrics.

Setup

Python: 2.7.6, PyTorch 0.4.1

Clone the repository and install dependencies from requirements.txt

git clone https://github.com/VamshiTeja/SMDL
cd smdl
pip install -r requirements.txt

Run

All the configurations are in a single place: ./config/smdl.yml

Modify the configurations are run smdl.py

python smdl.py

The code is well documented and you would be able to get along. In case of troubles please do raise an issue here. We would be agile.

Citation

@inproceedings{joseph2019Submodular,
  title = {Submodular Batch Selection for Training Deep Neural Networks},
  author={K J, Joseph and Vamshi Teja, R and Krishnakant, Singh and Vineeth, N Balasubramanian},
  booktitle={Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI, Macao, China.},
  organization={International Joint Conferences on Artificial Intelligence Organization},
  year={2019}}
You can’t perform that action at this time.