Skip to content
Tensorflow implementation of DropMax: Adaptive Variational Softmax (NeurIPS2019)
Python Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.


Type Name Latest commit message Commit time
Failed to load latest commit information. Update Dec 11, 2018 Add files via upload Sep 12, 2018
dropmax_poster.pdf Add files via upload Nov 23, 2018 Update Jan 26, 2019 Update Jan 26, 2019 Add files via upload Sep 12, 2018 Update Jan 26, 2019 Add files via upload Sep 12, 2018

DropMax: Adaptive Variationial Softmax

  • Hae Beom Lee (KAIST), Juho Lee (Univ. Oxford), Saehoon Kim (AITRICS), Eunho Yang (KAIST), and Sung Ju Hwang (KAIST)

This is the Tensor-Flow implementation for the paper DropMax: Adaptive Variationial Softmax (NeurIPS 2018) :


We propose DropMax, a stochastic version of softmax classifier which at each iteration drops non-target classes according to dropout probabilities adaptively decided for each instance. Specifically, we overlay binary masking variables over class output probabilities, which are input-adaptively learned via variational inference. This stochastic regularization has an effect of building an ensemble classifier out of exponentially many classifiers with different decision boundaries. Moreover, the learning of dropout rates for non-target classes on each instance allows the classifier to focus more on classification against the most confusing classes. We validate our model on multiple public datasets for classification, on which it obtains significantly improved accuracy over the regular softmax classifier and other baselines. Further analysis of the learned dropout probabilities shows that our model indeed selects confusing classes more often when it performs classification.


If you found the provided code useful, please cite our work.

    author    = {Hae Beom Lee and Juho Lee and Saehoon Kim and Eunho Yang and Sung Ju Hwang},
    title     = {DropMax: Adaptive Variationial Softmax},
    booktitle = {NeurIPS},
    year      = {2018}

Run examples

  1. Modify --mnist_path, in
  2. Specify --model among softmax or dropmax, in
  3. Run

Implementation Issue

Note that LeNet is used as the base network for this code, whereas in the paper we used the network in the Tensorflow-Tutorial (refer to But the training results and tendencies are similar.

You can’t perform that action at this time.