Variational Walkback, NIPS'17
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
layers updates Sep 29, 2017
lib updates Sep 29, 2017
models updates Sep 29, 2017
ConvLayer.py updates Sep 29, 2017
README.md updates Oct 18, 2017
__init__.py updates Sep 29, 2017
compute_inception_cifar.py updates Sep 29, 2017
compute_loss.py updates Sep 29, 2017
dataprep.py updates Sep 29, 2017
datasets.py updates Sep 29, 2017
distributions.py updates Sep 29, 2017
draw_fig.py updates Sep 29, 2017
experiment_celebA.sh updates Sep 29, 2017
experiment_celebA_nips.sh updates Sep 29, 2017
experiment_cifar.sh updates Sep 29, 2017
experiment_cifar_diff_bnorm.sh updates Sep 29, 2017
experiment_circle_online.sh updates Sep 29, 2017
experiment_compute_inception_cifar.sh updates Sep 29, 2017
experiment_lsun.sh updates Sep 29, 2017
experiment_mog_diff_bnorm.sh updates Sep 29, 2017
experiment_mog_online.sh updates Sep 29, 2017
experiment_online_mnist.sh
experiment_spiral_online.sh updates Sep 29, 2017
experiment_svhn.sh updates Sep 29, 2017
generate_figs.py updates Sep 29, 2017
graphing.py updates Sep 29, 2017
imagenet_data.py updates Sep 29, 2017
impainting.py updates Sep 29, 2017
inception_score.py updates Sep 29, 2017
load_data.py updates Sep 29, 2017
load_lsun.py updates Sep 29, 2017
log_analysis.py updates Sep 29, 2017
mixture_viz.py updates Sep 29, 2017
nn_layers.py updates Sep 29, 2017
optimizers.py updates Sep 29, 2017
parzen_estimate.py updates Sep 29, 2017
setup.py updates Sep 29, 2017
train_celebA.py updates Sep 29, 2017
train_cifar_online.py updates Sep 29, 2017
train_cifar_online_diff_bnorm.py updates Sep 29, 2017
train_impainting.py updates Sep 29, 2017
train_lsun.py updates Sep 29, 2017
train_mixture_online.py updates Sep 29, 2017
train_mog_different_batchnorm.py updates Sep 29, 2017
train_spiral.py updates Sep 29, 2017
train_spiral_online.py updates Sep 29, 2017
train_svhn.py updates Sep 29, 2017
util.py updates Sep 29, 2017
vector_field.py updates Sep 29, 2017
viz.py updates Sep 29, 2017
viz_cifar.py updates Sep 29, 2017

README.md

walkback_nips17

The authors would like to apologize for not pushing clean code, but you should be able to reproduce the experiments just by running the scripts. Also, in the current code setup, we are NOT learning the variance, which seems to improve the sample quality.

Requirements - The code requires theano(version -0.9.0.dev-c697eeab84e5b8a74908da654b66ec9eca4f1291) Unfortunately, the code is not compatible with the latent version of theano, to make it compatible, we need to make some changes.

This repo contains code for running VW on Cifar, SVHN, CelebA, LSUN, Circle, Spiral , Mixture of Gaussian. Some of these datasets are not mentioned in the paper.

In all the image experiments, we observed that by having different batchnorm papemeters for different steps, actually improves the result considerably. Having different batchnorm parameters was also necessery for making it work on mixture on gaussian. The authors were not able to make it work on MoG without different parameters. One possible way, could be to let optimizer know that we are on different step by giving the temperature information to the optimizer too. But there's a tradeoff, in some cases having different batchnorm parameters, improves the results (visually as well as lower bound) and in some cases it decreases the bound considerably, keeping the visual quality of images same.

We observed better results while updating the parameters in online-mode, as compared to batch mode. (i.e instead of accumulating gradients across different steps, we update the parameters in an online fashion)

The authors would also like to thank open-source contributors from all the different libraries, as the authors have used source code of other people too. (https://github.com/casperkaae/parmesan, https://github.com/Sohl-Dickstein/Diffusion-Probabilistic-Models)