Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
inferences
models
.gitignore
LICENSE
README.md
arrows_proximity_vi.png
arrows_vanilla_vi.png
deep_latent_gaussian_model_config.yml
deep_latent_gaussian_model_experiment.sh
deep_latent_gaussian_model_grid.sh
deep_latent_gaussian_model_train.py
environment.yml
get_binary_mnist.py
launch_job_hostname.example.com.cmd
sigmoid_belief_network_config.yml
sigmoid_belief_network_experiment.sh
sigmoid_belief_network_grid.sh
sigmoid_belief_network_train.py
stats.py
util.py

README.md

Proximity Variational Inference

This code accompanies the proximity variational inference paper: https://arxiv.org/abs/1705.08931

If you use this code, please cite us:

@article{altosaar2017proximity,
	author={Altosaar, J and Ranganath, R and Blei, DM},
	eprint={arXiv:1311.1704},
	title={Proximity Variational Inference},
	url={https://arxiv.org/abs/1705.08931},
	year={2017}
}

The promise: Variational inference (left) is sensitive to initialization. Proximity variational inference (right) can help correct this.

Data

Get the binarized MNIST dataset from Hugo & Larochelle (2011), write it to /tmp/binarized_mnist.hdf5.

python get_binary_mnist.py

Environment

I recommend anaconda: brew cask install anaconda on a mac, bash installer otherwise. To use the same environment:

conda env create -f environment.yml  # may need to edit to choose between CPU or GPU version of tensorflow
source activate proximity_vi

The code assumes you have set the following environment variables. This enables easy switching between local and remote workstations.

> export DAT=/tmp
> export LOG=/tmp

Sigmoid belief network experiment

This benchmarks proximity variational inference against deterministic annealing and vanilla variational inference, with good initialization and bad initialization (Tables 1 and 2 in the paper).

Each experiment takes about half a day on a Tesla P100 GPU:

./sigmoid_belief_network_grid.sh

# List final estimates of the ELBO and marginal likelihood
tail -n 1 $LOG/proximity_vi/*/*/*.log

# View training statistics on tensorboard
tensorboard --logdir $LOG/proximity_vi

Variational autoencoder experiment

This tests the orthogonal proximity statistic to make optimization easier in a variational autoencoder. (Table 3 in the paper)

Each run takes a few minutes on a Tesla P100 GPU:

./deep_latent_gaussian_model_grid.sh

# List final estimates of the ELBO and marginal likelihood
tail -n 1 $LOG/proximity_vi/*/*/*.log

# View training statistics on tensorboard
tensorboard --logdir $LOG/proximity_vi

Support

Please email me with any questions: altosaar@princeton.edu.

You can’t perform that action at this time.