Skip to content
Low-variance Black-box Gradient Estimates for the Plackett-Luce Distribution, AAAI 2020 and NeurIPS 2019 Bayesian Deep Learning Workshop
Jupyter Notebook Python
Branch: master
Clone or download
Latest commit 1a33d7e Nov 25, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
figures added toy_experiment Nov 21, 2019
README.md Update README.md Nov 25, 2019
critics.py added toy_experiment Nov 21, 2019
distributions.py added toy_experiment Nov 21, 2019
estimators.py added toy_experiment Nov 21, 2019
plot_toy.ipynb added toy_experiment Nov 21, 2019
toy_env.yml added toy_experiment Nov 21, 2019
toy_experiment.py added toy_experiment Nov 21, 2019
utils.py added toy_experiment Nov 21, 2019

README.md

Low-variance Black-box Gradient Estimates for the Plackett-Luce Distribution

This repo contains code for our paper Low-variance Black-box Gradient Estimates for the Plackett-Luce Distribution.

Abstract

Learning models with discrete latent variables using stochastic gradient descent remains a challenge due to the high variance of gradients. Modern variance reduction techniques mostly consider categorical distributions and have limited applicability when the number of possible outcomes becomes large. In this work, we consider models with latent permutations and propose control variates for the Plackett-Luce distribution. In particular, the control variates allow us to optimize black-box functions over permutations using stochastic gradient descent. To illustrate the approach, we consider a variety of causal structure learning tasks for continuous and discrete data. We show that for differentiable functions, our method outperforms competitive relaxation-based optimization methods and is also applicable to non-differentiable score functions.

Citation

Coming soon

Toy Experiment

Prepare environment (maybe you'll need to change cudatoolkit version in toy_env.yml or even use cpuonly version of PyTorch):

conda env create -f toy_env.yml
conda activate toy_env

Run toy_experiment.py:

python toy_experiment.py --estimator exact
python toy_experiment.py --estimator reinforce
python toy_experiment.py --estimator rebar
python toy_experiment.py --estimator relax

Plot figure using plot_toy.ipynb

alt text

Results were obtained using cpu. Quantitative results for cuda may vary slighlty due to randomness in cuda kernels, but qualitative results remain the same.

You can’t perform that action at this time.