Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
tan
 
 
 
 
 
 
 
 
 
 
 
 

Transformation Autoregressive Networks (TANs)

A state of the art density estimator

This is the official repository for the 2018 ICML article.

@inproceedings{oliva2018transformation,
  title={Transformation Autoregressive Networks},
  author={Oliva, Junier and Dubey, Avinava and Zaheer, Manzil and Poczos, Barnabas and Salakhutdinov, Ruslan and Xing, Eric and Schneider, Jeff},
  booktitle={International Conference on Machine Learning},
  pages={3895--3904},
  year={2018}
}

TLDR: Combining novel change of variables jointly with autoregressive conditional 1d-mixture-models yields an extremely powerful density estimator on general real valued data. A year later, TANs are still leading on several benchmark datasets. Watch the 5-minute video clip below for a quick summary:

video


Results

Below we compile some of the state of the art mean test log likelihoods on several benchmark dataset introduced in the MAF paper. Standard errors are reported when available. Please report the published results below when reporting TANs performance on these datasets. Please do not report unpublished early results from an ICLR submission as done in [1] [2].

Method POWER GAS HEPMASS MINIBOONE BSDS300
TANs [2018] 0.60 ± 0.01 12.06 ± 0.02 −13.78 ± 0.02 −11.01 ± 0.48 159.80 ± 0.07
SOS [2019] 0.60 ± 0.01 11.99 ± 0.41 -15.15 ± 0.10 -8.90 ± 0.11 157.48 ± 0.41
FFJORD [2019] 0.46 8.59 -14.92 -10.43 157.40
NAF DDSF (5) [2018] 0.62 ± 0.01 11.91 ± 0.13 -15.09 ± 0.40 -8.86 ± 0.15 157.73 ± 0.04
NAF DDSF (10) [2018] 0.60 ± 0.02 11.96 ± 0.33 -15.32 ± 0.23 -9.01 ± 0.01 157.43 ± 0.30
MAF (5) [2017] 0.14 ± 0.01 9.07 ± 0.02 -17.70 ± 0.02 -11.75 ± 0.44 155.69 ± 0.28
MAF (10) [2017] 0.24 ± 0.01 10.08 ± 0.02 -17.73 ± 0.02 -12.24 ± 0.45 154.93 ± 0.28
MAF MoG (5) [2017] 0.30 ± 0.01 9.59 ± 0.02 -17.39 ± 0.02 -11.68 ± 0.44 156.36 ± 0.28
Real NVP (5) [2016]* -0.02 ± 0.01 4.78 ± 1.80 -19.62 ± 0.02 -13.55 ± 0.49 152.97 ± 0.28
Real NVP (10) [2016]* 0.17 ± 0.01 8.33 ± 0.14 -18.71 ± 0.02 -13.84 ± 0.52 153.28 ± 1.78
MADE [2015]* -3.08 ± 0.03 3.56 ± 0.04 -20.98 ± 0.02 -15.59 ± 0.50 148.85 ± 0.28
MADE MoG [2015]* 0.40 ± 0.01 8.47 ± 0.02 -15.15 ± 0.02 -12.27 ± 0.47 153.71 ± 0.28

*As reported in the MAF paper.

To train model on exactly the same data as the MAF paper, first download and save data in pickle formats:

import tan.demos.mafdata_demo as mdemo
# Will save pickle files in ~/data/tan by default
# Assumes you are in outermost repo directory; i.e. can see ./tan/external_maf
mdemo.download_and_make_data()

After, train a model on a specified dataset by running

# Returns a dictionary of result stats
res = mdemo.main(dataset=dname)

where dname is one of {maf_power, maf_gas, maf_hepmass, maf_miniboone, maf_bsds}. A scatter plot of samples projected into the 2d space of first and last dimensions is also saved (by default in ~/data/tan/maf_*/); for example for maf_hepmass:

hepmass

Note that the above validates across multiple random initializations (open question: how to best initialize?), if you wish to only train on one initialization, then run res = mdemo.main(dataset=dname, ntrls=1).

About

No description, website, or topics provided.

Resources

License

Releases

No releases published

Packages

No packages published

Languages