Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Latest commit fabb09b May 31, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
conv2d Initial commit May 20, 2019
data_loaders Initial commit May 20, 2019
maf Initial commit May 20, 2019
README.md Update README.md May 31, 2019
graphics.py Initial commit May 20, 2019
memory_saving_gradients.py Initial commit May 20, 2019
model.py Initial commit May 20, 2019
optim.py Initial commit May 20, 2019
tfops.py Initial commit May 20, 2019
train.py Initial commit May 20, 2019
utils.py Initial commit May 20, 2019

README.md

Emerging Convolutions for Generative Flows

Code for invertible convolutions (or deconvolutions) in deep neural networks: paper, blog.

If you use our work, please cite us:

Emiel Hoogeboom, Rianne van den Berg, and Max Welling. Emerging Convolutions for Generative Normalizing Flows. International Conference on Machine Learning, 2019.

A BibTeX entry for LaTeX users is:

@inproceedings{
hoogeboom2019emerging,
title={Emerging Convolutions for Generative Normalizing Flows},
author={Emiel Hoogeboom and Rianne van den Berg and Max Welling},
booktitle={International conference on machine learning},
year={2019},
url={https://arxiv.org/abs/1901.11137},
}

The source is adapted from Glow: Generative Flow with Invertible 1x1 Convolutions

Requirements

  • Horovod (tested with 0.15.2)
  • Tensorflow (tested with 1.12)

Download datasets

CIFAR10 is automatically downloaded. Galaxy images need to be downloaded here.

ImageNet 32x32 and 64x64 was downloaded from the link on the Glow github: https://storage.googleapis.com/glow-demo/data/{dataset_name}-tfr.tar with imagenet-oord as dataset_name.

Galaxy images results

Periodic:

mpiexec -n 4 python train.py --problem space --image_size 32 --n_level 2 --depth 8 --flow_permutation 5 --flow_coupling 1 --seed 2 --lr 0.001 --n_bits_x 8 --epochs 6001

Emerging:

mpiexec -n 4 python train.py --problem space --image_size 32 --n_level 2 --depth 8 --flow_permutation 3 --flow_coupling 1 --seed 2 --lr 0.001 --n_bits_x 8 --epochs 6001

Baseline (Glow):

mpiexec -n 4 python train.py --problem space --image_size 32 --n_level 2 --depth 8 --flow_permutation 2 --flow_coupling 1 --seed 2 --lr 0.001 --n_bits_x 8 --epochs 6001
CIFAR-10 results

Emerging:

mpiexec -n 4 python train.py --problem cifar10 --image_size 32 --n_level 3 --depth 32 --flow_permutation 3 --flow_coupling 1 --seed 2 --learnprior --lr 0.001 --n_bits_x 8 --epochs 4001

Baseline (Glow):

mpiexec -n 4 python train.py --problem cifar10 --image_size 32 --n_level 3 --depth 32 --flow_permutation 2 --flow_coupling 1 --seed 2 --learnprior --lr 0.001 --n_bits_x 8 --epochs 4001
CIFAR-10 results (smaller architectures)

Replace ? with either 8 or 4, depending on the experiment.

Emerging:

mpiexec -n 4 python train.py --problem cifar10 --image_size 32 --n_level 3 --depth ? --flow_permutation 3 --flow_coupling 1 --seed 2 --learnprior --lr 0.001 --n_bits_x 8 --epochs 4001

Baseline (Glow):

mpiexec -n 4 python train.py --problem cifar10 --image_size 32 --n_level 3 --depth ? --flow_permutation 2 --flow_coupling 1 --seed 2 --learnprior --lr 0.001 --n_bits_x 8 --epochs 4001
ImageNet 32x32 results

Emerging:

mpiexec -n 4 python train.py --problem imagenet-oord --image_size 32 --n_level 3 --depth 48 --flow_permutation 3 --flow_coupling 1 --seed 0 --learnprior --lr 0.001 --n_bits_x 8

Baseline (Glow):

mpiexec -n 4 python train.py --problem imagenet-oord --image_size 32 --n_level 3 --depth 48 --flow_permutation 2 --flow_coupling 1 --seed 0 --learnprior --lr 0.001 --n_bits_x 8
ImageNet 64x64 results

Emerging:

mpiexec -n 4 python train.py --problem imagenet-oord --image_size 64 --n_level 4 --depth 48 --flow_permutation 3 --flow_coupling 1 --seed 0 --learnprior --lr 0.001 --n_bits_x 8

Baseline (Glow):

mpiexec -n 4 python train.py --problem imagenet-oord --image_size 64 --n_level 4 --depth 48 --flow_permutation 2 --flow_coupling 1 --seed 0 --learnprior --lr 0.001 --n_bits_x 8
1x1 Convolution results

QR 1x1:

mpiexec -n 4 python train.py --problem cifar10 --image_size 32 --n_level 3 --depth 8 --flow_permutation 2 --flow_coupling 1 --seed 2 --learnprior --lr 0.001 --n_bits_x 8 --epochs 3501 --decomposition QR

PLU 1x1 (Glow):

mpiexec -n 4 python train.py --problem cifar10 --image_size 32 --n_level 3 --depth 8 --flow_permutation 2 --flow_coupling 1 --seed 2 --learnprior --lr 0.001 --n_bits_x 8 --epochs 3501 --decomposition PLU

Baseline 1x1 (Glow):

mpiexec -n 4 python train.py --problem cifar10 --image_size 32 --n_level 3 --depth 8 --flow_permutation 2 --flow_coupling 1 --seed 2 --learnprior --lr 0.001 --n_bits_x 8 --epochs 3501
You can’t perform that action at this time.