---

Sinusoidal Flow: A Fast Invertible Autoregressive Flow

Description

Normalising flows offer a flexible way of modelling continuous probability distributions. We consider expressiveness, fast inversion and exact Jacobian determinant as three desirable properties a normalising flow should possess. However, few flow models have been able to strike a good balance among all these properties. Realising that the integral of a convex sum of sinusoidal functions squared leads to a bijective residual transformation, we propose Sinusoidal Flow, a new type of normalising flows that inherits the expressive power and triangular Jacobian from fully autoregressive flows while guaranteed by Banach fixed-point theorem to remain fast invertible and thereby obviate the need for sequential inversion typically required in fully autoregressive flows. Experiments show that our Sinusoidal Flow is not only able to model complex distributions, but can also be reliably inverted to generate realistic-looking samples even with many layers of transformations stacked.

How to run

First, install dependencies (Python 3.8 or above required)

# clone project   
git clone https://github.com/weiyumou/ldu-flow.git

# install project   
cd ldu-flow
pip install -e .

Next, download the data files for UCI, MNIST and CIFAR-10 experiments here, and place each data folder under project/datasets/maf/data. For example, the data file for the POWER dataset should be accessible from project/datasets/maf/data/power/data.npy.

Reproducibility Check

You may do a quick reproducibility check for the results in Table 4 by testing checkpoints. First, download all the checkpoints here and place each checkpoint folder corresponding to each dataset under project/experiments/ldu/checkpoints. For example, the ".ckpt" checkpoint file for MNIST should be accessible under project/experiments/ldu/checkpoints/mnist/version_21654323/checkpoints.

Then you may either test the checkpoint for MNIST by running

python -m project.experiments.ldu.run_mnist_cifar10 --dataset mnist --test_checkpoints --gpus 0

or the checkpoint for CIFAR-10 by running

python -m project.experiments.ldu.run_mnist_cifar10 --dataset cifar10 --test_checkpoints --gpus 0

Use --gpus 1 instead to test checkpoints on a single GPU.