RealNVP and simple Normalizing Flow in Pytorch

In this repo, I have implemented two different normalizing flows in Pytorch and tested them on three different toy datasets. The two flows implemented are

• A simple affine transform
• Real NVP [1]

The implementations of the flows are located in flow_models while a short presentation of the data and training is available in Normalizing Flows with Pytorch.

A flow can, if its transform is invertible, be used to both learn a probability density function and sample from it. Both flows implemented in this repo are invertible, so example samples drawn from the flows are presented in the notebook.

Bellow follows illustrations of how a 4 layer RealNVP-model transforms data sampled from a standard Multivariate Gaussian, into the three presented distributions. A separate model was trained for each distribution.

Performance of trained networks presented as the output of each layer

RealNVP performance on multiple Gaussians, located the same distance from the centre

RealNVP performance on a circle dataset

RealNVP performance on the moon dataset

References

[1] Density estimation using Real NVP, Laurent Dinh and Jascha Sohl-Dickstein and Samy Bengio, 2017.