Skip to content
Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders
Lua Python Shell
Branch: master
Clone or download
Latest commit 4c9b3f7 Feb 27, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
criteria comit Nov 9, 2016
datasets add spiral Nov 9, 2016
layers comit Nov 9, 2016
models add batchnorm for stability Nov 10, 2016
plot comit Nov 9, 2016
.gitignore comit Nov 9, 2016
ClusteringEvaluations.lua comit Nov 9, 2016
README.md readme Feb 27, 2017
UtilsNetwork.lua add readme Nov 9, 2016
main.lua edit readme Nov 9, 2016
run.sh add readme Nov 9, 2016

README.md

This branch is for version 1 of the paper. For updated version checkout branch version2 and svhn

Code for "Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoders"

(https://arxiv.org/abs/1611.02648) (version 1) By Nat Dilokthanakul, Pedro A.M. Mediano, Marta Garnelo, Matthew C.H. Lee, Hugh Salimbeni, Kai Arulkumaran, Murray Shanahan

Abstract

We study a variant of the variational autoencoder model with a Gaussian mixture as a prior distribution, with the goal of performing unsupervised clustering through deep generative models. We observe that the standard variational approach in these models is unsuited for unsupervised clustering, and mitigate this problem by leveraging a principled information-theoretic regularisation term known as consistency violation. Adding this term to the standard variational optimisation objective yields networks with both meaningful internal representations and well-defined clusters. We demonstrate the performance of this scheme on synthetic data, MNIST and SVHN, showing that the obtained clusters are distinct, interpretable and result in achieving higher performance on unsupervised clustering classification than previous approaches.

Requirements

Luarocks packages:

Python packages:

  • torchfile (pip install torchfile)

Instructions

To run a spiral experiment,

./run.sh spiral

To visualise spiral experiment (can be used while training)

cd plot
python plot_latent.py
python plot_recon.py

To turn off CV regularisation term (set alpha)

set flag

./run.sh spiral -cvWeight 0.0

To turn off z-prior term (set eta)

./run.sh spiral -zPriorWeight 0.0

To run MNIST

./run.sh mnist

To use GPU

set flag

./run.sh mnist -gpu 1

To run quick MNIST on fully-connected network

./run.sh mnist_fc

Code reading guide

The neural network components are constructed with nngraph in the folder 'models'.

The network's components are put together in the main.lua file at line 126.

During optimisation, the GMVAE objectives (5 contributing terms) are calculated from line 202 until 250 in main.lua.

Acknowledgements

I would like to thanks the following, whose github's repos were used as inital templates for implementing this idea.

  1. Rui Shu https://github.com/RuiShu/cvae

  2. Joost van Amersfoort https://github.com/y0ast/VAE-Torch

  3. Kai Arulkumaran https://github.com/Kaixhin/Autoencoders

You can’t perform that action at this time.