The information sieve for discrete variables.
Latest commit edd1691 Nov 4, 2016 @gregversteeg 1. Visualization code was accidentally commented out.
2. More default iterations for corex
3. Turn off Bayesian smoothing by default.
Failed to load latest commit information.

The Information Sieve

Recent work establishes results that allow the efficient construction of representations that are maximally informative about the data. A recent extension allows us to build maximally informative representations in an incremental way. The other advantage of this approach is that it keeps track of remainder information, so that we can enable lossy and lossless compression.

This is described here:
The Information Sieve
Greg Ver Steeg and Aram Galstyan, ICML 2016,

A much more practical version of the information sieve for continuous variables is introduced here:
Sifting Common Information from Many Variables
Greg Ver Steeg, Shuyang Gao, Kyle Reing, and Aram Galstyan
The code for this version is in this repository:

Papers describing previous work:
Discovering Structure in High-Dimensional Data Through Correlation Explanation
Greg Ver Steeg and Aram Galstyan, NIPS 2014,

Maximally Informative Hierarchical Representions of High-Dimensional Data
Greg Ver Steeg and Aram Galstyan, AISTATS 2015,


Sieve requires numpy and scipy. If you use OS X, I recommend installing the Scipy Superpack:


To install, download using the link on the right or clone the project by executing this command in your target directory:

git clone git://

Use git pull to get updates. The code is under heavy development. Please feel free to raise issues or request features using the github interface.

Basic Usage


import corex as ce

X = np.array([[0,0,0,0,0], # A matrix with rows as samples and columns as variables.
              [1,1,1,1,1]], dtype=int)

s = sieve.Sieve(max_layers=2)  # Initialize. Can limit number of layers (default is automatic) and pass kwargs to CorEx

s.labels  # Each variable/column is associated with one Y_j
s.tcs  # TC(X; Y) for each latent factor
s.layers[0].corex  # Access the CorEx object 

More examples are in the paper and in the tests and paper_figures folder.

CorEx outputs

See for more info about the CorEx module (the one included here is a simplified version.)