[WIP] Gaussian Bernoulli RBM

[IssamLaradji]

This extends current RBM (BB-RBM) to allow real-valued visible units. I implemented the algorithm in a separate file than BB-RBM, but I propose that a main file to contain BaseRBM, with GaussianBernoulliRBM (GB-RBM), BernoulliRBM, etc.. As RBM has many extensions in the literature, such main file would make extension easy. I would do that if the reviewers agree :).

Over BB-RBM, GB-RBM uses sigma to control the width of the parabola that adds a quadratic offset to the Energy function [1],

Much of the implementation are explained in (http://www.ini.rub.de/data/documents/tns/masterthesis_janmelchior.pdf), a masters thesis devoted partly to Gaussian RBMs.

Tests and examples are soon to come.

[1] Krizhevsky, Alex, and Geoffrey Hinton. "Learning multiple layers of features from tiny images."
Master's thesis, Department of Computer Science, University of Toronto (2009).

[untom]

I just browsed the source code a bit, but: why did you not extend the existing RBM or inherit from it? IIRC the code should be pretty much the same, apart from how you calculate your visible activations and your free energy.

[IssamLaradji]

@untom, yes, for some reason I decided first to push it as independent file :). In a while I will combine them in a main file, containing BaseRBM, BernoulliRBM, and GaussianRBM.

[ogrisel]

@IssamLaradji could you please rebase this branch on top of the current master so that travis can run the tests on it?

[amueller]

I think this should go into contrib. Anyone opposed? Feel free to reopen in this case.