Matlab Code for Restricted/Deep Boltzmann Machines and Autoencoders
Matlab M C++
Permalink
Failed to load latest commit information.
example compatibility with liblinear-1.9x Jul 28, 2014
MaxPooling.cpp convnet added Jul 15, 2013
MaxPooling.m convnet added Jul 15, 2013
MaxPooling.mexa64 convnet added Jul 15, 2013
README.md average pooling Jul 18, 2013
TODO.txt Initial Version. Read TODO.txt for future work. Apr 1, 2013
candidate_lrates.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
candidate_moments.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
convnet.m bug fixes Feb 5, 2014
convnet_avgpool.m average pooling added Jul 18, 2013
convnet_classify.m bug fix in convnet Aug 13, 2013
convnet_maxpool.m convnet added Jul 15, 2013
dae.m Bug fix for use_gpu Oct 12, 2013
dae_get_hidden.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
dbm.m use v0_next for adaptive lr during the second-stage pretraining Jan 16, 2014
dbm_energy.m fixed: dbm_energy.m Apr 15, 2013
dbm_entropy.m Fix: local likelihood computation in adaptive learning rate (dbm.m) Apr 3, 2013
dbm_get_hidden.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
dbm_get_hidden_raw.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
dbm_sample.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
dbn.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
dbn_sample.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
default_convnet.m define C.conv.use_tanh Jan 6, 2014
default_dae.m Rectified linear hidden units for MLP, DAE, sDAE May 13, 2013
default_dbm.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
default_dbn.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
default_gsn.m . Jul 5, 2013
default_mlp.m Fixed dropout Jun 13, 2013
default_rbm.m removed unnecessary switch mf_update Feb 17, 2014
default_sdae.m Rectified linear hidden units for MLP, DAE, sDAE May 13, 2013
dsigmoid.m Rectified linear hidden units for MLP, DAE, sDAE May 13, 2013
gpl-2.0.txt Initial Version. Read TODO.txt for future work. Apr 1, 2013
grbm_energy.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
grbm_pt.m removed unnecessary switch mf_update Feb 17, 2014
grbm_sample.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
gsn.m Removed post-sigmoid noise injection in gsn.m Jul 10, 2013
gsn_sample.m Added Generative Stochastic Network (Bengio et al., 2013) Jul 4, 2013
logdiff.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
logsum.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
mlp.m fixed a bug where gpuArray is used even if use_gpu == 0 Oct 25, 2013
mlp_classify.m bug in mlp fixed Aug 13, 2013
mlp_dbm.m Bug fix for use_gpu Oct 12, 2013
padimages.m image padding for convnet Jul 23, 2013
print_n_updates.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
rbm.m removed unnecessary switch mf_update Feb 17, 2014
rbm_ais.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
rbm_energy.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
rbm_get_hidden.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
rbm_get_visible.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
rbm_pt.m removed unnecessary switch mf_update Feb 17, 2014
rbm_sample.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
save_intermediate.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
sdae.m Sami: fixed for the case of not using gpu Jun 10, 2014
sdae_get_hidden.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
sdae_get_visible.m Fixed a bug in sdae_get_visible.m Apr 24, 2013
set_dbm_centers.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
set_mlp_dbm.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
sigmoid.m Rectified linear hidden units for MLP, DAE, sDAE May 13, 2013
softmax.m bug in mlp fixed Aug 13, 2013
train_rbm.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
visualize.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
visualize_adv.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
visualize_grbm.m Fixed display functions for grbm and rbm. Jun 27, 2013
visualize_rbm.m Initial Version. Read TODO.txt for future work. Apr 1, 2013
zca.m bug fixed Jul 17, 2013
zca_whiten.m bug fixed Jul 17, 2013

README.md

deepmat

WARNING: this is not my main code, and there is no warranty attached!

= Generative Stochastic Network =

  • A simple implementation of GSN according to (Bengio et al., 2013)

= Convolutional Neural Network =

  • A naive implementation (purely using Matlab)
  • Pooling: max (Jonathan Masci's code) and average
  • Not for serious use!

= Restricted Boltzmann Machine & Deep Belief Networks =

  • Binary/Gaussian Visible Units + Binary Hidden Units
  • Enhanced Gradient, Adaptive Learning Rate
  • Adadelta for RBM
  • Contrastive Divergence
  • (Fast) Persistent Contrastive Divergence
  • Parallel Tempering
  • DBN: Up-down Learning Algorithm

= Deep Boltzmann Machine =

  • Binary/Gaussian Visible Units + Binary Hidden Units
  • (Persistent) Contrastive Divergence
  • Enhanced Gradient, Adaptive Learning Rate
  • Two-stage Pretraining Algorithm (example)
  • Centering Trick (fixed center variables only)

= Denoising Autoencoder (Tied Weights) =

  • Binary/Gaussian Visible Units + Binary(Sigmoid)/Gaussian Hidden Units
  • tanh/sigm/relu nonlinearities
  • Shallow: sparsity, contractive, soft-sparsity (log-cosh) regularization
  • Deep: stochastic backprop
  • Adagrad, Adadelta

= Multi-layer Perceptron =

  • Stochastic Backpropagation, Dropout
  • tanh/sigm/relu nonlinearities
  • Adagrad, Adadelta
  • Balanced minibatches using crossvalind()