Matlab Environment for Deep Architecture Learning
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data
demo Bug fix in crbm.m object (Thanks Adepu Ravi Sankar) Feb 12, 2014
models
utils
visualizations
.config.mat
LICENSE
README
startLearning.m

README

-------------------------------------------------------------------------------
Matlab Environment for Deep Architecture Learning (MEDAL) - version 0.1
-------------------------------------------------------------------------------

   o   o
  / \ / \ EDAL
 o   o   o

Model Objects:
	mlnn.m        -- Multi-layer neural network
	mlcnn.m       -- Multi-layer convolutional neural network
	rbm.m         -- Restricted Boltzmann machine (RBM)
	mcrbm.m       -- Mean-covariance (3-way Factored) RBM
	drbm.m        -- Dynamic/conditional RBM 
	dbn.m         -- Deep Belief Network 
	crbm.m        -- Convolutional RBM
	ae.m          -- Shallow autoencoder 
	dae.m         -- Deep Autoencoder 
	
-------------------------------------------------------------------------------
To begin type:

>> startLearning

in the medal directory

To get an idea of how the model objects work, check out the demo script:

>> deepLearningExamples('all')

These examples are by no means optimized, but are for getting familiar with 
the code.If you have any questions or bugs, send them my way:
 
stan_s_bury@berkeley.edu
-------------------------------------------------------------------------------
References:

*Neural Networks/Backpropagations:
 Rumelhart, D. et al. "Learning representations by back-propagating errors".
 Nature 323 (6088): 533–536. 1986.

*Restricted Boltzmann Machines/Contrastive Divergence
 Hinton, G. E. "Training Products of Experts by Minimizing Contrastive
 Divergence". Neural Computation 14 (8): 1771–1800. 2002

*Deep Belief Networks:
 Bengio, Y., Lamblin, P., Popovici, P., Larochelle, H. "Greedy Layer-Wise
 Training of Deep Networks" NIPS 2006

*Deep & Denoising Autoencoders
 Hinton, G. E. and Salakhutdinov, R. R "Reducing the dimensionality of data with
 neural networks." Science, Vol. 313. no. 5786, pp. 504 - 507, 28 July 2006.

*Pascal, V. et al. “Stacked denoising autoencoders: Learning useful
 representations in a deep network with a local denoising criterion.“ The
 Journal of Machine Learning Research 11:3371-3408. 2010

*Mean-Covariance/3-way Factored RBMs:
 Ranzato M. et al. "Modeling Pixel Means and Covariances Using
 Factorized Third-Order Boltzmann Machines." CVPR 2012.

*Dynamic/Conditional RBMs:
 Taylor G. et al. "Modeling Human Motion Using Binary Latent
 Variables" NIPS 2006.

*Convolutional MLNNs:
 LeCun, Y., et al. "Gradient-based learning applied to document recognition".
 Proceedings of the IEEE, 86(11), 2278–2324. 2008

 Krizhevsky, A et al. "ImageNet Classification with Deep Convolutional Neural
 Networks." NIPS 2012.

*Convolutional RBMs:
 Lee, H. et al. “Convolutional deep belief networks for scalable unsupervised
 learning of hierarchical representations.”, ICML 2009

*Rectified Linear Units
 Nair V., Hinton GE. (2010) Rectified Linear Units Improve Restricted Boltzmann Machines. IMCL 2010.

 Glorot, X. Bordes A. & Bengio Y. (2011). "Deep sparse rectifier neural 
 networks". AISTATS 2011.

*Dropout Regularization:
 Hinton GE et al. Technical Report, Univ. of Toronto, 2012.
 
*General
 Hinton, G. E. "A practical guide to training restricted Boltzmann machines"
 Technical Report, Univ. of Toronto, 2010.
-------------------------------------------------------------------------