MatDL
MatDL is an open-source lightweight deep learning library native in MATLAB that implements some most commonly used deep learning algorithms. The library comprises functions that implement the following: (1) basic building blocks of modern neural networks such as affine transformations, convolutions, nonlinear operations, dropout, batch normalization, etc.; (2) popular architectures such as deep neural networks (DNNs), convolutional neural networks (ConvNets), and recurrent neural networks (RNNs) and their variant, the long short-term memory (LSTM) RNNs; (3) optimizers such stochastic gradient descent (SGD), RMSProp and ADAM; as well as (4) boilerplate functions for training, gradients checking, etc.
Installation
- Add MatDL to your path:
addpath(genpath('MatDL'));
- Compile the C MEX files using the
Makefile
or using the following:
cd MatDL/convnet; mex col2im_mex.c; mex im2col_mex.c; cd ..; cd ..;
Project Layout
common/
: Basic building blocks of neural networks such as nonlinear functions, etc.
convnet/
: ConvNet specific functions such as convolutional layers, max pooling layers, etc.
nn/
: NN specific functions.
optim/
: Optimization algorithms such as SGD, RMSProp, ADAM, etc.
rnn/
: RNN and LSTM functions.
train/
: Functions for gradients checking, training, and prediction.
zoo/
: Samples of various models definitions and initializations.
Usage
This is a sample complete minimum working example: (Examples: DNN (Below), ConvNet, RNN)
% A complete minimum working example.
%% Init
clear all
addpath(genpath('../MatDL'));
%% Load data
load('../Data/mnist_uint8.mat'); % Replace with your data file
X = double(train_x)/255; Y = double(train_y);
XVal = double(test_x)/255; YVal = double(test_y);
rng(0);
%% Initialize model
opt = struct;
[model, opt] = init_six_nn_bn(784, 10, [100, 100, 100, 100, 100], opt);
%% Hyper-parameters
opt.batchSize = 100;
opt.optim = @rmsprop;
% opt.beta1 = 0.9; opt.beta2 = 0.999; opt.t = 0; opt.mgrads = opt.vgrads;
opt.rmspropDecay = 0.99;
% opt.initialMomentum = 0.5; opt.switchEpochMomentum = 1; opt.finalMomentum = 0.9;
opt.learningRate = 0.01;
opt.learningDecaySchedule = 'stepsave'; % 'no_decay', 't/T', 'step'
opt.learningDecayRate = 0.5;
opt.learningDecayRateStep = 5;
opt.dropout = 0.5;
opt.weightDecay = false;
opt.maxNorm = false;
opt.maxEpochs = 100;
opt.earlyStoppingPatience = 20;
opt.valFreq = 100;
opt.plotProgress = true;
opt.extractFeature = false;
opt.computeDX = false;
opt.useGPU = false;
if (opt.useGPU) % Copy data, dropout, model, vgrads, BNParams
X = gpuArray(X); Y = gpuArray(Y); XVal = gpuArray(XVal); YVal = gpuArray(YVal);
opt.dropout = gpuArray(opt.dropout);
p = fieldnames(model);
for i = 1:numel(p), model.(p{i}) = gpuArray(model.(p{i})); opt.vgrads.(p{i}) = gpuArray(opt.vgrads.(p{i})); end
p = fieldnames(opt);
for i = 1:numel(p), if (strfind(p{i},'bnParam')), opt.(p{i}).runningMean = gpuArray(opt.(p{i}).runningMean); opt.(p{i}).runningVar = gpuArray(opt.(p{i}).runningVar); end; end
end
%% Gradient check
x = X(1:100,:);
y = Y(1:100,:);
maxRelError = gradcheck(@six_nn_bn, x, model, y, opt, 10);
%% Train
[model, trainLoss, trainAccuracy, valLoss, valAccuracy, opt] = train(X, Y, XVal, YVal, model, @six_nn_bn, opt);
%% Predict
[yplabel, confidence, classes, classConfidences, yp] = predict(XVal, @six_nn_bn, model, opt)
Contributing
Contributions are highly welcome!
If you wish to contribute, follow these steps:
- Create a personal fork of the MatDL GitHub repository.
- Make your changes in a branch named other than
master
. - Follow the same coding and documentation style for consistency.
- Make sure that your changes do not break any of the existing functionality.
- Submit a pull request.
Please use Github issues to report bugs or request features.
Citation
If you use this library in your research, please cite:
Fayek, (2017). MatDL: A Lightweight Deep Learning Library in MATLAB. Journal of Open Source Software, 2(19), 413, doi:10.21105/joss.00413
@article{Fayek2017,
author = {Haytham M. Fayek},
title = {{MatDL}: A Lightweight Deep Learning Library in {MATLAB}},
journal = {Journal of Open Source Software},
year = {2017},
month = {nov},
volume = {2},
number = {19},
doi = {10.21105/joss.00413},
url = {https://doi.org/10.21105/joss.00413},
publisher = {The Open Journal},
}
References
MatDL was inspired by Stanford's CS231n and Torch, and is conceptually similar to Keras and Lasagne. Torch, keras and Lasagne are more suited for large-scale experiments.
License
MIT