A Lightweight Deep Learning Library in MATLAB
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
Data
MatDL
LICENSE
README.md
paper.bib
paper.md

README.md

MatDL

status DOI

MatDL is an open-source lightweight deep learning library native in MATLAB that implements some most commonly used deep learning algorithms. The library comprises functions that implement the following: (1) basic building blocks of modern neural networks such as affine transformations, convolutions, nonlinear operations, dropout, batch normalization, etc.; (2) popular architectures such as deep neural networks (DNNs), convolutional neural networks (ConvNets), and recurrent neural networks (RNNs) and their variant, the long short-term memory (LSTM) RNNs; (3) optimizers such stochastic gradient descent (SGD), RMSProp and ADAM; as well as (4) boilerplate functions for training, gradients checking, etc.

Installation

  • Add MatDL to your path:
addpath(genpath('MatDL'));
  • Compile the C MEX files using the Makefile or using the following:
cd MatDL/convnet; mex col2im_mex.c; mex im2col_mex.c; cd ..; cd ..;

Project Layout

common/: Basic building blocks of neural networks such as nonlinear functions, etc.

convnet/: ConvNet specific functions such as convolutional layers, max pooling layers, etc.

nn/: NN specific functions.

optim/: Optimization algorithms such as SGD, RMSProp, ADAM, etc.

rnn/: RNN and LSTM functions.

train/: Functions for gradients checking, training, and prediction.

zoo/: Samples of various models definitions and initializations.

Usage

This is a sample complete minimum working example: (Examples: DNN (Below), ConvNet, RNN)

% A complete minimum working example.
%% Init
clear all
addpath(genpath('../MatDL'));

%% Load data
load('../Data/mnist_uint8.mat'); % Replace with your data file
X = double(train_x)/255; Y = double(train_y);
XVal = double(test_x)/255; YVal =  double(test_y);

rng(0);

%% Initialize model
opt = struct;
[model, opt] = init_six_nn_bn(784, 10, [100, 100, 100, 100, 100], opt);

%% Hyper-parameters
opt.batchSize = 100;

opt.optim = @rmsprop;
% opt.beta1 = 0.9; opt.beta2 = 0.999; opt.t = 0; opt.mgrads = opt.vgrads;
opt.rmspropDecay = 0.99;
% opt.initialMomentum = 0.5; opt.switchEpochMomentum = 1; opt.finalMomentum = 0.9;
opt.learningRate = 0.01;
opt.learningDecaySchedule = 'stepsave'; % 'no_decay', 't/T', 'step'
opt.learningDecayRate = 0.5;
opt.learningDecayRateStep = 5;

opt.dropout = 0.5;
opt.weightDecay = false;
opt.maxNorm = false;

opt.maxEpochs = 100;
opt.earlyStoppingPatience = 20;
opt.valFreq = 100;

opt.plotProgress = true;
opt.extractFeature = false;
opt.computeDX = false;

opt.useGPU = false;
if (opt.useGPU) % Copy data, dropout, model, vgrads, BNParams
    X = gpuArray(X); Y = gpuArray(Y); XVal = gpuArray(XVal); YVal = gpuArray(YVal); 
    opt.dropout = gpuArray(opt.dropout);
    p = fieldnames(model);
    for i = 1:numel(p), model.(p{i}) = gpuArray(model.(p{i})); opt.vgrads.(p{i}) = gpuArray(opt.vgrads.(p{i})); end
    p = fieldnames(opt);
    for i = 1:numel(p), if (strfind(p{i},'bnParam')), opt.(p{i}).runningMean = gpuArray(opt.(p{i}).runningMean); opt.(p{i}).runningVar = gpuArray(opt.(p{i}).runningVar); end; end
end

%% Gradient check
x = X(1:100,:);
y = Y(1:100,:);
maxRelError = gradcheck(@six_nn_bn, x, model, y, opt, 10);

%% Train
[model, trainLoss, trainAccuracy, valLoss, valAccuracy, opt] = train(X, Y, XVal, YVal, model, @six_nn_bn, opt);

%% Predict
[yplabel, confidence, classes, classConfidences, yp] = predict(XVal, @six_nn_bn, model, opt)

Contributing

Contributions are highly welcome!

If you wish to contribute, follow these steps:

  • Create a personal fork of the MatDL GitHub repository.
  • Make your changes in a branch named other than master.
  • Follow the same coding and documentation style for consistency.
  • Make sure that your changes do not break any of the existing functionality.
  • Submit a pull request.

Please use Github issues to report bugs or request features.

Citation

If you use this library in your research, please cite:

Fayek, (2017). MatDL: A Lightweight Deep Learning Library in MATLAB. Journal of Open Source Software, 2(19), 413, doi:10.21105/joss.00413

@article{Fayek2017,
    author       = {Haytham M. Fayek},
    title        = {{MatDL}: A Lightweight Deep Learning Library in {MATLAB}},
    journal      = {Journal of Open Source Software},
    year         = {2017},
    month        = {nov},
    volume       = {2},
    number       = {19},
    doi          = {10.21105/joss.00413},
    url          = {https://doi.org/10.21105/joss.00413},
    publisher    = {The Open Journal},
}

References

MatDL was inspired by Stanford's CS231n and Torch, and is conceptually similar to Keras and Lasagne. Torch, keras and Lasagne are more suited for large-scale experiments.

License

MIT