Skip to content

haythamfayek/MatDL

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 

MatDL

status DOI

MatDL is an open-source lightweight deep learning library native in MATLAB that implements some most commonly used deep learning algorithms. The library comprises functions that implement the following: (1) basic building blocks of modern neural networks such as affine transformations, convolutions, nonlinear operations, dropout, batch normalization, etc.; (2) popular architectures such as deep neural networks (DNNs), convolutional neural networks (ConvNets), and recurrent neural networks (RNNs) and their variant, the long short-term memory (LSTM) RNNs; (3) optimizers such stochastic gradient descent (SGD), RMSProp and ADAM; as well as (4) boilerplate functions for training, gradients checking, etc.

Installation

  • Add MatDL to your path:
addpath(genpath('MatDL'));
  • Compile the C MEX files using the Makefile or using the following:
cd MatDL/convnet; mex col2im_mex.c; mex im2col_mex.c; cd ..; cd ..;

Project Layout

common/: Basic building blocks of neural networks such as nonlinear functions, etc.

convnet/: ConvNet specific functions such as convolutional layers, max pooling layers, etc.

nn/: NN specific functions.

optim/: Optimization algorithms such as SGD, RMSProp, ADAM, etc.

rnn/: RNN and LSTM functions.

train/: Functions for gradients checking, training, and prediction.

zoo/: Samples of various models definitions and initializations.

Usage

This is a sample complete minimum working example: (Examples: DNN (Below), ConvNet, RNN)

% A complete minimum working example.
%% Init
clear all
addpath(genpath('../MatDL'));

%% Load data
load('../Data/mnist_uint8.mat'); % Replace with your data file
X = double(train_x)/255; Y = double(train_y);
XVal = double(test_x)/255; YVal =  double(test_y);

rng(0);

%% Initialize model
opt = struct;
[model, opt] = init_six_nn_bn(784, 10, [100, 100, 100, 100, 100], opt);

%% Hyper-parameters
opt.batchSize = 100;

opt.optim = @rmsprop;
% opt.beta1 = 0.9; opt.beta2 = 0.999; opt.t = 0; opt.mgrads = opt.vgrads;
opt.rmspropDecay = 0.99;
% opt.initialMomentum = 0.5; opt.switchEpochMomentum = 1; opt.finalMomentum = 0.9;
opt.learningRate = 0.01;
opt.learningDecaySchedule = 'stepsave'; % 'no_decay', 't/T', 'step'
opt.learningDecayRate = 0.5;
opt.learningDecayRateStep = 5;

opt.dropout = 0.5;
opt.weightDecay = false;
opt.maxNorm = false;

opt.maxEpochs = 100;
opt.earlyStoppingPatience = 20;
opt.valFreq = 100;

opt.plotProgress = true;
opt.extractFeature = false;
opt.computeDX = false;

opt.useGPU = false;
if (opt.useGPU) % Copy data, dropout, model, vgrads, BNParams
    X = gpuArray(X); Y = gpuArray(Y); XVal = gpuArray(XVal); YVal = gpuArray(YVal); 
    opt.dropout = gpuArray(opt.dropout);
    p = fieldnames(model);
    for i = 1:numel(p), model.(p{i}) = gpuArray(model.(p{i})); opt.vgrads.(p{i}) = gpuArray(opt.vgrads.(p{i})); end
    p = fieldnames(opt);
    for i = 1:numel(p), if (strfind(p{i},'bnParam')), opt.(p{i}).runningMean = gpuArray(opt.(p{i}).runningMean); opt.(p{i}).runningVar = gpuArray(opt.(p{i}).runningVar); end; end
end

%% Gradient check
x = X(1:100,:);
y = Y(1:100,:);
maxRelError = gradcheck(@six_nn_bn, x, model, y, opt, 10);

%% Train
[model, trainLoss, trainAccuracy, valLoss, valAccuracy, opt] = train(X, Y, XVal, YVal, model, @six_nn_bn, opt);

%% Predict
[yplabel, confidence, classes, classConfidences, yp] = predict(XVal, @six_nn_bn, model, opt)

Contributing

Contributions are highly welcome!

If you wish to contribute, follow these steps:

  • Create a personal fork of the MatDL GitHub repository.
  • Make your changes in a branch named other than master.
  • Follow the same coding and documentation style for consistency.
  • Make sure that your changes do not break any of the existing functionality.
  • Submit a pull request.

Please use Github issues to report bugs or request features.

Citation

If you use this library in your research, please cite:

Fayek, (2017). MatDL: A Lightweight Deep Learning Library in MATLAB. Journal of Open Source Software, 2(19), 413, doi:10.21105/joss.00413

@article{Fayek2017,
    author       = {Haytham M. Fayek},
    title        = {{MatDL}: A Lightweight Deep Learning Library in {MATLAB}},
    journal      = {Journal of Open Source Software},
    year         = {2017},
    month        = {nov},
    volume       = {2},
    number       = {19},
    doi          = {10.21105/joss.00413},
    url          = {https://doi.org/10.21105/joss.00413},
    publisher    = {The Open Journal},
}

References

MatDL was inspired by Stanford's CS231n and Torch, and is conceptually similar to Keras and Lasagne. Torch, keras and Lasagne are more suited for large-scale experiments.

License

MIT