Neural nets in SharpLearning

Introduction

This guide provides a short introduction to using SharpLearning.Neural for training and using neural nets.

SharpLearning.Neural contains layers for constructing standard fully-connected neural networks and convolutaional neural networks.

To create a NeuralNetLearner, the neural net architecture first has to be defined. This is done using the NeuralNet class together with the layer classes. Following the NeuralNet has to be parsed to a NeuralNetLearner in order to train the network. An example showing how to create a convolutional neural network can be seen below:

// Define the neural net architecture
var net = new NeuralNet();
net.Add(new InputLayer(28, 28, 1));
net.Add(new Conv2DLayer(5, 5, 32));
net.Add(new MaxPool2DLayer(2, 2));
net.Add(new Conv2DLayer(5, 5, 32));
net.Add(new MaxPool2DLayer(2, 2));
net.Add(new DropoutLayer(0.5));
net.Add(new DenseLayer(256, Activation.Relu));
net.Add(new DropoutLayer(0.5));
net.Add(new SoftMaxLayer(numberOfClasses));

// Create a classification neural net learner
var learner = new ClassificationNeuralNetLearner(net, loss: new LogLoss());

After the learner is created, it is used in exactly the same way as all the other learners in SharpLearning.

// Train the neural network model
var model = learner.Learn(observations, targets);

// Use the model for predicting new observations.
var predictions = model.Predict(testObservations);

There is support for classification and regression using the ClassificationNeuralNetLearner and the RegressionNeuralNetLearner.

More examples on how to use the neural net learners can be found in SharpLearning.Examples/NeuralNets

Data layout

The neural net models and learners expects the input data to have a layout of [Width, Height, Channel, Batch]. Since SharpLearning does not currently support tensors, image data for a neural net must be transformed to a matrix layout, where each image becomes a row in the matrix. An example of the layout can be seen below:

This layout generalises to fewer and more channels.

The corresponding InputLayer descripes the original dimensions of the data and would look like this:

new InputLayer(height: 2, width: 2, depth: 3)

Math.net numerics for matrix operations

SharpLearning.Neural uses Math.net numerics for matrix operations. This makes it possible to use the MKL and OpenBLAS providers for better performance. As default SharpLearning, uses the MKL provider and the nuget package for SharpLearning.Neural will automatically install Math.Net and the MKL provider.

Future work

• Support for CNTK backend.
• Tensor support in SharpLearning, to make it easier to work with data in the networks.
• Support for tensorflow backend via TensorFlowSharp