Skip to content

yimeitang/MNISTNeuralNetwork

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

8 Commits
 
 
 
 

Repository files navigation

MNISTNeuralNetwork

1. This model allows you to choose

(1)Different early stopping criteria
(2)Different cost functions: a.Quadratic; b.Cross-entropy; c.Log-likelihood;
(3)SGD or Momentum
(4)No regularization or L2 Rgularization
(5)Different transfer(activation) functions: a.tanh; b.softmax; c.ReLU;

2. The model uses better initial weights and minibatch shuffling, it will return learned network and accuracy/costs for your train set ,validation set and test set.

3. Parameters:

The parameters in the function are :
(1) inputs: a matrix with a column for each example, and a row for each input feature.
(2) targets: a matrix with a column for each example, and a row for each output feature.
(3) split: how the data will be divided into training set, test set and validation set.
(4) nodeLayers: a vector with the number of nodes in each layer (including the input and output layers). Important: Your code should not assume that there are just three layers of nodes. It should work with a network of any size.
(5) numEpochs: (scalar) desired number of epochs to run.
(6) batchSize: (scalar) number of instances in a mini-batch.
(7) eta: (scalar) learning rate.
(8) costFunOption: 0 stands for quadradic, 1 stands for cross entropy , 2 stands for log likelihood.
(9) actFunOption: 0 stands for sigmoid, 1 stands for tanh, 2 stands for softmax, 3 stands for ReLU.
(10) momentum: the value of momentum.
(11) lambda: the value of lambda.

4. Outputs:

(Early Stopping: In my code, I set the training to terminate early before all epochs have run when the difference between the accuracy score of training set and validation set is larger or equal to 0.15. The reason is that if the difference is larger or equal than 0.15, it means that the model is overfitting and result will not be well generalized.) testcases tc1 tc1_plot tc2 tc2_plot tc3 tc3_plot tc4 tc4_plot tc5 tc5_plot tc6 tc6_plot tc7 tc7_plot tc8 tc8_plot tc9 tc9_plot

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published