Skip to content
Simple GPGPU CUDA C++ code for training Deep Neural Networks (DNN)
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
DeeperThought
results
README.md
compile.sh

README.md

Deeper Thought

Deeper Thought is general framework for training Deep Neural Networks (and convolutional as well).

To compile on linux:

./compile.sh

To run training using your train and test data from command line execute:

./dt configFile trainFile testFile batchSize(integer) paramFile/null saveEveryNEpochs(integer) square/log wheremax/netflix/none

Input format of data:

expOut_1, ... , expOut_n, inp_1, ... , inp_m

For both trainFile and testFile (expOut - expected output, inp - input). One data point is one line.

Good research publications on the topic:

Results for MNIST dataset:

You can download MNIST dataset here. Mnist dataset can be found in other formats on Yann Lecun's web page or Kaggle's web page.

configA.txt (logistic regression)

batch size 4000, squared error

matrix,784,10,0.5,0.0001

sigmoid,10

graphA

Top accuracy on test data: 92.47 %

configC.txt (logistic regression)

batch size 4000, squared error, auto step size

matrix,784,10,0.5,-0.001

sigmoid,10

graphC

Top accuracy on test data: 92.53 %

configB.txt (simple 2 layered network with dropout)

batch size 4000, squared error

matrix,784,100,0.5,0.001

sigmoid,100

dropout,100,0.25

matrix,100,10,0.5,0.0001

sigmoid,10

graphB

Top accuracy on test data: 96.24 %

configD.txt (simple 2 layered network with dropout)

batch size 4000, squared error, auto step size

matrix,784,100,0.5,-0.01

sigmoid,100

dropout,100,0.25

matrix,100,10,0.5,-0.01

sigmoid,10

graphD

Top accuracy on test data: 96.33 %

configD.txt

batch size 4000, log loss error, auto step size

matrix,784,100,0.5,-0.01

sigmoid,100

dropout,100,0.25

matrix,100,10,0.5,-0.01

sigmoid,10

graphDlog

Top accuracy on test data: 96.73 %

configE.txt (convolutional neural network CNN)

batch size 1000, log loss error, auto step size

convolution,1,28,28,200,5,5,0.5,-0.001

max,200,24,24,8,8

matrix,1800,100,0.5,-0.001

sigmoid,100

dropout,100,0.25

matrix,100,10,0.5,-0.001

sigmoid,10

graphE

Top accuracy on test data: 99.13 %

configF.txt (convolutional neural network CNN)

batch size 4000, log loss error, auto step size

convolution,1,28,28,400,5,5,0.5,-0.001

max,400,24,24,6,6

matrix,6400,150,0.5,-0.001

sigmoid,150

dropout,150,0.25

matrix,150,10,0.5,-0.001

softmax,10

graphF

Top accuracy on test data: 99.36 %

configG.txt (convolutional neural network CNN)

batch size 4000, log loss error, auto step size

convolution,1,28,28,200,8,8,0.5,-0.001

max,200,21,21,7,7

matrix,1800,130,0.5,-0.001

sigmoid,130

dropout,130,0.5

matrix,130,10,0.5,-0.001

softmax,10

graphG

Top accuracy on test data: 99.41 %

What you need to download and install beforehand:

Other alternatives for training DNNs:

You can’t perform that action at this time.