This is a module that provides objects and functions that are useful for
constructing neural networks. The simplest way to use it is to go type
import neuralnetwork in an interactive Python shell, and go from there.
Supported layer types: tanh, sigmoid, relu, softplus, softmax
Supported cost functions: categorical crossentropy, binary crossentropy, quadratic
Training: minibatch SGD with optional momentum, optional early stopping
Regularization: optional L2
Evaluation methods: classification accuracy, autoencoder accuracy, cost value
Supported layer initialization methods: Glorot (for tanh and sigmoid), He (for relu and softplus)
Other features: saving and loading networks to the disk, constructing networks from arbitrary layers -- even ones trained in other networks, ensemble networks
Layer types: convolutional, lstm
Cost functions: negative log-likelihood
Training: optional learning rate schedule, optional momentum schedule
Regularization: optional dropout, optional minibatch normalization
Evaluation Methods: better autoencoder accuracy
Layer initialization methods: layer sequential unit variance (Mishkin, Matas 2016)
First install Theano
0.9.0 0.8.2 and
Numpy. Theano 0.9.0 works fine, but has a memory leak issue when
there is not a linked version of BLAS.
It's not noticable on small networks, but on large ones, it can lead to crashing the
system. You can configure Theano however you'd like, however, this code has not been
tested on a GPU, so procede with caution.
You can then clone the repository with
git clone https://github.com/pogrmman/NeuralNet.git
Run an interactive Python shell in the directory, and type
This repository contains three datasets frequently used for machine learning tasks - the MNIST handwriting dataset, the iris dataset, and the abalone dataset. The directories for each dataset contain pickles of the dataset, divided into a training set, a test set, and a validation set. The pickles are stored in a form usable by Network objects provided by neuralnetwork.py. You can access the datasets as follows:
with open($FILENAME$, "rb") as f: train, test, val = pickle.Unpickler(f).load()
The MNIST dataset provided here gives much better performance when it is scaled by dividing every value by 255.
load_imgs.py will properly load and scale the dataset into a training set, a test set, and a validaiton set.
After running it inside of a python instance, you'll have the variables train, test, and val that have their respective data.
It takes a moment because of the size of the dataset.
To create a basic neural network, you use a command like
net = neuralnetwork.Network(net_description,learning_rate) where
net_description is a list of tuples of the form
that describe the network.
You can train a network with
All functions, classes, and methods have a docstring containing usage
help() can be used to find usage information.