Simple feedforward neural network class "handmade" in pure python+numpy
Winner of Siraj Ravel's coding challange
- Fully connected feed forward neural network with as many hidden layers as needed, of any size.
- Implements backpropagation : link 1, link 2, link 3 (does not implement gradient checking)
- Uses gradient descent for optimization
- Uses Sum Squared Error cost function
- Uses the sigmoid activation function
How to use
- Create an instance
nn = NeuralNetwork()
- Add layers
The first input layer is created automatically
nn.train(features, targets, num_epochs, learning_rate)
nn.train() method has an optional parameter called
stop_accuracy. At the end of each epoch the mean loss is calculated and if it is under the specified threshold than the training stops. This avoids training longer than necessary. By looking at the number of epochs needed to reach the threshold, it gives us a good metric as to the performance of our hyperparameters.
Example: XOR function
The Jupyter Notebook shows how the network can be used to approximate the XOR function using a 3-layer neural network. We attempt to find the optimal network dimensions and the optimal learning rate.