willyai

Simple deep learning library based on one I built for a friend called Will (thus the silly names). By now I'm endeared to the names so they're staying :)

It's mostly a learning project but I can see some benefits of using this: it's based entirely on numpy so only needs a CPU; it's easily parallelisable, unlike a tensorflow or keras session; and it's pretty fast (if you don't compare to deep learning libraries that exploit GPUs -- it's about the same speed as an sklearn NN).

The way this is structured means deep networks can be built by layers, as in Keras:

from willyai import DeepWilly
from willyai.willies import ConnectedWilly, ConvolutionalWilly, DropoutWilly, StackingWilly, PoolingWilly

dnet = DeepWilly(cost_func = 'cross entropy')

dnet.add(ConvolutionalWilly(20, (3, 3), in_shape = tuple(X.shape[1:])))
dnet.add(PoolingWilly((2, 2), pool_type = 'max'))
dnet.add(StackingWilly())
dnet.add(ConnectedWilly(100, act_func = 'sigmoid'))
dnet.add(DropoutWilly(0.2))
dnet.add(ConnectedWilly(n_out, act_func = 'sigmoid'))

dnet.train(X_train, y_train, 
           num_iterations = 100, 
           batch_size = 75)

print('Accuracy:', dnet.accuracy(X_test, y_test, pred_type = 'argmax'))

There is also a separate class for building simple fully-connected neural networks. This works as in sklearn:

from willyai import WillyNet

net = WillyNet(shape = NN_shape, 
               problem = 'classification')

net.train(X_train, y_train, 
          num_iterations = 100, 
          batch_size = 75)

print('Accuracy:', net.accuracy(X_test, y_test, pred_type = 'argmax'))

There are two scripts that use these classes (testWN.py for WillyNet and testDW.py for DeepWilly) to classify handwritten digits using a subset of the MNIST dataset (available in data). The latter, testDW.py, hasn't had it's parameters optimised but that's not the point of the script -- I'm using it to show the classes work, not to actually try to classify digits. I'm going to turn those into a notebook for ease of reading but it's not done yet.

I may add more activations, costs, etc. but fear that would stray too far from the spirit of the project. I'm more likely to add a recurring (LSTM or other) layer and one or two other optimising methods (RMSProp and/or Adam). Maybe a softmax activation and log likelihood cost. I want to keep it simple.

The fully-connected WillyNet does have alternative optimisers (SGD, NAG, Adagrad and Adam).