Neural networks from scratch in Clojure

A series of neural network implementations in Clojure, building up from a version with no external dependencies to a version using neanderthal.

Structure

src/ contains four neural network implementation namespaces, each building upon the last.

resources/ contains the MNIST training data in a Clojure-friendly format.

mnist-scittle/ embeds code for handwritten digit recognition in a web app using scittle. Run it with bb mnist:

The recognition code — besides the weights and biases — is quite small, and doesn't have any dependencies. Of course, it's the training that's the hard part :)

Tests

It was enormously helpful to generate test vectors from existing neural network libraries and use them at the REPL during development, especially for the backpropagation algorithm.

Browse the files under src/ to see the tests inside rich comment blocks, for example this backpropagation gradient test.

Run tests with bb test.

Resources

I can recommend:

• 3Blue1Brown's video series on neural networks, with visualizations and intuitive explanations. Good initial context.
• Michael Nielsen's neural networks and deep learning tutorial, which uses Python and numpy.
• Andrej Karpathy's intro to neural networks and backpropagation, which is pure Python (no numpy).