Machine learning in Clojure
Clojure Mathematica Python Cuda Java Shell C++
Latest commit ae21190 Jan 18, 2017 @poojaramesh poojaramesh committed with cnuernber Added more information to the exception (#73)
* Clearer error output for network batching

* Clearer error output for network batching
Permalink
Failed to load latest commit information.
caffe Major change to travis to avoid deploying to clojars, ever. Jan 13, 2017
compute Added more information to the exception (#73) Jan 18, 2017
cortex [cortex/dataset] map-sequence->dataset docstring improvement. (#70) Jan 18, 2017
datasets Major change to travis to avoid deploying to clojars, ever. Jan 13, 2017
examples/suite-classification Update example instructions with front end build commands (#69) Jan 16, 2017
gpu-compute Upgrading project files to 0.4.0-SNAPSHOT in preparation for a 0.4.0 … Jan 11, 2017
keras Major change to travis to avoid deploying to clojars, ever. Jan 13, 2017
optimise Appease think.quality Jan 13, 2017
sage Local response normalization (#27) Nov 6, 2016
suite Upgrading project files to 0.4.0-SNAPSHOT in preparation for a 0.4.0 … Jan 11, 2017
visualization Upgrading project files to 0.4.0-SNAPSHOT in preparation for a 0.4.0 … Jan 11, 2017
.gitignore Kicking screaming 21st century (#49) Dec 9, 2016
.travis.yml fixing build. Jan 13, 2017
Dockerfile Major change to travis to avoid deploying to clojars, ever. Jan 13, 2017
LICENSE.md Add Eclipse Public License Jan 12, 2017
README.md Move cortex.optimise docs to README Jan 13, 2017
ROADMAP.md Added design information to README.md instead of in different document. Oct 11, 2016
build-and-test.sh Major change to travis to avoid deploying to clojars, ever. Jan 13, 2017
build-docker-and-test.sh Major change to travis to avoid deploying to clojars, ever. Jan 13, 2017
design.md Unify cortex compute optimise (#59) Jan 2, 2017
local-install.sh Adding more libraries to a local cortex install Nov 26, 2016

README.md

Cortex TravisCI

Neural networks, regression and feature learning in Clojure.

Cortex has been developed by ThinkTopic in collaboration with Mike Anderson.

Mailing List

https://groups.google.com/forum/#!forum/clojure-cortex

Usage

Cortex has a 0.3.0 release meaning all libraries are released on clojars. This is very preliminary and I would expect quite a few things to change over time but it should allow you to train some initial classifiers or regressions.

Cortex Design

Design is detailed here: Cortex Design Document

Please see the various unit tests and examples for training a model. Specifically see: mnist verification

Also, for an example of using cortex in a more real-world scenario please see: mnist example.

Existing Framework Comparisons

  • Stanford CS 231 Lecture 12 contains a detailed breakdown of Caffe, Torch, Theano, and TensorFlow.

TODO:

  • hdf5 import of major keras models (vgg-net). This requires each model along with a single input and per-layer outputs for that input. Please don't ask for anything to be supported unless you can provide the appropriate thorough test.

  • Recurrence in all forms. There is some work towards that direction in the compute branch and it is specifically designed to match the cudnn API for recurrence. This is less important at this point than running some of the larger pre-trained models.

  • Graph-based description layer. This will make doing things like res-nets and dense-nets easier and repeatable.

  • Better training - currently there are lots of things that don't train as well as we would like. This could be because we are using Adam exclusively instead of sgd, it could be because of bugs in the code or it could be because we need different weight initialization. In any case, building larger nets that train better is of course of critical importance.

  • Speaking of larger nets, multiple GPU support and multiple machine support (which could be helped by the above graph based description layer).

  • Profiling GPU system to make sure we are using as much GPU as possible in the single-gpu case.

  • Better data import/augmentation systems. Basically inline augmentation of data so the net never sees the same training example twice.

  • Better data import/visualization support. We need a solid panda-equivalent with some level of visualization and feature parity and it isn't clear the best way to get this. Currently there are three different 'dataset' abstractions in clojure it isn't clear if any of them support the level of indirection and features that panda does.

Getting Started:

  • Get the project and run lein test in both cortex and compute. The various unit tests train various models.

GPU Compute Install Instructions

Ubuntu

Basic steps include, at minimum: Installing nvidia-cuda-toolkit. and installing cudnn available from here: https://developer.nvidia.com/cudnn publicly.

$ sudo apt-get install nvidia-cuda-toolkit nvidia-361 libcuda1-361

The .zip contains some libraries that you will need to make available to the loader. I simply copied the library files to /usr/lib, though I'm sure there's a better way of doing this.

Depending on which distribution you're on you will either have cuda7.5 or cuda8.0. Current master is 7.5, if you're running 8.0 you will need to use the following branch (basically specifies a different dep for the jni bindings -- o/w code is identical):

https://github.com/thinktopic/cortex/tree/cuda-8.0

Mac OS

My install steps on Mac OSX were:

Followed the instructions for gpu setup from Tensor Flow Brew install coreutils and CUDA toolkit

$ brew install coreutils
$ brew tap caskroom/cask
$ brew cask install cuda

Add CUDA Tool kit to bash profile

export CUDA_HOME=/usr/local/cuda
export DYLD_LIBRARY_PATH="$DYLD_LIBRARY_PATH:$CUDA_HOME/lib"
export PATH="$CUDA_HOME/bin:$PATH"

Download the CUDA Deep Neural Network libraries

Once downloaded and unzipping, moving the files:

$ sudo mv include/cudnn.h /Developer/NVIDIA/CUDA-8.0/include/
$ sudo mv lib/libcudnn* /Developer/NVIDIA/CUDA-8.0/lib
$ sudo ln -s /Developer/NVIDIA/CUDA-8.0/lib/libcudnn* /usr/local/cuda/lib/

See also:

Roadmap