Awesome C++ BLAS/Matrix Libraries and Machine/Reinforcement Learning Frameworks

My curated list of C++ (GPU) matrix/BLAS libraries and machine learning/reinforcement learning frameworks.

Free/Open Source BLAS/Matrix Libraries

Libraries:

• ViennaCL: a free open-source linear algebra library for computations on many-core architectures (GPUs, MIC) and multi-core CPUs. The library is written in C++ and supports CUDA, OpenCL, and OpenMP (including switches at runtime).
• CUSP: a library for sparse linear algebra and graph computations based on Thrust. Cusp provides a flexible, high-level interface for manipulating sparse matrices and solving sparse linear systems.
• MAGMA: a dense linear algebra library similar to LAPACK but for heterogeneous/hybrid architectures, starting with current "Multicore+GPU" systems.
• NVidia cuBLAS: a fast GPU-accelerated implementation of the standard basic linear algebra subroutines (BLAS).
• CULA: a set of GPU-accelerated linear algebra libraries utilizing the NVIDIA CUDA parallel computing architecture to dramatically improve the computation speed of sophisticated mathematics.
• Boost uBlas: a C++ template class library that provides BLAS level 1, 2, 3 functionality for dense, packed and sparse matrices. The design and implementation unify mathematical notation via operator overloading and efficient code generation via expression templates.
• CUV: a C++ template and Python library which makes it easy to use NVIDIA(tm) CUDA.
• Eigen: a C++ template library for linear algebra: matrices, vectors, numerical solvers, and related algorithms.
• Armadillo: a high quality linear algebra library (matrix maths) for the C++ language, aiming towards a good balance between speed and ease of use.

The features listed here are based on my casual observations on Dec 2017. If you see a feature is not checked, it could be because it is not supported or I didn't find it/didn't have time to find it out.

Comparison:

	ViennaCL	CUSP	MAGMA	cuBLAS	CULA	uBLAS	CUV	Eigen	Armadillo
Language	C++	C++	C	C	C	C++	C++	c++98	C++98 - 14
License	BSD like	Apache	BSD like	free	free/$	Boost	BSD like	MPL2	Apache 2
Created	2010	2009		2007?	2009	2004		2009	2008
Last release	Jan 2016	Apr 2015	Nov 2017		Apr 2014	May 2016	Sep 2015	Jun 2017	Dec 2017
Active	?	✗	✓	✓	✗	?	✗	✓	✓
	ViennaCL	CUSP	MAGMA	cuBLAS	CULA	uBLAS	CUV	Eigen	Armadillo
Platforms:
- CPU	✓		✓			✓	✓	✓	✓
- GPU	✓	✓	✓	✓	✓		✓	✓	partial
- OpenCL	✓		✓
- Xeon Phi	✓		✓
- OpenMP	✓		✓						✓
Thrust compat	✓	✓
	ViennaCL	CUSP	MAGMA	cuBLAS	CULA	uBLAS	CUV	Eigen	Armadillo
Features:
- Column or row major	both	both		column	column		both	both	column
- Dense matrix	✓	✓			✓			✓	✓
- Sparse matrix	✓	✓			✓			✓	✓
- Slice/view	✓	✓				✓		✓	✓
- BLAS L1	✓	✓	✓	✓	✓	✓	✓	✓	✓
- BLAS L2	✓	✓	✓	✓	✓	✓		✓	✓
- BLAS L3	✓	✓	✓	✓	✓	✓		✓	✓
	ViennaCL	CUSP	MAGMA	cuBLAS	CULA	uBLAS	CUV	Eigen	Armadillo
Other:
- fancy operators	✓	-							✓
- need Boost?	partly					✓	✓
Notable users	Singa, <10			many				TensorFlow, Shogun, 70+	MLPACK, 30+
	ViennaCL	CUSP	MAGMA	cuBLAS	CULA	uBLAS	CUV	Eigen	Armadillo

Machine Learning Frameworks

Frameworks:

• Darknet: open source neural network framework written in C and CUDA. It is fast, easy to install, and supports CPU and GPU computation. Mainly geared towards CNNs but has some RNNs as well.
• MLPack: a scalable machine learning library, written in C++, that aims to provide fast, extensible implementations of cutting-edge machine learning algorithms.
• Shogun: open-source machine learning library that offers a wide range of efficient and unified machine learning methods.
• OpenNN: an open source class library written in C++ programming language which implements neural networks with deep architectures, a main area of machine learning research.
• DLib: contains a wide range of machine learning algorithms. All designed to be highly modular, quick to execute, and simple to use via a clean and modern C++ API. See paper (PDF).
• Caffe: a deep learning framework made with expression, speed, and modularity in mind. Caffe is geared towards CNN (Caffe stands for Convolutional Architecture for Fast Feature Embedding). See paper (PDF).
• Dynet: neural network library by Carnegie Mellon University, aimed to work well with networks that have dynamic structures that change for every training instance. Seems to be geared towards sequence model (RNN/LSTM). See paper (PDF).
• Shark: Shark is a fast, modular, general open-source C++ machine learning library. Contains many basic ML algorithms comparable to scikit-lean such as linear regression, SVM, neural networks, clustering, etc. See paper (PDF).
• Fido: Fido is a light-weight, open-source, and highly modular C++ machine learning library. The library is targeted towards embedded electronics and robotics.

Comparison:

	Darknet	MLPack	Shogun	OpenNN	DLib	Caffe	Dynet	Shark	Fido
License	copyleft	BSD like	BSD	LGPLv3	Boost	BSD	Apache	LGPL	MIT
Created	2013	2011	1999	2012	2006	2013	2015	2008	2015?
Active	~	✓	✓	✓	✓	✓	✓	✓	~
Platforms:
- GPU	✓	-	?	✓	✓	✓	✓
- OpenMP	✓	-	?	✓		✓
- OpenCL						~
- Windows			✓	✓	✓	~	✓	✓
Features:
- Supervised	-	✓	✓	✓	✓	✓	✓	✓	~
- Unsupervised	-		✓	-	✓			✓
- RL	-	✓	-	-	~	✓			✓
- CNN	✓				✓	✓
- RNN	✓						✓
Matrix lib	own	Armadillo		Eigen	own	own	Eigen	uBLAS	-/STL
Notable users					10+	1000+	10+

Others:

• frugally-deep: header-only library for using Keras models in C++, supporting CNN. Currently only runs on CPU.

Reinforcement Learning

• RLlib - C++ library for reinforcement learning. See the paper "A C++ Template-Based Reinforcement Learning Library: Fitting the Code to the Mathematics", Hervé Frezza-Buet, Matthieu Geist.
• RLLib (Saminda's) - a lightweight C++ template library that implements incremental, standard, and gradient temporal-difference learning algorithms in Reinforcement Learning. See the paper (PDF).

Other Lists

• Awesome Machine Learning (C++ Section)
• http://mloss.org/software/ - machine learning OSS.
• Comparison of deep learning software - Wikipedia

Other Stuff

Machine Learning for Trading

Software:

• Q Learning for Trading: an adaptive learning model to trade a single stock under the reinforcement learning framework.

Papers:

• Optimal Asset Allocation using Adaptive Dynamic Programming. Ralph Neuneier. 1996
• Artificial Markets and Intelligent Agents. N.T. Chan. 2001
• Learning to Trade via Direct Reinforcement. John Moody and Matthew Saffell, IEEE Transactions on Neural Networks, Vol 12, No 4, July 2001
• Reinforcement learning for trading systems and portfolios. John Moody , Matthew Saffell. 1998
• Algorithm Trading using Q-Learning and Recurrent Reinforcement Learning. Du, Xin, Jinjian Zhai, and Koupin Lv. 2009
• The price impact of order book events. Journal of financial econometrics. R. Cont, k. Arseniy, and S. Sasha. 2014
• Agent Inspired Trading Using Recurrent Reinforcement Learning and LSTM Neural Networks. David W. Lu. 2017.