Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Developed between 2002-2004, 2013-2018. Picture: GB-RBM reconstructed data (red) from examples (green). Variance of input data is approximated more or less correctly.
Machine learning component contains:
- unix commands line tools to do machine learning from scripts (nntool, dstool)
- multistart L-BFGS neural network 2nd order optimizer
- basic BayesianNeuralNetwork using hybrid monte carlo sampling
- principal component analysis (PCA) preprocessing method including symmetric eigenvalue decomposition code (eig)
- basic Hidden Markov Model (HMM) codes
- Stacked RBMs (GB-RBM & BB-RBM)
- [EXPERIMENTAL CODE]: basic reinforcement learning code (discrete/continuous actions, continuous states)
- [EXPERIMENTAL CODE]: parallel tempering sampling
- recurrent neural network code using BB-RBM output (RNN-RBM) and extended RTRL algorithm
- multi-threaded (multicore CPUs) code to support parallel search/optimization (std::threads and OpenMP)
- uses BLAS interface for matrix/vector math which is optimized for speed (OpenBLAS, Intel MKL)
- supports CSV ascii file import/export
- self-organizing map, quaternion, 3d matrix rotation and other codes
Additionally, the library contains basic linear algebra and various other general computer science algorithms like radix sorting (for floating point numbers through conversion code O(N)), hermite curve interpolation and cryptography (AES, DES..), arbitrary precision numbers C++ library etc. The compiler used is GCC https://gcc.gnu.org/ and the build process uses GNU Makefile and autoconf. The library also uses C++11 features meaning that only a new GCC C++ compiler is likely to work.
As of April 2017 there is ~100.000 lines of C++ code in the library.
Compiles both on Linux and on Windows (GCC).
NOTE: Please use experimental branch (RBM_test) if possible as the main branch is not actively maintained and I only plan to drop updates to it from experimental branch from time to time.
Releases are distributed using GNU GPL license. Links with BSD/LGPL libraries (zlib, gmp and blas).
Commercial license is also available. C++ implementation allows integration of machine learning into native C++ applications. Contact author(s) for details.
Author(s) (Tomas Ukkonen) keep full rights to use library in any way he or she wants in the future
including releases under different licenses.
Implement more modern algorithms (C++):
- add cuBLAS support for accelerated BLAS routines
- convolutional (sparse) neural networks
- LSTM RNNs
class LBFGS_nnetwork<> (neural network LBFGS optimizer)
class HMC<> (parallel hamiltonian monte carlo sampling for neural networks)
are the primary classes and tools to look at and use. Also read HowToLearnFromData page. Those interested in pretraining weights using RBMs, read docs/RBM_notes.tm and src/neuralnetwork/tst/test.cpp: rbm_test() and look at classes:
It is possible to convert layer-by-layer optimized DBN into feedforward supervised neural network.
Recurrent neural networks are activated using --recurrent DEPTH and --pseudolinear switches in nntool. For more information read:
which trains a primitive recurrent nnetwork using LBFGS. For examples, look at src/neuralnetwork/tst/test.cpp.
- cleanup the code and remove classes that are not needed and don't belong to this library
- improve threading code: fix known minor race conditions in parallel code
- try to remove few direct x86 dependencies (cpuid instruction)
- PROPER DOCUMENTATION (these wiki pages are also slightly out of date)
- small things: write bash autocomplete scripts for nntool and dstool
The software has been developed on Linux (x86 and amd64). It also compiles using MINGW-w64 64bit/Windows compiler (MSYS2 environment). It has been reported to work correctly in Cygwin 32bit environment (but cygwin license is a problem). Currently there are few x86 specific instructions (CPUID) and assumptions (size of int and float etc) so only x86 and amd64/x86_64 are fully supported.
Tomas Ukkonen firstname.lastname@example.org
GitHub is now (2015) the official location of this project.