What is nnForge?
nnForge is a framework for training convolutional and fully-connected neural networks. It includes CPU and GPU (CUDA) backends. The schema of the networks is DAG (directed acyclic graph).
The framework has a number of layers defined:
- Batch Normalization
- Sparse (in feature map dimension) convolutional
- Fully connected
- Local contrast subtractive
- RGB->YUV conversion
- Max subsampling
- Average subsampling
- Rectification - |x|
- Hyperbolic tangent
- Parametric ReLU
- Add (element-wise)
Error functions available:
- Negative Log-Likelihood
- Accuracy (top-n) (cannot be source of error for backward propagation)
The library implements mini-batch Stochastic Gradient Descent training algorithm with optional momentum (vanilla or Nesterov), ADAM is supported as well.
- Weight decay
- Dropout (as a separate layer)
- The framework can utilize multiple GPUs both for training and inference
- Training is parallelized with data parallel approach, where mini-batch is split across multiple GPUs
- Single node only
- NCCL is supported
nnForge is an open-source software distributed under the Apache License v2.0.
The package contains nnForge framework as well as examples - applications using the framework.
nnForge is designed and implemented by Maxim Milakov.