v1.9.0

Neural Network Models and Architectures

• Layers
  • Convolution layers now all use “same” padding instead of “valid”. However, this change greatly increases the computation time of CNNs.

Data

• Update the best neural networks on Datasets.

v1.8.3

Data

• Update the best neural networks on Datasets.

Bug Fixes

• Fix a bug where the error in convolution layers during training was miscalculated.

v1.8.2

Neural Network Models and Architectures

• Add a summary method.
• Allow to use NaN when a desired value is unknown.
• Allow to use a fixed seed.
• Layers
  • Allow to set the bias on convolutions.

Data

• Update the best neural networks on Datasets.

Tests

• Improve manual tests

Code Improvements and Optimizations

• Add a Clang-Tidy configuration
• Reduce the number of calculations during convolution layer training.

Bug Fixes

• Fix a bug where the error in convolution layers during training was miscalculated.
• Fix a bug on sequential dataset.

v1.8.1

Code Improvement and Optimization

• Clean some code.
• Use CodeFactor to note code quality.
• Remove GTest from sources and import it with CMake.

Data

• The normalization is no longer done automatically.

Tests

• Improve test stability.
• Fix CI compilation with GCC 13.
• Datasets
  • Improve the best CIFAR-10 neural network.
  • Improve the best Audio Cats and Dogs neural network.

v1.8.0

Neural Network Models and Architectures

• Layers
  • Rework all filter layers (Convolution, LocallyConnected, MaxPooling)
  • Change the shape of the Input layer to (C, X, Y) instead of (X, Y, C)
• Neurons
  • Fix the learning for the neurons in the convolutional layers
• Features
  • Update Boost to 1.80.0

Learning Algorithms and Optimizations

• Optimizers
  • Adapt SGD to new filter layers
  • Implement Softmax as LayerOptimizer

Tests

• Improve test stability
• Add tests on filter layers
• Datasets
  • Improve the best MNIST neural network
  • Improve the best Fashion-MNIST neural network
  • Improve the best CIFAR-10 neural network

v1.7.1

Neural Network Models and Architectures

• Layers
  • Improve 2D filter layers
• Neurons
  • Bias is no longer constant but always initialize to 1
• Features
  • Add function to display 2D filter layers as bitmap
  • Add function to display 2D input data as bitmap

Learning Algorithms and Optimizations

• Layer Optimizers
  • Add Error Multiplier

Data

• Normalize data between [0, 1] instead of [-1, 1]

Tests

• Improve test stability
• Datasets
  • Improve the best MNIST neural network
  • Improve the best Fashion-MNIST neural network
  • Improve the best CIFAR-10 neural network

v1.7.0

Neural Network Models and Architectures

• Layers
  • Fix Convolution backpropagation
  • Fix MaxPooling backpropagation
• Activation Functions
  • Fix ReLU
  • Add GELU

Learning Algorithms and Optimizations

• Learning
  • Fix MEA stop condition
  • Improve NaN detection
• Optimizations
  • Implement SIMD optimization with OpenMP

Tests

• Improve test stability
• Datasets
  • Improve the best MNIST neural network
  • Improve the best Fashion-MNIST neural network
  • Improve the best CIFAR-10 neural network

v1.6.0

Learning Algorithms and Optimizations

• Layer Optimizers
  • Add L1 Regularization
  • Add L2 Regularization
  • Fix bug on Dropout

Documentation

• Update documentation

v1.5.0

Neural Network Models and Architectures

• Layers
  • Add MaxPooling layer

Tests

• Dataset tests
  • Add a pre-trained neural networks for MNIST with highest accuracy
  • Add a pre-trained neural networks for Fashion-MNIST with highest accuracy
  • Add a pre-trained neural networks for CIFAR-10 with highest accuracy

v1.4.0

Learning Algorithms and Optimizations

• Add Batch
• Add synchronous training function
• Avoid evaluation twice in a row

Code Improvement and Optimization

• Manage NaN value during training
• Use C++ template for the neuron class

Documentation

• Update documentation