v0.9.1 - Add Noise Injection Layers and More Normalization Layers

Release Note for RustyML v0.9.1

Significant Update and Explanation

Added: Noise Injection Layers

GaussianDropout and GaussianNoise are added to neural network:

• GaussianDropout multiplies every input by Gaussian noise with mean 1 during training, providing smooth multiplicative regularization instead of hard zeroing.
• GaussianNoise adds zero-mean Gaussian noise to the input during training to improve model robustness.

Added: More Normalization Layers

InstanceNormalization and Group Normalization are added to neural network:

• InstanceNormalization normalizes each sample independently per channel using its own spatial mean and variance, reducing style/contrast variations without relying on batch statistics.
• GroupNormalization normalizes activations within groups of channels for each sample, making training stable even with small batch sizes.

Clarification

In the previous update, RustyML promised to adopt the new reference type when the ndarray was updated to v0.17. However, due to the generic constraint conflicts and incompatibility with parallel computing in RustyML's new reference type, RustyML will not switch to this type.

Update

Added

• Add Instance Normalization layer
• Add Group Normalization layer
• Add Gaussian Noise layer
• Add Gaussian Dropout layer

Changed

• Refactor AdaGrad parameter update logic in convolutional layers by introducing a reusable macro to reduce duplication and improve maintainability
• Refactor forward pass in train_batch and forward to prevent unnecessary cloning.
• Refactor weight serialization with dedicated macros for activation-based and simple layers.