Caspian v1.1.0
Adds new layers, new losses, and many new activations:
- Attention mechanisms (standard and multi-headed)
- N-Dimensional Convolution/Pooling/Upsampling layers
- Cosine Similarity layer, with potential for further similar distance tools
- Further Normalization Layers (Group, Instance, RMS)
- Expanded testing suite and many, MANY bug fixes
- Introduction of Parameterizable Activations
- GLU & similar activation functions
- Improvements to prior activations, increasing accuracy of the functions and learning methods
- Hinge, KL Divergence, and Huber losses now included