RustyML v0.17.0

Highlights

Composable layers and hierarchical parameter paths

Layers can now contain named sublayers through LayerBase::sublayers and sublayers_mut. Each sublayer has its own parameter address, gradients, optimizer buffers, state, and caches. Repeated calls to the same sublayer can use separate cache stacks through Ctx::sublayer_call.

The new LayerPath type identifies a layer within a model, including nested paths such as 2.forward. Optimizers, gradient clipping, state application, checkpoints, and weight enumeration traverse the complete layer tree. Sequential::summary and Graph::summary now include sublayer parameters in their totals.

Custom training code can use layer_path::walk, walk_mut, update_tree, and apply_state_tree to traverse and update a layer tree directly.

GELU, SiLU, and Mish

Three new activations are available as both Activation variants and standalone layers:

• GELU: exact by default, with an optional tanh approximation.
• SiLU: also known as Swish.
• Mish: x * tanh(softplus(x)).

Dense, all ten convolution layers, SimpleRNN, LSTM, and GRU support these activations.
For the standalone GELU layer, use GELU::new() for the exact form or .with_approximate(true) for the approximation.

Activation training now uses ActivationCache to retain the tensor needed by the backward pass. GELU, SiLU, and Mish retain their pre-activation input; existing activations retain their output. Each host layer caches only the required tensor.

Breaking changes and migration

Checkpoint format 4

Neural-network checkpoints from format 3 cannot be loaded by this release. The new format records the sublayer tree and uses hierarchical weight paths such as 2.forward.kernel. Weights belonging to layers without sublayers retain paths such as 2.kernel.

Rebuild the model, retrain it, and save a new checkpoint with RustyML 0.17.0.

Custom layers and optimizers

Update custom implementations for the following API changes:

• ParamId now contains { layer: LayerPath, name } and no longer implements Copy. Use ParamId::at(path, name) or path.param(name) for a sublayer parameter. ParamId::new(scope, name) still addresses a top-level layer.
• Grads::get now takes &ParamId, and Grads::iter yields (&ParamId, &Tensor).
• Optimizer::update now receives path: &LayerPath instead of scope: usize. Models invoke it for each layer and sublayer.
• Ctx::has_state and Ctx::state_slot now take &LayerPath. Use &LayerPath::root(scope) for a top-level layer.
• weight_path has moved from layers::checkpoint to layer_path and now takes &LayerPath instead of a model position.

Composite layers should expose their children through the sublayer methods instead of exposing child arrays as their own prefixed weights. Call children through Ctx::sublayer, or use Ctx::sublayer_call when a child is called repeatedly in one pass.

Activation backward passes

Activation::backward now takes &ActivationCache instead of an activated output tensor. Obtain the output and cache through Activation::forward_train(z), then pass the cache to the backward call. Inference continues to use Activation::forward(&z) without creating a cache.

Validation and fixes

• Layer-tree validation: model builds reject duplicate parameter or sublayer names, empty names, names containing ., inconsistent sublayer rosters, repeated layer nodes, and paths exceeding 64 sublayer levels. Errors identify the affected layer path.
• Sublayer call validation: training rejects calls whose target differs from the layer registered at that path, even when types and shapes match.
• Unconsumed state: training and direct mutable layer calls return Error::Computation when pending state is left unconsumed. This check also applies to release builds.
• Activation errors: invalid cache kinds return Error::InvalidInput, and gradient shape mismatches return Error::ShapeMismatch instead of causing a panic.
• Weight lookup: Sequential::weight and Graph::weight reject alternate spellings of layer positions, such as 00.kernel and +0.kernel.
• DBSCAN: predict returns Error::EmptyInput for an input matrix with zero rows.
• Single-sample tree predictions: DecisionTree::predict_one, DecisionTree::predict_proba_one, and IsolationForest::score_sample reject NaN and infinite values with Error::NonFinite.
• Normalization shapes: shape inference, model building, and forward execution now enforce consistent rank requirements. GroupNormalization accepts rank 2 and above; InstanceNormalization requires rank 3 and above.

Dependencies and documentation

• Updated thiserror from 2.0.20 to 2.0.21 and refreshed compatible transitive dependencies.
• Restricted indicatif to the show_progress feature, reducing dependencies for neural-network builds that do not display progress bars.
• Updated English and Chinese installation instructions, examples, and API reference links for 0.17.0.