v0.8.0

AbstractNeuralNetworks v0.8.0

Diff since v0.7.2

Breaking, on two counts: a rename and a narrowing.

ArrayNamedTuple is NamedTupleOfArrays

The name now says what the alias is about, which here is a network's inputs and outputs. GeometricOptimizers had an ArrayNamedTuple of parameters, and the two shared a name by coincidence rather than by meaning. That one is gone as of GeometricOptimizers 0.7.0, so this is the only alias of the shape left in the ecosystem — which is what it should have been from the start.

ArrayTuple is TupleOfArrays. ArrayOrNamedTuple keeps its name.

A whole set of parameters is a NetworkParameters

NeuralNetworkParameters 0.3.0 removes ParameterSet, the Union{NetworkParameters, NamedTuple} these signatures were written on: a method on it was a method on Base.NamedTuple, and it named two questions at once — a whole set, and a branch of one.

applychain, _compute_loss, _statify, update!(::Model, …) and changebackend take a NetworkParameters. A caller holding a bare NamedTuple writes NetworkParameters(ps), which shares the leaf arrays rather than copying them. initialparameters already returns one, so a network built here is unaffected.

Two places keep a NamedTuple signature, and neither is for breadth

• applychain(layers, x, ps::NamedTuple), because a reverse pass calls it that way. NeuralNetworkParameters' ZygoteRules.pullback for a container seeds the pass with the wrapped NamedTuple, since that is what yields a tangent keyed by the layers rather than a tangent for the wrapper's one field. test/custom_pullback_test.jl fails without this method.
• changebackend(backend, ::NamedTuple), because moving a single layer between backends is something a caller legitimately asks for, and a layer is a branch rather than a set.

Both are separate methods rather than one signature over a union: they answer different questions that happen to share a body.

The loss functors take ps untyped

_compute_loss and the NetworkLoss/FeedForwardLoss functors drop the annotation instead of naming both shapes, and the difference from applychain is deliberate. applychain's two methods normalise — they name the two shapes values is defined on before handing a Tuple to the @generated method that does the work. The loss functors forward ps untouched to model(input, ps) and read nothing of it.

What that costs: a wrong ps now fails inside model(input, ps) rather than at the call.

Requires

NeuralNetworkParameters 0.3.0.

Merged pull requests:

• Rename ArrayNamedTuple, and take NetworkParameters for a set of parameters (#38) (@michakraus)