v0.9.1

AbstractNeuralNetworks v0.9.1

Diff since v0.9.0

Changed

• CI uploads coverage from the Julia 1 - ubuntu-latest job instead of Julia min, and a test
  job saves the Julia cache only when it succeeds.

• Pkg.test() no longer runs the doctests. test/quality/doctests.jl is now the group
  doctests, which an empty ARGS does not run; Pkg.test(test_args = ["doctests"]) runs it. In
  CI the Doctests job stays their runner, so the test matrix no longer runs them a second time.

• test/neural_network_constructors.jl moves to test/integration/neural_network_constructors.jl.
  The test convention keeps a test file at the top level of test/ only where it mirrors
  src/<name>.jl, and there is no src/neural_network_constructors.jl. The file tests the
  NeuralNetwork constructors of src/neural_network.jl, whose mirror test/neural_network.jl
  already exists, so it goes to test/integration/.

Dependencies and tests

• Admits NeuralNetworkParameters 0.5 alongside 0.4.2 (compat only; no change to this
  package's own behaviour). The downstream packages move to NNP 0.5, and they can resolve only if
  this package admits it. NNP 0.5 gives an untouched leaf a zero gradient instead of nothing,
  gives a leaf's gradient the leaf's element type, rejects a NetworkParameters with two numeric
  element types, and makes + add storage leaf by leaf. The test suite passes against both 0.4.2
  and 0.5.

Merged pull requests:

• CI: coverage from Julia 1 on ubuntu, save the cache only from a job that succeeds (#54) (@michakraus)
• Move the doctests into their own test group (#55) (@michakraus)
• Move the top-level test files that mirror no source file (#56) (@michakraus)
• Admit NeuralNetworkParameters 0.5 in compat (#57) (@michakraus)

v0.9.0

AbstractNeuralNetworks v0.9.0

Diff since v0.8.0

Methods that GeometricMachineLearning (GML) defined on this package's types move here, so GML no longer pirates them. The full notes are in the 0.9.0 section of CHANGELOG.md.

Breaking Changes

• A GML that still defines the 3-tensor methods on Dense and Linear fails to precompile against this release ("Method overwriting is not permitted during Module precompilation"). GML deletes them in the same change that raises its AbstractNeuralNetworks bound.
• The backend keyword of load(NeuralNetwork, …) is removed. The network loads on the CPU; changebackend(backend, nn) moves it to a device.

New Features

• Dense, Linear and Affine accept inputs of rank 3 or more, through reshape and one matrix product, on any array type.
• A weak extension HDF5Ext provides save and load for NeuralNetwork.

Bug Fixes

• A top-level Zygote.pullback(applychain, layers, x, ps) gives the storage gradient of each leaf and keeps the leaf's type; a parameter cotangent of nothing no longer raises a MethodError.
• parameterlength(::Dense) reads the bias flag as a Bool.

Dependencies

• Requires NeuralNetworkParameters 0.4.2 (and so GeometricBase 0.15) and Julia 1.11.

Merged pull requests:

• Take in Dense/Linear/Affine 3-tensor and NeuralNetwork HDF5 methods from GML (#40) (@michakraus)
• Require NeuralNetworkParameters 0.4 (#41) (@michakraus)
• Give the storage gradient from applychain's pullback (#42) (@michakraus)
• Move the test suite to the common layout (#44) (@michakraus)
• Update DocumenterCitations requirement to 1.5.0 in /docs (#46) (@dependabot[bot])
• Update Documenter requirement to 1.19.0 in /docs (#47) (@dependabot[bot])
• Update SafeTestsets requirement to 0.1.0 in /test (#49) (@dependabot[bot])
• Update Documenter requirement to 1.19.0 in /test (#50) (@dependabot[bot])
• Read the Dense bias flag as a Bool (#51) (@michakraus)
• Remove the shared compat entries from test/Project.toml (#52) (@michakraus)
• Raise the NeuralNetworkParameters floor to 0.4.2 (#53) (@michakraus)

v0.8.0

AbstractNeuralNetworks v0.8.0

Diff since v0.7.2

Breaking, on two counts: a rename and a narrowing.

ArrayNamedTuple is NamedTupleOfArrays

The name now says what the alias is about, which here is a network's inputs and outputs. GeometricOptimizers had an ArrayNamedTuple of parameters, and the two shared a name by coincidence rather than by meaning. That one is gone as of GeometricOptimizers 0.7.0, so this is the only alias of the shape left in the ecosystem — which is what it should have been from the start.

ArrayTuple is TupleOfArrays. ArrayOrNamedTuple keeps its name.

A whole set of parameters is a NetworkParameters

NeuralNetworkParameters 0.3.0 removes ParameterSet, the Union{NetworkParameters, NamedTuple} these signatures were written on: a method on it was a method on Base.NamedTuple, and it named two questions at once — a whole set, and a branch of one.

applychain, _compute_loss, _statify, update!(::Model, …) and changebackend take a NetworkParameters. A caller holding a bare NamedTuple writes NetworkParameters(ps), which shares the leaf arrays rather than copying them. initialparameters already returns one, so a network built here is unaffected.

Two places keep a NamedTuple signature, and neither is for breadth

• applychain(layers, x, ps::NamedTuple), because a reverse pass calls it that way. NeuralNetworkParameters' ZygoteRules.pullback for a container seeds the pass with the wrapped NamedTuple, since that is what yields a tangent keyed by the layers rather than a tangent for the wrapper's one field. test/custom_pullback_test.jl fails without this method.
• changebackend(backend, ::NamedTuple), because moving a single layer between backends is something a caller legitimately asks for, and a layer is a branch rather than a set.

Both are separate methods rather than one signature over a union: they answer different questions that happen to share a body.

The loss functors take ps untyped

_compute_loss and the NetworkLoss/FeedForwardLoss functors drop the annotation instead of naming both shapes, and the difference from applychain is deliberate. applychain's two methods normalise — they name the two shapes values is defined on before handing a Tuple to the @generated method that does the work. The loss functors forward ps untouched to model(input, ps) and read nothing of it.

What that costs: a wrong ps now fails inside model(input, ps) rather than at the call.

Requires

NeuralNetworkParameters 0.3.0.

Merged pull requests:

• Rename ArrayNamedTuple, and take NetworkParameters for a set of parameters (#38) (@michakraus)

v0.7.2

AbstractNeuralNetworks v0.7.2

Diff since v0.7.1

Merged pull requests:

• Name the parameter union after the package that owns it (#37) (@michakraus)

v0.7.1

AbstractNeuralNetworks v0.7.1

Diff since v0.7.0

v0.7.0

AbstractNeuralNetworks v0.7.0

Diff since v0.6.4

Breaking changes

The parameter container moved to its own package (#33)

NeuralNetworkParameters is gone from this package. The container now lives in NeuralNetworkParameters.jl as NetworkParameters, along with the tree walks and the HDF5 path that used to be duplicated here. No alias is left behind, so that one type has one name across the ecosystem.

# before
using AbstractNeuralNetworks: NeuralNetworkParameters
# after
using NeuralNetworkParameters: NetworkParameters

NeuralNetwork now stores PT <: NetworkParameters. A model whose initialparameters returns a plain NamedTuple — a bare layer, for instance — cannot be stored in one, and now says so with an ArgumentError naming the model and the reason, instead of a MethodError against the inner constructor.

The recurrent cells were removed (#34)

Recurrent, LSTM, GRU, IdentityCell and GridCell are gone, along with src/cells/. They had not compiled against this package's own initialparameters interface since 0.6 and no test exercised them; GridCell additionally returned a Matrix of NamedTuples, which the parameter change above rules out. There is no replacement — the only consumer was GeometricMachineLearning, which is dropping its RecurrentNeuralNetwork/LSTMNeuralNetwork in step.

QPT and QPTOAT were removed (#31, #35)

These fixed a NamedTuple's keys to (:q, :p) — Hamiltonian phase-space vocabulary in a package that should not know what a Hamiltonian is. They are replaced by key-agnostic aliases:

# before
using AbstractNeuralNetworks: QPT, QPTOAT
# after
using AbstractNeuralNetworks: ArrayNamedTuple, ArrayOrNamedTuple

ArrayNamedTuple{T,S} is a NamedTuple of AbstractArray{T}; ArrayOrNamedTuple{T} is that or a bare array. Both are strictly wider than what they replace, so nothing that resolved before stops resolving. Neither is exported, matching the names they replace.

Bug fixes

• Losses over a NamedTuple could not be differentiated. _norm/_diff had generic NamedTuple fallbacks that threw under Zygote — ChainRulesCore.ProjectTo has no method for the Vector the implementation collected — so in practice only the hardcoded (q, p) case worked. Every NamedTuple now differentiates, with values unchanged (#35).
• A Chain refused any NamedTuple that was not (q, p). applychain is now generic in its input; a chain applies whatever its layers accept (#35).
• Dense's initialparameters now accepts the keyword arguments Chain forwards to it, as Chain's docstring has always promised (#34).

Other

• AbstractLayer{M,N} now declares its type parameters in the order everything uses them (M = input, N = output). Names only; no behaviour change (#34).

Merged pull requests:

• Take the parameter container from NeuralNetworkParameters (#33) (@michakraus)
• Remove the recurrent cells (#34) (@michakraus)
• Remove QPT and QPTOAT (#35) (@michakraus)

Closed issues:

• Remove Cells (#21)
• Remove QPT and QPTOAT. (#31)

v0.6.4

AbstractNeuralNetworks v0.6.4

Diff since v0.6.3

v0.6.3

AbstractNeuralNetworks v0.6.3

Diff since v0.6.2

Merged pull requests:

• Move HDF5 support to a package extension (#32) (@benedict-96)

v0.6.2

AbstractNeuralNetworks v0.6.2

Diff since v0.6.1

Merged pull requests:

• Removed default loss computation for abstract type NetworkLoss (#28) (@benedict-96)

Closed issues:

• Default functor should not be implemented for NeuralNetworkLoss (#27)

v0.6.1

AbstractNeuralNetworks v0.6.1

Diff since v0.6.0

Merged pull requests:

• Added custom pullback for NeuralNetworkParameters. (#25) (@benedict-96)