v0.9.0

GeometricMachineLearning v0.9.0

Diff since v0.8.0

The full notes, with a migration line for each change, are the 0.9.0 section of CHANGELOG.md.

Breaking Changes

• Training uses GeometricOptimizers 0.9: Optimizer, its callable methods and the aliases GradientOptimizer, MomentumOptimizer and AdamOptimizer are removed. Build TrainingOptimizer(nn.params; algorithm, linesearch, retraction) (re-exported) and call train!(nn, opt, dl, batch, n_epochs; loss). train!'s pullback keyword is gone; pass the pullback as loss.
• The Hamiltonian and Lagrangian neural networks move to HamiltonianNeuralNetworks.jl and LagrangianNeuralNetworks.jl. SymbolicNeuralNetworks and Symbolics leave [deps].
• DataLoader, Batch and train! are re-exported from ReducedComplexityModeling 0.5, with the data-loader routines. DataLoader(matrix) is a time series unless autoencoder = true.
• QPT and QPTOAT are removed; use AbstractNeuralNetworks' NamedTupleOfArrays and ArrayOrNamedTuple.
• PoissonTensor is removed. The trajectory methods are integrate, not iterate. update! and solve! are no longer exported. HRedSys requires the integrator keyword. NetworkLoss(::TransformerIntegrator, prediction_window) is gone.
• The 3-tensor functors of Dense and Linear and the HDF5 save/load methods come from AbstractNeuralNetworks 0.9.
• Julia 1.12 or later. [compat]: GeometricOptimizers 0.9.2, AbstractNeuralNetworks 0.9.2, NeuralNetworkParameters 0.5, GeometricBase 0.15, GeometricEquations 0.21.5, ReducedComplexityModeling 0.5.

New Features

• Multi-modal symplectic autoencoders, with a three-stage reproduction script and tutorial.
• flatten_system_parameters, concatenate_array_with_parameters, assign_q_and_p and check_forcing_type are public and documented.
• One matrix and tensor product, _mul, with faster forward and backward passes; VolumePreservingAttention uses one Cayley kernel at every sequence length and on device arrays.
• SymplecticAttentionQ/P accept a single sample; SymplecticAutoencoder builds with one block.

Merged pull requests:

• Run GPU tests on Metal on Apple silicon and in a macos-15 job (#307) (@michakraus)
• Bind every type parameter of the (q, p) NeuralNetworkIntegrator iterate method (#308) (@michakraus)
• Test the (q,) NeuralNetworkIntegrator iterate method (#309) (@michakraus)
• Move the Open Issues section of CHANGELOG.md into KNOWN_ISSUES.md (#312) (@michakraus)
• Update NeuralNetworkParameters requirement from 0.3 to 0.3, 0.4 (#313) (@dependabot[bot])
• Migrate the test suite to the flat SafeTestsets layout (#320) (@michakraus)
• Read show_progress as a Bool (#323) (@michakraus)
• Clear the fatou findings in src and test (#324) (@michakraus)
• Add test/quality/jet.jl for the kernel launchers (part J1) (#326) (@michakraus)
• Run the metal group by default on Apple silicon (part DV3) (#328) (@michakraus)
• Take the manual's TikZ figures and logo from GeometricFigures (part C1) (#331) (@michakraus)
• Raise the Julia floor to 1.12 (fixes #327) (#332) (@michakraus)
• Cut the compilation cost of the test suite (#334) (@michakraus)
• CI: coverage from Julia 1 on ubuntu, save the cache only from a job that succeeds (#339) (@michakraus)
• Move the doctests into their own test group (#340) (@michakraus)
• Move the top-level test files that mirror no source file (#341) (@michakraus)
• Train on GeometricOptimizers 0.9: NetworkOptimizer and train! (part M1a) (#342) (@michakraus)
• Losses and pullbacks: one default loss, one pullback shape, and the symbolic HNN gradient (B5) (#344) (@michakraus)
• Products through GEMM: one _mul, slicing assign_q_and_p, zero and other-precision seeds in the kernel rrules (part M3) (#345) (@michakraus)
• GML part M2: DataLoader samples against trajectories, one count and one gather (#346) (@michakraus)
• GML part M4: one row-pivoted Cayley kernel at every sequence length, with the closed forms for N = 2 to 5 kept (#347) (@michakraus)
• GML part M5: layers and architectures share one method per Q/P pair, Bool choice and triangle (#351) (@michakraus)
• Rename citation keys to the house scheme (#352) (@michakraus)
• Replace QPT and QPTOAT by AbstractNeuralNetworks' NamedTupleOfArrays and ArrayOrNamedTuple (#353) (@michakraus)
• Remove the Hamiltonian and Lagrangian networks (now in their own packages) (#354) (@michakraus)
• Save one decoder evaluation per call of the reduced vector fields (#355) (@michakraus)
• Correct the comments and docstrings that seven known issues name (#356) (@michakraus)
• Add the parametric core: apply_parametric, ParametricLoss, ParametricDataLoader (#357) (@michakraus)
• Part M6: small API corrections for GeometricMachineLearning 0.9 (#358) (@michakraus)
• Add WideResNetLayer, ParametricResNetLayer and ParametricResNet (#359) (@michakraus)
• Add the forcing layer and the forced G-SympNet (#360) (@michakraus)
• Part M7: device tests and both precisions for GeometricMachineLearning (#361) (@michakraus)
• Record the two Metal aborts and count them: K35 and the run driver (#362) (@michakraus)
• Add analytic symplectic layer inversion and WeightTiedSymplecticAutoencoder (#363) (@michakraus)
• Make the prose and the records describe the code: C6, C16, K4 and the inventory (#364) (@michakraus)
• Move the data loader to ReducedComplexityModeling and re-export its three names (#365) (@michakraus)
• Metal abort runs: count early deaths, record overlap before the timed call, record run 18 as a death (#366) (@michakraus)
• Add permutation-symmetric particle layers and SymmetricSymplecticAutoencoder (#367) (@michakraus)
• Add bilinear and low-rank velocity-parity Stiefel layers and trace loss (#368) (@michakraus)
• Make the slot frontend invariant and correct the particle layers' dimensions and claims (#369) (@michakraus)
• Add MultiModalSymplecticAutoencoder, multimodal losses, and HRedSys reference shift and VJP (#370) (@michakraus)
• Correct the multi-modal losses and the claims of the particle and parity frontends (#371) (@michakraus)
• Three-stage multi-modal symplectic autoencoder: reproduction script and tutorial (#372) (@michakraus)
• Bind ReducedComplexityModeling to 0.5 (#373) (@michakraus)
• Make the four names HamiltonianNeuralNetworks reaches public and documented (#374) (@michakraus)

Closed issues:

• Type piracy (#145)
• Naming of OptimizerMethod subtypes. (#149)
• Get rid of JLD2 in docs (#178)
• Extend HDF5 interface to custom arrays (like SymmetricMatrix). (#191)
• Import QPT and QPTOAT from AbstractNeuralNetworks (#195)
• solve! doesn't seem to work anymore for PSD (#197)
• Functions encoder and decoder not working anymore (#198)
• Possible type instability in definition of DataLoader (#211)
• save from AbstractNeuralNetworks seems to not be imported at the moment. (#218)
• Zygote gradient w.r.t. NetworkParameters loses the SymmetricMatrix leaf on a second access (#319)
• Julia min on macOS and Windows: stack overflow in type inference in the Lagrangian Neural Network tests (#327)
• SymbolicPullback(arch) gives the gradient of the per-sample sum, not of HNNLoss (#343)