Implementation-agnostic linear algebra optimisations for Reverse-Mode AD
Switch branches/tags
Nothing to show
Clone or download

README.md

DiffLinearAlgebra

Note: the current version of this package is not intended for general consumption.

Build Status Windows Build status codecov.io DiffLinearAlgebra DiffLinearAlgebra

DiffLinearAlgebra can be (very loosely) thought of as DiffRules.jl for linear algebra. For every sensitivity, we provide a function which, when provided with the input and output from the forward pass and the reverse-mode sensitvity w.r.t the output from the forward pass, computes the sensitivity of the specified argument.

  A, B = randn(5, 3), randn(3, 4)
  C, C̄ = A * B, randn(5, 4)
  Ā = (*, Val{1}, (), C, C̄, A, B)
  B̄ = (*, Val{2}, (), C, C̄, A, B)

In the above example, the sensitivities of A and B are computed from C and a random seeding of . (Note that the third argument is currently redundant; see this issue for motivation for its inclusion.)

We also expose some "metadata" for each implemented sensitivity. This is done via a set called ops contains DiffOp structs. These structs contain information regarding the arguments types supported by each sensitivity, and which arguments are differentiable.