[WIP] Testing compatibility of user-defined gradients approach

[andrjohns]

This PR is being used to test the downstream compatibility of an approach to user-defined gradients. Given that the approach is based on a lot of specialised tuple manipulation and general template trickery, I'd like to make sure that it's not fragile to any of the various compilation/testing layers in the pipeline.

I have replaced the gradient specifications for a unary function (exp), binary function (hypot), and a matrix/varmat function (diag_pre_multiply) to see whether compilation and values/gradients are stable throughout the pipeline

[andrjohns]

Looks like there's some compatibility issues with the older g++, closing to investigate

[wds15]

Cool beans! Is there some design doc around? Would this approach also allow for efficient Jacobian adjoint product things, which is more efficient in case we only want the gradient in reverse mode? Having analytic gradients is a great thing, don't get me wrong! If you want to performance sensitive test, then you can attempt to make this work with a non-stiff ODE problem and use the RK45 integrator. Supplying the gradient for the user defined ODE RHS should speed up things a lot if this approach is fast.

[andrjohns]

Thanks! Yep, the latest iteration added an implementation where the gradient function can be specified as an adjoint-jacobian product, either for just reverse-mode or for both reverse- and forward-mode.

A design-doc is still in the works, just want to make sure that all works downstream first!

[andrjohns]

Closing for now to clear PR backlog, will re-open once ready for review