Combined function/jacobian interface

[pkienzle]

With chain rule one often sees many common sub-expressions in the calculation of the function and its derivatives. It would save time to compute the two of these together rather than having one call to the function and one call to the jacobian.

[hahnec]

I see your point to gain computational performance here. However, this would require the user to provide only part of the derivative function? Such an interface would be quite custom/unconventional or at least not straightforward... Feel free to draft this as a separate and more performant function. Again, I am happy to accept PRs.

[pkienzle]

Figuring out a nice interface will take some effort.

In particular I would like it to be able to use the autodifferentiation tools in torch if the function supports them. I also have functions implemented in pure cuda which I'm planning to tie in through cupy. These will need either numerical differentiation or hard-coded analytic forms, so I don't want to require AD.