Diffhask integration

[stites]

@o1lo01ol1o I know you are already working on this with diffhask so here's the official ticket to say "hasktorch needs this, please keep doing you." If you need anything, feel free to ping people here.

Followup ticket will be "Implement Learning to learn by gradient descent by gradient descent".

[o1lo01ol1o]

👍

o1lo01ol1o/diffhask#2

[o1lo01ol1o]

I've resolved the current instance resolution issues and am filling out the numeric hierarchy for diffhask. While one can define arbitrary operations, adding concrete numeric instances for numhask and container and numeric instances for numhask-array will get you diffhask compatability for the corresponding linear algebra operations.

What's the sitch in the roadmap for numhask instances?

[stites]

Current status: totally forgotten. I've been getting baseline AD integration with backprop in place. Right now I'm trying to scope out what the NN library will look like with convolutions which, I believe, needs a bit of reshaping (unintended pun) for a better AD experience.

NumHask doesn't have an dependencies, as of >= 0.2.0.0, so hasktorch can have orphan instances directly added to the indefinites without too much overhead (I think). Tensors are already instances of Num, so I assume that about a quarter of the work is already done. Triaging which functions are pure and which are not, RE: #85, might pair well with the numhask instances -- opening an issue for that now.

[stites]

This is slowly getting booted back up and I'm adding @mitchellwrosen to this ticket. To start, you should check out Conal Elliot's The simple essence of automatic differentiation.

Added a followup and made this issue's title more specific.

[stites]

AD is now handled by libtorch.