Make work on IRTools

[oxinabox]

This swaps out Cassette for IRTools @dynamo.

Which was an attempt to solve #35 which it does.
Time of first call for SVD is down to about 1/4 what it was;

Original, Cassette

3.128563 seconds (11.08 M allocations: 577.564 MiB
0.005609 seconds (2.19 k allocations: 152.170 KiB)

No Overdubs (Best Case Senario)

0.794261 seconds (2.49 M allocations: 120.038 MiB, 7.86% gc time)
0.000015 seconds (23 allocations: 3.641 KiB)

IRTools, with precompile.jl (This branch)

0.881375 seconds (2.98 M allocations: 146.369 MiB, 6.39% gc time)
0.000031 seconds (33 allocations: 4.188 KiB)

Note however I added the precompile.jl which makes sure that IRTools itself has been compiled before anything else. Without that it is much worse than anything else.

Next I will try that for Cassette.

Commented out is a sketch of what it should looklike without usiing IRTools.IR
which is close to what it should looklike without IRTools at all.
Doesn't quiet work.
Which might be nice since no IRTools dep, no need to precompile IRTools.
This is a really simple transform.

[oxinabox]

Nope, Cassette does not do so well when I add the precompile

Cassette with precompile.jl

2.838656 seconds (11.13 M allocations: 580.525 MiB, 7.72% gc time)
0.004810 seconds (2.19 k allocations: 152.170 KiB)

[ararslan]

According to @jrevels, the use of Cassette in this package is purely an implementation detail and could be replaced with regular ol' multiple dispatch. The reason why Jarrett didn't do that to begin with is it requires defining a fair number of methods to resolve ambiguities between the various differential types, rule types, etc. I think going that route will be a cleaner solution overall than replacing the dependency on Cassette with on one IRTools.

[oxinabox]

We might want to do this first, which makes things faster.
Since it will take/is a whole ton less code than #38 will be

[jrevels]

Do we know what the reason for the overhead here is and why switching away from Cassette fixes it

[oxinabox]

I do not know.

[jrevels]

Would probably be good to debug the actual perf issue before churning through a lot of code. Would also be interested if performing the transform via a Cassette pass vs. using dispatch improved things.

[oxinabox]

Debugging Cassette performance issues is really really annoying.
Even more so when it is related to the compile cache, since that means restarting Julia between runs.
To which ever poor sap tries to do it. (probably future me)

StatsProfile.jl and Cthulu.jl are best tools

[willtebbutt]

Should we anticipate that a custom Cassette @pass that performs the same transformation as @dynamo would result in the same performance gains? i.e. is doing contextual dispatch (and leaving it up to the compiler to figure out that overdub does nothing both of the time), rather than manual IR re-writing, the thing that's degrading performance here?

[oxinabox]

resolved by JuliaDiff/ChainRulesCore.jl#35