Cassette Compatibility

[jrevels]

A few core developers have some exciting packages in the works for the Julia 1.x timeframe that rely on Cassette to implement native language AD, GPU transpilation, auto-parallelization, developer tooling, and PL research.

At the moment, however, Cassette's usability is blocked on certain Julia compiler improvements/features. Since the new language-level capabilities Cassette provides are planned dependencies in many areas of the Julia ecosystem, it was decided during the biweekly ML developer meeting that we should prioritize Cassette compatibility for Julia's 1.1 release.

Here are the key items that need to be addressed in Julia itself for Cassette compatibility:

• world age validation for certain generated functions JuliaLabs/Cassette.jl#6: world age validation for certain generated functions the main problems with this were essentially solved by fix type intersection for invalidation during method insertion to handle typevars properly #27351, though having computable-by-reflection world age bounds in CodeInfo might still be a good idea eventually
• generated CodeInfo inlining should happen in Base rather than in Cassette JuliaLabs/Cassette.jl#7, attempt to generalize Overdub{Intercept} to accept varargs JuliaLabs/Cassette.jl#14: generated CodeInfo inlining should happen in Base rather than in Cassette EDIT: This is probably as good as it's going to get without actually versioning a compiler API separately from Base.
• add mechanism for spoofing inference work-limiting heuristics #24852: support for spoofing inference's work-limiting heuristics (basically done, but blocked on review from the compiler team)
• optimization of "pass-through" varargs methods JuliaLabs/Cassette.jl#5, improve codegen of varargs functions #5402: splatting/varargs optimizations EDIT: turns out enable/improve constant propagation through varargs methods  #26826 fixes the problems we previously thought needed to be fixed in codegen
• isa_tfunc not working around broken subtyping in the way I hoped it would #27078: dispatching on kind types is broken/Base does this for some pretty core functions (see also handle unions of tuples better #22051, Dispatching on T where Core.Compiler.iskindtype(T) is broken #11840)

I'll update these issues as new things crop up. I can also move issues from Cassette's tracker to Base's tracker if that's helpful.

Some of these items (especially performance-related issues) might require significant follow-on work. For example, #24852 currently does provide a mechanism for inference spoofing, but does not update all of inference to leverage that mechanism.

Luckily, since much (if not all) of this work does not incur API breakage, it can be worked on and merged basically at any time (including during the 1.0-dev period).

While the official milestone here is Julia 1.1, that's more of a late deadline; it would be highly beneficial for us to finish as much as possible by 1.0 for two reasons:

1. There's a lot of Cassette-based work we'd like to evangelize at various conferences throughout 2018, and we'd hate for Julia to get scooped by similar work implemented in "lesser" languages. As brought up by @maleadt, the earliest of these is GTC 2018 in late March. We don't want Julia to be late to the game for these opportunities.

2. JuliaCon has always been a huge catalyst for development - it's well known that a week at JuliaCon is equal to months of non-JuliaCon development 😉 Thus, we'd like to have stable releases of Cassette and downstream tools (like Capstan) to introduce to the wider community at that time.

[jrevels]

I've added #22051 and #11840 to the checklist here, though we can probably get by with just fixing #22051 (according to @vtjnash, fixing #11840 would likely require a nontrivial subtyping redesign).

#26826 fixes most of the performance/inference issues Cassette that hits, but the improvements to inference cause #11840 to expose itself heavily in Cassette-ified code. Most practical offending cases, however, seem to come from #22051, which can and should be fixed by correcting the currently incorrect implementations of various tuple-handling functions. This should be relatively doable, and further improve Base anyway.

EDIT: The MWE that pretty much needs to be fixed for Cassette to pass tests is #27078

[jrevels]

Okay, everything on the TODO list here is marked resolved. It's looking like we might actually have a v0.7 that's Cassette-compatible (fingers crossed).

Unfortunately, one of the TODO items that was resolved regressed when the new optimizer was merged. This is probably my fault - I likely ported the relevant optimization to the new optimizer incorrectly in #26981. For more info, see JuliaLabs/Cassette.jl#45. As a side note, fixing this optimization should help performance of varargs functions all over the place, not just Cassette - it should, for example, cause some broken sparse array broadcast tests to start passing. I'll try to tackle this last regression in the upcoming week.

Once we get that issue resolved, we should be able to mark this issue as resolved.

[jrevels]

With #28079 merged, v0.7 should now serve as a solid foundation for an initial (but still experimental) release of Cassette.

While many hurdles were overcome to get here, we still have a long way to go for optimal and bug-free Cassette usage.

For example, while most overdubbed code now seems to infer well and doesn't suffer allocation penalties, Cassette-ified code still seems to run 3x-4x slower even if no "extra work" is being done. This likely points to a need for further tweaks to codegen/optimization for better handling of varargs methods.

Another important issue is JuliaLabs/Cassette.jl#6.

Finally, we have to figure out how to version Julia + Cassette in the long term, since Cassette's implementation depends on compiler details that are traditionally allowed to change between patch versions of Julia. Thus, each Cassette release would technically need to be tied exactly to a single patch version of Julia. Of course, this cumbersome versioning restriction would then infect packages that depend on Cassette, as well. There are several possible solutions to this problem, so I'm not too worried, but picking the right one will definitely require some deliberation.

Anyway, thanks to the compiler team at Julia Computing for all the work that has gone into this!

[ViralBShah]

It seems that we will need to start thinking about stability for the compiler API. Agree that picking a good workable solution for now is the right direction.