Remove Rule (or maybe all AbstractRules) and treat functions as Rules

[oxinabox]

Lyndon White:ox: Yesterday at 10:18 PM

I am wondering if we will be able to get rid of all subtypes of AbstractRule.
Idk if the general overloads re indexing and getindex are not used and are kinda confusing.
And would be removed by #31 (comment)
1-arg rule is just a function now.
2-arg rule maybe can be done just by overloading accumulate? (Not sure onthis one,)
DNERule is just a function that always returns DNE
I think we could refactor things so the WirtingerRule goes away and we just have a function that returns a Wirtinger.
4 replies

Lyndon White:ox: 16 hours ago

We can remove them a few at a time. See what breaks

Simeon Schaub 2 hours ago

I kind of like having a WirtingerRule because that usually needs to be handled seperately since it doesn't implement all the arithmetic. Having a special type means this handling can be done at compile time, which I don't think can be done in a non-hacky way with just a function

Lyndon White:ox: 2 hours ago

Definitely we want the Wirtinger differential.
I am less sure about the WirtingerRule pullback (edited)

Simeon Schaub 1 hour ago

The case I'm thinking of is if you're building up a tape for reverse AD, where you would store only the rules in some kind of tree structure. But to materialize the results, it was important in my case to know, whether the storage type needs to handle Wirtinger derivatives

[oxinabox]

We're going to go forth and make general functions act like rules for now.
And maybe get rid of Rule.
One argument to keep it is that it makes it easier to overload show

[oxinabox]

One thing this will do nicely is make implementing the equiv of Zygotes '@adjoint` much simpler, as will not need to autowrap everything.

Also while it is harder to overload show it means methods(pullback) is easier to use.

[nickrobinson251]

Can close? @oxinabox