[WIP] Use ChainRules #291
Closed
[WIP] Use ChainRules #291
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
oxinabox
force-pushed
the
ox/chainrules_step1
branch
from
001587a
to
232f838
Compare
3 tasks
Merged
bors bot
added a commit
that referenced
this pull request
May 28, 2020
366: Add ChainRules r=oxinabox a=oxinabox This replaces #291 The bits from that OP that still matter Step 1) Change Zygote to check for chainrules before doing its normal stuff, and adapt the stuff it gets back from chainrules to play nice with Zygote's expectations Step 2) adapt Zygote more deeply, so it can take full advantage of thunks etc. This PR is Step 1. <s> TODO: workout why this seems to segfault for me. </s> Co-authored-by: Lyndon White <lyndon.white@invenialabs.co.uk> Co-authored-by: Mike J Innes <mike.j.innes@gmail.com> Co-authored-by: Lyndon White <oxinabox@ucc.asn.au>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
my plan is
Step 1) Change Zygote to check for chainrules before doing its normal stuff,
and adapt the stuff it gets back from chainrules to play nice with Zygotes expectations
Step 2) adapt ChainRules and Zygote, so their expectations are more inline with each other and integrate more deeply.
This likely includes adding a macro that is basically @adjoint to ChainRules.
I think
@adjoint function f, y, Δ->-yd isthe same as ChainRules’s:
rrule(::typeof(f),...) = (y, Rule(Δ->yd))And that that should really have a similar connivence macro
This PR is Step 1.
Major limitations blocking this:
ChainRules does not know how to differentiate WRT the function itself, which matters for closures.
I know this imprinciple works as
without this branch:
With this branch
The matching PR for Nabla is:
invenia/Nabla.jl#178