Add a straight-through gradient wrapper method. #70
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![copybara-service[bot]](https://avatars.githubusercontent.com/in/44061?s=60&v=4){kind=small}
copybara-service
bot
force-pushed
the
test_409150269
branch
7 times, most recently
from
471ec14
to
dab3796
Compare
copybara-service
bot
force-pushed
the
test_409150269
branch
2 times, most recently
from
f77df8d
to
25bc697
Compare
Categorical distributions wrapped this way will behave the same in the forward pass, but use a biased gradient computed on `self.probs` in the backward pass. PiperOrigin-RevId: 413413594
copybara-service
bot
force-pushed
the
test_409150269
branch
from
25bc697
to
6ebedf1
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add a straight-through gradient wrapper method.
Categorical distributions wrapped this way will behave the same in the forward
pass, but use a biased gradient computed on
self.probsin the backward pass.