Pull request/9de97af9 #38
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…t what this is meant to be and update a couple lines)
|
Q is a projector that takes features from intermediate time steps and forwards them to the classifier. It can be any linear operator. It might well be that Jmv is not being used in the CIFAR example. First, note that Jmv is the Jacobian and not its transpose (used in backprop).In first order methods like SGD you wouldn't use Jacobians much. However, they'll be used in second-order methods and also to test derivatives. |
klensink
approved these changes
Feb 3, 2018
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@eldadHaber @lruthotto What is Q? It seems like its a projection matrix of some kind..? It can be either an operator (Identity) or a 2D Array?
Also - Jmv doesn't seem to get used anywhere in the CIFAR10 example...?