Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SameDiff: remove outputs() variable inference in favor of user specified outputs #8265

Closed
AlexDBlack opened this issue Oct 4, 2019 · 0 comments · Fixed by KonduitAI/deeplearning4j#12
Assignees
Labels

Comments

@AlexDBlack
Copy link
Contributor

@AlexDBlack AlexDBlack commented Oct 4, 2019

The SameDiff.outputs() method is a "best guess" inference of the predictions of the SameDiff instance.

https://github.com/eclipse/deeplearning4j/blob/master/nd4j/nd4j-backends/nd4j-api-parent/nd4j-api/src/main/java/org/nd4j/autodiff/samediff/SameDiff.java#L1824-L1831

This inference is unfortunately unreliable: the "best guess" can fail (and confuse users), for example when a loss function is added.
Even if we filter loss variables, we still can't infer automatically in the case of multi-op loss functions (like softmax -> difference -> squared difference -> (loss = sum(squared difference)).

Instead:

  • User should call setOutputs(String... ) to set the outputs explicitly
  • outputs() should throw a useful exception if outputs have not been set
  • We should have no-arg feed forward methods that use these stored output variable names (or again, exception if not set)

And of course, outputs should be serialized with the graph.

@AlexDBlack AlexDBlack self-assigned this Oct 24, 2019
AlexDBlack added a commit to KonduitAI/deeplearning4j that referenced this issue Oct 24, 2019
…nreliable 'best guess'

Signed-off-by: AlexDBlack <blacka101@gmail.com>
AlexDBlack added a commit to KonduitAI/deeplearning4j that referenced this issue Oct 25, 2019
…nreliable 'best guess'

Signed-off-by: AlexDBlack <blacka101@gmail.com>
AlexDBlack added a commit to KonduitAI/deeplearning4j that referenced this issue Oct 26, 2019
* eclipse#8160 Remove resolvePrepertiesFromSameDiffBeforeExecution

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* SameDiff API cleanup

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More SameDiff cleanup

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8248 Switch SameDiff variable init from lazy to creation time for more predictable behaviour

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8252 TanhDerivative javadoc

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8225 Deconvolution2D input validation

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8265 Switch SameDiff.outputs() to user settable, instead of unreliable 'best guess'

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8224 SameDiff.zero and .one create constants, not variables

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* More cleanup and fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small test fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* DL4J SameDiff fixes

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Re-add hack for Deconvolution2DLayer until eclipse#8315 is resolved

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* eclipse#8270 Move CUDA device/version logging to Java; can be disabled via existing org.nd4j.log.initialization system property

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* All ND4J init logging checks system property

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Small tweak

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Remove redundant device logging

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* One more fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* UX improvements

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Deconv fix

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Add deconv tests

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Cleanup

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Remove debug code

Signed-off-by: AlexDBlack <blacka101@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
1 participant
You can’t perform that action at this time.