Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DL4J: Output Layers - validate incompatible loss/activation function combinations #6280

Closed
AlexDBlack opened this issue Aug 27, 2018 · 3 comments · Fixed by #6302

Comments

@AlexDBlack
Copy link
Contributor

commented Aug 27, 2018

Common mistakes we should detect + warn users about:

  • Softmax + nOut=1
  • Softmax + XENT
  • Sigmoid + MCXENT
  • Non 0-1 activation fns (identity/tanh/relu etc) + XENT or MCXENT (or others that expect probabilities)

A warning would be a good start - an exception (that can be disabled via outputValidation(false) or something) might be even better - lower risk of users missing the warning.

@Charele

This comment has been minimized.

Copy link

commented Aug 27, 2018

Why dont' throw exception directly?

@AlexDBlack

This comment has been minimized.

Copy link
Contributor Author

commented Aug 27, 2018

@Charele

Why dont' throw exception directly?

Why no exception at present? Because we just haven't implemented that.
Why no exception in the future? I'm proposing to do exactly that, but allow users to disable the exception manually. There's often edge cases we can't anticipate.

@AlexDBlack AlexDBlack self-assigned this Aug 29, 2018

AlexDBlack added a commit that referenced this issue Aug 29, 2018
AlexDBlack added a commit that referenced this issue Aug 30, 2018
DL4J: Misc fixes (#6302)
* Another pass on javadoc link formatting

* #6299 Mnist iterator subset shuffling repeatability

* #6128 fix StackVertex output type

* #6101 DataVec ObjectDetectionRecordReader image center validation

* #6280 validate and throw exception for invalid loss/activation combinations

* Cleanup and fix tests given new validation

* Another round of javadoc link fixes

* Re-enable some now passing tests

* Tweak arbiter max candidates condition to exclude queued candidates

* Small final test fix
sshepel added a commit that referenced this issue Aug 30, 2018
DL4J: Misc fixes (#6302)
* Another pass on javadoc link formatting

* #6299 Mnist iterator subset shuffling repeatability

* #6128 fix StackVertex output type

* #6101 DataVec ObjectDetectionRecordReader image center validation

* #6280 validate and throw exception for invalid loss/activation combinations

* Cleanup and fix tests given new validation

* Another round of javadoc link fixes

* Re-enable some now passing tests

* Tweak arbiter max candidates condition to exclude queued candidates

* Small final test fix
@lock

This comment has been minimized.

Copy link

commented Sep 29, 2018

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked and limited conversation to collaborators Sep 29, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
2 participants
You can’t perform that action at this time.