/caffe

Sigmoid Cross-Entropy Loss: ignore selected targets by `ignore_label` #4986

Merged
merged 2 commits into from Nov 17, 2016

Conversation

Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
1 participant
@shelhamer
@shelhamer
Owner

shelhamer commented Nov 15, 2016 • edited

Sigmoid cross-entropy loss learns to ignore by zeroing out the loss/diff at targets equal to
the configured ignore_label.

  • ignore targets by ignore_label
  • properly normalize loss/diff when there are ignored targets. see #3296 for how this was handled for softmax loss.

Note that sig-ce loss and softmax loss have different defaults for normalization: batch size (sig-ce) and valid (softmax).

TODO: the common normalization and ignore logic should be pulled up into LossLayer, and layers should die loudly by default instead of silently ignoring these options.
@shelhamer shelhamer sigmoid cross-entropy loss: ignore selected targets by `ignore_label` 
sig-ce learns to ignore by zeroing out the loss/diff at targets equal to
the configured `ignore_label`.

n.b. as of now the loss/diff are not properly normalized when there are
ignored targets. sig-ce loss should adopt the same normalization options
as softmax loss.
c6ab965

shelhamer added the enhancement label Nov 15, 2016

@shelhamer shelhamer sigmoid cross-entropy loss: normalize loss by different schemes 
sig-ce loss handles all the same normalizations as the softmax loss;
refer to #3296 for more detail.

this preserves the default normalization for sig-ce loss: batch size.
3d62e3c

@shelhamer shelhamer merged commit 28c135c into BVLC:master Nov 17, 2016
1 check passed

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment