focal loss modulator

[bei-startdt]

how to infer the modulator

the code in your repo

modulator = tf.exp(-gamma * labels * logits - gamma * tf.log1p(
          tf.exp(-1.0 * logits)))

for focal loss in tensorflow/models/blob/master/research/object_detection
the focal loss form is the same as what is shown in paper

    prediction_probabilities = tf.sigmoid(prediction_tensor)
    p_t = ((target_tensor * prediction_probabilities) +
           ((1 - target_tensor) * (1 - prediction_probabilities)))
    modulating_factor = 1.0
    if self._gamma:
      modulating_factor = tf.pow(1.0 - p_t, self._gamma)

Could you please tell me how to transform the paper form to your form?

Thank you very much!

[richardaecn]

Hi @bei-startdt

Thanks for pointing this out! The implementation you mentioned is not very numerically stable (same for the implementation in https://github.com/tensorflow/tpu/blob/master/models/official/retinanet/retinanet_model.py#L130-L162). When gamma is small (< 1), there might be NaN occurs during back-propagation.

The full derivation can be found in the figure below. Hope this will help!

[bei-startdt]

Thanks a lot!

[Angzz]

@richardaecn Hi,have you experiment on detection datasets such as coco, and the results?

[richardaecn]

Hi @Angzz , we haven't tried it on detection datasets.

[XCRobert]

@richardaecn Hi , have you compared the class balanced focal loss with the orignal focal loss using resnet 50 or 101 ? When did such comparsion , you used resnet 32 in your paper. Will stronger networks weaken the framework you proposed ?

[shawnthu]

modulator = tf.exp(-gamma * labels * logits - gamma * tf.log1p( tf.exp(-1.0 * logits)))
should be
modulator = tf.exp(-gamma * labels * logits - gamma * tf.log1p( tf.exp(-1.0 * labels * logits)))
labels in {-1, 1}

[richardaecn]

Hi @shawnthu, in the formulation, we are using 1 for positive labels and 0 for negative labels.

[shawnthu]

Hi @shawnthu, in the formulation, we are using 1 for positive labels and 0 for negative labels.

in fact we are both right, but your solution more concise (^o^)/~