Combine two kinds of loss function

[wang5566]

I want to train my model using soft max
Loss and triplet loss together. How can I combine two loss in a net together? Any reply will be appreciated.

[Rizhiy]

You need to write a new loss, similar to the one reid/loss/triplet.py and use that instead. Usually, you can just add losses together, just make sure to scale them since softmax can be much larger than triplet.

You would probably need to write your own data loader since softmax and triplet use different sampling techniques.

[Cysu]

@Rizhiy Thank you very much for the answer!

[wang5566]

@Rizhiy Thank you for your reply. But how to scale this two loss? In my network, I got softmax loss 1.5 and triplet loss 1.27, seeming unusual.

[Rizhiy]

To be honest I have never tried to train a network with multi-loss. Perhaps train softmax and triplet loss separately first, then calculate the scale such that they become the same value.

[zydou]

You can add a parameter to control the balance between them.
For example:

loss = loss1 + lambda * loss2 

[Cysu]

@Rizhiy @zydou Thanks a lot for your answers!