add contrastive loss

[willduan]

add contrastive loss for siamese network
Computes the constrative loss between y_pred (logits) and y_true (labels).
(http://yann.lecun.com/exdb/publis/pdf/chopra-05.pdf)
Sumit Chopra, Raia Hadsell and Yann LeCun (2005).
Learning a Similarity Metric Discriminatively, with Application to Face Verification.

Arguments:
y_pred: Tensor. Predicted values.
y_true: Tensor. Targets (labels).
margin: . A self-set parameters that indicate the distance between the expected different identity features. Defaults 1.

[aymericdamien]

LGTM! Thanks for adding the contrastive loss :)

[polajnta]

This version of contrastive loss is a bit weird. The other definitions I saw basically compare two sets of outputs and whether they belong to the same class or not. This version is missing the label that indicates whether the outputs belong to the same class. So it cannot be used to do paired loss correctly.

[willduan]

@polajnta Hi, the comparision of two siamese nets' outputs this is the Euclidean distance need to add by yourself when you build the network, such as this:

 distance = tf.sqrt(tf.reduce_sum(tf.square(tf.subtract(out1, out2)), 1, keep_dims=True))
 distance = tf.div(distance,  tf.add(tf.sqrt(tf.reduce_sum(tf.square(out1), 1, keep_dims=True)),  tf.sqrt(tf.reduce_sum(tf.square(self.out2), 1, keep_dims=True))))
 distance = tf.reshape(distance, [-1], name="distance")

And the contrastive loss just compute the residual of y_true and the y_pred (that is distance).
So I guess you do not have a complete understanding of siamese network.

[polajnta]

right i was just expecting a self contained function like in:
https://github.com/tiagofrepereira2012/examples.tensorflow/blob/master/examples/tensorflow/script/train_mnist_siamese.py
L = 0.5 * (Y) * D^2 + 0.5 * (1-Y) * {max(0, margin - D)}^2