question on the equation 1 of origin paper

[littleredxh]

I just doubt the correctness of equation 1 in the origin paper. if f_i and f_j (feature before the normalization) is set to have the norm of 1, then the derivate in equation 1 should be f_j. But the right side of the equation looks like not to be f_j. Can you explain it?

[Dyfine]

Hi! In equation 1, we assume that \hat{f_i} = f_i / ||f_i|| and \hat{f_j} = f_j / ||f_j||, and these normalization operations are differentiable. What we want to obtain is the derivative of the inner product between two normalized embeddings (i.e. \hat{f_i} and \hat{f_j}) to one unnormalized embedding (i.e., f_i), thus we get the result in the paper. On the other hand, if we consider the inner product between two unnormalized embeddings (i.e. f_i and f_j), then f_j is correct.

[littleredxh]

Suppose we have ||f_i||=||f_j||=1, then \hat{f_i} = f_i and \hat{f_j} = f_j.
In equation 1, the left side should be partial{<\hat{f_i},\hat{f_j}>}/partial{f_i} = partial{<f_i,f_j>}/partial{f_i} = f_j; the right side should be 1/1*(f_j-cos_theta_ij f_i).

Problem: In this special case, the right part cannot equal the left part

[Dyfine]

In fact, the left side of equation 1 should be partial{< f_i/||f_i||, f_j/||f_j|| >} / partial{f_i}. When we have ||f_i||=||f_j||=1, <\hat{f_i},\hat{f_j}>=<f_i,f_j> is correct, however, we don't have partial{<\hat{f_i},\hat{f_j}>}/partial{f_i} = partial{<f_i,f_j>}/partial{f_i}, since we always use the normalization operation which should always be considered in the derivative calculation.