Add CosineSimilarity

[SkafteNicki]

🚀 Feature

Add Cosine similarity metric
https://en.wikipedia.org/wiki/Cosine_similarity
Measures the similarity between two feature vectors by calculating the angle between them.

Motivation

Pitch

Alternatives

Additional context

[avinashsai]

Hi,
https://pytorch.org/docs/stable/generated/torch.nn.CosineSimilarity.html. There is already cosine similarity in torch. Can I use it?

[SkafteNicki]

@avinashsai depends on your purpose.
If you want the calculate the mean score on a single batch, no problem.
But lets say that you want to accumulate over a complete dataset, then you need some kind of custom accumulation:

preds = [torch.randn(5, 2), torch.randn(10, 2)]
target = [torch.randn(5, 2), torch.randn(10, 2)]

individual_scores = []
for p,t in zip(preds, target):
    individual_scores.append(torch.nn.CosineSimilarity()(p, t).mean())
# total score and correct score is different
total_score = torch.mean(torch.stack(individual_scores))
correct_score = torch.nn.CosineSimilarity()(torch.cat(preds, dim=0), torch.cat(target, dim=0)).mean()

the global mean does not equal the mean of the score on the individual batches.
This is basically the reason why we also have custom implementation of mean squared error, mean absolute error ect even though these are also a part of pytorch, because when you want to accumulate over multiple batches you need to be careful about the order of operations.

[avinashsai]

@SkafteNicki Thanks for the clarification. So, this cosine similarity metric will compute between 2 n-dim tensors??

[SkafteNicki]

It should still be two [N, d] tensors meaning N tensors each with dimension d.
First the function should calculate the cosine similarity between the two tensors and then it should have a reduction argument that determines if we should take the sum or the mean over the batch dimension

[N-georgakopoulos]

Hey, I would like to work on this, could you assign it to me?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.