Some questions about Supervised_NT_xent

[A-New-Page]

Excuse me, I recently paid attention to this paper.

when I use the Supervised_NT_xent loss, I find that there are some question maybe.

In SupCLR paper, when calculate the loss, the positive pair is (i, j), where label_i is the same as label_j, and the pair (i, i) is not regarded as positive pair, even though label_i must be the same as label_i.

However, when i use the Supervised_NT_xent loss from your code, and calculate Mask, I notice that Mask[i，i] is not zero. Therefore, the pair(i, i) will also be regarded as positive pair to calculate loss.

CSI/training/contrastive_loss.py

Lines 72 to 74 in 60742b6

	Mask = torch.eq(labels, labels.t()).float().to(device)
	#Mask = eye * torch.stack([labels == labels[i] for i in range(labels.size(0))]).float().to(device)
	Mask = Mask / (Mask.sum(dim=1, keepdim=True) + eps)

Maybe line 72 should be
Mask = torch.eq(labels, labels.t()).float().to(device) * (1 - eye)

I have some questions about it. May I trouble you to answer it?
Looking forward to your reply！