Is this implementation correct?

[Hhhhhhao]

Hi, I wonder if this implementation is correct?

When I run the code with softmax, I get weights like this:

tensor([[0.8824, 0.8824, 0.8824, 0.8824, 0.8824],
        [0.8824, 0.8824, 0.8824, 0.8824, 0.8824],
        [0.8824, 0.8824, 0.8824, 0.8824, 0.8824],
        [0.8824, 0.8824, 0.8824, 0.8824, 0.8824],
        [1.7646, 1.7646, 1.7646, 1.7646, 1.7646],
        [1.7646, 1.7646, 1.7646, 1.7646, 1.7646],
        [0.8824, 0.8824, 0.8824, 0.8824, 0.8824],
        [0.8824, 0.8824, 0.8824, 0.8824, 0.8824],
        [0.8824, 0.8824, 0.8824, 0.8824, 0.8824],
        [0.8824, 0.8824, 0.8824, 0.8824, 0.8824]])

where each sample enjoin the same weights.

This is weird, right? According to the original paper, we should have different weight for each class (column), right?

[zdaiot]

@HHHHHHHHHHao I have the same question. Can you help me?

[wmf1991yeah]

@HHHHHHHHHHao I have the same question too. Why?

[Hhhhhhao]

sorry I didn't figure out yet...

[Saadmaghani]

I know this is an old issue but I just want to give my understanding to any one who has the same question (as I did too)

Yes, the columns should have different values. But since Cross Entropy and Focal Loss take the one-hot labels of the target and multiply it by this weight matrix, essentially the final result is that you are only taking the weight for a particular class. That particular class' weight is actually the one you see above, repeated along the entire column.

Hope I helped!