energy_new in CAM

[eugenelawrence]

Hi @junfu1115 ,
why do you use the energy_new in attention.py?
Correct me if I miss something.

[junfu1115]

Prevent loss divergence during training

[weix-liu]

To prevent overflow or underflow, it should be x - max(x), rather than max(x) - x

[HanzhiC]

Yes. I am also wondering why using max(x) - x, seems that your correlation will favor the most different channel information instead the most similar one

[kamiLight]

Yes. I am also wondering why using max(x) - x, seems that your correlation will favor the most different channel information instead the most similar one

i have the same opinion with you, using max(x) - x will give bigger weights to the feature with less similarity after performing softmax

[carolo44]

Yes. I am also wondering why using max(x) - x, seems that your correlation will favor the most different channel information instead the most similar one

I think it should be x - max(x) and actually this operation is also redundant, pytorch itself has done it already.