About TV loss

[mountains-high]

Hi~
Thank you for this great work.

My question is about the TV loss. Could you give the reason why you took mean while calculating the TV loss?
The paper about that work did not mention the 'mean'. Thank you

CMI/datafree/criterions.py

Line 48 in 9e79fa9

diff3.abs() / 255.0).mean() + (diff4.abs() / 255.0).mean()

[VainF]

Hi @mountains-high, diff1-4 measure the pixel-wise gradient in four directions. So we reduce these gradients to a scalar value for training.

[mountains-high]

Good day ~

Thank you for your reply, I got the point about the diff 1-4, however, didn't understand the taking "mean" of them. I found these lines in the paper Data-free Knowledge Distillation for Object Detection

According to equation 4 which they(the same authors) say that used on [44] isn't there to be 1/N if we consider the mean?
What do you think about it?

Thank you

[VainF]

Yes as you mentioned, the only difference between mean and sum lies in the scaling factor 1/N. You can adjust the their weight to get the same loss during training. In other word, if you want to use the summed TV loss, you need to lower down the weight of $\mathcal{L}_{TV}$ by N$\times$. However, there is not too much difference from the perspective of training.

[mountains-high]

Good day~
Thank you for the answer