A question about variable self.acc_sum in GHMC_loss

[sjtuliuqin]

Hi, Buyu,
Thanks for your great work !
In line 22 of ghm_loss.py, the variable self.accu_sum in GHMC_loss class is initialized as a list filling with zeros. In line 47, you update it with the following code:
self.acc_sum[i] = mmt * self.acc_sum[i]
+ (1 - mmt) * num_in_bin
As self.acc_sum[i] is initialized as zero, mmt * self.acc_sum[i] has no meaning. Could you explain a little bit about your idea here?

Best Regards
Qin

[sjtuliuqin]

I think I have understood your intention.
As the name 'acc_sum' indicates, you want to record the accumulated sum of all samples whose gradient lies within the bin. Thus, as batch iteration goes on, self.acc_sum would consider not only samples in the current batch, but also its previous value. The ratio is controlled by 'mmt'.
Thanks again !

[libuyu]

@sjtuliuqin Yes, you got it. And for the theory of the "acc_sum", you can refer to the part "EMA" on page 4 in our paper https://arxiv.org/pdf/1811.05181.pdf. Thank you for your interest.