Question regarding hard rounding

[KwangHoonAn]

BRECQ/quant/adaptive_rounding.py

Lines 50 to 51 in 819d440

	else:
	x_int = x_floor + (self.alpha >= 0).float()

Would you elaborate how you derived hard-rounding scheme and what's the use of it?

[yhhhli]

Hi,

In hard rounding mode, we first round down the tensor (i.e, flooring). Then (self.alpha >= 0).float() is used to determine either rounding up or rounding down. When alpha is less than 0, it is 0, otherwise, it would be 1 and therefore round up the weights.

For mathematical explanation, you can find it on the paper. https://arxiv.org/abs/2004.10568