Bug in the Gaussian noise layer?

[kjerk]

Hello, I know it's been quite a while since the publication of the code but I wanted to check on a difference between the paper and the code.

According to the paper, Gaussian noise is meant to be injected in each of the RRDB blocks, but it looks like in the code there is an extra check disabling the noise unless the model is in training mode only. block.py:118

...
if self.training and self.sigma != 0:
    scale = self.sigma * x.detach() if self.is_relative_detach else self.sigma * x
    sampled_noise = self.noise.repeat(*x.size()).normal_() * scale
    x = x + sampled_noise
...

This would mean that for all times the model is used in model.eval() mode including in both test.py:18 test.py:56 scripts, that the Gaussian noise is actually disabled, which would affect the metrics and output.

I have double checked this myself by running an evaluation tensor through the model in training and eval modes and the outputs are different. So my questions is, is Gaussian noise meant to only be injected during training, or should the self.training check be removed?

...
if self.sigma != 0:
    scale = self.sigma * x.detach() if self.is_relative_detach else self.sigma * x
    sampled_noise = self.noise.repeat(*x.size()).normal_() * scale
    x = x + sampled_noise
...

[ncarraz]

Hello,
Thank you for pointing that out. The noise injection for this work is only for training. We extented this in another work (https://arxiv.org/abs/2009.12177) which exploits the noise during test time.

[kjerk]

Ah very cool, thank you for the clarification! (And the link to the new paper.)