ActNorm reinitializes on model load

[djd1283]

Thank you for the repo! Upon reloading the Glow model from save after training, cross entropy performance diminished. This was likely because the ActNorm module was being reinitialized on the first input batch to a value the model didn't expect. I added a constructor flag to specify whether to reinitialize, and this seemed to fix the problem. I would imagine this is a larger issue for datasets with larger variance and smaller batch size.

Fix:

• add an initialized flag to ActNorm constructor, set True when loading model

[MrHuff]

Hi! Well spotted, would you mind sharing your code/the bug fix?

Thank you!

[rosinality]

Hello, thanks for your suggestions! I haved missed that cases...I fixed this at f332121 by changing initialized flag as module buffers. So it will be saved with model weights.

[djd1283]

Awesome! Thanks for the fix.