Abount offset_noise

[suzhenghang]

What is the effect of the offset noise in training?

[JCBrouwer]

Apparently at the moment nothing (the variable with the offset noise added is never used):

Text-To-Video-Finetuning/train.py

Line 204 in 9cfcae5

if use_offset_noise:

But the idea is that the starting noise distribution does not have a mean of 0. For image-based diffusion models there is some evidence that this helps generate a more varied output distribution. It especially helps to generate very dark or very light images. See https://www.crosslabs.org/blog/diffusion-with-offset-noise for a more in depth discussion.