-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Frozen parameters in GaussianFourierProjection #166
Comments
Hey @vvvm23, It's set to False because we don't want to train those parameters. I followed the implementaton of the original model here: https://github.com/yang-song/score_sde_pytorch/blob/1618ddea340f3e4a2ed7852a0694a809775cf8d0/models/layerspp.py#L37 Does this make sense? |
I somewhat misphrased my original question, I'm aware setting But why would we not want to train the noise level embeddings? Or is this just a simple, fixed (albeit randomly initialised) projection from a per-batch noise value to a different space, which would later have some learned transformation applied to it? Thanks! |
Hey @vvvm23, sinusoidal position features like If one wants to train position embedding vectors (or time embedding vectors here), one can just randomly initialize such a vector and let the model learn it. If however we use sinusoidal embeddings, there is no need to learn it |
Okay thank you @patrickvonplaten ! That explanation makes a lot of sense~ |
Hi, just a beginner with diffusion models and have been using your implementations as reference. I have a question about this class
Why is
requires_grad
set to false in the weight parameter? Won't this mean, during training, the noise level embeddings won't be updated?Thanks!
The text was updated successfully, but these errors were encountered: