-
Notifications
You must be signed in to change notification settings - Fork 133
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A question about gradient #15
Comments
Hi, this is not a bug. We need to backdrop gradient signal from these two losses to the embedding function in order to jointly train. |
@XiangLi1999 Hi, I'm curious about the loss funtion too. I can not understand why we need to compute |
Hi @summmeer , Thanks for your reply in advance. It would help me a lot. Best, |
Diffusion-LM/improved-diffusion/improved_diffusion/gaussian_diffusion.py
Lines 1549 to 1557 in 352b28c
Hi, I see there are three main losses. MSE, decoder_nll and kl_T. I think decoder_nll is designed for the decoder and MSE is designed for the diffusion model. However, I see x_start is not detached from these two losses, so these two losses also influence the embedding part. Is this a bug or a particular design?
The text was updated successfully, but these errors were encountered: