New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GaussianCriterion gradient issue #13

Closed
syyeung opened this Issue Jul 18, 2016 · 1 comment

Comments

Projects
None yet
2 participants
@syyeung

syyeung commented Jul 18, 2016

In GaussianCriterion.lua the gradient with respect to log(sigma^2) is computed as:

self.gradInput[2] = torch.exp(-input[2]):cmul(torch.add(target,-1,input[1]):pow(2)):mul(-1):add(0.5)

But it seems to me that the multiplication should be by -0.5, not -1, such that

self.gradInput[2] = torch.exp(-input[2]):cmul(torch.add(target,-1,input[1]):pow(2)):mul(-0.5):add(0.5)

which would also be consistent with the expression in your comment (after flipping the sign for neg. log likelihood): - 0.5 + 0.5 * (x - mu)^2 / sigma^2

I am seeing a gradient check fail with the original code but it passes after the change.

@y0ast y0ast closed this in f820c7f Aug 30, 2016

@y0ast

This comment has been minimized.

Show comment
Hide comment
@y0ast

y0ast Aug 30, 2016

Owner

You are absolutely right, I have no idea why the comment and code were inconsistent. I also flipped the sign of all the equations in the comments, to be fully consistent.

Thanks!

Owner

y0ast commented Aug 30, 2016

You are absolutely right, I have no idea why the comment and code were inconsistent. I also flipped the sign of all the equations in the comments, to be fully consistent.

Thanks!

y0ast added a commit that referenced this issue Dec 19, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment