Questions about GaussianCriterion.lua

[xcyan]

self.gradInput[2] = torch.exp(-input[2]):cmul(torch.add(target,-1,input[1]):pow(2)):add(-0.5)

It seems to me that gradient updating step should be:

self.gradInput[2] = torch.exp(-input[2]):cmul(torch.add(target,-1,input[1]):pow(2)):mul(0.5):add(-0.5)

[y0ast]

Hmm, there is definitely something wrong but your solution is not correct.

input[1] = mu
input[2] = log(sigma^2)

The forward is:
-0.5 * (log(sigma) + log(2pi)) - 0.5 * (x - mu)^2/sigma^2

The backward for log(sigma^2) is:

• 0.25 + 0.5 * (x - mu)^2 / sigma^2

derivative of 1/sigma^2 with respect to log(sigma^2) is:
-exp(-log(sigma^2)) (using: 1/sigma^2 = exp(-log(sigma^2)))

The minus cancels the minus of - 0.5 * (x - mu)^2 and you reach the above mentioned backward.

See the commit for further details :). Thanks for pointing it out!