StyleLoss dividing by n twice

[rrshaban]

In StyleLoss:updateOutput neural_style.lua:398:

function StyleLoss:updateOutput(input)
  self.G = self.gram:forward(input)
  self.G:div(input:nElement())
  self.loss = self.crit:forward(self.G, self.target)
  ...
end

Our criterion function (self.crit = nn.MSECriterion()) already divides by n after calculating MSE: nn.MSECriterion. So it seems to me that we are dividing by n twice; once before passing it to MSE and then once again within MSE.

Is this intentional, or am I missing something?

[jcjohnson]

You're right - we do divide by the same thing twice. Per-layer, this is basically a no-op since it just scales the style loss for that layer by a constant.

However since we use the same style loss weight for all layers, any normalization has the effect of scaling the contributions of the style losses from different layers, since n is different for each layer. I didn't really have a good theoretical justification for this particular type of normalization, but empirically it tends to give good results.

[rrshaban]

Thanks for the detailed reply, as well as the great code!