Permalink
Browse files

Fixed weight decay to scale with learning rate.

  • Loading branch information...
1 parent abbbab1 commit 0369e3b334eeeb26495b8b6812f2d3076f7e4596 @clementfarabet clementfarabet committed Jan 18, 2012
Showing with 1 addition and 1 deletion.
  1. +1 −1 sgd.lua
View
@@ -43,7 +43,7 @@ function optim.sgd(opfunc, x, state)
-- (2) weight decay
if wd ~= 0 then
- x:add(-wd, x)
+ x:add(-wd*lr, x)
end
-- (3) learning rate decay (annealing)

0 comments on commit 0369e3b

Please sign in to comment.