Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also compare across forks.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also compare across forks.
base fork: schaul/torch7-demos
base: 8e24ed0fc6
...
head fork: schaul/torch7-demos
compare: 329c6c3903
  • 2 commits
  • 1 file changed
  • 0 commit comments
  • 1 contributor
Showing with 49 additions and 5 deletions.
  1. +49 −5 xorexample.lua
View
54 xorexample.lua
@@ -121,14 +121,21 @@ net:updateParameters(0.2) -- this is the learning rate
-- the weights and error have changed now
print("1st epoch", totalMSE(), "w",ll2.weight)
--- train more
-for i=1,99 do
+
+function oneEpoch()
net:zeroGradParameters()
- for i=1,4 do
- net:forward(inpt[i])
+ local s=0
+ for i=1,inpt:size(1) do
+ s = s + criterion:forward(net:forward(inpt[i]), targ[i])
net:backward(inpt[i], criterion:backward(net.output, targ[i]))
net:accGradParameters(inpt[i], criterion.gradInput)
- end
+ end
+ return s
+end
+
+-- train more
+for i=1,99 do
+ oneEpoch()
net:updateParameters(0.2)
end
print("100th epoch", totalMSE(),"w", ll2.weight)
@@ -150,6 +157,11 @@ end
optimizer = nn.SGDOptimization{module = net,
criterion = criterion,
learningRate = 0.2}
+
+-- if we don't want to guess at the best learning rate we can call this method:
+-- optimizer:optimalLearningRate(inpt, targ)
+-- print("Optimal learning rate", optimizer.learningRate)
+-- It turns out that this does not work in our case (much too big, why?)
-- for some reason, we provide virtually the same information
-- again to construct the trainer
@@ -170,3 +182,35 @@ print = _print
-- note that often the weights found in this second run are very different from the ones
-- in the first run, which is due to the random re-initialization.
print("100th epoch", totalMSE(),"w", ll2.weight)
+
+
+
+-- (7) A third approach is the optimizer framework with "evaluate()" closures
+netReset(net)
+print("new init", totalMSE(), "w",ll2.weight)
+
+-- For this we need a flattened view on the parameters and derivatives
+parameters = nnx.flattenParameters(nnx.getParameters(net))
+gradParameters = nnx.flattenParameters(nnx.getGradParameters(net))
+
+-- For example, using the LBFGS algorithm
+require 'liblbfgs'
+
+-- this needs to be a closure that does a forward-backward
+lbfgs.evaluate = oneEpoch
+
+-- init LBFGS state
+maxIterations = 100
+maxLineSearch = 10
+maxEvaluation = 100
+linesearch = 0
+sparsity = 0
+verbose = 0
+lbfgs.init(parameters, gradParameters,
+ maxEvaluation, maxIterations, maxLineSearch,
+ sparsity, linesearch, verbose)
+
+-- here's where the optimization takes place
+output = lbfgs.run()
+
+print("100th epoch", totalMSE(),"w", ll2.weight)

No commit comments for this range

Something went wrong with that request. Please try again.