Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BFGS and Newton: reset initial linesearch step to one #328

Merged
merged 5 commits into from
Jan 10, 2017
Merged

BFGS and Newton: reset initial linesearch step to one #328

merged 5 commits into from
Jan 10, 2017

Conversation

anriseth
Copy link
Contributor

@anriseth anriseth commented Dec 23, 2016

Resetting the initial step for the linesearch to one after each iteration in Newton and BFGS improves convergence:
Newton CUTEst tests with maxvar = 5:
f_err_cutest
BFGS CUTEst tests with maxvar = 5:
f_err_bfgs_cutest

BT3 is backtracking with cubic interpolation, added in JuliaNLSolvers/LineSearches.jl#20. The behaviour for quadratic bactracking is similar.
LBFGS already has a variant of this implemented, so I haven't added it there.
Ref #91

@codecov-io
Copy link

codecov-io commented Dec 23, 2016

Current coverage is 89.19% (diff: 97.22%)

Merging #328 into master will decrease coverage by <.01%

@@             master       #328   diff @@
==========================================
  Files            27         28     +1   
  Lines          1684       1693     +9   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits           1502       1510     +8   
- Misses          182        183     +1   
  Partials          0          0          

Powered by Codecov. Last update f29374a...42f457a

@pkofod
Copy link
Member

pkofod commented Dec 23, 2016

Just out of curiosity, did you choose 5 because of optimtests.jl? I chose that to quickly debug the code when migrating. I love that we're beginning to have a tool to say something about the quality of changes. I would love to see it for some large(r) scale problems as well. But it looks very promising!

@anriseth
Copy link
Contributor Author

anriseth commented Dec 23, 2016

Yep, I just copied your benchmark code from OptimTests.jl.
Here's a run with Newton and maxvar = 20 (my laptop is slow, so I can't go much higher):
f_err_cutest

@pkofod
Copy link
Member

pkofod commented Dec 30, 2016

It almost looks good to me, but the entry in the docs for the constructors have to be updated.

@pkofod
Copy link
Member

pkofod commented Jan 5, 2017

@anriseth pinging in case you missed it. I know you might be busy, but :)

@anriseth
Copy link
Contributor Author

anriseth commented Jan 5, 2017 via email

@pkofod
Copy link
Member

pkofod commented Jan 5, 2017

Great, Sorry if I bothered you on your vacation!

@anriseth
Copy link
Contributor Author

Updated the docs now. Are the changes clear enough?

@pkofod
Copy link
Member

pkofod commented Jan 10, 2017

Great, thanks for doing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants