Skip to content
This repository has been archived by the owner on Nov 23, 2018. It is now read-only.

Linesearchers should test function decrease to within a tolerance #128

Closed
btracey opened this issue Sep 24, 2015 · 2 comments
Closed

Linesearchers should test function decrease to within a tolerance #128

btracey opened this issue Sep 24, 2015 · 2 comments

Comments

@btracey
Copy link
Member

btracey commented Sep 24, 2015

The linesearches we implement should not test for strict function decrease (fnew <= fmin), but instead test within a tolerance. This helps get closer to the true minimum (based on the gradient norm), as the function value converges faster than the gradient for a smooth problem near the optimum.

This was originally part of the line search, but was removed because of confusion with updating OptLoc. Now that the optimizer is strictly in charge of updating OptLoc, we can add this test back in.

It should be a tunable parameter. It should work properly with #127 .

@vladimir-ch
Copy link
Member

Just a simple observation to support that we should do this: requiring strict decrease of fnew in Bisection breaks two tests for BFGS and LBFGS. Permitting <= allows to get to a point with smaller gradient norm, so if we permit even a tiny increase, we could get closer to the minimizer for some problems.

@btracey
Copy link
Member Author

btracey commented Mar 16, 2017

We tried this, and it turned out to be a bad idea. We replaced this with the use of MajorIteration for the optimizer to control when stats get updated.

@btracey btracey closed this as completed Mar 16, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants