Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENH: Only use Halley's adjustment to Newton if close enough. #8882

Merged
merged 3 commits into from Jul 13, 2018

Conversation

pvanmulbregt
Copy link
Contributor

@pvanmulbregt pvanmulbregt commented May 29, 2018

Closes gh-8881.

Halley's enhancement to the Newton-Raphson rule divides the step by (1 - 0.5 * f/f' * f''/f')
If (1 - 0.5 * f/f' * f''/f') is not close to 1, then the iteration could easily go in the wrong direction.
This only happens when the approximation is not that close. See gh-8881.

Algorithm change: If the denominator (1 - 0.5 * f/f' * f''/f') in the adjustment is not close to 1, don't use it, as the current approximation is not close enough to the root to guarantee that the necessary assumptions hold. Just fall-back to a Newton step.

Expected changes in behavior in existing usage:

  1. Some convergence failures will be replaced with successes.
  2. For an approximation now considered not close enough, the algorithm may take one or more Newton steps until close enough. This may lead to an increase in the number of iterations, or convergence to a different root.
  3. In some cases, newton(f,...,fprime2=fpp, ) may no longer converge when previously it did. This may occur if the derivative has a zero between the current approximation and the root, or if the sign of f'(x0) and f'(root) are different. [E.g f(x) = 16*x*(x-1)*(x+1)+7, which has a single real root -1.17 and is situated on an upward sloping part of the graph. Starting at x0 in a positive but downward sloping part of the graph (say between -0.5 and 0.5), the algorithm may or may not converge.]
    I.e. If Newton's algorithm would be on shaky ground due to the presence of stationary points, then using the modification here may change some previous Halley convergences to failures. And vice-versa.

@rgommers
Copy link
Member

Sounds like a reasonable change to me. Could you add the example of gh-8881 as a test?

If the denominator in Halley's adjustment to Newton is not close to 1, don't
use it, as the current approximation is not close enough to the root to
guarantee that the necessary assumptions hold.
@pvanmulbregt
Copy link
Contributor Author

Added an example of gh-8881 as a test. Rebased on a recent master.

@rgommers rgommers added this to the 1.2.0 milestone Jul 13, 2018
@rgommers rgommers merged commit 642c6f4 into scipy:master Jul 13, 2018
@rgommers
Copy link
Member

Merged, thanks Paul!

@pvanmulbregt pvanmulbregt deleted the halley_notclose branch July 18, 2018 00:23
@pvanmulbregt pvanmulbregt mentioned this pull request Sep 2, 2018
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement A new feature or improvement scipy.optimize
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Halley's enhancement to Newton can causes convergence failures
2 participants