Allow Python functions in minimize with NCG method

[mforets]

If algorithm="ncg" is used in the top-level minimize function, one should be able to work with Python functions, as with the other algorithms.

CC: @dimpase

Component: numerical

Keywords: minimize

Author: Marcelo Forets, Vincent Delecroix

Branch/Commit: public/23074 @ 36d0f31

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/23074

[mforets]

comment:1

For instance, this use case is currently broken:

sage: f = lambda x : 100*(x[1]-x[0]^2)^2+(1-x[0])^2+100*(x[2]-x[1]^2)^2+(1-x[1])^2
sage: gradient = lambda x : scipy.array([400*(x[0]**2 - x[1])*x[0] + 2*x[0] - 2, \
                                    -200*x[0]**2 + 400*(x[1]**2 - x[2])*x[1] + 202*x[1] - 2, \
                                     -200*x[1]**2 + 200*x[2]])
sage: x0 = [.1,.3,.4]
sage: minimize(f, x0, gradient, algorithm="ncg")
Traceback (most recent call last)
...
UnboundLocalError: local variable 'min' referenced before assignment

but it shouldn't, since this works:

sage: import scipy
sage: from scipy import optimize
sage: optimize.fmin_ncg(f, [float(_) for _ in x0], fprime=gradient, disp=True)
Optimization terminated successfully.
...
array([ 0.99999732,  0.99999463,  0.99998926])

Moreover, the Hessian can also be passed to ncg (but it's optional => docstring is inaccurate)

sage: hessian = lambda x : [[1200*x[0]^2 - 400*x[1] + 2, -400*x[0], 0], \
                    [-400*x[0], 1200*x[1]^2 - 400*x[2] + 202, -400*x[1]], \
                    [0, -400*x[1], 200]]
sage: optimize.fmin_ncg(f, [float(_) for _ in x0], fprime=gradient, fhess=hessian, disp=True)
Optimization terminated successfully.
...
array([ 1.,  1.,  1.])

[mforets]

comment:2

on a separate line, in the doc it says that

Only one of hessp or hess needs to be given.

Hence, in the current if clause of algorithm=="ncg" one does not need to compute & pass both hessian and hessian_p, but just one of them, i think.

[mforets]

Dependencies: 23062

[mforets]

Changed dependencies from 23062 to #23062

[mforets]

Branch: u/mforets/23074

[mforets]

Commit: 9416552

[mforets]

Author: Marcelo Forets

[mforets]

Last 10 new commits:

ff2f590	change disp -> verbose keyword
1f1fecf	fix keyword in doctests
be7fcf5	add rename keyword deprecation warning
469bd82	set default -> False, and update doctests
46e3497	add missing algorithm descriptions and wiki link
f0373a0	update verbose in docstring
540bdea	add complementary note
83e9a04	remove pair of backticks (all or nothing)
8e8289d	Merge branch 'develop' into t/mforets/23074
9416552	add ncg method to python funcs and add test

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

b658d33	add example with hessian
74da21c	update test

[sagetrac-git]

Changed commit from 9416552 to 74da21c

[sagetrac-git]

Changed commit from 74da21c to ac91ce2

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

ac91ce2

fix typo

[videlec]

comment:10

rebased

---

New commits:

c2fa6a5	add ncg method to python funcs and add test
e8c3208	add example with hessian
e2d29a1	update test
f7a5fb4	fix typo

[videlec]

Changed branch from u/mforets/23074 to public/23074

[videlec]

Changed dependencies from #23062 to none

[videlec]

Changed commit from ac91ce2 to f7a5fb4

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

4388049	add ncg method to python funcs and add test
e6c8e38	clean doctests

[sagetrac-git]

Changed commit from f7a5fb4 to e6c8e38

[videlec]

comment:12

Some refactorization in progress..

[sagetrac-git]

Changed commit from e6c8e38 to 36d0f31

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

36d0f31

refactor

[videlec]

Changed author from Marcelo Forets to Marcelo Forets, Vincent Delecroix

[dimpase]

comment:15

Thanks, looks good.

[dimpase]

Reviewer: Dima Pasechnik

[vbraun]

comment:16

On 32-bit:

**********************************************************************
File "src/sage/numerical/optimize.py", line 352, in sage.numerical.optimize.?
Failed example:
    minimize(f, [.1, .3, .4], algorithm="ncg") # abs tol 1e-6
Expected:
    (1.0, 1.0, 1.0)
Got:
    (1.0000003254615812, 1.000000652320023, 1.0000013080122123)
Tolerance exceeded in 1 of 3:
    1.0 vs 1.0000013080122123, tolerance 2e-6 > 1e-6
**********************************************************************
1 item had failures:
   1 of  20 in sage.numerical.optimize.?
    [104 tests, 1 failure, 1.15 s]
----------------------------------------------------------------------
sage -t --long src/sage/numerical/optimize.py  # 1 doctest failed
----------------------------------------------------------------------

[mkoeppe]

comment:17

Moving tickets to milestone sage-9.2 based on a review of last modification date, branch status, and severity.

[mkoeppe]

comment:19

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[mkoeppe]

comment:20

Setting a new milestone for this ticket based on a cursory review.