Skip to content

Loading…

ENH: use memoization in MINPACK routines #236

Closed
wants to merge 3 commits into from

4 participants

@dlax
SciPy member

This implements memoization of the objective function and Jacobian for MINPACK routines.
See #130 for a rationale and use-case.

@pv pv commented on an outdated diff
scipy/optimize/optimize.py
((4 lines not shown))
+
+class MemoizeFun(object):
+ """ Decorator that caches the value of the objective function each time
+ it is called or only the first time if `first_only==True`."""
+ def __init__(self, fun, first_only=False):
+ self.fun = fun
+ self.calls = 0
+ self.first_only = first_only
+
+ def __call__(self, x, *args):
+ if self.calls == 0:
+ self.x = numpy.asarray(x).copy()
+ self.f = self.fun(x, *args)
+ self.calls += 1
+ return self.f
+ elif self.first_only:
@pv SciPy member
pv added a note

I think this doesn't work --- if first_only is True, the cached function value is never used.

@dlax SciPy member
dlax added a note
@dlax SciPy member
dlax added a note
def __call__(self, x, *args):
    if not hasattr(self, 'x'):
        self.x = numpy.asarray(x).copy()
        self.f = self.fun(x, *args)
        self.calls = 1
        return self.f
    elif self.first_only and self.calls > 1:
        return self.fun(x, *args)
    elif numpy.any(x != self.x):
        self.x = numpy.asarray(x).copy()
        self.f = self.fun(x, *args)
        self.calls += 1
        return self.f
    else:
        return self.f

Does this look better?

@dlax SciPy member
dlax added a note

fixed in a16c58a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@rgommers
SciPy member

OK, I'll wait; let me know when it's ready.

dlax added some commits
@dlax dlax ENH: use memoization in MINPACK routines cce626f
@dlax dlax FIX: copy __name__ attribute upon memoization in optimize
This is needed since, for some kinds of failures, MINPACK routines output
an error message which refers to the __name__ attribute of the objective
function and jacobian.
cd8ba57
@dlax
SciPy member

I've rebased the branch in order to ease merge.
It is ready now, I think.

@yosefm

Tested, it is slower than vanilla SciPy for me.
I have to apologize - when I tested in #130 I made a PYTHONPATH mistake, so I actually didn't test your version there. But now it's ok.

@dlax
SciPy member
@pv
SciPy member
pv commented

Yosef's problem is probably mainly Python overhead. Adding caching however will be useful in the opposite case when the objective function is slow. One can however make the memoization still faster by just adding the assumption that the first two calls are at same point, rather than explicitly checking for this condition which is always true...

@dlax
SciPy member

One can however make the memoization still faster by just adding the assumption that the first two calls are at same point, rather than explicitly checking for this condition which is always true...

I don't understand this. What do you suggest?

If this were to be dropped, note (to self as well) that cd8ba57 contains a fix that has to be applied anyways.

@pv
SciPy member
pv commented

@dlaxalde: assume that np.all(x == self.x) is true for the first call. The optimization algorithms probably always first evaluate the function value at the input point, so there is no need to actually check what the input argument actually is.

@dlax
SciPy member

assume that np.all(x == self.x) is true for the first call. The optimization algorithms probably always first evaluate the function value at the input point, so there is no need to actually check what the input argument actually is.

This is skipped in the first call since self.calls == 0 (self.x does not exist yet btw).

@pv
SciPy member
@dlax
SciPy member

Please see 3d24d3b.
Not sure it will solve your problem @yosefm but at least it skips one more call...

@pv pv added the PR label
@pv pv removed the PR label
@dlax dlax closed this
@dlax dlax deleted the dlax:enh/optimize/memoize-minpack branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jun 5, 2012
  1. @dlax
  2. @dlax

    FIX: copy __name__ attribute upon memoization in optimize

    dlax committed
    This is needed since, for some kinds of failures, MINPACK routines output
    an error message which refers to the __name__ attribute of the objective
    function and jacobian.
Commits on Jun 7, 2012
  1. @dlax
Showing with 40 additions and 1 deletion.
  1. +5 −1 scipy/optimize/minpack.py
  2. +35 −0 scipy/optimize/optimize.py
View
6 scipy/optimize/minpack.py
@@ -4,7 +4,7 @@
from numpy import atleast_1d, dot, take, triu, shape, eye, \
transpose, zeros, product, greater, array, \
all, where, isscalar, asarray, inf, abs
-from optimize import Result
+from optimize import Result, MemoizeFun
error = _minpack.error
@@ -183,6 +183,7 @@ def _root_hybr(func, x0, args=(), jac=None, options=None):
diag = options.get('diag', None)
full_output = True
+ func = MemoizeFun(func, first_only=True, nskip=1)
x0 = array(x0, ndmin=1)
n = len(x0)
if type(args) != type(()):
@@ -199,6 +200,7 @@ def _root_hybr(func, x0, args=(), jac=None, options=None):
retval = _minpack._hybrd(func, x0, args, full_output, xtol, maxfev,
ml, mu, epsfcn, factor, diag)
else:
+ Dfun = MemoizeFun(Dfun, first_only=True, nskip=1)
_check_func('fsolve', 'fprime', Dfun, x0, args, n, (n,n))
if (maxfev == 0):
maxfev = 100*(n + 1)
@@ -348,6 +350,7 @@ def leastsq(func, x0, args=(), Dfun=None, full_output=0,
params
"""
+ func = MemoizeFun(func, first_only=True, nskip=1)
x0 = array(x0, ndmin=1)
n = len(x0)
if type(args) != type(()):
@@ -361,6 +364,7 @@ def leastsq(func, x0, args=(), Dfun=None, full_output=0,
retval = _minpack._lmdif(func, x0, args, full_output, ftol, xtol,
gtol, maxfev, epsfcn, factor, diag)
else:
+ Dfun = MemoizeFun(Dfun, first_only=True, nskip=1)
if col_deriv:
_check_func('leastsq', 'Dfun', Dfun, x0, args, n, (n,m))
else:
View
35 scipy/optimize/optimize.py
@@ -39,11 +39,46 @@
'pr_loss': 'Desired error not necessarily achieved due '
'to precision loss.'}
+
+class MemoizeFun(object):
+ """ Decorator that caches the value of the objective function.
+ If `first_only==True`, only the first point is memoized.
+ If `nskip > 0`, it is assumed that the function is evaluated `nskip`
+ times at the first point, so that the memoized values is used. """
+ def __init__(self, fun, first_only=False, nskip=0):
+ self.fun = fun
+ if hasattr(fun, '__name__'):
+ self.__name__ = fun.__name__
+ self.first_only = first_only
+ self.nskip = nskip
+
+ def __call__(self, x, *args):
+ if hasattr(self, 'calls'):
+ self.calls += 1
+ else:
+ self.calls = 0
+ if self.calls == 0:
+ self.x = numpy.asarray(x).copy()
+ self.f = self.fun(x, *args)
+ return self.f
+ elif self.calls <= self.nskip:
+ return self.f
+ elif self.first_only and self.calls > 1:
+ return self.fun(x, *args)
+ elif numpy.any(x != self.x):
+ self.x = numpy.asarray(x).copy()
+ self.f = self.fun(x, *args)
+ return self.f
+ else:
+ return self.f
+
class MemoizeJac(object):
""" Decorator that caches the value gradient of function each time it
is called. """
def __init__(self, fun):
self.fun = fun
+ if hasattr(fun, '__name__'):
+ self.__name__ = fun.__name__
self.jac = None
self.x = None
Something went wrong with that request. Please try again.