Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

ENH: use memoization in MINPACK routines #236

Closed
wants to merge 3 commits into from

4 participants

Denis Laxalde Ralf Gommers Yosef Meller Pauli Virtanen
Denis Laxalde
Collaborator

This implements memoization of the objective function and Jacobian for MINPACK routines.
See #130 for a rationale and use-case.

scipy/optimize/optimize.py
((4 lines not shown))
+
+class MemoizeFun(object):
+ """ Decorator that caches the value of the objective function each time
+ it is called or only the first time if `first_only==True`."""
+ def __init__(self, fun, first_only=False):
+ self.fun = fun
+ self.calls = 0
+ self.first_only = first_only
+
+ def __call__(self, x, *args):
+ if self.calls == 0:
+ self.x = numpy.asarray(x).copy()
+ self.f = self.fun(x, *args)
+ self.calls += 1
+ return self.f
+ elif self.first_only:
Pauli Virtanen Owner
pv added a note

I think this doesn't work --- if first_only is True, the cached function value is never used.

Denis Laxalde Collaborator
dlax added a note
Denis Laxalde Collaborator
dlax added a note
def __call__(self, x, *args):
    if not hasattr(self, 'x'):
        self.x = numpy.asarray(x).copy()
        self.f = self.fun(x, *args)
        self.calls = 1
        return self.f
    elif self.first_only and self.calls > 1:
        return self.fun(x, *args)
    elif numpy.any(x != self.x):
        self.x = numpy.asarray(x).copy()
        self.f = self.fun(x, *args)
        self.calls += 1
        return self.f
    else:
        return self.f

Does this look better?

Denis Laxalde Collaborator
dlax added a note

fixed in a16c58a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Ralf Gommers
Owner

OK, I'll wait; let me know when it's ready.

dlax added some commits
Denis Laxalde dlax ENH: use memoization in MINPACK routines cce626f
Denis Laxalde dlax FIX: copy __name__ attribute upon memoization in optimize
This is needed since, for some kinds of failures, MINPACK routines output
an error message which refers to the __name__ attribute of the objective
function and jacobian.
cd8ba57
Denis Laxalde
Collaborator

I've rebased the branch in order to ease merge.
It is ready now, I think.

Yosef Meller

Tested, it is slower than vanilla SciPy for me.
I have to apologize - when I tested in #130 I made a PYTHONPATH mistake, so I actually didn't test your version there. But now it's ok.

Denis Laxalde
Collaborator
Pauli Virtanen
Owner
pv commented

Yosef's problem is probably mainly Python overhead. Adding caching however will be useful in the opposite case when the objective function is slow. One can however make the memoization still faster by just adding the assumption that the first two calls are at same point, rather than explicitly checking for this condition which is always true...

Denis Laxalde
Collaborator

One can however make the memoization still faster by just adding the assumption that the first two calls are at same point, rather than explicitly checking for this condition which is always true...

I don't understand this. What do you suggest?

If this were to be dropped, note (to self as well) that cd8ba57 contains a fix that has to be applied anyways.

Pauli Virtanen
Owner
pv commented

@dlaxalde: assume that np.all(x == self.x) is true for the first call. The optimization algorithms probably always first evaluate the function value at the input point, so there is no need to actually check what the input argument actually is.

Denis Laxalde
Collaborator

assume that np.all(x == self.x) is true for the first call. The optimization algorithms probably always first evaluate the function value at the input point, so there is no need to actually check what the input argument actually is.

This is skipped in the first call since self.calls == 0 (self.x does not exist yet btw).

Pauli Virtanen
Owner
pv commented
Denis Laxalde
Collaborator

Please see 3d24d3b.
Not sure it will solve your problem @yosefm but at least it skips one more call...

jnothman jnothman referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
Clemens ClemensFMN referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
Pauli Virtanen pv added the PR label
Pauli Virtanen pv removed the PR label
Denis Laxalde dlax closed this
Denis Laxalde dlax deleted the branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jun 5, 2012
  1. Denis Laxalde
  2. Denis Laxalde

    FIX: copy __name__ attribute upon memoization in optimize

    dlax authored
    This is needed since, for some kinds of failures, MINPACK routines output
    an error message which refers to the __name__ attribute of the objective
    function and jacobian.
Commits on Jun 7, 2012
  1. Denis Laxalde
This page is out of date. Refresh to see the latest.
Showing with 40 additions and 1 deletion.
  1. +5 −1 scipy/optimize/minpack.py
  2. +35 −0 scipy/optimize/optimize.py
6 scipy/optimize/minpack.py
View
@@ -4,7 +4,7 @@
from numpy import atleast_1d, dot, take, triu, shape, eye, \
transpose, zeros, product, greater, array, \
all, where, isscalar, asarray, inf, abs
-from optimize import Result
+from optimize import Result, MemoizeFun
error = _minpack.error
@@ -183,6 +183,7 @@ def _root_hybr(func, x0, args=(), jac=None, options=None):
diag = options.get('diag', None)
full_output = True
+ func = MemoizeFun(func, first_only=True, nskip=1)
x0 = array(x0, ndmin=1)
n = len(x0)
if type(args) != type(()):
@@ -199,6 +200,7 @@ def _root_hybr(func, x0, args=(), jac=None, options=None):
retval = _minpack._hybrd(func, x0, args, full_output, xtol, maxfev,
ml, mu, epsfcn, factor, diag)
else:
+ Dfun = MemoizeFun(Dfun, first_only=True, nskip=1)
_check_func('fsolve', 'fprime', Dfun, x0, args, n, (n,n))
if (maxfev == 0):
maxfev = 100*(n + 1)
@@ -348,6 +350,7 @@ def leastsq(func, x0, args=(), Dfun=None, full_output=0,
params
"""
+ func = MemoizeFun(func, first_only=True, nskip=1)
x0 = array(x0, ndmin=1)
n = len(x0)
if type(args) != type(()):
@@ -361,6 +364,7 @@ def leastsq(func, x0, args=(), Dfun=None, full_output=0,
retval = _minpack._lmdif(func, x0, args, full_output, ftol, xtol,
gtol, maxfev, epsfcn, factor, diag)
else:
+ Dfun = MemoizeFun(Dfun, first_only=True, nskip=1)
if col_deriv:
_check_func('leastsq', 'Dfun', Dfun, x0, args, n, (n,m))
else:
35 scipy/optimize/optimize.py
View
@@ -39,11 +39,46 @@
'pr_loss': 'Desired error not necessarily achieved due '
'to precision loss.'}
+
+class MemoizeFun(object):
+ """ Decorator that caches the value of the objective function.
+ If `first_only==True`, only the first point is memoized.
+ If `nskip > 0`, it is assumed that the function is evaluated `nskip`
+ times at the first point, so that the memoized values is used. """
+ def __init__(self, fun, first_only=False, nskip=0):
+ self.fun = fun
+ if hasattr(fun, '__name__'):
+ self.__name__ = fun.__name__
+ self.first_only = first_only
+ self.nskip = nskip
+
+ def __call__(self, x, *args):
+ if hasattr(self, 'calls'):
+ self.calls += 1
+ else:
+ self.calls = 0
+ if self.calls == 0:
+ self.x = numpy.asarray(x).copy()
+ self.f = self.fun(x, *args)
+ return self.f
+ elif self.calls <= self.nskip:
+ return self.f
+ elif self.first_only and self.calls > 1:
+ return self.fun(x, *args)
+ elif numpy.any(x != self.x):
+ self.x = numpy.asarray(x).copy()
+ self.f = self.fun(x, *args)
+ return self.f
+ else:
+ return self.f
+
class MemoizeJac(object):
""" Decorator that caches the value gradient of function each time it
is called. """
def __init__(self, fun):
self.fun = fun
+ if hasattr(fun, '__name__'):
+ self.__name__ = fun.__name__
self.jac = None
self.x = None
Something went wrong with that request. Please try again.