Skip to content
This repository

ENH: use memoization in MINPACK routines #236

Open
wants to merge 3 commits into from

4 participants

Denis Laxalde Ralf Gommers Yosef Meller Pauli Virtanen
Denis Laxalde
Collaborator
dlax commented June 04, 2012

This implements memoization of the objective function and Jacobian for MINPACK routines.
See #130 for a rationale and use-case.

scipy/optimize/optimize.py
((4 lines not shown))
  42
+
  43
+class MemoizeFun(object):
  44
+    """ Decorator that caches the value of the objective function each time
  45
+    it is called or only the first time if `first_only==True`."""
  46
+    def __init__(self, fun, first_only=False):
  47
+        self.fun = fun
  48
+        self.calls = 0
  49
+        self.first_only = first_only
  50
+
  51
+    def __call__(self, x, *args):
  52
+        if self.calls == 0:
  53
+            self.x = numpy.asarray(x).copy()
  54
+            self.f = self.fun(x, *args)
  55
+            self.calls += 1
  56
+            return self.f
  57
+        elif self.first_only:
4
Pauli Virtanen Owner
pv added a note June 04, 2012

I think this doesn't work --- if first_only is True, the cached function value is never used.

Denis Laxalde Collaborator
dlax added a note June 04, 2012
Denis Laxalde Collaborator
dlax added a note June 04, 2012
def __call__(self, x, *args):
    if not hasattr(self, 'x'):
        self.x = numpy.asarray(x).copy()
        self.f = self.fun(x, *args)
        self.calls = 1
        return self.f
    elif self.first_only and self.calls > 1:
        return self.fun(x, *args)
    elif numpy.any(x != self.x):
        self.x = numpy.asarray(x).copy()
        self.f = self.fun(x, *args)
        self.calls += 1
        return self.f
    else:
        return self.f

Does this look better?

Denis Laxalde Collaborator
dlax added a note June 04, 2012

fixed in a16c58a

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Ralf Gommers
Owner

OK, I'll wait; let me know when it's ready.

added some commits June 04, 2012
Denis Laxalde ENH: use memoization in MINPACK routines cce626f
Denis Laxalde FIX: copy __name__ attribute upon memoization in optimize
This is needed since, for some kinds of failures, MINPACK routines output
an error message which refers to the __name__ attribute of the objective
function and jacobian.
cd8ba57
Denis Laxalde
Collaborator
dlax commented June 05, 2012

I've rebased the branch in order to ease merge.
It is ready now, I think.

Yosef Meller
yosefm commented June 07, 2012

Tested, it is slower than vanilla SciPy for me.
I have to apologize - when I tested in #130 I made a PYTHONPATH mistake, so I actually didn't test your version there. But now it's ok.

Denis Laxalde
Collaborator
dlax commented June 07, 2012
Pauli Virtanen
Owner
pv commented June 07, 2012

Yosef's problem is probably mainly Python overhead. Adding caching however will be useful in the opposite case when the objective function is slow. One can however make the memoization still faster by just adding the assumption that the first two calls are at same point, rather than explicitly checking for this condition which is always true...

Denis Laxalde
Collaborator
dlax commented June 07, 2012

One can however make the memoization still faster by just adding the assumption that the first two calls are at same point, rather than explicitly checking for this condition which is always true...

I don't understand this. What do you suggest?

If this were to be dropped, note (to self as well) that cd8ba57 contains a fix that has to be applied anyways.

Pauli Virtanen
Owner
pv commented June 07, 2012

@dlaxalde: assume that np.all(x == self.x) is true for the first call. The optimization algorithms probably always first evaluate the function value at the input point, so there is no need to actually check what the input argument actually is.

Denis Laxalde
Collaborator
dlax commented June 07, 2012

assume that np.all(x == self.x) is true for the first call. The optimization algorithms probably always first evaluate the function value at the input point, so there is no need to actually check what the input argument actually is.

This is skipped in the first call since self.calls == 0 (self.x does not exist yet btw).

Pauli Virtanen
Owner
pv commented June 07, 2012
Denis Laxalde
Collaborator
dlax commented June 07, 2012

Please see 3d24d3b.
Not sure it will solve your problem @yosefm but at least it skips one more call...

jnothman jnothman referenced this pull request from a commit May 08, 2013
Commit has since been removed from the repository and is no longer available.
Clemens ClemensFMN referenced this pull request from a commit August 06, 2013
Commit has since been removed from the repository and is no longer available.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 3 unique commits by 1 author.

Jun 05, 2012
Denis Laxalde ENH: use memoization in MINPACK routines cce626f
Denis Laxalde FIX: copy __name__ attribute upon memoization in optimize
This is needed since, for some kinds of failures, MINPACK routines output
an error message which refers to the __name__ attribute of the objective
function and jacobian.
cd8ba57
Jun 07, 2012
Denis Laxalde ENH: allow MemoizeFun to skip first points in MINPACK 3d24d3b
This page is out of date. Refresh to see the latest.
6  scipy/optimize/minpack.py
@@ -4,7 +4,7 @@
4 4
 from numpy import atleast_1d, dot, take, triu, shape, eye, \
5 5
                   transpose, zeros, product, greater, array, \
6 6
                   all, where, isscalar, asarray, inf, abs
7  
-from optimize import Result
  7
+from optimize import Result, MemoizeFun
8 8
 
9 9
 error = _minpack.error
10 10
 
@@ -183,6 +183,7 @@ def _root_hybr(func, x0, args=(), jac=None, options=None):
183 183
     diag      = options.get('diag', None)
184 184
 
185 185
     full_output = True
  186
+    func = MemoizeFun(func, first_only=True, nskip=1)
186 187
     x0 = array(x0, ndmin=1)
187 188
     n = len(x0)
188 189
     if type(args) != type(()):
@@ -199,6 +200,7 @@ def _root_hybr(func, x0, args=(), jac=None, options=None):
199 200
         retval = _minpack._hybrd(func, x0, args, full_output, xtol, maxfev,
200 201
                                  ml, mu, epsfcn, factor, diag)
201 202
     else:
  203
+        Dfun = MemoizeFun(Dfun, first_only=True, nskip=1)
202 204
         _check_func('fsolve', 'fprime', Dfun, x0, args, n, (n,n))
203 205
         if (maxfev == 0):
204 206
             maxfev = 100*(n + 1)
@@ -348,6 +350,7 @@ def leastsq(func, x0, args=(), Dfun=None, full_output=0,
348 350
          params
349 351
 
350 352
     """
  353
+    func = MemoizeFun(func, first_only=True, nskip=1)
351 354
     x0 = array(x0, ndmin=1)
352 355
     n = len(x0)
353 356
     if type(args) != type(()):
@@ -361,6 +364,7 @@ def leastsq(func, x0, args=(), Dfun=None, full_output=0,
361 364
         retval = _minpack._lmdif(func, x0, args, full_output, ftol, xtol,
362 365
                 gtol, maxfev, epsfcn, factor, diag)
363 366
     else:
  367
+        Dfun = MemoizeFun(Dfun, first_only=True, nskip=1)
364 368
         if col_deriv:
365 369
             _check_func('leastsq', 'Dfun', Dfun, x0, args, n, (n,m))
366 370
         else:
35  scipy/optimize/optimize.py
@@ -39,11 +39,46 @@
39 39
                    'pr_loss': 'Desired error not necessarily achieved due '
40 40
                               'to precision loss.'}
41 41
 
  42
+
  43
+class MemoizeFun(object):
  44
+    """ Decorator that caches the value of the objective function.
  45
+    If `first_only==True`, only the first point is memoized.
  46
+    If `nskip > 0`, it is assumed that the function is evaluated `nskip`
  47
+    times at the first point, so that the memoized values is used. """
  48
+    def __init__(self, fun, first_only=False, nskip=0):
  49
+        self.fun = fun
  50
+        if hasattr(fun, '__name__'):
  51
+            self.__name__ = fun.__name__
  52
+        self.first_only = first_only
  53
+        self.nskip = nskip
  54
+
  55
+    def __call__(self, x, *args):
  56
+        if hasattr(self, 'calls'):
  57
+            self.calls += 1
  58
+        else:
  59
+            self.calls = 0
  60
+        if self.calls == 0:
  61
+            self.x = numpy.asarray(x).copy()
  62
+            self.f = self.fun(x, *args)
  63
+            return self.f
  64
+        elif self.calls <= self.nskip:
  65
+            return self.f
  66
+        elif self.first_only and self.calls > 1:
  67
+            return self.fun(x, *args)
  68
+        elif numpy.any(x != self.x):
  69
+            self.x = numpy.asarray(x).copy()
  70
+            self.f = self.fun(x, *args)
  71
+            return self.f
  72
+        else:
  73
+            return self.f
  74
+
42 75
 class MemoizeJac(object):
43 76
     """ Decorator that caches the value gradient of function each time it
44 77
     is called. """
45 78
     def __init__(self, fun):
46 79
         self.fun = fun
  80
+        if hasattr(fun, '__name__'):
  81
+            self.__name__ = fun.__name__
47 82
         self.jac = None
48 83
         self.x = None
49 84
 
Commit_comment_tip

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.