ENH: optimize, returning True from callback function halts minimization

[andyfaff]

If the callback functions in scipy.optimize return True, then the minimization halts.
I have not modified optimize.TNC because the callback function is called in the Fortran somewhere.

[andyfaff]

As part of this PR I'm wondering it it would be possible to have some way of tracking the progress of the minimization. For example, for a long minimization one could use this PR to halt the minimization if it's taking too long. But you might want to stop it after so many nit or nfev. Perhaps the callback functions could have an optional progress keyword. If progress = 1 (or 100%), then the minimization would be complete. My personal quest is to add a progress dialogue in my minimizer GUI.

At the moment the callback functions have a mostly uniform signature. All the minimize methods use callback(xk), basinhopping uses callback(x, f, accept), differential_evolution uses callback(x, convergence=0).

[dlax]

Why did you choose to return True in the callback after all? I actually liked better the exception approach you suggested on the ML.

[andyfaff]

I noticed that the basinhopping code halts if the callback returned True.
So I used that way for everything.
At the moment raising an exception will stop the minimization, but it's a
hard stop and you lose the state of the minimizer - nfev, etc.
Instead of testing for True from the callback I could wrap the callbacks in
a try/except block?
On 10/01/2015 11:53 PM, "Denis Laxalde" notifications@github.com wrote:

Why did you choose to return True in the callback after all? I actually
liked better the exception approach you suggested on the ML.

—
Reply to this email directly or view it on GitHub
#4384 (comment).

[dlax]

Andrew Nelson a écrit :

Instead of testing for True from the callback I could wrap the callbacks in
a try/except block?

I guess so. One advantage of this approach is that you can carry
additionnal information through the exception (like a message), which
could then be added to the OptimizeResult.

[argriffing]

The question of tracking the state of the minimization after halting reminds me of the approach taken in this project: https://github.com/mikecroucher/nearest_correlation/blob/master/nearest_correlation.py. I suggested that they could use an interface modeled on scipy's minimize interface, but they wanted to track more information (so that the minimization could be restarted, or whatever). So they do it like @dlax has mentioned, by carrying additional information through an exception.

[andyfaff]

So there are two places a user could raise an Exception, in the objective function or the callback.
One could catch either of those, does it make sense to catch one or both?

My use case is tracking what a minimizer is doing. I am using differential_evolution and a minimization can go on for a few minutes. I am doing this in a GUI program. I want to track (and display) what's happening to chi2, how many iterations have been done, how close to convergence I am, etc. If I notice a problem I would like to halt, but keep the best solution so far. As I see it there are two places for improvement:

1. Being able to halt the minimization. This is what this PR is for. Raising an Exception is a good way to do it. The status message in OptimizeResult would then have some variation.
2. At the moment the information provided to the callback function is limited, limited to the best solution so far, xk. I would like to see richer information being provided (function value, best solution vector so far, number of iterations, etc). But I don't know how to go about this without breaking backward compatibility.

[dlax]

Andrew Nelson wrote:

So there are two places a user could raise an |Exception|, in the
objective function or the callback.

One could catch either of those, does it make sense to catch one or both?

Working at the callback level is probably easier and makes more sense,
in my opinion.

My use case is tracking what a minimizer is doing. I am using
differential_evolution and a minimization can go on for a few minutes. I
am doing this in a GUI program. I want to track (and display) what's
happening to chi2, how many iterations have been done, how close to
convergence I am, etc. If I notice a problem I would like to halt, but
keep the best solution so far. As I see it there are two places for
improvement:

1. Being able to halt the minimization. This is what this PR is for.
   Raising an Exception is a good way to do it. The status message in
   OptimizeResult would then have some variation.

In addition, we could simply bind the exception object to the
OptimizeResult, hence everything the user would have put in would be
preserved.

1. At the moment the information provided to the |callback| function is
   limited, limited to the best solution so far, |xk|. I would like to
   see richer information being provided (function value, best solution
   vector so far, number of iterations, etc). But I don't know how to
   go about this without breaking backward compatibility.

So, in essence, you'd like to pass a state object similar to the final
OptimizeResult to the callback? Somthing like callback(xk, state=current_state).
I'd say we could handle backwards incompatibility through deprecation.

[andyfaff]

So, in essence, you'd like to pass a state object similar to the final OptimizeResult to the callback? Somthing like callback(xk, state=current_state).
I'd say we could handle backwards incompatibility through deprecation.

Actually one could just have callback(current_state) because the xk would already in current_state.

[andyfaff]

I was thinking that it could be achieved relatively simply by wrapping the callback function, similar to what a decorator can do. The wrapping function could take care of constructing the OptimizeResult and hold the try catch block.

[andyfaff]

@dlax, @argriffing how about this example:
https://github.com/andyfaff/scipy/tree/callback_exception_halts

[dlax]

Andrew Nelson a écrit :

@dlax https://github.com/dlax, @argriffing
https://github.com/argriffing how about this example:
https://github.com/andyfaff/scipy/tree/callback_exception_halts

AFAICT, this does not preserve backwards compatibility. In other words,
any callback function following the current specification and accepting
x as the only parameter and using it as an array will not work with
your proposal.

[andyfaff]

AFAICT, this does not preserve backwards compatibility. In other words, any callback function following the current specification and accepting x as the only parameter and using it as an array will not work with your proposal.

That's correct. Unfortunately I don't know how one would handle backwards compatibility. I suppose it could be caught as an exception in the wrapper code?
Obviously at the moment a callback function should only accept xk, but this code would require them to accept an OptimizeResult.
I would note that the callbacks for differential evolution and basinhopping have extra arg and keyword arguments.

[andyfaff]

this will be superseded by #8414, if it gets merged.

[tupui]

Closing this as superseded by #17486