BUG: optimize: ensure line search is not confused by float issues

[pv]

Detect non-finite (nan, inf) results from _cubicmin and _quadmin so that
the rest of the line search doesn't become confused. Avoids problems
when float overflows etc. occur.

Also replace intermediate value checks by exceptions.

Fixes gh-2169

[josef-pkt]

@pv do you know which other optimizers or rootfinders are using this linesearch, besides bfgs?

[argriffing]

which other optimizers or rootfinders are using this linesearch, besides bfgs?

The ones that use line_search_wolfe2 or _line_search_wolfe12 or optimize.line_search, including at least cg and newton cg. I see that this is implemented from nocedal and wright, which is littered with errata.

[pv]

I think the change here is quite safe. The local minimizers either should return None or a finite number. Previously they could also return nan, which confused the zoom algorithm.

[rgommers]

LGTM. @josef-pkt @argriffing did you still want to look at this?

[josef-pkt]

I don't have any comment about the changes themselves, I don't know this code well enough.
I'm mainly interested in these fixes as a user, keeping track of where we might run into optimization problems in statsmodels, or where the problems disappeared.

[pv]

Some problems with e.g. using inf or very large numbers for penalties may disappear due to these changes.

[josef-pkt]

If my reading is correct, it should also help if the objective function returns nan for some parameter values, doesn't it?

[pv]

Yep.