DOC: optimize: separate legacy interfaces from new ones

[antonior92]

Hi all, in this PR I try to improve the structure of scipy.optimize API reference.

I tried to separate the new interfaces, such as minimize and root, from the deprecated ones. I think this should make the API reference easier to navigate for new users (and for the more experienced users as well).

[josef-pkt]

The multivariate fminxxx are not really deprecated, are they?

I don't remember seeing a decision to deprecate them.

[antonior92]

Ok, maybe deprecated is the wrong word.

But they are interfaces that should be avoided in new scripts and I think it would improve the documentation to separate them from the newer and more consistent interfaces.

Maybe I can just change the section name to "Old Interfaces"

[dlax]

Perhaps "legacy" is a more appropriate term than "deprecated".

[antonior92]

Thank you @dlax. "Legacy" is a much better term for describing this functions.

[rgommers]

This restructuring seems like a good idea to me.

[antonior92]

Thank you all for the comments. I have just amend a few minor modifications.

Anyone, please let me know if there are other parts that need to be improved or better choice of words that I could use.

[dlax]

It seems that the HessianUpdateStrategy line got dropped (unless I missed it). I guess it should stay along with LbfgsInvHessProduct in "Hessian Approximation" section above.

[antonior92]

Ok, just included this line back.

[antonior92]

For easy reference, the documentation can be accessed through this link:
https://7187-1460385-gh.circle-artifacts.com/0/html-scipyorg/optimize.html

[twmr]

Does it make sense to deprecate those legacy optimizers and throw them out in future release?

[rgommers]

Does it make sense to deprecate those legacy optimizers and throw them out in future release?

Not really, they're very widely used and they're not broken. We've decided to leave them in place, and just in the docs direct people to the newer interface.

[josef-pkt]

Does it make sense to deprecate those legacy optimizers and throw them out in future release?

statsmodels still relies dominantly on the "legacy" interface, even though we have minimize now as optional method.
Changing this would be quite a bit of work, and would need a lot of checking. The scipy optimizers are at the core of almost all our model estimation. So if something doesn't work it would have a very large impact.
scipy minimize looks stable enough now, I guess even for the minimum scipy version that statsmodels supports, so I would not rule out anymore switching over to minimize, but it's still low priority work given that the current interface has worked for us for 8 or 9 years.

[josef-pkt]

To add to statsmodels comment:
We have not added any new wrappers since basinhopping, so we need to go through minimize for the newer optimizers. This will most likely lead to a slow migration to have more parts using the minimize interface, for now the main default methods are still fmin_bfgs and fmin_lbfgsb.