MAINT: Benchmarks for optimize.leastsq

[nmayorov]

The general idea is to add benchmarks for the current and future nonlinear least squares optimization solvers. But now we only have optimize.leastsq and the shape of the future methods is not know yet.

I kept benchmarking class very simple and without strong considerations of future. But nevertheless the added problems will suit well for the future methods too, and the benchmarking class is easy to modify (and it's not public). So I decided to show this PR early to get a feedback.

I added almost all LSQ problems from MINPACK-2 problems set. They are rather small unconstrained problems, but perfectly fine for basic benchmarking. I'm planning to expand the problem set as I progress during GSoC.

Here is how the report looks like:

[  0.00%] ·· Building for /Users/nmayorov/anaconda/bin/python
[  0.00%] ·· Benchmarking /Users/nmayorov/anaconda/bin/python
[100.00%] ··· Running optimize.BenchLeastSquares.track_all                                                                            ok
[100.00%] ···· 
               ====================== ======================= ====== =========
               --                                   result type               
               ---------------------- ----------------------------------------
                      problem               average time       nfev   success 
               ====================== ======================= ====== =========
                 AlphaPineneDirect       0.0959238052368164     22       1    
                ChebyshevQuadrature     0.08988590240478515     41       1    
                  CoatingThickness      0.020872282981872558    6        1    
                   EnzymeReaction      0.0005537033081054688    11       1    
                 ExponentialFitting    0.0006913900375366211    18       1    
                  GaussianFitting      0.0015572071075439452    16       1    
                ThermistorResistance    0.005420684814453125   216       1    
               ====================== ======================= ====== =========

[rgommers]

This looks promising. Just a note about gh-4191, which contains lots of global optimization benchmarks. Doesn't overlap with this, just wanted to point out related work in progress.

[ev-br]

This is probably OK, but I cannot help but notice that ASV selects the number of runs with a more sophisticated way:
https://github.com/spacetelescope/asv/blob/master/asv/benchmark.py#L451

@pv what is the recommended way of making multiple measurements on a benchmark?

[pv]

@ev-br: not possible to do that with asv currently, so hard to do better than the above. This is on the asv todo list, however. The old optimizer benchmarks also have this problem, see above in the code, so we'll refactor once it becomes possible to do.

[ev-br]

Thanks Nikolay, Pauli.