optimize: curve-fitting - implement Levenberg-Marquardt algorithm (damped least-squares)

[sbinet]

just a follow-up on:
https://www.reddit.com/r/golang/comments/4atete/levenbergmarquardt_or_gaussnewton_go

https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm

Paraphrasing the above wikipedia link:

LMA is used to solve non-linear least squares problems.
The LMA is used in many software applications for solving generic curve-fitting problems.

[btracey]

I'm not sure where it should be located. Levenberg-Marquardt isn't the same as the other methods in optimize as it only solves a specific kind of problem. It may still belong in the base package, but I'm not sure.

[btracey]

Maybe optimize/nlls? Otherwise maybe in addition to local we could have another function.

[vladimir-ch]

I would prefer another function (LeastSquares?) taking another kind of problem (LeastSquaresProblem or LSProblem?).

[phil-mansfield]

I wanted to check in on the status of this issue (and about curve fitting in general in the Go environment).

The reason I'm asking is that I'm currently working on a Go-based affine-invariant Markov chain Monte Carlo sampler (similar to emcee) for one of my own projects. If you guys are interested, this could be used as a backend for general purpose fitting routines.

(MCMC doesn't fill exactly the same niche as LM, so it would probably be best to have an LM implementation, too.)

[sbinet]

AFAIK, nobody is actively working on this.

I still think it is indeed worthy of having in the gonum toolkit.
Feel free to either open a PR or drop a message on the gonum mailing list.

[sbinet]

@phil-mansfield FYI, I have started a design investigation at go-hep/fit with discussion on the gonum mailing list: https://groups.google.com/forum/#!topic/gonum-dev/AGekBtNCfJ8.

[vladimir-ch]

Closing because there is now gonum/gonum#295 in the new repository.