StepLength Extrapolation

[cortner]

Aside from

• search direction
• line search
  the step-length extrapolation (from step-length, gradient and function history) is maybe the most crucial aspect of the optimisation algorithms implemented in Optim.

I suggest to (minimally) refactor Optim to allow the user to choose how to perform this? E.g. cg.jl uses alphatry which is HZ's method, LBFGS uses just the previous step. But I'd really like to play around or even try my own variants.

CC: @pkofod @anriseth
(this should probably be done in parallel in Optim and LineSearches)

[cortner]

Maybe to simplify this, it might be better to make step length extrapolation part of the line-search type.

[anriseth]

I'll try to deal with this tomorrow or Friday

[pkofod]

Awesome!