OEP: more general state for user-supplied functions

[timholy]

It would be lovely to be able to support sparse Hessians, Jacobians (for constrained optimization), etc. It would be good to move to an API where one can supply HessState and JacState objects of any kind. The user-supplied functions would update the state at the current point, and then such state objects would have to support J*x and H\x type of operations. Obviously H could be a dense n-by-n matrix, but it wouldn't have to be.

[pkofod]

I will try to fit this in when I have a shot at the dispatch based function interface. I intend to target this after #304 , as quite a few open issues are related to this.

[pkofod]

Currently #337 simply opens up the fields for storing gradients and hessians. This means that if you use the full constructor (with all the fields), you can input your own gradient type as storage as long as you've implemented

copy
copy!
getindex
vecdot(s::Array, g::G)

for your type G. For the Hessian you can in principle do the same, but currently you'll run into a wall when it gets passed to the NewtonTrustRegion solver, and in Newton you'd need to support a method that can be passed to PositiveFactorizations. I do intend to make a switch, such that you can simply do good old renegade Newton steps: -H\g. Then you just need your state to be supported by a \ method.

I'm going to leave Newton alone for now, but I will try to make some stupid examples with the gradient, such as

type MyStupidGradient{T}
    g::Vector{T}
end

just to make sure it works all the way through. Some changes might have to be made over at LineSearches.jl, but I will find out :)

[pkofod]

Should be possible with #337. If you create a OnceDifferentiable "manually" that is you also provide the g yourself, then this is possible. (the gradient storage is parametric now, as is the Hessian)