Directional derivatives

[briochemc]

It would be great to have the API for directional derivatives for functions from ℝⁿ to ℝⁿ. I keep coming back to this problem and I can only guess that I'm not the only one.

I would like to suggest adding a directionalderivative function, such that for a function f(x::Vector) that returns another Vector of the same size,

ForwardDiff.directionalderivative(f, x, y)

gives the derivative of f at x in the y direction.

Right now my solution is to do something like

directionalderivative(f, x, y) = ForwardDiff.jacobian(λ -> f(x + λ .* y), [0.0])

But this feels a bit hacky and I'm sure there are some issues with it... But maybe that's OK? (I @btime'd it on vectors of size ~200'000 and it seemed to perform similar to using DualNumbers' dualpart.(f(x .+ ε * y))

[briochemc]

FWIW I realized from a slack comment that one should do

directionalderivative(f, x, y) = ForwardDiff.derivative(λ -> f(x .+ λ .* y), 0.0)

instead. Maybe that's as good as it gets?

[mcabbott]

Would still be neat to build this in as a convenience function. If it took a modified DerivativeConfig then it could save the allocation of Dual.(x,v), and handle f!, etc, too.

function ForwardDiff.derivative(f, x::AbstractArray, v::AbstractArray)
    size(x) == size(v) || throw(ArgumentError("argument and direction must be arrays of the same size"))
    y = f(ForwardDiff.Dual.(x,v))
    if y isa Real
        return ForwardDiff.partials(y,1)
    elseif y isa AbstractArray
        return ForwardDiff.partials.(y,1)
    else
        throw(ArgumentError("expected f(x) to be either a real number or an array"))
    end
end

Edit -- I guess this is a duplicate of #319