specialize on input function on a few more wrapper functions

[KristofferC]

Benchmark

using ForwardDiff
using BenchmarkTools

function run0(x, f, n)
    out = similar(x);
    cfg = ForwardDiff.GradientConfig(f, x, ForwardDiff.Chunk{5}());
    for i = 1:n
        ForwardDiff.gradient!(out, f, x, cfg);
    end
    return out
end

V(x) = (dot(x,x)-1)^2;

x5 = rand(5);

@btime run0(x5, V, 100);

Before:

  42.694 μs (103 allocations: 6.92 KiB)

After:

  10.979 μs (3 allocations: 688 bytes)

Ref: https://discourse.julialang.org/t/forwarddiff-and-gradientconfig-memory-usage/10145/2.
Arguably, StaticArrays should have been used for this size but perhaps this is still worth it.
Same "issue" might exist for derivatives / jacobians etc.

[simonbyrne]

Wait, specialization is affected by the existence of type parameters in method definitions?

[ChrisRackauckas]

Wait, specialization is affected by the existence of type parameters in method definitions?

Only for functions IIRC (and Datatype has their own exception). I really don't like this exception...

[simonbyrne]

Is it documented anywhere?

[ChrisRackauckas]

JuliaLang/julia#19137