Support for real-valued function with complex arguments

[Uroc327]

If I understood #157 correctly, ForwardDiff should be able to differentiate a real-valued function with complex arguments. When I try this, I get the following error instead:

julia> ForwardDiff.gradient(v -> sum(abs2, v), [1.0+2.0im 3.0+4.0im 5.0+6.0im])
ERROR: ArgumentError: Cannot create a dual over scalar type Complex{Float64}. If the type behav
es as a scalar, define FowardDiff.can_dual.
Stacktrace:
 [1] throw_cannot_dual(::Type{T} where T) at /home/crunge/.julia/packages/ForwardDiff/qTmqf/src
/dual.jl:36
 [2] ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Complex{Float64}},Complex{Float64},3}(::Compl
ex{Float64}, ::ForwardDiff.Partials{3,Complex{Float64}}) at /home/crunge/.julia/packages/Forwar
dDiff/qTmqf/src/dual.jl:18
 [3] _broadcast_getindex_evalf at ./broadcast.jl:648 [inlined]
 [4] _broadcast_getindex at ./broadcast.jl:621 [inlined]
 [5] getindex at ./broadcast.jl:575 [inlined]
 [6] macro expansion at ./broadcast.jl:932 [inlined]
 [7] macro expansion at ./simdloop.jl:77 [inlined]
 [8] copyto! at ./broadcast.jl:931 [inlined]
 [9] copyto! at ./broadcast.jl:886 [inlined]
 [10] materialize! at ./broadcast.jl:848 [inlined]
 [11] materialize! at ./broadcast.jl:845 [inlined]
 [12] seed!(::Array{ForwardDiff.Dual{ForwardDiff.Tag{var"#21#22",Complex{Float64}},Complex{Floa
t64},3},2}, ::Array{Complex{Float64},2}, ::Tuple{ForwardDiff.Partials{3,Complex{Float64}},Forwa
rdDiff.Partials{3,Complex{Float64}},ForwardDiff.Partials{3,Complex{Float64}}}) at /home/crunge/
.julia/packages/ForwardDiff/qTmqf/src/apiutils.jl:65
 [13] vector_mode_dual_eval(::var"#21#22", ::Array{Complex{Float64},2}, ::ForwardDiff.GradientC
onfig{ForwardDiff.Tag{var"#21#22",Complex{Float64}},Complex{Float64},3,Array{ForwardDiff.Dual{F
orwardDiff.Tag{var"#21#22",Complex{Float64}},Complex{Float64},3},2}}) at /home/crunge/.julia/pa
ckages/ForwardDiff/qTmqf/src/apiutils.jl:36
 [14] vector_mode_gradient(::var"#21#22", ::Array{Complex{Float64},2}, ::ForwardDiff.GradientCo
nfig{ForwardDiff.Tag{var"#21#22",Complex{Float64}},Complex{Float64},3,Array{ForwardDiff.Dual{Fo
rwardDiff.Tag{var"#21#22",Complex{Float64}},Complex{Float64},3},2}}) at /home/crunge/.julia/pac
kages/ForwardDiff/qTmqf/src/gradient.jl:99
 [15] gradient(::Function, ::Array{Complex{Float64},2}, ::ForwardDiff.GradientConfig{ForwardDif
f.Tag{var"#21#22",Complex{Float64}},Complex{Float64},3,Array{ForwardDiff.Dual{ForwardDiff.Tag{v
ar"#21#22",Complex{Float64}},Complex{Float64},3},2}}, ::Val{true}) at /home/crunge/.julia/packa
ges/ForwardDiff/qTmqf/src/gradient.jl:19
 [16] gradient(::Function, ::Array{Complex{Float64},2}, ::ForwardDiff.GradientConfig{ForwardDif
f.Tag{var"#21#22",Complex{Float64}},Complex{Float64},3,Array{ForwardDiff.Dual{ForwardDiff.Tag{v
ar"#21#22",Complex{Float64}},Complex{Float64},3},2}}) at /home/crunge/.julia/packages/ForwardDi
ff/qTmqf/src/gradient.jl:17 (repeats 2 times)
 [17] top-level scope at REPL[558]:1
 [18] run_repl(::REPL.AbstractREPL, ::Any) at /build/julia/src/julia-1.5.3/usr/share/julia/stdl
ib/v1.5/REPL/src/REPL.jl:288

Is this use-case supported? If yes, how do I do this? I didn't find anything in the documentation.

[mcabbott]

I don't think there's an official support, or not yet. Making a crude attempt by fixing whatever gives an error:

julia> using ForwardDiff: ForwardDiff, Dual, value, partials
julia> ForwardDiff.can_dual(::Type{ComplexF64}) = true
julia> Base.abs2(x::Dual{Z,<:Complex{T}}) where {Z,T} = Dual{Z}(abs2(value(x)), 2 .* real.(partials(x).values) .+ 2 .* imag.(partials(x).values))

julia> ForwardDiff.gradient(v -> sum(abs2, v), [1.0+2.0im, 3.0+4.0im, 5.0+6.0im]' )
1×3 adjoint(::Vector{Float64}) with eltype Float64:
 2.0  2.0  2.0
 
 julia> Dual(1.0+2im, 3) isa Real
true

julia> Zygote.gradient(v -> sum(abs2, v), [1.0+2.0im, 3.0+4.0im, 5.0+6.0im]' )[1]
1×3 Matrix{ComplexF64}:
 2.0-4.0im  6.0-8.0im  10.0-12.0im

Notice that the vector this has created contains Dual{...,Complex}, while I thought the idea was to prefer Complex{Dual}, so more changes would be needed.