Ambiguity with forwarddiff ?

[lrnv]

In lrnv/Copulas.jl#301,

we have a

julia> using ForwardDiff, TaylorDiff

julia> x = ForwardDiff.Dual(1, 1)
Dual{Nothing}(1,1)

julia> y = ForwardDiff.Dual(1,1)
Dual{Nothing}(1,1)

julia> TaylorDiff.derivative(z -> z*y, x, one(x), Val{3}())
ERROR: MethodError: *(::TaylorScalar{ForwardDiff.Dual{Nothing, Int64, 1}, 3}, ::ForwardDiff.Dual{Nothing, Int64, 1}) is ambiguous.

Candidates:
  *(a::TaylorScalar, b::Number)
    @ TaylorDiff ~\.julia\packages\TaylorDiff\N6Pcf\src\primitive.jl:119
  *(x::Real, y::ForwardDiff.Dual{Ty}) where Ty
    @ ForwardDiff ~\.julia\packages\ForwardDiff\Or6Qh\src\dual.jl:149

Possible fix, define
  *(::TaylorScalar, ::ForwardDiff.Dual{Ty}) where Ty

Can we force the choice here between the two somehow, so the code goes through ? eg by defining

*(::TaylorScalar{ForwardDiff.Dual{Ty}}, ::ForwardDiff.Dual{Ty}) where Ty

?

[lrnv]

Happend once more in another context, where I first define a function through taylordiff and then try to differentiate through it, but did not work for the same ambiguity reasons. Has it something to do with the fact that you moved from Real to Numbers ?

MethodError: *(::ForwardDiff.Dual{ForwardDiff.Tag{…}, Float64, 1}, ::TaylorDiff.TaylorScalar{ForwardDiff.Dual{…}, 3}) is ambiguous.

Candidates:
  *(x::ForwardDiff.Dual{Tx}, y::Real) where Tx
    @ ForwardDiff ~\.julia\packages\ForwardDiff\Or6Qh\src\dual.jl:148
  *(a::Number, b::TaylorDiff.TaylorScalar)
    @ TaylorDiff ~\.julia\dev\TaylorDiff\src\primitive.jl:114

Possible fix, define
  *(::ForwardDiff.Dual{Tx}, ::TaylorDiff.TaylorScalar) where Tx

[tansongchen]

I agree this is problematic, but I'm not pretty sure what to do here. Both ForwardDiff and TaylorDiff works around a bundle type (Dual or TaylorScalar), and when binary operations are defined, one need to have a function working on (primal number type, bundle type) and (bundle type, primal number type). However, without knowing each other package, it is impossible to unambigiously dispatch (Dual, TaylorScalar) to either (Dual, number) or (number, TaylorScalar). The best thing we can do is to create an extension for that (so that two package know each other).

But then there is another question: if we have an extension to implement functions for (Dual, TaylorScalar), what should be in the function body? There is a well-known problem called perturbation confusion for type-based automatic differentiation. That is, one does not know how to proceed this computation, because it can be either "treating TaylorScalar as a number and using Dual's arithmetic" or "treating Dual as a number and using TaylorScalar's arithmetic". Which is correct depends on how you compose these two packages: if you are using TaylorDiff-on-ForwardDiff, then you should do the latter; if you are using ForwardDiff-on-TaylorDiff, then you should do the former.

Therefore, since we don't have a very general solution to this, you can define your own dispatch as you like to solve your specific problem. In your specific case, you are using TaylorDiff as the "outer AD", so the correct definition should be

using ForwardDiff, TaylorDiff
x = ForwardDiff.Dual(1, 1)
y = ForwardDiff.Dual(1, 1)

Base.:*(a::TaylorScalar, b::ForwardDiff.Dual{Ty}) where Ty = TaylorScalar(TaylorDiff.value(a) * b, TaylorDiff.partials(a) .* b)

TaylorDiff.derivative(z -> z*y, x, one(x), Val{3}()) # Dual{Nothing}(0,0)

but remember, this is only correct if you are using TaylorDiff as "outer AD". So it wouldn't be very appropriate to put this in the package.

[lrnv]

Thanks for the explanation. I am using TaylorDiff to define a function, and then my users want to ForwardDiff through it, so I guess the outer AD is ForwardDiff for me.

1. What would be the solution for using ForwardDiff as the outer AD ?
2. Isnt that type piracy ? I do not own TaylorScalar, Base.*, nor ForwardDiff.Dual
3. I think I'll need more than only Base.*...

Basically, I need a way to "Set TaylorDiff as Inner, ForwardDiff as outer" for one and only one specific method, this one: https://github.com/lrnv/WilliamsonTransforms.jl/blob/21de468ceb0b6ff91640025c8647d0cc07eb4d25/src/WilliamsonTransforms.jl#L68-L71C4

[tansongchen]

Ok so if this is just one function taylor in your package, we can have an overload of that with ForwardDiff.Dual to avoid piracy.

Please give me a minimal example that represents your whole workflow, and I'll make a pr to your WilliamsonTransforms.jl package to make this work.

[lrnv]

Something like that triggers the issue:

using WilliamsonTransforms: taylor
using ForwardDiff: derivative

f(a, x) = expm1(a * x^2) # This could be any function, try also with a + x or a/x for example. 

df(a, x, ::Val{p}) where p = taylor(y -> f(a, y), x, Val{p}())[end] * factorial(p) # pth derivative w.r.t. x

derivative(a -> df(a, 0.3, Val{4}()), 0.1) # derivative w.r.t. a, triggers the previous issue
derivative(x -> df(0.3, x, Val{4}()), 0.1) # This one goes through, is equal to df(0.3, 0.1, Val{5}())

Basically df defined here needs to behave like a normal function for ForwardDiff.

note: The first call triggers the previous issue, while the second goes through, but might be more efficiently remapped to taylor(y -> f(a, y), x, Val{6}()), which is strictly equivalent.