ForwardDiff cannot determine ordering of Dual tags for Laplace distributions

[ManojGopalkrishnan]

I asked about my problem here: https://discourse.julialang.org/t/forwarddiff-cannot-determine-ordering-of-dual-tags/40345 and was directed to open an issue at Turing.

Here’s my MWE

@model function mwe(::Type{T} = Vector{Float64}) where {T}

    lvar ~ Uniform(5.0,15.0)

    u ~ filldist(truncated(Laplace(55.0, lvar), 10, Inf),10) 

    return u

end

mwemodel = mwe()

chainmwe = sample(mwemodel, NUTS(.65),3)

The error is:

Cannot determine ordering of Dual tags ForwardDiff.Tag{Turing.Core.var"#f#7"{DynamicPPL.VarInfo{NamedTuple{(:lvar, :u),Tuple{DynamicPPL.Metadata{Dict{DynamicPPL.VarName{:lvar,Tuple{}},Int64},Array{Uniform{Float64},1},Array{DynamicPPL.VarName{:lvar,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{DynamicPPL.VarName{:u,Tuple{}},Int64},Array{Product{Continuous,Truncated{Laplace{Float64},Continuous,Float64},FillArrays.Fill{Truncated{Laplace{Float64},Continuous,Float64},1,Tuple{Base.OneTo{Int64}}}},1},Array{DynamicPPL.VarName{:u,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64},DynamicPPL.Model{var"###evaluator#333",(:T,),Tuple{Type{Array{Float64,1}}},(),DynamicPPL.ModelGen{var"###generator#334",(:T,),(:T,),Tuple{Type{Array{Float64,1}}}}},DynamicPPL.Sampler{NUTS{Turing.Core.ForwardDiffAD{40},(),AdvancedHMC.DiagEuclideanMetric},Turing.Inference.SamplerState{DynamicPPL.VarInfo{NamedTuple{(:lvar, :u),Tuple{DynamicPPL.Metadata{Dict{DynamicPPL.VarName{:lvar,Tuple{}},Int64},Array{Uniform{Float64},1},Array{DynamicPPL.VarName{:lvar,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}},DynamicPPL.Metadata{Dict{DynamicPPL.VarName{:u,Tuple{}},Int64},Array{Product{Continuous,Truncated{Laplace{Float64},Continuous,Float64},FillArrays.Fill{Truncated{Laplace{Float64},Continuous,Float64},1,Tuple{Base.OneTo{Int64}}}},1},Array{DynamicPPL.VarName{:u,Tuple{}},1},Array{Float64,1},Array{Set{DynamicPPL.Selector},1}}}},Float64}}}},Float64} and Nothing
...

and goes on for more characters than allowed in the body here. Not able to figure this out! Why isn’t this working?

[devmotion]

You can avoid the issue by defining your model as

@model function mwe()
    lvar ~ Uniform(5.0,15.0)
    T = typeof(lvar)
    u ~ filldist(truncated(Laplace(55.0, lvar), T(10.0), T(Inf)), 10)
    return u
end

(Note that there is no reason to specify the input argument ::Type{T} since you have access to the type of lvar inside the function definition, and you didn't actually use T in your original version.) The underlying problem is the comparison in https://github.com/JuliaStats/Distributions.jl/blob/feae3ba9ada360d2980c901e7c737ca02bda7a31/src/truncate.jl#L116. Truncated normal distributions are handled differently and don't end up there, but for Laplace distributions Distributions falls back to Truncated. The comparison in this line errors if d.lower and d.upper are regular numbers but x is a dual number (i.e., lvar in your case). So the issue should be fixed properly upstream in Distributions, but hopefully for the time being the fix is helpful.

[devmotion]

I think the proper fix upstream is just adding

Base.eltype(::Type{Laplace{T}}) where {T} = T

Then https://github.com/JuliaStats/Distributions.jl/blob/feae3ba9ada360d2980c901e7c737ca02bda7a31/src/truncate.jl#L25 should guarantee that the lower and upper bound of the truncated distribution are also dual numbers (as in the manual fix mentioned above). I'll quickly check if my intuition is correct.

[devmotion]

I can confirm that defining eltype of Laplace fixes the problem. However, I think one should maybe just always promote the provided inputs and bounds in the implementation of Truncated instead. In general, it seems a bit unclear what eltype in Distributions refers to, it's missing for many distributions (so many more examples would still fail), it's mixed with partype, and it would still lead to problems if x (in this line) is a Dual number but the element type of the underlying distribution (in contrast to the example here) and the bounds were just regular numbers.

[ManojGopalkrishnan]

Thanks for the replies! I could get around the problem by typecasting the limits to truncated as suggested.