Overdubbing things using llvmcall fail

[oxinabox]

@KristofferC found this fun bug in MagneticReadHead
oxinabox/MagneticReadHead.jl#60

I have mare a reproducer that does not have MRH involved.
So I think it is a Cassette problem.

This reproduces in 0.2.4 and current master.

MWE:

using Cassette, Tensors

Cassette.@context Ctx

# This overdub does nothing
function Cassette.overdub(ctx::Ctx, f, args...)
    if Cassette.canrecurse(ctx, f, args...)
        Cassette.recurse(ctx, f, args...)

    else 
        Cassette.fallback(ctx, f, args...)
    end
end

t = rand(Tensor{2,3})

Cassette.@overdub Ctx() t+t

Output:

ERROR: this intrinsic must be compiled to be called
Stacktrace:
 [1] call(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::Function, ::Tuple{String,String}, ::Type, ::Type, ::Vararg{Any,N} where N) at /Users/oxinabox/.julia/packages/Cassette/qs8Qq/src/context.jl:447
 [2] fallback(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::Function, ::Vararg{Any,N} where N) at /Users/oxinabox/.julia/packages/Cassette/qs8Qq/src/context.jl:445
 [3] overdub(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::Function, :
:Tuple{String,String}, ::Type, ::Type, ::NTuple{9,VecElement{Float64}}, ::Vararg{NTuple{9,VecElement{Float64}},N} where N) at ./REPL[19]:6
 [4] macro expansion at /Users/oxinabox/.julia/packages/SIMD/nqizO/src/SIMD.jl:602 [inlined]
 [5] llvmwrap at /Users/oxinabox/.julia/packages/SIMD/nqizO/src/SIMD.jl:583 [inlined]
 [6] recurse(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::typeof(SIMD
.llvmwrap), ::Type{Val{:+}}, ::SIMD.Vec{9,Float64}, ::SIMD.Vec{9,Float64}, ::Type{Float64}) at /Users/oxinabox/.julia/packages/Cassette/qs8Qq/src/overdub.jl:0
 [7] overdub(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::Function, :
:Type, ::SIMD.Vec{9,Float64}, ::SIMD.Vec{9,Float64}, ::Type) at ./REPL[19]:3
 [8] llvmwrap at /Users/oxinabox/.julia/packages/SIMD/nqizO/src/SIMD.jl:583 [inlined]
 [9] overdub at ./REPL[19]:3 [inlined]
 [10] + at /Users/oxinabox/.julia/packages/SIMD/nqizO/src/SIMD.jl:1057 [inlined]
 [11] overdub at ./REPL[19]:3 [inlined]
 [12] + at /Users/oxinabox/.julia/packages/Tensors/hQW1A/src/simd.jl:98 [inlined]
 [13] recurse(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::typeof(+),
 ::Tensor{2,3,Float64,9}, ::Tensor{2,3,Float64,9}) at /Users/oxinabox/.julia/packages/Cassette/qs8Qq/src/overdub.jl:0
 [14] overdub(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::Function,
::Tensor{2,3,Float64,9}, ::Tensor{2,3,Float64,9}) at ./REPL[19]:3
 [15] recurse(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::getfield(M
ain, Symbol("##38#39"))) at /Users/oxinabox/.julia/packages/Cassette/qs8Qq/src/overdub.jl:677
 [16] top-level scope at none:0

[oxinabox]

I have made a smaller MWE, with no libraries except Cassette and Core.

MWE:

using Cassette

Cassette.@context Ctx

# This overdub does nothing
function Cassette.overdub(ctx::Ctx, f, args...)
    if Cassette.canrecurse(ctx, f, args...)
        Cassette.recurse(ctx, f, args...)

    else 
        Cassette.fallback(ctx, f, args...)
    end
end


function llvm_sin(x::Float64)
    Core.Intrinsics.llvmcall(
        (
            """declare double @llvm.sin.f64(double)""",
            """%2 = call double @llvm.sin.f64(double %0)
               ret double %2"""
        ),
        Float64, Tuple{Float64}, x
    )
end

Cassette.@overdub Ctx() llvm_sin(4.0)

Output

julia> llvm_sin(4.0)
-0.7568024953079282

julia> Cassette.@overdub Ctx() llvm_sin(4.0)
ERROR: this intrinsic must be compiled to be called
Stacktrace:
 [1] call at /Users/oxinabox/.julia/packages/Cassette/qs8Qq/src/context.jl:447 [inlined]
 [2] fallback at /Users/oxinabox/.julia/packages/Cassette/qs8Qq/src/context.jl:445 [inlined]
 [3] overdub(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::Function, ::Tuple{String,String}, ::Type, ::Type, ::Float64) at ./REPL[3]:7
 [4] llvm_sin at ./REPL[6]:2 [inlined]
 [5] overdub at ./REPL[3]:4 [inlined]
 [6] recurse(::Cassette.Context{nametype(Ctx),Nothing,Nothing,getfield(Cassette, Symbol("##PassType#365")),Nothing,Nothing}, ::getfield(Main, Symbol("##10#11"))) at /Users/oxinabox/.julia/packages/Cassette/qs8Qq/src/overdub.jl:677
 [7] top-level scope at none:0

[vchuravy]

Note from my brief investigation this morning. @inline on the overdub makes this go away.

[oxinabox]

Fun:
In julia 1.3, all print statements are protected by locks
which use Atomics,
which are written using LLVM call.

Which makes Cassette unusable in jula 1.3 on a lot of code bases.

[vchuravy]

We could just blacklist the atomic functions until I figure out how do deal with the llvmcall in general

[carstenbauer]

Just hit this. Any update/news?