GeneralizedGenerated with @tullio Macro

[roflmaostc]

Hey,

I wanted to use GeneralizedGenerated together with Tullio.jl.
I was directed to your Package via this post.

So my minimal failing example:

using Tullio
using GeneratedGeneralized
r = mk_function(Meta.parse("arr -> (@tullio r = (arr[i] - arr[i + 1])^2)"))

returns

ERROR: LoadError: UndefVarError: @tullio not defined
Stacktrace:
 [1] top-level scope
 [2] #macroexpand#36 at ./expr.jl:108 [inlined]
 [3] macroexpand at ./expr.jl:107 [inlined]
 [4] mk_function(::Module, ::Expr) at /home/fxw/.julia/packages/GeneralizedGenerated/hIoV7/src/GeneralizedGenerated.jl:20
 [5] mk_function(::Expr) at /home/fxw/.julia/packages/GeneralizedGenerated/hIoV7/src/GeneralizedGenerated.jl:16
 [6] top-level scope at REPL[5]:1
 [7] run_repl(::REPL.AbstractREPL, ::Any) at /build/julia/src/julia-1.5.2/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
in expression starting at none:1

So it looks like that it didn't find the package. To fix that, I built a local version of your package with included Tullio (using Tullio added to the top).
However, the stack trace is even worse:

ERROR: GG doesn't support type parameters or return type annotations.
Stacktrace:
 [1] (::GeneralizedGenerated.var"#conv#6"{Module})(::Expr) at /tmp/GeneralizedGenerated.jl/src/closure_conv.jl:42
 [2] iterate at ./generator.jl:47 [inlined]
 [3] _collect(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Base.EltypeUnknown, ::Base.HasShape{1}) at ./array.jl:699
 [4] collect_similar(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}) at ./array.jl:628
 [5] map(::Function, ::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}) at ./abstractarray.jl:2162
 [6] (::GeneralizedGenerated.var"#conv#6"{Module})(::Expr) at /tmp/GeneralizedGenerated.jl/src/closure_conv.jl:64
 [7] iterate at ./generator.jl:47 [inlined]
 [8] collect_to!(::Array{LineNumberNode,1}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Int64, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:732
 [9] collect_to_with_first!(::Array{LineNumberNode,1}, ::LineNumberNode, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:710
 [10] _collect(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Base.EltypeUnknown, ::Base.HasShape{1}) at ./array.jl:704
 [11] collect_similar(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}) at ./array.jl:628
 [12] map(::Function, ::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}) at ./abstractarray.jl:2162
 [13] (::GeneralizedGenerated.var"#conv#6"{Module})(::Expr) at /tmp/GeneralizedGenerated.jl/src/closure_conv.jl:64
 [14] iterate at ./generator.jl:47 [inlined]
 [15] collect_to!(::Array{Any,1}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Int64, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:732
 [16] collect_to!(::Array{LineNumberNode,1}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Int64, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:740
 [17] collect_to_with_first!(::Array{LineNumberNode,1}, ::LineNumberNode, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:710
 [18] _collect(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Base.EltypeUnknown, ::Base.HasShape{1}) at ./array.jl:704
 [19] collect_similar(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}) at ./array.jl:628
 [20] map(::Function, ::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}) at ./abstractarray.jl:2162
 [21] (::GeneralizedGenerated.var"#conv#6"{Module})(::Expr) at /tmp/GeneralizedGenerated.jl/src/closure_conv.jl:64
 [22] iterate at ./generator.jl:47 [inlined]
 [23] collect_to!(::Array{Symbol,1}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Int64, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:732
 [24] collect_to_with_first!(::Array{Symbol,1}, ::Symbol, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:710
 [25] _collect(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Base.EltypeUnknown, ::Base.HasShape{1}) at ./array.jl:704
 [26] collect_similar(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}) at ./array.jl:628
 [27] map(::Function, ::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}) at ./abstractarray.jl:2162
 [28] (::GeneralizedGenerated.var"#conv#6"{Module})(::Expr) at /tmp/GeneralizedGenerated.jl/src/closure_conv.jl:64
 [29] iterate at ./generator.jl:47 [inlined]
 [30] collect_to!(::Array{LineNumberNode,1}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Int64, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:732
 [31] collect_to_with_first!(::Array{LineNumberNode,1}, ::LineNumberNode, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Tuple{Base.OneTo{Int64},Int64}) at ./array.jl:710
 [32] _collect(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}, ::Base.EltypeUnknown, ::Base.HasShape{1}) at ./array.jl:704
 [33] collect_similar(::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}, ::Base.Generator{SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true},GeneralizedGenerated.var"#conv#6"{Module}}) at ./array.jl:628
 [34] map(::Function, ::SubArray{Any,1,Array{Any,1},Tuple{UnitRange{Int64}},true}) at ./abstractarray.jl:2162
 [35] (::GeneralizedGenerated.var"#conv#6"{Module})(::Expr) at /tmp/GeneralizedGenerated.jl/src/closure_conv.jl:64
 [36] (::GeneralizedGenerated.var"#conv#6"{Module})(::Expr) at /tmp/GeneralizedGenerated.jl/src/closure_conv.jl:25
 [37] (::GeneralizedGenerated.var"#conv#6"{Module})(::Expr) at /tmp/GeneralizedGenerated.jl/src/closure_conv.jl:47
 [38] closure_conv(::Module, ::Expr) at /tmp/GeneralizedGenerated.jl/src/closure_conv.jl:78
 [39] mk_function(::Module, ::Expr) at /tmp/GeneralizedGenerated.jl/src/GeneralizedGenerated.jl:25
 [40] mk_function(::Expr) at /tmp/GeneralizedGenerated.jl/src/GeneralizedGenerated.jl:18
 [41] top-level scope at REPL[24]:1
 [42] run_repl(::REPL.AbstractREPL, ::Any) at /build/julia/src/julia-1.5.2/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:2

I'm sorry to bother you with that, since I'm not even sure if it's an issue by your package or if I'm misusing it.

Thanks a lot,

Felix

[thautwarm]

Hello Felix,

You can specify the module(global scope) of your generated code by

using Tullio
using GeneralizedGenerated
r = mk_function(Tullio, Meta.parse("arr -> (@tullio r = (arr[i] - arr[i + 1])^2)"))

However so far you'll get

ERROR: GG doesn't support type parameters or return type annotations.

because GG cannot handle multiple dispatch functions in runtime generated code, and @tullio generates

local function ���!(::Type, ℛ::�, arr, ��i, ♻️ = nothing, � = true) where �
   ...
end

In order to leverage GG, you might choose to generate

local ���! = function(_1, ℛ, arr, ��i, ♻️ = nothing, � = true) where �
   _1::Type
   ℛ::�
end

---

Best,
-- Taine

[thautwarm]

Some of my suggestions here: https://discourse.julialang.org/t/dynamically-create-a-function-initial-idea-with-eval-failed-due-to-world-age-issue/49139/9

For libraries generating code based on information of large-scale data, I would recommend using RuntimeGeneratedFunctions, see issues #45 for the reason.