serialization of Functions chooses the wrong module for scoping

[timholy]

I've got a situation where I have a "base" module (here called Shell) that defines a general API, and several "specialized" versions (here called Instance1 and Instance2) which specialize the API for types defined in their respective modules. It seems that functions defined in Shell get serialized by scoping with respect to Instance1 rather than Shell. This interacts quite badly with JuliaLang/Distributed.jl#17, in which different workers added at different times may not have access to the same modules---in my situation, all processes know about Shell but some workers know about Instance1 while other workers know about Instance2.

Demo:

julia> module Shell

       export foo

       foo(args) = error("Instances must implement foo")

       end
Shell

julia> using Shell

julia> io = IOBuffer()
IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1)

julia> serialize(io, foo)

julia> takebuf_string(io)
"\x13\x02#\x02\x05Shell/\x02\x03foo"

julia> module Instance1

       using Shell
       import Shell: foo

       foo(x::Int) = "This is an Int"

       end
Instance1

julia> using Instance1

julia> serialize(io, foo)

julia> takebuf_string(io)
"\x13\x02#\x02\tInstance1/\x02\x03foo"

julia> module Instance2

       using Shell
       import Shell: foo

       foo(x::Float64) = "This is a Float64"

       end
Instance2

julia> using Instance2

julia> serialize(io, foo)

julia> takebuf_string(io)
"\x13\x02#\x02\tInstance1/\x02\x03foo"

julia> serialize(io, Shell.foo)

julia> takebuf_string(io)
"\x13\x02#\x02\tInstance1/\x02\x03foo"

Here's why this happens and what we might do about it:

julia> foo.env.defs.func.code.module
Instance1

julia> for m in methods(foo)
           @show m.func.code.module
       end
m.func.code.module = Instance1
m.func.code.module = Instance2
m.func.code.module = Shell

But I'm uncertain how one decides to choose Shell among these options; is there some place we can look to decide that Instance1 and Instance2 both require Shell?

[timholy]

I should clarify that the reason I said this interacts with JuliaLang/Distributed.jl#17 is that it arises naturally in testing situations like this:

File runtests.jl:

include("instance1.jl")
include("instance2.jl")

where each instance looks like

wpids = addprocs(2)
using Stuff, Instance1
# Work around JuliaLang/Distributed.jl#17
@sync for p in wpids
     @spawnat p eval(:(using Stuff, Instance1))
end

# do stuff

rmprocs(wpids)

Each test file works fine on its own, but not in combination. But the way to fix this is not through fixing JuliaLang/Distributed.jl#17, obviously.