Finding macro-defined methods

[timholy]

Methods have a couple of fields, .file and .line, that document their origin. But when a method is defined via a macro, it gets a little ambiguous: do you attribute the definition to the macro or to the caller of the macro? There's nothing really new here: the various complaints about figuring out where a @deprecate ... command was issued from have a long and storied history. Perhaps the most important thing here is that there is a viable (but insane) strategy for finding it, see below.

Let's consider the following example:

julia> m = @which Float32(π)
Float32(::Irrational{:π}) in Base.MathConstants at irrationals.jl:167

There are a couple of interesting features here:

• m.module is Base.MathConstants, which is the caller
• m.file is irrationals.jl, the file in which the @irrational macro is defined. This is loaded into Base, not Base.MathConstants. m.line is likewise associated with the macro.
• Figuring out the macro-caller's source location is tricky. For example,

julia> code = Base.uncompressed_ast(m)
CodeInfo(
1 ─     return 3.1415927f0
)

julia> code.linetable
1-element Array{Any,1}:
 Core.LineInfoNode(Base.MathConstants, :Type, Symbol("irrationals.jl"), 167, 0)

so again there doesn't seem to be any way of figuring out the caller's file.

I've found one viable strategy to find these methods:

• Figure out which specific macro was responsible for the definition. For example, here that means parsing irrationals.jl and looking to see which top-level expression contains line 167.
• Grab m.module's eval or include method and ask which file it's from. This is the file in which the module was created.
• Parse that file and see if the module contains any additional include directives, so that you get a complete list of all files that were used to define that module.
• Then for each file:
  • parse the file and look for :macrocall expressions to that particular macro
  • lower those calls and compute the method signature
  • when you find a match, return the caller LineNumberNode.

Obviously this is not a small amount of work; it would make life easier if Method contained enough information to figure this out more directly. To achieve that, it seems that CodeInfo needs some enhancements so that it can store the macro-caller's LineNumberNode. For example, allowing Vector{LineInfoNode} entries in code.linetable would be sufficient to document the origin. (Or make LineInfoNode a linked-list?)

[yha]

I find that I run into this issue a lot, and usually give up on finding out where the method is defined.
I don't know much about the internals here, but can't method objects' file and line fields simply be changed to refer to the topmost macro caller, rather than the innermost macro line (would that be a breaking change)? Going down the macro expansion stack is much easier than going up: just a series of @which calls. So changing file and line to refer to the top of the stack makes the entire stack available.

EDIT: I've created a package (https://github.com/yha/FindDefinition.jl) for finding the definition of a macro-defined method, using roughly the method @timholy outlined above. One difference is that I expand all macros in the module to look for the definition, in order to cover cases where it's not syntactically inside the expanded macro (e.g. through a function or another macro).