Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix show for MethodList when methods are from another module #52354

Merged
merged 2 commits into from
Dec 22, 2023

Conversation

ararslan
Copy link
Member

When a type is defined in one module but its constructor's methods are defined elsewhere, show_method_table errors due to an incorrect lookup of the defining module used to determine colors for printing. In particular, the code had been assuming that the type is defined (in the sense of isdefined) in the module in which its constructor's first method (in the sense of first(methods())) is defined, which isn't always true.

To fix this, we can look through the available methods and choose the first in which the type is defined in the method's module, falling back to the method table's module otherwise. I'm not sure that this is necessarily the best way to fix the issue but it does fix it.

Fixes #49382
Fixes #49403
Fixes #52043

@ararslan ararslan added kind:bugfix This change fixes an existing bug domain:display and printing Aesthetics and correctness of printed representations of objects. backport 1.9 Change should be backported to release-1.9 backport 1.10 Change should be backported to the 1.10 release labels Nov 30, 2023
@ararslan ararslan added the status:awaiting review PR is complete and seems ready to merge. Has tests and news/compat if needed. CI failures unrelated. label Dec 11, 2023
@KristofferC KristofferC mentioned this pull request Dec 12, 2023
17 tasks
@ararslan ararslan mentioned this pull request Dec 20, 2023
Comment on lines 304 to 305
i = findfirst(m -> isdefined(m.module, m.name), ms.ms)
i === nothing ? mt.module : which(ms.ms[i].module, ms.ms[i].name)
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
i = findfirst(m -> isdefined(m.module, m.name), ms.ms)
i === nothing ? mt.module : which(ms.ms[i].module, ms.ms[i].name)
sig = Base.unwrap_unionall(ms[1].sig)::DataType # Tuple{Type{...}}
((sig.parameters[1]::DataType).parameters[1]::DataType).name.module

This seems very hacky, but would be more correct?

Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think most of those type asserts are invalid, because the general assumption here (that method tables are owned by modules) is invalid itself

Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking over the code a bit closer, it looks like it wants to check it the method itself is being defined on a function / type in the same module or in a dependency. But that needs to be a per-method query, and is not valid to derive from the method table

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the difficulty is defining what "same module" means here since you need to select a starting point for comparison in order to cycle the colors. @aviatesk's suggestion does seem better than what I've implemented, except it uses ms[1] as on current master, the defining module of which isn't necessarily meaningful. But it at least shouldn't error since it isn't using which as on master.

Copy link
Member Author

@ararslan ararslan Dec 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Combining part of @aviatesk's suggestion with part of what's currently implemented in this PR:

Suggested change
i = findfirst(m -> isdefined(m.module, m.name), ms.ms)
i === nothing ? mt.module : which(ms.ms[i].module, ms.ms[i].name)
i = something(findfirst(m -> isdefined(parentmodule(m), m.name), ms), 1)
parentmodule(fieldtype(unwrap_unionall(ms[i].sig), 1).parameters[1])

Does this seem reasonable, @vtjnash?

Copy link
Sponsor Member

@vtjnash vtjnash Dec 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a bit better. The more generally correct version of this needs to be done for each method however, first attempting to derive the method table it is being added to, then deriving if it is the same as the method itself. Something like this:

mmt = get_methodtable(meth)
modulecolor = if mmt.module === parentmodule(meth)
        nothing
    else
        # mmt is only particularly relevant for external method tables. Since the primary method table is shared, we now need to distinguish "primary" methods by trying to check if there is a primary DataType to identify it with
        # c.f. how `jl_method_def` would derive this same information (for the name)
        ft = argument_datatype((unwrap_unionall(meth.sig)::DataType).parameters[1]) # try to get the type associated with the first argument
        isType(ft) && (ft = argument_datatype(ft.parameters[1])) # if the ft is `Type`, try to unwrap it again
        if ft === nothing || parentmodule(meth) === parentmodule(ft) !== Core
            nothing
        else
            # get color for meth
        end
end
...

n.b. In C, the first ft computation is called jl_nth_argument_datatype, but it seems we don't export that to Julia

When a type is defined in one module but its methods are defined
elsewhere, `show_method_table` errors due to an incorrect lookup of the
defining module used to determine colors for printing. In particular,
the code had been assuming that the type is defined in the module in
which its constructor's first method (in the sense of
`first(methods())`) is defined, which isn't always true.

The color used for printing the module name needs to be determined on a
per-method basis and can't be correctly done based on the method table's
module. For each method, we attempt to derive the module for the method
table to which the method was added, then determine whether it's the
same as the defining module for the method.

Fixes #49382
Fixes #49403
Fixes #52043

Co-Authored-By: Jameson Nash <vtjnash@gmail.com>
Copy link
Sponsor Member

@aviatesk aviatesk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I've looked into the discussion and it seems reasonable to me.

Copy link
Sponsor Member

@vtjnash vtjnash left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why would the method table lookup ever return nothing?

@ararslan
Copy link
Member Author

Why would the method table lookup ever return nothing?

I wondered the same, turns out it does so for builtins. This is actually handled explicitly in a couple of places, e.g.:

mt = ccall(:jl_method_get_table, Any, (Any,), method)
mt === nothing && return nothing

Method table lookup uses the first item of the method's signature, but the method for a builtin has signature Tuple, which causes nth_methodtable to return nothing.

@ararslan ararslan merged commit 40bc64c into master Dec 22, 2023
7 checks passed
@ararslan ararslan deleted the aa/fix-methodlist-show branch December 22, 2023 20:21
ararslan added a commit that referenced this pull request Dec 22, 2023
)

When a type is defined in one module but its methods are defined
elsewhere, `show_method_table` errors due to an incorrect lookup of the
defining module used to determine colors for printing. In particular,
the code had been assuming that the type is defined in the module in
which its constructor's first method (in the sense of
`first(methods())`) is defined, which isn't always true.

The color used for printing the module name needs to be determined on a
per-method basis and can't be correctly done based on the method table's
module. For each method, we attempt to derive the module for the method
table to which the method was added, then determine whether it's the
same as the defining module for the method.

Fixes #49382
Fixes #49403
Fixes #52043

Co-Authored-By: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit 40bc64c)
KristofferC pushed a commit that referenced this pull request Dec 23, 2023
)

When a type is defined in one module but its methods are defined
elsewhere, `show_method_table` errors due to an incorrect lookup of the
defining module used to determine colors for printing. In particular,
the code had been assuming that the type is defined in the module in
which its constructor's first method (in the sense of
`first(methods())`) is defined, which isn't always true.

The color used for printing the module name needs to be determined on a
per-method basis and can't be correctly done based on the method table's
module. For each method, we attempt to derive the module for the method
table to which the method was added, then determine whether it's the
same as the defining module for the method.

Fixes #49382
Fixes #49403
Fixes #52043

Co-Authored-By: Jameson Nash <vtjnash@gmail.com>
(cherry picked from commit 40bc64c)
@aviatesk aviatesk removed the backport 1.10 Change should be backported to the 1.10 release label Dec 24, 2023
@inkydragon inkydragon removed the status:awaiting review PR is complete and seems ready to merge. Has tests and news/compat if needed. CI failures unrelated. label Feb 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 1.9 Change should be backported to release-1.9 domain:display and printing Aesthetics and correctness of printed representations of objects. kind:bugfix This change fixes an existing bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

methods(Core.OpaqueClosure) errors show_method_table error Error when showing method list
4 participants