add morespecific rule for Type{Union{}} #49349
Merged
+369
−157
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KristofferC
added
the
needs pkgeval
Removes some overly strict `@test_throws MethodError` for calls with
`Union{}` or `Any`, in preparation for making those errors more precise.
Make Type{Union{}} in method definitions always the most specific type
(normally these would end up being ambiguous). This ensures we do not
invalidate them, nor need to consider ambiguities that might arise from
intersections with them.
Based on the new morespecific rule for Union{} and method definitions of
the specific form `f(..., Type{Union{}}, Vararg)`. If a method
definition exists with that specific form, the intersection visitor will
ignore all intersections that have that as their only result, saving
significant effort when working with lookups involving `Type{<:T}`
(which usually ended up mostly ambiguous anyways).
Fixes: #33780
This pattern turns out to have still to been making package loading
slow. We could keep adding methods following the ambiguity pattern
#46000 for the couple specific
functions that need it (constructor, eltype, IteratorEltype,
IteratorSize, and maybe a couple others) so the internals can detect
those and optimize functions that have that method pair. But it seems
somewhat odd, convoluted, and non-obvious behavior there. Instead, this
breaks all ambiguities in which Union{} is present explicitly in favor
of the method with Union{}. This means that when computing method
matches, as soon as we see one method definition with Union{}, we can
record that the method is the only possible match for that slot.
This, in essence, permits creating a rule for dispatch that a TypeVar
lower bound must be strictly a supertype of Union{}, but this creates it
at the function level, instead of expecting the user to add it to every
TypeVar they use to define methods.
This also lets us improve the error message for these cases (generally
they should error to avoid polluting the inference result), since we can
be assured this method will be called, and not result in an ambiguous
MethodError instead!
Reverts the functional change of #46000
Since T cannot be Union{} here, the prior optimization would not get
detected post facto, but a priori this cannot be inhabited for
T=Union{}, so we can exclude it immediately. This does not happen during
inference, but shows up during edge validation somewhat often.
vtjnash
force-pushed
the
jn/morespecific-bottom
branch
from
d1a2c9e
to
285c770
Compare
vtjnash
added a commit
that referenced
this pull request
Apr 19, 2023
Even if we have a new method that is more specific than the method it is
replacing, there still might exist an existing method that is more
specific than both which already covers their intersection.
An example of this pattern is adding
Base.IteratorSize(::Type{<:NewType})
causing invalidations on
Base.IteratorSize(::Type)
for specializations such as
Base.IteratorSize(::Type{<:AbstractString})
even though the intersection of these is fully covered already by
Base.IteratorSize(::Type{Union{}})
so our new method would never be selected there.
This won't detect ambiguities that already cover this intersection, but
that is why we are looking to move away from that pattern towards
explicit methods for detection in closer to O(n) instead of O(n^2): #49349.
Similarly, for this method, we were unnecessarily dropping it from the
MethodTable cache. This is not a significant latency problem (the cache
is cheap to rebuild), but it is also easy to avoid in the first place.
Refs #49350
|
@nanosoldier |
|
Your package evaluation job has completed - possible new issues were detected. |
vtjnash
removed
the
needs pkgeval
vtjnash
added a commit
that referenced
this pull request
Apr 20, 2023
Even if we have a new method that is more specific than the method it is
replacing, there still might exist an existing method that is more
specific than both which already covers their intersection.
An example of this pattern is adding
Base.IteratorSize(::Type{<:NewType})
causing invalidations on
Base.IteratorSize(::Type)
for specializations such as
Base.IteratorSize(::Type{<:AbstractString})
even though the intersection of these is fully covered already by
Base.IteratorSize(::Type{Union{}})
so our new method would never be selected there.
This won't detect ambiguities that already cover this intersection, but
that is why we are looking to move away from that pattern towards
explicit methods for detection in closer to O(n) instead of O(n^2): #49349.
Similarly, for this method, we were unnecessarily dropping it from the
MethodTable cache. This is not a significant latency problem (the cache
is cheap to rebuild), but it is also easy to avoid in the first place.
Refs #49350
|
This PR appears to fail analyzegc. |
andreasnoack
added a commit
that referenced
this pull request
Feb 5, 2024
The link to #49349 was missing.
ViralBShah
pushed a commit
that referenced
this pull request
Feb 5, 2024
The link to #49349 was missing.
Drvi
pushed a commit
to RelationalAI/julia
that referenced
this pull request
Jun 7, 2024
The link to JuliaLang#49349 was missing.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Make Type{Union{}} in method definitions always the most specific type
(normally these would end up being ambiguous). This ensures we do not
invalidate them, nor need to consider ambiguities that might arise from
intersections with them.
Based on the new morespecific rule for Union{} and method definitions of
the specific form
f(..., Type{Union{}}, Vararg). If a methoddefinition exists with that specific form, the intersection visitor will
ignore all intersections that have that as their only result, saving
significant effort when working with lookups involving
Type{<:T}(which usually ended up mostly ambiguous anyways).
Fixes: #33780
This pattern turns out to have still to been making package loading
slow. We could keep adding methods following the ambiguity pattern
#46000 for the couple specific
functions that need it (constructor, eltype, IteratorEltype,
IteratorSize, and maybe a couple others) so the internals can detect
those and optimize functions that have that method pair. But it seems
somewhat odd, convoluted, and non-obvious behavior there. Instead, this
breaks all ambiguities in which Union{} is present explicitly in favor
of the method with Union{}. This means that when computing method
matches, as soon as we see one method definition with Union{}, we can
record that the method is the only possible match for that slot.
This, in essence, permits creating a rule for dispatch that a TypeVar
lower bound must be strictly a supertype of Union{}, but this creates it
at the function level, instead of expecting the user to add it to every
TypeVar they use to define methods.
This also lets us improve the error message for these cases (generally
they should error to avoid polluting the inference result), since we can
be assured this method will be called, and not result in an ambiguous
MethodError instead!
Reverts the functional change of #46000
The list of methods to add here was mainly autogenerated by Test.detect_ambiguities.
On loading OmniPackage, I see this give a few seconds savings. Roughly 26s -> 23s (>10% faster)