-
-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NTuples made me sad (so I nixed them) #11242
Conversation
@@ -1583,11 +1582,197 @@ DLLEXPORT void jl_(void *jl_value) | |||
in_jl_--; | |||
} | |||
|
|||
// Useful because the jl_typeof macro isn't available from debugger |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i found out today that if you type macro define jl_typeof jl_typeof
then it works in gdb (it already works in lldb).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice tip, thanks!
Oh my. Nice work, impressive dedication. Travis appears to be failing an assertion, did you run with julia-debug locally?
|
Do you think this is feasible for 0.4 in contrast to the "type system design iteration" which is tagged 0.5? |
@tkelman, lots of time with the debug version, but I didn't run the whole test suite that way. Thanks for flagging this, and of course @garrison and @vtjnash for #11155. Should be fixed now, but we'll see what CI says. @mschauer, there's an important distinction between the two: this is done, and that one isn't 😄. This should be very useful for #10525 (the whole reason I started this, oh-so-long-ago), if the codegen bugs there can be worked out. |
@mschauer, I should also say that this probably makes the "type system design iteration" easier. This gets rid of a whole class of objects that needed special treatment in our type system, and it turns out we weren't always being consistent about how that class was treated. |
@timholy I was asking because I was hoping so. ;-) |
To avoid overwhelming Jeff with requests, I had planned on waiting a while before bumping this. But this probably deserves at least a brief glance before making decisions about what makes it in to 0.4, and it sounds as if final decisions about 0.4 could happen soon. The main motivation for this is #10525, and if that doesn't get merged then this is not pressing and could wait for Arraymagedon. But it has advantages of its own. Of course, it could also break stuff (and almost inevitably will, despite strenuous efforts to the contrary). |
There's a little bit of dispatch wonkiness that I uncovered with #11547: julia> f{N}(::Type{NTuple{N}}) = N
f (generic function with 1 method)
julia> f(Tuple{})
ERROR: MethodError: `f` has no method matching f(::Type{Tuple{}})
julia> f(Tuple{Int,})
1
julia> g(::Type{NTuple{0}}) = 0
g (generic function with 1 method)
julia> g(Tuple{})
0
julia> NTuple{0}
Tuple{}
julia> Tuple{} <: NTuple{0}
true
julia> isa(Tuple{}, Type{NTuple{0}})
true (Also, I resolved a few simple conflicts with master and figured I'd push it on the off-chance that it'd save someone else the work. Or you can totally ignore it and force-push over it. Edit: whoops, looks like my master was a bit behind… there are still newer conflicts. Oh well.) |
Awesome work. I'll start looking at this seriously, since it would be best to merge it before tackling #11320. With this, it seems more pressing to consider renaming |
Yes, you're right that the name is no longer appropriate. I'd vote for @mbauman, thanks for kicking the tires here, and for the start on conflict resolution. I'll see about doing a rebase, just to keep the size down (which might nix, sorry, your conflict resolution changes, since I think you did those as a merge commit). |
@@ -1433,8 +1440,12 @@ Too numerous to mention. | |||
[#10994]: https://github.com/JuliaLang/julia/issues/10994 | |||
[#11105]: https://github.com/JuliaLang/julia/issues/11105 | |||
[#11145]: https://github.com/JuliaLang/julia/issues/11145 | |||
<<<<<<< HEAD |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops indeed. Fixed.
The PR was hard to review as a merge with master (because of all the extra changes), so I rebased it. BUT I had to disable two tests, see the final commit (one of these was passing when I first submitted this PR, the other of which is a "new-style" This does not yet change the name from |
Yeah, I wish git/github did better with Merge commits. In some senses they're nice since they don't mess with history, but they're really hard to deal with. I take no exception here :). |
👍 to getting this in, 👍 for |
@mbauman, I fixed your "wonkiness" above. It turns out this was due to the one case I listed above where I thought master made more sense than this PR. Now in all cases of disagreement, this seems more consistent (to me). |
Usually, spelling things out – i.e. |
I prefer |
On current master, this gave true but Type{Tuple{}} <: Type{Tuple{Vararg}} gave false. Now they both give true.
c.f. "Instantiate Tuple{Vararg{Int,3}} as Tuple{Int,Int,Int}"
Method sorting question: is the precedence here intentional? It feels like it might be ambiguous… or maybe even more natural to be reversed: julia> f{T,N}(::Array{T,N}, ::Vararg{Int, N}) = 1
f (generic function with 1 method)
julia> f(::Array, ::Int) = 2
f (generic function with 2 methods)
julia> f([1,2,3], 1)
2
julia> methods(f)
# 2 methods for generic function "f":
f(::Array{T<:Any,N<:Any}, ::Int64) at REPL[10]:1
f{T,N}(::Array{T,N}, ::Vararg{Int64,N}) at REPL[9]:1 It'd be slightly easier to define the indexing methods were the order here reversed — I'd like to reshape the array to a vector in method |
I think this is a missing ambiguity since both of them can match arguments that the other one cannot. |
While our ambiguity detection isn't perfect now, it should mean "two non-identical signatures, neither of which is consistently more specific than the other over the domain of their intersection." Using that definition, this is not an ambiguous case: the intersection of these signatures is equivalent to When passed types that do not fall in this intersection, only one of the two methods applies, and therefore such cases cannot be counted as ambiguity. Fixed in #16249. |
What does it mean by |
With that background, consider a set of arguments to which both methods apply (the only condition in which you have to worry about ambiguity and specificity): it requires that |
Does it mean that f{T,N}(::Array{T,N}, ::Int)
f{T,N}(::Array{T,N}, ::Vararg{Int,N}) Should be processed differently? and therefore f{T,N}(::Array{T,N}, ::Int)
f(::Array, ::Int) Are non-identical signatures even though they matches exactly the same set of types? |
And following that argument should this not be ambiguous? julia> f{T}(::T, ::Int) = 1
f (generic function with 1 method)
julia> f(::Int, ::Any) = 2
f (generic function with 2 methods)
julia> f(1, 1)
ERROR: MethodError: f(::Int64, ::Int64) is ambiguous. Candidates:
svec(Tuple{#f,Int64,Int64},svec(),f(::Int64, ::Any) at REPL[2]:1)
svec(Tuple{#f,Int64,Int64},svec(Int64),f{T}(::T, ::Int64) at REPL[1]:1)
in eval(::Module, ::Any) at ./boot.jl:230 |
And the following are also along the same line (this is on #16249). Following the argument above, 2 and 3 should return 1 and 4 and 5 should be ambiguous. julia> type A{T}
end
julia> f2{T}(::A{T}, ::A{T}) = 1
f2 (generic function with 1 method)
julia> f2(::A, ::A{Int}) = 2
f2 (generic function with 2 methods)
julia> f2(A{Int}(), A{Int}())
2
julia> f3{T}(::T, ::T) = 1
f3 (generic function with 1 method)
julia> f3(::Any, ::Int) = 2
f3 (generic function with 2 methods)
julia> f3(1, 1)
2
julia> f4{T}(::A{T}, ::A{T}) = 1
f4 (generic function with 1 method)
julia> f4{T}(::A{T}, ::A{Int}) = 2
f4 (generic function with 2 methods)
julia> f4(A{Int}(), A{Int}())
2
julia> f5{T}(::T, ::T) = 1
f5 (generic function with 1 method)
julia> f5{T}(::T, ::Int) = 2
f5 (generic function with 2 methods)
julia> f5(1, 1)
2 Although IMHO all of them should be ambiguous (which I personally think is more consistent since the specificity of a signature is then purely determined by the set of types it matches) but I'm not 100% sure which approach is more intuitive/easier to implement. |
#11242 (comment): I'm not quite sure I understand the point you're trying to make in the #11242 (comment), given that we give a bonus for binding TypeVars in other places, I agree that should not be ambiguous. That seems like a bug. #11242 (comment): more great examples. Personally, I think there's only one conclusion one can draw: |
Trying this out a bit today and just wanted to say that this stuff is amazing! Thanks for all the work making this get merged @timholy. |
This rabbit hole turned out to be a giant dungeon,
instead, and I haven't yet found the exit. I think I'm very close, but I've spent far too much time on this and I'm beginning to run out of ideas and energy., but yay, I found the exit.This is the latest attempt at
NTuple{N,T} -> Tuple{Vararg{T,N}}
, aka #10911, #10691. Julia builds and passesmanyall tests,but borks on some (e.g.,subarray
). The current problem seems to be that aVarState(Tuple{_<:Vararg{T,N}},false)
gets injected somewhere (leading ultimately to an attemptedTuple{Vararg, ASCIIString}
andVararg type in non-final position
error). It's probably a 3-line fix, but I have not yet succeeded in finding the correct 3 lines (inference.jl is difficult for not-Jeff to hack on).This still containsThe intermediate commits contain a lot of debugging code thatwill eventually bewas deleted in the later commits. In the end, the way I got thisfarto work was to write new versions ofjl_intersect_type
,jl_subtype_le
,type_morespecific
, andtype_match_
, which (1) called the old algorithm and stored the result, then (2) translated NTuples on-the-fly and then called the new algorithm with the translated arguments, and (3) printed some output if the two disagreed.This branch is partway through the deletion of all that debugging code; before completing the migration I'd love to figure out where the problem is.It turns out that we have enough differences in how
NTuple
s andTuple{Vararg}
s behave that some disagreements are unavoidable; one of the best things about this, presumably, is that such differences will go away. I even went to the trouble to catalog the differences; in the "best" column I use "N" to indicate that I think the new algorithm yields the most sensible result, and "O" to indicate the old algorithm. Only two of those four algorithms had differences, of which representative examples are:jl_intersect_type
:none (as measured by a <: b && b <: a)
jl_subtype_le
:_<:Tuple
Tuple
_<:Tuple{_<:Vararg{T<:Any, N<:Any}}
Tuple{Vararg{#T<:Any, N<:Any}}
Tuple{}
NTuple{#_<:Any, #T<:Any}
01OsameTuple{Tuple{_<:Tuple{Any, Any}}}
Tuple{Tuple{Tuple{Any, Any}}}
_<:Tuple{Vararg{Int64, #N<:Any}}
Tuple{Vararg{#T<:Any, #_<:Any}}
Tuple{Symbol, Integer}
Tuple{Vararg{#T<:Any, N<:Any}}
Tuple{Char, Char}
Tuple{Vararg{#T<:Any, N<:Any}}
Tuple{Type{Tuple{Int64, Int64}}}
Tuple{Type{Tuple{Int64, Vararg{Int64, N<:Any}}}}
Tuple{Type{Tuple{Char}}, Tuple{Vararg{Any, N<:Any}}}
Tuple{Type{Tuple{Vararg{#T<:Any, N<:Any}}}, Tuple}
Tuple{Tuple{_<:Tuple{Vararg{Any, N<:Any}}}, Int64}
Tuple{Tuple{Tuple{Vararg{Any, N<:Any}}}, Int64}
Tuple{Tuple{Symbol, Vararg{Symbol, N<:Any}}, DataType, Type{T<:Any}}
Tuple{Vararg{#T<:Any, N<:Any}}
_<:Tuple{Symbol, Vararg{Symbol, N<:Any}}
Tuple
Tuple{DataType, Type}
Tuple{Vararg{#T<:Any, N<:Any}}
_<:Tuple{Tuple{Symbol, Vararg{Symbol, N<:Any}}, DataType, Type{T<:Any}}
Tuple{Tuple{Vararg{Any, N<:Any}}, DataType, Vararg{Any, N<:Any}}
Tuple{Type{Tuple{Type{Base.IteratorsMD.CartesianIndex}, Tuple{Int64, Int64, Int64}}}, UInt64}
Tuple{Type{Tuple{Type{Base.IteratorsMD.CartesianIndex}, Tuple{Vararg{Int64, N<:Any}}}}, UInt64}
jl_type_morespecific
:none
type_match_
:NTuple{#N<:Any, Int}
NTuple{#N<:Any, Integer}
Tuple{Ptr{#T<:Any}, NTuple{#N<:Any, Int64}}
Tuple{Ptr{#T<:Any}, NTuple{#N<:Any, Integer}}
Note that
in only one case doesthere are no cases where the result in current master seems to make more sense to me than the version here. I think there's some chance that, once working, this will fix some ambiguity and similar problems.