Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pair construction can bork convert #8631

Closed
timholy opened this issue Oct 8, 2014 · 36 comments
Closed

Pair construction can bork convert #8631

timholy opened this issue Oct 8, 2014 · 36 comments
Labels
kind:bug Indicates an unexpected problem or unintended behavior status:priority This should be addressed urgently

Comments

@timholy
Copy link
Sponsor Member

timholy commented Oct 8, 2014

A minimal testcase for the issue reported here:

julia> immutable RGB
           r::Float32
           g::Float32
           b::Float32
       end

julia> 
WARNING: deprecated syntax "{a=>b, ...}".
Use "Dict{Any,Any}(a=>b, ...)" instead.
julia> const colormaps_sequential = {
           #single hue
           #name        hue   w     d     c     s     b     wcolor      dcolor
           "blues"   => (255, 0.3,  0.25, 0.88, 0.6,  0.75, RGB(1,1,0), RGB(0,0,1)),
           "greens"  => (120, 0.15, 0.18, 0.88, 0.55, 0.9,  RGB(1,1,0), RGB(0,0,1)),
           "grays"   => (0,   0.0,  0.0,  1.0,  0.0,  0.75, RGB(1,1,0), RGB(0,0,1)),
           "oranges" => (20,  0.5,  0.4,  0.83, 0.95, 0.85, RGB(1,1,0), RGB(1,0,0)),
           "purples" => (265, 0.15, 0.2,  0.88, 0.5,  0.7,  RGB(1,0,1), RGB(1,0,0)),
           "reds"    => (12,  0.15, 0.25, 0.8,  0.85, 0.6,  RGB(1,1,0), RGB(0.3,0.1,0.1))
       }

WARNING: deprecated syntax "{a=>b, ...}".
Use "Dict{Any,Any}(a=>b, ...)" instead.
Dict{Any,Any} with 6 entries:
  "reds"    => (12,0.15,0.25,0.8,0.85,0.6,RGB(1.0f0,1.0f0,0.0f0),RGB(0.3f0,0.1f0,0.1f0))
  "purples" => (265,0.15,0.2,0.88,0.5,0.7,RGB(1.0f0,0.0f0,1.0f0),RGB(1.0f0,0.0f0,0.0f0))
  "oranges" => (20,0.5,0.4,0.83,0.95,0.85,RGB(1.0f0,1.0f0,0.0f0),RGB(1.0f0,0.0f0,0.0f0))
  "greens"  => (120,0.15,0.18,0.88,0.55,0.9,RGB(1.0f0,1.0f0,0.0f0),RGB(0.0f0,0.0f0,1.0f0))
  "blues"   => (255,0.3,0.25,0.88,0.6,0.75,RGB(1.0f0,1.0f0,0.0f0),RGB(0.0f0,0.0f0,1.0f0))
  "grays"   => (0,0.0,0.0,1.0,0.0,0.75,RGB(1.0f0,1.0f0,0.0f0),RGB(0.0f0,0.0f0,1.0f0))

julia> 
WARNING: deprecated syntax "{a=>b, ...}".
Use "Dict{Any,Any}(a=>b, ...)" instead.
julia> const colormaps_diverging = {
           #name         h1   h2    w     d1    d2    c      s     b      wcolor      dcolor      dcolor2
           "rdbu"    => (12,  255,  0.2,  0.6,  0.0,  0.85,  0.6,  0.65,  RGB(1,1,0), RGB(1,0,0), RGB(0,0,1))
       }

WARNING: deprecated syntax "{a=>b, ...}".
Use "Dict{Any,Any}(a=>b, ...)" instead.
Dict{Any,Any} with 1 entry:
  "rdbu" => (12,255,0.2,0.6,0.0,0.85,0.6,0.65,RGB(1.0f0,1.0f0,0.0f0),RGB(1.0f0,0.0f0,0.0f0),RGB(0.0f0,0.0f0,1.0f0))

And now:

julia> convert((Int...), (uint(1),))
ERROR: `convert` has no method matching convert(::Type{Int64...}, ::Uint64)
 in convert at base.jl:9
 in convert at base.jl:17
@timholy
Copy link
Sponsor Member Author

timholy commented Oct 8, 2014

That's just a copy-paste of the maps_data.jl file from Color.

@timholy timholy changed the title Deprecated Dict syntax can bork convert Dict construction can bork convert Oct 8, 2014
@timholy
Copy link
Sponsor Member Author

timholy commented Oct 8, 2014

Actually, even if you use the new syntax, the error remains.

@timholy
Copy link
Sponsor Member Author

timholy commented Oct 8, 2014

An even more minimal case:

julia> immutable RGB
           r::Float32
           g::Float32
           b::Float32
       end

julia> const colormaps_sequential = Dict(
           "blues"   => (255, 0.3,  0.25, 0.88, 0.6,  0.75, RGB(1,1,0), RGB(0,0,1)))
Dict{ASCIIString,(Int64,Float64,Float64,Float64,Float64,Float64,RGB,RGB)} with 1 entry:
  "blues" => (255,0.3,0.25,0.88,0.6,0.75,RGB(1.0f0,1.0f0,0.0f0),RGB(0.0f0,0.0f0,1.0f0))

julia> convert((Int...), (uint(1),))
ERROR: `convert` has no method matching convert(::Type{Int64...}, ::Uint64)
 in convert at base.jl:9
 in convert at base.jl:17

@simonster
Copy link
Member

Even more minimal:

julia> 1 => (1,1,1,1,1,1,1,1,1.,false);

julia> convert((Int...), (uint(1),))
ERROR: `convert` has no method matching convert(::Type{Int64...}, ::Uint64)
 in convert at base.jl:9
 in convert at base.jl:17

@simonster simonster changed the title Dict construction can bork convert Pair construction can bork convert Oct 8, 2014
@JeffBezanson
Copy link
Sponsor Member

Good, this is the level of breakage I was hoping to get to in 0.4-devel :) On the other hand, oh f*** there goes my week.

@timholy
Copy link
Sponsor Member Author

timholy commented Oct 8, 2014

I sympathize. Meanwhile, I'll be curious to see whether this might be the same bug that's also behind #8618, #8128 (which seems to be working now, even though #8618 is not). Those others aren't obviously pair-related, of course, so the pessimistic view is that there is more than one 😦.

@jakebolewski
Copy link
Member

Playing around with this

julia> 1=>(1,1,1,1,false)
1=>(1,1,1,1,false)

julia> convert((Int...), (uint(1),))
(1,)
julia> 1=>(1,1,1,1,1,false)
1=>(1,1,1,1,1,false)

julia> convert((Int...), (uint(1),))
ERROR: `convert` has no method matching convert(::Type{Int64...}, ::Uint64)
julia> 1=>(1,1,1,1,2.0)
1=>(1,1,1,1,2.0)

julia> convert((Int...), (uint(1),))
(1,)
julia> 1=>(1,1,1,1,1,2.0)
1=>(1,1,1,1,1,2.0)

julia> convert((Int...), (uint(1),))
ERROR: `convert` has no method matching convert(::Type{Int64...}, ::Uint64)
julia> 1=>(1,1,1,1,1,2.0,1)
1=>(1,1,1,1,1,2.0,1)

julia> convert((Int...), (uint(1),))
(1,)
julia> 1=>(1,1,1,1,1,1,2.0,1)
1=>(1,1,1,1,1,1,2.0,1)

julia> convert((Int...), (uint(1),))
ERROR: `convert` has no method matching convert(::Type{Int64...}, ::Uint64)
julia> 1=>(1.0,2,3,4, false)
1=>(1.0,2,3,4,false)

julia> convert((Int...), (uint(1),))
(1,)
julia> 1=>(1.0,2,3,4,5, false)
1=>(1.0,2,3,4,5,false)

julia> convert((Int...), (uint(1),))
ERROR: `convert` has no method matching convert(::Type{Int64...}, ::Uint64)
julia> 1=>(false,1.0,2,3,4)
1=>(false,1.0,2,3,4)

julia> convert((Int...), (uint(1),))
(1,)
julia> 1=>(false,1.0,2,3,4,5)
1=>(false,1.0,2,3,4,5)

julia> convert((Int...), (uint(1),))
ERROR: `convert` has no method matching convert(::Type{Int64...}, ::Uint64)
julia> 1 => (false,1,2,3,4,5)
1=>(false,1,2,3,4,5)

julia> convert((Int...), (uint(1),))
(1,)

So it seems like it is potentially related to these magic numbers although I don't offhand know why it starts at N+1.

@jakebolewski
Copy link
Member

Actually changing those numbers does not seem to impact behavior.

@garborg
Copy link
Contributor

garborg commented Oct 9, 2014

In case it's not old news, convert is breaking for me on 0.3:

julia> aa = convert(Array{Ufixed8}, a)
ERROR: `convert` has no method matching convert(::Type{Array{UfixedBase{Uint8,8},N}}, ::Array{Float64,2})
 in convert at base.jl:13

julia> Pkg.status()
ERROR: `convert` has no method matching convert(::Type{UTF8String}, ::ASCIIString)
 in convert at base.jl:13
 in dir at pkg/git.jl:10
 in git at pkg/git.jl:16
 in success at pkg/git.jl:27
 in iscommit at pkg/git.jl:44
 in installed_version at ./pkg/read.jl:65
 in installed at ./pkg/read.jl:122
 in status at pkg/entry.jl:107
 in anonymous at pkg/dir.jl:28
 in cd at ./file.jl:20
 in cd at pkg/dir.jl:28
 in status at pkg.jl:28 (repeats 2 times)

julia> versioninfo()
Julia Version 0.3.2-pre+48
Commit bf35ce6 (2014-10-08 14:44 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin13.4.0)
  CPU: Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

@timholy
Copy link
Sponsor Member Author

timholy commented Oct 16, 2014

@bjarthur noted that this can be triggered on 0.3, too, although the minimal testcase above is not sufficient to do it.
JuliaAttic/Color.jl#68

@davidssmith
Copy link
Contributor

I'm getting this on my 3.3 install when I try to return a tuple of Float64 matrices from fetch. The code is hundreds of lines long, and I can't really reduce it to a minimal test case. I could try to write one if that would help, but I suspect it is the same issue with convert.

Julia Version 0.3.3-pre+8
Commit 142798d (2014-10-23 10:59 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin14.0.0)
  CPU: Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas
  LIBM: libopenlibm
  LLVM: libLLVM-3.3

@timholy
Copy link
Sponsor Member Author

timholy commented Nov 2, 2014

I needed to take a short break from more tedious stuff, so I got started here; only a tiny step forward, but I thought I'd throw it out there since I still have deadlines that will occupy most of my attention.

I began by inserting a print statement:

diff --git a/src/gf.c b/src/gf.c
index ee226d9..0b21779 100644
--- a/src/gf.c
+++ b/src/gf.c
@@ -1841,6 +1843,8 @@ static jl_value_t *ml_matches(jl_methlist_t *ml, jl_value_t *type,
         ti = lookup_match(type, (jl_value_t*)ml->sig, &env, ml->tvars);
         if (ti != (jl_value_t*)jl_bottom_type) {
             assert(ml->func->linfo);  // no builtin methods
+           if (strcmp(ml->func->linfo->name->name, "convert") == 0)
+               jl_printf(JL_STDOUT, "match function %s, file %s, line %d\n", ml->func->linfo->name->name, ml->func->linfo->file->name, ml->func->linfo->line);
             assert(jl_is_tuple(env));

             int skip = 0;

and then used the following debug file:

1 => (1,2)  # A non-breaking warmup
# 1 => (1,1,1,1,1,1,1,1,1.,false);
println("Starting")
convert((Int...), (uint(1),))

If I leave the comment in place (so convert is not broken), then the log after Starting looks like this:

match function convert, file bool.jl, line 4
match function convert, file int.jl, line 154
match function convert, file int.jl, line 156
match function convert, file int.jl, line 154
match function convert, file int.jl, line 156
match function convert, file bool.jl, line 4
match function convert, file int.jl, line 154
match function convert, file int.jl, line 156
match function convert, file int.jl, line 154
match function convert, file int.jl, line 156
match function convert, file bool.jl, line 3
match function convert, file bool.jl, line 4
match function convert, file int.jl, line 163
match function convert, file int.jl, line 163
match function convert, file int.jl, line 163
match function convert, file int.jl, line 163
match function convert, file base.jl, line 36
match function convert, file base.jl, line 36
match function convert, file base.jl, line 36
match function convert, file base.jl, line 36

but if I uncomment that line, then it looks like this:

match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file bool.jl, line 4
match function convert, file int.jl, line 154
match function convert, file int.jl, line 156
match function convert, file int.jl, line 154
match function convert, file int.jl, line 156
match function convert, file base.jl, line 39
match function convert, file bool.jl, line 4
match function convert, file int.jl, line 154
match function convert, file int.jl, line 156
match function convert, file int.jl, line 154
match function convert, file int.jl, line 156
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file bool.jl, line 3
match function convert, file bool.jl, line 4
match function convert, file int.jl, line 163
match function convert, file int.jl, line 163
match function convert, file int.jl, line 163
match function convert, file int.jl, line 163
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 36
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 36
match function convert, file base.jl, line 39
match function convert, file base.jl, line 36
match function convert, file base.jl, line 36
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file pointer.jl, line 10
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file base.jl, line 39
match function convert, file pointer.jl, line 10

Basically, the two start out identically except for the presence of a bunch of base.jl line 39 matches. This is my line 39. Kinda weird.

@timholy
Copy link
Sponsor Member Author

timholy commented Nov 3, 2014

Surprisingly, "convertalypse" (or at least, this specific manifestation) turns out to be a previously undetected consequence of @vtjnash's inlining patches. The problem lies somewhere in this range of commits:
cd4666d
e9e5606
31969bf
ec2fa85
4f71f59
5b91981
2bc684b
350c8e7
19d20ac
51c9a5b
baa1cb4
f4357ac
f8bfa2f

I can't narrow it down any better than this, because julia won't build within this range.

Here was my test script:

type Pair{A,B}
    first::A
    second::B
end
Pair(1, (1,1,1,1,1,1,1,1,1.,false));
convert((Int...), (uint(1),))

@vtjnash
Copy link
Sponsor Member

vtjnash commented Nov 3, 2014

In the inlining work, I assume that the subtype relationship must hold between the input arguments and the function type signature, however, for Tuples of typevars, Julia computes the wrong answer for subtype. Perhaps that is at issue here? I already opened an issue for it.

@timholy
Copy link
Sponsor Member Author

timholy commented Nov 3, 2014

Not sure I follow entirely. Would that explain why it works in a fresh julia session, but that once you construct a Pair it's broken henceforth?

The Tuple may be a red herring, if it turns out that a single bug underlies #8818, #8836. I think it's too early to tell, but we should at least be aware of the possibility.

@timholy
Copy link
Sponsor Member Author

timholy commented Nov 4, 2014

Unfortunately, there seems to be more than one problem underlying "convertalypse." Disabling all inlining by building julia with

diff --git a/base/inference.jl b/base/inference.jl
index 0ecb096..ed6765e 100644
--- a/base/inference.jl
+++ b/base/inference.jl
@@ -2032,6 +2032,7 @@ end
 # static parameters are ok if all the static parameter values are leaf types,
 # meaning they are fully known.
 function inlineable(f, e::Expr, atypes, sv, enclosing_ast)
+    return NF
     if !(isa(f,Function) || isstructtype(f) || isa(f,IntrinsicFunction))
         return NF
     end

"fixes" the problem here but does not fix #8818. So your suggestion that this may be limited to tuple-types is probably well-founded.

GunnarFarneback added a commit to GunnarFarneback/Color.jl that referenced this issue Dec 6, 2014
Switching the colormaps_sequential and colormaps_diverging values from tuples to vectors works around the Julia bug JuliaLang/julia#8631, solving JuliaAttic#68 and JuliaIO/HDF5.jl#160.
@vtjnash
Copy link
Sponsor Member

vtjnash commented Dec 28, 2014

the issue may be that in the call to ml_matches in need_guard_entries (

julia/src/gf.c

Line 781 in fb730f3

temp = ml_matches(mt->defs, (jl_value_t*)type, lambda_sym, -1);
) there is a method missing from the returned list.

in particular, the following call:

julia> methods(convert, ((Type, Type...), (Any, Any...)))
2-element Array{Any,1}:
 convert(::Type{Tuple},x::Tuple) at base.jl:39                   
 convert(T::(Type,Type...),x::(Any,Any...)) at base.jl:44

is missing the following function definition:

convert{T}(::Type{(T...)}, x::Tuple) = cnvt_all(T, x...)

therefore, when we specialize convert(::(Type{Int64}, Type{Int64}, Type{Int64}, Type{Int64}, Type{Int64}, Type{Int64}, Type{Int64}, Type{Float64}, Type{Bool}), ::(Int64, Int64, Int64, Int64, Int64, Int64, Int64, Float64, Bool)) to match convert(::(Type, Type...), ::(Any, Any...)) the guard entry for Type... matching the empty list is not generated.

@timholy
Copy link
Sponsor Member Author

timholy commented Dec 30, 2014

Nice detective work. I assume you don't yet have an idea why that method isn't showing up?

@vtjnash
Copy link
Sponsor Member

vtjnash commented Dec 30, 2014

no, usually i just hope jeff will chime in at some point.

although, looking back at it, i'm not sure if my statement was entirely correct either. It's not exactly missing the match for the empty list so much as missing the match for when (Type, Type...) can be reduced to (T...)

but it does let us make a more reduced test case:

julia> broken(::(Type, Type...), ::(Any, Any...)) = 1
broken (generic function with 1 method)

julia> broken{T}(::Type{(T...)}, x::Tuple) = 2
broken (generic function with 2 methods)

julia> methods(broken, ((Type, Type...), (Any, Any...)))
1-element Array{Any,1}:
 broken(::(Type{T<:Top},Type{T<:Top}...),::(Any,Any...)) at none:1

and sticking a breakpoint in jl_matching_methods, i can see that lookup_match returns Union() when attempting to compare the requested signature to that first method signature.

tracing through the code execution, we see that (Type, Type...) turns into Type{(Top, Top...)}. Note the presence of Top there. The type intersection of Type{(T<:Any...)} and Type{(T<:Top...)} is Union()

which means that we can "fix" it (and the original bug) with one liner (kids, don't try this at home):

julia> broken(::(Type, Type...), ::(Any, Any...)) = 1
broken (generic function with 1 method)

julia> broken{T}(::Type{(T...)}, x::Tuple) = 2
broken (generic function with 2 methods)

julia> methods(broken, ((Type, Type...), (Any, Any...)))
1-element Array{Any,1}:
 broken(::(Type{T<:Top},Type{T<:Top}...),::(Any,Any...)) at none:1

julia> Type.parameters = (TypeVar(:T),)
(T,)

julia> methods(broken, ((Type, Type...), (Any, Any...)))
2-element Array{Any,1}:
 broken{T}(::Type{(T...,)},x::(Any...,)) at none:1      
 broken(::(Type{T},Type{T}...),::(Any,Any...)) at none:1

so, now that you got me started again, it seems I do now have some idea why this isn't showing up

@vtjnash
Copy link
Sponsor Member

vtjnash commented Dec 30, 2014

@JeffBezanson can you verify the validity of the following patch (since you can't make a tuple containing #undef)

fwiw, the following change doesn't seem to break things (and it fixes this bug):

diff --git a/src/jltypes.c b/src/jltypes.c
index 7d4d53c..50a838e 100644
--- a/src/jltypes.c
+++ b/src/jltypes.c
@@ -751,10 +751,12 @@ static int tuple_to_Type(jl_tuple_t *a, jl_tuple_t **ptemp)
     for(i=0; i < alen; i++) {
         jl_value_t *el = jl_tupleref(a, i);
         if (jl_is_type_type(el)) {
-            jl_tupleset(*ptemp, i, jl_tparam0(el));
+            jl_value_t *T = jl_type_intersection(jl_tparam0(el), (jl_value_t*)jl_any_type); // removes Undef
+            jl_tupleset(*ptemp, i, T);
         }
         else if (i==alen-1 && jl_is_vararg_type(el) && jl_is_type_type(jl_tparam0(el))) {
-            jl_tupleset(*ptemp, i, jl_wrap_vararg(jl_tparam0(jl_tparam0(el))));
+            jl_value_t *T = jl_type_intersection(jl_tparam0(jl_tparam0(el)), (jl_value_t*)jl_any_type); // removes Undef
+            jl_tupleset(*ptemp, i, jl_wrap_vararg(T));
         }
         else {
             *ptemp = NULL;

with the following test case:

f8631(::(Type, Type...), ::(Any, Any...)) = 1
f8631{T}(::Type{(T...)}, x::Tuple) = 2
@test length(methods(f8631, ((Type, Type...), (Any, Any...)))) == 2

i'm not sure if this is entirely complete, however, or if the following answer is also wrong:

julia> Base.typeintersect(Type{(TypeVar(:T,Core.Top), TypeVar(:T,Core.Top)...)}, Type{(TypeVar(:T)...,)})
Union()

@JeffBezanson
Copy link
Sponsor Member

That patch looks safe. In the second part T might need to be rooted before passing it to jl_wrap_vararg.

@vtjnash
Copy link
Sponsor Member

vtjnash commented Dec 31, 2014

ok. however, i don't feel like that patch is complete. i can't quite decide if the following are correct:

julia> Base.typeintersect((Type{TypeVar(:A,Integer)}, Type{TypeVar(:B)}...), Type{(TypeVar(:C,Integer)...,)})
Union()

julia> Base.typeintersect((Type{TypeVar(:A)}, Type{TypeVar(:B)}...), Type{(TypeVar(:C,Integer)...,)})
Type{(C<:Integer...,)}

it seems to be a covariance/invariance complication arising from the Type{(T,)}, (Type{T},) duality. in the former case, we expect that Type{(T,)} must be matched invariantly (including the TypeVar, because it is in a type parameter), but in the second case, we expect that the TypeVar can be matched covariantly. however, it seems that these two type specifications are assumed to be equivalent (by the existence of tuple_to_Type).

so that leads to this alternative patch, which also fixes the convert bug, but in a very different way:

diff --git a/src/jltypes.c b/src/jltypes.c
index 7d4d53c..69982b1 100644
--- a/src/jltypes.c
+++ b/src/jltypes.c
@@ -539,6 +539,14 @@ static jl_value_t *intersect_tag(jl_datatype_t *a, jl_datatype_t *b,
                     continue;
                 }
             }
+            else if (jl_is_tuple(ap)) {
+                if (!jl_is_tuple(bp)) {
+                    JL_GC_POP();
+                    return jl_bottom_type;
+                }
+                ti = intersect_tuple((jl_tuple_t*)ap, (jl_tuple_t*)bp, penv,eqc,invariant);
+                jl_tupleset(p, i, ti);
+            }
             else {
                 int tva = jl_has_typevars_(ap,0);
                 int tvb = jl_has_typevars_(bp,0);

and now we get:

julia> Base.typeintersect((Type{TypeVar(:A,Integer)}, Type{TypeVar(:B)}...), Type{(TypeVar(:C,Integer)...,)})
Type{(C<:Integer,C<:Integer...)}

julia> Base.typeintersect((Type{TypeVar(:A)}, Type{TypeVar(:B)}...), Type{(TypeVar(:C,Integer)...,)})
Type{(C<:Integer...,)}

(although now this fails other tests)

additionally, 98a1165 seems closely related to this issue.

@vtjnash
Copy link
Sponsor Member

vtjnash commented Dec 31, 2014

or instead:

diff --git a/src/jltypes.c b/src/jltypes.c
index 7d4d53c..8ccb59b 100644
--- a/src/jltypes.c
+++ b/src/jltypes.c
@@ -539,6 +539,9 @@ static jl_value_t *intersect_tag(jl_datatype_t *a, jl_datatype_t *b,
                     continue;
                 }
             }
+            else if (jl_is_tuple(ap) && jl_is_tuple(bp)) {
+                ti = intersect_tuple((jl_tuple_t*)ap, (jl_tuple_t*)bp, penv,eqc,invariant);
+            }
             else {
                 int tva = jl_has_typevars_(ap,0);
                 int tvb = jl_has_typevars_(bp,0);

although that can mucks things up pretty badly, since it turns
convert(::Type{Tuple}, x::Tuple) = x
into the equivalent:
convert(::Type{(Any...)}, x::Tuple) = x
and therefore starts to consider it to be a possible (and preferable) method match for pretty much any tuple:

julia> methods(convert, typeof(((Int...), (l,))))
2-element Array{Any,1}:
 convert(::Type{(Any...,)},x::(Any...,)) at base.jl:39                   
 convert(T::(Type{T<:Top},Type{T<:Top}...),x::(Any,Any...)) at base.jl:44

@garrison
Copy link
Sponsor Member

garrison commented Feb 4, 2015

No comments here in over a month, so I'll ask: What is the current status of this issue with convert? From the above breakthroughs I thought it was nearly fixed and was hoping it would go in 0.3.5. Now there is talk of 0.3.6 (planning issue: #10058) and I'm curious how likely it is that this will be fixed soon.

JeffBezanson added a commit that referenced this issue Feb 14, 2015
This reverts commit 84e80f4.

Major performance regressions. This commit will be back shortly when
fixed.
@ivarne
Copy link
Sponsor Member

ivarne commented Feb 14, 2015

🍰 🎆

@ivarne ivarne reopened this Feb 14, 2015
@aviks
Copy link
Member

aviks commented Feb 14, 2015

👏

@ivarne did you intend to reopen this?

[edit: saw the revert, sorry for the noise]

@tkelman
Copy link
Contributor

tkelman commented Feb 15, 2015

Nice work @JeffBezanson! How much different would this look like on release-0.3? We're probably tagging 0.3.6 soon, might be good to do this as one of the first commits for 0.3.7 (assuming it doesn't introduce any new issues on master).

@JeffBezanson
Copy link
Sponsor Member

Thanks! This is a tricky one. Since this removes the Top and Undef types it's technically a breaking change, though user code should never have used those types. Tough call. Other than that it should be possible to backport.

@tkelman
Copy link
Contributor

tkelman commented Feb 15, 2015

Right, don't want to rush this, but wanted to get the question out there since this has been bothering people for a while. Let's see who can break it first :)

@timholy
Copy link
Sponsor Member Author

timholy commented Feb 15, 2015

Great to have this closed! Many thanks.

@tlnagy
Copy link
Contributor

tlnagy commented Mar 10, 2016

Is this supposed to be working in 0.4.3?

julia> 1 => (1,1,1,1,1,1,1,1,1.,false)
1=>(1,1,1,1,1,1,1,1,1.0,false)

julia> convert((Int...), (uint(1),))
ERROR: MethodError: `start` has no method matching start(::Type{Int64})
 in append_any at essentials.jl:127

@JeffBezanson
Copy link
Sponsor Member

convert(Tuple{Vararg{Int}}, (UInt(1),)) is what you want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:bug Indicates an unexpected problem or unintended behavior status:priority This should be addressed urgently
Projects
None yet
Development

No branches or pull requests