Dispatch on tuples of types

[dcjones]

This may be intentional, but I found it surprising.

               _
   _       _ _(_)_     |  A fresh approach to technical computing
  (_)     | (_) (_)    |  Documentation: http://docs.julialang.org
   _ _   _| |_  __ _   |  Type "help()" for help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 0.4.0-dev+4452 (2015-04-22 15:05 UTC)
 _/ |\__'_|_|_|\__'_|  |  Commit 19b2dd5* (0 days old master)
|__/                   |  x86_64-apple-darwin14.3.0

julia> type A end

julia> isa(A, Type{A})
true

julia> isa((A,), Tuple{Type{A}})
false

julia> isa(convert(Tuple{Type{A}}, (A,)), Tuple{Type{A}})
false

Whereas in 0.3

julia> type A end

julia> isa(A, Type{A})
true

julia> isa((A,), (Type{A},))
true

This is in the context of JuliaStats/KernelDensity.jl#14. @sebastiang is trying to dispatch on something that boils down to foo{T <: A, T <: A}(::Tuple{Type{T}, Type{T}}) but can't figure out how to make in work, nor can I. Am I wrong in expecting it to work?

[sebastiang]

My expectation would be that (Int,String) would be Tuple{DataType,DataType)}, and this is indeed the case in v0.4. And the following inferences work as I expect, so I remain stumped by the particulars of JuliaStats/KernelDensity.jl#14.

julia> f{A <: Number,B}(_::Tuple{A,B}) = 1
f (generic function with 1 method)

julia> f{A <: String,B}(_::Tuple{A,B}) = 2
f (generic function with 2 methods)

julia> f((1,2))
1

julia> f(("a",2))
2

[toivoh]

This seems like a bug. Arguably, it should be possible to use isa(args, argstype) to check whether the type args matches to a method signature given by the tuple type argstype.

[cdsousa]

I cannot test right now, but what about julia> isa((A,), Tuple{Datatype}) ?

[sebastiang]

julia> isa((Int,), Tuple{DataType})
true

[sebastiang]

Though I'm not clear why

(Int,) <: Tuple{DataType}
ERROR: TypeError: subtype: expected Type{T}, got Tuple{DataType}

[JeffBezanson]

(Int,) is not a type at all; it's a tuple that contains a type. So you can't use <: on it.

Tuples now "infer" their parameters from constructor arguments just like other parametric types do:

julia> typeof(Int=>String)
Pair{DataType,DataType}

julia> typeof((Int,String))
Tuple{DataType,DataType}

julia> [Int]
1-element Array{DataType,1}:
 Int64

The difference is that tuples don't provide a way to request parameter types. If you want, you can construct Pair{Any,Any}(1,2), but you can't do the same thing with tuples. Tuple "parameters" are always inferred.

This is normally what you'd expect, but the corner case is that it's impossible to construct a Tuple{Type{Int}}.

To do this dispatch, I think you'll have to pass tuple types --- dispatch on Type{Tuple{T,T}}. Unfortunately the input syntax Tuple{A,B} is much less pleasant than (A,B). Another example in favor of using {A,B} for tuple types.

[mbauman]

The difference between Tuple types and Pair is that Tuple types are covariant, specifically so they behave like dispatch. This seems like a bug:

julia> isa(Int, Any)
true

julia> isa(Int, Type{Int})
true

julia> isa(Int, DataType)
true

julia> isa(("",), Tuple{Any})
true

julia> isa(("",), Tuple{String})
true

julia> isa(("",), Tuple{ASCIIString})
true

julia> isa((Int,), Tuple{Any})
true

julia> isa((Int,), Tuple{Type{Int}})
false

julia> isa((Int,), Tuple{DataType})
true

Edit: Actually, upon further reflection, I understand now. typeof((Int,)) is a Tuple{DataType}. And this is behaving like dispatch:

julia> f(::Tuple{Type{Int}}) = 1
f (generic function with 1 method)

julia> f((Int,))
ERROR: MethodError: `f` has no method matching f(::Tuple{DataType})
Closest candidates are:
  f(::Tuple{Type{Int64}})

[sebastiang]

Dispatching on the types works, though it's a bit unsatisfying. The fact that Normal and Uniform are types is sort of an implementation detail to the user of this package, though to be fair the function in question (kernel_dist) is only to be extended by implementers of additional distributions.

To overcome the breaking change of (Normal,Normal) needing to be Tuple{Normal,Normal} I'll try to add another method that dispatches one to the other.

I have to say in wandering through packages fixing tuple syntax, there seems to be a lot of abuse of tuples where I would have expected named types. Are anonymous tuples used extensively in Julia's base library or compilers? I've always thought that a few tuples here and there were useful internally, but should essentially never be found in public interfaces.

[JeffBezanson]

Yes, it should work to add a method to convert (Normal,Normal) to Tuple{Normal,Normal}, although the resulting dispatch will probably be slow.

There's nothing wrong with using tuples. In fact now using them is a great idea, since they perform well and don't require declarations. One huge benefit is that if everybody who needs "two integers" uses Tuple{Int,Int}, then they can share lots of code. Declaring new types every time can be highly redundant.

[JeffBezanson]

Closing because I don't plan to change anything here. However any further ideas about the future of Tuple{Type{T}} are welcome.

[toivoh]

So if isa shouldn't be used to check the applicability of a method signature to an argument tuple, what should? Isn't this an inconsistency in the type system? (Making tuples not completely covariant?) I can understand if it's tricky to do something about it, but it would be good to get some more perspective about how it could be allowed to behave in the long run.

[JeffBezanson]

Tuples are still covariant. Tuple{Type{Int}} <: Tuple{DataType} is true, but of course not the other way around. It's just that you can't construct a value of the left-hand type.

One thing worth considering is making typeof(T) always return Type{T} for types, and doing reflection some other way. We could maybe even have DataType{T}. There's an awful lot of isa(T,Type) ? Type{T} : typeof(T).

[toivoh]

But if tuples are covariant, then

isa(A, Type{A})

should imply

isa((A,), Tuple{Type{A}})

(right?) which is clearly not the case currently in 0.4, according to @dcjones original example above.

[sebastiang]

I suspect this is perhaps only tangentially related, but it seems a reasonable place to ask. I'm trying to update https://github.com/timholy/HDF5.jl to use the new syntax and am running into problems I can summarize here. Given

> immutable Y{T<:Tuple{Vararg{AbstractVector}}} end

Why can I not construct the type as I expect here

> Y{(Vector{Int},)}()
ERROR: TypeError: apply_type: in Y, expected Type{T}, got Tuple{DataType}
> Y{(Vector{Int},Vector{String})}()
ERROR: TypeError: apply_type: in Y, expected Type{T}, got Tuple{DataType,DataType}

I'm hoping this is just a syntax confusion on my part.

> Y{Tuple{Vector{Int}}}()
Y{Tuple{Array{Int64,1}}}()

But the distinction in this case between the type of the tuple and the tuple itself is confusing for me. I've done one or the other wrong.

[sebastiang]

Tim has code which returns a tuple of dimensions. e.g. (10,5,2) for describing fixed-size arrays. Ideally, we'd want to be able to parameterize a type with a tuple of ints, e.g. FixedArray{(10,5,2)}. I can do this in v0.4 by just declaring immutable FixedArray{T}, but I don't know how to restrict the type. This is an error:

> immutable FixedArray{T<:Tuple{Vararg{Int}}} end
> FixedArray{(1,2,3)}
ERROR: TypeError: FixedArray: in T, expected T<:Tuple{Vararg{Int64}}, got Tuple{Int64,Int64,Int64}

It makes me wonder why we can't just have vararg type parameters, e.g. using the (nicer imo!) syntax

FixedArray{T...}

[carnaval]

What you would want is something like immutable A{T::Tuple{Vararg{Int}}} not <:. It does not work (syntax error) but probably could without too much effort. Jeff ?

[JeffBezanson]

@toivoh No, that's not what covariance means.

@sebastiang ( ) is now never syntax for a type. If you're writing a type, use Tuple{ }: Y{Tuple{Vector{Int},}}.

[JeffBezanson]

isa-restrictions on type parameters are not implemented yet, but have come up from time to time. I was pretty sure there was an issue about it, but I can't find it.

[sebastiang]

But how do I parameterize a type with a vararg tuple then?

julia> immutable A{T<:Tuple{Vararg{Int}}} end

This is conceptually what I want, but I see why it doesn't work: (1,2,3) is a tuple of ints, not a type which describes a tuple of ints.

julia> A{(1,2,3)}()
ERROR: TypeError: A: in T, expected T<:Tuple{Vararg{Int64}}, got Tuple{Int64,Int64,Int64}

Of course this doesn't work, because 1, 2, and 3 aren't types either

julia> A{Tuple{1,2,3}}()
ERROR: TypeError: A: in T, expected T<:Tuple{Vararg{Int64}}, got Type{Tuple{1,2,3}}

And this works, but doesn't help me in the least. Where did my (1,2,3) go?

julia> A{Tuple{Int}}()
A{Tuple{Int64}}()

So I'm still confused as to how I can retain the parameterization by a value (1,2,3) and restrict that value to be the right kind of Tuple, i.e. such that isa((1,2,3), Tuple{Vararg{Int}})

[sebastiang]

@JeffBezanson, I guess this is what your isa-restriction comment meant; perhaps it's not possible to describe this restriction in v0.4 and it worked 'by accident' in v0.3 because tuples of types were also types? My head spins; thanks for your patience.

Really all we want is FixedArray{T::Int...}

[JeffBezanson]

That's right; we've never had isa restrictions. This doesn't work in 0.3 either:

julia> immutable FixedArray{T<:(Int...)} end

julia> FixedArray{(1,2,3)}
ERROR: type: FixedArray: in T, expected T<:(Int64...,), got (Int64,Int64,Int64)

In 0.3 tuples could be types, but 1 was not, so this hasn't really changed.

[sebastiang]

This is why in HDF5, the wily Tim created a sentinel type DimSize{N} to hold those integers.

# Stub types to encode fixed-size arrays for H5T_ARRAY
immutable DimSize{N}; end  # Int-wrapper (can't use tuple of Int as param)
immutable FixedArray{T,D<:(DimSize...)}; end
dimsize{N}(::Type{DimSize{N}}) = N
size{T,D}(::Type{FixedArray{T,D}}) = map(dimsize, D)::(Int...)
eltype{T,D}(::Type{FixedArray{T,D}}) = T

[toivoh]

@JeffBezanson: You're right, that's not what covariance means. Thinking about it some more, I thought that was what tuple (actually 1-tuple) types mean.

Anyway, making typeof(T) always return Type{T} for types definitely has some appeal. It would make things more consistent, e.g. since it would make

isa(x, T)

and

typeof(x) <: T

equivalent. I have no idea about the implications when it comes to performance, breakage, etc though.