Generator Type Instability

[cortner]

r = rand(10)
test1(r) = sum( t^2 for t in r )
test2(r)= sum( [t^2 for t in r] )
@code_warntype test1(r)   # return type Any is inferred
@code_warntype test2(r)   # return type Float64 is inferred

This is discussed at length here

Apparently this should be a feature request, but I think it is actually a bug.

The "bug" was confirmed on 0.5.0 and on 0.6.0-dev.787

[yuyichao]

Ref #18645 although I don't think we want exactly that.

[JeffBezanson]

I believe the problem plaguing inference in this case is that, while sum throws an error when the argument is empty, the specific error thrown is a missing method for zero(Any). Inference is currently conservative in this case, and infers Any when there is no method instead of Bottom. With @vtjnash's work we will hopefully be able to improve this soon. In the meantime, we can work around it by throwing more explicit errors in mr_empty.

[stevengj]

Can't we do better than eltype(t^2 for t in r) == Any?

[Jutho]

In general, the eltype of a generator remains uninferrable, basically due to this line. I agree that we might not want #18645 , but one paradigm I use a lot is where the function of the generator is just a type constructor, i.e. something as simple as (T(i) for i in some_iter). Would it be too bad to add something like

iteratoreltype(::Type{Generator{I,T}}) where {I,T<:Type} = HasEltype()
eltype(g::Generator{I,T}) where {I,T<:Type} = T

I can prepare a PR if this has any change of being accepted?

[yuyichao]

That lowering is wrong.
#18621

[Jutho]

My apologies but I don't understand the relationship?

[yuyichao]

It is wrong to lower (T(i) for i in some_iter) to Generator{...,T}

[Jutho]

Ok thanks. But I don't care much about the syntax. So I could just specify that I explicitly want a
g=Base.Generator(T,some_iterator)
and I would like to have eltype(g) return T. Is the above addition of code acceptable, or are there corner cases (where T(i) does not actually return a type T) that one need to worry about?

[yuyichao]

It's also wrong when T is not a concret type.