Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFC: Change iteratorsize trait of product(itr1, itr2) #16437

Merged
merged 5 commits into from May 29, 2016
Merged

RFC: Change iteratorsize trait of product(itr1, itr2) #16437

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
ce96a51
Change iteratorsize trait of `product(itr1, itr2)`
May 18, 2016
5c082ec
Improved behaviour for multi-dimensional arguments
May 21, 2016
e2ca636
Removed trailing whitespace
May 21, 2016
d924a7a
merge with upstream
May 28, 2016
c6106f7
Change iteratorsize trait of product
May 28, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
40 changes: 35 additions & 5 deletions base/iterator.jl
View file
Open in desktop
Expand Up @@ -298,10 +298,33 @@ done(it::Repeated, state) = false

repeated(x, n::Int) = take(repeated(x), n)

# product

# Product -- cartesian product of iterators

abstract AbstractProdIterator
length(p::AbstractProdIterator) = prod(size(p))

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one is ambiguous

# one iterator
immutable Prod1{I} <: AbstractProdIterator
a::I
end
product(a) = Prod1(a)

eltype{I}(::Type{Prod1{I}}) = Tuple{eltype(I)}
size(p::Prod1) = size(p.a)
ndims(p::Prod1) = ndims(p.a)

@inline start(p::Prod1) = start(p.a)
@inline function next(p::Prod1, st)
n, st = next(p.a, st)
return (n, ), st
end
@inline done(p::Prod1, st) = done(p.a, st)

iteratoreltype{I}(::Type{Prod1{I}}) = iteratoreltype(I)
iteratorsize{I}(::Type{Prod1{I}}) = iteratorsize(I)

# two iterators
immutable Prod2{I1, I2} <: AbstractProdIterator
a::I1
b::I2
Expand All @@ -323,11 +346,13 @@ changes the fastest. Example:
(1,5)
(2,5)
"""
product(a) = Zip1(a)
product(a, b) = Prod2(a, b)

eltype{I1,I2}(::Type{Prod2{I1,I2}}) = Tuple{eltype(I1), eltype(I2)}
size(p::Prod2) = (length(p.a), length(p.b))
ndims(p::Prod2) = 2

iteratoreltype{I1,I2}(::Type{Prod2{I1,I2}}) = and_iteratoreltype(iteratoreltype(I1),iteratoreltype(I2))
length(p::AbstractProdIterator) = length(p.a)*length(p.b)
iteratorsize{I1,I2}(::Type{Prod2{I1,I2}}) = prod_iteratorsize(iteratorsize(I1),iteratorsize(I2))

function start(p::AbstractProdIterator)
Expand Down Expand Up @@ -355,13 +380,17 @@ end
@inline next(p::Prod2, st) = prod_next(p, st)
@inline done(p::AbstractProdIterator, st) = st[4]

# n iterators
immutable Prod{I1, I2<:AbstractProdIterator} <: AbstractProdIterator
a::I1
b::I2
end

product(a, b, c...) = Prod(a, product(b, c...))

eltype{I1,I2}(::Type{Prod{I1,I2}}) = tuple_type_cons(eltype(I1), eltype(I2))
size(p::Prod) = (length(p.a), size(p.b)...)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is tricky: depending on who HasShape or HasLength,
(length(p.a), size(p.b)...) or (size(p.a)..., size(p.b)...) or (size(p.a)..., length(p.b)). You can define _prod_size in a similar way as _min_length or _diff_length

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmm, not sure I follow. Can you write a test here that would make the current implementation fail?

ndims(p::Prod) = length(size(p))

iteratoreltype{I1,I2}(::Type{Prod{I1,I2}}) = and_iteratoreltype(iteratoreltype(I1),iteratoreltype(I2))
iteratorsize{I1,I2}(::Type{Prod{I1,I2}}) = prod_iteratorsize(iteratorsize(I1),iteratorsize(I2))

Expand All @@ -370,8 +399,9 @@ iteratorsize{I1,I2}(::Type{Prod{I1,I2}}) = prod_iteratorsize(iteratorsize(I1),it
((x[1][1],x[1][2]...), x[2])
end

prod_iteratorsize(::Union{HasLength,HasShape}, ::Union{HasLength,HasShape}) = HasLength()
prod_iteratorsize(::Union{HasLength,HasShape}, ::Union{HasLength,HasShape}) = HasShape()
prod_iteratorsize(a, ::IsInfinite) = IsInfinite() # products can have an infinite last iterator (which moves slowest)
prod_iteratorsize(::IsInfinite, b) = IsInfinite()
prod_iteratorsize(a, b) = SizeUnknown()
Copy link
Contributor

@mschauer mschauer May 27, 2016 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one too, but this should be removed, because a or b might be zero and 0*infty=0 in terms of taking products.

Copy link
Contributor Author

@gasagna gasagna May 28, 2016 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But that would be a special case. How would you handle the majority of cases where you have (finite nonzero size) * (infinite size) ?


_size(p::Prod2) = (length(p.a), length(p.b))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed anymore?

Expand Down
126 changes: 118 additions & 8 deletions test/functional.jl
View file
Open in desktop
Expand Up @@ -177,14 +177,124 @@ end
# product
# -------

@test isempty(Base.product(1:2,1:0))
@test isempty(Base.product(1:2,1:0,1:10))
@test isempty(Base.product(1:2,1:10,1:0))
@test isempty(Base.product(1:0,1:2,1:10))
@test collect(Base.product(1:2,3:4)) == [(1,3),(2,3),(1,4),(2,4)]
@test isempty(collect(Base.product(1:0,1:2)))
@test length(Base.product(1:2,1:10,4:6)) == 60
@test Base.iteratorsize(Base.product(1:2, countfrom(1))) == Base.IsInfinite()
# empty?
for itr in [Base.product(1:0),
Base.product(1:2, 1:0),
Base.product(1:0, 1:2),
Base.product(1:0, 1:1, 1:2),
Base.product(1:1, 1:0, 1:2),
Base.product(1:1, 1:2 ,1:0)]
@test isempty(itr)
@test isempty(collect(itr))
end

# collect a product - first iterators runs faster
@test collect(Base.product(1:2)) == [(i,) for i=1:2]
@test collect(Base.product(1:2, 3:4)) == [(i, j) for i=1:2, j=3:4]
@test collect(Base.product(1:2, 3:4, 5:6)) == [(i, j, k) for i=1:2, j=3:4, k=5:6]

# iteration order
let expected = [(1,3,5), (2,3,5), (1,4,5), (2,4,5), (1,3,6), (2,3,6), (1,4,6), (2,4,6)]
i = 1
for el in Base.product(1:2, 3:4, 5:6)
@test el == expected[i]
i+=1
end
end

# is this the correct behaviour?
@test collect(Base.product([1 2; 3 4], [5 6; 7 8])) == [(1,5) (1,7) (1,6) (1,8);
(3,5) (3,7) (3,6) (3,8);
(2,5) (2,7) (2,6) (2,8);
(4,5) (4,7) (4,6) (4,8)]

let
a, b, c, d, e = 1:2, 1.0:10.0, 4f0:6f0, 0x01:0x08, Int8(1):Int8(0)

# length
@test length(Base.product(a)) == 2
@test length(Base.product(a, b)) == 20
@test length(Base.product(a, b, c)) == 60
@test length(Base.product(a, b, c, d)) == 480
@test length(Base.product(a, b, c, d, e)) == 0

# size
@test size(Base.product(a)) == (2, )
@test size(Base.product(a, b)) == (2, 10)
@test size(Base.product(a, b, c)) == (2, 10, 3)
@test size(Base.product(a, b, c, d)) == (2, 10, 3, 8)
@test size(Base.product(a, b, c, d, e)) == (2, 10, 3, 8, 0)

# eltype
@test eltype(Base.product(a)) == Tuple{Int}
@test eltype(Base.product(a, b)) == Tuple{Int, Float64}
@test eltype(Base.product(a, b, c)) == Tuple{Int, Float64, Float32}
@test eltype(Base.product(a, b, c, d)) == Tuple{Int, Float64, Float32, UInt8}
@test eltype(Base.product(a, b, c, d, e)) == Tuple{Int, Float64, Float32, UInt8, Int8}

# ndims
@test ndims(Base.product(a)) == 1
@test ndims(Base.product(a, b)) == 2
@test ndims(Base.product(a, b, c)) == 3
@test ndims(Base.product(a, b, c, d)) == 4
@test ndims(Base.product(a, b, c, d, e)) == 5

f, g, h = randn(1, 1), randn(1, 1, 1), randn(1, 1, 1, 1)
@test ndims(Base.product(f)) == ndims(collect(Base.product(f))) == 2
@test ndims(Base.product(g)) == ndims(collect(Base.product(g))) == 3
@test ndims(Base.product(h)) == ndims(collect(Base.product(h))) == 4
@test ndims(Base.product(f, f)) == ndims(collect(Base.product(f, f))) == 2
@test ndims(Base.product(f, g)) == ndims(collect(Base.product(f, g))) == 2
@test ndims(Base.product(g, g)) == ndims(collect(Base.product(g, g))) == 2
@test ndims(Base.product(g, h)) == ndims(collect(Base.product(g, h))) == 2
@test ndims(Base.product(h, h)) == ndims(collect(Base.product(h, h))) == 2
@test ndims(Base.product(f, f, f)) == ndims(collect(Base.product(f, f, f))) == 3
@test ndims(Base.product(f, f, g)) == ndims(collect(Base.product(f, f, g))) == 3
@test ndims(Base.product(f, g, g)) == ndims(collect(Base.product(f, g, g))) == 3
@test ndims(Base.product(g, g, g)) == ndims(collect(Base.product(g, g, g))) == 3
@test ndims(Base.product(g, g, h)) == ndims(collect(Base.product(g, g, h))) == 3
@test ndims(Base.product(g, h, h)) == ndims(collect(Base.product(g, h, h))) == 3
@test ndims(Base.product(h, h, h)) == ndims(collect(Base.product(h, h, h))) == 3
@test ndims(Base.product(f, f, f)) == ndims(collect(Base.product(f, f, f))) == 3
@test ndims(Base.product(g, g, g, g)) == ndims(collect(Base.product(g, g, g, g))) == 4
@test ndims(Base.product(h, h, h, h)) == ndims(collect(Base.product(h, h, h, h))) == 4
end

# iteratorsize trait business
let f1 = Filter(i->i>0, 1:10)
@test Base.iteratorsize(Base.product(f1)) == Base.SizeUnknown()
@test Base.iteratorsize(Base.product(1:2, f1)) == Base.SizeUnknown()
@test Base.iteratorsize(Base.product(f1, 1:2)) == Base.SizeUnknown()
@test Base.iteratorsize(Base.product(f1, f1)) == Base.SizeUnknown()
@test Base.iteratorsize(Base.product(f1, countfrom(1))) == Base.IsInfinite()
@test Base.iteratorsize(Base.product(countfrom(1), f1)) == Base.IsInfinite()
end
@test Base.iteratorsize(Base.product(1:2, countfrom(1))) == Base.IsInfinite()
@test Base.iteratorsize(Base.product(countfrom(1), 1:2)) == Base.IsInfinite()
@test Base.iteratorsize(Base.product(1:2)) == Base.HasShape()
@test Base.iteratorsize(Base.product(1:2, 1:2)) == Base.HasShape()
@test Base.iteratorsize(Base.product(take(1:2, 1), take(1:2, 1))) == Base.HasShape()
@test Base.iteratorsize(Base.product(take(1:2, 2))) == Base.HasLength()
@test Base.iteratorsize(Base.product([1 2; 3 4])) == Base.HasShape()

# iteratoreltype trait business
let f1 = Filter(i->i>0, 1:10)
@test Base.iteratoreltype(Base.product(f1)) == Base.HasEltype() # FIXME? eltype(f1) is Any
@test Base.iteratoreltype(Base.product(1:2, f1)) == Base.HasEltype() # FIXME? eltype(f1) is Any
@test Base.iteratoreltype(Base.product(f1, 1:2)) == Base.HasEltype() # FIXME? eltype(f1) is Any
@test Base.iteratoreltype(Base.product(f1, f1)) == Base.HasEltype() # FIXME? eltype(f1) is Any
@test Base.iteratoreltype(Base.product(f1, countfrom(1))) == Base.HasEltype() # FIXME? eltype(f1) is Any
@test Base.iteratoreltype(Base.product(countfrom(1), f1)) == Base.HasEltype() # FIXME? eltype(f1) is Any
end
@test Base.iteratoreltype(Base.product(1:2, countfrom(1))) == Base.HasEltype()
@test Base.iteratoreltype(Base.product(countfrom(1), 1:2)) == Base.HasEltype()
@test Base.iteratoreltype(Base.product(1:2)) == Base.HasEltype()
@test Base.iteratoreltype(Base.product(1:2, 1:2)) == Base.HasEltype()
@test Base.iteratoreltype(Base.product(take(1:2, 1), take(1:2, 1))) == Base.HasEltype()
@test Base.iteratoreltype(Base.product(take(1:2, 2))) == Base.HasEltype()
@test Base.iteratoreltype(Base.product([1 2; 3 4])) == Base.HasEltype()



# flatten
# -------
Expand Down