mypy complains if itertools.chain.from_iterable is given an iterator, tuple, or list.

[fgregg]

chunks = ([1] * 3 for _ in range(10))
flattened = itertools.chain.from_iterable(chunks)

Is valid python code, but mypy throws this error:

error: Argument 1 to "from_iterable" of "chain" has incompatible type Iterator[Iterable[...]]; expected Iterable[Iterable[...]]

[gvanrossum]

Hm... Maybe Iterable should be covariant?

On Saturday, September 3, 2016, Forest Gregg notifications@github.com
wrote:

chunks = ([1] * 3 for _ in range(10))
flattened = itertools.chain.from_iterable(chunks)

Is valid python code, but mypy throws this error:

error: Argument 1 to "from_iterable" of "chain" has incompatible type Iterator[Iterable[...]]; expected Iterable[Iterable[...]]

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#521, or mute the thread
https://github.com/notifications/unsubscribe-auth/ACwrMlsjou1jgo7_9WlbSPodSDHT-0zsks5qmcmqgaJpZM4J0YD3
.

--Guido (mobile)

[fgregg]

Also failing:

> chunks = tuple([1] * 3 for _ in range(10))
> flattened = itertools.chain.from_iterable(chunks)
error: Argument 1 to "from_iterable" of "chain" has incompatible type Tuple[Iterable[...], ...]; expected Iterable[Iterable[...]]

> chunks = list([1] * 3 for _ in range(10))
> flattened = itertools.chain.from_iterable(chunks)
error: Argument 1 to "from_iterable" of "chain" has incompatible type List[Iterable[...]]; expected Iterable[Iterable[...]]

[gvanrossum]

I can't actually repro this -- which version of mypy are you using? Can you show a complete program that reproduces the problem?

[fgregg]

Here's code that reproduces the problem I ran into.

import itertools
from typing import Iterable, Sequence, Tuple

def flat_pairs(blocks : Iterable[Sequence[int]]) -> Iterable[Tuple[int, int]]:
    pairs = (itertools.combinations(block, 2) for block in blocks)
    return itertools.chain.from_iterable(pairs)

print(list(flat_pairs([[1, 2, 3], [4, 5, 6]])))

error: Argument 1 to "from_iterable" of "chain" has incompatible type Iterator[...]; expected Iterable[...]

mypy 0.4.4

[gvanrossum]

I can repro, and I've reduced it to:

from typing import *

def flat_pairs(pairs: Iterator[Sequence[int]]) -> Iterable[Tuple[int, int]]:
    return pairs

The type of pairs here is what is returned by the genexpr in your original repro.

The error message from mypy is about the return, and the reason is that Sequence[int] is not a subtype of Tuple[int, int].

That doesn't seem to be a bug in mypy or typeshed, but in your code (or in your expectations of how mypy would work).

Note that the error is the same if I change Iterator to Iterable, so that's not the reason either.

Also note that putting reveal_type() calls in various places will help show the exact type (in case the error message suppresses some of the type).

[fgregg]

In my flat_pairs, if the function is given a iterable of sequences of integers it will return a iterable of 2-tuples of integers. This is not true of your flat_pairs

It does seem strange that I can't express that fact with typing and mypy. Strange things are often true, and I accept that it is my expectation that is buggy.

It would be very good to have documentation of what types of things are inexpressible.

[fgregg]

It seems like the breakdown is in itertools.combinations(iterable, r) which produces an iterable of r-tuples. The current typeshed stub for combinations is:

def combinations(iterable: Iterable[_T], r: int) -> Iterable[Sequence[_T]]: ...

My expectations would be met if it was possible to do something like

def combinations(iterable: Iterable[_T], r: int) -> Iterable[Tuple[_T * r]]: ...

Where the n of the n-tuples in the return type depends upon r.

I can imagine why that would be quite difficult.

[gvanrossum]

I think that would require a concept named "dependent types" where the
type of a return value (or another argument) depends on the value of an
argument. This is not something that can be expressed in PEP 484.

[fgregg]

In documentation, itertool's product might be an excellent examples of how dependent types are not supported. This is a commonly used function that's easy to explain (slightly easier, I think, then combinations or permutations).

[gvanrossum]

Opened python/mypy#2115