[Typing] `batched` revealed type?

[ego-thales]

Bug report

Bug description:

Hi!

Consider the following.

from functools import reduce
from itertools import batched

x = reduce(batched, [1], (1,))

It makes mypy unhappy!

error: Argument 1 to "reduce" has incompatible type "type[batched[Any]]"; expected "Callable[[tuple[int], int], tuple[int]]"  [arg-type]

I did try to previously type hint the tuple with

the_tuple: tuple[int, ...] = (1,)

But the error remains the same.

It may be that I'm not understanding something elementary in which case, thanks for the help! Otherwise, is this a problem of batched or something deeper?

Thanks!

CPython versions tested on:

3.13

Operating systems tested on:

Linux

[anjuraghuwanshi]

Hi @ego-thales ,

It seems like the issue is arising because the reduce function expects a binary function as its first argument, but batched from itertools is not a binary function—it's a function that returns an iterator, which is why mypy is complaining about the type mismatch.

To fix this, you should use batched directly with the iterable and not inside reduce. reduce is typically used with a binary function that takes two arguments (e.g., a summing function), but batched groups elements of the iterable into fixed-sized chunks, so it's not meant to be used in this way.

[sobolevn]

Hi, this is clearly not an issue in CPython, because it does not even have annotations :)

So, I transfered issue here. It might also be a mypy specific issue, or even not an issue at all. I didn't look deep enough.

[JelleZijlstra]

Pyright doesn't show an error here, but it produces this rather interesting type:

$ cat /tmp/bat.py 
from functools import reduce
from itertools import batched

x = reduce(batched, [1], (1,))
reveal_type(x)

$ pyright /tmp/bat.py 
/tmp/bat.py
  /tmp/bat.py:5:13 - information: Type of "x" is "batched[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[_T_co@batched, ...], ...], ...], ...], ...], ...], ...], ...], ...], ...] | Literal[1], ...], ...]] | tuple[Literal[1]] | batched[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[<Recursive>, ...], ...], ...], ...], ...], ...], ...] | Literal[1], ...], ...], ...], ...], ...], ...], ...], ...], ...], ...] | Literal[1]] | batched[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[<Recursive>, ...], ...], ...], ...], ...], ...], ...] | Literal[1], ...], ...], ...], ...], ...], ...], ...], ...], ...], ...] | tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[<Recursive>, ...], ...], ...], ...], ...] | Literal[1], ...], ...], ...], ...], ...], ...], ...], ...], ...], ...] | Literal[1], ...] | tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[_T_co@batched, ...], ...], ...], ...], ...], ...], ...], ...], ...], ...] | Literal[1], ...], ...], ...] | Literal[1]]"
0 errors, 0 warnings, 1 information 

Similarly, pyright thinks reveal_type(list(x)) is

/tmp/bat.py:6:13 - information: Type of "list(x)" is "list[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[<Recursive>, ...], ...], ...], ...], ...], ...] | Literal[1], ...], ...], ...], ...], ...], ...], ...], ...], ...], ...] | tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[<Recursive>, ...], ...], ...], ...] | Literal[1], ...], ...], ...], ...], ...], ...], ...], ...], ...], ...] | Literal[1], ...] | tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[_T_co@batched, ...], ...], ...], ...], ...], ...], ...], ...], ...], ...] | Literal[1], ...], ...], ...] | Literal[1], ...] | tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[tuple[_T_co@batched, ...], ...], ...], ...], ...], ...], ...], ...], ...], ...] | Literal[1], ...], ...], ...] | Literal[1]]"

Which is incorrect since it's [(1,)] at runtime. So there might be something wrong with the stubs here, but I haven't looked into whether it's easily fixable.

[ego-thales]

It feels like there are two reasons this happens.

First: reduce seems a bit restrictive to me.

typeshed/stdlib/functools.pyi

Lines 38 to 39 in 5caaf2e

	@overload
	def reduce(function: Callable[[_T, _S], _T], sequence: Iterable[_S], initial: _T, /) -> _T: ...

This would not be applicable to functions such as, say,

def function_append[_S](iter: Iterable[_S], append: _S) -> list[_S]:
    return list(iter) + [append]

There might be a good reason for such restriction, but I'm not type-fluent enough to say. Also, I don't really know how to fix it, we would somehow need to extend to

def reduce(function: Callable[[_U, _S], _T], sequence: Iterable[_S], initial: _U, /) -> _T: ...

with the added condition that _T be "compatible" with _U for function. I'm not sure it's possible orif I'm making sense here.

Second: batched is de facto a class with following stub.

typeshed/stdlib/itertools.pyi

Lines 329 to 336 in 5caaf2e

	class batched(Generic[_T_co]):
	if sys.version_info >= (3, 13):
	def __new__(cls, iterable: Iterable[_T_co], n: int, *, strict: bool = False) -> Self: ...
	else:
	def __new__(cls, iterable: Iterable[_T_co], n: int) -> Self: ...
	def __iter__(self) -> Self: ...
	def __next__(self) -> tuple[_T_co, ...]: ...

Which obsiouvly doesn't sit well with the Callable expected by reduce. I don't know how to fix this.

[sobolevn]

@ego-thales all classes are callables. So, it is not really a problem.

[ego-thales]

Ok, thanks for confirming that mypy knows that. It should be pretty obvious from the __new__-specific hints (by the way, is there a good reason to type hint __new__ instead of __init__ in general?).

So in the end, I think the focus should turn to the restrictive stub for reduce. Would it be fair to used the following?

_S = TypeVar("_S")
_T = TypeVar("_T")
_U = TypeVar("_U", bound=_T)

@overload
def reduce(function: Callable[[_T, _S], _U], sequence: Iterable[_S], initial: _T, /) -> _U: ...

[JelleZijlstra]

is there a good reason to type hint __new__ instead of __init__ in general?).

If the runtime class has __new__, add a type hint for __new__; otherwise, don't. Stubs should reflect the runtime.

Would it be fair to used the following?

TypeVars can't have bounds that depend on other TypeVars. Possibly a TypeVar default (PEP 696) would work but I haven't fully thought it through.