Unpacking tuples of variable length

[mihneagiurgea]

Given the following code:

from typing import Any, Dict, Tuple, Union

VarTuple = Union[Tuple[int, int], Tuple[int, int, int]]

def make_tuple():
    # type: () -> VarTuple
    return None

x = make_tuple()
a = b = c = 0
if len(x) == 3:
    a, b, c = x
else:
    a, b = x
    c = 0

mypy complains about the following error:

sandbox.py:12: error: 'Union[Tuple[builtins.int, builtins.int], Tuple[builtins.int, builtins.int, builtins.int]]' object is not iterable
sandbox.py:14: error: 'Union[Tuple[builtins.int, builtins.int], Tuple[builtins.int, builtins.int, builtins.int]]' object is not iterable

[JukkaL]

The error message is incorrect. However, mypy may not be able to type check this properly since it doesn't recognize the len(x) == 3 check. So there seem to be two related issues:

• Union types aren't recognized as iterables when type checking assignment statements.
• len(x) == n checks can't be used to narrow down tuple types or unions. These could work similarly to isinstance checks.

The recommended way to work around the issue is to use casts:

if len(x) == 3:
    a, b, c = cast(Tuple[int, int, int], x)
else:
    a, b = cast(Tuple[int, int], x)
    c = 0

[JukkaL]

Also, mypy could recognize len(x) when x is a variable-length tuple. Example:

def f(x: Tuple[int, ...]) -> None:
    assert len(x) == 3 
    # Type of x should be Tuple[int, int, int] now
    ...

[neiljp]

I just ran into this issue and was going to post the following test case, but didn't realise that len(x) wasn't yet a way to distinguish among unions:

def f_4(a: Tuple[int, int, int, int]) -> None:
    pass

def f_2_4(a: Union[Tuple[int, int], Tuple[int, int, int, int]]) -> None:
    if len(a) == 2:
        aa = a + (0,0)
    else:
        aa = a
    f_4(aa)

[pgunn]

I'm getting a different error, but I'm guessing this is the same issue:

#!/usr/bin/env python

from typing import Tuple, Union

####################
# test

def patient_caller():
	a,b,c = annoying_returner(calculate_more=True)
	print(f"I got {a}, {b}, {c}")

def annoying_returner(calculate_more:bool=False) -> Union[Tuple[int,int], Tuple[int,int,int]]:
	if calculate_more:
		return 1,2,3
	else:
		return 4,5

def main():
	patient_caller()

##########

main()

mypytest.py:9: error: Need more than 2 values to unpack (3 expected)

(I have a mypy.ini specifying check_untyped_defs)

[ilevkivskyi]

@pgunn In your particular case you can use @overload on Literal[True]/Literal[False] for the flag.

[pgunn]

@ilevkivskyi Will look into that; unfortunate that it's actually necessary, but it's great that it's possible. Thanks.

[pgunn]

@ilevkivskyi I'm trying to follow the docs here:
https://mypy.readthedocs.io/en/latest/literal_types.html
And I'm getting a

caiman/components_evaluation.py:539: error: Overloaded function signatures 1 and 2 overlap with incompatible return types

This is also getting very verbose. Not sure if it's that these are keyword rather than positional arguments that's giving me the problem.

# The below tells the type-checker that if return_all is a literal True, it returns 5 values
@overload
def estimate_components_quality(traces, Y, A, C, b, f, final_frate=30, Npeaks=10, r_values_min=.95,
                                fitness_min=-100, fitness_delta_min=-100, return_all:Literal[True]=True, N=5,
                                remove_baseline=True, dview=None, robust_std=False, Athresh=0.1, thresh_C=0.3, num_traces_per_group=20) -> Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.array]: ...
# and if it's a literal False, it returns two
@overload
def estimate_components_quality(traces, Y, A, C, b, f, final_frate=30, Npeaks=10, r_values_min=.95,
                                fitness_min=-100, fitness_delta_min=-100, return_all:Literal[False]=False, N=5,
                                remove_baseline=True, dview=None, robust_std=False, Athresh=0.1, thresh_C=0.3, num_traces_per_group=20) -> Tuple[np.ndarray, np.ndarray]: ...
# Fallback overload
def estimate_components_quality(traces, Y, A, C, b, f, final_frate=30, Npeaks=10, r_values_min=.95,
                                fitness_min=-100, fitness_delta_min=-100, return_all:bool=False, N=5,
                                remove_baseline=True, dview=None, robust_std=False, Athresh=0.1, thresh_C=0.3, num_traces_per_group=20) -> Union[Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.array], Tuple[np.ndarray, np.ndarray]]: ...

def estimate_components_quality(traces, Y, A, C, b, f, final_frate=30, Npeaks=10, r_values_min=.95,
                                fitness_min=-100, fitness_delta_min=-100, return_all:bool=False, N=5,
                                remove_baseline=True, dview=None, robust_std=False, Athresh=0.1, thresh_C=0.3, num_traces_per_group=20) -> Union[Tuple[np.ndarray, np.ndarray, np.ndarray, np.ndarray, np.array], Tuple[np.ndarray, np.ndarray]]:
     # Function body

This may be wandering away from the topic of the issue though; if it's wandered too far I'm fine with not getting an answer; don't want to clutter things for devs.

[ilevkivskyi]

Yes, this is because they are keyword args, see #6580

Anyway, in your case I would just return a less precise type like Tuple[np.ndarray, ...] (if this works).

[pgunn]

@ilevkivskyi It does. Thanks! I have some other code not well-covered by this tack, but perhaps by the time we're ready to switch to requiring mypy to pass in CI before merges, this kind of thing will have a more graceful solution.

[goodmami]

There seems to be an issue even with fixed-length tuples:

# file: ex.py
from typing import Tuple
points = (1, 2)  # type: Tuple[int, int]
x, y, z = *points, 0

This results in incorrect inference about the tuple size:

$ mypy ex.py 
ex.py:4: error: Need more than 2 values to unpack (3 expected)
Found 1 error in 1 file (checked 1 source file)

A workaround is to use cast(), as @JukkaL suggested earlier in this thread:

x, y, z = cast(Tuple[int, int, int], (*points, 3))

[ethanhs]

@goodmami could you open a new issue for that? It seems like a different (enough) problem.

[ilevkivskyi]

I think this is a missing feature rather than a bug (although this does cause a false positive).

[rodo-hf]

If your list to unpack is never greater than your left side and you know how many variables are on the left side, then you can do this (assuming there are 5 variables on the left side):

list_to_unpack = [1, 2, 3]

a, b, c, d, e = list_to_unpack + (5 - len(list_to_unpack)) * [None]

print(a)
print(b)
print(c)
print(d)
print(e)

# output
# 1
# 2
# 3
# None
# None

if you also want to catch the cases where the list is bigger than the variables, then you can do as follows:

list_to_unpack = [1, 2, 3, 4, 5, 6]
count_variables = 5
a, b, c, d, e = list(list_to_unpack + (count_variables - len(list_to_unpack)) * [None])[:count_variables]

[wxgeo]

I ran into a similar issue today.

Minimal example:

transition: tuple[int, int] | tuple[int, int, str] = (1, 1, "a")
if len(transition) == 3: 
    print(transition[2])

An error is raised by mypy for last line: error: Tuple index out of range [misc]

[JelleZijlstra]

This is getting very off-topic. Let's leave this issue focused on narrowing tuple types with len().

[w238liu]

Just wondering if this issue has been resolved? I still get a mypy error for the following code snippet:

from typing import Tuple


def f0(n: int) -> Tuple[int, ...]:
    return tuple(range(n))


def f2() -> Tuple[int, int]:
    ret = f0(2)

    assert len(ret) == 2

    return ret


if __name__ == '__main__':
    print(f2())

Save the code in tmp/mypy_test/main.py and run mypy tmp/mypy_test/main.py, I get the following error message:
tmp/mypy_test/main.py:13: error: Incompatible return value type (got "Tuple[int, ...]", expected "Tuple[int, int]") [return-value]

[JelleZijlstra]

It's not resolved, that's why this issue is still open.

[w238liu]

Thanks for quick reply. I’ll use the workaround mentioned in this thread then.

…

On Fri, Aug 25, 2023 at 4:41 PM Jelle Zijlstra ***@***.***> wrote: It's not resolved, that's why this issue is still open. — Reply to this email directly, view it on GitHub <#1178 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASLXBGMUAWX5U76IV23HOWTXXEEXNANCNFSM4BZXWZWQ> . You are receiving this because you commented.Message ID: ***@***.***>