Recognize comparison with Ellipsis

[davidfstr]

Consider the following program:

import builtins
from typing import Any, Optional, Union

def func1() -> Optional[str]:
    multi_state = 'value'  # type: Optional[str]
    if multi_state is None:
        return None
    reveal_type(multi_state)
    return multi_state

def func2() -> Any:
    multi_state = 'value'  # type: Union[str, builtins.ellipsis]
    if multi_state is Ellipsis:
        return None
    reveal_type(multi_state)
    return multi_state

func1()
func2()

If I typecheck with:

$ mypy --strict-optional --suppress-error-context test_recognize_ellipsis_comparison.py

I get:

test_recognize_ellipsis_comparison.py:8: error: Revealed type is 'builtins.str'
test_recognize_ellipsis_comparison.py:15: error: Revealed type is 'Union[builtins.str, builtins.ellipsis]'

But expect:

test_recognize_ellipsis_comparison.py:8: error: Revealed type is 'builtins.str'
test_recognize_ellipsis_comparison.py:15: error: Revealed type is 'builtins.str'

This suggests that mypy does not recognize the comparison value is Ellipsis in the same way that it recognizes value is None.

In my own codebase having explicit recognition for Ellipsis would be useful in some functions that use both None and Ellipsis as sentinel values.

[gvanrossum]

I believe we are already tracking this in #1803.

[davidfstr]

#1803 only talks about Enums, not the Ellipsis constant. Is Ellipsis internally implemented as an Enum?

[gvanrossum]

The PEP just says "singleton types" and although it only gives None and enums as examples I think it applies here too. I'll add a note to #1803 just to be sure.

Do note though, there is no actual type builtins.ellipsis at runtime (it's a figment of the stubs). Do you have to use Ellipsis for this?

[davidfstr]

Do note though, there is no actual type builtins.ellipsis at runtime (it's a figment of the stubs).

I tried to use type(Ellipsis) but mypy's parser choked on it. Is there a more reliable symbol than builtins.ellipsis, which apparently is an implementation detail?

Do you have to use Ellipsis for this?

Since you're curious: The specific function that prompted this issue does an os.walk of a directory to locate all contained files. It returns a Mapping[FileName, Union[FilePath, builtins.ellipsis]] that allows O(1) mapping of a short filename to a filepath. If you do a get() on the dictionary there are 3 cases:

1. None: No file with the specified name exists.
2. Ellipsis: Multiple files with the specified name exist. An ambiguous reference.
3. FilePath: There is a unique file with the specified name and this is its path.

[gvanrossum]

Hm. I recommend defining some dedicated constant for that. It doesn't look like using Ellipsis here has advantages other than not needing to define/import it. Also note that the type of None is similarly hard to get at, but PEP 484 special-cases it. There are other similar singletons, e.g. NotImplemented. All these probably deserve their types named and exported from either builtins or types in Python itself (not just in the stubs).

[relikd]

I'd like to repoen this issue. Here is a legitimate example where a constant is not needed.

def test() -> None:
    arr = {}  # type: Dict[str, Optional[List]]
    val = arr.get('key', Ellipsis)
    if val is Ellipsis:
        return
    if val is None:
        val = [1, 2, 3]
        arr['key'] = val

    for i in val:
        print(i)

mypy: error union-attr - Item "ellipsis" of "Union[List[Any], ellipsis]" has no attribute "__iter__" (not iterable)

Interestingly, using NotImplemented is fine but I don't want to use non-descriptive and potentially misleading flags solely to make mypy happy.

[erictraut]

This is an incredibly specific and rare edge case. I wouldn't think that this merits custom type narrowing logic in a type checker — especially given that there is a simple workaround.

    if isinstance(val, ellipsis):

[relikd]

The Ellipsis is just a placeholder, I do not specifically need the Ellipsis here. The only thing that bugs me is why does mypy not recognize if I specifically test for the same variable. And testing for isinstance(ellipsis) requires an additional import whereas Ellipsis is built-in. But if you say it is not going to change, then fine.

[erictraut]

Mypy evaluates types of expressions and symbols. It doesn't track specific values. Mypy knows that the type of val is builtin.ellipsis, but it doesn't know that it is the same value as Ellipsis. We happen to know that it's the same value (the same instance), but a static type checker cannot make that assumption.

If you want to avoid the additional import, you can do this:

if isinstance(val, type(Ellipsis)):