Added Scanner to re.pyi

[jonathanslenders]

No description provided.

[srittau]

Thanks, left a few comments below.

[srittau]

Please add an undocumented flag. Also, this needs to be generic over AnyStr for the methods to have correct types:

Suggested change

	class Scanner:
	# undocumented
	class Scanner(Generic[AnyStr]):

[srittau]

It seems the action can be anything, not just a callable:

Suggested change

	lexicon: Sequence[Tuple[AnyStr, Callable[[Scanner, AnyStr], Any]]],
	lexicon: Sequence[Tuple[AnyStr, Any]],

Alternatively, another TypeVar for the return value might make sense, in which case this would be:

Suggested change

	lexicon: Sequence[Tuple[AnyStr, Callable[[Scanner, AnyStr], Any]]],
	lexicon: Sequence[Tuple[AnyStr, Union[None, _T, Callable[[Scanner, AnyStr], Optional[_T]]]],

[srittau]

In the case of introducing another type var, List[Any] -> List[_T].

[jonathanslenders]

Would this make sense?

# undocumented

_T = TypeVar('_T', bound=AnyStr)
_T2 = TypeVar('_T2')

class Scanner(Generic[_T, _T2]):
    def __init__(
        self,
        lexicon: Sequence[Tuple[_T, Union[None, _T2, Callable[[Scanner, _T], Optional[_T2]]]]],
        flags: _FlagsType = ...,
    ) -> None: ...
    def scan(self, string: _T) -> Tuple[List[_T2], _T]: ...

Should we break it down in smaller pieces?

[srittau]

Looks good with two changes: The # undocumented should directly precede the class and AnyStr is already a type-var, so instead of _T you can use AnyStr directly (and don't need to list it in Generic).

[jonathanslenders]

So, something like this would do it?

_T = TypeVar('_T')

# undocumented

class Scanner(Generic[_T]):
    def __init__(
        self,
        lexicon: Sequence[Tuple[AnyStr, Union[None, _T, Callable[[Scanner, AnyStr], Optional[_T]]]]],
        flags: _FlagsType = ...,
    ) -> None: ...
    def scan(self, string: AnyStr) -> Tuple[List[_T], AnyStr]: ...

What is the reason that AnyStr does not have to be listed in the Generic, while the other type variable should?

[JelleZijlstra]

What is the reason that AnyStr does not have to be listed in the Generic, while the other type variable should?

In this case you have a generic method on a generic class. The class is generic over _T, and the method is generic over AnyStr. So for example, if you have a Scanner[int], scanner.scan("") would return a Tuple[List[int], str], and scanner.scan(b"") would return a Tuple[List[int], bytes].

[srittau]

Also, flake8 fails because of white space in line 156. @jonathanslenders Are you still interested in this PR?

[srittau]

Closing for now, please reopen if you find time to work on this.