Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add typing and enforce checking via tox/CI #135

Merged
merged 1 commit into from Dec 23, 2021

Conversation

pgjones
Copy link
Member

@pgjones pgjones commented Aug 28, 2021

This uses the same mypy settings as wsproto.

The Sentinel values are problematic, but I've found no good solution
that also has the property that type(Sentinel) is Sentinel - so this
should suffice for now.

Whilst I've been lazy with the tests, I've mostly avoided type ignores
in the main code. This should ensure that mypyc can be used if
desired.

@pgjones pgjones force-pushed the typing branch 2 times, most recently from 9fb1a67 to dfd06ba Compare August 28, 2021 16:24
__all__ += _events.__all__
__all__ += _connection.__all__
__all__ += _state.__all__
__all__ = (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change already is going to do a ton for discoverability 🎉 IDEs don't like dynamic __all__.

Copy link
Member

@sethmlarson sethmlarson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall this looks good. What are your thoughts on having the edges of the API be fully type-hinted but using # type: ignore[...] more liberally within APIs to not require using assert, isinstance, etc? Wondering if these changes would impact performance.

h11/_connection.py Outdated Show resolved Hide resolved
return [(raw_name, value) for raw_name, _, value in self._full_items]


def normalize_and_validate(headers, _parsed=False):
HeaderTypes = Union[
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this be List[Tuple[Union[str, bytes], Union[str, bytes]] instead of the individual permutations?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mypy seems to get confused by that (although technically it is more correct as it doesn't need to uniformly the same type in the tuple for all elements of the list) - I think this covers enough of the likely use cases and mypy understands it.

h11/_headers.py Outdated Show resolved Hide resolved
h11/_readers.py Outdated Show resolved Hide resolved
h11/_readers.py Outdated Show resolved Hide resolved
h11/_writers.py Outdated Show resolved Hide resolved
h11/_writers.py Outdated Show resolved Hide resolved
@pgjones
Copy link
Member Author

pgjones commented Aug 29, 2021

What are your thoughts on having the edges of the API be fully type-hinted but using # type: ignore[...] more liberally within APIs

I'm hoping to use mypyc next to gain more performance (it was reasonably effective 2 years ago). So I'm not keen on this unless it doesn't affect mypyc.

h11/_util.py Outdated Show resolved Hide resolved
This uses the same mypy settings as wsproto.

The Sentinel values are problematic, but I've found no good solution
that also has the property that type(Sentinel) is Sentinel - so this
should suffice for now.

Whilst I've been lazy with the tests, I've mostly avoided type ignores
in the main code. This should ensure that mypyc can be used if
desired.
SERVER = make_sentinel("SERVER")

class CLIENT(Sentinel, metaclass=Sentinel):
pass
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised that metaclass=... is needed here.

(I guess I would have assumed that inheriting from a class with a custom __new__ method was sufficient.)

Not sure this is an actionable comment, but just "this is a bit fiddly, innit?" 🙂

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea it is. There is some extra discussion here.

class_ = InformationalResponse if status_code < 200 else Response
http_version = (
b"1.1" if matches["http_version"] is None else matches["http_version"]
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an interesting change. Much neater/more obvious this way, now that we're not passing an open-ended **matches to the response class.

@@ -113,7 +135,7 @@ def __call__(self, buf):
self._remaining -= len(data)
return Data(data=data)

def read_eof(self):
def read_eof(self) -> NoReturn:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL: typing.NoReturn

assert normalize_data_events(
[
Data(data=bytearray(b"1")),
Data(data=b"2"),
Response(status_code=200, headers=[]),
Response(status_code=200, headers=[]), # type: ignore[arg-type]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious about these ignore cases, since it looks correct to me?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think mypy gets confused about the type of the empty list. There is an open bug I can't find at the moment

Callable[[Union[InformationalResponse, Response], Writer], None],
Callable[[Request, Writer], None],
],
]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this typing here helps illuminate that the WRITERS usage here is a bit fiddly, with it's special case for SEND_BODY. 😀
Not actionable tho - just a passing observation.


@overload
def normalize_and_validate(headers: HeaderTypes, _parsed: Literal[False]) -> Headers:
...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand this overloading?
What's with the Literal[True], Literal[False], bool cases?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It allows mypy to determine that the type of headers changes depending on _parsed being True or False. See also this

Copy link
Contributor

@tomchristie tomchristie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed as best as possible.
Pretty substantial bit of work here. 👍

A few passing comments along the way but no obvious blockers.

@pgjones
Copy link
Member Author

pgjones commented Dec 22, 2021

Thanks, I'll merge tomorrow

@pgjones
Copy link
Member Author

pgjones commented Dec 23, 2021

Thanks again for the reviews and comments 😄

@pgjones pgjones merged commit fe249c9 into python-hyper:master Dec 23, 2021
@pgjones pgjones deleted the typing branch December 23, 2021 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants