Refactor WSMessage to use tagged unions #7319
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
✅ All tests successful. No failed tests found. Additional details and impacted files@@ Coverage Diff @@
## master #7319 +/- ##
==========================================
- Coverage 98.60% 98.59% -0.01%
==========================================
Files 108 108
Lines 35160 35207 +47
Branches 4184 4184
==========================================
+ Hits 34668 34714 +46
- Misses 329 330 +1
Partials 163 163
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
Creating the dataclasses will be significantly slower than the NamedTuple, but the property accesses will be faster...probably need to benchmark it to see what the tradeoff will be |
for more information, see https://pre-commit.ci
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
A quick test locally suggests that NamedTuple will also work with unions. Let me know which you want to go with and I'll update (and then try to revert the micro-optimisations that got caught in conflict resolutions). |
|
Also worth noting that with dataclass we get errors because DataQueue expects a Sized thing. This suggests to me that DataQueue is probably not the best solution here as there seems to be no meaning to the size when processing WS messages. Maybe an unsized queue could be used here to be a little more optimised. |
|
I'll write a benchmark for all the options later today |
|
Options compared from dataclasses import dataclass
from typing import NamedTuple
from aiohttp.http_websocket import WSMsgType
from typing import Optional, Literal
import timeit
BINARY_TYPE = WSMsgType.BINARY
@dataclass
class _WSMessageDataClass:
data: object
type: WSMsgType
extra: Optional[str] = None
@dataclass
class WSMessageBinaryDataClass(_WSMessageDataClass):
data: bytes
type: Literal[WSMsgType.BINARY] = BINARY_TYPE
@dataclass(frozen=True)
class _WSMessageFrozenDataClass:
data: object
type: WSMsgType
extra: Optional[str] = None
@dataclass(frozen=True)
class WSMessageBinaryFrozenDataClass(_WSMessageFrozenDataClass):
data: bytes
type: Literal[WSMsgType.BINARY] = BINARY_TYPE
class _WSMessageNamedTuple(NamedTuple):
data: object
type: WSMsgType
extra: Optional[str] = None
class WSMessageBinaryNamedTuple(_WSMessageNamedTuple):
data: bytes
type: Literal[WSMsgType.BINARY]
data = b"Hello, world!"
named_tuple_direct = timeit.timeit(
"x=WSMessageBinaryNamedTuple(data, BINARY_TYPE);x.data;x.type",
globals=locals(),
number=1000000,
)
named_tuple_tuple_new = timeit.timeit(
"x=tuple.__new__(WSMessageBinaryNamedTuple, (data, BINARY_TYPE));x.data;x.type",
globals=locals(),
number=1000000,
)
dataclass_direct = timeit.timeit(
"x=WSMessageBinaryDataClass(data);x.data;x.type", globals=locals(), number=1000000
)
frozen_dataclass_direct = timeit.timeit(
"x=WSMessageBinaryFrozenDataClass(data);x.data;x.type",
globals=locals(),
number=1000000,
)
print(f"named_tuple_direct: {named_tuple_direct}")
print(f"named_tuple_tuple_new: {named_tuple_tuple_new}")
print(f"dataclass_direct: {dataclass_direct}")
print(f"frozen_dataclass_direct: {frozen_dataclass_direct}")named_tuple_direct: 0.13957041699904948
|
psf-chronographer
bot
added
the
bot:chronographer:provided
for more information, see https://pre-commit.ci
CodSpeed Performance ReportMerging #7319 will not alter performanceComparing Summary
|
|
If you rebase, you should get a better idea of the performance impact, however from looking at the https://codspeed.io/aio-libs/aiohttp/branches/websocket_benchmarks it looks like changes here may be shadowed by the performance of |
|
Think we're all good now. |
This reworks WSMessage to be a union of dataclasses which can provide much better type safety using tagged unions (on the
typeattribute).I need to go over a couple of details (like what should be exported and from where, imports are a little messy currently). But, my main question before I finish it off, is whether it makes sense to have
.json()only on the TEXT/BINARY messages? If we do this, it becomes easy for mypy to validate the code, requiring amsg.type is WSMsgType.TEXTcheck before calling.json().Fixes #7313