-
Notifications
You must be signed in to change notification settings - Fork 193
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimize dataclasses with __slots__ #50
Comments
Gonna give this one a shot 👍 |
>>> @dataclass
... class Data:
... t: int
... m: str
... n: float
...
>>> Data(t=3, m=3, n=3).__sizeof__()
32
>>> @dataslots.with_slots
... @dataclass
... class Data:
... t: int
... m: str
... n: float
...
>>> Data(t=3, m=3, n=3).__sizeof__()
40 Really not seeing much in the ways of benefits here. Attribute access is marginally faster as is instantiation but this doesn't seem like a worthwhile trade off against the extra overhead and memory size. I might be seriously missing something or it might be Python 3.9 doing stuff not too sure though. |
I'm curious did you find a measurable speed improvement? Admittedly I've not done enough investigation to validate this, but my thinking was the if the technique from dataslots were implemented directly in the Message class then it should give the most benefit. |
The there didn't appear to be a point at which the un-slotted class used more memory. The speed improvement wasn't noteworthy. Might have to test some more however. |
Actually I take this back it appears to be working now 🤔 |
All tests were ran with timeit.repeat and a sped up custom implementation of dataslots. So for a single slotted class attribute access 0.11287586920000003s vs unslotted 0.1498976172s Init appears to be slightly slower slotted 25.6038842112s vs 22.337924385599997s. Overhead is relatively low at 3.2458480566e-05s per class per decorator directly using the implementation of the above library finishes in 3.7460388750999996e-05s Memory usage temperamental which I think is caused by the use of |
The size issues might be due to using it on instances this sure is very strange. |
I'm gonna give a Message metaclass a go for the best performance as you suggested. |
Adding __slots__ to a class improves memory usage and performance of attribute access, at the expense of not being able to set arbitrary attributes on the resulting objects.
This project demonstrates how dataclasses can be augmented to use slots, whilst still having class attributes set (normally adding the slots attribute to a class definition would cause an error with dataclassess that declare default values).
Making generated dataclasses use slots in this way would be a small breaking change, though I don't see how it could be too inconvenient to work with, and I expect the performance improvement would be worthwhile.
The text was updated successfully, but these errors were encountered: