-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove RLock from BZ2File #87951
Comments
The original issue is reported here.
lzma and gzip don't use RLock(). It adds significant performance overhead.
BZ2File.readline() is implemented in C so it is slightly slow than C implementation. If I add this def __iter__(self):
self._check_can_read()
return iter(self._buffer) If this __iter__ method is safe, it can be added to gzip and lzma too. |
I will create a separated issue for __iter__, because it is same to gzip and lzma. |
This change is backwards incompatible, it may break some code silently. If someone really needs better performance, they can write a BZ2File class without RLock by themselves, it should be easy. FYI, zlib module was added in 1997, bz2 module was added in 2002, lzma module was added in 2011. (Just curious for these years) |
I'm not worried about compatibility on this for 3.10. Nobody realistically expects to be able to have multiple readers from a single BZ2File stream. They don't for the gzip or lzma equivalents. That isn't even a normal thing to do on an actual file. Lets go forward with this for 3.10betas and see if anyone even notices. I doubt they will. But the addition of __iter__ deferring to iter(self._buffer) belongs in its own PR and issue and should be done for all of bz2, gzip, and lzma all at once. I've edited the PR branch to remove __iter__ and cleanup a couple of minor nits. |
I see you're already tracking __iter__ in https://bugs.python.org/issue43787, perfect! :) |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: