-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ZlibStorage.iterator() leaks the underlying iterator #4
Labels
Comments
jamadden
added a commit
that referenced
this issue
Jan 5, 2017
Fixes #4. Also add PyPy support. Modernize the .travis.yml: sudo false, pip caching, same testrunner as tox.ini. Pin ZODB/ZEO to old versions because ServerZlibStorage is broken for ZODB 5. Will address in a subsequent PR.
jamadden
added a commit
that referenced
this issue
Jan 5, 2017
Fixes #4. Also add PyPy support. Modernize the .travis.yml: sudo false, pip caching, same testrunner as tox.ini. Pin ZODB/ZEO to old versions because ServerZlibStorage is broken for ZODB 5. Will address in a subsequent PR.
jamadden
added a commit
that referenced
this issue
Jan 5, 2017
Fixes #4. Also add PyPy support. Modernize the .travis.yml: sudo false, pip caching, same testrunner as tox.ini. Pin ZODB/ZEO to old versions because ServerZlibStorage is broken for ZODB 5. Will address in a subsequent PR.
jamadden
added a commit
that referenced
this issue
Jan 5, 2017
Fixes #4. Also add PyPy support. Modernize the .travis.yml: sudo false, pip caching, same testrunner as tox.ini. Pin ZODB/ZEO to old versions because ServerZlibStorage is broken for ZODB 5. Will address in a subsequent PR.
jamadden
added a commit
that referenced
this issue
Jan 5, 2017
Fixes #4. Also add PyPy support. Modernize the .travis.yml: sudo false, pip caching, same testrunner as tox.ini. Pin ZODB/ZEO to old versions because ServerZlibStorage is broken for ZODB 5. Will address in a subsequent PR.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ZlibStorage.iterator is implemented as a generator, calling the underlying storage's iterator method. Because it's a generator, it doesn't expose the
close
method (if any) of the underlying iterator. This can lead to resources being leaked; at the very least, it's dependent on the CPython garbage collector for timely cleanup (which can be very tricky with generators).For example, in RelStorage, the
zodbconvert
utility uses the storage's iterator to see if it has data. The tests methods that use zodbconvert with a zlibstorage wrapping a FileStorage produce this ResourceWarning (under 3.6 with tracemalloc to get a stacktrace):close
isn't part of the storage iterator interface, AFAICS, but FileStorage's iterator provides aclose
method, as does the iterator provided by RelStorage. ZEO's doesn't, instead relying on a manual-ish_iterator_gc
method (but maybe it could benefit from one?).I'm happy to provide a PR to address this, possibly by making the returned iterator a new class that proxies unknown attributes to the underlying iterator.
EDIT: Or we could just use a
try/finally
block. But if memory serves, it's the try/finally blocks in a generator that lead to GC trouble...So we could catchGeneratorExit
instead, but that's actually discouraged in the release notes.The text was updated successfully, but these errors were encountered: