-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for ZODB on PyPy #20
Support for ZODB on PyPy #20
Conversation
…tion needs to bypass the _setattr_ implementation when working with its own internal state variables. This much gets the ZODB broken tests to pass. Need to figure out how to test this locally.
…the C version in more cases. Also, make the behaviour of comparisons and hashing match as well; test all this.
…'s testCache passes. Required some small updates and additions to our own test cases; the update_object_size_estimation is not tested here yet.
…for _p_changed in pure-python.
…es and ghosts the evicted objects.
…, and clear out references to ejected objects when done. This makes the ZODB testConnection and testConnectionSavepoint tests pass but is not tested here.
… be changed. ZODB depends on this.
…t. There are some interconnected lifetime issues that this solves for ZODB's cache tests, and what appear to be some genuine bugs in invalidation.
…on tests depend on it.
…tion needs to bypass the _setattr_ implementation when working with its own internal state variables. This much gets the ZODB broken tests to pass. Need to figure out how to test this locally.
…'s testCache passes. Required some small updates and additions to our own test cases; the update_object_size_estimation is not tested here yet.
…for _p_changed in pure-python.
…es and ghosts the evicted objects.
…, and clear out references to ejected objects when done. This makes the ZODB testConnection and testConnectionSavepoint tests pass but is not tested here.
… be changed. ZODB depends on this.
…t. There are some interconnected lifetime issues that this solves for ZODB's cache tests, and what appear to be some genuine bugs in invalidation.
…on tests depend on it.
I was actually able to restore 100% coverage pretty easily, but I'm not sure that really covers all of ZODB's use cases. |
Speaking of test coverage, I couldn't stop thinking about Hypothesis when I was looking at your recent series of pull requests. Sounds like the perfect tool to discover differences between C and Python implementations, e.g. the timestamp second rounding mismatch. Unfortunately I don't have time to play with it :( |
… tox/virtualenv/pip come together soon to allow official testing and mention.
… I'm not sure why I was reaching into the internals of the Python implementation.
…hon2. The existing behaviour matches the C implementation.
from .. import ring | ||
|
||
if _is_pypy or os.getenv('USING_CFFI'): | ||
assert ring.Ring is ring._CFFIRing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you can use 'self.assertTrue(ring.Ring is ring._CFFIRing)`.
…OSA for consistency; also avoid some duplicate calls through __getattribute__ which were more obvious in the OGA form.
…he_size as specified in the interface.
Whew! That's a lot of back and forth -- thanks for the cheerful effort! I have opened issues for the non-consensus issues here:
|
Thank you, @tseaver and everyone else involved. It's been a learning experience and the interactions have been great. |
Whow, I saw this thread way too late. Extremely well thought things going on. |
…ally. Performance is way up again, beating CPython on half of the tests: "Transaction", mysql "Add 3000 Objects", 8259 "Update 3000 Objects", 9454 "Read 3000 Warm Objects", 5460 "Read 3000 Cold Objects", 5454 "Read 3000 Hot Objects", 24943 "Read 3000 Steamin' Objects", 1099616 Still needs some GC work to get all the ZODB tests to pass like they do under zopefoundation#20
…pport Support for ZODB on PyPy
This PR brings the necessary changes to make the complete set of ZODB unit and functional tests pass (1410 in total). (See zopefoundation/ZODB#33)
Mostly this consisted of making the Python implementation more closely mirror the C implementation. A few notable things:
total_estimated_size
._p_invalidate
and_p_activate
without calling super (ZODB's Blob is an example of a deliberate use of both)._p
attributes in more circumstances than before (ZODB's Broken class relies on this)._p_oid
, something that could easily crop up in test code in the wild like it did in ZODB) ; a few restrictions had to be added to match the C implementation (e.g., requiring objects in the PickleCache to have a jar).All of the existing test cases should pass with very minor changes. I've already added a few test cases for the easier things and will see about adding a few more if I can extract them from ZODB in a reasonable form to try to get the coverage a bit higher (although it's not too bad as it is). Some of the scenarios are fairly complex though so it may be difficult to separate them. (Although, especially in the case of the C implementations it seems like many of the test cases still effectively live in ZODB.)
I'm opening this PR a bit early because I know it's large and involves some changes in the way the cache and Persistent objects have to interact, so I'm open to any feedback and suggestions. Plus I probably can't open a PR for the ZODB changes until this gets merged and released :)