Support for ZODB on PyPy #20
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…tion needs to bypass the _setattr_ implementation when working with its own internal state variables. This much gets the ZODB broken tests to pass. Need to figure out how to test this locally.
…the C version in more cases. Also, make the behaviour of comparisons and hashing match as well; test all this.
…'s testCache passes. Required some small updates and additions to our own test cases; the update_object_size_estimation is not tested here yet.
…for _p_changed in pure-python.
…es and ghosts the evicted objects.
…, and clear out references to ejected objects when done. This makes the ZODB testConnection and testConnectionSavepoint tests pass but is not tested here.
… be changed. ZODB depends on this.
…t. There are some interconnected lifetime issues that this solves for ZODB's cache tests, and what appear to be some genuine bugs in invalidation.
…on tests depend on it.
…tion needs to bypass the _setattr_ implementation when working with its own internal state variables. This much gets the ZODB broken tests to pass. Need to figure out how to test this locally.
…'s testCache passes. Required some small updates and additions to our own test cases; the update_object_size_estimation is not tested here yet.
…for _p_changed in pure-python.
…es and ghosts the evicted objects.
…, and clear out references to ejected objects when done. This makes the ZODB testConnection and testConnectionSavepoint tests pass but is not tested here.
… be changed. ZODB depends on this.
…t. There are some interconnected lifetime issues that this solves for ZODB's cache tests, and what appear to be some genuine bugs in invalidation.
…on tests depend on it.
|
I was actually able to restore 100% coverage pretty easily, but I'm not sure that really covers all of ZODB's use cases. |
|
Speaking of test coverage, I couldn't stop thinking about Hypothesis when I was looking at your recent series of pull requests. Sounds like the perfect tool to discover differences between C and Python implementations, e.g. the timestamp second rounding mismatch. Unfortunately I don't have time to play with it :( |
… tox/virtualenv/pip come together soon to allow official testing and mention.
… I'm not sure why I was reaching into the internals of the Python implementation.
…hon2. The existing behaviour matches the C implementation.
| from .. import ring | ||
|
|
||
| if _is_pypy or os.getenv('USING_CFFI'): | ||
| assert ring.Ring is ring._CFFIRing |
There was a problem hiding this comment.
you can use 'self.assertTrue(ring.Ring is ring._CFFIRing)`.
…OSA for consistency; also avoid some duplicate calls through __getattribute__ which were more obvious in the OGA form.
…he_size as specified in the interface.
This was referenced May 19, 2015
|
Whew! That's a lot of back and forth -- thanks for the cheerful effort! I have opened issues for the non-consensus issues here:
|
|
Thank you, @tseaver and everyone else involved. It's been a learning experience and the interactions have been great. |
|
Whow, I saw this thread way too late. Extremely well thought things going on. |
fabiomaggio
pushed a commit
to fabiomaggio/persistent
that referenced
this pull request
Feb 20, 2017
…ally. Performance is way up again, beating CPython on half of the tests: "Transaction", mysql "Add 3000 Objects", 8259 "Update 3000 Objects", 9454 "Read 3000 Warm Objects", 5460 "Read 3000 Cold Objects", 5454 "Read 3000 Hot Objects", 24943 "Read 3000 Steamin' Objects", 1099616 Still needs some GC work to get all the ZODB tests to pass like they do under zopefoundation#20
fabiomaggio
pushed a commit
to fabiomaggio/persistent
that referenced
this pull request
Feb 20, 2017
…pport Support for ZODB on PyPy
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR brings the necessary changes to make the complete set of ZODB unit and functional tests pass (1410 in total). (See zopefoundation/ZODB#33)
Mostly this consisted of making the Python implementation more closely mirror the C implementation. A few notable things:
total_estimated_size._p_invalidateand_p_activatewithout calling super (ZODB's Blob is an example of a deliberate use of both)._pattributes in more circumstances than before (ZODB's Broken class relies on this)._p_oid, something that could easily crop up in test code in the wild like it did in ZODB) ; a few restrictions had to be added to match the C implementation (e.g., requiring objects in the PickleCache to have a jar).All of the existing test cases should pass with very minor changes. I've already added a few test cases for the easier things and will see about adding a few more if I can extract them from ZODB in a reasonable form to try to get the coverage a bit higher (although it's not too bad as it is). Some of the scenarios are fairly complex though so it may be difficult to separate them. (Although, especially in the case of the C implementations it seems like many of the test cases still effectively live in ZODB.)
I'm opening this PR a bit early because I know it's large and involves some changes in the way the cache and Persistent objects have to interact, so I'm open to any feedback and suggestions. Plus I probably can't open a PR for the ZODB changes until this gets merged and released :)