Skip to content

Make all classes new-style. - #160

Merged
jimfulton merged 3 commits into
masterfrom
only-new-style
Apr 17, 2017
Merged

Make all classes new-style.#160
jimfulton merged 3 commits into
masterfrom
only-new-style

Conversation

@jamadden

@jamadden jamadden commented Apr 13, 2017

Copy link
Copy Markdown
Member

On PyPy, it's documented (http://pypy.org/performance.html#id3) that old-style classes are slow, and classes that derive from both old and new are especially slow. I checked with the PyPy devs on IRC today and they confirmed that's still true with the latest PyPy2 releases.

Unfortunately, FileStorage was one such mixed class (due to UndoLogCompatible), as was Connection (due to ExportImport).

Moving to new-style classes seems to have a positive impact in the benchmarks. Here's zodbshootout on PyPy2 5.7.1 against a FileStorage (running in --threads and with 5 reps to be sure we get warmed up). First, current ZODB:

"Transaction",                fs
"Add 1000 Objects",            31,738
"Update 1000 Objects",         42,444
"Read 1000 Cold Objects",      51,894
"Read 1000 Hot Objects",       53,187
"Read 1000 Steamin' Objects", 835,877

And with this PR:

"Transaction",                fs
"Add 1000 Objects",             35,651
"Update 1000 Objects",          54,906
"Read 1000 Cold Objects",      103,484
"Read 1000 Hot Objects",        84,721
"Read 1000 Steamin' Objects", 2,112,095

The tests that hit the storage extensively are notably faster, as are steamin and hot (Connection having been a mixed class).

I ran all tests multiple times. The data files were removed between runs. There's some variation, but the new-style classes always seem better.

For comparison, here's CPython 2.7.13:

"Transaction",                fs
"Add 1000 Objects",            19,531
"Update 1000 Objects",         16,201
"Read 1000 Cold Objects",      22,111
"Read 1000 Hot Objects",       21,851
"Read 1000 Steamin' Objects", 880,582

Locally I've run the tests under 2.7 and they all passed.

Besides the benchmark improvements, this makes Python 2 closer to Python 3 for those classes.

This was a mechanical transform (with only one mis-change reverted) produced by this script:

$ sed -E -i "" "s/class ([[:alnum:]_]*):/class \1(object):/g" `find src -name "*.py"`

Comment thread src/ZODB/serialize.py Outdated

# Now we have a list of references. Need to convert to list of
# oids and class info:
# oids and class info(object):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you recheck with s/^(\s*)class ([[:alnum:]_]*):/\1class \2(object):/g ?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for finding that!

Unfortunately the other regex misses not just the things that are in docstrings/doctests (which aren't important for performance, granted, but are important for consistency) but it also missed a handful of classes (E.g this one) for some reason.

jamadden added a commit to zopefoundation/ZEO that referenced this pull request Apr 13, 2017
See zopefoundation/ZODB#160 for rationale.

Also restore PyPy to the build matrix.
On PyPy, it's documented (http://pypy.org/performance.html#id3) that
old-style classes are slow, and classes that derive from both old and
new are especially slow. I checked with the PyPy devs on IRC today and
they confirmed that's still true with the latest PyPy2 releases.

Unfortunately, FileStorage was one such mixed class, as was Connection.

Moving to new-style classes seems to have a positive impact in the
benchmarks. Here's zodbshootout on PyPy2 5.7.1 against a
FileStorage (running in --threads and with 5 reps to be sure we get
warmed up). First, current ZODB:

"Transaction",                fs
"Add 1000 Objects",            31,738
"Update 1000 Objects",         42,444
"Read 1000 Cold Objects",      51,894
"Read 1000 Hot Objects",       53,187
"Read 1000 Steamin' Objects", 835,877

And with this PR:

"Transaction",                fs
"Add 1000 Objects",             35,651
"Update 1000 Objects",          54,906
"Read 1000 Cold Objects",      103,484
"Read 1000 Hot Objects",        84,721
"Read 1000 Steamin' Objects", 2,112,095

The tests that hit the storage extensively are notably faster, as are
steamin and hot, Connection having been a mixed class.

I ran all tests multiple times. The data files were removed between
runs. There's some variation, but the new-style classes always seem
better.

For comparison, here's CPython 2.7.13:

"Transaction",                fs
"Add 1000 Objects",            19,531
"Update 1000 Objects",         16,201
"Read 1000 Cold Objects",      22,111
"Read 1000 Hot Objects",       21,851
"Read 1000 Steamin' Objects", 880,582

Locally I've run the tests under 2.7 and they all passed.
@jamadden

Copy link
Copy Markdown
Member Author

Rebased on master to resolve CHANGES.rst conflict with #161.

@jamadden
jamadden requested a review from jimfulton April 17, 2017 11:35
@jimfulton
jimfulton merged commit e714f51 into master Apr 17, 2017
@jimfulton

Copy link
Copy Markdown
Member

Thanks.

@jamadden
jamadden deleted the only-new-style branch April 17, 2017 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants