Skip to content

Speed up ZODB.blob.BushyLayout.oid_to_path on Python 3 - #161

Merged
jimfulton merged 1 commit into
masterfrom
faster-bush-oid-to-path
Apr 14, 2017
Merged

Speed up ZODB.blob.BushyLayout.oid_to_path on Python 3#161
jimfulton merged 1 commit into
masterfrom
faster-bush-oid-to-path

Conversation

@jamadden

Copy link
Copy Markdown
Member

Profiling (zodb/zodbshootout#32) showed that
this method was the only blob-related method that showed up in a test
of creating blobs, other than those that actually performed IO.

With this change its total and cumulative time drops from 0.003/0.004
to 0.001/0.002 in a small benchmark. Blobs created per second shows a
small but consistent improvement.

Before:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x1080d3a60}
      200    0.003    0.000    0.004    0.000 blob.py:576(oid_to_path)
      101    0.003    0.000    0.003    0.000 {built-in method mkdir}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
        1    0.002    0.002    0.034    0.034 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     5633    0.001    0.000    0.002    0.000 {built-in method isinstance}

After:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      101    0.005    0.000    0.005    0.000 {built-in method mkdir}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x10636aa60}
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
        1    0.002    0.002    0.035    0.035 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     4033    0.001    0.000    0.001    0.000 {built-in method isinstance}
   ....
      200    0.001    0.000    0.002    0.000 blob.py:576(oid_to_path)

Profiling (zodb/zodbshootout#32) showed that
this method was the only blob-related method that showed up in a test
of creating blobs, other than those that actually performed IO.

With this change its total and cumulative time drops from 0.003/0.004
to 0.001/0.002 in a small benchmark. Blobs created per second shows a
small but consistent improvement.

Before:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x1080d3a60}
      200    0.003    0.000    0.004    0.000 blob.py:576(oid_to_path)
      101    0.003    0.000    0.003    0.000 {built-in method mkdir}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
        1    0.002    0.002    0.034    0.034 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     5633    0.001    0.000    0.002    0.000 {built-in method isinstance}

After:

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100    0.005    0.000    0.005    0.000 {built-in method rename}
      101    0.005    0.000    0.005    0.000 {built-in method mkdir}
      100    0.004    0.000    0.004    0.000 {function BlobFile.close at 0x10636aa60}
      402    0.002    0.000    0.005    0.000 {method 'dump' of '_pickle.Pickler' objects}
      100    0.002    0.000    0.002    0.000 blob.py:333(__init__)
        1    0.002    0.002    0.035    0.035 Connection.py:553(_store_objects)
      201    0.002    0.000    0.002    0.000 {built-in method stat}
     4033    0.001    0.000    0.001    0.000 {built-in method isinstance}
   ....
      200    0.001    0.000    0.002    0.000 blob.py:576(oid_to_path)
@jimfulton
jimfulton merged commit 3580ddd into master Apr 14, 2017
@jimfulton

Copy link
Copy Markdown
Member

Thanks!

@jimfulton
jimfulton deleted the faster-bush-oid-to-path branch April 14, 2017 15:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants