Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify get_row() to use the CPython tuple API #178

Merged
merged 1 commit into from Jan 22, 2014

Conversation

damoxc
Copy link
Contributor

@damoxc damoxc commented Jan 20, 2014

This drops the previous method of building up a row tuple and switches
to using the CPython API, which allows you to create a correctly sized
tuple at the beginning and simply fill it in. This appears to offer
around a 10% boost when fetching rows from a table where the data is
already in memory.

The CPU usage is still up around 100% so more optimisations should be
able to be made somewhere if some proper profiling was done.

Without patch

$ python2 test_pymssql.py
Grabbed 936222 row(s) in 10.957s
Grabbed 936222 row(s) in 10.728s
Grabbed 936222 row(s) in 10.669s
Grabbed 936222 row(s) in 10.610s
Grabbed 936222 row(s) in 10.738s
Grabbed 936222 row(s) in 10.620s
Grabbed 936222 row(s) in 10.855s
Grabbed 936222 row(s) in 10.844s
Grabbed 936222 row(s) in 10.603s
Grabbed 936222 row(s) in 10.697s
Max: 10.957s
Min: 10.603s
Avg: 10.732s

With patch

$ python2 test_pymssql.py
Grabbed 936222 row(s) in 9.676s
Grabbed 936222 row(s) in 9.537s
Grabbed 936222 row(s) in 9.481s
Grabbed 936222 row(s) in 9.424s
Grabbed 936222 row(s) in 9.281s
Grabbed 936222 row(s) in 9.421s
Grabbed 936222 row(s) in 9.291s
Grabbed 936222 row(s) in 9.454s
Grabbed 936222 row(s) in 9.438s
Grabbed 936222 row(s) in 9.443s
Max: 9.676s
Min: 9.281s
Avg: 9.445s

This drops the previous method of building up a row tuple and switches
to using the CPython API, which allows you to create a correctly sized
tuple at the beginning and simply fill it in. This appears to offer
around a 10% boost when fetching rows from a table where the data is
already in memory.

The CPU usage is still up around 100% so more optimisations should be
able to be made somewhere if some proper profiling was done.
@damoxc
Copy link
Contributor Author

damoxc commented Jan 20, 2014

@msabramo @ramiro @rsyring

damoxc added a commit that referenced this pull request Jan 22, 2014
Modify get_row() to use the CPython tuple API
@damoxc damoxc merged commit 022647b into master Jan 22, 2014
@ramiro ramiro deleted the slight-tuple-perf-improvement branch April 5, 2015 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant