Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve callback performance (C to Python calls) #477

Closed
9 tasks done
rogerbinns opened this issue Aug 26, 2023 · 0 comments
Closed
9 tasks done

Improve callback performance (C to Python calls) #477

rogerbinns opened this issue Aug 26, 2023 · 0 comments

Comments

@rogerbinns
Copy link
Owner

rogerbinns commented Aug 26, 2023

There are two things that can be done:

  • Cache a PyUnicode object for the string of the callback name instead of looking it up each time
  • Use the vector calling protocol

A quick benchmark where only the xColumn callback was changed to use PyObject_VectorcallMethod, measured in calls per second by vtbench

2,085,699 Base (Old calling, no string cache)
2,253,161 Vector, PyUnicode_FromString called each time
2,678,927 Vector, PyUnicode_FromString called once
Update after initial implementation has 3 million calls per second.

  • vtable vector calls
  • vfs vector call
  • Other Call_PythonMethod sites
  • PyObject_Call sites
  • getfunctionargs needs vargs rewrite
  • Make it work on Python 3.8
  • Memory leak checking
  • Connection.execute{,many} use fastcall
  • PyObject_Get/SetAttrString - use string table
rogerbinns added a commit that referenced this issue Aug 26, 2023
This especially helps with #477
@rogerbinns rogerbinns changed the title Improve callback performance Improve callback performance (C to Python calls) Aug 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant