Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Share SQLite database handle among all contexts #2738

merged 7 commits into from Jun 9, 2021


Copy link

@rouault rouault commented Jun 3, 2021

Fixes #2728

  • The sqlite3* handle to a same file (which is the nominal case) is shared among all contexts of all threads using the SQLITE3_FULLMUTEX mode which enables a thread-safe use of such handle. Note that this is restricted to opening a single database for now, and not using auxiliary databases which will use a per-context handle (could probably be extended to that case later)
  • Sharing of cached objects is not implemented yet.
  • proj_context_set_autoclose_database() is now deprecated and a no-op. Implementing to work now would be a complication and would defeat the purpose of the whole work. proj_context_set_autoclose_database() was mostly a workaround for the non-shared sqlite3* handle

Seems to work well for me, but testing from others would be appreciated

@rouault rouault added this to the 8.1.0 milestone Jun 3, 2021
Copy link

snowman2 commented Jun 6, 2021

Just tested it out with the pyproj test suite and the speed improvement was fantastic. Tests went from ~3 minutes with previous versions of PROJ to ~30 seconds with this MR. All tests pass (Note: I haven't done any serious threading tests).

Minimal speed test:

python -m timeit -s "from pyproj import CRS" "CRS('EPSG:4326')"


PROJ 8.0.1

20 loops, best of 5: 15.1 msec per loop

This MR

500 loops, best of 5: 759 usec per loop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

Successfully merging this pull request may close these issues.

Sharing connections to proj.db
2 participants