-
-
Notifications
You must be signed in to change notification settings - Fork 31.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Wrap sqlite3_serialize API in sqlite3 module #86096
Comments
It would be useful to provide a wrapper (in the Python sqlite3 stdlib module) for the sqlite3_serialize API. This API allows you to get a database's content as a byte string, as if you had called open('foo.sqlite3', 'rb').read(). sqlite3_serialize can even be used for :memory: databases, which I think is the most interesting use-case. Concretely, I'd propose adding a .serialize() method on sqlite3.Connection objects that returns a bytes object representing the serialized database. It would be similar to the .iterdump() method except that it would use the binary format instead of the text format, and it would return all the data at once instead of an iterator (because that's how the underlying sqlite API works). |
What would be the use case for this? |
FYI, this was just checked into the SQLite fossil repo: $ fossil update trunk
updated-to: 6df3b03e00b1143be8fed3a39a58ce8106302027 2021-05-08 17:18:23 UTC
tags: trunk
comment: Enable the sqlite3_serialize() and sqlite3_deserialize() interfaces by default. Omit the SQLITE_ENABLE_DESERIALIZE option and replace it with the SQLITE_OMIT_DESERIALIZE option. (user: drh) Ref. the SQLite docs: https://www.sqlite.org/compile.html#enable_deserialize |
FYI, the pending release log of the upcoming SQLite 3.36.0 now mentions that these API's are enabled by default: |
Quoting D. Richard Hipp, from the sqlite-users mailing list, 2018-03-18, FWIW: For example: A client program might communicate with a server using Advantages of using an SQLite database as a container: (1) It is easy to mix text and binary data without having to worry (2) Applications can be easily enhanced and extended in (3) Easy to manually view or modify the container content during (4) No need to write encoder/decoder logic. (5) There is a high-level query language available to extract the (6) No need to worry with big-endian vs. little-endian translation, Using the serialise/deserialise API's to pickle an in-memory database to a data stream is an interesting thought. |
I've been fiddling with this between others projects lately; the PR is mostly ready. The only remaining issue is how to include this in the Connection object:
Solution 1: enable the Python sqlite3 serialize API if SQLITE_VERSION_NUMBER >= 3023000 and force people to build their SQLite library _with_ SQLITE_ENABLE_DESERIALIZE defined for SQLite versions 3.23.0 through 3.35.x and _without_ SQLITE_OMIT_DESERIALIZE defined for SQLite versions 3.36.0 and onward. Solution 2: enable the Python sqlite3 serialize API if SQLITE_VERSION_NUMBER >= 3036000 and force people to build their SQLite library _without_ SQLITE_OMIT_DESERIALIZE defined. Solution 3: build the Python sqlite3 serialize API as a "sub module" to _sqlite3 (for example _sqlite3._serialize) and add conditionally add it to sqlite3.Connection in Lib/sqlite3/init.py or Lib/sqlite3/dbapi2.py. Solution 4: try to autodetect SQLite compile-time options in setup.py, autogenerate a sqlite-config.h header file, and conditionally enable the API based on that. I suggest solution 2, as it adds little code (low maintenance), and will require no or minimal changes to the devguide, because (wild guess) in most cases SQLITE_OMIT_DESERIALIZE will not be a widely used compile-time option. (For the python.org macOS and Windows installers, this will not be a problem, because we control the SQLite compile-time options.) |
Thanks, this will be in Python 3.11. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: