Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(archive): patch the sqlite driver to solve results inconsistencies #1415

Merged
merged 1 commit into from
Nov 24, 2022

Conversation

LNSD
Copy link
Contributor

@LNSD LNSD commented Nov 24, 2022

This is a temporary patch until the SQLite library is improved. Making the SQLite manage a copy is far from ideal. Binding the parameters with this "destructor configuration" option, implies that SQLite is making a copy of the buffer. This is inefficient. A more performant solution should be investigated.

From SQLite C documentation (link):

The fifth argument to the BLOB and string binding interfaces controls or indicates the lifetime of the object referenced by the third parameter. These three options exist:

  1. A destructor to dispose of the BLOB or string after SQLite has finished with it may be passed. It is called to dispose of the BLOB or string even if the call to the bind API fails, except the destructor is not called if the third parameter is a NULL pointer or the fourth parameter is negative.
    2.The special constant, SQLITE_STATIC, may be passsed to indicate that the application remains responsible for disposing of the object. In this case, the object and the provided pointer to it must remain valid until either the prepared statement is finalized or the same SQL parameter is bound to something else, whichever occurs sooner.
  2. The constant, SQLITE_TRANSIENT, may be passed to indicate that the object is to be copied prior to the return from sqlite3_bind_*(). The object and pointer to it must remain valid until then. SQLite will then manage the lifetime of its private copy.

The tests I have performed so far seem to confirm that these changes solve issue #1400. But to be 100% sure, more thorough testing needs to be done.

@LNSD LNSD self-assigned this Nov 24, 2022
@@ -127,10 +127,15 @@ template prepare*(env: Sqlite, q: string, cleanup: untyped): ptr sqlite3_stmt =

proc bindParam*(s: RawStmtPtr, n: int, val: auto): cint =
when val is openarray[byte]|seq[byte]:
# The constant, SQLITE_TRANSIENT, may be passed to indicate that the object is to be copied
# prior to the return from sqlite3_bind_*(). The object and pointer to it must remain valid
# until then. SQLite will then manage the lifetime of its private copy.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume this means it handles the malloc but always free it again - more memory but (hopefully) not leaky memory?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We call the sqlite3_finalize function later in the code to release the prepared statement memory.

This is what the documentation says about the SQLite statement destructor:

The application must finalize every prepared statement in order to avoid resource leaks.

I interpret this as deallocating any memory associated with the prepared statement. Bindings included.

Copy link
Contributor

@jm-clius jm-clius left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really great work. Quite something to find this. Well done!

@LNSD
Copy link
Contributor Author

LNSD commented Nov 24, 2022

macOS test2 job failing due to the same offending test case. Test failure already tracked in #1357. Merging without re-running the checks and Ubuntu test2 job is passing.

Copy link
Contributor

@rymnc rymnc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@LNSD LNSD merged commit 41b2f87 into master Nov 24, 2022
@LNSD LNSD deleted the fix-archive-sqlite-driver branch November 24, 2022 17:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants