Skip to content

fix(rivetkit-sqlite): preserve text with embedded nul bytes#4735

Draft
NathanFlurry wants to merge 1 commit into04-23-chore_rivetkit_add_child_spans_for_loggingfrom
04-24-fix_sqlite_text_nul_roundtrip
Draft

fix(rivetkit-sqlite): preserve text with embedded nul bytes#4735
NathanFlurry wants to merge 1 commit into04-23-chore_rivetkit_add_child_spans_for_loggingfrom
04-24-fix_sqlite_text_nul_roundtrip

Conversation

@NathanFlurry
Copy link
Copy Markdown
Member

Description

Please include a summary of the changes and the related issue. Please also include relevant motivation and context.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

This was referenced Apr 24, 2026
Copy link
Copy Markdown
Member Author

NathanFlurry commented Apr 24, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@claude
Copy link
Copy Markdown

claude Bot commented Apr 24, 2026

PR Review: fix(rivetkit-sqlite): preserve text with embedded nul bytes

This is a targeted and correct bug fix. The core change — replacing CString::new() with an explicit-length sqlite3_bind_text call — is the right approach for handling text with embedded NUL bytes in SQLite bindings.

Correctness

The fix is technically sound:

  • Bind side: sqlite3_bind_text with an explicit byte length correctly passes the full buffer including embedded NULs. The old CString::new() would return Err for any string containing NUL bytes, silently dropping the bind.
  • Read side: Using sqlite3_column_bytes + slice::from_raw_parts matches the existing blob read path and correctly recovers the full byte range.

Issues

Potential truncation on large strings

value.len() as i32 will silently truncate if the string exceeds i32::MAX (~2 GB). This is consistent with the existing blob binding code so it is not a regression, but worth a checked cast:

let len = i32::try_from(value.len()).expect("text value exceeds i32::MAX");

Missing newline at end of file

The diff shows no newline at end of file on message.incoming_too_long.json after the change — the missing newline still is not added.

Duplicated MemoryDb helper

The new test file tests/query_text_nul.rs duplicates the MemoryDb helper struct that already exists in the inline #[cfg(test)] mod tests block inside query.rs. Per CLAUDE.md, Rust tests should live under tests/, not inline in src/. The existing inline tests in query.rs should be moved to tests/ alongside this new test, and MemoryDb extracted into a shared helper to avoid duplication.

Misleading test assertion

Both the Rust and TypeScript tests assert sqlite_length === 1 because the SQLite length() function stops at the first NUL for TEXT columns. While technically correct, it may confuse readers into thinking only 1 byte was stored. A brief comment would help clarify this is expected SQLite behavior, while hex() proves all bytes are actually stored.

Unrelated change

The punctuation fix in message.incoming_too_long.json is unrelated to the NUL byte fix. Not a blocker, but worth separating into its own commit.

Minor Notes

  • from_utf8_lossy silently replaces invalid UTF-8 with the replacement character. This matches the pre-fix behavior (CStr::to_string_lossy) so it is not a regression, but malformed UTF-8 in a TEXT column gets silently corrupted. Acceptable for SQLite TEXT (which is defined to be UTF-8), just worth acknowledging.
  • The SQL preparation paths still use CString::new(sql) — this is appropriate since SQL itself should not contain NUL bytes. The fix is correctly scoped to bound parameters and result columns.

Summary

The core fix is correct and well-tested. Main asks before merging: (1) address the i32 truncation with a checked cast, (2) add a comment to the sqlite_length == 1 assertions explaining the expected SQLite behavior, (3) move existing inline query.rs tests to tests/ per CLAUDE.md conventions and share the MemoryDb helper.

@NathanFlurry NathanFlurry mentioned this pull request Apr 24, 2026
11 tasks
@NathanFlurry NathanFlurry force-pushed the 04-24-fix_sqlite_text_nul_roundtrip branch from 38f839d to e8072b7 Compare April 24, 2026 09:52
@NathanFlurry NathanFlurry mentioned this pull request Apr 24, 2026
11 tasks
@NathanFlurry NathanFlurry force-pushed the 04-24-fix_sqlite_text_nul_roundtrip branch from e8072b7 to 4f00fb4 Compare April 24, 2026 10:19
@NathanFlurry NathanFlurry force-pushed the 04-23-chore_rivetkit_add_child_spans_for_logging branch from 475a302 to 36cef20 Compare April 24, 2026 10:19
@NathanFlurry NathanFlurry force-pushed the 04-23-chore_rivetkit_add_child_spans_for_logging branch from 36cef20 to a70e758 Compare April 24, 2026 10:32
@NathanFlurry NathanFlurry force-pushed the 04-24-fix_sqlite_text_nul_roundtrip branch from 4f00fb4 to 5e61ccd Compare April 24, 2026 10:32
@NathanFlurry NathanFlurry force-pushed the 04-23-chore_rivetkit_add_child_spans_for_logging branch from a70e758 to e676e10 Compare April 24, 2026 11:48
@NathanFlurry NathanFlurry force-pushed the 04-24-fix_sqlite_text_nul_roundtrip branch from 5e61ccd to 3fd37c2 Compare April 24, 2026 11:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant