Skip to content

perf(context,memory): binary search truncation and batch fidelity tag updates#4624

Merged
bug-ops merged 2 commits into
mainfrom
perf-fidelity-truncate-batch
May 29, 2026
Merged

perf(context,memory): binary search truncation and batch fidelity tag updates#4624
bug-ops merged 2 commits into
mainfrom
perf-fidelity-truncate-batch

Conversation

@bug-ops
Copy link
Copy Markdown
Owner

@bug-ops bug-ops commented May 29, 2026

Summary

  • Replace sequential halving in truncate_to_tokens with binary search — preserves up to 2× more content per Compressed message at the same token budget
  • Replace N sequential UPDATE messages SET fidelity_tag calls with a single batched CASE WHEN expression — reduces SQLite round-trips from N to ⌈N/333⌉ per scoring pass
  • Add empty-slice early return to skip transaction entirely when no updates needed

Performance

Binary search (fidelity.rs):

  • Before: O(log² N) — each halving halves the string, takes O(log N) iterations, each on a shorter prefix. At 90% truncation point, result was ~50% of content.
  • After: O(N) total byte reads across O(log N) iterations converging on the exact maximum valid prefix. At 90% truncation point, result is ~90% of content.

Batch SQL (messages/mod.rs):

  • Before: N sequential execute calls inside a transaction (N=200 → 200 awaits)
  • After: ⌈N/333⌉ execute calls (N=200 → 1 await; N=500 → 2 awaits)
  • MAX_FIDELITY_BATCH=333 keeps bind variable count within SQLITE_MAX_VARIABLE_NUMBER=999 (3 vars/row: 2 for CASE arm + 1 for IN list)

Test plan

  • 7 new unit tests for truncate_to_tokens: no-op, exact limit, minimal truncation, 90%-preservation, empty string, max_tokens=0, multibyte UTF-8 boundary
  • 2 new unit tests for update_fidelity_tags: single-item and multi-item batch
  • 1 chunk-boundary regression test: N=334 (333+1 split) validates MAX_FIDELITY_BATCH constant
  • 1640 tests pass, 8 skipped

Closes #4618
Closes #4619

@github-actions github-actions Bot added performance Performance improvements memory zeph-memory crate (SQLite) rust Rust code changes and removed performance Performance improvements labels May 29, 2026
…delity tag updates

truncate_to_tokens now uses binary search (O(log N) token-count calls) instead
of sequential halving, preserving up to 2x more content per Compressed message.
Uses usize::midpoint and floor_char_boundary for safe UTF-8 handling; stall guard
prevents infinite loops on multibyte chars spanning the midpoint.

update_fidelity_tags replaces N sequential UPDATE statements with a single batched
CASE WHEN expression per chunk, reducing SQLite round-trips from N to ceil(N/333).
MAX_FIDELITY_BATCH=333 keeps bind variable count within SQLITE_MAX_VARIABLE_NUMBER=999
(3 variables per row). A single transaction wraps all chunks for atomicity.
Empty-slice early return skips the transaction entirely.

Closes #4618
Closes #4619
@github-actions github-actions Bot added the size/L Large PR (201-500 lines) label May 29, 2026
@bug-ops bug-ops enabled auto-merge (squash) May 29, 2026 13:52
@bug-ops bug-ops force-pushed the perf-fidelity-truncate-batch branch from f935735 to 9413b08 Compare May 29, 2026 13:52
@github-actions github-actions Bot added the performance Performance improvements label May 29, 2026
@bug-ops bug-ops merged commit 791dfba into main May 29, 2026
32 checks passed
@bug-ops bug-ops deleted the perf-fidelity-truncate-batch branch May 29, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

memory zeph-memory crate (SQLite) performance Performance improvements rust Rust code changes size/L Large PR (201-500 lines)

Projects

None yet

1 participant