Skip to content

perf(core): make /history transcript_len/transcript_page O(1)/O(count)#6434

Merged
bug-ops merged 1 commit into
mainfrom
fix/6427-history-scan-perf
Jul 18, 2026
Merged

perf(core): make /history transcript_len/transcript_page O(1)/O(count)#6434
bug-ops merged 1 commit into
mainfrom
fix/6427-history-scan-perf

Conversation

@bug-ops

@bug-ops bug-ops commented Jul 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • MessageAccessImpl::transcript_len/transcript_page (crates/zeph-core/src/agent/command_context_impls.rs), which back the /history command, re-scanned the entire message vector on every call — transcript_len was O(total non-system messages) unconditionally, and transcript_page's .skip(start) ran on top of a .filter(), so it walked from index 0 even though the common bounded-default case (/history N) computes start near the end of history. Repeated /history all -> /history next paging did O(total²/page_size) aggregate work.
  • MessageState now maintains an incrementally-updated non_system_count (O(1) transcript_len), and transcript_page scans from whichever end of the message vector is closer to the requested page, making both named hot patterns (/history N bounded-default tail read, /history all's first page) O(count) instead of O(total).
  • Middle-of-history pages reached via a long /history next chain remain up to O(min(start, total-end)) by design — documented in the transcript_page doc comment. A full position index of non-system ordinals was rejected: arbitrary-position System-message churn (LSP notes, focus checkpoints) would make that index itself O(n) per turn, moving cost onto the hotter per-turn path instead of the /history read path.
  • Audited and updated every messages-vector mutation site in zeph-core (~15 production call sites across 9 files) to keep the counter in sync: single push/insert/remove/pop sites use an O(1) delta (track_single_message); batch mutations, and mutations that happen through a borrowed view in a foreign crate (zeph-agent-context, zeph-agent-persistence), use an O(n) full recount (recompute_non_system_count) mirroring the existing recompute_prompt_tokens() cache-invalidation pattern already used at the same call sites.

Background

Follow-up from #6420 (resume-session history visibility, spec-068 §13.6, INV-SP-6), explicitly tracked as a known perf gap when that PR merged.

Went through the reduced performance chain: developer (root cause + fix + mutation-site audit + 10 unit tests) -> parallel validation (perf: microbenchmark confirming the O(1)/O(count) claim, 2400x-26000x speedup on transcript_len, 10.7x-87.6x on the bounded-default tail read, no regressions; tester: confirmed 2099 tests pass, found a real coverage gap — none of the batch-mutation sites were exercised by a test that reads transcript_len/transcript_page afterward; impl-critic: verdict minor, re-verified the mutation-site audit and the bidirectional-scan boundary logic independently, no blocking bugs) -> code review (one changes_requested round: required 2 targeted regression tests closing the coverage gap tester found, plus a zero-cost saturating_add hardening; approved on re-review).

Non-blocking follow-ups noted during review, not addressed in this PR:

  • 5 #[cfg(test)]-gated context wrappers in assembly.rs have asymmetric recompute treatment vs their System-only siblings — dead in every real build (confirmed by direct read), so no live bug, but worth a uniformity pass if any of them are ever promoted to production.
  • Cosmetic double-recompute in focus.rs's apply_compression_removals -> rebuild_knowledge_block path — not worth changing unless that path proves hot.

Test plan

  • cargo +nightly fmt --check — clean
  • cargo clippy --profile ci --workspace --all-targets --features "desktop,ide,server,chat,pdf,scheduler,testing" -- -D warnings — clean
  • cargo nextest run --config-file .github/nextest.toml --workspace --features "desktop,ide,server,chat,pdf,scheduler" --lib --bins — 14280/14280 passed, 0 failed
  • RUSTFLAGS="-D warnings" RUSTDOCFLAGS="--deny rustdoc::broken_intra_doc_links" cargo doc --no-deps --workspace --features "desktop,ide,server,chat,pdf,scheduler" — clean
  • gitleaks protect --staged — no leaks
  • 12 unit tests in command_context_impls.rs::tests (10 initial + 2 added during review to close a coverage gap on the batch-mutation/foreign-crate-view recompute pattern)
  • Standalone microbenchmark confirming the O(1)/O(count) complexity claim at 5k/20k/50k synthetic messages
  • .local/testing/playbooks/session-resume-history.md and .local/testing/coverage-status.md updated (main repo, not this worktree)
  • Reviewed by rust-code-reviewer: approved after one fix round (test coverage gap)

Closes #6427

MessageAccessImpl::transcript_len/transcript_page re-scanned the entire
message vector on every /history call: transcript_len was O(total
non-system messages) unconditionally, and transcript_page's .skip(start)
ran on top of a .filter(), so it walked from index 0 even for the common
bounded-default case where start sits near the end of history. Repeated
/history all -> /history next paging did O(total^2/page_size) aggregate
work.

MessageState now maintains an incrementally-updated non_system_count
(O(1) transcript_len), and transcript_page scans from whichever end of
the vector is closer to the requested page, making both named hot
patterns (/history N bounded-default, /history all's first page)
O(count) instead of O(total). Middle-of-history /history next pages
remain up to O(min(start, total-end)) by design, documented in the
transcript_page doc comment.

Audited and updated every messages-vector mutation site in zeph-core to
keep the counter in sync: single push/insert/remove/pop sites use an
O(1) delta (track_single_message), batch mutations and mutations that
happen through a borrowed view in a foreign crate use an O(n) full
recount (recompute_non_system_count) mirroring the existing
recompute_prompt_tokens() pattern.

Closes #6427
@github-actions github-actions Bot added documentation Improvements or additions to documentation rust Rust code changes core zeph-core crate performance Performance improvements size/L Large PR (201-500 lines) labels Jul 18, 2026
@bug-ops
bug-ops enabled auto-merge (squash) July 18, 2026 00:52
@bug-ops
bug-ops merged commit 17a5f2a into main Jul 18, 2026
43 checks passed
@bug-ops
bug-ops deleted the fix/6427-history-scan-perf branch July 18, 2026 01:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core zeph-core crate documentation Improvements or additions to documentation performance Performance improvements rust Rust code changes size/L Large PR (201-500 lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MessageAccessImpl::transcript_len/transcript_page re-scan the full message Vec on every /history call

1 participant