Skip to content

fix(memory): prevent stuck singleton leaves from starving TiMem consolidation#6402

Merged
bug-ops merged 1 commit into
mainfrom
fix/6393-timem-leaf-starvation
Jul 17, 2026
Merged

fix(memory): prevent stuck singleton leaves from starving TiMem consolidation#6402
bug-ops merged 1 commit into
mainfrom
fix/6393-timem-leaf-starvation

Conversation

@bug-ops

@bug-ops bug-ops commented Jul 17, 2026

Copy link
Copy Markdown
Owner

Summary

  • load_tree_leaves_unconsolidated loaded the oldest batch_size unconsolidated leaves every consolidation sweep. Leaves that never cluster (singletons) keep parent_id NULL forever, so once more than batch_size singletons accumulate, every sweep re-loads and re-embeds the same stuck oldest set — newer leaves with real memorable content never get a chance to be considered.
  • Pre-existing bug, unreachable before TiMem tree consolidation loop runs forever as a no-op: insert_tree_leaf has zero production callers #6384 (PR fix(memory): wire insert_tree_leaf and recent-embeddings fetch into production paths #6392) gave insert_tree_leaf a production caller; flagged out of scope during that PR's implementation critique.
  • Adds a consolidation_attempts/last_attempted_at pair on memory_tree (migration 111, sqlite + postgres) that run_tree_consolidation_sweep bumps for every loaded node that does not end up consolidated. Both load_tree_leaves_unconsolidated and load_tree_level now order by true LRU (COALESCE(last_attempted_at, created_at) ASC, with consolidation_attempts/id as deterministic tiebreakers) instead of strict age — a bounded-wait guarantee, not just a bias, since a stuck leaf's touch time freezes in the past while competing leaves' touch times keep refreshing to "now".
  • Selection priority and presentation order are decoupled via a subquery, since cluster_by_similarity's greedy leader algorithm requires created_at ASC for deterministic clustering regardless of which rows were selected.

Closes #6393

Test plan

  • cargo +nightly fmt --check
  • cargo clippy --profile ci --workspace --all-targets --features "desktop,ide,server,chat,pdf,scheduler,testing" -- -D warnings
  • cargo nextest run --config-file .github/nextest.toml --workspace --features "desktop,ide,server,chat,pdf,scheduler" --lib --bins (14119 passed, 0 failed, 35 skipped)
  • RUSTFLAGS="-D warnings" RUSTDOCFLAGS="--deny rustdoc::broken_intra_doc_links" cargo doc --no-deps --workspace --features "desktop,ide,server,chat,pdf,scheduler"
  • gitleaks protect --staged --no-banner --redact
  • zeph-db migration parity tests (sqlite/postgres dialect symmetry for migration 111)
  • New regression tests directly disprove the original starvation scenario and the critic's sustained-new-arrival counterexample via explicit backdated multi-sweep timelines (not real wall-clock dependent)
  • Production wiring confirmed real (not dead code): run_tree_consolidation_sweep is reached from the supervised mem-tree-consolidation background task, not just from tests

@github-actions github-actions Bot added bug Something isn't working size/XL Extra large PR (500+ lines) documentation Improvements or additions to documentation memory zeph-memory crate (SQLite) rust Rust code changes and removed size/XL Extra large PR (500+ lines) labels Jul 17, 2026
@bug-ops
bug-ops enabled auto-merge (squash) July 17, 2026 17:42
@bug-ops
bug-ops force-pushed the fix/6393-timem-leaf-starvation branch from 7d75818 to e5c79c8 Compare July 17, 2026 17:42
@github-actions github-actions Bot added the size/XL Extra large PR (500+ lines) label Jul 17, 2026
…lidation

load_tree_leaves_unconsolidated loaded the oldest batch_size unconsolidated
leaves every sweep; leaves that never cluster (singletons) keep parent_id
NULL, so once more than batch_size singletons accumulate, every sweep
re-loads the same stuck set and newer leaves never get a chance.

Add a consolidation_attempts/last_attempted_at pair on memory_tree that the
sweep bumps for any loaded node that doesn't end up consolidated. Both load
queries now order by true LRU (COALESCE(last_attempted_at, created_at) ASC)
instead of strict age, giving a bounded-wait guarantee: a leaf's touch time
freezes in the past while competing leaves' touch times keep refreshing, so
a stuck leaf's relative priority strictly increases until it wins. Selection
priority and presentation order are decoupled via a subquery, since
cluster_by_similarity requires created_at ASC for deterministic clustering
regardless of which rows were selected.
@bug-ops
bug-ops force-pushed the fix/6393-timem-leaf-starvation branch from e5c79c8 to 0bab92a Compare July 17, 2026 18:06
@bug-ops
bug-ops merged commit bb26f9a into main Jul 17, 2026
43 checks passed
@bug-ops
bug-ops deleted the fix/6393-timem-leaf-starvation branch July 17, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation memory zeph-memory crate (SQLite) rust Rust code changes size/XL Extra large PR (500+ lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TiMem tree consolidation: singleton leaves can permanently starve newer leaves from being swept

1 participant