Skip to content

fix: batch consolidation for large entry counts#474

Merged
BYK merged 1 commit into
mainfrom
fix/batched-consolidation
May 27, 2026
Merged

fix: batch consolidation for large entry counts#474
BYK merged 1 commit into
mainfrom
fix/batched-consolidation

Conversation

@BYK
Copy link
Copy Markdown
Owner

@BYK BYK commented May 27, 2026

Summary

Follow-up to #473. Onur's logs revealed 1143 knowledge entries — far beyond what the single-pass consolidation can handle. The previous consolidation sent all entries in one prompt, but with 1143 entries (~343K tokens of input) this overflows the context window, and the 4096 output token budget can only express ~80-100 delete ops (vs the ~1118 needed).

Context from Onur's logs

entry count 1143 exceeds maxEntries 25 — running consolidation
entry count 1143 exceeds maxEntries 25 — running consolidation
entry count 1143 exceeds maxEntries 25 — running consolidation
...repeating every ~60s...

cost-tracker: worker overhead=$1140.8017 (distillation-only=$2.5246)

The retry storm (#473) burned $1,138 in consolidation calls that could never succeed due to the token budget constraint.

Changes

Adds batched consolidation mode in curator.ts:

  • When entries ≤ 50: unchanged — sends all entries in a single prompt
  • When entries > 50 (batched mode): takes the lowest-confidence entries (tail of the confidence-sorted list from forProject()) as candidates for deletion. Each pass targets removing ~25 entries (half the batch).
  • The idle scheduler's cooldown (from fix: consolidation retry storm, idle curation frequency, and session memory leak #473) clears when entry count changes, automatically triggering the next batch on the following idle tick.
  • Converges to maxEntries over multiple passes: 1143 → 1118 → 1093 → ... → 25

For Onur's case: ~45 passes × 1 Sonnet call each ≈ $2-3 total to clean up 1143 entries, spread across idle periods. vs the previous behavior of infinite retries that never made progress.

@BYK BYK self-assigned this May 27, 2026
When entry count massively exceeds maxEntries (e.g., 1143 vs 25),
sending all entries in a single consolidation prompt overflows the
context window and the 4096 output token budget can't express enough
delete ops. The LLM can only output ~80-100 ops per call.

Adds batched consolidation: when entries > 50, takes the lowest-
confidence entries (tail of the confidence-sorted list) as candidates.
Each pass deletes ~25 entries (half the batch). The idle scheduler's
cooldown clears when entry count changes, triggering the next batch
automatically. Converges to maxEntries over multiple idle ticks.

For small overshoots (≤50 entries), behavior is unchanged — all
entries sent in a single prompt.
@BYK BYK force-pushed the fix/batched-consolidation branch from 93b2ae2 to e8a73f9 Compare May 27, 2026 13:35
@BYK BYK merged commit 3a4159c into main May 27, 2026
7 checks passed
@BYK BYK deleted the fix/batched-consolidation branch May 27, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant