Skip to content

v1.16.0 — cursor-chunked checkpoint + um-drain

Choose a tag to compare

@goldenwo goldenwo released this 17 Aug 03:40
· 41 commits to main since this release
a1691a8

Session-end checkpoints read the raw capture backlog cumulatively from zero on every fire, with a single unbounded summarization call. Once the backlog outgrew one model call (~4MB after a month of quiet failure), every checkpoint attempt hit the same cliff, failed, and — because the failure path emitted no counter — reported nothing. Capture looked alive; digestion had been dead since early August. Issue #247 carries the full investigation.

This release makes checkpoint input bounded by construction, and makes the stats surface incapable of hiding an outage like this again.

What changed

  • Positional cursor + chunked summarization (#248). Checkpoint now maintains a per-project cursor (file + byte offset, strictly validated, safe-side recovery) and digests the backlog in ≤200KB chunks, at most 3 per run, each a committed transaction: summarize → write doc → merge state → advance cursor. A crash redoes at most one chunk — summary ids are deterministic over the content window, so a redo adopts the orphan instead of duplicating it. Verified by a constructed post-crash disk-state matrix in CI plus a live spawned-process SIGKILL smoke (kill at a mid-run chunk boundary → held lock refused conservatively → stale takeover → drain to EOF with no lost turns).
  • Stats tell instrumented truth (#249). outcomes_7d counts only landings, turns_7d counts turn appends separately, and summarizer failures land in a failed bucket instead of vanishing.
  • Per-layer freshness (#250). The stats payload gains a layers block (per-project capture/digest freshness), um-alert.sh escalates on stale layers, and the control page grew a freshness tile.
  • um-drain.sh (#251). The operator loop that turns "3 chunks per run" into "keep going until done": repeated /api/checkpoint POSTs per project with the full response taxonomy, a cost/pending estimate and confirm gate up front, --probe <project> for a one-project rehearsal, and a legacy-server preflight. Session-end 502s are now disambiguated (stage, provider_class).

Compatibility

  • Config: five new chunking keys in checkpoint.json, all env-overridable (UM_CHECKPOINT_CHUNK_MAX_BYTES, UM_CHECKPOINT_MAX_CHUNKS_PER_RUN, …); shipped defaults preserve existing behavior for small backlogs.
  • API: /api/checkpoint responses gain additive fields (chunks_done, backlog_remaining, thin_tail, stopped); the deprecated truncated alias is retained this cycle and will be removed in the next major.
  • No reindex, no migration; the cursor initializes from existing session summaries (or a full digest from zero for never-summarized projects).

Multi-arch image on GHCR (amd64 + arm64), both variants boot-verified from the published registry before release.