v1.16.0 — cursor-chunked checkpoint + um-drain
Session-end checkpoints read the raw capture backlog cumulatively from zero on every fire, with a single unbounded summarization call. Once the backlog outgrew one model call (~4MB after a month of quiet failure), every checkpoint attempt hit the same cliff, failed, and — because the failure path emitted no counter — reported nothing. Capture looked alive; digestion had been dead since early August. Issue #247 carries the full investigation.
This release makes checkpoint input bounded by construction, and makes the stats surface incapable of hiding an outage like this again.
What changed
- Positional cursor + chunked summarization (#248). Checkpoint now maintains a per-project cursor (
file+ byteoffset, strictly validated, safe-side recovery) and digests the backlog in ≤200KB chunks, at most 3 per run, each a committed transaction: summarize → write doc → merge state → advance cursor. A crash redoes at most one chunk — summary ids are deterministic over the content window, so a redo adopts the orphan instead of duplicating it. Verified by a constructed post-crash disk-state matrix in CI plus a live spawned-process SIGKILL smoke (kill at a mid-run chunk boundary → held lock refused conservatively → stale takeover → drain to EOF with no lost turns). - Stats tell instrumented truth (#249).
outcomes_7dcounts only landings,turns_7dcounts turn appends separately, and summarizer failures land in afailedbucket instead of vanishing. - Per-layer freshness (#250). The stats payload gains a
layersblock (per-project capture/digest freshness),um-alert.shescalates on stale layers, and the control page grew a freshness tile. um-drain.sh(#251). The operator loop that turns "3 chunks per run" into "keep going until done": repeated/api/checkpointPOSTs per project with the full response taxonomy, a cost/pending estimate and confirm gate up front,--probe <project>for a one-project rehearsal, and a legacy-server preflight. Session-end 502s are now disambiguated (stage,provider_class).
Compatibility
- Config: five new chunking keys in
checkpoint.json, all env-overridable (UM_CHECKPOINT_CHUNK_MAX_BYTES,UM_CHECKPOINT_MAX_CHUNKS_PER_RUN, …); shipped defaults preserve existing behavior for small backlogs. - API:
/api/checkpointresponses gain additive fields (chunks_done,backlog_remaining,thin_tail,stopped); the deprecatedtruncatedalias is retained this cycle and will be removed in the next major. - No reindex, no migration; the cursor initializes from existing session summaries (or a full digest from zero for never-summarized projects).
Multi-arch image on GHCR (amd64 + arm64), both variants boot-verified from the published registry before release.