Skip to content

v1.6.23

Choose a tag to compare

@marmutapp marmutapp released this 21 May 07:01
· 14 commits to main since this release

fix(store): token counts now MAX-upgrade on re-emit (Copilot snapshot staleness)

Closes operator-reported "total cost for Copilot sessions doesn't fully
add up after new session-level updates land." Root cause: snapshot-based
adapters (Copilot's modern adapter) re-emit the same
source_event_id = requestID + ":usage" with REFINED token counts as an
in-flight request progresses (partial → mid → complete state in the
JSONL snapshot's result.metadata). Pre-fix, InsertTokenEvents ON
CONFLICT preserved the EXISTING counts and only upgraded the model
label — so a partial-state row with (in=100, out=0) could permanently
persist even when the final state arrived with (in=150, out=200).

Fix (internal/store/store.go::InsertTokenEvents): change the ON
CONFLICT clause to MAX(existing, new) for every token-count column
(input_tokens, output_tokens, cache_read_tokens,
cache_creation_tokens, cache_creation_1h_tokens,
reasoning_tokens). Token counts are monotonically non-decreasing per
logical event — newer ≥ older always — so MAX is universally correct:
re-emit with identical counts → no-op; re-emit with refined (larger)
counts → upgrade; accidental regression (smaller new value from an
adapter bug) → existing larger value preserved.

estimated_cost_usd follows the same monotonic-upgrade pattern (a
proxy-sourced row may later get a non-zero cost when initial emit had
none; the larger non-zero value wins on conflict).

Live evidence from the maintainer DB (2 of 4 copilot token rows show
the partial-state pattern: request_59a80a92...:usage has in=0, out=177 and request_dbbb2805...:usage has in=0, out=137 — the
complete versions never landed because the conflict-resolution rule
silently dropped them).

Universal scope (not Copilot-only): the fix applies on conflict
across all adapters. JSONL adapters (claude-code, codex, cowork,
antigravity, cursor-hook) emit final-on-write events where re-parse
produces identical counts — MAX returns the same value, no behavioral
change. Snapshot/patches adapters (Copilot modern, potentially
opencode/openclaw if they re-emit) get the refinement upgrade.

fix(watcher): per-file mutex eliminates fsnotify/poller race

Closes operator-reported store.SetCursor: database is locked (5) and
store.UpsertProject: database is locked (5) warnings in the watcher
log. Root cause: processFile is called by TWO independent goroutines
for the same source file — the fsnotify-debounced fire() and the
poller's pollCursors(). Both BEGIN IMMEDIATE against SQLite; one
holds the write lock while the other waits. On slow filesystems
(WSL2 /mnt/c, OneDrive-synced directories) the holder's transaction
can exceed the 30s busy_timeout window, surfacing as
SQLITE_BUSY (5) on the waiter.

Pre-fix the doc-comment claimed the race was "safe via UNIQUE
constraints" — true for data integrity (the loser's retry hits the
UNIQUE index and silently dedupes), but the wasted-acquisition cost +
visible WARN spam was poor. The race also meant the cursor on the
loser's processFile was never advanced — the next poll re-processed
the same bytes a second time, doubling the work for the steady-state
file-growth case.

Fix (internal/watcher/watcher.go): add a sync.Map-backed
per-file sync.Mutex keyed by source_file. processFile acquires
the file's mutex before reading the cursor and releases it after
SetCursor. fsnotify and poller for the same file now queue instead of
race. Inter-file contention is still handled by SQLite's busy_timeout
backoff (unchanged).

Data integrity unchanged: pre-fix retry path via UNIQUE constraints

  • idempotent SourceEventID already prevented data loss. This fix is
    about reducing wasted work and eliminating the misleading WARN logs;
    no operator action required.

fix(cursor,cost): four bugs from the 2026-05-21 cursor adapter audit

Closes the four high-impact findings from
docs/cursor-audit-2026-05-21.md. Each was a silent dropout invisible
to existing tests and CI — only visible by reconciling the live
hook captures at /tmp/cursor-hook-capture/ (Cursor 3.4.20,
2026-05-19) against the live DB.

F1 — model:"default" (Cursor's Auto mode) silently billed $0.
The pricing table had no default entry and no family prefix that
default matched. All 8 cursor token rows in the maintainer DB
(model=default, ~120K input + 7.7K output + 245K cache_read across
the corpus) were billing at zero. Pinned to Composer 2.5 Fast rates
($3 / $15 / $0.30 per 1M) per cursor.com/blog/composer-2-5 — that's
the underlying default model behind the Auto router. Also added
explicit pins for composer-2.5 ($0.50 / $2.50) and
composer-2.5-fast ($3 / $15) so direct model-name picks resolve
without a family-prefix fallback. The new default entry is
override-friendly via Settings → Pricing — Cursor's bundled-usage
economics differ from the API-list-equivalent baked-in rates, and
the maintainer-facing UI surfaces this for adjustment.

F2 — postToolUse duration field name mismatch. Cursor's
postToolUse payload sends "duration": 6.332 (float, in SECONDS);
observer's rawHookPayload struct expected "duration_ms" (int64).
Result: 0 of 79 cursor actions in the live DB had a non-zero
duration_ms. The struct now reads both — DurationSecs (the
postToolUse seconds float) is converted to ms in BuildAfterOutcome,
falling back to the legacy DurationMs for events that genuinely
emit it (afterAgentThought, postToolUseFailure).

F3 — postToolUse tool_output body not captured. The
postToolUse payload carries the tool's response (Read body, shell
stdout, etc.) but OutcomeUpdate had no slot for it. Extended
OutcomeUpdate.Output + Store.UpdateActionOutcome signature to
carry the output through, with FTS5 indexing into action_excerpts
via the new Store.WithIndexer binding (best-effort; index failures
log but don't fail the row update).

F4 — beforeReadFile content body not captured. Cursor's
beforeReadFile payload carries content (the file body cursor just
read). BuildEvent now stamps it on ev.ToolOutput (scrubbed +
capped at 4000 chars matching the cross-adapter convention). With
the indexer wired in handleCursorHook, the body lands in
action_excerpts the same way claudecode's Read body does. Closes
the cross-tool body-visibility gap for cursor.

Plumbing

  • Store gains an optional indexer *indexing.Indexer field with
    a WithIndexer chainable setter. Store.Ingest falls back to
    this when IngestOptions.Indexer is nil — daemon watchers
    continue passing their long-lived indexer through opts; hook
    handlers attach via the chainable setter on hook entry.
  • handleCursorHook (cmd/observer/hook.go) now creates an
    indexing.Indexer from cfg.Compression.Indexing.MaxExcerptBytes
    and binds it to its Store. Both Ingest-path emission
    (beforeReadFile.content) and UpdateActionOutcome-path enrichment
    (postToolUse.tool_output) populate action_excerpts cleanly.
  • Store.UpdateActionOutcome signature extended with
    toolOutput, toolName, target string. Existing callers updated;
    Cursor's OutcomeUpdate now carries the tool name and derived
    target so the FTS5 row matches the surrounding action's columns.

docs

  • docs/cursor-audit-2026-05-21.md — full audit using the
    docs/adapter-audit-playbook.md methodology. Captured field-map
    per event for Cursor 3.4.20. Seven findings ranked by severity;
    eight G-ok rows of verified-correct behavior; carry-forward
    items for future sessions.

Validation

Live empirical evidence from /tmp/cursor-hook-capture/
(post-v1.6.18 captures on Cursor 3.4.20):

  • F1 reproduction: Lookup("default") returned PricingSourceMiss
    before this ship; returns PricingSourceExact with Composer 2.5
    Fast rates after.
  • F2 reproduction: postToolUse-7yNeQw.json has "duration": 6.332;
    test TestBuildAfterOutcome_PostToolUseDurationSecondsTranslated
    pins the seconds→ms translation.
  • F3/F4 reproductions: corresponding test pins for tool_output
    capture and beforeReadFile.content truncation.

Deferred to a follow-up (per audit doc §4)

  • F5 — observability fields (composer_mode, cursor_version,
    stop.status, stop.loop_count, attachments[]) still ignored.
    All five fit actions.metadata (JSON) cleanly; deferred for a
    metadata-batch ship rather than smearing across this one.
  • F6 — soften v1.6.18 walker-dead-code docstring on Cursor 3.4.x.
    Doc-only; not yet edited in this ship.
  • F7 — delete /tmp/cursor-tee-shim.sh and re-register hooks.
    Operator-approval cleanup — kept current here to avoid breaking
    the maintainer's running session.

Downloads

Pre-built binaries for each supported platform are attached below. Linux variants bundle antigravity-bridge.exe next to the observer binary for WSL2 users of the Antigravity adapter.

Platform Asset
Linux x86_64 observer-v1.6.23-linux-x64.tar.gz
Linux arm64 observer-v1.6.23-linux-arm64.tar.gz
macOS x86_64 (Intel) observer-v1.6.23-darwin-x64.tar.gz
macOS arm64 (Apple Silicon) observer-v1.6.23-darwin-arm64.tar.gz
Windows x86_64 observer-v1.6.23-win32-x64.zip

Verify with sha256sum -c SHA256SUMS (or shasum -a 256 -c SHA256SUMS on macOS) from the directory containing the downloads.

Also available via npm: npm install -g @superbased/observer@1.6.23