session-query-sqlite: one corrupt cold log fails every search, and live-session churn prevents the observation from stabilising

[fbeeg]

Symptoms

Two failures in the SQLite session search observation (SqliteSessionQueryEngine._observeStable), both seen on long-lived instances with many stored sessions:

1. One corrupt cold log fails every search. readColdSessionLog throws for a single session (for example a released-format log with a seq gap), the catch around the whole cold-read loop turns it into SESSION_QUERY_PERSISTENCE_FAILED, and nothing is indexed or searchable for as long as that log exists.
2. The observation never stabilises while sessions are being written. samePersistenceSnapshots compares the revision of every listed session, including live sessions whose logs are appended to while a multi-minute cold read runs. Attempt 1 fails the comparison, attempt 2 re-reads every cold log from scratch and fails the same way (STABLE_OBSERVATION_ATTEMPTS = 2). On a busy instance with about 1,400 stored logs the search gave no answer for minutes and then errored with "did not stabilize after one retry".

Proposal

• Treat a non-abort failure while reading or decoding one cold log as "skip this session": it stays in the observation without loaded, so it is neither indexed nor does it delete a row the index already holds. Remember the failing id and revision so it is not re-read on every search, warn once, and read it again when its revision changes. Aborts and a moved persistence binding keep their current paths.
• Compare only cold sessions in the stability check. A live session is indexed from memory by observeLive, never from its log, so its log churn cannot invalidate the observation.

Reference branch (two new specs that fail on master and pass with the change): https://github.com/gkinter/deepseek-harness/tree/fix/session-query-sqlite-observation

vitest run packages/session-query/session-query-sqlite/tests/sqlite.spec.ts: 60 of 62 pass; the other two need the native flock addon, which is not built in my environment, and they fail the same way on master. oxlint clean. A further improvement, not in the branch, is caching cold logs across the two attempts keyed by id and revision. Based on master at 0.2.1-alpha.3.

[micw]

I found a related performance issue even when reconciliation succeeds and nothing needs updating.

With eight attached sessions, searching xyzx took 2,069 ms:

• persistence listings: 713 ms
• remaining observation work: 1,354 ms
• actual SQL query: 1.2 ms

No cold session reads or live index replacements occurred. Unchanged live histories were still cloned, projected, serialized and hashed on every search.

I tested reusing the derived observation while the same Session object's append-only seq remains unchanged. In paired tests against copies of the real index, this removed repeated history processing and reduced selective warm API searches from ~1.8 s to ~0.7 s, with identical ordered results.