fix(sqlite): move every long-lived daemon store off WAL to rollback journalling

[ziomik]

Why

Production incident on 2026-08-01: four daemon stores (conversations, tasks, agent_comms, research) were found holding …-wal (deleted), with 54 messages, 5 inbox rows and 2 tasks reachable only from the daemon's address space. /proc/<pid>/locks showed the daemon holding no locks at all.

The failure mode is structural, not a one-off:

1. Every store opens the same database file once per thread (self._thread_local.connection), so one process holds several independent connections to one inode.
2. POSIX record locks are owned per (process, inode), not per descriptor. The moment any of those connections closes — a worker thread ending, an explicit _reset_connection() — the process drops all of its advisory locks on that file, including the ones the surviving connections still rely on.
3. The daemon is then invisible to other processes. The next external opener (a backup, an operator running sqlite3, a script) sees an unlocked database, believes it is the sole connection, and on close checkpoints and unlinks the -wal/-shm.
4. The daemon's surviving connections keep writing into the now-orphaned inode. Those writes are visible only through the daemon; every external reader sees data frozen at the last checkpoint, and the whole tail is lost when the process exits.

Note that #956's _reset_connection() healing closes a connection — i.e. it is itself a trigger for step 2. That pattern is untouched here; under rollback journalling there is no -wal for anyone to unlink, so the trigger no longer has a payload.

What

New module src/pinky_daemon/sqlite_journal.py with configure_rollback_journal(conn, db_label=…):

• checkpoints any hot WAL first, so nothing committed is stranded when the wal-index is dropped;
• switches to journal_mode=TRUNCATE with bounded retries and busy_timeout (default 30s, same value as fix(sqlite): add PRAGMA busy_timeout=30000 to all thread-local store connections #942);
• fails loud (SqliteJournalConfigError) rather than silently running on WAL — a silent fallback is exactly the state that loses writes.

It is called from the thread-local connection factory of every long-lived daemon store, so each per-thread connection is configured, not just the first:

activity, agent_comms, apps, audit (hooks.AuditStore), conversations, dreams, mesh, message_context, outreach_config, presentations, research, sessions (×2), skills, tasks, triggers, user_profiles, voice_calls — plus the per-call connections in kb_store, librarian_runner, analytics_store and plugin_manager (×2).

This converges the hand-rolled copies already living in conversations_agents.db (#797/#220) on one implementation instead of letting them drift.

dreams.db and skills.db are included: the module docstring notes them as already-remedied, but the thread-local refactor put both back on PRAGMA journal_mode=WAL in their connection factory. This restores them.

Trade-off, deliberately accepted: rollback mode serialises readers against the writer. These are low-throughput control-plane stores; correctness of the write path matters more here than reader concurrency.

Out of scope

Still on WAL, untouched by this PR: pinky_identity (2 sites), pinky_hub, pinky_federation, pinky_memory (2 sites). Worth a follow-up.

Tests

• New tests/test_sqlite_journal.py: mode conversion, hot-WAL drain, retry/fail-loud behaviour, and a subprocess test pinning that the FTS5 shadow tables in conversations.db survive the checkpoint on a genuinely hot WAL (not just on the reasoning).
• The existing per-store concurrency hammers now assert truncate instead of wal on each thread-local connection.

🤖 Opened by Engineer