v0.8 [1/4]: Daemon core — server, watcher, queue, handlers

[ashu17706]

Part of #71. The daemon process itself, no service-file plumbing, no CLI wiring yet.

Scope

Create `src/daemon/` with five focused modules:

• `src/daemon/server.ts` — Unix socket bind at `/.cache/smriti/daemon.sock`, single-instance guard via bind-fail, signal handling (SIGTERM/SIGINT graceful drain), diagnostic PID file at `/.cache/smriti/daemon.pid`, log rotation to `~/.cache/smriti/daemon.log`
• `src/daemon/watcher.ts` — `fs.watch` wrapper. macOS: `{ recursive: true }` native. Linux: walk-and-watch with dir-create re-watch (use `chokidar` to abstract; verified Bun-compatible)
• `src/daemon/queue.ts` — per-project debounce queue, default 30s via `SMRITI_DAEMON_DEBOUNCE_MS`, in-process timers, exposes `schedule(projectId)` and `flush(projectId)`
• `src/daemon/handlers.ts` — socket connection handler for Claude Stop hook poke; protocol: any byte received = "wake up and check Claude logs"
• `src/daemon/client.ts` — socket client used by `smriti daemon stop` / `status`. Parses daemon socket messages

The watcher's events feed the queue; the queue's timer fires call into `src/ingest/index.ts`'s `ingest()` function in-process (no subprocess).

Verified prerequisites

• ✅ `ingest()` and `ingestAll()` are clean async exports from `src/ingest/index.ts` — importable standalone
• ✅ Bun 1.3.6 supports Node-compatible `fs.watch` with `{ recursive: true }` on macOS

Acceptance

• `bun src/daemon/server.ts` runs in foreground, binds socket, exits cleanly on SIGTERM
• Second instance attempting to bind same socket exits 0 with a "already running" message
• Touching a file under any configured agent log dir triggers ingest after debounce window
• Sending bytes to the socket triggers immediate Claude ingest
• SIGKILL leaves no stale socket file blocking the next start

Out of scope (other issues in milestone)

• Service-file generation (v0.8 [2/4]: Service installer — LaunchAgent + systemd-user #73)
• CLI subcommands (v0.8 [3/4]: CLI subcommands + hook template #74)
• Tests (v0.8 [4/4]: Tests + release readiness #75)

[ashu17706]

⚠️ Pre-impl smoke test findings — design changes required

Ran the three verification tests recommended in the parent issue. Two findings that change the implementation plan.

Finding 1: Don't use chokidar — use native fs.watch 🔴

Test: chokidar 5.0.0 vs native node:fs.watch watching ~/.claude/projects for 8s with self-touched files at t=2s and t=4s.

Library	Events fired
chokidar 5.0.0	0 (watcher reaches ready but no events on real writes)
Native fs.watch with { recursive: true }	4 (caught self-touches + live writes from this very Claude session)

Chokidar appears to have a Bun compatibility issue. Native fs.watch works perfectly on macOS with recursive: true.

Decision for src/daemon/watcher.ts:

• macOS: use Node's fs.watch(root, { recursive: true }) directly. No abstraction layer.
• Linux: walk-and-watch fallback (Linux fs.watch doesn't support recursive). Hand-rolled, not chokidar.
• Remove chokidar from package.json if it was added.

Finding 2: Unix socket bind does NOT enforce single-instance under Bun 🔴

Test: open one Bun process, bind /tmp/cx.sock. In the same process, try to bind a second server to the same path.

s1 listening
s2 listening (or errored above)
s1 got: 0 / s2 got: 3

The second listen() silently succeeds and steals all incoming connections from s1. No error event. s1 continues to think it's listening but receives nothing. This is Bun-specific behavior (Node throws EADDRINUSE).

Implication: the PRD's recommended single-instance mechanism (socket-bind contention) is fundamentally broken under Bun. If two smriti daemon processes ever start simultaneously, one silently zombies and the other does all the work — and we can't tell which is which.

Decision for src/daemon/server.ts:
Switch to QMD's pattern (PID file + kill(pid, 0) liveness check). Concretely:

const pidPath = `${HOME}/.cache/smriti/daemon.pid`;
if (existsSync(pidPath)) {
  const existingPid = parseInt(readFileSync(pidPath, "utf-8").trim());
  try {
    process.kill(existingPid, 0);
    // Process is alive — we should not start
    console.error(`Already running (PID ${existingPid})`);
    process.exit(0);
  } catch (e) {
    if (e.code !== "ESRCH") throw e;
    // Stale PID file — clean up and continue
  }
}
// We are the daemon
writeFileSync(pidPath, String(process.pid));
process.on("exit", () => unlinkSync(pidPath));

Unix socket is still used for IPC (the hook poke), but not for single-instance guarding. Order at startup:

1. Check PID file → exit if a live daemon owns it
2. Write our PID
3. Bind the IPC socket
4. Begin watching

Finding 3: Native fs.watch event semantics ✅

Verified that on macOS, native fs.watch with recursive: true fires:

• rename on new file creation
• change on content append (we even saw the live Claude session file fire mid-test)

The PRD's debounce design (per-project 30s window, settle then ingest) works as expected against these event types. No design changes needed for the queue.

What this means for the PRD

I'll update docs/internal/daemon-prd.md to:

• Note that socket-bind single-instance was attempted and ruled out due to Bun behavior
• Switch the documented single-instance mechanism to PID-file + liveness probe
• Remove the chokidar suggestion; mandate native fs.watch

Hard findings before any production code gets written. Both would have been silent footguns at runtime.

[ashu17706]

⚠️ Smoke test finding 3 — Bun crashes on repeated in-process ingest() calls 🔴

The third smoke test (call ingest() 5 times in the same Bun process to verify FD/state cleanup) panicked Bun with a segfault.

FD DELTA: 20 (expect ~0; <5 is fine)
Elapsed: 28599ms | User: 37365ms | Sys: 10264ms
RSS: 3.37GB | Peak: 6.82GB
panic(main thread): Segmentation fault at address 0xC9AB8
oh no: Bun has crashed. This indicates a bug in Bun, not your code.

Three things going on:

a. +20 FDs across 5 ingest() calls

Bounded but not zero. Roughly 4 FDs per call retained. Likely from SQLite prepared statements or transient file handles not closing between iterations. Daemon does thousands of ingest cycles per day — this would compound.

b. RSS peaks at 6.82 GB

A 5-call ingest test should not approach 7 GB of memory. Something inside the QMD-side store gateway / FTS / vector path is holding references across iterations. Could be transcript buffers, could be parser state.

c. Bun panics

A segfault inside Bun itself, triggered by the workload (not our code). Repeatable. Stack-trace URL: https://bun.report/1.3.6/Ma1d530ed9ij... (full URL in the test output). Worth filing upstream with a minimal repro.

What this means for the daemon design

The original assumption was "daemon holds the SQLite DB open for the process lifetime, calls ingest() repeatedly." That assumption is unsafe under current Bun + current QMD.

Three mitigation options, ranked by complexity:

1. Fresh DB connection per ingest cycle — open DB at the start of each debounce flush, close at the end. Defeats some of the daemon's warm-state benefit, but the cost is ~20–50ms per cycle which is fine at a 30s debounce window. Probably the right MVP move.
2. Daemon self-recycle every N cycles — call process.exit(0) after every N ingests; systemd/launchd Restart=on-failure brings the daemon back up cleanly. Hides the leak rather than fixing it.
3. Find and fix the leak upstream — investigate where the FDs/RSS are accumulating in QMD's store layer. Real fix, but takes time.

Recommendation: ship Phase 1 with option 1 (DB connection per cycle). Open a follow-up issue for option 3. File the Bun crash with a repro.

Updated daemon contract

src/daemon/queue.ts should:

async function flushProject(projectId: string) {
  const db = openSmritiDb(); // fresh connection
  try {
    await ingest(db, agentForProject(projectId), { projectId });
  } finally {
    db.close();
  }
}

This is a meaningful design change to bake into the implementation plan before any code lands.

Updated PRD already reflects findings 1 and 2 (chokidar, socket-bind). Finding 3 to be added separately.