Skip to content

Phase 1: cross-agent capture daemon (macOS + Linux) #71

@ashu17706

Description

@ashu17706

Context

Smriti's first long-running process. Captures sessions across all configured coding agents (Claude, Codex, Cline, Cursor, Copilot) so users don't have to remember which agent they used or to run `smriti ingest ` manually.

Full design and rationale in `docs/internal/daemon-prd.md` — that document is the source of truth; this issue tracks the work.

Supersedes #7 and #70.

Scope

A single-process daemon, started automatically at user login on macOS and Linux, that:

  • Watches all configured agent log roots via `fs.watch` (macOS/Windows recursive natively, Linux walks the tree)
  • Debounces per-project for 30s, then runs the existing ingest pipeline (`src/ingest/`) in-process
  • Accepts a poke from the Claude Stop hook over a Unix socket as an additional signal
  • Survives crashes (re-triggered by next FS event) and reboots (via LaunchAgent/systemd-user)
  • Writes to the same SQLite the CLI reads from — no protocol, no IPC for reads

The daemon is parse-and-write only. The existing CLI surface (`smriti search`, `smriti recall`, `smriti share`, `smriti embed`) keeps working unchanged.

Tasks

Daemon core (`src/daemon/`)

  • `src/daemon/server.ts` — Unix socket bind (`/.cache/smriti/daemon.sock`), single-instance guard via bind failure, signal handling (SIGTERM/SIGINT graceful), diagnostic PID file at `/.cache/smriti/daemon.pid`
  • `src/daemon/watcher.ts` — `fs.watch` wrapper with Linux recursion fallback (walk + per-dir watch + dir-create re-watch). Consider `chokidar` as the abstraction
  • `src/daemon/queue.ts` — per-project debounce queue (default 30s, configurable via `SMRITI_DAEMON_DEBOUNCE_MS`)
  • `src/daemon/handlers.ts` — socket connection handler for the Claude Stop hook poke
  • `src/daemon/client.ts` — `smriti daemon stop` / `status` socket client helpers
  • Wire the daemon to call the existing orchestrator in `src/ingest/index.ts` in-process — no subprocess spawning

Service installer (`src/daemon/install.ts`)

  • macOS: generate `~/Library/LaunchAgents/dev.zero8.smriti.plist`, register via `launchctl bootstrap` (or `launchctl load` for older macOS)
  • Linux: generate `~/.config/systemd/user/smriti.service`, register via `systemctl --user daemon-reload && systemctl --user enable --now smriti`
  • Idempotent: running `smriti daemon install` twice produces the same registered service, no duplicates
  • `smriti daemon uninstall` reverses everything — service file removed, daemon stopped, no orphaned state

CLI subcommands (in `src/index.ts`)

  • `smriti daemon` — run in foreground (debugging, systemd target)
  • `smriti daemon install` / `uninstall`
  • `smriti daemon status` — PID, uptime, pending queues, last ingest per project
  • `smriti daemon stop` — graceful via socket, fallback PID-file SIGTERM
  • `smriti daemon logs` — tail `~/.cache/smriti/daemon.log` (rotating)

Hook update

  • Update the documented `~/.claude/hooks/save-memory.sh` template to poke-with-lockf-fallback:
    ```bash
    #!/bin/bash
    SOCK="$HOME/.cache/smriti/daemon.sock"
    if [ -S "$SOCK" ]; then
    : | nc -U "$SOCK" 2>/dev/null
    else
    /usr/bin/lockf -t 0 /tmp/smriti-ingest.lock smriti ingest claude 2>/dev/null
    fi
    exit 0
    ```

Tests

  • Socket bind contention (second daemon exits cleanly when first holds the socket)
  • Debounce coalescing — N rapid writes within window produce 1 ingest
  • FS watch correctness under macOS recursive + Linux walk-and-watch
  • Auto-install idempotency (run install twice, no duplicates)
  • Stale socket cleanup after SIGKILL
  • Claude poke handler ingests the right project

Docs

  • Update `CLAUDE.md` quick-reference with `smriti daemon` commands
  • Update `docs/internal/ingest-architecture.md` to note the daemon trigger layer
  • Link `docs/papers/stop-hook-never-stopped.md` and `docs/papers/only-by-staying.md` from the daemon PRD

Acceptance criteria

From the PRD:

  1. After `smriti daemon install`, the daemon survives a logout/login cycle and a full reboot without user intervention.
  2. Opening Cursor on a new project, doing some work, and never running a smriti command — that project's sessions appear in `smriti search` within `30s + ingest_time` of the work being saved.
  3. Claude Code's Stop hook completes within 50 milliseconds when the daemon is running.
  4. SIGKILLing the daemon leaves no stale socket, no stale PID file, no corrupt SQLite state. Re-running it works cleanly.
  5. Running `smriti daemon install` twice produces idempotent results — same plist/service file, same registered job, no duplicates.
  6. `smriti share` continues to work, unchanged, with its existing sanitization. No new redaction error paths.
  7. Removing the daemon (`smriti daemon uninstall`) leaves the system in exactly the state it was before installation — no orphaned files, no lingering processes.

Out of scope (deferred to later phases)

Enumerated here so the boundary is unambiguous:

  • No redaction pipeline yet. `smriti share` keeps its current sanitization. A real redaction pipeline is the next phase after this.
  • No read-side routing. `smriti search` / `recall` remain one-shot CLI invocations — ~150ms cold-start is tolerable, not worth daemon-routing complexity.
  • No QMD MCP daemon coordination. Both daemons can run concurrently; SQLite WAL handles it.
  • No QMD upstream proposal (`searchFTS({ joins })` is orthogonal).
  • No Windows. macOS and Linux only.
  • No embedding model in the daemon. `qmd embed` stays the way embeddings happen — adding it to the daemon would duplicate QMD's model load and add ~300–500 MB resident for no gain.

Design notes

A summary of the reasoning behind the choices above, for future maintainers landing on this issue.

Why a daemon at all?

The lockf mitigation in `~/.claude/hooks/save-memory.sh` solves the Claude pile-up problem. But it only covers Claude — Codex CLI, Cursor, Cline, and Copilot have no equivalent Stop hook. Their session logs sit on disk and only enter Smriti when the user remembers to run `smriti ingest `. The actual product gap is "I switch coding agents during the week and my memory has holes." A filesystem watcher closes that gap; nothing simpler does.

Why auto-start at login and not a foreground `smriti watch` command?

A foreground command means users have to remember to run it after every reboot. The point of capture is that it happens whether the user thinks about it or not. The cost is per-OS service-file generators (LaunchAgent for macOS, systemd-user for Linux) — meaningful engineering but bounded, and a one-time install step.

Why not capture at commit time (Entire-style)?

Considered seriously. Entire CLI uses git hooks (`post-commit`) to capture sessions, which dodges the daemon problem entirely. The trade-off: commit-time capture loses pre-commit exploratory work (the rabbit holes that don't end in a commit). For Smriti's personal-recall use case ("what did I try yesterday that didn't work?") that exploratory work is the point. Session-boundary capture preserves it; commit-boundary doesn't. We may add a commit-hook layer later as an additional signal, but session-level capture stays primary.

Why no embedding work in the daemon?

The embedding model is ~300–500 MB resident and takes 2–5 seconds to load. QMD already owns embedding via `qmd embed` (one-shot batch CLI). Putting embedding in the daemon would either duplicate the model (if `qmd mcp --daemon` is also running) or fight for ownership of it. Neither is worth it for phase 1, when BM25 search works perfectly well against newly-ingested content. Vectors lag ingest by however long until the next `qmd embed` run — fine.

Why ship without a real redaction pipeline?

Phase 1 is local-only. Nothing in this phase makes captured content leave the user's machine. The existing `smriti share` already does basic sanitization (XML noise, interrupt markers, API errors). That's sufficient for the current curated-knowledge sharing flow. A proper redaction pipeline (high-entropy strings, credentialed URIs, vendor secret patterns, typed placeholders) becomes load-bearing the moment Smriti starts handling raw transcripts at scale — but that's the next phase, not this one.

Why per-project debounce and not global?

A busy session in project A shouldn't delay project B's ingest. Per-project keeps the latency floor at `DEBOUNCE_MS` regardless of how many projects are active. The cost is one timer per active project, which is trivial.

Why a Unix socket for single-instance, not a PID file?

The socket file is kernel-cleaned on process exit — no stale-recovery code. The same socket doubles as the IPC transport for the Claude Stop hook poke. One mechanism, two jobs. (We still write a diagnostic PID file at `~/.cache/smriti/daemon.pid` for `smriti daemon status` and human grep, but it's not load-bearing for mutual exclusion.)

Trajectory

  • Phase 1 (this issue) — cross-agent daemon, auto-start, macOS + Linux.
  • Phase 1.5 — real redaction pipeline. Re-shapes `smriti share` to handle raw transcripts safely alongside the curated knowledge it already produces.

Beyond that, the trajectory tracks what users actually ask for — additional agent integrations, search quality improvements as QMD evolves, and ergonomics around the team-sharing flow.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions