Pi Daemon

Pi Daemon is a standalone, general-purpose service that multiplexes many on-demand Pi SDK agent sessions inside one long-lived Node process.

It shares process-global Pi module code, provider authentication storage, and model metadata while retaining a separate session tree, settings instance, command queue, events, and idempotency state for every logical session. The implemented no-tools scaffold creates no process per session or wake. The full target retains that unisolated in-process mode while adding durable session CRUD by ID/name, an additive authenticated JSON API, multi-reader Pi RPC attachment, an ACP adapter, and a stock-compatible RPC stdio attach client. Stronger isolation modes are future backends rather than an in-process security claim.

Pi Daemon is not a Cacophony component. Cacophony can deploy and consume it, but the service protocol contains no beads, Cacophony agents, messages, profiles, or Cacophony credentials.

Status: durable resident/dormant CRUD, bearer-authenticated JSON admission, asynchronous mutation/prompt tickets, exact Pi conversation recovery, bounded serialization, per-session runtime configuration, truthful bounded recovery/shutdown, full multi-reader Pi RPC attachment, the remote pi-daemon-rpc stdio bridge, in-process ACP translation, and the first Pi Daemon Dash foundation are implemented. Dash now includes a packaged content-hashed SPA, preview-only transcript projection, secure browser BFF, signed revocable cookie exchange, revisioned workspace/UI-settings stores, a native durable IANA-timezone scheduler with restart-safe prompt tickets and Dash countdown/editor/history, persisted 10k-session inventory, rich transcript rendering, durable direct/fork ownership and export, the neutral service-bearer Dash API, coalesced embedded and dedicated Rich/TUI backends with bounded reconnect, and a bounded in-process shadow-terminal host, authenticated browser stream routing, and a standalone dedicated pi-daemon web lifecycle over the remote backend, and the production same-origin browser client for input-only login, persisted inventory/workspace/settings, preview-first hydration, multiplexed Rich/TUI streams, correlated commands, replay recovery, liveness, extension UI, inert server-validated declarative extension views with scoped actions/TUI fallback, a virtualized full branch-tree navigator with compare/edit/fork/clone and capability-gated in-place summarize/navigation, durable activation-recency ordering independent of source mtime, owner-configurable home/Pi-settings/ trusted-policy New Session defaults, revisioned Enter-send/Shift-newline composer behavior with a multiline alternative, an administrator-only safe Diagnostics panel with bounded normalized service events, and a polished lazy New Session flow that performs no Pi runtime/model/tool work until its exactly-once first message. Full credential-free install/CRUD/RPC/ACP/restart/security acceptance is green; the repository is a release candidate, but no release tag is cut yet.

Start here: Operator quickstart — run collision-free Home Manager instances, use the authenticated session API, and attach RPC or ACP clients without putting the service bearer in process arguments.

Fresh-node test runner: from a checkout, run nix develop -c just test-daemon. It creates a safe node-local config if absent, Nix-builds/tests exact main, and starts an isolated non-service tmux instance. See Rolling non-launchd test instance.

Why

Starting one pi --mode rpc process for every logical agent or every cold wake repeats Node startup, module loading, provider/model setup, and process supervision. Pi's supported SDK can host multiple independent AgentSession instances in one Node process. Pi Daemon turns that capability into a bounded, durable, observable service with a neutral local protocol.

A feasibility probe created two concurrent live SDK sessions in one process, shared AuthStorage and ModelRegistry, received independent model responses, and observed zero child_process calls during session creation or no-tool turns.

CLI

pi-daemon serve --socket /run/user/1000/pi-daemon.sock \
  --state-dir ~/.local/state/pi-daemon \
  --allow-root ~/work

# Optional authenticated JSON/WebSocket admission boundary.
# With no explicit bearer source, first launch creates and reuses the
# owner-only token at STATE_DIR/api-token:
pi-daemon serve --socket /run/user/1000/pi-daemon.sock \
  --state-dir ~/.local/state/pi-daemon --allow-root ~/work \
  --api-bind 127.0.0.1 --api-port 7463

# The same process can expose its embedded Dash on a distinct loopback port;
# it shares the resident session catalog and never spawns `pi-daemon web`:
pi-daemon serve --socket /run/user/1000/pi-daemon.sock \
  --state-dir ~/.local/state/pi-daemon --allow-root ~/work \
  --api-enabled true --api-bind 127.0.0.1 --api-port 7463 \
  --web-enabled true --web-bind 127.0.0.1 --web-port 7464

# Equivalent service values may come from
# ~/.config/pi/daemon/INSTANCE/config.yaml; individual CLI flags override YAML.
pi-daemon serve --config ~/.config/pi/daemon/work/config.yaml --instance work

# Dedicated Dash over the authenticated service API (defaults: API 7463,
# dedicated Dash 7465; the token is read from its owner-only file).
pi-daemon web --config ~/.config/pi/daemon/work/config.yaml --instance work

pi-daemon probe --socket /run/user/1000/pi-daemon.sock
pi-daemon request --socket /run/user/1000/pi-daemon.sock --json '{...}'
pi-daemon version

# Present a retained session as stock Pi RPC JSONL on stdin/stdout.
# PI_DAEMON_BEARER_TOKEN is memory-only; --token-file/--token-fd are preferred.
pi-daemon-rpc --url http://127.0.0.1:7463 --session exact-id-or-name \
  --token-file ~/.local/state/pi-daemon/api-token

With an enabled embedded web block, the same serve process starts the packaged browser BFF after its owner socket/API are ready. Open /dash/ on the configured loopback web port (for example http://127.0.0.1:7464/dash/). Enter the generated owner-private STATE_DIR/web-token, configured legacy web.auth.tokenFile, or your operator-issued identity credential. The SPA exchanges it once for an HttpOnly browser session and never stores it. A web.mode: dedicated configuration is instead served by pi-daemon web, which uses the service API on 7463 and defaults its independent browser listener to 7465. Never put either service or web credentials in the URL.

Remote browser deployments should keep the listener on loopback behind an HTTPS reverse proxy or configure native HTTPS/WSS with an exact web.publicOrigin plus bounded certificate/private-key file or inherited-fd sources. Native TLS enforces matching SNI, Host, Origin and optional loopback proxy evidence, emits HSTS with Secure __Host- cookies, and atomically reloads valid file-backed pairs while retaining the last good context on failure. A reviewed high-trust exception may bind plaintext Dash to 0.0.0.0 or :: only when an exact non-loopback web.publicOrigin and web.allowInsecureHttp: true are both explicit; startup emits dashboard_insecure_http_exposure, while authentication and exact Host/Origin checks remain mandatory. GET|HEAD /dash/healthz is a content-free no-store transport probe; /dash/readyz additionally performs one fresh, bounded dedicated-backend check while revealing no backend data. Certificate/key bytes never enter YAML, argv, Nix store values, status, or logs. See Dashboard transport security.

Every browser HTTP, inventory, ticket, draft, schedule, Rich, and TUI boundary now enforces the identity-bound owner-private central read/control/admin ledger. Inventory uses bounded principal/query-bound opaque cursors; absent and unauthorized resources share one content-free denial; live streams revalidate policy and provider state before sensitive frames/events. Revisioned grant, workspace sharing/revocation, ownership transfer, content-free audit, and an explicit release-before-grant controller handoff are available through the BFF and accessible SPA administration dialog. Multi-user activation is explicit: configure a bounded static identity provider in strict YAML or select a strict provider document with --web-identity-provider-file; identity credential bytes come only from owner-only files or inherited descriptors. Home Manager can emit provider metadata containing runtime secret paths, never bytes. With no provider, existing deployments remain the exact generated-token local-owner mode. The remote backend still receives only its separate machine bearer. See Dashboard identity and authorization.

Both executables treat an EPIPE from stdout or stderr as a normal early-closing Unix pipeline consumer and exit quietly with status 0. Other stream errors remain fatal and are never hidden by the closed-pipe guard.

On first launch, serve creates and validates its private state, socket, and Pi agent directories. When a custom --agent-dir has no auth.json, it seeds once from Pi's normal owner-private auth file if present; --auth-seed-file selects a required source explicitly. Existing auth and bearer files are never overwritten or rotated.

serve uses one process-global Pi ModelRuntime and credential store from the configured --agent-dir while creating one AgentSessionRuntime with isolated session managers, settings, resource loaders, and rebound event subscriptions per logical session. Authenticated creation supports bounded typed Pi-equivalent model/tool/resource/settings policy and a memory-only environment overlay. Because arbitrary extensions share the process trust domain, they require an explicit trusted project/configuration policy; unisolated is not a sandbox. Dashboard activations use the owner-configured web.runtimePolicy for that bounded authority and otherwise remain no-tools/no-extensions. An explicit resources.inheritInstalledPackages: true loads only global Pi packages already installed under the configured agent directory, honoring package manifests and filters without installing, updating, spawning a package manager, or using the network. Direct and fork activation restore the source active branch's provider, model, and thinking level before the first turn; model selection never falls through to arbitrary registry order when the source records an authenticated model. Configured trusted sessions may deliberately enable Pi built-ins, including the cwd-bound bash tool, and explicit reviewed extension tools. The logical AgentSessionRuntime and conversation persist across requests; each session has one active model turn, while the host's bounded global semaphore runs independent sessions concurrently. A bash call is a scoped child invocation, not a guaranteed persistent shell or PTY, and intra-session parallel command execution is not part of the public contract.

The protocol is versioned UTF-8 NDJSON over an owner-only Unix socket. Host status and handshake responses include bounded counters, recovery/degraded state, resident-session state, nondestructive redacted adapter readiness, uptime, and memory. Probe returns temporary failure while recovering/degraded. Service logs omit prompts, outputs, credentials, and private paths. SIGTERM uses a 30-second whole-shutdown deadline, SIGINT five seconds, and idle SDK sessions are evicted after 30 minutes by default (--idle-session-ttl-ms 0 disables it).

The language-neutral v1 protocol.schema.json, additive v2 protocol-v2.schema.json, scoped tool-adapter.schema.json, checked fixtures under fixtures/, and exported TypeScript protocol types are the compatibility contract. Protocol v2 additionally advertises strict configured open support for an isolated per-session agentDir and a deterministic sessionDir confined to its canonical Pi sessions subtree; v1 remains unchanged. See the protocol design and, as implementation lands, the published documentation site.

Safety defaults

Legacy Unix NDJSON v1 and unconfigured browser activations deliberately start narrow. These defaults are not a product-wide capability limit; the authenticated Session API may explicitly opt a mutually trusted session into the configured runtime described above.

• no built-in tools until configured session policy enables them;
• no arbitrary project extensions until explicit trusted resource policy enables them;
• explicit canonical working roots;
• owner-only local socket;
• bounded requests, queues, sessions, turns, buffers, and drain;
• content/auth redaction in logs and status;
• no blind replay after an indeterminate accepted request;
• durable wakes replay only after the exact resolved Pi conversation reopens;
• memory sessions are resident-only and never journaled for crash replay;
• one isolated AgentSessionRuntime, SessionManager, and settings domain per logical session.

A shared process is a shared trust boundary, not a sandbox. Explicit trusted sessions may load extensions and process/filesystem tools in that domain; mutually untrusted workloads need separate daemon processes, containers, or VMs. See SECURITY.md.

Development

Requirements:

• Node.js 22.19 or newer;
• npm;
• Nix with flakes (recommended).

nix develop
npm ci
npm test
nix flake check

The Pi SDK version and npm dependency graph are pinned. Strict TypeScript is the source language; built JavaScript and declarations are emitted under dist/.

Caravan policy

The repository-owned .caravan/config.yaml selects the reviewed GitHub-native Stack backend. Cacophony owns the external ten-minute cara sync --all schedule; this repository deliberately defines no second loop. Inspect policy without provider mutation with:

cara config check
cara --json status

config check is offline. status performs provider reads only; neither command joins, rewrites, labels, or merges a pull request. The configured lifecycle calls scripts/cara-runtime.sh, which selects a launchable system Cara, strips loader-only environment overrides, and emits a bounded, secret-redacted runtime receipt before execution.

Nix consumer contract

The checked-in flake exposes packages.default, packages.pi-daemon, packages.npm-deps, apps.default, apps.pi-daemon, apps.pi-daemon-rpc, homeManagerModules.default (homeManagerModules.pi-daemon), package/site/module checks, and devShells.default on Linux and macOS:

inputs.pi-daemon.url = "github:harryaskham/pi-daemon";
inputs.pi-daemon.inputs.nixpkgs.follows = "nixpkgs";

The standalone lock follows the fleet's warm nixpkgs baseline; consumers should use nixpkgs.follows so Pi Daemon shares their own evaluated package set. A release lane can pre-materialize the exact fixed-output npm cache with nix build .#npm-deps; once cached, the same command with --offline proves no ambient registry is needed. First materialization retries only recognized transient transport failures and still fails integrity/identity errors immediately. A Cacophony node can therefore consume the reproducible package without copying service source into Cacophony. The Home Manager module creates independently named user services through systemd on Linux, launchd on Darwin, or conditional supervisord on nix-on-droid. API-enabled instances also receive a separate semantic watchdog by default: it distinguishes HTTP responses from a mere PID or listener, records slow responses without restarting, and permits one exact-instance recovery per failure epoch rather than forming a restart storm. See Operations. An opt-in stable service shim can prefer verified atomic releases from ~/.local/bin/pi-daemon while retaining the immutable Nix package as fallback, so subsequent pi-daemon update runs do not require a full system rebuild; see User-local release updates. A separate non-cancelling Attic publisher builds, executes, and pushes the exact package closure for all four supported Linux/Darwin architectures, with per-target runner-capability checks and signed-substituter preservation; see Supported-system closure cache.

Documentation

• docs/quickstart.md — secure Home Manager, session API, RPC, and ACP operator quickstart
• PLAN.md — architecture, protocol, rollout, and provisional beads
• docs/session-api.md — additive session CRUD, Pi RPC attach, and /apc ACP contract
• docs/blob-transfer.md — neutral SHA-256-addressed streaming uploads, quarantine, exact-generation authorization, daemon-owned references, and cleanup
• docs/tool-adapter-protocol.md — protocol-v2 host/session-bound fixed filesystem capability
• docs/dashboard-protocol.md — Dash browser/backend contract, limits, identity, and performance budgets
• docs/dashboard-session-tree.md — virtual branch navigation, active-leaf truth, compare/edit/fork/clone, and framed summarize/navigation
• docs/transcript-projection.md — preview-only Pi JSONL active-branch projection, cache, paging, and bounds
• docs/dashboard-ownership.md — direct/fork ownership, leases, conflict guards, export, and release
• docs/dashboard-session-drafts.md — lazy no-runtime session drafts and first-send crash semantics
• docs/dashboard-service-api.md — neutral service-bearer Dash API and TUI negotiation for remote backends
• docs/dashboard-acceptance.md — live dual-mode browser, security, bundle/performance, and soak receipts
• docs/shadow-tui.md — in-process virtual terminal, control policy, performance proof, and Pi view seam
• docs/session-cli.md — high-level session CRUD, tickets, prompt/control, and endpoint discovery
• docs/schedules.md — native durable timers, authenticated schedule CRUD/status, ETags, safe file-backed CLI/config imports, and external-timer coexistence
• docs/scheduler-acceptance.md — measured all-IANA DST, clock-jump, restart, overlap, secrecy, and soak release receipt
• session-api.schema.json / session-api.openapi.json — machine-readable session API
• SECURITY.md — trust boundary and vulnerability reporting
• CONTRIBUTING.md — development workflow
• CHANGELOG.md — release history
• docs/acceptance.md — live multiplex/zero-child-process proof
• docs/pi-sdk-compatibility.md — exact SDK acquisition, compatibility gates, upgrades, and rollback
• docs/pi-rpc-host.md — in-process full Pi RPC command/event/UI semantics
• docs/rpc-bridge.md — authenticated stock-RPC stdio client and reconnect semantics
• docs/acp-adapter.md — in-process upstream ACP translation at the /apc route
• docs/release.md — release and rollback checklist
• docs/ — published protocol, operations, security, and integration guides

License

MIT. See LICENSE.