Skip to content

Operations

Peter Pak edited this page Jul 15, 2026 · 14 revisions

Operations

Deployment model, runbook sequences, and the gotchas that were learned the hard way. (The repo's CLAUDE.md is the compact agent-facing version of this page.)

Services

Three systemd user units (deploy/systemd/, installed via deploy/systemd/install.sh, linger enabled):

Unit Port Role
agentic-recorder :3000 Fastify recorder (telemetry, frames, build lifecycle, registry API)
agentic-broker :3100 Harness chat broker (SSE)
agentic-web :5173 React/Vite dashboard

Units run tsx without watch — editing server/src or harness/ does NOT hot-reload. A deploy is an explicit systemctl --user restart agentic-<name> at a safe moment.

Never restart the recorder while a print is being recorded. Check /api/health/recording first; for a graceful maintenance window use POST /api/admin/stop-after-build. Logs: journalctl --user -u agentic-recorder -f.

The recorder lifecycle self-heals: startup orphan reconciliation, spool recovery (15 s delay so the job detector re-adopts an active build first), unreachable-printer finalization.

Post-print refresh sequence

  1. uv run sls-export — refresh data/exports/
  2. uv run sls-summarize-builds — rebuild build_summaries
  3. New PrintSessions? rsync ppak@inova:/home/ppak/SLS4All/PrintSessions/ into SLS4All-Backup + the Database repo, then uv run sls-sync-reference
  4. New builds need a sls-match-build-sessions propose/review pass (live session stamping not yet built — see Architecture)

Tests

  • Root: uv run pytest
  • plugin/: npm test + npm run typecheck + npx tsx tests/smoke.ts (needs DATABASE_URL exported)
  • Headless agent run: npx tsx harness/run.ts <harness> --prompt '…'

Printer-side deploys

See Inova API Plugin for the build.sh → install.sh → reboot loop. Destructive printer actions (reboot/kill/restart) are user-executed — agents recommend and verify only. One-shot firmware restarts default to sudo reboot over the firmware's soft-restart hooks.

Conventions

  • TypeScript over JavaScript for new JS-adjacent code; Python scripts run via uv run; CLI tools installed with brew.
  • Postgres: two-arg round() needs a ::numeric cast.
  • MCP knowledge tools are read-only by construction (forced read-only pool); agent_actions logging of every *_set is expected behavior.

Gotchas (hard-won)

  • Old-era telemetry (pre-2026-07-13-restore) lives only in data/exports/telemetry/*.parquet — summarize from parquet, never --source postgres for big builds.
  • Profile names can lie (a "20mJ/mm" profile containing 15) — read the JSON field.
  • Sandboxed shells can't see listening ports — ss/curl under the sandbox falsely report healthy services as down; verify health with an unsandboxed curl.
  • Wedged watchers (historical, from the pre-systemd tsx watch era): an orphaned process holding a port while a childless watcher ignores file edits. Diagnose with ss -tlnp and compare the owning PID against ps. The systemd migration removed the day-to-day exposure; server/scripts/run-recorder.sh remains for dev only (flock-guarded).
  • Worktree development during prints: for non-trivial recorder changes while recording, build in a git worktree and merge at a safe moment (worktrees need node_modules + submodule symlinks).
  • Storage tiering exists for a reason: Postgres lives on an SMR HDD that cannot absorb 10 Hz inserts — see Data Plane before "simplifying" the spool away.

Clone this wiki locally