Repository navigation
Data Plane
The recorder stack: how print data gets from the printer into Postgres, and why it's shaped that way. Design overview: Architecture §A.
-
server/— Node + Fastify recorder on:3000(ESM, strict TypeScript). Subscribes to the Inova API Plugin's/state/streamWebSocket, polls the firmware's port-80 endpoints for camera/thermal/galvo images and rich status, and detects build start/end from the firmware heartbeats (500 ms cadence). -
Postgres — docker container
agentic-sls-postgres(pgvector/pgvector:pg16, defined inservices/postgres/compose.yml), datadir bind-mounted atdata/postgres/(co-located with the repo, trivially backed up, safe fromdocker compose down -v). A pgAdmin UI (services/pgadmin/) sits alongside on:5050. Chosen over Timescale/Influx/ClickHouse because MK1 scale (<1M rows/day) doesn't need them, pgvector covers future embeddings in the same DB, and LLMs speak SQL. Timescale remains a nondestructiveCREATE EXTENSIONaway if ever needed. -
Frames — camera/thermal/galvo images land in
data/frames/<build_id>/with metadata rows in theframestable.
High-rate streams never INSERT into Postgres during a print. The original design did, and Postgres on the SMR HDD (/mnt/storage2) couldn't keep up — WAL fsync stalls pushed ingestion lag to ~2.5–3 s against a 3 s health budget, causing the dashboard's RECORDING/NOT RECORDING flapping.
The fix (deployed 2026-07-01, mid-build-40):
- During a print, telemetry / position / plotter / frame-metadata streams append raw NDJSON to a spool on the NVMe (
SPOOL_DIR, default~/.agentic-sls/spool/<buildId>/<stream>.ndjson). Append lag: 10–50 ms. - After the build closes,
recorder/importer.tsbulk-imports into Postgres. Idempotent viaspool_importedmarker events written in the same transaction; files renamed*.ndjson.importedafter commit; startup recovery is delayed 15 s so the job detector can re-adopt an active build first. - Imported rows are identical to live-recorded ones — the importer reuses the recorders' own row mapping and takes timestamps from the frame payloads.
Low-rate writes (builds, events, health events) still go to Postgres live. /api/health/recording measures in-memory spool freshness, not MAX(ts) queries.
| Stream | Source | Destination | Cadence |
|---|---|---|---|
| telemetry | plugin WS /state/stream
|
telemetry table |
10 Hz (TELEMETRY_HZ; plugin can do 100 Hz, deliberately not raised) |
| position | plugin WS /movement/position/stream
|
position_hf |
event-driven — machine axes only move during recoat, so bursts between long silences |
| bedmatrix | plugin WS thermal matrix | JSON frame files + frames rows |
1 Hz (native 6 Hz; storage-conservative) |
| camera/thermal/galvo | firmware :80 image endpoints |
data/frames/ + frames rows |
5/2/5 Hz |
| job detector | firmware :80 status endpoints |
builds + events
|
500 ms |
| plotter_commands | plugin | plotter_commands |
ImageCodePlotter singleton, and the intended feeder LoggingMovementClient is disabled over a boot-hang DI cycle — see Roadmap
|
At build-open, builds.params stores the full raw firmware status payloads — the LLM picks fields later; don't over-curate at write time.
-
Recording:
builds(⊕inova_session_id),telemetry,position_hf,frames,events,plotter_commands -
Reference (synced copies):
inova_jobs,inova_print_profiles,inova_print_sessions,astm_specimens— fix data at the origin repo, re-runuv run sls-sync-reference -
Derived:
build_summaries,build_registry(the linkage view) -
Agent:
agent_conversations→agent_sessions→agent_messages,agent_actions
The recorder mints integer build_ids; the firmware mints PrintSession UUIDs. sls-match-build-sessions maps them by timestamp-interval alignment with hand review in data/exports/build_to_inova_session.csv (46/46 mapped). One firmware PrintSession can span multiple recorder builds (restarts/false starts). Live stamping of the session UUID at print start is the one remaining gap — until the plugin exposes it, each new build needs a matcher propose/review pass.
The pre-2026-07-13 era was restored and renumbered into a single 46-build history, but old-era raw telemetry lives only in data/exports/telemetry/*.parquet, not Postgres. Summarize big old builds from parquet, never --source postgres.
See also: Operations for deploy/restart rules · Datasets for the export contract