Brainstem 1.3.0

First release on the way to 2.0: one database for all state (SQLite or Postgres) and a durable decide/deliver pipeline. Includes the unreleased 2.0 Phase 0 and Phase 1 work. See "Behaviour changes" below before upgrading.

Brainstem 2.0 Phase 0 (safety net), see docs/roadmap-2.0.md:

• CI workflow running the syntax check, unit/integration tests, and acceptance tests on every PR and push to main.
• Tests for the runtime polling loop, shutdown and abort handling, poll-retry re-delivery (1.x behaviour), per-ID ordering and decision caching in the core, config saves, and the SSE stream.
• Multi-core acceptance harness (npm run test:acceptance) that runs several real core processes against shared state. The 2.0 cluster guarantees are tracked as todo tests until Phase 3.
• Architecture decision records for the 2.0 design in docs/adr/.

Brainstem 2.0 Phase 1 (storage abstraction):

• One database for all state, via Kysely: SQLite (default, sqlite:data/brainstem.sqlite) or Postgres (BRAINSTEM_DATABASE_URL=postgres://...). The runtime and the embedded API share one connection.
• Records, checkpoints, telemetry, stats, and auth stores are async and database-backed. Checkpoints are read from the database on every poll, with no per-process cache.
• New ctx.state plugin API for durable plugin state. The Matrix destination keeps its suppression state there, claiming a notification before sending so two processes can't both send it.
• Record writes check the expected revision (groundwork for multi-core).
• Safe first-admin creation under concurrent setup requests; login rate limiting stored in the database and keyed by client IP behind trusted proxies (BRAINSTEM_TRUSTED_PROXIES); retention pruning moved to a periodic maintenance job.
• brainstem migrate, migrate status, migrate import-v1, and migrate copy / sqlite-to-pg commands.
• 1.x state (web-stats.sqlite, checkpoints.json, matrix-notifications.json) is imported automatically on first start; legacy files are never modified.
• CI runs every storage test against both SQLite and Postgres 17.
• Behaviour change: with no records/checkpoints config, state now persists to the database (previously records were in memory and checkpoints in JSON). Explicit memory, json, and sqlite types still work.

Brainstem 2.0 Phase 2 (durable pipeline), see docs/delivery-pipeline.md:

• The runtime is split into poll, decide and deliver stages connected by jobs in the new jobs table. Each stage commits its outputs and the next stage's jobs in one transaction. Checkpoints advance in the same transaction that stores the polled observations.
• One deliver job per (decision, destination). A failing destination is retried on its own with exponential backoff and jitter (runtime.queue.deliver.retry, or retry per destination). It no longer re-triggers other destinations, the decision or the poll, and a slow destination no longer stalls polling.
• Jobs that run out of attempts become dead jobs. The new Deliveries page in the Web UI shows queue depth, the delivery log and dead jobs, with redrive and discard. API: GET /api/pipeline, GET /api/jobs, GET /api/deliveries, POST /api/jobs/redrive, POST /api/jobs/discard.
• Delivery log (deliveries table) keyed by an idempotency key, which is checked before sending. Delivered rows are pruned after runtime.deliveries.pruneAfterDays (14).
• Decide jobs are serialized per observation ID through the queue (queue_key), and record writes are compare-and-set on the revision.
• Jobs hold renewable leases. Jobs from crashed processes are reclaimed, and jobs interrupted by shutdown are handed back without using up an attempt.
• Destination plugin SDK v2 (brainstem.destination/v2): handle(ctx, output) receives the serializable brainstem.output envelope, plus ctx.idempotencyKey and ctx.attempt. v1 destinations keep working through an adapter, which logs a deprecation warning, until 3.0.
• Built-in destinations are on v2:
  • http-json sends an Idempotency-Key header.
  • Matrix uses the key as txnId, replacing the random id per attempt.
  • opencode-runner resumes the session of a retried delivery instead of starting a new one.
  • log-decisions is migrated.
• Migration 0002_pipeline adds the jobs and deliveries tables. migrate copy includes them.
• New telemetry: brainstem.core.failed. brainstem.destination.failed now carries attempt, dead and retryInMs. New metrics: decisions.failed, decisions.dead and destinations.dead.
• Behaviour changes:
  • runtime.destinations.blockInputOnFailure is ignored, with a warning.
  • A failing decision no longer holds back the input checkpoint. It stays queued and is retried.
  • --once runs every due job after polling, and exits non-zero if any poll or job failed.
  • Config callbacks of the built-in destinations (notificationKey, titleForEvent, promptTemplate) receive output.payload.
  • Give destinations an id: it is part of the idempotency key.

Docker image: ghcr.io/jurrebuunk/brainstem:v1.3.0. Upgrade notes: docs/delivery-pipeline.md#upgrading-from-1x.