Skip to content

v0.1.0 — tracing, replay, evals, and governance for agent runs

Choose a tag to compare

@haeliotang haeliotang released this 04 Jul 17:37

First complete reference implementation.

What's in it

Runtime substrate

  • Deterministic scripted agent runtime; policy gating (allow / deny / needs_review) evaluated before every step
  • Digest-addressed JSONL traces; replay engine with per-field divergence detection (catches code drift, environment drift, and tampering)
  • Golden-task eval harness gating CI on every commit

Governance objects (field-aligned with wutai & stillmirror-review — see docs/object-model-alignment.md)

  • PolicyDecision, RiskSignal, ReviewerSeat as first-class data
  • Attestations: scoped ratification by a named human (declared_scope / excluded_scope, proposed_byattested_by), consuming recorded review debt
  • Run-level trust verdict (trusted / review_required / blocked), per-run coverage declarations

Observability

  • OTel spans with digests and policy verdicts as attributes; Prometheus metrics incl. aro_review_debt_total and aro_attestations_total
  • Pre-provisioned Grafana dashboard (screenshot in README, captured live)
  • SLOs incl. two governance SLOs (docs/slo.md), full error taxonomy (docs/error-taxonomy.md)

Infra

  • Postgres store with FOR UPDATE SKIP LOCKED queue claims (SQLite fallback); worker retry with exponential backoff, dead-lettering, deterministic chaos injection; API rate limiting
  • docker-compose stack (api, worker, Postgres, Prometheus, Grafana) with healthchecks — verified end to end; optional k8s manifests
  • CI: lint, unit, integration, Postgres service-container job, golden replay regression, web build

Surfaces: FastAPI (runs / traces / replay / attestations / queue / metrics), queue worker, React + Vite dashboard.

62 files of Python across five workspace packages, 49 tests, benchmark numbers measured not asserted (docs/benchmark.md).