Repository navigation
v0.8.0 — eval/ops, self-review critic, dynamic handoffs
Three features across the harness/loop/graph layers, built to keep data / state / flow / logic cleanly separated. The engine gained exactly one additive, emit-only line.
Highlights
critic node — self-review that can't rubber-stamp
An independent judge scored against explicit criteria in a fresh context — it only ever sees the artifact + criteria, never the producing agent's conversation, so a model can't quietly grade its own work. LLM or deterministic handler: backend; the verdict routes via edges (loop to revise on fail, forward on pass), bounded by the producer's max_iterations.
Dynamic handoffs on agent
A handoffs: list adds one synthetic handoff_to_<target> tool per target; calling one transfers control (goto) and the full conversation (a state channel the target reads at <state.handoff.*>). Bounded by the existing step/iteration guards.
eval/ops — a read-only observer
A new graphx.eval package over the existing events + checkpoints tables — never touches the execution path, state, or routing.
- Per-run + per-node reports: cost, latency, retries, fallbacks, dead-letters, interventions.
- Dataset-driven eval: deterministic assertions + optional LLM judge (reuses the critic).
- Version compare: same dataset vs two graphs → outcome / metric / trace diff.
- Golden-trace normalization so behavioral regressions diff cleanly.
- CLI:
runs,run-show,eval,eval-compare. TUI run browser onb.
Verification
283 tests (20 new), ruff-clean, Python 3.12+. Live-verified against a local vLLM: critic review loop, handoff, eval run, and the TUI run browser.
Examples
examples/review_loop.yaml, examples/handoff.yaml, examples/hello.eval.yaml.
Full changelog: v0.7.1...v0.8.0