Skip to content

v3.17.0 — QA chain (seconder fusion + test-engineer/test-runner)

Choose a tag to compare

@Jaggerxtrm Jaggerxtrm released this 31 May 18:27
· 1130 commits to master since this release

[v3.17.0] — 2026-05-31

Added

  • seconder specialist (NEW, package tier). The fused post-writer gate from canon docs/design/chain-templates.md §2.3 — collapses the old split between scope/compliance (reviewer phase-1) and code-quality smell (code-sanity) into one READ_ONLY dispatch (openai-codex/gpt-5.4-mini) emitting a structured dual-verdict JSON: scope_verdict + scope_findings + quality_verdict + quality_findings + overall_verdict. The chain reducer reads overall_verdict to advance or route back to the writer; the reviewer reads the dimension-tagged findings. Replaces code-sanity, which is removed this release (see Removed) (unitAI-4e194, unitAI-wz2ag, unitAI-321ir).
  • test-engineer specialist (NEW, package tier). Post-implementation behavioral-test author from the actual diff (canon §2.5). MEDIUM, openai-codex/gpt-5.5, requires_worktree. Produces tests + fixtures + smoke/E2E harnesses + telemetry assertions and emits exact test-runner commands via a structured schema (status, files_changed, coverage_map, smoke_e2e_commands, telemetry_assertions, test_runner_commands, known_deferred_paths, source_bug_suspicions). Ambidextrous role (§3.16): the same spec is the primary writer in test-only chains and the secondary writer in code-with-tests chains — the system prompt is mode-agnostic and the position arrives via the dispatch-time mandate. Forbidden from patching production source by default (unitAI-sfwe1, unitAI-sfwe1.1).
  • Two NEW chain-template formulas — code-with-tests (§3.14) and test-only (§3.15). code-with-tests: dual-writer production chain (executor writes the diff, test-engineer writes tests against it) at high/critical scrutiny. test-only: single-writer chain when scope is test-paths only (test-engineer as primary writer). Both carry the ambidextrous test-engineer mandate in the step description (pre-substrate position-injection mechanism, §3.16) (unitAI-f9kku).
  • Seconder dual-verdict eval + QA-routing eval. .specialists/evals/seconder/ — a reproducible static eval with three fixtures (wrong-scope → scope_verdict FAIL, bad-quality → quality_verdict FAIL, clean → overall_verdict PASS) each carrying an expected-verdict.json, plus an operator-run run.sh and a token-cost note (unitAI-o7j1a). config/skills/using-specialists-v3/evals/ gained four QA-routing eval cases (test-engineer primary vs secondary writer, test-runner owner-routing, reviewer-consumes-QA-evidence) + a passing vitest harness (unitAI-sfwe1.5).
  • transcriber specialist (NEW, package tier). Promotes the documentation-grade YouTube transcriber prompt to the shipped package catalog at config/specialists/transcriber.specialist.json. v1.6.0 uses openai-codex/gpt-5.3-codex, title-derived transcript/analysis filenames, narrow subtitle language extraction to avoid YouTube 429 fanout, immediate section-by-section writes, dense technical DETAILED SECTION ANALYSIS, TECHNICAL EXTRACTION TABLES, and a coverage/quality audit to prevent shallow “2-line per 5 minutes” outputs (unitAI-jfw26).
  • sp log runtime/provenance stream — new operator-facing log command for specialist runtime debugging. It reads observability.db, shows dispatch/control/status/error/auto-commit provenance separate from sp feed, supports --json NDJSON for full payloads, --follow, --since, --limit, job/bead/specialist/node filters, and --all-events for raw feed-like internals (unitAI-gqpvw, unitAI-vfqgq).
  • sp log parent-directory/global mode — when run outside a repo root with no local specialists DB, sp log discovers immediate child repos containing .specialists/db/observability.db and aggregates their runtime rows as one global log; --repo <name> narrows output to a single child repo (unitAI-v5xfu).
  • obligations-scanner specialist (NEW) — READ_ONLY, cheap (openai-codex/gpt-5.4-mini, bare: true, ~30s target) pre-review marker scan. Scans executor/debugger diffs for newly-introduced TODO/FIXME/HACK/XXX/TEMP/WIP/NOTE(release) markers in production code. Distinguishes production vs test/fixture surfaces. Recognizes structured // TODO(<bead-id>): reason format and treats it as TRACKED when the linked bead is open. Verdict: CLEAN | OBLIGATIONS_FOUND | BLOCKED with a JSON output_schema the reviewer consumes directly. Iron-style obligations tracking (unitAI-kglvm.3).
  • docs/design/iron-review-hardening.html — design doc visualizing the new pipeline (SCRUTINY taxonomy, old-vs-new chain flow, per-specialist changes, git-state precondition, manual execution plan). Mirrored to ~/second-mind/1-projects/Mercury/ for sync (unitAI-fpwbr, unitAI-1n56e, unitAI-ejdi1).
  • service-skills-sync specialist (NEW, package tier). Promotes the Service Skills Librarian (previously a market-data user-tier override) to a shipped package specialist at config/specialists/service-skills-sync.specialist.json. MEDIUM, openai-codex/gpt-5.4-mini; keeps per-service expert-persona SKILL.md docs in sync with code drift using gitnexus (detect_changes/impact/context) + Serena, gated by a drift_detector.py pre-scan. The per-service knowledge layer the future devops agent reads (DevOps PRD §7.1) (unitAI-g8zr3).
  • researcher specialist v1.2.0 → v1.3.0 — general-web pipeline (Mode 4). Adds a fourth research mode closing the web-research gap (previously the researcher reached library docs/repos/code/social but had no general web search or arbitrary-URL read): ddgs (DuckDuckGo search CLI, no API key — uv tool install ddgs) discovers authoritative URLs, then agent-browser (native Rust CLI + Chrome daemon — npm i -g agent-browser) reads any URL including JS-rendered pages. Documented in prompt.system Mode 4 + config/mandatory-rules/research-tool-routing.md. capabilities.external_commands deliberately left empty — it is a hard pre-run gate (runner.ts validateBeforeRun throws on a missing PATH binary), so declaring these heavy tools would break the shipped researcher in projects without them; documented as available-on-demand with install hints instead (unitAI-qgvld).
  • notes_mode specialist field + markdown-native 3-state handoff. New top-level notes_mode enum (full-trail default | final-only) controls how each turn's handoff is persisted to BOTH the input bead notes and output_file. The supervisor renders a markdown-native 3-state handoff — ### <specialist> · <model> · [turn N · WAITING] trail blocks plus a canonical ## <specialist> · <model> · [FINAL · DONE] block — with the specialist's output verbatim, a single italic metadata footer (empty/zero/unknown fields omitted), and a provider-prefix-stripped model string; no divider rules or emoji. One shared content source feeds bead notes, output_file, and sp result. final-only persists only the canonical FINAL block and overwrites output_file, for non-coding/chained pipelines where the next specialist reads the previous one's note or file as input (unitAI-10y07, unitAI-yiazs).

Changed

  • test-runner specialist — upgraded to the QA failure-routing contract (canon §2.5). Now prefers exact commands from test-engineer/orchestrator and falls back to manifest-detected runners only when none are supplied (clearly labeled as fallback). Classifies every failure by owner — test_engineer (test/fixture/harness wrong, or new untested feature), debugger_or_executor (missing telemetry / source behavior regression), infrastructure, pre_existing — and never writes tests or patches source (LOW). Backed by config/mandatory-rules/test-runner-execution-scope.md (unitAI-sfwe1.2).
  • reviewer specialist — refactored to phase-2-only (seconder fusion, canon §2.3). The phase-1 compliance/scope check now lives in seconder's scope_verdict; the reviewer keeps only phase-2 (adversarial deep code-quality audit + machine-readable Release Checklist + ddiff re-review on PARTIAL) and treats a seconder PASS as the upstream scope gate. Two-phase framing removed from the prompt (unitAI-4e194, unitAI-sowpa).
  • 13 chain-template formulas rewired for the canonical QA pipeline. Every production-diff template (code-standard, code-with-advisors, debug, security-deep, restitch) now wires writer → seconder → test-engineer → test-runner → [security-auditor if sensitive] → obligations-scanner → reviewer (canon §2.1) — code-sanity renamed to seconder, test-engineer + test-runner inserted. README overlay table + roadmap Opp 14/15 status updated (unitAI-f9kku).
  • using-specialists-v3 skill — canonical seconder-fusion pipeline. SKILL.md now teaches writer → seconder → test-engineer → test-runner → [security] → obligations → reviewer, the QA failure-routing matrix (§2.5), and SCRUTINY reframed as a chain property that modulates structure, not quality (§2.2) — including the none tier for read-only chains and the required-at-creation rule. seconder replaces the code-sanity seconder slot; reviewer documented as phase-2-only (unitAI-096re, folds unitAI-sfwe1.3).
  • sp log human output is leaner and calmer. Default output now hides agent-internal turn/tool/text/thinking/token rows already covered by sp feed, keeps runtime-owned rows only, collapses repo/path/branch/worktree metadata into one compact worktree=<repo>/<worktree> field, uses a restrained professional color palette (dim metadata, plain job ids, bold specialist names, color-coded status=<state>, green/yellow/red/cyan only for semantic state), and collapses adjacent duplicate display rows caused by duplicated runtime events while preserving full payloads in --json (unitAI-vfqgq, unitAI-npjlq, unitAI-f5k0p).
  • reviewer specialist — Iron-inspired prompt overhaul. Five new system-prompt sections, additions only (existing source-of-truth priority and AUTHORITATIVE REVIEW CONTEXT preserved verbatim) (unitAI-kglvm.1):
    • SCRUTINY tier behavior (low | medium | high | critical) — reads field from bead contract; defaults to medium; tiers reviewer depth from seconder-only spot-check (low) through file-by-file sign-off with mandatory gitnexus_impact (high) to required second-opinion (critical).
    • Scrutiny auto-escalation — surface-pattern floor table raises level regardless of bead's stated SCRUTINY when diff touches auth/*, **/credentials*, **/token* (→ high), config/specialists/*.json (→ high), src/specialist/{runner,schema}.ts (→ high), **/*.lock (→ medium + security-auditor required), migrations/** (→ high), src/permissions/* / hooks/** (→ critical). Author's level is a floor, not a ceiling.
    • Re-review after PARTIAL (Ddiff mode) — when re-reviewing a fixed PARTIAL, scope to delta since prior verdict, carry forward prior approvals, audit only newly-touched files/symbols.
    • Obligations scan — consumes obligations-scanner JSON output if present, else scans diff inline; production markers → PARTIAL unless accepted via bead NON_GOALS or structured // TODO(<bead-id>): reference; test/fixture markers noted but not blocking.
    • Release Checklist (REQUIRED) — machine-readable block appended to every verdict for future sp merge enforcement.
  • executor and debugger specialists — Obligations discipline. New system-prompt section instructs both codegen specialists to avoid introducing in-code obligation markers in production paths by default; if work is genuinely deferred, file a follow-up bead via bd create --deps discovered-from:<current>; if a marker is truly needed at a code site, use structured form // TODO(<follow-up-bead-id>): <reason> where the linked bead is open and listed in current bead's NON_GOALS. Prevents PARTIAL fix-loops from the new obligations-scanner gate. Test/fixture paths exempt (unitAI-kglvm.4).
  • using-specialists-v3 skill: v3.4 → v3.5 (Iron-style orchestration). Substantial restructure aligned with the above specialist changes (unitAI-kglvm.5):
    • "Advisory Passes" section reframed as three mandatory gates: Seconder Gate (code-sanity), Security Gate (security-auditor on sensitive surfaces), Obligations Gate (obligations-scanner). Skip rules tightened.
    • NEW SCRUTINY taxonomy section: tier behavior + auto-escalation surface table. SCRUTINY field added to task/epic, executor, reviewer bead contract templates.
    • NEW Git State Precondition section: four-check pre-flight (working tree clean, HEAD contains prior chain commits, no orphaned worktrees, in-sync integration branch) required before dispatching any chain that depends on prior chain output. Strictness-by-scenario table.
    • Rule #9 INVERTED: manual git workflow is now canonical; sp merge and sp epic merge are PROHIBITED (known broken, awaiting separate rework epic). Cherry-Pick Playbook promoted to canonical multi-chain merge path. sp finalize removed from documented orchestrator workflow.
    • Rule #13 exception clause added for epics that restructure the specialists themselves (operator-authorized manual-orchestrator-direct work).
    • Rule #14 NEW: Git State Precondition reference.
    • obligations-scanner row added to Choosing The Specialist table. parallel-review marked deprecated.
    • Escalation Matrix and Failure Recovery tables rewritten: sp-merge rows replaced with git-workflow recovery patterns (stale .git/index.lock, info/exclude vs tracked beads file, FF-via-git update-ref when checkout blocked).
  • CLAUDE.md "Common gotchas" section rewritten to match the new canonical: manual merge, explicit sp stop for keep-alive cleanup, Iron-style gates mandatory, Git State Precondition, bd auto-export churn handling, package-tier specialist edits via direct JSON.
  • bd auto-export pain fix. bd config set export.git-add false disables per-write auto-staging of .beads/issues.jsonl (silent mid-work; no checkout aborts; no .git/index.lock races). Paired with a custom block added to .git/hooks/pre-commit AFTER bd's managed markers — runs git add -f .beads/issues.jsonl so commits naturally include the fresh JSONL snapshot via the existing pre-commit hook chain. Eliminates the runaway chore(beads): export state commits that plagued every multi-bd-op session. Verified end-to-end in this repo (commits 63ac83f6, 4c1f19a5, 1e014f33) (unitAI-mg18o).
  • output_file decoupled from SPECIALISTS_JOB_FILE_OUTPUT. A specialist that sets output_file now always writes its full result — foreground and --background (tmux) — independent of the env flag, which now only gates the debug file-mirrors (events.jsonl / status.json / result.txt). Previously --background (tmux) runs silently dropped output_file because the env var did not propagate into the tmux session. The single-writer invariant is preserved (the supervisor owns the file in supervised runs; suppressRunnerFileOutput still skips the runner write), and .specialists/*-result.md was added to .gitignore since specs with output_file now always write. output_file, notes_mode, and the handoff envelope are documented in docs/authoring.md and the specialists-creator skill (unitAI-f58ma, unitAI-g8rqg).

Removed

  • code-sanity specialist removed — superseded by seconder. The Iron seconder gate (briefly promoted as code-sanity mid-cycle, unitAI-kglvm.2) is replaced by the new seconder specialist, which fuses its code-quality smell pass with the reviewer's old phase-1 scope check into one dual-verdict gate (canon §2.3). config/specialists/code-sanity.specialist.json deleted; all operational references across the v3/auto/v2 skills, reviewer.specialist.json, chain-template formula prose, and docs/specialists-catalog.md renamed to seconder. Two historical lineage notes preserved (seconder.specialist.json's absorbed-mandate section + the v3 SKILL §2.3 fusion explanation) (unitAI-321ir, unitAI-4e194).
  • sp merge / sp epic merge / sp finalize removed from documented orchestrator workflow in using-specialists-v3. Commands still exist in the sp binary (no source-code removal) but the skill explicitly prohibits their use pending a separate rework epic. Operators reaching for them should use the documented manual git workflow instead.

Changed (prior)

  • All 17 package-shipped specialists in config/specialists/ now declare the v3.16.0 schema additions explicitly: execution.bare: false and prompt.system_prompt_mode: "append". Values match the previous absent-field defaults — pure-mechanical, zero behavior change — but every shipped spec is now self-documenting at the schema level instead of relying on per-runner legacy fallbacks. bare.specialist.json retains its explicit bare: true + replace (unitAI-51r2w).

Fixed

  • sp stop / sp resume control-plane actions now treat observability writes as best-effort: status/control telemetry failures no longer prevent SIGTERM delivery or falsely report a delivered resume as a steer-pipe write failure (unitAI-dkhi3).
  • OSV scan now resolves GHSA-q8mj-m7cp-5q26 by overriding all qs lockfile entries to 6.15.2; bun.lock no longer contains vulnerable qs@6.15.1 entries (unitAI-dkhi3).
  • Supervisor status reads now reconcile dead starting/running specialist jobs to terminal error with a run_complete(ERROR) event, so reviewer crashes during heavy bash validation no longer leave sp ps/sp result stuck on stale running rows (unitAI-6x6p6, unitAI-uzyut).
  • Per-turn handoff notes now append instead of replace. appendBeadNote called bd update --notes (whole-field replace), so each per-turn specialist handoff clobbered the previous one — multi-turn jobs left only the last (often empty) note on the bead, recoverable only from observability.db. Switched to bd update --append-notes and exported formatBeadNotes. (The appended handoff format was subsequently finalized to the markdown-native 3-state form — see the notes_mode entry under Added.) Tests in bead-notes.test.ts + new supervisor-bead-notes.test.ts (sibling, since supervisor.test.ts is excluded from the default run) (unitAI-sx5qk).
  • [FINAL · DONE] handoff block now emitted on sp stop for keep-alive jobs. The canonical FINAL block was silently skipped on the dominant keep-alive→sp stop path because src/specialist/control.ts constructed the Supervisor without a beadsClient, so finalizeWaitingJob's bead_id && beadsClient guard never fired; stopJob also never invoked finalizeWaitingJob for waiting jobs. Both fixed, and keep-alive turn summaries skip the duplicate non-final done write, so a keep-alive run yields one [turn N · WAITING] per turn plus one [FINAL · DONE] at stop (unitAI-mis38).