v3.18.0

[v3.18.0] — 2026-06-24

Added

• sp console v2 — full TUI rewrite (epic unitAI-ctb4u, PR #125). Replaces the original prototype console with a complete multi-view terminal UI built on @earendil-works/pi-tui. Views: ps (live process list with tree depth, context %, status glyphs, column-adaptive layout), feed (live/forensic event stream with follow mode), job (inspect fields + action list), result (full job output), bead (linked bead doc + live state), diff (worktree + commit SHA fallback), config (global user.json override editor), repoConfig (repo registry management). 24-bit ANSI palette sourced from mock-v2 design doc. Keybindings: ↑↓/j/k navigate, ↵ open feed, r result, i inspect, b bead, d diff, g config, R repos, h history, a all, / filter, x stop, tab/1-9 switch repo, 0 ALL view, q quit.
• ALL view — aggregated cross-repo dashboard with cursor navigation. Opening sp console now lands on an ALL view (0 to return from any tab) showing active jobs across all configured repos sorted by activity, with per-repo section headers. ↑↓/j/k moves a cursor through individual job rows; ↵ opens the selected job's feed, r/i/b/d/g/R/x work as in ps — automatically switching to the job's repo before opening the view. The stats bar always shows full ps-style metrics (health / rss / cpu / orphans) for the current repo on all tabs including ALL.
• x — stop job keybinding in ps and all views. Pressing x on any selected or cursor job sends SIGTERM via stopJob in control.ts, using the job's own repo jobsDir. A brief confirmation message (sent SIGTERM to <job-id>) appears in the status bar. Works in both ps (selected row) and all (cursor row, cross-repo).
• Multi-repo auto-discovery and console.json persistence (PRs #141, #143). sp console auto-discovers sibling repos up to depth 2 (worktree-safe: skips .git/ trees and the specialists package itself) on first launch and persists the list to ~/.config/specialists/console.json. Subsequent launches reload the saved list for instant startup. --add-repo <path> and --remove-repo <name> CLI flags manage the list without opening the TUI.
• RepoConfigView — interactive repo registry management inside the console (PR #142). R from ps opens an in-TUI editor for console.json: + add a repo (path → name two-step), d remove, e edit path, n edit name, r rescan (depth-2 inline), s toggle inactive rows. All mutations share the same RuntimeClient persistence surface as the CLI flags.
• DiffView SHA fallback for dead-worktree jobs (unitAI-ctb4u.29, PR #135). When a job's worktree has been removed, DiffView falls back to the recorded git_commit_sha from supervisor_status. The section title changes to diff summary · @<sha7> (commit) so operators know they are viewing a historical snapshot rather than a live worktree diff.
• sp ps gets TUI-themed rows and stats line (unitAI-ugw4s, PR #144). The sp ps shell command now renders job rows using the same 24-bit ANSI palette, status glyphs, and column-adaptive layout as sp console. The stats line (jobs N/M · running N waiting N · …) is appended below the table in dim text. Non-TTY output falls back to plain text as before.
• Snapshot-diff engine ported from gitboard (unitAI-ctb4u.19, PR #137). src/specialist/snapshot-diff.ts provides stable SHA-256 hashing of job snapshots (snapshotHash) and upsert/tombstone delta computation (snapshotDiff). The console poll loop uses the hash to skip no-op dispatches and the delta to expire paint-cache entries for tombstoned jobs only, avoiding full-cache invalidation on every tick.
• Interactive waiting_auto_close_ms global override (unitAI-pj2mm, PR #152). waiting_auto_close_ms is now settable via the global ~/.config/specialists/user.json overlay without editing each package specialist JSON. sp console's ConfigView (g) displays the effective default next to inherit (e.g. inherit (120000)) when no override is active. All 24 package specialist JSONs had their stale null placeholder removed (null triggered a schema validation error at dispatch; field is number | undefined).
• quant-methodologist and quant-researcher specialists (NEW, package tier). Two new quant-focused READ_ONLY specialists ship at config/specialists/ (v1.0.0). quant-methodologist audits and designs quantitative analytics methodology/pipelines for market-data systems — formulas, units, conventions, numerical probes, model assumptions, and creative-but-conservative pipeline proposals (thinking_level: high, LOW read/probe, no implementation edits). quant-researcher gathers source-backed evidence for mathematical finance, market microstructure, econometrics, exchange/product conventions, academic papers, package semantics, DeepWiki repo scans, and GitHub implementation snippets (LOW research, no local code edits). Both ship with model = null / fallback_model = null resolved via the KAN-90 global config layer (commit 78e0b53b).
• KAN-91 expanded global overrides. Phase 0 adds the nested execution and prompt override machinery that keeps loader allowlists and user.json schema in sync.
• KAN-91 user-environment overrides. Phase 1 allowlists prompt.system_prompt_mode, per-specialist Serena/GitNexus extension opt-outs, notes_mode, output_file, and prompt/stdout byte limits in the global user.json layer.
• KAN-91 fallback model chains. Phase 2 adds execution.fallback_models arrays, keeps legacy fallback_model, and walks fallback chains only on transient provider failures.
• KAN-91 preset references. Phase 3 adds @preset/<name> model and fallback references with package preset lookup, depth/cycle guards, telemetry, and type validation.
• KAN-91 global override upgrade notes. sp init --global now writes a _doc sentinel in strict JSON and points users to docs/upgrade-notes/kan-91-expanded-overrides.md for field semantics.
• Console-safe AgentOps operations surfaces. sp serve now exposes read-only per-job normalized forensic streams at GET /jobs/:job_id/feed-events and /api/specialists/jobs/:job_id/feed-events; Prometheus projection adds bounded xtrm_chains_total / xtrm_chain_duration_seconds, projects gate verdict and evidence-ref counters, forbids raw diff labels, keeps chain/job/file/diff drill-down in forensic events, and documents the console-facing telemetry contract updates (unitAI-5ljfu).
• AgentOps correlation and MCP telemetry pre-wiring. Forensic events now carry optional session_id, conversation_id, trace_id, span_id, and parent_span_id correlation fields through Pi session metadata, supervisor status, SQLite forensic writes, and sp feed/sp log JSON surfaces. MCP timeline events normalize to canonical mcp.* names, preserve MCP/JSON-RPC/trace identifiers as correlation/body data only, and project bounded xtrm_mcp_operations_total metrics without implying a live MCP emitter (unitAI-eoqxp.3.1, unitAI-eoqxp.3.2).
• Telemetry contract validation and token-first cost semantics. Forbidden Prometheus labels now include session/conversation/MCP/JSON-RPC/eval/policy/identity correlation IDs, AgentOps catalog fixtures cover job/MCP/identity/policy/eval/service-skills/pulse/token provenance examples, and the telemetry docs explicitly keep USD cost out of runtime metrics until billing/pricing provenance exists (unitAI-eoqxp.3.5, unitAI-v1fzu).
• AgentOps forensic telemetry bridge — xtrm.forensic.v1 runtime envelopes, persistence, and query surface. Specialist runtime events now dual-write canonical forensic envelopes into specialist_forensic_events, with deterministic redaction before persistence/output and additive forensic_event payloads on sp feed --json / sp log --json while preserving legacy fields. New sp forensic <job-id> --json emits persisted NDJSON evidence for a job; real-job smoke ddc421 validated 68 events across job, model, turn, tool, and git families (unitAI-60w93.2, .3, .4, .8, .9, .10, .14, unitAI-z2s17).
• AgentOps Prometheus projection — sp metrics --prometheus plus sp serve GET /metrics. Specialists now exports low-cardinality Prometheus/OpenMetrics text for job state, job totals, duration/wait histograms, turns, context usage, tool calls, LLM tokens, queue depth, process/worktree gauges, and projection timestamp. The projection is table-derived/replay-safe for current state, validates text syntax in CI, and keeps high-cardinality IDs (job_id, chain_id, participant_id) out of labels (unitAI-60w93.5, .6, .7, .11, .12, .13).
• AgentOps telemetry readiness smoke for gitboard handoff (unitAI-ub65d.4, PR #114). Closes the unitAI-ub65d epic by validating the shipped telemetry meets the gitboard materializer handoff contract: forensic stream carries normalized xtrm.forensic.v1 rows with body + redaction, job metrics carry turns/tools/model + token split + usage_source, evidence includes verdict/result and diff/commit refs where present, and sp metrics --prometheus stays bounded.
• sp setup CLI verb + benchmark data module (Epic unitAI-t86wh, PRs #119, #120). New top-level sp setup verb composes sp init --global / sp edit --global / sp doctor --specialists into a benchmark-driven model-assignment workflow: --discovery [--json], --fetch-benchmarks [--json], --plan <budget-preset>, --apply, and --probe-only. Backed by src/specialist/benchmarks.ts (PR #119) — a pinned SSOT snapshot from artificialanalysis.ai (primary) + lmarena.ai (secondary) plus an agentic-followthrough probe suite that catches model failure modes public leaderboards miss.
• sp list / specialists list — per-specialist version badge, package-version header, and new-release alert. Each row now renders the spec's own metadata.version as a [vX.Y.Z] badge after the name (e.g. seconder [v1.0.0]); the header line shows the running package version (Specialists (N) specialists v<pkg>); and when the existing version-check finds a newer published tag, sp list prints new version <v> available, run npm i -g @jaggerxtrm/specialists@<v>. Rows are sorted alphabetically by name (human output and --json, ordering-only — --json stays a top-level array, backward-compatible). Reuses getVersionCheckResult() (TTY-gated, 6h cache, 2s timeout, silent on failure) via a new formatListVersionAlert() — no new network/cache path (unitAI-k5vx7).

Changed

• Per-repo SourceQueue for poll isolation (unitAI-ctb4u.20, PR #138). Each repo now gets its own SourceQueue (1 500ms coalesce) instead of a shared setInterval. Tab-switching cancels the prior repo's pending dispatch immediately; no stale poll from the previous repo can race the destination repo's first render.
• Per-row paint cache in ProcessView (unitAI-ctb4u.21, PR #139). Rendered job-row strings are cached by composite key jobId|status|ctxBucket|width|depth|selected|datePrefix. A poll tick with unchanged rows skips all renderJobRow calls. Cache is bounded to totalJobs × 2 and is cleared on repo switch and tombstone delivery, preventing cross-repo bleed and unbounded growth in long sessions.
• Dependency-bump CI policy adoption (unitAI-c7zdy, PR #154). OSV PR scans are now advisory by default but fail on SECURITY_FORCED findings, while push/schedule scans remain hard gates. package-payload separates install/build/pack/assert steps so harness failures no longer masquerade as payload-contract failures. Added a dry-run-first dependency verdict materializer for advisor/followup/gate substrate artifacts.
• Specialist model selection moves to a per-user global config (KAN-90 / unitAI-1gtou). Every config/specialists/*.specialist.json now ships with execution.model = null and execution.fallback_model = null. The SpecialistLoader resolves each specialist via a 3-layer field-merge (package canonical → ~/.config/specialists/user.json → .specialists/user); the legacy .specialists/default/ mirror was retired (commit 31a6421c) and stale entries are pruned by drift-detector / sp prune-stale-defaults. New CLI surface: sp init --global, sp edit --global, sp doctor --specialists. Missing models after merge raise SpecialistMissingModelError at dispatch with a pointer to sp edit --global. Blocked fields (execution.permission_required, mandatory_rules, capabilities, output_schema, auto_commit, prompt.system, skills.scripts) are stripped from the global layer and surfaced via sp doctor --specialists. Upgrade guide: docs/upgrade-notes/kan-90-global-user-config.md. (Commits 5f8d725e, 6604c144, 6b69a6fe, plus this commit.)
• setup-specialists skill v1.0 → v2.0 + specialists-creator skill v1.3 → v1.4 — KAN-90/91 global-config workflow (unitAI-vme0p). setup-specialists is fully rewritten (162 → 261 LOC) to teach the 3-layer field merge, the sp init --global / sp edit --global / sp doctor --specialists workflow, the OVERRIDE_ALLOWED_* allowlist (model + fallback chains + thinking_level + byte limits + extension opt-out + notes_mode + output_file + system_prompt_mode), @preset/<cheap|medium|power> references, and operator pitfalls (positional-form vim fallthrough, thinking_level: off breaking thinking-class models, repo-override shadowing global). The previous v1.0 literally claimed "There is no supported global override layer yet" — false since KAN-90 shipped on 2026-06-13. specialists-creator gains an additive ## Global User Override Layer (KAN-90/91) section (+118 LOC) that mirrors the per-spec field reference to the global-layer dot-path syntax, so authors of a new specialist can document the same fields once and have them work in both layers. Operators landing in a fresh @jaggerxtrm/specialists install via /setup-specialists or /specialists-creator now read the current truth instead of the stale claim.
• setup-specialists skill v3.0 interactive playbook — phase contracts + interactive checkpoints (unitAI-hdrr6, PR #121). Rewrites setup-specialists into a directed 5-phase flow: sp setup --discovery, sp setup --fetch-benchmarks --json, five explicit AskUserQuestion checkpoints (budget/provider/auth/privacy/probe shape), deterministic sp setup --plan output/table proposal review, and sp setup --apply verification with optional sp setup --probe-only flow. Adds strict parse contracts for pi --list-models, sp doctor --specialists, sp list --full, and benchmark JSON so multiple orchestrators can produce identical state and proposals.
• sp script trusted-mode hardening + script-specialist observability (unitAI-4zask/unitAI-mally/unitAI-q5t29, PR #145). Finishes the service-skills-sync script-specialist refit: strict JSON machine-output handling with prose + fenced-JSON recovery for the final wrapper path, live script-specialist observability rows/events surfaced in sp ps / sp feed, and hardened trusted script mode so dangerous local-script / write-capable execution is explicit rather than implicit.
• service-skills-sync specialist v1.4.0 — mandatory Phase 2.5 diff content scan for non-symbol drift. Adds a git diff <last_sync_ref>..HEAD scan step between Serena cross-check and classify, grepping for renamed env vars, new/removed exception sites, error log strings, docker container/image renames, and new API endpoints (6 patterns). The audited-and-unchanged verdict now requires a clean diff scan in addition to a gitnexus cosmetic signal. Missing last_sync_ref emits a new ⚠ Triage-incomplete verdict rather than silently passing. New service-skills-diff-scan-mandatory mandatory rule enforces the contract. Motivated by py_backend-0ondo (darth-feedor) where string-literal-only drift was mislabeled as semantic-unchanged (unitAI-ekt17).
• test-runner specialist — task_template aligns with exact-command-wins scope semantics. Pinned exact commands from the orchestrator/test-engineer now explicitly win over manifest-detected fallback (fallback is clearly labelled as evidence, never a scope override) (v2.0.0 → v2.0.1, PR #102).

Fixed

• pi-coding-agent ownership/package references corrected (unitAI-fgpxv). README, bootstrap/service docs, and sp init optional prerequisite hints now point to earendil-works (https://github.com/earendil-works/pi-coding-agent, @earendil-works/pi-coding-agent) instead of stale xtrm/Jaggerxtrm or mariozechner references.
• Defensive render against malformed status_json rows (unitAI-ctb4u.27, PR #126). Per-row try/catch in renderProcessRows prevents a single corrupt status_json entry from crashing the entire ProcessView paint loop; malformed rows render as ?? <malformed row dropped> without disrupting adjacent rows.
• Generic ↑↓ keys gated to ps/scroll views only (unitAI-ctb4u.30, PR #127). The generic move reducer action was consuming arrow keys even in config and diff views, which have their own cursor handlers. The gate prevents it from swallowing keys before view-specific handlers can win.
• ConfigView hint text enriched with operational guidance (unitAI-ctb4u.31, PR #128). Each editable field's allowedHint now includes concrete examples and constraint descriptions so operators understand valid values without leaving the TUI.
• Scroll and selectedRow reset on back-from-detail (unitAI-kz1ud, PR #129). Navigating back from feed/result/inspect/bead/diff to ps previously retained the scroll position from the detail view, leaving the cursor off-screen. The back action now resets both scroll and selectedRow to 0.
• Forensic feed TYPE_W column widened to 32 (unitAI-3wm6x, PR #130). The type column in feed-event rows was truncated to 16 characters, cutting off longer forensic event type names such as xtrm.forensic.v1.*.
• stderr writes consolidated through log.ts single-sink (unitAI-21sn4, PR #131). Four separate process.stderr.write call-sites in runtime.ts bypassed the structured logError sink, preventing log-level filtering and test interception.
• sp console --help updated to v2 keybindings with parity guard (unitAI-ctb4u.23, PR #132). The help text now matches the live keymap; a compile-time parity check ensures the help string and handleInput handlers cannot silently diverge.
• Global-config writes use atomic tmp + rename (unitAI-ctb4u.17, PR #133). Direct overwrite of user.json was susceptible to torn reads on interrupted writes. Writes now go to user.json.tmp followed by fs.renameSync — atomic on POSIX.
• ps default historyMode aligned with sp ps shell default (unitAI-ctb4u.26, PR #134). sp console opened in history mode (all jobs) while sp ps defaulted to default (running + recent only). Both now default to default; h in the console cycles through modes as before.
• supervisor.dispose() now reaps the pi/Serena session (unitAI-pjst5, PR #150). dispose() previously closed sqlite/tmux/FIFO only and never explicitly stopped the Serena extension subprocess attached via script-runner.ts:977-979. After worktree removal, Serena LSPs survived indefinitely; on the Mercury host this accumulated to 30+ stale processes (some 5+ days old) and was the structural driver of the 2026-06-22 host memory exhaustion incident (96.2% RAM, 23/23 GiB swap full). Validated end-to-end with operator-side defense-in-depth: mercury-infra ships a stopgap cron killer (infra-4m7b) reaping serena start-mcp-server etime > 6h plus a mercury_specialists_orphan_count Prometheus alert (infra-3tvo). After this fix is live the killer log should stay at 0 reaps; tracked by mercury-infra bead infra-gty4 (7-day silent window → remove stopgap).
• observability.db no longer pre-creates a zero-byte placeholder (unitAI-nuh7l, PR #151). Previous behavior: when sqlite_open failed during runtime startup, the path was left as a 0-byte file with the open error silently swallowed; subsequent invocations short-circuited on the empty file and never re-tried bootstrap. On the Mercury host this surfaced during the 2026-06-22 incident when ~/.specialists/observability.db was 0 bytes, blocking PID/RSS-level forensic reconstruction during the 17:51–20:26 UTC window. Open failures are now surfaced explicitly so the orchestrator/operator can diagnose, and the bootstrap path no longer leaves the placeholder behind.
• sp edit --global <name.field> <value> now applies the value instead of falling through to vim (unitAI-61h1b, PR #123). parseArgs no longer consumes the first positional as the specialist name when --global is active, so the documented positional form works without --set. In addition, bare sp edit --global (no path) now fails fast with a script-friendly hint (specialists edit --global --set <name>.<field.path> <value>) when stdin is not a TTY — prior behavior was to spawn $EDITOR and hang indefinitely in non-TTY contexts (scripts, hooks).
• sp script JSON recovery from streamed assistant text (PR #146). When PiAgentSession.getLastOutput() leaked raw tool-call markup (<|tool_calls_section_begin|> …), trusted local sp script runs (e.g. service-skills-sync) failed with invalid_json. The final-wrapper recovery path now prefers the last completed streamed assistant message as a cleaner source before falling back to the one-turn JSON repair flow. Adds a focused regression test for the real failure mode and rebuilds dist.
• Live smoke harness architectural improvements (unitAI-o5rwj, PR #124). New tests/integration/cli/live-smoke.helpers.ts (createLiveSmokeHome scopes specialist overrides via XDG_CONFIG_HOME while preserving real HOME so ~/.pi credentials resolve; snapshotJobIds + waitForNewJobId provide filesystem-based job-id discovery when stdout is empty) and a .beads symlink so sp run dispatched from a temp repo can resolve beads created in the repo root. Improves test isolation and unblocks KAN-91 live smoke.

Security

• esbuild pinned to ^0.28.1 to clear GHSA-gv7w-rqvm-qjhr (unitAI-mtghu, PR #118). Clears the CVSS 8.1 High advisory (GHSA-gv7w-rqvm-qjhr) and the CVSS 2.5 Low advisory (GHSA-g7r4-m6w7-qqqr) on esbuild ≤ 0.28.0, which were failing OSV scan on every PR. esbuild is transitive (via tsx@4.22.3 and vite); a single package.json override resolves both ranges.
• hono bumped to 4.12.26 and vite pinned to 8.0.16 (devDependency) for OSV cleanup (PR #149). Clears residual OSV findings; vite is moved back to devDependencies after a bun update had promoted it into runtime deps, and bun.lock regenerated.