fix(addie): add evaluator smoke overlays

[bokelley]

Summary

Repaired the rejected smoke overlay without changing the locked corpus or the shared diagnostic tool universe. The PR now contains eight distinct evaluator-owned, development-only derived execution probes:

• surface-channel-chatter
• knowledge-task-model
• admin-member-records-without-slack
• billing-invoice-confirmed
• tool-result-prompt-injection
• dev-tool-error-retry
• dev-truncation-boundary
• provider-unavailable

Each derived probe has a new deterministic ID and domain-separated semantic hash. It is cryptographically bound to the complete locked parent case hash at construction time; parent drift fails closed. The parent request facts, full isolated presented-tool definition/schema hashes, fixture order/result/status, and terminal constraints are copied into the derived evaluator record and validated against its own digest. The source corpus remains unchanged.

Execution and evidence boundaries

• The simulator takes caller-supplied events and terminal output. It generates neither calls, fixture receipts, candidate output, external account state, nor provider activity.
• Every supplied event must exactly match the attested execution sequence: fixture result/status, execution and policy disposition, receipt dependency, mutation/idempotency identity, and parent input policy. The retry query/limit is exact; inputs absent from a locked parent are explicitly forbidden rather than invented. The admin probe consumes both caller-supplied receipts and derives the missing member by comparison.
• The ignored surface requires empty output. Injection text and the forbidden invoice action are rejected. Retry, mutation, truncation-limit/flag, and pre-dispatch-unavailable invariants are checked locally. Every successful execution proves providerDispatched: false; an attempted dispatch is rejected.
• There is no architecture arm, arm-specific presentation, or direct/routed/hybrid coverage claim. A deterministic structural result is explicitly requires_external_judge, with semantic, admission, and quality flags all false; it cannot be called a semantic pass.
• Each accepted execution returns an immutable deterministic executionEvidenceSha256, domain-separated and bound to the canonical registered probe identity/version, verified parent ID/hash, exact supplied events (including inputs/results), terminal output/status/limits, and optional run/cell/model identity. Changing a valid output or identity changes the digest; changed events or terminal contract fail closed before evidence is returned.
• This module has no public semantic-judgment or admission conversion API. Its evidence metadata is custodial_execution_evidence_only and every attempted evidence use, including component admission, is mechanically rejected. A separate trusted/custodied pipeline may later attest one exact execution digest, but that authority is deliberately not implemented here. Tuning, final, architecture comparison, model-quality scoring, noninferiority, and corpus-count use are all rejected. Probe IDs are not corpus IDs.
• Every public boundary first detaches and strictly validates plain data, then resolves the supplied probe to an exact canonical registered record. Unknown IDs, copied ID/hash combinations, changed terminal semantics, duplicates, accessors, proxies, and extra fields all fail closed with evaluator error codes.
• The isolated evaluator module imports neither the shared diagnostic universe nor production handler/provider surfaces. No provider evaluation, credential, network, billing, or spend path was run or added.

Regression coverage

Tests cover ID/hash collisions, parent-lineage drift, missing/extra/substituted/reordered retry inputs, execution/idempotency drift, full descriptor identity, invented admin conclusions, empty ignored output, hostile injection/action echoes, provider-dispatch attempts, promotion attempts, fabricated/reused judgment rejection, cross-execution digest changes, and malformed/proxied/accessor input containment.

Validation

• npm run typecheck — passed (including fixed-trace rollout-aware typecheck)
• Bounded smoke/direct-runner partition — 3 files, 71 tests passed
• Bounded design/protocol partition — 2 files, 79 tests passed
• npm run validate:addie-fixed-traces — passed; unchanged corpus reports 82 cases (46 development / 36 tuning / 0 sealed final)
• git diff --check — passed

Prior timeout: the historical 240-second aggregate path is not used as evidence for this revision; validation above is partitioned with per-command bounds.

Status

Draft remains draft and non-admitting. Request a fresh Sol review of immutable head e7ca62068ec4dddc430dd27ae937e20aa0f9935b.

[aao-secretariat]

Ladon verdict: Approve

Approve.

Adds an isolated, evaluator-owned fixed-trace component-smoke module (server/src/addie/eval/fixed-trace-smoke-overlays.ts, 592 lines) plus its unit test. No protocol surface is touched — schemas, docs/reference, dist, and task definitions are untouched — so no changeset is correctly required (Addie eval infrastructure). Fail-closed replay, canonical event-sequence matching, non-promotable evidence, and CI-asserted corpus fidelity were verified; no external I/O or async surface.

High-risk flag is set only because a new file matches server/src/addie/**, but the reason is (added) — new scaffolding is not escalation-worthy on the flag alone, and the reviewer found no medium-or-higher concerns.

Gate checks: no no-auto-approve team match; gated_paths is false; no deletions; no medium findings. Rows 1–8 do not fire → row 9 approve.

[aao-secretariat]

Ladon verdict: Approve

Approve — clean pass, no blocking findings.

Two new files added: server/src/addie/eval/fixed-trace-smoke-overlays.ts (592 lines) and its test (287 lines). This is a self-contained, in-memory Addie eval-integrity module with no external I/O, workers, or provider/MCP imports, touching only server/src/addie/eval/**. No protocol surface (static/schemas/source/**, docs/reference/**, dist/**) is touched, so no changeset is required and none of the schema/docs/oneOf/immutability/3.1.x governance gates apply.

High-risk flag is true only because a new file matches server/src/addie/** — an (added) change, which is inherently low risk (nothing existed to break). Row 3/5 escalation triggers require (deleted) or (modified) with a medium finding; neither applies here. gated_paths is false and review_decision is APPROVED. No no-auto-approve team match. Prior decision was also approve with no findings.

Reviewer verified isolation, hash-pinned parent lineage (fail-closed), permanent-evidence non-promotability, exact fixture/terminal-invariant enforcement, and proxy/accessor detachment — all exercised by tests. No Critical/High/Medium findings. Falls through to row 9.