Skip to content

sdlc-process 0.4.10 — Haiku 5.5 observer

Choose a tag to compare

@cchatfield cchatfield released this 07 Oct 20:41
0f2c37f

Use explicit claude-haiku-5-5 for the SDLC observer and require Claude Code 2.1.293 through the existing setup check. Regenerate the packaged observer/setup runtime. Preserve the user's MAIN model and the Sonnet Grok/Codex supervisors. Release sdlc-process 0.4.10.

The same ten maintained observer-decision fixtures were replayed once per model at medium effort on Claude Code 2.1.293. Actual assistant records confirmed claude-haiku-5-5 and claude-sonnet-5-5; no fallback. Both matched all ten existing acceptance outcomes:

Cases Both models
f1, f2 Findings: omitted setup validation / checkout backstop
f3, f4 Assessments: acknowledgement/plan accepted, execution still pending
f5, f6 No report: retained remainder / genuine supersession
f7 Finding: unsupported completion
f8 No report: missing decision stays unknown
f9, f10 Findings: omitted cleanup / retained visible decision despite partial history

Both produced parseable markers with actual decision/action UUIDs, no false-positive findings, and no false resolution claims. Haiku's CLI-reported list cost was $0.01283817 versus Sonnet's $0.16614780 (92.27% lower). Total host time was 68.440s versus 108.773s; median 7.010s versus 7.183s. One ordered batch, not randomized or an invoice estimate.

Limits retained: both skipped the required native delivery-identity Read in f3, so this is not a full instruction-conformance pass (9/10 on that broader accounting). Haiku f2 attempted unresolved source-template paths; the finding rests on the visible decision, not claimed successful source loading. Replay used the documented text fallback for unavailable ObserverReport, not a live MAIN session. No first-turn/native attachment qualification or supervisor migration is claimed.

Evaluation source: 9a67df7. Exact unchanged observer instruction-body SHA256: f63f44383c74a90a87090031cb48903d1e965c843a465673828722614c0fd0c8. Existing materializer produced baseline/digest inputs; acceptance was withheld; existing markers() parser checked reports. Disposable transcripts are not committed.

Native evaluation command per model/case: claude -p --safe-mode --model <full-model-id> --effort medium --system-prompt-file <exact-observer-body> --tools Read --allowedTools Read --permission-mode dontAsk --setting-sources '' --strict-mcp-config --no-session-persistence --output-format stream-json --verbose, with the materializer output and documented fallback on stdin, isolated temporary configuration and no model override environment variables.

Official model sources: https://www.anthropic.com/claude-haiku-5-5 and https://code.claude.com/docs/en/model-config. Prior comparison: #50; separate Sonnet supervisor qualification: #53.

Validation: 38 affected Node tests, 9 packaging tests, 74-file generator check, source/generated parity, and diff checks passed. Tests cover version boundaries, exact installed observer model, and preservation of MAIN model/settings. Independent review results follow before integration.

Native model-selection smoke: Claude --agent sdlc-process:sidkik-sdlc-observer --plugin-dir <candidate>/sdlc-process with no --model override returned NO_REPORT; the actual assistant/modelUsage record is claude-haiku-5-5. This verifies packaged frontmatter selection, not live observer attachment.

Independent Standards and Spec reviews: no candidate findings; conditional publication pass for reviewed tree 0a6fbc987a287695e2e36b0ee70d255c8bae61b3. Conditions: preserve these bytes through integration/release, passing applicable checks, and supported installation with released-byte/settings/path verification. No claim that installation proves live observation. Known replay limitations above remain owned by the accountable session for any later observer-conformance qualification.