Skip to content

v1.5.0 — The Opus 4.8 Reliability Rework

Choose a tag to compare

@RaNDoM6913 RaNDoM6913 released this 04 Jul 18:56

This is the Opus 4.8 reliability rework: every prompt surface in the kit — all 56 agents, 16 commands, 22 skills, and the always-loaded rules layer — rebuilt from the ground up for a more literal, weaker-executor model against a written failure-mode playbook. It started with an audit: a 16-agent sweep over 117 kit files surfaced systemic rot the old prose had hidden — internal contradictions (checklists that disagreed with their own enforcement hooks), broken cross-references (dead references/ pointers, ghost /dev phases, pointers to skills that don't exist), and enum drift (three competing severity vocabularies, overlapping confidence bands, fractional phase numbers). The method was uniform and unglamorous: rewrite every file to one canonical contract, then have an independent adversarial instance verify it — no self-grading, no file left on trust. Nothing about what the kit does changes; everything about how reliably it does it does.

🤖 Agents (56)

Rewritten in three batches, each rewrite verified by a separate instance (0 unresolved escalations across all three).

  • Batch 1 — all 31 core agents. Every agent now carries a top Hard Rules block (≤7 bullets) and a bottom Recap (a lost-in-the-middle guard), an exact fenced Output Contract with a filled mini-example, and either an Evidence Gate (reviewers emit findings only with a file:line actually read + a concrete failure mode; NOT FOUND instead of invented content) or a Done-gate (generators verify artifacts on disk, actually run what they generate, separate VERIFIED from ASSUMED). Canonical enums throughout — severity CRITICAL/WARNING/SUGGESTION, confidence HIGH ≥80 / MEDIUM 60–79 / LOW <60 with LOW routed to Open Questions, never silently dropped. Named contradictions killed: critic's ghost "/dev Phase 8.5", plan-checker's overlapping REVISE/BLOCK thresholds (now a clean verdict precedence), security-scanner's triple severity vocabulary, visual-reviewer's impossible scoring math, plus code-reviewer's drop-vs-route LOW conflict, debug-observer's read-only-but-edits contradiction, and comment-rot-analyzer's file-age-instead-of-TODO-age command.
  • Batch 2 — 25 stack / frontend-3d / GAN / extras files. Same playbook for the 9 stack reviewers (go×6, ts, py, rs), 4 frontend-3d reviewers, 4 extras, and the whole GAN package. Highlights: stack references/ pointers fixed for the real installed layout (.claude/agents/references/ + Glob fallback + SKIPPED note; go-reviewer's index now lists every v1.4.0 doc), the nonexistent maps.NewWithSize() corrected to make(map[K]V, n), and INFO severity / percentage-confidence normalized to the canonical bands. GAN redesigned end-to-end — the planner→evaluator rubric handoff now carries a mandatory ## Rubric section, the evaluator scores X/N against a both-rubrics default, rubric row counts corrected to the real 21/15, BLOCKED reserved for un-runnable evaluations, and the generator fix-loop capped at 3 attempts, with all 3 Codex mirrors synced.
  • Batch 3 — 6 frontend-ui agents. Frontmatter repaired: the pre-fix inject-tokens bug had wedged tokens: mid-description in every one of these agents, making their Dispatch / Do-NOT-dispatch conditions invisible to Claude Code's dispatcher — descriptions were reconstructed as single-line YAML (PyYAML-parse-verified) with dispatch conditions restored. The umbrella ui-reviewer no longer claims to "dispatch" specialists (subagents can't spawn subagents) — it recommends them by name for the caller; specialists gained their own standalone Output Contracts; reflex-audit thresholds and the typography 1.25x scale unified across the set.

⚡ Commands (16)

  • /dev rewritten by hand to linear phases 0–15, matching the flow SVGs exactly — the old fractional 1.5/1.7/2.1 numbering is retired in favor of a single Skip Matrix (Simple skips 2,3,5,6,8,11,13; Standard skips 2,3,13), per-phase Done-when, Hard Rules + Recap, gate enums consumed verbatim, and the ghost "--quick mode" / PASS-instead-of-PROCEED bugs fixed.
  • /review — goal-verifier moved to a verdict track that bypasses finding-validation, and Step 2/3 dispatch synced so silent-failure-hunter, comment-rot-analyzer, api-contract-sync, and database-reviewer are actually dispatched.
  • /migrate gained a destructive-action gate: prisma migrate reset is no longer sold as rollback (honest "no true down" + a migrate diff alternative), Drizzle apply corrected to drizzle-kit migrate, Knex rollback contradiction fixed.
  • /audit now states the two-way PASS/WARN/FAIL contract with the audit trio + a Grand Summary aggregation table.
  • /capture-screen rebuilt on flags actually verified against npx playwright screenshot --help — the nonexistent --device-scale-factor is gone, 2x capture routed through a device descriptor, and the previously-fictional data-screen-id element capture is now really implemented.
  • /superkit-init and /superkit-evolve pinned to the canonical shipped doc names (killing the permanent false [MISSING]), with the Russian scaffold leak, find-precedence bug, and GNU-only grep -oP / BSD-only stat -f all fixed to POSIX-portable, executed-and-verified constructs.
  • /benchmark --compare got a guaranteed git-restore epilogue plus a benchstat pre-check with raw fallback.

📚 Skills (22)

  • The two meta-skills now encode the new kit conventions, so every future component inherits them: writing-agents teaches the full playbook skeleton (Hard Rules → Phase 0 → Process → fenced Output Contract with a complete, placeholder-free example) and the canonical enums; writing-commands encodes the command contract ($ARGUMENTS exactly once, per-step Done-when, gated report, destructive-action gates, linear numbering).
  • writing-hooks timeout units corrected to SECONDS — the old text would have produced 5000-second timeouts on copy-paste — and its 3-vs-4 profile contradiction reconciled.
  • ru-text now obeys its own rules: every canonical example ends with proper „лапки" (hexdump-verified) and invisible NBSPs are shown as visible ␣ markers with a legend.
  • Verifiers caught copy-paste-breaking code bugs in the reworked skills: redis-patterns' duplicate const results redeclaration, drizzle-orm-expert's undefined pool/orders tables, telegram-bot-builder's undefined items, threejs-color-management's undefined texture — plus dated advice corrected (PostgreSQL 12+ inlines plain CTEs — AS MATERIALIZED is the real optimization fence).

📏 Rules (20)

The always-loaded layer is now compact, generic, and hook-accurate.

  • documentation.md — the "15-Point" checklist that actually had 19 rows and disagreed with the enforcement hook's own path→doc map is now ONE canonical 11-row table, verified against doc-check-on-commit.sh's case statements; TGApp-specific rows moved to a marked "ADAPT" block and the config-exemption contradiction resolved with a precedence rule.
  • dev-workflow.md — fractional skip list replaced by the new /dev Skip Matrix, and the rule reconciled against the real hook wiring (no MultiEdit matcher, correct [hotfix:] reason logic, exempt commits don't reset the edit counter).
  • The 10 frontend rules were reworked under binding cross-file decisions and a cluster consistency pass: a single canonical reflex_fonts_to_reject list, unified reflex-audit thresholds (3+ checked → ONE CRITICAL), the gsap-vs-motion ease-in conflict resolved by explicit mutual scoping, and both heading anchors that impeccable-craft depends on preserved verbatim.

🔧 Tooling & hooks

  • bin/inject-tokens.js blank-line fix — it no longer corrupts YAML frontmatter of files with multi-paragraph block descriptions; blank lines inside a description: block scalar are treated as continuations, so tokens: lands after the block instead of mid-description (this was the root cause of the frontend-ui dispatch breakage above).
  • subagent-stop-validate.sh — stopped validating ghost agents (playwright-test-generatore2e-test-generator, so its generated-but-not-run guard actually fires; the nonexistent audit-security dropped) and its security branch now matches the severity token CRITICAL only, so confidence "HIGH" in the new [WARNING/HIGH] finding format no longer false-triggers the do-not-merge warning.
  • /security-scan auto-triggers realigned to capability — the command audits .claude/ configuration, so its triggers fire on config/MCP/install events, while application-code security is explicitly routed to the security-scanner agent via /review and /dev.
  • ui-reviewer name collision documented as intended precedence — on a fresh install the frontend-ui umbrella (richer superset) overwrites the core agent, merge mode keeps the pre-existing file, verified against copyFile semantics.

🔬 How it was verified

  • Method: a written failure-mode playbook plus a per-file audit brief for every surface — each rewrite targeted named defects, not vibes.
  • Every rewrite was adversarially verified by an independent instance (tiered models — Fable/Opus verifiers matched to each file's ownership), so no file graded its own homework.
  • Cluster consistency checks ran over the interlocking sets — the GAN planner/generator/evaluator/rubrics and the 10 frontend rules + their agents — to catch drift that per-file review can't see (e.g. a rule with 12 boxes where the agents had 11).
  • ~200 subagents across the batches, 0 unresolved escalations — the handful of caught regressions were fixed inline (ts-reviewer's dropped "no prop drilling" check restored, red-blue-auditor's Risk Score math corrected, GAN's toHaveScreenshot() capability restored, and more).
  • A final kit-wide consistency sweep came back clean: zero fractional /dev phase references, zero confidence-band or severity-enum drift, zero TGApp leaks in core, every references/*.md pointer and dispatch-table name resolves, every "N-point" label matches its real item count.
  • Green on the way out: 28/28 npm tests and 11/11 hook test suites.

📦 Compatibility & next

  • No breaking changes. This is a reliability and coherence pass — behavior surfaces are the same, they just hold together now.
  • All counts unchanged: agents 56 / commands 16 / hooks 42 / skills 22 / rules 19. Nothing added or removed; everything rewritten.
  • To pick it up, re-run the installer over your project — it refreshes the agents, commands, skills, and rules in place.
  • Coming next release: the remaining Codex skill mirrors (beyond the already-synced /dev trio and GAN package) will be brought up to the new contract — an owner decision to sync those with GPT directly rather than in this pass.
  • The showcase package was left intentionally untouched — it's a frozen snapshot of a real production .claude/ setup and stays as-is.