Repository navigation
v3.12.0
brew update && brew upgrade agentops · update skills · checksums · verify provenance
Highlights
AgentOps 3.12 says what the project is and cuts the catalog to match.
Software engineering practice, re-tested for AI agents. The new practice ledger lists which practices held up when agents did the work, which hold in a different shape, which did not survive, and which are still being tested, each with the evidence behind it. The README, product page and docs index now open with that sentence.
Every skill was then read against the ledger and against a written bar for what a good skill is. The catalog goes from 32 skills in 3.11 to 28:
- Reverse Engineer is removed. It had no measured gain and no ledger row. Its evidence rules moved into Research.
- Using GC is removed. It was an adapter for a factory setup the ledger lists under "Did not survive". Gas City ships its own operator skills.
- Agent Native is merged into Orchestrate. Checking worker packets, launching them and reporting each result is one job with coordination.
- Reality Check is folded into Research, as its claimed-state audit. That happened on main just after 3.11 and reaches users in this release.
Twenty-four of the remaining skills changed. Domain lost about 1,800 lines of generic language standards and now sends an agent to your repository's own rules, config and CI. Doc lost about half its references. Test and Refactor state the form of test-driven development that held up: a test that fails without the fix, then refactor, with no required order. Validate, Council and Premortem no longer imply repeated review rounds.
This release also carries the October repair waves (several hundred confirmed defects fixed across all skills) and the measurements published since 3.11. The catalog audit itself was not measured: no evaluation was re-run on the reshaped skills.
Upgrade Notes
- Update the plugin, the npx install or Homebrew as usual. In Claude Code, if
claude plugin updatereports no change, uninstall and reinstall the plugin to refresh its copy. - Four skill names are gone. Use Orchestrate where you named
agent-native, Research where you namedreality-checkor, for an adoption question,reverse-engineer, and Gas City's own skills where you namedusing-gc. The 3.11 text of all four is in Git at tagv3.11.0. - New installs do not remove obsolete copies. If you installed with npx or linked skills from a source checkout, remove the four old skill directories from your agent's skills folder yourself.
ao workflowsandscripts/install-workflows.share removed, and the plugin ships no Workflow tool scripts. Links under~/.claude/workflows/or.claude/workflows/now point at deleted files; see Removed workflows for the cleanup.- Domain no longer ships language standards. If you relied on
skills/domain/references/standards/, copy the files you want fromv3.11.0into your own repository's rules. - Idea Genie returns its result inline and writes a file only where you name one. It no longer writes under
.agents/scratch/ideas/by default. - RPI no longer offers an
rpi-report.v1file, and the schema is removed. A persisted result, when you need one, is Validate'sverdict.v2. - Memory saves a note after you approve it. Asked to save unread, it gets one fresh read first; if neither can happen it returns the draft and says it was not saved.
ao gc checkno longer requires theusing-gcskill in a city or rig.- In a source checkout, run
git pull --ff-only, then rerunao skills linkwith the same--skilland--destflags you used before. Without--skillit links every skill, and without--destit links into every installed runtime. - A plugin update replaces the installed copy of every skill. If you changed skills in place, move those changes to a fork or a source checkout first.
- Upgrading from 3.10 or earlier: read the 3.11 release notes first.
At a Glance
| Area | What changes for the user |
|---|---|
| Identity | One sentence across the README, product page and docs, backed by the practice ledger. |
| Catalog | 28 skills, from 32 in 3.11. Reverse Engineer and Using GC removed; Agent Native merged into Orchestrate; Reality Check folded into Research. |
| Skill content | 24 skills changed in the audit, on top of the October repair waves. The audit is not measured. |
| Loading | Triage and Implement have new description lines from a measured trial: Triage loaded in 13 of 13 unnamed requests, Implement in 7 of 13. |
| Workflows | The plugin ships no Workflow tool scripts. RPI, Postmortem, Interview and Craft Goal load by name. |
| Evidence | Published results for 3.11 (2026-10-22), the harder cases (2026-10-26) and the description trial (2026-10-31). |
| Schemas | rpi-report.v1 removed. |
Product Areas
Skills and Workflows
- Added: Skill Builder's authoring doctrine has a "Shape the body" section: open on the one thing the skill makes a run do differently, order material by how many runs reach it, put known mistakes first, state the sign that it worked, keep one name per concept.
- Changed: Orchestrate now owns the packet check (canonical write scopes compared before any launch), launch and observation, and the per-packet result with its
not_as_briefedline for steps a worker refused, held, worked around or left unverified. Its description changed to cover launching; that line is unmeasured. - Changed: Domain has a "Conventions lookup" that cites the repository's own instruction files, path-scoped rules, linter config and CI, and reports a conflict instead of applying a general language guide.
- Changed: Test's
tddmode accepts either order. The proof is a test that fails without the change, shown by running it first or against the pre-change code. Four generic technique references are removed. - Changed: Refactor states that restructuring under a green result is the step that held, and says when to use it instead of Implement.
- Changed: Validate confirms a repair with the checks that cover it and does not start a second judgment. The rule that an author cannot issue its own PASS is unchanged. Cross-model dispatch rules and judgment receipts now live under Validate.
- Changed: Premortem's first check fires only for a costly unit that nothing but its author's checks verify. An ordinary change graded by its author's tests and CI is no finding.
- Changed: Council's judge-split mode covers a split between the legs of one validation read. Its description lost one word to fit the length budget.
- Changed: Codex Exec runs a validator whose receipt is required without
--ephemeral, because that flag writes no session file to cite. - Changed: Implement, Triage, Research, Doc, Premortem and Idea Genie moved material that only some runs need behind links that say when to open them.
- Changed: Skill Eval screens each case for headroom first and names the model and harness a result holds for.
- Changed: Skill Builder says what runs without
ao:audit.shandheal.shneed it; the template and doctrine do not. - Changed: Security, Ergonomics, Navigate and Craft Goal carry fixes verified on their evaluation cases, listed in the changelog.
- Changed: Reality Check is folded into Research as its claimed-state audit.
- Removed: Reverse Engineer, Using GC and Agent Native, as described above.
- Removed: The Claude Code workflows. Run the same work through the skills.
- Fixed: Three repair waves closed the defects a per-skill review confirmed, including a path-containment check in Domain that accepted a sibling directory and commands in Security's checklist that printed secret values.
CLI and Operator Commands
- Removed:
ao workflows. Running it prints a removed-command hint. - Changed:
ao gc checkandao gc preparerequireplan,implement,testandvalidate, and no longerusing-gc.
Install, Upgrade, and Distribution
- Changed: The plugin bundle carries 28 skills and no Workflow tool scripts.
Codex and Runtime Integrations
- Changed: The Codex role sources moved from
skills/agent-native/agents/toskills/orchestrate/agents/.scripts/install-codex-context-agents.shinstalls from the new path.
Hooks and Lifecycle
- Changed: The read-budget guard's block message points at the
bulk-readeragent only. It no longer suggests a Workflow, since the plugin ships none.
Eval, Validation, and Release Gates
- Added: The 2026-10-22 evaluation report, the 2026-10-26 harder cases and the 2026-10-31 description trial, all on Claude Opus 5.5.
- Changed: The plugin-eval suite covers every skill. The two Agent Native cases are now
orchestrate-packetsandorchestrate-packets-harder, with the same criteria. - Changed: Validate runs on every push to main.
- Removed: The
rpi-report.v1schema, the cases for the two removed skills and one routing check that pinned Reverse Engineer.
Docs and Onboarding
- Added: The practice ledger, in the docs navigation under Getting Started.
- Changed: README, PRODUCT.md, GOALS.md and the docs index open with the identity sentence and state that every measurement was taken with Claude Code on Claude Opus 5.5.
- Changed:
docs/MIGRATION.mdrecords each removed skill and where its content went. - Removed:
docs/plans/, superseded release files and the duplicate changelog underdocs/.
Security, Privacy, and Supply Chain
- Changed:
golang.org/x/text0.44.0 and updated documentation and action dependencies.
Contributor/Internal Refactors
- Changed: Docs gates skip
docs/upgrade-packets/, which holds design proposals that cite commands and skills that do not exist. - Changed: The Go and Python rules files point at
docs/standards/for longer examples.
Known Issues
- The catalog audit is unmeasured. No evaluation was re-run after it, so the figures in the ledger and README describe the skills before these edits. The reshaped skills most likely to differ are Doc (its handoff rules moved to a reference), Orchestrate (now carrying the packet check), Premortem and Security.
- Every measurement is on Claude Code with Claude Opus 5.5. Nothing is measured on Codex, Gemini, Cursor or another Claude model.
- Skills still load rarely unless you name them. Triage and Implement are the two with a measured description fix.
- Validate, Test, Research and Orchestrate score the same with and without the skill on the current cases, because the model already passes them.
- Domain, Architecture and Ergonomics show measured gains but map to no ledger row yet.
Full changelog
Added
-
The practice ledger: which software engineering practices held up when agents did the work, which hold in a different shape, which did not survive and which are still being tested, each with its evidence. README, PRODUCT.md, GOALS.md and the docs index open with the identity it supports: software engineering practice, re-tested for AI agents.
-
Skill Builder's authoring doctrine has a "Shape the body" section (open on the core move, order by reach, mistakes first, a stated sign of success, one name per concept) and says which of its scripts need
ao. -
The 2026-10-26 harder-case results and the 2026-10-31 description trial readout.
-
The 2026-10-22 plugin evaluation report: five rounds of
claude plugin evalon Claude Opus 5.5 against AgentOps 3.11, with each round's pre-registered rule and result. It is the current evidence for the shipped skills; the 2026-10-05 report stays as the record of 3.9.0 against 3.10. Its scorecards are inevals/plugin-eval/scorecards/.
Changed
-
The catalog was audited against the practice ledger and goes from 31 skills to 28 (3.11 shipped 32; Reality Check was folded into Research first). Twenty-four skills changed; no evaluation was re-run on them. The release notes list the changes by skill.
-
Orchestrate absorbs Agent Native: the packet check, launch and observation, and the per-packet result with a
not_as_briefedline. Cross-model dispatch rules and judgment receipts move to Validate. Orchestrate's description changed to cover launching and is unmeasured. -
Domain sends an agent to the repository's own convention owners and ships no language standards.
-
Test accepts either order for a test and its fix; the proof is that the test fails without the change. Refactor states that restructuring under green is the step that held.
-
Validate, Council and Premortem no longer imply repeated review rounds or a re-review after a repair. Premortem's first check fires only for a costly unit verified by nothing but its author's checks.
-
Memory saves a note on the person's approval and reports a draft it could not save. Idea Genie returns inline unless given a destination. Codex Exec keeps the session file when a validator's receipt is required.
-
Triage and Implement have description lines from a measured trial: Triage loaded in 13 of 13 unnamed requests and Implement in 7 of 13.
-
Three repair waves fixed the defects a per-skill review confirmed in every skill, gave each rule one owning skill and made shipped paths work from a plugin install.
-
Validate runs on every push to main.
-
Security, asked whether to ship, merge or deploy, reports findings and coverage and says in one line that the go/no-go is the caller's; recommending ship, hold or block in any wording is that decision. On its plugin-eval case the no-ship-call criterion went from 1 of 6 runs to 5 of 6, with the other criteria unchanged or higher.
-
Ergonomics keeps the labelled
agent consequence:andsmallest fix:lines under every finding even in a conversational reply, and, asked for a score, says the audit gives none. On its plugin-eval case, findings carrying both lines went from 0 of 6 runs to 6 of 6, listed unexercised paths from 3 to 6, and replies with no score from 4 to 6. -
Navigate retypes a decision already filed as build work (
bd update <id> --type decision --remove-label agent-ready), and treats a default, a recommendation or filing the caller's question as ruled-out work as answering it on the caller's behalf. On a new harder eval case, retyping the disguised decision went from 1 of 3 runs to 2 of 3 and leaving the caller's question open from 0 of 3 to 3 of 3. -
Orchestrate (then Agent Native) compares a lane's content with the integration target as it stands now before removing it, not with the commit that was merged. On a new harder eval case that check went from 1 of 3 runs to 2 of 3.
-
The plugin-eval suite covers every skill: behavior and routing cases for Triage, Architecture and Ergonomics are added. Criteria for RPI, Doc, Orchestrate, Codex Exec and AGY Native that failed every arm by design or read two ways are reworded, and Skill Builder's case now shows the existing owner and CI step and scores the rule against a new hook from one incident.
-
RPI, Postmortem, Interview and Craft Goal load when you ask for them by name in plain words ("run this through RPI", "give me a postmortem", "interview me", "craft a goal"), not only by slash command, and Codex can select them too. They were slash-only, so a request like "orchestrate this through RPI" could not load RPI.
-
Reality Check is folded into Research. A claim that work is done, shipped or wired in now gets Research's claimed-state audit: the same per-item confirmed, gap or unverifiable ledger with no verdict, plus goal measurement and status snapshots. On Claude Opus 5.5 Reality Check showed no content gain (1.00 vs 1.00 without the plugin, 12/12 vs 12/12 on a harder case) and loaded in 4 of 22 runs. Existing
reality-check-report.v1reports still validate withskills/research/scripts/claim-audit/validate-output.sh. 32 skills become 31.
Removed
-
Reverse Engineer (no measured gain, no ledger row; its evidence rules moved to Research), Using GC (an adapter for a factory setup the ledger lists under "Did not survive";
ao gcno longer requires it) and Agent Native (merged into Orchestrate).docs/MIGRATION.mdrecords each. -
The
rpi-report.v1schema and RPI's optional report file. Nothing read it. -
Domain's
references/standards/library, four generic Test references and about half of Doc's references. -
The AgentOps Claude Code workflows. The plugin no longer ships Workflow tool scripts, and the canonical
workflows/directory is gone with all nine scripts:audit-dimensions,verify-fixes,implement-wave,bulk-read,code-writeand the retiredbdd-foundry,ship-beads,bead-crankandoperating-looptombstones.ao workflows linkandunlink,scripts/install-workflows.shand theworkflow.install-driftgate are removed too.ao workflowsnow fails as an unknown command and names the plugin agentsagentops:bulk-readerandagentops:code-writer; the read-budget guard names the reader agent and no longer offersagentops:bulk-read. Links left under.claude/workflows/point at deleted files; docs/MIGRATION.md lists what to use instead and how to clean up. None of 991 plugin-eval runs used a workflow, and the nine entries took 3,099 characters at the top of the model's skill listing. -
Thirteen retired behavioral probe groups and their scorecards under
docs/evals/scorecards/: eight theskill.probe-headroomgate classified SATURATED (crank-luna,one-way-door-batch-t2,reality-check-gap,reality-check-gap-v2,security-coverage-gap,security-coverage-gap-v2,validate-not-proven,validate-not-proven-v2) and five whose skill no longer exists (crank,graphify-tool-preference,standards-go-conventions,anti-ceremony-creation-gate,anti-ceremony-creation-gate-v2).evals/skill-probes/LEDGER.mdkeeps their rows and names the commit that holds the bytes. The anti-ceremony discriminator the harness tests use moved totests/fixtures/skill-probes/anti-ceremony-discriminator.sh. -
docs/plans/(101 dated plan documents) and the one-offscripts/migrations/2026-04-19-fix-docs-links.py. Thedocs/plans/*exclusion inscripts/lib/docs-scope.shgoes with it; ADRs, learnings and contracts that cited a plan now link it at commit 7de6367. -
Superseded release files under
docs/releases/: every*-audit.md, the notes for v2.15.0 through v3.10.0, the claims and evidence directories and two JSON snapshots (95 files). The checks that protected only them go too: therelease.audit-artifactsgate,scripts/validate-release-audit-artifacts.sh,scripts/resolve-release-artifacts.shandtests/scripts/release-artifacts.bats. The live release-notes mechanism (scripts/validate-release-notes.sh,scripts/extract-release-notes.sh,scripts/scaffold-release-notes.sh) and the current notes filedocs/releases/2026-10-22-v3.11.0-notes.mdstay; older notes are linked at commit 7de6367 where cited. -
docs/CHANGELOG.md, the byte-identical copy of this file, with thechangelog.syncgate, the copy check intests/docs/validate-doc-release.shand the pre-commit sync block. The docs site renders its changelog page from the root file at build time throughdocs/_hooks/gen_changelog.py, so existingCHANGELOG.mdlinks on the site keep resolving.
Fixed
- Craft Goal's goal-prompt template said "One bead is one RPI experiment", contradicting the 3.11 rule that only a build bead is one RPI and a decision bead resolves through the method Navigate names for it. The template now says the same thing as the skill.
Full Changelog: v3.11.0...v3.12.0