Repository navigation
brew update && brew upgrade agentops · update skills · checksums · verify provenance
Highlights
AgentOps 3.12.1 fixes one line. The 3.12.0 audit rewrote Orchestrate's description, and with the new line Orchestrate mostly stopped loading on requests where it used to load. Agy Native loaded less often too, although its own text did not change. This release replaces that line with one chosen in a measured trial.
The fix comes from the first measurement of 3.12: all 28 skills, with and without the plugin, on Claude Code with Claude Opus 5.5.
- Orchestrate loads again. Over five requests and six runs each, 3.12.0 loaded Orchestrate on 6 of 30 runs; the 3.11 text loaded its skill on 21 of 30. The new line loaded it on 22 of 30 in the trial.
- Agy Native loads again. On its harder request it loaded on 4 of 12 runs in 3.12.0 and on 6 of 6 with the new Orchestrate line.
- Nothing else regressed on these cases. No other skill fell more than 0.15 below its 3.11 score, with the same judge grading both, on a request the unaided agent fails. Implement did once when named (10/12 against 12/12) and a same-day re-run did not reproduce it (24/24 on both). With the plugin, the agent met 89% of the practice criteria against 65% without it.
The recheck report has every case, the rules fixed before the runs and the limits.
Upgrade Notes
- Update the plugin, the npx install or Homebrew as usual. In Claude Code, if
claude plugin updatereports no change, uninstall and reinstall the plugin to refresh its copy. Start a new session afterward: a running session keeps the old description. - No skill, command, flag or schema is added, removed or renamed. If you are on 3.12.0 there is nothing else to do.
- In a source checkout, run
git pull --ff-only. Linked skills pick up the new line without relinking. - Upgrading from 3.11 or earlier: read the 3.12.0 release notes first. Four skill names were removed there.
At a Glance
| Area | What changes for the user |
|---|---|
| Loading | On the tested requests, which ask what idle or finished agents should do or ask to run work in parallel, Orchestrate loaded on 22 of 30 runs without being named (6 of 30 in 3.12.0). Agy Native loaded as often as it did before 3.12.0. |
| Skill content | None. Only Orchestrate's description line changed. |
| Evidence | A published measurement of all 28 skills in 3.12.0, replacing the "not measured" caveat. |
| Build | The Go toolchain is 1.27.4. |
Product Areas
Skills and Workflows
- Fixed: Orchestrate's description. It now reads "Dispatch parallel workers without write collisions and decide what idle or finished ones do next. Use when: managing agents, or running or planning agents in parallel, even two." The skill body is unchanged.
Eval, Validation, and Release Gates
- Added: The 2026-11-11 recheck of 3.12.0 and its scorecard: 42 behavior cases in three arms, graded blind by Claude Fable 5.2 with the stored 3.11 responses regraded by the same judge, and 30 routing cases that count whether the skill loaded.
- Added: A test that every
docs/evals/report the README links to exists.
Docs and Onboarding
- Changed: The README evidence section, the practice ledger and the eval suite README carry the 3.12 figures and no longer say the audit is unmeasured.
Security, Privacy, and Supply Chain
- Changed: The Go toolchain is 1.27.4.
Known Issues
- Every measurement is on Claude Code with Claude Opus 5.5. Nothing is measured on Codex, Gemini, Cursor or another Claude model.
- Why Orchestrate's old line kept Agy Native from loading is not explained. The runs that missed loaded no skill at all.
- Doc, Navigate, Research, Review, Security and Validate did not load on their unnamed routing request in 3.11 or 3.12. Name the skill when you want its rules applied.
- On 22 of the 42 cases the agent meets more than 70% of the criteria without the plugin, so they cannot show whether a skill's content got worse. Codex Exec, Council, Doc, Domain, Postmortem, Review, Skill Eval and Test are measured only on such cases.
- The new Orchestrate line was written after seeing the requests it was tested on, so its loading counts may be higher than on requests it has not seen.
- The harder of the two packet requests still loads Orchestrate rarely (1 of 6 runs).
Full changelog
Fixed
- Orchestrate's description. The line the 3.12.0 audit wrote loaded Orchestrate on 6 of 30 runs where the 3.11 text loaded its skill on 21 of 30, and it cut Agy Native's loading from 10 of 12 runs to 4 of 12. The new line, chosen in a same-day trial of four candidates, loads Orchestrate on 22 of 30 runs and Agy Native on 6 of 6. The skill body is unchanged.
Added
- The 2026-11-11 recheck of 3.12.0 on Claude Opus 5.5: all 28 skills with and without the plugin, graded blind by Claude Fable 5.2, with the stored 3.11 responses regraded by the same judge. No other skill fell more than 0.15 below its 3.11 score on a case with headroom, except Implement when named (10/12 against 12/12), which a same-day re-run did not reproduce. Its scorecard is in
evals/plugin-eval/scorecards/. - A test that every
docs/evals/report the README links to exists.
Changed
- README, the practice ledger and the eval suite README carry the 3.12 figures: 89% of practice criteria met with the plugin against 65% without.
- The Go toolchain is 1.27.4.
Full Changelog: v3.12.0...v3.12.1