Skip to content

SeniorOS 0.9.0

Choose a tag to compare

@joshsegatt joshsegatt released this 03 Nov 23:36
· 115 commits to main since this release
77422a6

SeniorOS beats the strongest models alone on Claude Opus, Grok and Gemini: a reported bug is fixed at its cause, even in another module, and the guard holds in Grok.

Added

  • A reported bug goes to senior-debugging, which reproduces it through the real entry points the data goes through, not only the function the report names, and fixes the cause where it is, even in another module. A code fix of the cause needs no approval; rewriting data already stored still does, and the agent says which records stay wrong until then. senior-engineering runs the changed path at the repository's real volume (a seed, a config value, a note on production size) before saying done. Every host's routing rules carry the change.

    • Measured on Claude Opus with tasks written by agents that never saw SeniorOS, held out from tuning (o17, o19, o20, 5 trials): SeniorOS 15/15, Opus alone 0/15, Superpowers 0/15. Every arm found the cause; only SeniorOS fixed it where it was. On grok-4.7 and Gemini Pro, on the five tasks of this kind: 9/10 and 7/10 for SeniorOS, at most 1/10 without it.
  • Bench: an Opus set of 32 tasks (o1–o32) written blind to the pack in four rounds, and a Grok backend (--backend grok): each run gets its own Grok home and an empty profile folder, and the login is copied without its refresh token, so runs cannot rotate it. Results on 13 tasks, 2 trials: Grok 23/26 with SeniorOS, 15/26 alone, 14/26 with Superpowers; Gemini Pro 21/26, 16/26, 16/26. Opus core set, 3 trials: 24/24, 21/24, 19/24.

  • Planning before the first edit, in the routing rules of every host and in senior-engineering: a change with several steps, a shared contract or stored data, or a step that can fail halfway is planned before editing. The agent finds every reader of what changes and the data already saved in the old shape, and decides what happens if a step fails halfway, runs twice or meets old data. Renaming or reshaping a field, key or type that other modules or saved data read is no longer a low-risk edit. senior-finish checks the plan against the result.

    • Measured on Haiku, 8 trials each on h12-cancel-order and h13-line-rename: 15/16 with these rules, against 12/16 for the rules before them and 9/16 for the model alone. A separate senior-plan skill with the full procedure scored the same 15/16 at the same cost, so it was not shipped: it is kept in experiments/senior-plan/ with its measurement. See evals/bench/README.md.
  • The finish gate puts senior-finish to work: when the checks pass on a change senior risk rates medium or higher, it sends the agent back once per session to name the two likeliest ways the change is still wrong and the evidence that rules each out, then give the ship status. Low-risk changes are untouched. When nothing failed but the proof baseline is not passing (no tests ran, checks skipped), the review says NOT PROVEN instead of "the checks pass". In the Claude Sonnet bench, no run had loaded senior-finish on its own. Applies to Claude Code, Codex, Cursor, OpenCode and the Antigravity hooks.

  • senior-code-review gains a "Try to break it" pass for medium-risk changes and above, and senior-finish a step to name what could still be wrong before choosing a status.

  • Bench: Claude runs get an empty config folder and authenticate with CLAUDE_CODE_OAUTH_TOKEN (the login file is never copied), and run refuses to start when the user's own CLAUDE.md still reaches the runs, which Claude Code reads from the profile folder whatever the environment says. With it moved aside, Claude Sonnet on the core and hard sets: SeniorOS 38/38, Superpowers 7.0.1 35/38.

  • A test command that runs no tests is no longer a pass. senior finish and the finish gate read how many tests ran from node:test, Jest, Vitest, Mocha, pytest, unittest, cargo test and go test output; zero is SKIP (no tests ran), and the count is shown next to the check and reported as tests in --json. Before, npm test with no test files, or go test with no test files, reported PASS. Output that is not recognised changes nothing.

    • A node:test file with no tests in it is not counted either. node:test reports such a file as one passing test named after the file (Node 20, 22 and 24, spec and TAP), so a suite of placeholder files read as PASS (1 test). Only a top-level entry that is not a suite and has no subtests, naming a test file that exists and has no code in it (only comments, imports, requires or export {}), is subtracted, once per file in each run. So describe('utils.ts'), a test named index.js or after its own file, and a build line like ✔ dist/index.js (12ms) still count. When the test command runs from another directory (npm workspaces, cd pkg && node --test), the printed paths are resolved from the project root, so they are usually not found and nothing is subtracted: the count stays as node:test printed it.
  • Provenance: every proof report says when and on which code it ran (provenance: commit, uncommitted changes, time). senior finish --record keeps the report in .senior/evidence/<commit>.json as a durable record of what was verified; it stores no command output. Changes under .senior/ no longer count as changes to a project.

  • The guard catches equivalents of the commands it already guarded, listed in a bypass corpus test (tests/guard-bypass.test.ts) where each case either asks or is documented as out of scope with its reason. Applies to every host.

    • git options before the subcommand (git -C ../app push --force, git -c …, --git-dir, --no-pager) no longer hide it from the git rules. Their values are still judged, since they can run commands (-c alias.x='!…', -C "$(…)") or name the agent's own configuration (git -C ~/.claude/plugins/… rm).
    • Newly guarded: git push --mirror and --prune; git checkout/restore of a folder, . or a glob, and checkout -f/switch -f (a dot-folder such as .github counts as a folder; one file, such as git restore src/a.ts, stays free); git branch --delete --force, update-ref -d, reflog expire, gc --prune=now, git prune, worktree remove --force.
    • Deletion by other tools: rm with its options after the operands (rm src -rf), find -delete and -exec rm, rimraf (build folders stay free, as with rm -rf dist), PowerShell's abbreviated -r/-Rec and the ri/rm/del aliases (where a command starts, so grep del -r is a search), rsync --delete, and recursive deletes in python -c, node -e/-p and ruby -e one-liners.
    • Publishing and remote changes: npm unpublish/deprecate, cargo publish, twine upload, poetry/uv publish, gem push, docker push (--dry-run stays free); gh release delete, gh repo archive, gh api -X DELETE; aws s3 rm --recursive/rb/sync --delete and aws … delete-*/terminate-*, also after global options (aws --profile prod …), as kubectl -n prod delete now is; helm uninstall, heroku apps:destroy, pulumi up/destroy, az … delete; redis-cli FLUSHALL, MongoDB dropDatabase(); dd of=/dev/…, mkfs.
    • Secrets: printenv, env and Get-ChildItem env: on their own or piped (env | grep TOKEN), printenv or echoing a variable named like a secret ($GITHUB_TOKEN, $OPENAI_API_KEY).
    • In git, gh and glab, commit messages, titles and search patterns are read as text, so git commit -m "remove rimraf" no longer asks; a $(…) or backticks inside double quotes still does, and so does a message containing ; or &&. Other tools are not touched: in ssh host -t "…" or watch -t "…" the quoted word is the command that runs.
    • Out of scope, by decision: commands built at run time or decoded ($(echo rm), base64 -d | sh) need an OS sandbox, and everyday file operations (rm of named files, mv, writing a file) would make it ask too often to keep. git restore of a file without an extension (Makefile) also asks, since the text cannot tell it from a folder.
  • senior doctor shows each supported host on this machine and whether SeniorOS is set up in it: the host's command on PATH or folder, the plugin (yes, no, or unknown when the files that would say are missing or unreadable, never a guess; a Claude Code install scoped to another project is no), Codex hook trust (yes only with all three trusted), Antigravity hooks, and the file behind each answer. It also says whether the senior CLI is on PATH. Detection reads files only and never runs a host's binary. Cursor's plugin state shows as unknown unless a cached copy is found, since Cursor does not document where it records plugins.

  • senior setup prints the step left for each host found. senior setup --apply also does the steps the CLI owns: it adds senioros to the user's opencode.json (backed up once as .bak-senioros; a JSONC file, or one that is not a plain JSON object, is left alone and the step says to edit it by hand) and runs the Antigravity installer when Antigravity is present without SeniorOS's skills or hooks. Plugins for Claude Code, Codex and Cursor are installed in the app, so setup prints those commands. When a step could not be applied, setup --apply exits 1.

  • The guard asks before senior setup --apply, senior install --target antigravity and senior uninstall, also as npx senioros@<version> … or node …/index.js …: they write a host's hooks or plugin list, the same change as editing those files. Before, an agent could run the Antigravity installer without a prompt.

  • Proof beyond Node: senior finish and the finish gate now check Python, Go and Rust projects. Before, the finish gate stayed silent on any repository without package.json or .senior/config.json.

    • Python: pytest (continuing past import errors) or unittest, plus mypy and ruff when configured, in the project's .venv/venv or Poetry/Pipenv environment. Go: go vet, go test. Rust: cargo test (a workspace covers its members).
    • Projects are found at the root and up to two folders below it; a repository with several runs them all.
    • With uncommitted changes, only the projects the changes touch run; senior finish --all runs everything.
    • A missing tool or no collected tests is SKIP, never a pass. A Python run that fails only because declared dependencies are not installed is SKIP with that reason, not a failure blamed on the change; an undeclared or local module, or a test that ran and failed, stays FAIL.
    • finish --json reports source: "standard" when these checks are part of the baseline.
  • senior risk recognises payment and password/encryption code. Two path rules of 4 points each, so such a change is at least medium risk and gets the finish gate's review: payments (payment, billing, checkout, invoice, refund, chargeback, payout, Stripe, PayPal, Braintree, Adyen) and crypto (password, crypto, encrypt, decrypt, cipher, signing, signature, HMAC, bcrypt, argon2, scrypt, PBKDF2, KMS, AES, RSA). The path is split into words on separators, spaces, camelCase and acronyms, so processPayment.ts, APIPayment.ts and lib/HMAC.ts count and rehashing.ts, discharge.ts or battery/charge.ts do not. A password file the auth rule already counts is not counted twice. Before, a refund-permission change in src/billing/ was rated low.

Changed

  • senior-code-review: one fresh reviewer per change. After fixing its findings the agent proves each fix itself with a test that failed before and passes now; a second review is for a fix that changed the design. On Grok, chained reviews had sent 10 of 26 runs past 12 minutes.
  • The finish gate's risk review asks only for the two risks, their evidence and the ship status, without repeating the report, and it is skipped when a proven change's final reply already names a risk and a status. senior-debugging proves the regression in one shell command when the CLI is missing. The Claude Code session start leaves out the .senior/ and CLI lines when neither exists. On Opus the score held; the cost per run did not fall.

Fixed

  • The guard works in Grok, which loads the Claude Code plugin: it judges Grok's shell tool (run_terminal_command, Shell), and under Grok's always-approve, where a hook's question is approved by itself, it refuses the command instead and tells the agent to let the user run it. Before, git reset --hard ran without a question there.
  • The Claude Code session-start hook finds itself by its path suffix, like the other hooks; an exact comparison failed behind a symlink or a different drive-letter case and started the session without rules.
  • Bench: each Claude run gets its own config folder, removed when the run ends without blocking the other runs, and retried while Windows still holds its files; before, runs in parallel shared one folder, removed only at exit. A run that times out on Windows is stopped with the processes it started. The isolation check no longer refuses None. or No, … for not being exactly NONE, and names its reason: a leak is a run that names a heading of the user's CLAUDE.md, or a distinctive part of one.
  • The finish gate no longer lets a slow suite pass in silence. When senior finish does not finish within 270 s, the agent is sent back once per change set with NOT VERIFIED: do not say the checks pass, run the tests that cover the change, or report the status as NOT VERIFIED. Before, a timeout was cached as a pass and the agent could report checks that never completed. The message counts as sent only once the hook answers, so a hook the host killed still tells the agent on the next stop, and a risk review that was due follows it (as NOT PROVEN). In Cursor and Antigravity, all repositories share one 270 s budget, so a second slow one cannot push the hook past the host's 300 s limit, and Antigravity saves each repository's result as it goes. Applies to Claude Code, Codex, Cursor, OpenCode and the Antigravity hooks.
  • The Antigravity stop gate falls back to npx senioros finish when the CLI is not on PATH, as the other hosts' gate does. Before, it did nothing without the CLI. The fallback now runs only when the shell says the senior command is missing, not when a run fails with "not found" in its own output.
  • The guard follows cd in a chain: cd ~/.claude/plugins/seniorOS && git rm hooks/guard.mjs, cd ~/.claude && rm settings.json and pushd ~/.cursor; rm hooks.json ask, like the same commands with the full path, also with ./ or .., in a subshell, after -- or cmd's /d, with a redirection, and with Push-Location; popd goes back. Any write made from inside a plugin folder asks. Reads there (cat, ls, git log) stay free.
  • Cursor, multi-root workspaces: the finish gate's risk review no longer goes out in the same follow-up as another repository's failing checks. Failures are sent alone; the review is kept for its change set and sent once no repository fails, without running the checks again. loop_limit goes from 2 to 3, so a review after two fix rounds still has a follow-up left. Cursor's cap is what ends a long run of failures, since each fix that changes the code starts a new change set, so an agent that keeps failing can now get a third follow-up.

npm: npx senioros@0.9.0