Skip to content

Releases: B0yko/obey

obey 0.1.1

Choose a tag to compare

@github-actions github-actions released this 24 Oct 11:37

Documentation and packaging only; the CLI, the checkers and the hooks behave as in 0.1.0.

  • The README opens with what obey is for and how to try it, installs from PyPI (uvx --from obey-cli obey demo),
    and uses absolute image and link URLs, so it also renders on PyPI.
  • A new social preview, built from the study's numbers by eval/make_social_preview.py.
  • Release assets are only the wheel and the sdist; study charts are written with stable element ids, so rebuilding
    them changes no bytes.

obey 0.1.0

Choose a tag to compare

@github-actions github-actions released this 24 Oct 04:16

First release of obey: unit tests for AGENTS.md.

obey turns the rules in AGENTS.md, CLAUDE.md, GEMINI.md, Cursor rules and Copilot instructions into deterministic
checks, runs your coding agent headless on small tempting tasks in a sandbox, measures per-rule compliance from what
the agent did, and compiles the rules it breaks into hooks for Claude Code, Codex CLI, Cursor, Gemini CLI, Copilot
CLI and OpenCode, plus a git pre-commit check.

uvx --from git+https://github.com/B0yko/obey@v0.1.0 obey demo     # no key, no Docker
uvx --from git+https://github.com/B0yko/obey@v0.1.0 obey init     # in your repository

The study

  • With the rule in AGENTS.md, Claude Haiku 5.5 in Claude Code and OpenCode followed it in all 1,474 exercised
    tempting-task runs; without it, 81% (stratified difference +17.6 points [95% CI 4.9–34.1]).
  • Phrasing (imperative, prohibition, with a reason) made no measurable difference: every phrasing was at 100%.
  • Hooks without the rule raised compliance from 34% to 100% for the rules agents broke; 0 of 123 blocked runs broke
    the rule another way.
  • 600 public AGENTS.md files: median 7 rules per file, 23% of rules mechanically checkable.

These are probe compliance numbers, after correcting 136 checker false positives found by reading the runs (94%
before the correction). The pre-registration, every deviation and both analyses are in
study/REPORT.md and
study/DEVIATIONS.md.

Assets

  • obey-v0.1.0-raw-runs.tar.gz: the raw records of all 2,663 study runs (events, diffs, commits, hook logs, usage),
    for eval/rescore.py and eval/audit_obeyed.py. Scanned for secrets and personal paths before upload.

The sandbox image is built locally with obey doctor --build-image; it is not published because it contains
third-party agent CLIs.

What's in it

  • obey extract: rules from AGENTS.md (nested), CLAUDE.md with imports, GEMINI.md, Cursor .mdc rules and Copilot
    instruction files, mapped to twelve deterministic checker types; optional LLM proposals that are never accepted
    automatically.
  • obey probe: a tempting task per rule from templates filled with repository facts.
  • obey run: Claude Code, Codex CLI, OpenCode, Copilot CLI and Gemini CLI run headless in a container behind an
    allow-listing egress proxy and a key-holding model gateway; per-rule compliance with Wilson intervals, a no-rule
    baseline, budgets, --estimate, --compare with evasion counts, and --unsafe-local.
  • obey enforce: hooks for Claude Code, Codex CLI, Cursor, Gemini CLI, Copilot CLI and OpenCode plus a git pre-commit
    check, all calling one standard-library rule engine; idempotent, JSON-preserving merges with --dry-run and
    --remove.
  • obey lint, the GitHub Action, .pre-commit-hooks.yaml, obey demo, obey doctor, obey init.
  • The study: a survey of 600 public AGENTS.md files, a pre-registered experiment on rule phrasing, and a before/after
    measurement of hooks with evasions (study/REPORT.md).