Skip to content

Releases: KenSuh/open-agent-harness

oah v0.3.0

Choose a tag to compare

@KenSuh KenSuh released this 04 Nov 04:11

oah (open-agent-harness) is a Python 3.11+ standard-library CLI for macOS and native Windows. It is a guideline for a
harness that keeps adapting as models and skills change. Out of the box it makes Claude Code and Codex CLI work as one team
on a repository: Claude Code writes the first version, and Codex CLI reviews, corrects and re-reviews. You talk to your AI in
plain words; the rules oah init installs make it run the harness, and you answer when it stops for a decision and confirm
before anything lands.

v0.3.0 is about choosing models from evidence and showing how they were chosen.

Install

  • macOS: download install.sh from this release and run bash install.sh, or clone the repository and run
    bash install/install.sh.
  • Windows (PowerShell): download install.ps1 from this release and run
    powershell -ExecutionPolicy Bypass -File .\install.ps1 (the policy applies to that run only), or clone the
    repository and run powershell -ExecutionPolicy Bypass -File .\install\install.ps1.

This release attaches both installers with their pins filled in, the source archive as .tar.gz and .zip, and
SHA256SUMS for all four files. The installers download this tag's archive and refuse to unpack it unless its SHA-256
matches the digest pinned in the installer. You can instead download the archive and
SHA256SUMS yourself, check the digest and run the included installer, as docs/install-macos.md and
docs/install-windows.md describe. Then run oah doctor and oah init in your project.

What is new

  • Benchmark scores pick the model. The task's difficulty (its tier) and [models] profile (performance,
    balanced or efficient) set the score a task needs. balanced takes the cheapest model that reaches it, efficient
    the best score for its price, and performance the top-scoring model for the hardest work. Scores come from Epoch AI
    without a key, or from Artificial Analysis with your own key. A newer model joins once it has a score, a retired one
    drops out, and OAH_MODEL_CHANGED says what moved. Pins, tier_pools, allowed_models and effort_caps keep their
    meaning. See "How a model is chosen" in docs/concepts.md.
  • TypeSafe Jev judges difficulty. With Jev on, it reads the task's Intent and Scope paths and judges how hard the
    task is. At or above [rerank] difficulty_min_confidence its tier runs, never below the floor, and the run record
    keeps the rule tier beside it. With Jev off, failing or unsure, the rule tier runs, and a Jev failure never blocks a
    call. A review never runs below its risk floor, whatever a flag, packet, profile or judgement says.
  • Your tools can stay current. After the last lease finishes, oah can update Claude Code, Codex and Claude Code
    plugins with each tool's own update command, and the model list is read again on the next call. oah init asks once,
    and you can turn it off. Codex plugins and MCP servers are not updated; the oah skills keep their daily check.
  • See how models were chosen. oah models report counts, from this repository's run records, how each run's tier
    and model were chosen. oah models export prints one JSON line per run with only the decision data and the outcome;
    no prompt, path, command or task name leaves. Both only read.
  • The Stop hook can judge only the agents you name. [gates] stop_agents lists the lease agents whose gates the
    Claude Code Stop hook checks; with the default empty list nothing changes.
  • oah land removes its private object folder reliably. The folder goes when the landing context is collected,
    even inside a reference cycle, with the same permission handling as before, so no ResourceWarning can surface at
    an unrelated moment where warnings are shown, such as a test run.
  • A README anyone can read. The README leads with what oah does for you: the impact map, the wiki, leases, gates,
    the build-and-review loop and Jev, in plain words. Every diagram is redrawn with the diagram-design skill, the banner
    shows the ask-and-confirm loop, the follow-along example reads as a conversation, and the design plan is now in
    Korean too (docs/ko/PLAN.md).

Upgrading from 0.2.0

Update oah itself and then run oah rules sync in each project, as sections 1 and 2 of docs/upgrade.md describe.
The sync brings in the rule that has your AI judge a design's difficulty first; without it, oah packet new warns
OAH_DESIGN_UNJUDGED. oah does not edit your config. The full list is in
"6. Models" in docs/upgrade.md. What changes:

  • Codex's default flag and the model in your Codex configuration no longer choose. To keep one model, pin it with
    [engines.<engine>.tiers] <tier> = { model = "<model>" } in .oah/config.toml.
  • The old default quality_order = ["opus", "sonnet", "haiku"] counts as unset, so scores choose
    (OAH_MODEL_ORDER_IGNORED). Pins, tier_pools, your own quality_order, allowed_models and effort_caps keep
    their meaning.
  • At a catalog rebuild oah downloads Epoch AI's public benchmark file from epoch.ai. The request sends nothing about
    your project. [models] fetch_scores = false stops it, and the snapshot shipped with oah then chooses.
  • With Jev already on, Jev now receives the task's Intent text and Scope paths, up to 16,000 characters, where it used
    to receive only the task name; never code, diffs or file contents. The first such request in a project prints
    OAH_RERANK_SCOPE_WIDENED once. enabled = false under [rerank] stops it.
  • Tool updates stay off in existing projects: a .oah/config.toml without after_finish under [updates] counts as
    off, and oah doctor mentions the setting once.

What can stop a launch after you update:

  • A review below its floor refuses with OAH_MODEL_BELOW_FLOOR, for example with --tier low, and a narrow
    allowed_models no longer lowers a floor: a review stops with OAH_MODEL_BELOW_FLOOR and any other call with
    OAH_MODEL_UNRESOLVED when no allowed model reaches it.
  • With no model list, or an unreadable claude --help, automatic choice stops with OAH_MODEL_CATALOG_UNAVAILABLE, and
    with no usable scores with OAH_MODEL_NO_SCORES. A pin gets past both.

Security

No API key, token, password, cookie, OAuth secret or account identifier enters the repository, its history,
releases, issues or CI logs. Keys are read only at runtime, from environment variables or the OS keychain. When Jev
judges difficulty it receives the Intent text and Scope paths, never code, diffs, file contents or keys. The Epoch AI
download at a catalog rebuild sends nothing about your project; [models] fetch_scores = false turns it off. Report
vulnerabilities privately through the repository's Security tab.

Known limits

  • Linux is not supported yet.
  • TypeSafe Jev needs your own TypeSafe key, and TypeSafe may charge for each request. Without it, the rules decide.
  • No benefit from Jev has been measured. oah models report and oah models export show how choices were made, not
    whether Jev helped.
  • Public benchmark scores measure a model's general performance, not its performance on your repository.
  • The diagrams were checked in macOS previews; Windows and Linux fonts may wrap a few labels differently.
  • The design conversation (say what you want to build, and oah asks one design question at a time) is planned for
    v0.4.0 and is not in this release.

oah v0.2.0

Choose a tag to compare

@KenSuh KenSuh released this 01 Nov 21:56

oah (open-agent-harness) is a Python 3.11+ standard-library CLI for macOS and native Windows. It is a guideline for a
harness that keeps adapting as models and skills change. Out of the box it makes Claude Code and Codex CLI work as one team
on a repository: Claude Code writes the first version, and Codex CLI reviews, corrects and re-reviews.

v0.2.0 is about reviews and evidence you can hand off.

Install

  • macOS: download install.sh from this release and run bash install.sh. You can also clone the repository and
    run bash install/install.sh.
  • Windows (PowerShell): download install.ps1 from this release and run it. You can also clone the repository and
    run install\install.ps1.

The release installers download the source archive of this tag. They refuse to unpack it unless its SHA-256
matches the digest pinned in the installer (also listed in SHA256SUMS). Then run oah doctor and oah init in
your project. See docs/install-macos.md and docs/install-windows.md. To upgrade, see docs/upgrade.md.

What is new

  • Jev can choose the model and effort within pools you allow. [engines.<engine>.tier_pools] in .oah/config.toml
    lists the models you accept for each tier, and Jev picks only from them. Without a key, or when Jev is unused or its
    answer is rejected, the rule-based choice runs, and pools do not limit that choice. See "Routing and difficulty" in
    docs/concepts.md.
  • Models outside your pools are reported. With pools set, when a CLI lists a model that is in no pool, oah models
    and a launch that rebuilt the catalog say so with OAH_MODEL_NEW_UNPOOLED and the one manual step. oah doctor shows
    the same list from the stored catalog. oah never edits your pools.
  • One stop and budget gate covers the whole task. oah call, oah fix and oah review pass the same gate before
    every paid engine call. Rounds count task-wide and the recorded limit survives restarts. Unreadable or structurally invalid
    saved state refuses and is left unchanged.
  • A not-converged review says what is left. oah review summary adds a handoff block: the last reviewed head, P0 and
    P1 findings, gates not met and a suggested next command.
  • Impact reports state their limits. oah impact reports which layers it analysed and which it did not. An empty
    result says nothing was found in the analysed scope, which is not proof of no impact. An incomplete analysis adds
    conservative fallback checks.
  • Real cases can be replayed. oah eval freezes a plan, runs variants in isolated workspaces with the expected
    answers hidden, grades by explicit rules and reports with its limits. It is not a release gate and never claims one
    variant is better in general. See docs/eval.md.
  • Codex effort can be verified on exact versions. Shipped rules cover Codex CLI 0.164.0 and 0.166.0 with
    gpt-6.1-sol and gpt-6.1-terra, at low, medium, high, xhigh and max.

Security

No API key, token, password, cookie, OAuth secret or account identifier enters the repository, its history,
releases, issues or CI logs. Keys are read only at runtime, from environment variables or the OS keychain. Report
vulnerabilities privately through the repository's Security tab.

Known limits

  • Linux is not supported yet.
  • Effort verification says the mapping is expected from public CLI source and the header matched. It is not proof of
    what the backend computed. Any other CLI version or model stays unverified, including newer Codex CLI versions, until
    a rule ships for them.
  • The first catalog rebuild with pools configured reports every unpooled model, not only new ones.
  • Pools limit Jev, not the rule-based choice. With no tier pin and an empty quality_order, the rule-based choice follows
    the model Codex marks as default, so a new default can run even if it is in no pool. Pin the tier's model to prevent
    this.

oah v0.1.0

Choose a tag to compare

@KenSuh KenSuh released this 28 Oct 23:36

oah (open-agent-harness) is a Python 3.11+ standard-library CLI for macOS and native Windows. It is a
guideline for a harness that keeps adapting as models and skills change. Out of the box it makes Claude Code and
Codex CLI work as one team on a repository: Claude Code writes the first version, and Codex CLI reviews, corrects
and re-reviews. oah keeps the work honest with leases, intents, gates and receipts.

Why this split: in the author's own use, Claude is good at filling in what a beginner missed while implementing,
and Codex is good at checking and correcting within a fixed scope. That is a personal experience, not a measured
claim. The harness parts work with one AI or several, fitted to your own setup.

This is the first release. It is meant for developers who are comfortable with git and pull requests.

Install

  • macOS: download install.sh from this release and run bash install.sh. You can also clone the repository and
    run bash install/install.sh.
  • Windows (PowerShell): download install.ps1 from this release and run it. You can also clone the repository and
    run install\install.ps1.

The release installers download the source archive of this tag. They refuse to unpack it unless its SHA-256
matches the digest pinned in the installer (also listed in SHA256SUMS). Then run oah doctor and oah init in
your project. See docs/install-macos.md and docs/install-windows.md.

What is in it

  • Guided setup: one-command installers, oah doctor, and oah init. oah init shows its whole plan first,
    asks before installing tools and never stores existing file contents.
  • Leases and intents: oah start/renew/status/finish/release/log keep parallel sessions apart. oah packet,
    oah intent and oah preflight bind each task to accepted acceptance criteria.
  • Routing and engines: oah call picks the engine, model and effort per stage and difficulty.
    oah recommend and oah models show the tiers. There is an optional Orca launcher.
  • Gates: oah gates and oah check handle acceptance gates with evidence. There is a finish guard and opt-in
    Claude Code and git hooks (oah hooks).
  • Review loop:
    • oah review runs Codex review rounds with receipts and repeat checks.
    • oah fix runs one recorded correction.
    • oah land checks that a task's exact reviewed head may land. CI must be verified unless an explicit recorded
      exception exists.
  • Knowledge:
    • oah impact lists what to re-check when files or DB objects change.
    • oah wiki keeps a project wiki.
    • oah rules keeps a shared rules block.
    • oah graph uses graphify when it is installed.
  • Toolkit: oah skills covers vendored and linked skills with provenance, safe updates of the kit cache, and
    MCP server templates with exact pins. oah ci gives you a CI starter for your own project.
  • Docs: README, concepts, a worked example (examples/hello-oah), troubleshooting, upgrade, toolkit and key
    guides in English and Korean, plus contributing, security and conduct files.

Security

No API key, token, password, cookie, OAuth secret or account identifier enters the repository, its history,
releases, issues or CI logs. Keys are read only at runtime, from environment variables or the OS keychain. Report
vulnerabilities privately through the repository's Security tab.

Known limits

  • Linux is not supported yet.
  • Effort verification: a run's reasoning effort is recorded as unverified unless a shipped rule proves the
    mapping. This release ships no such rule, so effort stays unverified. Evidence-backed rules for exact Codex CLI
    versions are planned for a later release.