Harnessie 1.5.0: Harbor and OpenEnv interoperability

Harnessie 1.5.0: Harbor and OpenEnv interoperability

Prepared for publication on 2026-10-06 from the signed tag v1.5.0. Package version is 1.5.0. The GitHub Release, the PyPI distributions and the downstream pins follow the release sequence in RELEASE_CHECKLIST.md; the exact commit, artifact digests and publication evidence are recorded in audits/release-1.5.0.md as each stage completes. Nothing in this document asserts that a publication stage has completed unless that audit records it.

Harnessie 1.5.0 makes a Harnessie result identifiable and reconcilable from outside the harness. Every verify report, run report and event log names the harness that produced it. The registry refusals a brain triggers by speaking another harness's tool vocabulary are counted as tool-contract breaks and published per brain. A run's hash-chained event log exports as an ATIF-v1.7 trajectory that an independent capture can be checked against. Two examples put Harnessie on either side of a Harbor trial, as the verifier and as the external agent, without changes to either project.

Included

• Harness identity on every result (harness/identity.py). harnessie verify reports and harnessie report print the installed harness version, the inward-manifest digest (or absent) and a digest of the registered tool surface beside the brain's bundle identity. The standalone verifier and the workflow runner emit the same fields as one harness_identity event, and the ATIF exporter carries them under agent.extra.
• Tool-contract break metrics. harness/trace_eval.py adds refusals_by_error, tool_contract_breaks (refusals with action_unsupported, malformed_arguments or bad_arguments), completed_tasks and tool_calls_per_completed_task (null when no loop completed). Policy refusals such as consent, role and approval are not counted. Loop eval scenarios accept expect_trace, and evals/tool-contract.yaml proves one foreign tool call is recoverable, a repeated one ends stuck, malformed arguments count, and a consent refusal does not.
• The tool_contract live scorecard row and the first measured brains. docs/brains.md gains the per-brain table, filled from a Sam-approved local-only run on 2026-10-05 recorded in audits/live-scorecard-2026-10-05/: qwen3.8:latest and gpt-oss:20b completed with zero contract breaks, qwen3.8:latest regressed under placeholder substitution, and granite4.1-guardian:8b made no tool calls so its break count is not measurable.
• harnessie atif RUN_ID_OR_DIR [--out PATH] [--force] exports a run's or verify report's event log as an ATIF-v1.7 trajectory.json, Harbor's trajectory format. It makes no model calls and refuses a broken hash chain, a loop without loop_finished, a step gap, or an existing output without --force. Fields the log does not hold are reported as absent. To make the record reconcilable, model_turn events now also carry input_tokens, output_tokens, tool_call_ids, model, provider and effort, and tool_result events carry call_id; the existing tokens total is unchanged. The export acceptance records a Harbor-valid trajectory whose call count matched OpenEnv's capture proxy.
• examples/harbor-verifier/: Harnessie as a Harbor verifier in both shapes Harbor supports. harnessie_verifier.py is the host-side BaseVerifier that runs harnessie verify --no-verifier under Harnessie's own OS sandbox; tasks/calc-add-mean/tests/test.sh is the in-sandbox form. Exit 0 maps to reward 1, exit 1 to reward 0, and exit 2 to no reward file. Only deterministic checks reach the reward. The verifier acceptance records four Harbor 0.24.0 trials on the Docker backend.
• examples/harbor-agent/: Harnessie as a Harbor external agent. harbor_tools.py registers read_file, list_files, write_file, run_shell and task_complete on an ordinary ToolRegistry with the builtin worker policy unchanged, with effects landing in the Harbor environment; harnessie_harbor_agent.py is the BaseAgent wrapper that writes events.jsonl, an ATIF trajectory.json and harnessie-result.json into the trial and turns Harbor's agent timeout into a recorded model_error stop through CancellableModel. The agent acceptance records three trials, one with Harnessie as both agent and verifier. The calc-add-mean task's agent timeout is now 1,200 s.
• Maintenance. The 2026-10-05 dependency-lock-refresh proposal is applied to the runtime, development and release locks: urllib3 2.7.0 to 2.8.0 (resolving GHSA-8988-9cw3-xx77, GHSA-vxq7-64xx-v4gw and GHSA-gh4c-6fx4-qh6g on requirements/dev.txt and requirements/release.txt) plus build 1.6.1, charset-normalizer 3.5.2, cryptography 50.0.2, fqdn 1.6.0, hypothesis 6.168.4, idna 3.20, pyparsing 3.3.3, pyproject-hooks 1.3.3, rpds-py 2026.9.1 and tzdata 2026.5. Public dependency ranges in pyproject.toml are unchanged. .github/dependabot.yml now documents that exclude-paths governs version updates only. All four CodeQL Action subactions are aligned to the immutable 4.38.2 commit.
• Fixes. The default local tier points at qwen3.8:latest, the model actually installed and already pinned by the pilot; the previous default was no longer installed, so a stock local run failed before any call. Two pilot tests are portable to Linux, where the hosted jobs had failed with Argument list too long; production code is unchanged.
• Pilot controls, source only. The experimental Claude pilot enforces its configured output-token threshold against aggregate reported output, exposes each delegate's actual aggregate output budget in reviewer notices, and adds a separately approved, consume-once local Qwen smoke driver. These are excluded from distributions and do not cap already-generated provider usage.
• Planning records. ROADMAP.md gains the Harbor and OpenEnv interoperability candidates section and marks which candidates shipped here; the controlled-review live pilot is recorded as the next active tranche; human accessibility acceptance moves to the nonblocking roadmap.

Authority and evidence limits

The ATIF export is derived from the audit stream and invents nothing. Model messages and system prompts are absent (message is empty with extra.message_recorded: false), tool arguments are absent (arguments_recorded: false), and tool output is the 300-character excerpt the log keeps. The exporter reports those gaps rather than reconstructing them. A schema-valid trajectory whose call count matches a capture is evidence that the harness's record and the wire agree on structure; it is not evidence that the model behaved well, and it is not token-level agreement.

The harness identity names the installed version, the inward-manifest digest and a digest over the registered tool surface. These are reported identities. They do not authenticate who ran the harness, prove the source was unmodified beyond what the inward manifest covers, or establish that a result was produced independently. The tool set digest covers names, schemas, effects and grants, not implementations, so the remote Harbor tool surface and the builtin registry have different digests under the same policy.

Tool-contract counts are per bundle: one brain, one endpoint, one prompt hash, one parser version, one sampling setting, one short tool-using task. The only measured brains so far ran on a local Ollama endpoint; no cloud provider was reachable in that run and both cloud targets report SKIP. A brain that makes no tool calls has no measurable break count, and a safety classifier's passes on no-action rows are not evidence about the small-model floor. Any change to model, endpoint, prompts, parser or sampling requires a new run; the numbers do not carry over.

The Harbor examples are contract and plumbing results on one backend (Docker through Colima), two tasks, and one local model. They establish that Harnessie's verdict contract maps onto Harbor's reward contract and that Harnessie's policy enforcement survives the move to a remote sandbox. They are not evidence about any brain's quality. The in-sandbox verifier form failed closed because the container runtime denies bubblewrap its namespaces; that is the harness working as specified, not a scoring result. The host-side agent runs with the operator's authority, as any Harbor external agent does; Harnessie's own OS sandbox is not engaged for it because nothing executes on the host.

Cannot-verify is unscorable, never zero. A harnessie verify exit 2 produces no reward file. Harbor carries that outcome two ways, a completed trial with no score from a custom verifier or an errored trial from a missing reward file, and neither is a zero. Writing a zero would teach a policy that an unverifiable result is a failure to avoid. Verifier model verdicts are never used as a training signal in these examples.

Verification status

The local release gates (pytest, harness.cli eval, verify-manifest, verify-inward-manifest, validate, build_docs_html.py --check, ecosystem_status.py --validate, git diff --check and scripts/release_gate.py) are run against the exact release commit and their counts recorded in audits/release-1.5.0.md. Test and eval counts are dated observations, not contracts; this document does not restate them.

Four dated acceptance records carry the 1.5.0 feature evidence: ATIF export acceptance, Harbor verifier acceptance, Harbor agent acceptance and the live scorecard. Each names the Harbor, OpenEnv, backend, model and Harnessie versions it ran under; a different version of any of them requires a new run.

The 1.4.1 guide earned hosted GuideCheck Level 4 under profile 2.0.0 for its frozen bytes. That receipt covers only those bytes. Any 1.5.0 guide change requires a new sidecar, trust-bundle pins, independent anchor and hosted receipt before a level is claimed. Guide conformance does not establish software safety or accessibility conformance.

Deferred work and downstreams

Verify Action 0.2.2 and stable v0 pin core 1.4.1 at 9f18d70017f395ef4d15ac5746e30ac633f00f8e. Homebrew 1.4.1 is published at tap merge 953760f3968200f99658bfb068609532cb9fbf4d. Both are separately propagated downstreams and are updated after the 1.5.0 PyPI distributions are published and verified; until then they continue to serve 1.4.1. Engine wrappers remain on their independent 0.1.0 release train.

Token-level ATIF agreement, the full validated-harness bar OpenEnv applies, needs the capture proxy in front of a token-returning engine such as vLLM or SGLang. Local Ollama yields evaluation-only rollouts, which is enough for call-count agreement and for every other shipped candidate. That remains a GPU-host decision.

The in-sandbox Harbor verifier form could score if the task image ran with the capabilities bubblewrap needs, or if Harnessie gained a backend that confines without namespaces. Neither was attempted. Record-mode tool aliases, the containment note for proxied endpoints and the training-surface scope note remain proposed candidates in ROADMAP.md; the two marked contested need human-arbitrated decisions.

The controlled-review pilot's local Qwen queue continues as recorded in NEXT.md; its consumed attempts and approvals are unchanged by this release. Human accessibility acceptance remains a deferred nonblocking roadmap item; no full-conformance claim is made. Live review panels, bidding, commentary, follow mode, automatic runner observation and bid-driven model selection remain an evidence-gated capability sequence.