rung

At which rung did your agent verify?

rung grades how real a verification was and who checked it, on two independent axes: a RUNG from 0 (reasoning about the code) to 4 (drove the real surface, before and after the change), and a CONTEXT from author to cross-lab. It ships a shared vocabulary, a reference schema, and a deterministic gate. Jump to what you need: Tutorial for a first verdict, How-to for task recipes, Reference for the exact fields and policy, Explanation for the two-axis model and the reasoning behind it.

Tutorial

Install

CLI, on your machine. The distribution is rung-ai on PyPI; the import package and the installed command are both rung.

pip install rung-ai

On a system Python that refuses a bare pip install with externally-managed-environment, install into an isolated environment with pipx: pipx install rung-ai. With uv: uv tool install rung-ai puts rung on your PATH, or run it once without installing via uvx --from rung-ai rung gate bundle.json. On macOS or Linux with Homebrew:

brew tap rung-dev/tap
brew install rung

Installed, rung is a single command on your PATH:

rung gate bundle.json          # gate an authored bundle
rung run --rung 3 --surface cli -- mytool --check
rung doctor                    # read-only preflight
rung version

Container. A pinned image is published to the container registry (GHCR) on each release; its exit code is the gate verdict, so it drops into any runner that can pull an image:

docker run --rm -v "$PWD:/w" -w /w ghcr.io/rung-dev/rung:0.2.0 gate bundle.json

GitHub Actions. Gate a bundle with no install step; see wiring it into CI for the uses: rung-dev/rung@v0.2.0 recipe.

No install. The runtime is stdlib-only, dependency-free, Python 3.9+ only. Run from a checkout with PYTHONPATH=src python3 -m rung.gate bundle.json (likewise -m rung.run, -m rung.cli), or vendor the single self-contained file src/rung/gate.py into your own repo. Every rung gate / rung run example below is the installed command; the module form is the drop-in equivalent.

Your first verdict

The worked-example bundles live in the repo, so git clone https://github.com/rung-dev/rung and gate one from the checkout:

rung gate cases/sync-connector-stdio-purity/bundle.json

It prints a JSON verdict and exits 0 (pass) or 30 (block); unreadable or malformed input exits 2. That is the whole surface. To author your own bundle instead of gating a shipped one, follow How to use rung.

How-to

How to use rung

Three steps, none needing a dependency beyond Python 3.9+.

1. Get the gate. pip install rung-ai (see Install), or, to keep the gate as an auditable single file, vendor a trusted, pinned copy of src/rung/gate.py into your repo. A standalone vendored gate.py has no bundled default policy beside it, so gate with an explicit --policy (vendor policy/default.json too, or point at your own); with no --policy a loose copy fails closed to exit 2 rather than guessing. Either way, do not run the subject repo's own copy (that is code execution as the judge, see the operator contract below).

2. Author a bundle. After you drive the change through the real surface (rung 3 or 4, not an import), write one evidence-bundle/v1 document. The smallest thing that runs is a single low-tier claim with no artifacts:

{
  "schema": "evidence-bundle/v1",
  "change": { "producer": { "lab": "your-lab" } },
  "claims": [
    { "id": "c1", "risk_tier": "low", "rung": 2, "context": "author", "verdict": "pass" }
  ]
}

Real bundles climb higher: rung 3 and 4 carry capture artifacts with their sha256 and, at rung 4, an S0/S1 (before/after) differential plus expected_delta. Rather than author those by hand (and hand-type the hashes), let rung run drive the surface and write the capture-backed bundle for you: rung run --rung 3 for a single-surface witness, rung run --rung 4 --diff for the S0/S1 differential (see Witnessing a run with rung run below). The cases/ bundles are working templates, and Enforced vs advisory fields is the field reference.

3. Run the gate, and wire it into CI.

rung gate bundle.json [policy.json]   # default policy if omitted

It exits 0 (pass), 30 (block), or 2 (unreadable or malformed input). In CI, fail the build on anything that is not exit 0. The bundled .github/workflows/ci.yml does exactly this. Pin your policy.json too: a structurally valid policy can still be toothless (see THREAT-MODEL.md).

In GitHub Actions you can skip the install step and gate a bundle with the action directly (it installs a pinned rung-ai and runs the gate; the job fails unless the gate passes):

- uses: rung-dev/rung@v0.2.0
  with:
    bundle: cases/sync-connector-stdio-purity/bundle.json
    # policy: policy/default.json   # optional; omit for the bundled default

A pinned container image (ghcr.io/rung-dev/rung) drops into any runner that can pull an image; its exit code is the gate verdict. See Install for the docker run invocation.

Witnessing a run with rung run

rung run (src/rung/run.py) is the first-class way to earn a rung-3 or rung-4 bundle; hand-authoring (above) is the fallback for evidence a tool cannot drive. Instead of driving the surface and then writing down a sha256, you let the tool drive it and write the bundle for you: an agent's cheapest path to a rung claim should be to actually run the surface, not to hand-type a hash for bytes that were never produced. It executes the probe directly (no shell), captures the exact bytes off the child's own stdout/stderr, hashes them into an evidence-bundle/v1, then runs the gate over that bundle and exits with the gate's verdict, never the probe's exit code.

# Drive a CLI surface, declare rung 3, let the tool witness + gate it.
rung run --rung 3 --surface cli --tier medium -- mytool --help

--rung and --surface are required: the tool witnesses bytes, never a rung, so it never mints one for you. A bare rung run -- <cmd> emits nothing and so can never satisfy a rung-3 policy; over-claiming (--rung 3 on an import) is a deliberate, logged act, not a silent default. Zero captured bytes at rung >= 3 on a process that ran to completion is refused ("nothing observed"); a process that hung is diagnosed as a timeout and blocked, not refused. What it does not prove: which program ran (a cat file emits bytes too) and whether that is the real subject surface. To keep that visible rather than laundered, the resolved path and sha256 of the launcher and of every file argument are recorded under surface.executed; surface authenticity stays judge-only.

Server surfaces. A correct persistent stdio server answers a request and then keeps its stream open, so a plain run always hits --timeout and blocks. Two flags treat answered-then-alive as a completed observation (capture the frames, then kill the still-running child, record no timeout):

• --expect-frames N: stop after N newline-terminated stdout frames. Use it for newline-delimited protocols (JSON-RPC / MCP over stdio).
• --until-idle [SECS]: stop once the probe produced output and then went quiet for SECS (bare flag = 2.0s). It is framing-agnostic; prefer it for non-newline framing such as LSP Content-Length, where --expect-frames would under-count and time out.

# MCP/JSON-RPC stdio server: one initialize frame, then it stays alive.
rung run --rung 3 --surface server --tier medium \
  --expect-frames 1 --stdin initialize.json --timeout 10 -- my-mcp-server

# LSP-style Content-Length framing (no trailing newline): idle-bounded instead.
rung run --rung 3 --surface server --tier medium \
  --until-idle 2 --stdin handshake.bin --timeout 10 -- my-lsp-server

hung-producing-nothing still times out and blocks under both. A capture that exceeds the 64 MiB cap is truncated and recorded as an undismissed capture-truncated blocker gap (flagged in the bundle, never silently dropped); RUNG_MAX_CAPTURE_BYTES tunes the cap (down for constrained environments, or up).

Rung 4: witnessing a differential with --diff. Rung 4 is a baseline vs candidate differential, so it needs two runs, not one; rung run refuses --rung 4 without --diff (a single run cannot earn it). With --diff, the probe argv is split on a bare ::: into an S0 (baseline) side and an S1 (changed) side; the tool drives each on the same bounded exec path, captures each off its own fds into exactly one s0_capture and one s1_capture, and emits a rung-4 bundle. The key part: the tool never asserts the delta. It only declares intent with --expect-delta, and the gate decides change-vs-invariance polarity from the captured bytes.

# Change claim: S0 and S1 must differ on the compared channel.
rung run --rung 4 --diff --surface cli --tier medium \
  -- mytool --old-flag ::: mytool --new-flag

# Invariance claim (refactor / dep bump / "no behavior change"): must match.
rung run --rung 4 --diff --expect-delta invariance --surface cli --tier medium \
  --s0-cwd ./before --s1-cwd ./after -- mytool run ::: mytool run

• --expect-delta change (default) or invariance: the polarity you claim. The gate blocks a change that produced identical bytes and an invariance that differed; declaring pass yourself grants nothing.
• --diff-channel stdout (default) | stderr | both: which captured channel the comparison uses. Use both for the strictest invariance check; a change confined to a channel you did not compare is not witnessed.
• --s0-cwd / --s1-cwd: run each side in its own working directory (e.g. a before/after checkout). Relative probe paths are resolved and hashed where the probe ran, not where the runner sits.

Determinism boundary (why --diff re-runs each side). A byte-level S0/S1 delta only proxies the claimed change when each side's output is deterministic: a timestamp, PID, or hash seed would make identical code read as a change, and an invariant refactor that happens to emit such noise would read as a delta. So rung run --diff runs each side twice and, if the compared channel is not byte-stable across the two runs, records a nondeterministic-output blocker gap and blocks (exit 30) rather than feeding an untrustworthy delta to the gate. It does not silently normalize the noise away; it refuses to certify it. Pin the environment (or point --diff-channel at a stable channel) and re-run. This is an author-context witness; whether the two sides are the real before/after surface, and independence, remain judge-only.

Operator contract for rung run (in addition to the gate's contract below; rung run is the more privileged tool because it executes the probe):

• Trusted code only. The gate is safe on untrusted input (it only hashes files); rung run executes its probe, so pointing it at an adversarial repo is arbitrary code execution. Keep it to trusted inputs. A sandboxed production recorder is a separate, privileged tool, not this one.
• Policy integrity is enforced by the tool. The policy is loaded, parsed, and hash-pinned before the probe runs, stamped into the bundle as policy_pin, and re-verified after the run: a probe that rewrites the policy file mid-run gets a block on the tamper, not a silently weakened gate. You still pin the policy itself (a structurally valid policy can be toothless).
• Only exit 0 is pass. rung run's exit is the gate verdict; treat both 30 (block) and 2 (usage / cannot-evaluate) as no-ship, exactly as for the gate.
• Redaction and env scrubbing. Captures can contain secrets; redacting before a bundle is published is an operator responsibility (the tool prints a reminder and does not scan). The probe inherits this process's environment by default, so a token in the operator env can surface in a capture; pass --env-clear to run the probe with a scrubbed, minimal environment (PATH, HOME, locale, TERM, TMPDIR).

Using rung with other tools

rung is deliberately narrow: it fixes the vocabulary and bundle format, and ships one deterministic check. What it leaves to other tools is the rest of the chain:

• the machinery that performs the drive (reaching the running system is what the upper rungs are about: a rung-3 or rung-4 claim does not exist without it); and
• the model-based judging that decides independence, which the gate can only check for, never perform.

Shipping neither is what keeps the gate dependency-free. So the practical flow is a chain: a producer drives the change and writes a bundle, rung grades how real that was, an independent judge attests, and rung re-checks the attestation. Two projects sit on either side of that chain:

• devloop produces the evidence. It runs an AI development loop (spec, plan, review, implement, verify) that ends in a verification of the running change; rung is the vocabulary and gate for saying how real that verification was (which rung it reached) and recording it as a portable bundle.
• syncade performs the independent judgment. It orchestrates blind, cross-judge review (isolated reviewers with no producer state, optional cross-model diversity) into one ship/no-ship verdict: the independence rung names on the CONTEXT axis but the deterministic gate can only check for, never perform. A reviewer at a different lab is the cross-lab independence the gate rewards, and what would stand behind a cross-lab attestation in the mostly-empty rung 3 to 4 × cross-lab cell.

These are examples of the layers on either side of rung, not a required stack. Any producer that emits an evidence-bundle/v1 and any judge that attests to one composes the same way: rung is only the interchange format and the deterministic gate between them.

Reference

What's here

schema/evidence-bundle-v1.schema.json   the portable per-claim record (JSON Schema, draft 2020-12)
policy/default.json                      declarative ship policy: min rung + independence per risk tier
src/rung/gate.py                         single-file, stdlib-only, dependency-free: gate(bundle, policy) -> verdict
src/rung/run.py                          `rung run`: drives a probe, captures its bytes, writes+gates the bundle
src/rung/cli.py                          the `rung` umbrella command (run / gate / check / doctor / version)
cases/                                   real, reproducible worked examples
skill/                                   an agent skill: how to use rung, plus a CLI and config reference
VERIFYING-RUNG.md                        rung applied to itself: an external blind review, then dogfooding

An evidence bundle records, per claim: the rung reached, the context, the surface driven, content-addressed artifacts, the S0/S1 differential (for rung 4), a verdict, and any cross-lab attestation. Gaps are listed in the bundle, not left out. evidence-bundle/v1 is the stable interchange: additive fields stay within v1, and a breaking change bumps the major (/v2).

The gate is a pure function of (bundle, policy). Its only disk I/O is reading the bundle and policy and re-hashing the artifacts they reference. It can only ever lower trust. A claim cannot pass above its own rung, and a producer cannot pass by declaring its own verdict: a declared pass grants nothing (the gate's own checks are the only thing that can pass a claim), while a declared fail or blocked still blocks. Exit 0 = pass, 30 = block, 2 = unreadable or malformed input.

rung gate cases/sync-connector-stdio-purity/bundle.json

The default policy

{
  "version": 1,
  "require_context": { "high": "cross-lab", "critical": "cross-lab" },
  "no_skip_tiers": ["high", "critical"],
  "allow_dismiss_gaps": false,
  "min_rung": { "low": 2, "medium": 3, "high": 4, "critical": 4 }
}

The policy is plain JSON: same format and stdlib parser as the bundles, no third-party dependency and no Python 3.11 floor. min_rung maps each risk tier to the minimum rung to ship, and require_context names the tiers where independence is mandatory; kept consistent, they close the self-report trap (a self-reported rung 4 blocks at high/critical until a cross-lab reviewer attests). The gate fails closed on an unknown or missing key rather than shipping with a disabled check. See policy/README.md for the full field reference, per-tier calibration rationale, and the self-report trap detail.

Enforced vs advisory fields

A schema-valid bundle is not necessarily gate-passing. The schema admits many fields for humans and tooling; the gate only reads a subset when it decides a verdict. Authors should know which is which.

Enforced (read by the gate; affect the verdict):

• Top-level: schema (must equal "evidence-bundle/v1"), change.producer.lab, claims (non-empty array).
• Per claim: risk_tier, rung, context, verdict, expected_delta, artifacts[] with each artifact's role/uri/sha256, differential.s0_observed/s1_observed (cross-checked against capture bytes at rung 4), attestation.lab/attestation.verdict (required when the policy demands cross-lab for the tier).
• Rung 4: needs exactly one s0_capture and one s1_capture artifact (zero, duplicate, or padded captures per role block); polarity (change vs invariance) is decided from that single verified pair of capture bytes.
• Gaps: severity, dismissed (an undismissed blocker gap blocks unless policy allows dismissal).

Advisory (in the schema for humans; the gate does not check them):

• change.repo/s0/s1/diff_range/created_at/policy_ref, producer.agent/model.
• claim.claim, claim.surface.*, claim.how_established.
• artifact.media/summary, differential.probe/observed_delta, attestation.judge_id/note, gap.desc/why_unverified.
• Note: id and gap.desc appear in the gate's human-readable reason output but are not enforcement inputs.

Conditional requirements the gate enforces beyond the schema: rung >= 3 requires >= 1 artifact; rung 4 requires exactly one s0_capture and one s1_capture plus a differential with byte-verified polarity; a cross-lab tier requires a matching attestation.

Explanation

The problem

"Verified" gets used for two different things, and most reports use the one word for both. "The tests pass" reads the same as "I ran the actual thing and watched it work." "I checked it" reads the same as "someone independent checked it." Those aren't the same claim. When they all sound alike, a change nobody ran can pass for verified, and there's no shared way to say which one you have.

Why "rung"

A rung is a step on a ladder. The core axis (0 to 4) is a ladder from reasoning about the code up to driving the running surface (the real CLI, server, library API, or GUI a user or program touches) and capturing what it did. How far you climbed is how real your verification is, and you don't get to claim the top while standing on the first. The name keeps the question in front of you: which rung did you actually reach?

Two axes: how real, and who checked

Pull them apart:

RUNG: how real (0 to 4)

Rung	Meaning
0	Read-only reasoning about the code
1	Import the unit and call it
2	Test suite green
3	Drove the real surface and observed it
4	Drove the real surface before and after the change, and compared the two runs (S0 vs S1) to confirm the difference matches the change

A rung-4 claim comes in two directions. A change claim expects the before and after to differ; an invariance claim (a refactor, a dependency bump, "nothing changed") expects them to match.

CONTEXT: who evaluated

Context	Meaning
author	The producer of the change
fresh-blind	An independent reviewer with no producer state
cross-lab	An independent reviewer at a different lab

Put them on a grid, and the empty cell is the one to look at:

rung ↓ · context →	author	fresh-blind	cross-lab
2 tests green	generic CI
3 drove the real surface	runtime-verification tools
4 drove + S0/S1 differential			← real and independent; what rung targets

The axes are independent: "drove it blind, cross-lab" is not a higher rung, it is a different cell (rung 3 to 4 × cross-lab). Generic CI sits at rung 2 × author, and runtime-verification tools reach rung 3 × author: they drive the real surface, but the producer still grades its own work. The right-hand column, real verification done by someone other than the producer, is where almost nothing lives today, and that is the column rung is built to name and reward.

The gate can check only one context, cross-lab, and only for presence, not authenticity: it requires the claim to declare context: cross-lab and the bundle to carry an attestation whose lab differs from the producer's and whose verdict is pass. It does not verify that the attestation is authentic (nothing is signed in v1; see THREAT-MODEL.md). author vs fresh-blind is the producer's word, which nothing can check; the gate treats both as "not independent."

When you need the gate

The vocabulary stands on its own. The deterministic gate earns its keep when you can't take the producer's word for it:

• Machine-made claims at volume. When agents emit "verified" by the hundreds and nobody reads each one, you want a fail-closed check that can say "you claimed rung 4 but there's no S0/S1 differential" and mean it. That is the case rung was built for.
• Producers who inflate a claim. A checker matters when the party making the claim benefits from overstating it. A declared pass grants nothing, a rung-4 claim with no differential is caught, and a change claim whose bytes don't differ blocks. That raises the cost of a bogus claim to fabricating consistent capture bytes, which v1 does not detect (see THREAT-MODEL.md); it also catches an innocent mislabel, so it's not only for bad actors.
• Automated gating. If ship/no-ship must block a merge, you need a machine-readable verdict, and vocabulary alone cannot fail a build.

Worked examples

Three real, reproducible cases, each driven at a different surface kind and every bundle re-checkable by the gate in this repo:

• cases/sync-connector-stdio-purity/: server (stdio), change polarity. A protocol server whose first stdout line was a logging banner instead of a protocol frame. Rungs 0 to 2 all pass it; rung 3 catches it by reading byte one off the real stdio surface; rung 4 shows the S0/S1 differential. Carries a declared gap: the auth-gated, data-mutating ops were not driven.
• cases/ctl-usage-error-doubleprint/: CLI, one commit that exercises both polarities. Human-mode stderr changes (a usage error printed 3× -> 1×); the --json machine channel is invariant (byte-identical S0 vs S1, exit 2 both). The invariance claim is the reason expected_delta exists: a change-only rung-4 gate would wrongly reject perfectly good evidence for it.
• cases/ical-text-escaping-rfc5545/: library boundary, change polarity. RFC 5545 TEXT escaping in a calendar export library, driven through the public generate() API. Flags up front that the GUI export button (the surface a user taps) was not driven.

Each case README shows the exact gate invocations, including the high-tier block on a self-reported rung-4 claim.

Threat model and limitations

The gate is a deterministic function of (bundle, policy) whose only disk I/O is reading the bundle and policy and re-hashing the artifacts they reference. In short: it can only ever lower trust, it detects post-bundle mutation but not fabrication, and a handful of properties (risk_tier, author/fresh-blind context, attestation authenticity, gate substitution) are trusted on assertion in v1, by design, until signing lands.

The full model, what the gate enforces, what rung run enforces, what is trusted on assertion, the distribution/supply-chain boundary, the operator contract, and the v2 signing direction, lives in THREAT-MODEL.md. For the layered verification rung itself has been through (an external blind review, and dogfooding rung on its own packaging), see VERIFYING-RUNG.md.

Prior art

The two-axis split is not new; rung names it and makes the cell checkable. GRADE / EBM already separate the quality of evidence from the strength of a recommendation, which is the model for splitting rung from policy: the RUNG axis is a software analogue of GRADE's evidence-quality tiers. That who evaluated is orthogonal to how well comes from DO-178C / SIL (independence of verification from development) and chain-of-custody (the artifact trail), which is where the CONTEXT axis originates. The test pyramid is the folk version of the RUNG axis for the lower rungs. rung's contribution is the two-axis vocabulary and a portable, checkable evidence bundle that names the cell, especially the rung 3 to 4 × cross-lab cell that runtime-verification tools and eval harnesses, running in author context, do not fill.

License

Apache 2.0. See LICENSE.