Repository navigation
greenwash 0.5.1
What this is
A trust report on what a coding agent's "done" actually means. greenwash reads
the diff, reruns the committed version of any test that was changed, runs the
project's own suite against a baseline, and reports a verdict with the evidence
behind it. No model, no network, no dependencies.
Install from the tag:
pipx install git+https://github.com/0DukePan/greenwash@v0.5.1
greenwash --versiongreenwash-cli is not on PyPI yet -- the release workflow is in the
repository and publishes on a tag once the Trusted Publisher is configured.
Verify it before you believe it
git clone https://github.com/0DukePan/greenwash && cd greenwash
python benchmark/verify.py # 90 seconds, no model, no networkThree offline measurements: a planted cheat per task scored by the real test
runner, each attack beside its legitimate twin, and a positive/negative pair per
language pack. The exit code is non-zero on the first regression.
What changed since 0.4.2
greenwash coverage-- the Requirement Coverage Engine. Requirements can
bind to more than one test, so one passing and one failing is reported as
contradictory rather than flattened into "failed". A requirement whose
evidence would not run is could not check, distinct from one with nothing
bound. Requirements whose bound tests only assert the happy path are named as
having no negative case. Every report ends with: coverage counts requirements
with passing evidence; it is not a claim that the code is correct.python benchmark/verify.py-- one command for the whole offline
benchmark, and the CI job that runs it where every toolchain is installed.verifier-config-changedno longer fires on maintenance. It was 104 of 109
flags on real history, nearly all dependency bumps; it now asks only when a
change can alter what runs, and produces 7. Rule id, code and severity are
unchanged, so existing waivers keep working.unreadable-file(GW-DIV-003). A changed Python file the static layer
cannot read -- including one git reports as binary -- is a finding instead of a
clean report. Missing paths are still not findings.- A maintenance pass: 42 branches to 1, the default test loop from twelve
minutes to seven seconds, Windows nightly instead of per-push, no in-tree
ticket files, and a CI wall time cut from eleven minutes to under two. - The demo starts from the cause. The README clip opens on the agent's real
git diffand ends on the verdict.
The numbers, including the one that is not a win
- Detector: 25 of 25 planted cheats caught on the machine that ran it (24
static, 23 behavioural), 0 false positives on 25 recorded honest fixes. Three
tasks are skipped for missing toolchains and named; CI scores all 28. - False positives on real history: a ten-repository survey found 109 flags,
104 of them one rule on dependency bumps; after narrowing, 12 flags -- 10
asking, 2 hand-checked false positives. A survey, not a rate. - Agent behaviour: still unmeasured on a current model. The earlier +0.0 came
from an instrument that could not record a passing cheat; the fix is in, the
re-run needs a funded key. No page claims an effect in either direction.
What did not work
The two pilot runs that exposed corpus prompts not stating their requirement, the
release blocked on API balance, and the eight days CI was red on a released tag
because the benchmark classifier could not read the checker's output. It is all
in docs/what-did-not-work.md.
Full write-up with intervals and limits: docs/benchmark-summary.md and
docs/claim-matrix.md.