Skip to content

unwash 1.0.0

Latest

Choose a tag to compare

@0DukePan 0DukePan released this 17 Nov 21:44
· 10 commits to main since this release

unwash 1.0.0

The project has a name that matches what it does, and a name nobody else owns.

What it is

A trust report on what a coding agent's "done, tests pass" actually means. It
reads the diff, reruns the committed version of any test that was changed, runs
the project's own suite against a baseline, and reports a verdict with the
evidence behind it. No model, no network, no dependencies.

pip install unwash-cli
cd your-repo
unwash

unwash-cli is the distribution; unwash is the command and the import.

The rename

The old name described the problem rather than the tool, and an unrelated
package already held it on PyPI. That forced a different distribution name and
an explanation in every install instruction -- a defect in a README whose whole
argument is that an instruction which does not work is a lie.

Now the distribution, the import, the command and the repository share one
name. This is a major version because that breaks things on purpose:

Before After
greenwash unwash
GW-TEST-001 UW-TEST-001
.greenwash/config.json .unwash/config.json
GREENWASH_ENFORCE UNWASH_ENFORCE
github.com/0DukePan/greenwash github.com/0DukePan/unwash

A waiver written against an old rule code still resolves, and a new waiver
counts on from whatever ids the ledger already holds, so an id is never
reissued. GitHub redirects the old repository URL, so existing clones keep
fetching.

Verify it before you believe it

git clone https://github.com/0DukePan/unwash && cd unwash
python benchmark/verify.py     # 90 seconds, no model, no network

What is in 1.0.0

  • unwash coverage -- the Requirement Coverage Engine. Requirements can bind
    to more than one test, so one passing and one failing is reported as
    contradictory rather than flattened into "failed". A requirement whose
    evidence would not run is could not check, distinct from one with nothing
    bound. Requirements whose bound tests only assert the happy path are named as
    having no negative case. Every report ends with coverage counts requirements
    with passing evidence; it is not a claim that the code is correct.
  • benchmark/verify.py -- one command for the whole offline benchmark:
    a planted cheat per task scored by the real test runner, each attack beside
    its legitimate twin, a positive/negative pair per language pack.
  • unreadable-file (UW-DIV-003) -- a changed Python file the static layer
    cannot read, including one git reports as binary, is a finding rather than a
    clean report.
  • verifier-config-changed narrowed -- it was 104 of 109 flags on real
    history, nearly all dependency bumps; it now asks only when a change can alter
    what runs, and produces 7.
  • The maintenance pass, the demo rebuild, and the first live-model pilot are
    described in CHANGELOG.md.

The numbers, including the one that is not a win

  • Detector: 25 of 25 planted cheats caught on the machine that ran it (24
    static, 23 behavioural), 0 false positives on 25 recorded honest fixes. Three
    tasks are skipped for missing toolchains and named; CI scores all 28.
  • False positives on real history: 109 flags across ten repositories, 104 of
    them one rule on dependency bumps; after narrowing, 12 -- 10 asking, 2
    hand-checked false positives. A survey, not a rate.
  • Agent behaviour: still unmeasured on a current model. No page claims an
    effect in either direction.

What did not work

Five days of red nightly because a repaired task changed label and its test did
not; the release blocked on API balance; the earlier +0.0 that came from an
instrument that could not record a passing cheat. All of it is in
docs/what-did-not-work.md.