Verdict Contract

How another plugin reads artwright's judgement. The caller hands artwright-judge a folder of images and a brief; artwright writes one file, <folder>/artwright/verdict.json, and touches nothing else of the caller's. The contract is version 1 (the artwright field). The repository treats it as a public interface: a breaking change bumps that field and the plugin's minor version. The source of truth is verdict-contract.md in the repository.

Calling

Start the file with the images and the brief, then ask artwright-judge to judge the folder:

python aw.py verdict init batch batch/fox_01.png batch/fox_02.png batch/fox_03.png batch/fox_04.png --kind pairwise --brief "red fox in snow at dusk, side view, no text"

prints:

{
 "verdict": "batch/artwright/verdict.json",
 "items": [
  {
   "id": "A",
   "file": "fox_01.png",
   "sha256": "a281f69710d4b9e2c7c204ee01e0b99def061bf9092bad6f060ed1a77ad60aa2"
  },
  {
   "id": "B",
   "file": "fox_02.png",
   "sha256": "666fa6e5311dae41e94c6a76419982ae459480b39f498676d2267f92a609dfaa"
  },
  {
   "id": "C",
   "file": "fox_03.png",
   "sha256": "f152d4d30a807dc9d6b4ed2fe8ad398d2fe16c7e8db4e96c0c3b45505beb5307"
  },
  {
   "id": "D",
   "file": "fox_04.png",
   "sha256": "50a3b0a92ef22cbd28159ab1bd6e70e147072bd5aa9bef871a884e1b012cd6d5"
  }
 ],
 "ok": true
}

The skill fills the rest and sets status to final. Before it reports, aw.py verdict validate <folder> must print "valid": true.

Fields

Field	Holds
artwright	the contract version, "1"
kind	pairwise, brief-fit, critique or pixel
brief	the brief as given
rubric	{"id": "artwright-judge", "version": "3"} since 0.5.0 ("2" in 0.3.0 and 0.4.x, "1" before); verdict init writes it. The contract version stayed "1"
judge	{"model": ..., "prompt_version": "1"}; model is null until the skill fills it
created, updated	local time, ISO 8601 to the second
status	draft or final
items	{"id", "file", "sha256"} per image; ids A, B, C in the order added
ranking	{"file", "rank", "strength", "wins", "losses", "ties", "close_to_next"} per image, from aw.py rank --write
fit	per image: verdict (FITS, FITS WITH FIXES or DOES NOT FIT), hard_fails, and checks of {"check", "pass", "region"}
fixes	per image: channel (prompt, setting, control, reference, inpaint or model), fix, and for a prompt fix prompt: {"model", "text", "diff"}
abstained	{"files", "reason"} per pair a person should see
notes	free text
score, score_anchors	only when the user asked for an absolute score; score needs score_anchors (three labelled references)

Rules for both sides, from the contract:

• Paths in the file are relative to the folder that holds artwright/, with forward slashes.
• items[].sha256 lets a caller see whether a file changed after the verdict; validate reports it.
• DOES NOT FIT lists its hard_fails.
• close_to_next: true means the order between that row and the next is not established. aw.py writes null on the last row.
• A caller must not silently pick between the files of an abstained pair.
• The working files beside it (pairs.jsonl, matches.jsonl, pairs/*.png) are artwright's own; callers ignore them.

A batch end to end

--kind counts only when the file is created. Running init again on the same folder keeps the kind and the items:

python aw.py verdict init batch --kind critique > /dev/null
python -c "import json; print(json.load(open('batch/artwright/verdict.json'))['kind'])"

pairwise

Prepare every pair of the four images (a round robin is six pairs) and ask for a first round of suggestions:

set -- batch/fox_01.png batch/fox_02.png batch/fox_03.png batch/fox_04.png
for a in "$@"; do
  shift
  for b in "$@"; do
    python aw.py compare prepare "$a" "$b" --work batch/artwright > /dev/null
  done
done
python aw.py rank --work batch/artwright --next 3

prints:

{
 "ranking": [
  {
   "rank": 1,
   "file": "batch/fox_04.png",
   "sha256": "50a3b0a92ef22cbd28159ab1bd6e70e147072bd5aa9bef871a884e1b012cd6d5",
   "strength": 1.0,
   "wins": 0,
   "losses": 0,
   "ties": 0,
   "close_to_next": true
  },
  {
   "rank": 2,
   "file": "batch/fox_02.png",
   "sha256": "666fa6e5311dae41e94c6a76419982ae459480b39f498676d2267f92a609dfaa",
   "strength": 1.0,
   "wins": 0,
   "losses": 0,
   "ties": 0,
   "close_to_next": true
  },
  {
   "rank": 3,
   "file": "batch/fox_01.png",
   "sha256": "a281f69710d4b9e2c7c204ee01e0b99def061bf9092bad6f060ed1a77ad60aa2",
   "strength": 1.0,
   "wins": 0,
   "losses": 0,
   "ties": 0,
   "close_to_next": true
  },
  {
   "rank": 4,
   "file": "batch/fox_03.png",
   "sha256": "f152d4d30a807dc9d6b4ed2fe8ad398d2fe16c7e8db4e96c0c3b45505beb5307",
   "strength": 1.0,
   "wins": 0,
   "losses": 0,
   "ties": 0,
   "close_to_next": null
  }
 ],
 "games": 0,
 "abstained": [],
 "cycles": [],
 "cycle_count": 0,
 "top_in_cycle": false,
 "note": "close_to_next: strengths within 1.5x, never compared directly, or inside a cycle (A beat B, B beat C, C beat A); do not claim an order there. top_in_cycle: the winner is close, say so",
 "next_pairs": [
  [
   "batch/fox_04.png",
   "batch/fox_02.png"
  ],
  [
   "batch/fox_01.png",
   "batch/fox_03.png"
  ]
 ],
 "ok": true
}

Nothing is judged yet, so every strength is 1 and every row is close_to_next. next_pairs stops at two: with four images, each in at most one pair, there is no third.

The judge then records both orders of all six pairs with compare record. For this example the picks were scripted: fox_03 beats every other image, fox_01 beats fox_02 and fox_04, and the two orders of fox_02 against fox_04 disagree. Then:

python aw.py rank --work batch/artwright --write

prints:

{
 "ranking": [
  {
   "rank": 1,
   "file": "batch/fox_03.png",
   "sha256": "f152d4d30a807dc9d6b4ed2fe8ad398d2fe16c7e8db4e96c0c3b45505beb5307",
   "strength": 5.275,
   "wins": 3,
   "losses": 0,
   "ties": 0,
   "close_to_next": false
  },
  {
   "rank": 2,
   "file": "batch/fox_01.png",
   "sha256": "a281f69710d4b9e2c7c204ee01e0b99def061bf9092bad6f060ed1a77ad60aa2",
   "strength": 1.592,
   "wins": 2,
   "losses": 1,
   "ties": 0,
   "close_to_next": false
  },
  {
   "rank": 3,
   "file": "batch/fox_04.png",
   "sha256": "50a3b0a92ef22cbd28159ab1bd6e70e147072bd5aa9bef871a884e1b012cd6d5",
   "strength": 0.345,
   "wins": 0,
   "losses": 2,
   "ties": 1,
   "close_to_next": true
  },
  {
   "rank": 4,
   "file": "batch/fox_02.png",
   "sha256": "666fa6e5311dae41e94c6a76419982ae459480b39f498676d2267f92a609dfaa",
   "strength": 0.345,
   "wins": 0,
   "losses": 2,
   "ties": 1,
   "close_to_next": null
  }
 ],
 "games": 6,
 "abstained": [
  {
   "pair": "5673c6320db1",
   "files": [
    "batch/fox_02.png",
    "batch/fox_04.png"
   ],
   "reason": "orders disagree: show a human"
  }
 ],
 "cycles": [],
 "cycle_count": 0,
 "top_in_cycle": false,
 "note": "close_to_next: strengths within 1.5x, never compared directly, or inside a cycle (A beat B, B beat C, C beat A); do not claim an order there. top_in_cycle: the winner is close, say so",
 "verdict": "batch/artwright/verdict.json",
 "ok": true
}

The ranking and abstained in verdict.json now hold the same rows with paths relative to batch (fox_03.png, not batch/fox_03.png) and without sha256 or pair. The file is still a draft, and a draft is valid:

python aw.py verdict validate batch

prints:

{
 "verdict": "batch/artwright/verdict.json",
 "valid": true,
 "problems": [],
 "ok": true
}

validate checks the shape, not whether the judging is finished: a caller must check status itself. The skill then adds the fit checks and the fixes and marks it final. In Python:

import json
from pathlib import Path

path = Path("batch/artwright/verdict.json")
v = json.loads(path.read_text(encoding="utf-8"))
v["judge"]["model"] = "claude-opus-5-5"
v["fit"] = [{"id": "C", "verdict": "FITS", "hard_fails": [],
             "checks": [{"check": "one fox", "pass": True, "region": "centre"}]}]
v["fixes"] = [{"id": "A", "channel": "prompt", "fix": "the fox faces the camera; the brief asks for a side view",
               "prompt": {"model": "z-image-turbo", "text": "red fox in profile walking through snow at dusk",
                          "diff": "+ in profile"}}]
v["status"] = "final"
path.write_text(json.dumps(v, indent=1) + "\n", encoding="utf-8")

python aw.py verdict validate batch

prints:

{
 "verdict": "batch/artwright/verdict.json",
 "valid": true,
 "problems": [],
 "ok": true
}

Reading it from a calling plugin

import json
from pathlib import Path

v = json.loads(Path("batch/artwright/verdict.json").read_text(encoding="utf-8"))
if v["status"] != "final":
    raise SystemExit("artwright has not finished")
best = v["ranking"][0]
print("best:", best["file"], "strength", best["strength"])
if best["close_to_next"]:
    print("not clearly ahead of", v["ranking"][1]["file"])
for t in v["abstained"]:
    print("ask a person:", " vs ".join(t["files"]), "-", t["reason"])
for f in v["fixes"]:
    print("next prompt for", f["prompt"]["model"] + ":", f["prompt"]["text"])

prints:

best: fox_03.png strength 5.275
ask a person: fox_02.png vs fox_04.png - orders disagree: show a human
next prompt for z-image-turbo: red fox in profile walking through snow at dusk

fox_03 won 3 to 0 with a strength more than 1.5 times the next, so there is no "not clearly ahead" line.

What validate catches

The same file with an absolute path, a DOES NOT FIT without hard fails, an unknown fix channel and a lone score:

import json
from pathlib import Path

path = Path("batch/artwright/verdict.json")
v = json.loads(path.read_text(encoding="utf-8"))
v["items"][0]["file"] = "/home/you/renders/fox_01.png"
v["fit"][0]["verdict"] = "DOES NOT FIT"
v["fixes"][0]["channel"] = "negative prompt"
v["score"] = 8
path.write_text(json.dumps(v, indent=1) + "\n", encoding="utf-8")

python aw.py verdict validate batch

prints, and exits 1:

{
 "verdict": "batch/artwright/verdict.json",
 "valid": false,
 "problems": [
  "item A: paths are relative with forward slashes",
  "item A: /home/you/renders/fox_01.png not found",
  "DOES NOT FIT needs hard_fails",
  "fix channel must be one of prompt, setting, control, reference, inpaint, model",
  "an absolute score needs score_anchors (three labelled references)"
 ],
 "ok": false
}

An image changed after the verdict was written:

mkdir -p changed && cp batch/fox_01.png changed/
python aw.py verdict init changed changed/fox_01.png --kind critique > /dev/null
python aw.py degrade changed/fox_01.png -o changed/fox_01.png --kind blur > /dev/null
python aw.py verdict validate changed

prints, and exits 1:

{
 "verdict": "changed/artwright/verdict.json",
 "valid": false,
 "problems": [
  "item A: file changed since the verdict (sha256 differs)"
 ],
 "ok": false
}

Every check validate makes:

Problem	Message
a required field missing	missing artwright (or kind, rubric, judge, items, status)
an unknown kind	kind must be one of pairwise, brief-fit, critique, pixel
an unknown status	status is draft or final
an item without id or file	item without id or file
an absolute path or a backslash	item A: paths are relative with forward slashes
a missing file	item A: <file> not found
a changed file	item A: file changed since the verdict (sha256 differs)
an unknown fit verdict	fit verdict must be one of FITS, FITS WITH FIXES, DOES NOT FIT
a fit row naming no item	fit refers to unknown id X
DOES NOT FIT with no hard fails	DOES NOT FIT needs hard_fails
an unknown fix channel	fix channel must be one of prompt, setting, control, reference, inpaint, model
a ranking row without rank	ranking row without rank
score without score_anchors	an absolute score needs score_anchors (three labelled references)

No verdict.json at all is an error rather than a problem list: {"ok": false, "error": "no <folder>/artwright/verdict.json"}.