What is missing
gradeFor is the calibrated verdict function every grader in the stack shares, and three of its rules mis-grade in ways that were measured on live claims. Each fix is a few lines inside gradeFor; each is currently carried by a downstream copy that says it is mirroring this function.
1. The unreachable-input signature list is consulted only on a nonzero exit. src/claim-evidence.ts:215-217 applies UNRUNNABLE_SIGNATURES inside if (execution.exitCode !== 0). A shell pipeline exits with the status of its LAST stage, so sha256sum <missing path> | cut -d' ' -f1 prints "No such file or directory" and exits 0. gradeFor then compares that error text to the expectation and returns contradicted — a refutation of research that was never run. Apply the signature list to the output whatever the exit status, and let it only ever downgrade contradicted to unrunnable, never turn a real failure into a pass. Two further details from the downstream copy: match case-insensitively, because a tool that prints its own message lowercases it; and consult the expectation first, so a claim that PREDICTED the error is still genuinely contradicted.
2. A check killed at its deadline is graded as a refutation. A process killed at a timeout (exit 124) never tested the claim: the deadline is a budget, not a verdict. It should return unrunnable with the timeout named, and it must fire before the expectation comparison, so a slow solver that prints nothing is not recorded as contradicted.
3. A passing check is refused for having a precise expectation. expectationRefusalNote at src/claim-evidence.ts:296-300 returns uncheckable when the expectation carries three or more key=value tokens — after the check exited 0 and immediately above the comparison that would have passed. The rule inverts its own intent: expect: OK is one token, passes the guard, and is satisfied by any output containing "OK" anywhere, while GRID OK cells=8 WIN=1 PARETO=6 NEGATIVE=1 is refused, though a false pass there needs four independent numbers to coincide. This half is already filed as tangle-network/agent-runtime#1006, which is open and in the wrong repository; it belongs here.
A fourth, smaller ask that rides with these: a duplicate flag on a grade. When several claims in one run share one check and one expectation but carry distinct titles, that is a check-quality defect and should be flagged, not silently counted as N independent verifications. Only a full match on title, check, and expectation is a true duplicate.
Where it lives today
discovery-lab/tools/oracle.mjs 701 lines, whose gradeExecution at :235 wraps verdictFor and applies all of the above. oracle.mjs:290-292 states the relationship: "Mirrors the upstream list in @tangle-network/agent-knowledge's gradeFor, which consults it only on a nonzero exit — a trailing pipe hides that, as measured above." The duplicate flag is at oracle.mjs:191-215.
Where it belongs
src/claim-evidence.ts:204 — export function gradeFor(evidence, execution), and its two helpers checkRefusalNote (:285) and expectationRefusalNote (:293).
Measured motive
- Unreachable input. Swept 2026-08-22 over all 272 grade files: 61 claims stood recorded as
contradicted, and 29 of them (48%) carry an output showing the check never ran against the claim — 7 a SyntaxError from a malformed check body, 7 a shell that could not parse the check at all, 6 a solver that could not open its problem file, 5 Permission denied, 4 cannot create directory. None is a refutation; each is a measurement that did not happen, recorded as evidence against a claim.
- Precise expectation. Measured 2026-08-23: 173 of 1,001 rung-4-or-higher claims carrying an expectation, 17.3%, were discarded as
uncheckable after their check exited 0 — including one whose output matched its expectation byte for byte.
Acceptance
oracle.mjs can drop its gradeExecution wrapper and call gradeFor directly when four tests pass in this repository: a zero-exit pipeline whose output carries an unreachable-input signature grades unrunnable, not contradicted; the same case with an expectation that names the signature still grades contradicted; an exit-124 execution grades unrunnable with the deadline named; and a zero-exit check whose output contains a four-token expectation grades verified.
What is missing
gradeForis the calibrated verdict function every grader in the stack shares, and three of its rules mis-grade in ways that were measured on live claims. Each fix is a few lines insidegradeFor; each is currently carried by a downstream copy that says it is mirroring this function.1. The unreachable-input signature list is consulted only on a nonzero exit.
src/claim-evidence.ts:215-217appliesUNRUNNABLE_SIGNATURESinsideif (execution.exitCode !== 0). A shell pipeline exits with the status of its LAST stage, sosha256sum <missing path> | cut -d' ' -f1prints "No such file or directory" and exits 0.gradeForthen compares that error text to the expectation and returnscontradicted— a refutation of research that was never run. Apply the signature list to the output whatever the exit status, and let it only ever downgradecontradictedtounrunnable, never turn a real failure into a pass. Two further details from the downstream copy: match case-insensitively, because a tool that prints its own message lowercases it; and consult the expectation first, so a claim that PREDICTED the error is still genuinely contradicted.2. A check killed at its deadline is graded as a refutation. A process killed at a timeout (exit 124) never tested the claim: the deadline is a budget, not a verdict. It should return
unrunnablewith the timeout named, and it must fire before the expectation comparison, so a slow solver that prints nothing is not recorded ascontradicted.3. A passing check is refused for having a precise expectation.
expectationRefusalNoteatsrc/claim-evidence.ts:296-300returnsuncheckablewhen the expectation carries three or morekey=valuetokens — after the check exited 0 and immediately above the comparison that would have passed. The rule inverts its own intent:expect: OKis one token, passes the guard, and is satisfied by any output containing "OK" anywhere, whileGRID OK cells=8 WIN=1 PARETO=6 NEGATIVE=1is refused, though a false pass there needs four independent numbers to coincide. This half is already filed as tangle-network/agent-runtime#1006, which is open and in the wrong repository; it belongs here.A fourth, smaller ask that rides with these: a duplicate flag on a grade. When several claims in one run share one check and one expectation but carry distinct titles, that is a check-quality defect and should be flagged, not silently counted as N independent verifications. Only a full match on title, check, and expectation is a true duplicate.
Where it lives today
discovery-lab/tools/oracle.mjs701 lines, whosegradeExecutionat:235wrapsverdictForand applies all of the above.oracle.mjs:290-292states the relationship: "Mirrors the upstream list in @tangle-network/agent-knowledge's gradeFor, which consults it only on a nonzero exit — a trailing pipe hides that, as measured above." The duplicate flag is atoracle.mjs:191-215.Where it belongs
src/claim-evidence.ts:204—export function gradeFor(evidence, execution), and its two helperscheckRefusalNote(:285) andexpectationRefusalNote(:293).Measured motive
contradicted, and 29 of them (48%) carry an output showing the check never ran against the claim — 7 aSyntaxErrorfrom a malformed check body, 7 a shell that could not parse the check at all, 6 a solver that could not open its problem file, 5Permission denied, 4cannot create directory. None is a refutation; each is a measurement that did not happen, recorded as evidence against a claim.uncheckableafter their check exited 0 — including one whose output matched its expectation byte for byte.Acceptance
oracle.mjscan drop itsgradeExecutionwrapper and callgradeFordirectly when four tests pass in this repository: a zero-exit pipeline whose output carries an unreachable-input signature gradesunrunnable, notcontradicted; the same case with an expectation that names the signature still gradescontradicted; an exit-124 execution gradesunrunnablewith the deadline named; and a zero-exit check whose output contains a four-token expectation gradesverified.