Skip to content

claim-check v1.0.0

Choose a tag to compare

@tonydzi tonydzi released this 04 Aug 17:21
· 11 commits to main since this release

Every number you publish is a claim. Prove it in CI.

claim-check binds each figure in your README or report to the artifact it came from — a JSON file, a log, a command's output — recomputes it on every run, and fails the build naming both sides when they disagree.

- uses: Palo-Alto-AI-Research-Lab/claim-check@v1
  with:
    claims-file: .github/claims.yml
what went wrong:
  DRIFT    unique-events    doc says 700, source says 744  (REPORT.md:4)

CLAIM-CHECK: 6 claims - 4 ok - 2 drift - 0 errors -> DRIFT

In this release

  • two ways to bind a number: <!--claim:id--> markers, or the sentence itself via pattern:
  • sources: json: + path:, text: + regex:, cmd: (opt-in), value:
  • formats (comma, round:N, percent:N, any Python format spec) and tolerance — formatting differences forgiven, arithmetic not
  • --fix rewrites drifted values in place, then re-verifies from disk
  • exit-code contract 0 / 1 DRIFT / 2 ERROR, in which a source it could not read never presents as a clean result
  • workflow annotations with file and line, a job-summary table, an optional JSON report
  • 90 selftest checks, each one proven by a mutant that breaks it first

Security, because on a fork's pull request the claims file belongs to a stranger: cmd: is off unless you ask for it and refused outright on fork PRs, every config-supplied path is confined to the checkout, and a hanging command is killed after 60 seconds instead of idling the runner.

No LLM, no API key, no network, no dependencies. Python 3.8+ on the runner.