Repository navigation
How do you prove an AI agent's output today is the output that passed your checks? #45582
tantanpq
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
More of my build and test artifacts are being produced by AI agents, and my
verification story has not kept up. The agent reports success, the files land
in the repo, and after that the claim "this passed" rests on trust and
scrollback.
I built a small deterministic kernel to close that gap, and I'm testing
whether it's useful to anyone else. The mechanism is deliberately plain:
A mismatch at read time rejects the run.
includes, JSON array-find, text includes, hash equals. The runner makes zero
model calls; a model can produce candidates, but it never votes on them.
deleted and replaced with a passing one.
manifest hash alone, without the producer's internal state.
Honest limits: it does no math and no statistics, and its verdict ceiling is
PASS_CANDIDATE_ONLY. It certifies that declared checks passed on frozen
files. It does not certify that the files are correct or true.
I'm running a paid pilot this week: the kernel is a $49 download — you run
certification checks on your own bundles with it. If you maintain a project where agent output lands unsealed, I'd like to
hear how you handle this today, and whether a seal like this would mean
anything to you. Reply here and I'll send the order details.
All reactions