Capability honesty is making an AI report the lowest proven state instead of the highest hoped-for one: it advertises an ability only after a probe has exercised it and passed, and it calls work "done" only when a verifiable artifact backs the claim. An AI that says "I haven't proven I can do that yet" is worth more than one that always says yes.
Distilled from two mechanisms in a production system — a boot-conformance check that probes each ability before advertising it, and a completion gate built the day a trial agent marked a task "done" with nothing on disk. Published with its own attack suite: every test hands the system an overclaim and passes only when the ability is withheld or the completion is downgraded.
"It claims abilities it doesn't have." Registering a capability does
not make it real. Here, a capability is unproven until its probe runs and
passes — and a probe that crashes is never read as a pass.
from capability_honesty import CapabilityRegistry
caps = CapabilityRegistry()
caps.register("send_email", probe=lambda: smtp_connect_and_noop())
caps.require("send_email") # ok: False — unproven, refused rather than assumed
caps.prove_all() # run the probes
caps.require("send_email") # ok: True — only now, and only if the probe passed
caps.status()["advertised"] # lists proven abilities only; the rest are 'withheld'"It says done when it isn't." A completion claim is a claim, not a
fact. settle downgrades it to a proposal unless evidence or an artifact
backs it.
from capability_honesty import settle, is_done
settle("wrote the report")
# state: 'proposal' — no artifact, no verifying evidence
settle("wrote the report", evidence=lambda: Path("out/report.md").exists())
# state: 'done' — only because the check passed
settle("wrote the report", evidence=lambda: False) # 'proposal'
settle("shipped it", artifact="out/report.md") # 'done' — artifact presentA model can be talked into saying "done" on a bad day. The gate cannot.
done and claimed done are different states, and this library keeps them
different mechanically. A crashing probe or a raising evidence-check is
treated as no proof — never optimistically as a pass. The system reports
what it proved, never what it hopes.
pip install capability-honesty
Zero dependencies.
Part of The Operator's Honesty Stack — open-source honesty engines from the same production system, composed, never merged:
- trust-skeleton: a "done" that is really a proposal belongs in INFERRED, not OBSERVED.
- evidence-binding-compiler: the artifact that settles a completion is the evidence EBC would bind.
- memory-integrity: a proven completion is an observation; a claimed one is an interpretation.
Apache 2.0. Copyright 2026 Dave DePew Enterprises, Inc.