SymbolicCheck: formally-verified invariants on structured outputs #2447
Replies: 1 comment
|
Thank you for your proposal and for your interest ! I've had a look at ephemaral, it is very nice, and if I understood correctly it automatically verifies a Hoare logic kind of invariants (ie with identical pre- and post-conditions) on a json representation of a function, using z3 to provide the proof of invariance. The flexibility to deal with any structured document is the reason for which we use rego in our guardails solution (out of the OSS scope though). It is a datalog-like language extended with structured data types, comprehensions, rich builtins, and finite-domain quantification over JSON data. I reckon it addresses the gap you are pointing out. |
Uh oh!
There was an error while loading. Please reload this page.
giskard-checksv3 covers single-field predicates (Equals, comparisons,RegexMatching,StringMatching) and LLM-based judgements (Groundedness,Conformity,LLMJudge,SemanticSimilarity), andFnCheck. What is missing is a way to verify multi-field invariants on structured outputs without re-writing the same lambda in every test.The gap
The properties I have in mind span multiple fields, require arithmetic, or quantify over bounded collections are too structured for an LLM judge, too composite for a single comparison check. A few examples from regulated agent workflows:
order.total == sum(line_items, price * quantity) + tax— arithmetic invariant across a structured outputaccount.balance >= 0after a tool-call sequenceforall item in cart: 0 <= item.discount <= item.price * item.quantity— universal quantification on a bounded domaintransaction.amount <= account.daily_limit - account.amount_used_today— cross-field policy invariantEach of these can be written today as an
FnChecklambda. That works for the specific scenario it is written into but does not generalise. The same invariant gets re-written in every test, with no shared specification, and it only verifies the observed output, not the property across the input domain.The proposal
SymbolicCheckwould be a new check type that takes a declarative invariant and verifies it against the trace output. Shape-wise, it slots in as a peer toLLMJudgein the existing hierarchy. Pseudocode for what it would look like at the call site:At verification time, the spec compiles to an SMT problem. SMT (Satisfiability Modulo Theories) is the branch of automated reasoning that combines propositional logic with decidable theories like arithmetic, equality, and arrays. An SMT solver such as Z3 (Microsoft Research, Apache 2.0, widely used in both academic and industrial verification) takes the constraints and decides whether the invariant is consistent with the actual trace value. The verdict is categorical rather than probabilistic:
SAT(invariant holds) → successUNSAT(counterexample exists) → failure, with the counter-model includedUNKNOWN(timeout, decidability boundary) → failure with a clear reasonScope
Of course this does not replace
LLMJudge. Symbolic reasoning has no traction over open-ended text - hallucination, tone, bias, multi-turn coherence are all outside its scope. The aim is not to compete with LLM-as-judge but to sit alongside it for the slice of checks where:It is also worth saying what this is not. It is not an LLM-output verifier (the LLM judges still do that better). It is not a replacement for
FnCheck(FnCheck remains the right tool for anything that does not fit a declarative spec). It is not a general theorem prover - the spec language is deliberately scoped to the decidable fragment Z3 handles well in practice.Related
There is an existing open-source tool (that I am also working on) exploring exactly this pattern — [Ephemaral](https://ephemaral.com), a Lean 4 + Z3 verifier that takes a similar declarative spec language and checks code against it. Its parser-and-spec architecture is portable: parser sits outside the trust boundary, spec compiled to Lean for kernel-checking, Lean to SMT, Z3 returns a verdict. The same architecture maps cleanly onto the check API surface here, with the trace value substituted for the code-under-test as the input to the SMT problem.
Questions
giskard-checks?Happy to spike a minimal version if there is interest. The core capability already exists - the question is whether and how it should land in
giskard-checks.All reactions