feat(vocab): agreement test — glossary relations vs blast-radius graph (task #13) - #259
Merged
Conversation
…h (task #13) The neurosymbolic check that the vocabulary tells the truth about structure. tools/agreement_test.py projects the glossary's composition relations (has-a/has-member) onto the estate via alignment.estateBinding and compares them to a CONSUMED blast-radius/dependency graph (GBRG owns that graph; this only compares). Fail-closed on OVERCLAIM (a declared dependency with no observed edge = governance hole); reports DRIFT (an observed edge no relation names) as a remediation candidate (a proposed has-a relation), same treatment as the vocab-currency loop's candidate terms. Only composition predicates imply runtime dependencies; is-a/skos:* excluded. validate-agreement (in make validate) teeth: aligned agrees; overclaim refused; drift reported as candidate.
A draft term hasn't passed the alignment gate and doesn't regulate state, so its relations must not overclaim the estate. Restrict declared edges + bindings to status==approved terms.
There was a problem hiding this comment.
Pull request overview
Adds a new “agreement test” that cross-checks the glossary’s declared composition relations (has-a / has-member, projected via alignment.estateBinding) against an observed blast-radius dependency graph, with CI “teeth” that fail closed on overclaims and report drift as remediation candidates.
Changes:
- Introduces
tools/agreement_test.pyto compute agreements / overclaims / drift candidates between declared relations and observed edges. - Adds
tools/validate_agreement.pyplusmake validate-agreement(and wires it intomake validate) to enforce the expected fail-closed vs remediation behavior on fixtures. - Documents the agreement test in
specs/agreement-test.mdand records it inCHANGELOG.md, with fixtures underfixtures/agreement/.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/validate_agreement.py | CI harness (“teeth”) exercising aligned/overclaim/drift scenarios against fixtures. |
| tools/agreement_test.py | Core comparison logic: projects glossary relations into estate edges and compares to observed graph edges. |
| specs/agreement-test.md | Spec doc describing semantics (overclaim fail-closed; drift reported). |
| Makefile | Adds validate-agreement target and includes it in validate. |
| fixtures/agreement/glossary.json | Minimal glossary bundle fixture used by the validator scenarios. |
| fixtures/agreement/blast_radius_graph.json | Minimal blast-radius graph fixture used by the validator scenarios. |
| CHANGELOG.md | Notes the addition of agreement test v0.1 and its validation gate. |
Comment on lines
+60
to
+75
| # DRIFT: an observed edge between two bound entities with no declared dep relation -> propose one. | ||
| ent_to_term = {ref: tid for tid, ref in bind.items()} | ||
| drift = [] | ||
| for (ef, et) in sorted(observed): | ||
| if ef in ent_to_term and et in ent_to_term and (ef, et) not in declared: | ||
| drift.append({"edge": [ef, et], | ||
| "proposedRelation": {"subject": ent_to_term[ef], "predicate": "has-a", | ||
| "target": ent_to_term[et]}, | ||
| "detail": "estate shows a dependency the vocabulary does not name"}) | ||
|
|
||
| return { | ||
| "ok": not overclaims, # fail-closed on overclaims; drift is remediation, not failure | ||
| "agreements": agreements, | ||
| "overclaims": overclaims, | ||
| "driftCandidates": drift, | ||
| } |
| def agreement(glossary: dict, graph: dict) -> dict: | ||
| terms = {t["id"]: t for t in glossary["terms"]} | ||
| bind = {tid: _binding(t) for tid, t in terms.items() if _binding(t)} | ||
| observed = {(e["from"], e["to"]) for e in graph.get("edges", [])} |
… (Copilot #259) - two approved terms binding the SAME estate entity made drift attribution ambiguous (dict silently dropped a duplicate). Now surfaced as bindingConflicts and fails closed; only unambiguously-bound entities participate in drift. - a graph edge missing from/to would KeyError-crash. Now malformed edges are collected into malformedEdges and fail closed instead of crashing. Two teeth added (ambiguous-binding, malformed-edge); 5 total.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Does the vocabulary tell the truth about structure?
The neurosymbolic agreement check. Two independent views of the same estate:
relations(composition:has-a,has-member)Projected onto each other via
alignment.estateBinding, they must agree:A has-a B(both bound) but the graph shows no edge → the vocabulary asserts a dependency the estate doesn't exhibit → fail-closed (governance hole)has-arelation), not a hard failure (same treatment as the vocab-currency loop's candidate terms)tools/agreement_test.pyconsumes a blast-radius graph (GBRG owns it; this only compares). Only composition predicates imply runtime dependencies;is-a/skos:*are subsumption/lexical and excluded.Teeth (
make validate-agreement)has-arelation, not silently passed and not hard-failedCompletes the last self-contained thread of the vocabulary program: currency (#255), proposal (#256), dogfood (#257), ingest+gate (ontogenesis#133), alignment promotion (#258), and now relations↔structure agreement. The estate's words and its wiring are now held to match.