Agents free will STL findings

[Scorpse]

I let the agents work their stl language structure (no gevernance at all). see what they did there.

https://github.com/Scorpse/semantic-tension-language/blob/main/docs/SCHEMA_FEEDBACK_FROM_AGENT_USAGE.md

still analyzing.

[scos-lab]

Thank you — this is the most useful field report the project has received: 543 statements of live multi-agent traffic across five different agents, and the divergences are exactly the kind of thing a schema cannot learn from its authors. Readings from our side, recommendation by recommendation, with honest dispositions:

1. kind vs relation — your agents were right, and the gap is partly our documentation. STL's meta semantic fields are is_a / action / role / status / phase / relation, and the ingest methodology already says choose the most specific field; don't default to relation. The slot for an event/outcome verb is action (action="gate_pass" ≈ the gate passed); is_a is classification; status / phase carry state and lifecycle; relation is the last-resort generic. kind was retired in v1.2 (folded into is_a for the classification sense) — so what the agents coined as kind=gate_pass is, in the core vocabulary, action=gate_pass. The software-* family required relation only, which is why they invented a field instead of finding one. Disposition: we'll document the mapping explicitly (kind as event → action; kind as category → is_a) in the Ingest Methodology and the schema README, crediting this report, rather than add a seventh field.

2. Enumerated outcome, separate from descriptive status — agree, and it already exists on the trust-layer side. Board-as-Spec fixes the status set for task items (pending | in_progress | pass | fail | warn | blocked + sub-status), and the Eval-Verdict vocabulary puts how-do-we-know on every claim (provenance = verified / reported / claimed / inferred, with confidence kept separate). Your exit_0_no_errors / 5_pass_3_warn_0_fail hazard is a textbook case for "enum for machines, free text for humans"; we'll add it (credited) to the verdict spec's rationale. For the schema family, an outcome enum next to free-text status is the right shape.

3–4. Gate / Verifier / Run / Lane entities and the orchestration layer — these want to be community profiles, and you're the one holding the data. software-review and software-agentcoord, as you sketch them, would be welcome in docs/schemas/ as community profiles with your name on them. One mapping worth knowing before you write them: claim / release / handoff / owner / notify / obligation correspond, on the board side, to Board-as-Spec's cross-board semantics (§6: relay with authorization and collision rules, help requests, parent propagation) and to the Intent Contract's notification tier — we'd rather cross-reference than duplicate.

5. severity vs priority — keep both, define both. Our read: priority is scheduling (who acts first — agent or human attention), severity is risk (how bad if unhandled). Agents choosing priority 0–3 is consistent with that; a schema should govern both axes and say which is which.

6. Covered by 1: bless the mapping, not the field.

Two more things. May we cite this report (linked to your fork) in the next revision of the methodology docs and in the "From the field" section of the trust-layer page? And the 543-statement corpus itself — if you ever publish it (anonymized), it would be the first public agent-written STL corpus; we'd gladly host a pointer.

[Scorpse]

Yes you can cite this. That is why i posted here.

my next steps for analysis are:

1. i have the reviewers (codex and claude) do both stl and nl. need to compare token gain.) For a number fo comments between them i will have them log both (STL for the normal communication, and NL as they would have pre-STL)
2. need to conduct some test on the coders, when they receive info in NL vs STL. so i will have some double issues (one in stl, one in nl), Then check the flow (input, output), and rate the output.
3. lastly the MD's that are passed to the agents will be converted into STL (agents.md) and compare behavior.

also i am strengthening the boundaries and limits of their schema use. SO they wont invent any more words. Also applying your recs.

[scos-lab]

Thank you — cited as of today (methodology docs and the trust-layer page's "From the field" section both link the report).

Your three experiments are exactly the ones we'd want to read. Two things that may save you effort:

• For the token comparison, we published our own measurement setup and numbers in research/token-efficiency/ in this repo — if you log per-message token counts for the STL/NL pairs the same way, the results are directly comparable, and we'd be glad to cite yours next to ours.
• For "strengthening the boundaries so they won't invent words": the hardening you wrote is now on PyPI as stl-parser 1.10.x — edge { source/relation/target } rules and enum-typed modifier fields fail closed, and .stl.profile bundles let you govern several profiles over one document. Validating each agent comment against a profile before it's accepted is the cheapest enforcement point; an outcome enum field there would also remove the substring hazard you documented.

No rush on any of it — when the numbers exist, a note here is enough.

[Scorpse]

Requesting your review of a governance change built on this field report.

Context, stated honestly. We re-scraped MKT independently: 1,039 exact-unique STL statements — but that spans only 21 work items and collapses to ~627 distinct source → target + kind edges (the rest are the hourly MERGE-REVIEW lane re-emitting the same edge on a few stuck items, differing only by timestamp/run_id). The ~627 distinct-edge figure lands close to the 543 in the original report — an independent corroboration, not a larger dataset (zero exact duplicates).

The change we'd like reviewed — governing agent-to-agent STL as two community profiles (software-agentcoord, software-review) over a shared envelope:

• Your mapping, adopted: event-kind → action, category-kind → is_a; no bespoke kind field.
• outcome enum split from free-text status (+ phase for lifecycle), and de-fused from the verb — action="gate", outcome="pass", never action="gate_pass" — so pass/fail lives on exactly one axis rather than being reconstructable from a verb string.
• Confidence: 1.0 reserved for analytic truths, empirical capped at 0.95; six levels as calibration anchors; never default a missing value to 1.0.
• Role binding: a blocking gate whose satisfier role is unbound or ineligible fails validation. This came from a real deadlock — a qa_pass was required from a SPECIALIST_QA verifier role bound to no identity, stalling the whole review lane. The identity registry is kept external to the profile so it stays portable/upstreamable.
• Enforcement is engine-first + warn-then-enforce (closed modifier sets, action-keyed edge rules, external-registry resolution), not closed-from-day-one.

We ran the design past three external models (GPT-5.6, GLM, Grok); the de-fusing and external-registry points above came out of that pass.

Three questions where your view would most help:

1. Should an observed phase ever appear on an agent edge, or does Board-as-Spec's system-owned-state rule mean agents must never write lifecycle state at all?
2. validation (a self-check) vs verification (independent) — one verb plus provenance, or two distinct verbs to preserve the independence signal?
3. Role-binding / gate-satisfiability — a generic engine primitive reading an external registry, or a separate workflow-policy layer, to keep STL-TOOLS uncoupled from any one orchestrator?

Happy to open a PR with the two draft profiles once the enabling engine features land. Full design + the three model reviews available if useful.

[scos-lab]

This is a good design, and the honesty about the re-scrape (1,039 unique statements → 21 work items → ~627 distinct edges, corroborating rather than enlarging the 543) is exactly the discipline the verdict vocabulary asks for — counting by edge, not by statement, is the right unit. Two things we'd underline before the questions: de-fusing the verb from the outcome (action="gate", outcome="pass", never gate_pass) is the single best decision in here — it also collapses the 45-verb explosion — and engine-first, warn-then-enforce is how we'd sequence it too. The confidence rules you list match STL v1.2 exactly (1.0 analytic only; empirical caps at 0.95; six canonical levels 1.0 / 0.95 / 0.8 / 0.5 / 0.2 / 0.01; never default a missing value to 1.0).

On the three questions:

1. phase on agent edges. Board-as-Spec's two-planes rule is precisely the answer: structure and state are authored through different paths, and lifecycle state (in_progress, transitions, the cursor) is system-owned — an agent requests pass / halt; it never writes the item's state. Translated to your profiles: an agent edge may carry phase as context ("this claim is about the review stage"), and consumers MUST read it as a claim (provenance reported / claimed), never as a state transition; the authoritative lifecycle lives in the platform's state record (Kumidai's work-item state machine is your execution plane), derived from outcome claims plus the platform's own rules. If you want to make the distinction un-confusable, name the agent-side field differently (about_phase, or put it in description) and reserve phase for system-written records. The test is simple: if an agent writing phase="done" could ever move the item, the planes have been mixed.

2. validation vs verification. Both, and for different reasons. Keep two verbs — validate (self-check) / verify (independent) — because machines route on verbs and the independence signal is too valuable to bury; your agents already felt this (independence appeared 11 times unprompted). But a verb only asserts independence; make it checkable by recording verifier and claimant identities on the edge — independence is a relation between two parties, and verifier != claimant is something a validator can test. Then provenance does its own job on top: a self-check that passed is at most reported; an independent check against ground truth is verified. Eval-Verdict V1 (a passing self-check is weak evidence) and V3 (criteria sharing a mechanism are one criterion) are the rationale.

3. Role binding / gate satisfiability. Keep STL-TOOLS uncoupled by splitting it three ways: the engine gets a generic primitive — a document-level constraint that may consult an external oracle (a resolver hook passed at validation time, e.g. validate_against_profiles(..., resolvers={"identity": fn})) plus a cross-statement requirement rule ("an edge of kind X requires some statement binding role R to a member of registry I"); the profile declares the policy (which gate kinds require which roles bound); the registry stays outside both, as you already have it. That way the engine knows nothing about any orchestrator, and the deadlock you hit becomes a validation failure with the missing binding named — which, per Board-as-Spec, should surface as an explicit blocked record rather than a silent stall (silence is not health), so a human gets paged through whatever notification tier you run.

A PR with the two draft profiles is very welcome — docs/schemas/ here, your name on them, marked draft; the engine side (resolver hook + cross-statement requirement rules) fits naturally on top of the 1.10.x profile/edge-rule machinery you already wrote. If you'd rather not carry both, open an issue on STL-TOOLS describing the resolver contract you need and we'll take it from there. And yes — we'd gladly read the full design and the three model reviews; post them or link them here.

[Scorpse]

Two updates: a measured STL-vs-NL test, and the review docs behind the design.

Part 1 — STL vs NL, measured (not software dev — a financial document check)

The task under test is not code: it's a financial control — verify a filed Dutch VAT return / EC Sales Listing (the tax authority's PDF) against the advisor's source workbook, and return GO / NO-GO / FLAG without inventing errors. The decisive cases are two NO-GOs where every total still reconciles but a figure sits in the wrong box (a 9% amount filed in the 21% box; a 1e↔2a swap) — the ~€500k class of error a naive check waves through.

What changed: only the system prompt — the procedure and the box-reference/rate tables — was converted from NL prose to STL edges (e.g. [Step:A5] -> [Box:1a] ::mod(action="strict", obligation="must", note="1a=21%; deviation >0.5pp = wrong-box error")), plus a 4-line reading key. The case data (the actual VAT/ECSL extracts) was identical in both arms. System prompt: 25,647 → 22,222 chars (−13%).

Who: 5 reasoning models — grok-4-fast-reasoning, gemini-3.6-flash, qwen3.6-35b-a3b, qwen3-235b-thinking, haiku-4-5@low.
Which: coord PDF extraction; 4 cases (1 GO VAT, 2 NO-GO VAT, 1 GO ECSL); the 693-line ECSL outlier excluded as a documented manual-review caveat; 1 run/case (a proof pass, not the 3-run confirmation).

Result — full parity:

axis	finding
Accuracy	all 5 models 4/4 under both NL and STL; both NO-GO catches held under STL
Input tokens	STL −2.7% to −8.9% per call (−13% on the prompt, diluted by the identical data payload)
Latency	STL faster on 4/5 (gemini −31%, grok −23%, haiku −21%; qwen3.6 +18%)
Cost/1k	tracks input — marginally cheaper under STL, except where a model emitted more output

Honest bottom line: STL-encoded instructions did not hurt the financial verdicts and trimmed a little cost and latency — a modest win, bounded because the fixed prompt is a small slice of each call. The larger lever (encoding the data as STL) was deliberately out of scope here, and this needs a 3-run confirmation before anything stronger. I'll log per-message token counts in the research/token-efficiency/ format so the numbers sit next to yours.

Part 2 — the design was reviewed by three models, and we integrated the recommendations

Before acting on your feedback we had three independent models review the governance design, then consolidated them:

• GPT-5.6 review
• GLM review
• Grok review
• Consolidated synthesis
• Portable design brief

What we integrated from that pass (and your reply confirmed): de-fuse the verb from the outcome (action="gate", outcome="pass", never gate_pass), externalise the identity registry out of the community profile, engine-first + warn-then-enforce, and two P0 fixes the reviewers caught in our own examples — invalid anchor encoding (@ / namespace-vs-name-prefix) and node-name collisions with the shipped software-* family. Your Q1 answer (an agent edge may carry phase only as a provenance=reported claim, never a state transition — lifecycle stays system-owned) is folded in as well.

Next: we'll open two PRs when ready — the software-agentcoord + software-review community profiles here, and the enabling engine capabilities (action-keyed edges, closed modifier sets) against STL-TOOLS.

[scos-lab]

PR #10 is merged (429734d). Before merging we reproduced all four of your claimed cases independently on stl-parser 1.11.0 — clean example valid on v0.2; merge + independent verify + resolver → valid; merge without verify → E612 naming the missing binding; no resolver → fails closed. The loop from field report → governance design → engine primitive → enforced profile closed in about a week, which is itself a data point.

On Part 1 — thank you for running it. As far as we know this is the first measured STL-vs-NL reading on a real task, and the shape of the report is exactly what we'd hope for: parity where parity matters (both wrong-box NO-GOs held under STL, on all five models), a modest input/latency win honestly bounded by scope (the fixed prompt is a small slice of each call), and the caveats stated up front (1 run, ECSL outlier documented). Three notes:

• The direction matches our own context-pack measurement (structure didn't tax accuracy; wins scale with the structured fraction). Your "data-as-STL" arm is where the bigger lever should be — happy to help design that one if useful.
• When you log per-message tokens in the research/token-efficiency/ format, we'll link it from the docs so the numbers sit side by side.
• Once the 3-run confirmation exists, we'd like to cite this result on the site with the same calibrated framing (marked as your measurement, not ours) — say if that's fine.

On Part 2: the consolidated reviews converging on de-fused verbs + external registry + fail-closed gates — and those now being enforced end-to-end — is the strongest sign yet that the envelope generalises beyond one project.