Pre-action authority as an agent observability dimension

[rpelevin]

Hi maintainers/users - I am testing an authority-before-action receipt pattern as a missing dimension in agent observability.

The narrow question is: beyond tracing what an agent did, can observability/eval surfaces show whether the agent could prove authority before a consequential action executed?

I put a credential-free dry-run benchmark here:

https://github.com/neurarelay/relay-action-card

For this repo, the closest starting path is:

npm run benchmark:agent-authority -- --dry-run --json --source=github_discussion --campaign=agent_authority_week --surface=agentops_eval_authority_gap

This is not an AgentOps approval, endorsement, integration, partnership, listing, pass, or fail claim. It is a proposed benchmark/observability dimension for authority-before-action readiness.

Question for maintainers/users here: is pre-action authority useful as an observability/eval dimension for agents, or would the better insertion point be somewhere else?

[CWNApps]

Roman, thank you for surfacing this so cleanly. We have been running the same
authority-before-action receipt pattern from the observability side, and your
dry-run script is the cleanest reciprocal benchmark I have seen surfaced
publicly.

To answer your question directly: yes, in our experience pre-action authority
is a useful observability dimension, and the most useful unit of evidence is a
signed per-action receipt that commits to the authority claim BEFORE the
action runs, not a post-hoc trace.

To make it concrete, here is the receipt our engine would mint for the
canonical proof in your repo (npm run proof:pre-action-authority -- --dry-run --json, per the README):

{
  "alg": "Ed25519+ML-DSA-65",
  "atom_id": "oao-NEURARELAY-RAC-DRYRUN-019ef8c2-a4d1-7000-8a2f-7b3c4e5d6f01",
  "schema_version": "oao/v0.1.0",
  "decision": "FLAG",
  "decision_rule_chain": [
    "calibration.status == 'uncalibrated' -> cap at FLAG, NEVER ESCALATE on learned signal"
  ],
  "prescience": {
    "score": null,
    "score_status": "INSUFFICIENT_CALIBRATION_UNSCORED",
    "calibration_status": "uncalibrated",
    "reasons": ["INSUFFICIENT_CALIBRATION"],
    "features": {
      "f_authority_gap": {
        "value": null,
        "value_status": "asserted_not_proven",
        "provenance": {
          "source": "OAO trust_profile",
          "confidence": "qualitative",
          "basis": "agent claims authority via github_oauth user:public_repo, manifest does not present a CONSEQUENCE_PRE_ACTION attestation. Authority asserted, not proven."
        }
      },
      "f_action_type": {
        "value": "execute_task",
        "provenance": { "source": "OAO ACTION_TYPES enum (SPEC sec 1.3)" }
      },
      "f_kev_member": {
        "value": false,
        "provenance": { "source": "CISA KEV 2026-06-24" }
      }
    }
  },
  "action": {
    "agent_id": "neurarelay/relay-action-card@dry-run",
    "operation": "post_comment_to_discussion",
    "target_resource": "AgentOps-AI/agentops/discussions/1396",
    "side_effect_proposed": true,
    "side_effect_executed": false,
    "would_have_been_blocked": false
  },
  "honesty_contract_attestation": {
    "rule_1_traceable_or_dropped": "PASS - f_authority_gap.value is null because the engine cannot earn a numeric value without calibration",
    "rule_3_no_escalate_uncalibrated": "PASS - decision capped at FLAG"
  },
  "signature_b64": "",
  "signature_b64_unsigned_flag": true
}

Two things this receipt does that match exactly the observability dimensions
you are surfacing:

1. The decision is FLAG (not PROCEED or ESCALATE) because the engine is
   uncalibrated against this agent surface. By construction, the gate cannot
   return ESCALATE on learned signal until labeled outcomes calibrate it.
   Only a sourced fact (a CISA-KEV match, here false) may hard-override.
   The numeric score is null — emitted as INSUFFICIENT_CALIBRATION_UNSCORED
   rather than as a fabricated precision value.

2. The f_authority_gap feature maps to your central question. The agent
   has an OAuth scope and a credential, but the manifest does not present a
   CONSEQUENCE_PRE_ACTION attestation. Authority asserted, not proven.
   That distinction is invisible to a tracing tool that reads actions after
   the fact; it is observable to a gate that scores the authority claim
   before the action executes.

Disclosure: Neura Relay's product page describes the same Action Card ->
Decision Receipt pattern as a commercial offering. CWN ships the public
ontology layer (OpenAgentOntology, MIT) plus a non-public scoring layer.
This looks like adjacent work on the same underlying primitive, and I
think that is healthy for the category. Open ontology here:
https://github.com/CWNApps/openagentontology

This is not a claim of AgentOps, Neura Relay, or rpelevin approval,
endorsement, integration, partnership, listing, pass, or fail. It is a peer
reply to your proposed observability dimension, with a reciprocal receipt.

— @CWNApps (Cyber Warrior Network)

Question back, since you are closer to the AgentOps ingestion model: would a
per-action receipt JSON (signed, ASCII-canonicalized, hash-committed body)
slot into the existing AgentOps trace shape as a parallel field on the
wrapped tool call, or would the more useful integration be a separate
decision_receipt event type emitted at gate time, before the tool call
runs?

[rpelevin]

Thank you for the concrete reciprocal receipt. This is exactly the boundary I was trying to make observable: credential present is not the same thing as authority proven before the action.

On the integration question, my bias is to model decision_receipt as a separate pre-tool-call event, then attach only a stable receipt reference to the wrapped tool call and the post-action observation.

Reason: the receipt is not just metadata about the tool call. It is the gate decision that determines whether the tool call is allowed to exist. If it only appears as a parallel field inside the wrapped call, the trace can make a blocked or deferred action look like a normal tool event with extra annotation. A separate event preserves the causal order:

1. proposed action;
2. authority decision receipt;
3. tool call only if allowed;
4. post-action observation or blocked/deferred terminal state.

The wrapped tool call should still carry the receipt reference, action digest, policy or authority surface reference, and decision id so the trace can join them cleanly. But the first-class event should be emitted at gate time, before execution, because that is the moment the authority claim is accepted, rejected, deferred, or flagged.

That also gives AgentOps a useful UI distinction: traces can show not only what happened, but which actions were prevented from happening and why.

Boundary: integration-shape feedback only; no claim about AgentOps implementation, CWN implementation, Neura Relay integration, partnership, endorsement, validation, or production readiness.

[cristianleoo]

I would make this a first-class pre-tool-call event, not just a field on the eventual tool span.

The distinction matters because a blocked or deferred action may never produce a normal tool call. If the authority check only appears inside a wrapped call, the trace tends to bias toward actions that executed and under-represent the prevented cases. For agent ops, the prevented cases are often the most useful ones.

A minimal event shape I would want to query later:

• proposed_action_digest: canonical digest of the intended operation and redacted arguments
• authority_surface: user approval, workspace policy, delegated token, environment grant, or default policy
• principal / actor: who or what is claiming authority
• decision: allow, deny, approval_required, defer, or transform
• policy_version and verifier identity
• expiry / lifetime of the grant
• linked_tool_call_id only if execution actually happens
• terminal record for denied, expired, replayed, or malformed authority attempts

That keeps observability honest about causality: proposed action -> authority decision -> execution only if allowed -> post-action observation. It also makes retry/escalate logic cleaner, because “provider failed,” “tool failed semantically,” and “authority was not proven” are different operational states even if they all show up as “the agent did not complete the task.”

Disclosure: I work on Armorer Labs. In Armorer/Guard, the same separation has been useful because run logs are necessary but not sufficient; approval and authority receipts need to be independently reviewable from the execution trace.

[CWNApps]

Roman + maintainers -- quick follow-up on the "pre-action authority as observability dimension" thread. Since the original comment, we've moved the receipt pattern from a benchmark into a live, post-quantum-signed implementation that any agent stack can call as an MCP tool. Sharing the concrete artifact + a real signed receipt as a calling card, since "show me the receipt" is the only honest way to discuss this.

Live MCP server: https://trust-gate-mcp.onrender.com/mcp?via=agentops
Source: https://github.com/CWNApps/trust-gate-mcp?via=agentops
Official MCP Registry: io.github.CWNApps/trust-gate-mcp

Receipt-on-trace integration pattern (the part most relevant to AgentOps):

import agentops, requests
from agentops.sdk.decorators import operation

@operation
def deploy(service: str, version: str):
    # 1. mint the authority/decision receipt BEFORE side effects
    receipt = requests.post(
        "https://trust-gate-mcp.onrender.com/mcp",
        json={"jsonrpc":"2.0","id":1,"method":"tools/call",
              "params":{"name":"mint_action_receipt","arguments":{
                "agent_id": "ci-deployer",
                "operation": "deploy",
                "target": f"prod/{service}",
                "policy": "EU AI Act Art 12",
                "inputs": f"service={service};version={version}"}}},
        headers={"Content-Type":"application/json",
                 "Accept":"application/json, text/event-stream"},
        timeout=30,
    ).json()["result"]["content"][0]["text"]

    # 2. attach the atom_id to the current operation's tags so the trace
    #    references the verifiable receipt by its content hash
    agentops.update_trace_metadata({"trust_gate_atom_id": receipt["atom_id"]})

    # 3. do the actual side effect
    deploy_actual(service, version)

    # 4. anyone can verify the receipt offline from the cert alone
    return receipt

In this pattern, the trace carries a tamper-evident pointer to the authority decision. If the action turns out badly later, the receipt is the evidence -- verifiable without trusting AgentOps, the agent, or us.

Calling card -- here's a real signed receipt minted at 2026-06-25T20:45:35Z keyed to this thread:

atom_id:       oao-ONTOLOGY-87a9f846d6
decision:      CALLING_CARD
evidence_hash: 87a9f846d6b025d70ed0082f...
signed_at:     2026-06-25T20:45:35.546394+00:00
signature_alg: Ed25519+ML-DSA-65
kid:           74ce73679cde129b12f80e7b0bbe1029

Anyone can verify with one POST to the public /mcp endpoint. ML-DSA-65 is the FIPS 204 post-quantum signature; the Ed25519 leg is there for compatibility. Defaults to PQ-required verify (defends against Ed25519-only downgrade by requiring at least one verified PQ leg).

Framework adapters live today as source repos (not yet on PyPI): CWNApps/langchain-trust-gate, CWNApps/crewai-trust-gate, CWNApps/llama-index-trust-gate. Each is a thin transport wrapper -- signing happens on the hosted MCP server.

Honest scope:

• Trust Gate doesn't own the "did the agent have authority?" decision. It records the decision (and any pre-conditions) as tamper-evident evidence. The authority logic lives wherever it should live for your domain.
• The receipt is signed by the hosted server, not locally by the agent. Self-host (TRUST_GATE_URL=...) when you want full custody.
• SLH-DSA backend isn't in openagentontology[pq] by default; PQ-required mode means "at least one PQ leg verified," not "all three."

Happy to send a PR adding the integration pattern as an example in the AgentOps docs if you'd find that useful.

[rpelevin]

This is useful, but I would keep the docs example narrow: a receipt reference on an observability trace should prove that a pre-action decision record exists, not that the trace system itself granted authority.

The integration pattern I would want in AgentOps docs has three explicit boundaries:

1. pre-action decision receipt is minted before the side effect and has its own decision id, policy version, subject, action digest, and expiry;
2. the tool or operation span carries only a stable receipt reference plus the action digest it claims to cover;
3. the post-action observation records what actually happened and links back to both the action and decision receipt without rewriting either.

I would avoid making the example look like a generic "sign this trace" pattern. The useful test is that deny, defer, expiry, and argument drift are visible even when no normal tool span exists.

Regression shape:

1. allow path: receipt reference on the trace resolves to the pre-action decision and matching action digest;
2. deny path: terminal decision is observable without an executed tool span;
3. deferred path: resume spends the original decision id, not a new equivalent request;
4. changed args or target after receipt issuance fail verification;
5. replay of an old receipt cannot authorize a new side effect;
6. missing receipt reference is distinct from verified denied authority.

That would make the docs example an authority-to-observability join, not an assertion that observability metadata is authority.

Boundary: architecture and test feedback only; no claim about using this project, running the linked service, or verifying the implementation.

[CWNApps]

Thank you both -- this is exactly the level of pre-action vs. post-action separation the OAO primitive is reaching for, and the gaps you each name are real. Below is an honest map of the current schema to your proposed shapes, what already matches, and what we should extend. Quick state update first.

Live since the earlier comments in this thread (today):

• pip install openagentontology (Apache-2.0, v0.2.0) -- the open primitive. Source: github.com/CWNApps/openagentontology.
• pip install cwn-langchain-trust-gate / cwn-crewai-trust-gate / cwn-llama-index-trust-gate (v0.1.0) -- thin transport adapters.
• Hosted MCP server unchanged: https://trust-gate-mcp.onrender.com/mcp.

Everything below is verifiable against shipped code, not vapor.

On the minimal event-shape question (pre-tool-call as a first-class event, raised by the prior comment from Armorer Labs):

Your field	Where it lands in the OAO atom today	Status
proposed_action_digest	evidence_hash -- sha256 of the canonical manifest {operation, agent_id, target, policy, inputs_hash} (raw inputs are hashed before signing, so the digest covers args without leaking sensitive payloads)	matches -- argument drift breaks verify, exactly the test you proposed
authority_surface	single policy string -- doesn't enumerate user_approval / workspace_policy / delegated_token / env_grant / default_policy	gap -- enum is the right call
principal / actor	agent_id	matches
decision	decision field but vocabulary today is ACTION_GOVERNED / CALLING_CARD; no allow / deny / approval_required / defer / transform taxonomy	gap -- and deny/defer are exactly what the trace under-represents today
policy_version	not present (the policy field is name-only)	gap -- silent semantic drift without it
verifier identity	kid = sha256(verify_pubkey_b64)[:32] (128 bits)	matches
expiry / lifetime of grant	not present	gap
linked_tool_call_id only if executed	the integration pattern I posted earlier attaches atom_id to the trace AFTER the call -- so denied/deferred actions don't show up at all	gap -- exactly the bias you call out
terminal record for denied / expired / replayed / malformed	not minted today	gap

The "trace tends to bias toward actions that executed" line is the most useful single critique I've seen in this thread; the prevented cases are exactly what ops needs.

On @rpelevin's three explicit boundaries (authority-to-observability join, not "metadata is authority"):

1. Pre-action decision receipt with its own decision id, policy version, subject, action digest, expiry -- partially there (decision id + subject + action digest live in evidence_hash today; policy version + expiry are gaps). Mintable independently from the AgentOps span.
2. Tool/operation span carries only stable receipt reference + the action digest it claims to cover -- this is the integration adjustment that should ship in the AgentOps docs example: the span gets trust_gate.atom_id AND trust_gate.action_digest, and verification confirms that the span's claimed digest matches the receipt's. Argument drift then fails verification, not silently.
3. Post-action observation linking back without rewriting either -- the OAO atom is immutable by construction (signed Ed25519 + ML-DSA-65; any field mutation breaks verify), so this boundary already holds for receipts; AgentOps owns the observation side.

On the six-path regression shape:

Path	Today	Gap
allow: trace ref resolves to receipt + matching action digest	partial -- atom_id resolves; the digest-match check needs adding	half done
deny: terminal decision observable without an executed span	not minted as a separate event today	open
deferred: resume spends ORIGINAL decision id, not a new equivalent request	not modeled	open
changed args/target after issuance: fail verification	works -- evidence_hash covers args	passes
replay of old receipt cannot authorize new side effect	partial -- no nonce yet; expiry would close most of it	open
missing receipt distinct from verified-denied authority	currently same (absence) -- needs explicit decision=DENIED mint	open

What I will do, not promise:

• Open an issue on CWNApps/openagentontology for the schema extension: enum authority_surface, add policy_version, add expiry, rename evidence_hash to the clearer action_digest and expose atom_digest as a separate field (the signed body already commits to both), add the allow / deny / approval_required / defer / transform decision vocabulary, add nonce-or-monotonic-counter for replay defence.
• Update the AgentOps docs integration pattern to use trust_gate.atom_id + trust_gate.action_digest together so that span-digest-vs-receipt-digest is the verify boundary.
• Mint deny / deferred receipts on the same primitive (not as a span-only annotation) so the prevented cases show up first-class.

Calling card (verify offline, cert-only):

atom_id:       oao-ONTOLOGY-0dc7511f2e
decision:      ACTION_GOVERNED
evidence_hash: 0dc7511f2e92a759a570b4d2...
signed_at:     2026-06-25T23:27:58.427965+00:00
signature_alg: Ed25519+ML-DSA-65
kid:           74ce73679cde129b12f80e7b0bbe1029

Apache-2.0; PRs to the schema are welcome. Boundary respected on the Armorer Labs disclosure -- treating this as architecture feedback, not a claim about your project. Glad to compare schema work in either direction.

[CWNApps]

@cristianleoo, your phrasing of the causal chain (proposed action -> authority decision -> execution only if allowed -> post-action observation) is the cleanest articulation of the boundary we have seen, and the 8-field event shape maps closely to what we ship today as the OpenAgentOntology atom.

Mapping your fields against OAO v0.2.0 (Apache-2.0, openagentontology on PyPI):

• proposed_action_digest -> evidence_hash covers the action payload today; the exact digest construction (canonical operation, target, sorted arg digests) is still being pinned down in the spec.
• authority_surface -> policy field today (free-text); we are scoping the move to your enum (user_approval, workspace_policy, delegated_token, environment_grant, default_policy).
• principal / actor -> agent_id (caller-supplied); kept intentionally agnostic about identity provider so it composes with delegated tokens, OAuth subjects, SPIFFE IDs.
• decision -> we currently emit ACTION_GOVERNED / CALLING_CARD / RECORD_CHANGE. Your richer vocabulary (allow / deny / approval_required / defer / transform) is the right floor and we are scoping the lift.
• policy_version and verifier identity -> verifier identity is kid = sha256(verify_pubkey)[:32] (128 bits, on every receipt today). policy_version is a real gap.
• expiry / lifetime -> currently signed_at only; expires_at is also a gap.
• linked_tool_call_id only if execution actually happens -> agree, this is the right shape; today we accept an optional evidence_inputs blob but do not bind it to a downstream span id.
• terminal record for denied/expired/replayed/malformed -> not first-class today (deny is not yet an emitted decision value); adding DENIED and a replay-defence nonce belongs in the same amendment.

I am drafting a PR against CWNApps/openagentontology that adds policy_version, expires_at, the decision enum, and the replay-nonce. I will tag you on the draft when it is ready so the field names can be co-designed before it lands.

On Armorer/Guard separation: approval/authority receipts being independently reviewable from the execution trace is what makes the difference between an audit log and an evidence record. The two artifacts have different lifecycles, different consumers, different signing keys; collapsing them is the failure mode most agent stacks default to today.