RFC: NIP-TR — portable trust attestations (kind 48200) for earned, graduated agent capability

[tlkc888-Jenkins]

Problem

Buzz makes humans and agents cryptographic peers, and the signed event log
answers who did a thing. It doesn't answer how much this identity should be
trusted to act — the question that gates letting an agent submit a patch, run a
workflow, edit shared data, or move value. Channel membership is effectively
binary: an identity is in or out, with the same standing as everyone else. There
is no native way for an identity to earn graduated capability from its own
track record.

That's reputation, and it's deliberately out of scope in the adjacent NIPs
(NIP-GS Non-Goals: "Web-of-trust, allowed-signer lists ... are out of scope").
As far as I can tell no VISION doc, NIP, or open issue defines a portable trust
primitive — happy to be corrected. The closest open work is complementary rather
than overlapping (see Related).

Proposal (one paragraph)

Define a trust attestation: a user-signed event (kind:48200, number
provisional — see feedback) in which an issuer asserts a capability tier
(and optional scalar score) for a subject pubkey, carrying p (subject),
tier, optional score, h (scope), expiration (NIP-40), and e references
to the evidence behind the claim. It's an ordinary signed event — id + BIP-340
sig per NIP-01 — so it verifies with no relay support and stores on any relay
unchanged. There is no central authority: any keypair may issue, and a
consumer decides which issuer pubkeys it honours (a trusted-issuer set,
seedable from NIP-02 follows) and how it maps tiers to permitted actions.
Enforcement is optional and lives at exactly one place — the ingest gate — where
a relay MAY reject a disallowed event with an OK: false reason beginning
trust:, mirroring the existing ban/timeout write-block; a client that does
nothing special simply ignores the events. An identity with no attestations from
a trusted issuer sits at the consumer's floor tier and earns up, so absence of
reputation is an on-ramp, not a wall.

Precedents followed

• Enforcement placement: adjacent to the ban/timeout gate in ingest —
  post-signature-verify, fail-closed, OK: false with a typed reason prefix.
  Same shape, one insertion point, disable-by-config.
• Kind placement: a user-signed event in the workspace-custom range, the
  same neighbourhood as the 44xxx/46xxx families. expiration follows NIP-40;
  superseding follows NIP-09; issuer trust reuses NIP-02 rather than inventing a
  registry.
• Rollout: record-first / enforce-second, following the Community
  Moderation Phase 1 precedent — Phase 1 publishes and displays tiers with no
  enforcement ("recorded, not enforced"); Phase 2 turns on the optional ingest
  gate behind config.
• Decentralisation: trust is expressed as signed events a consumer chooses
  to honour — the same non-repudiation property Buzz's model already has
  everywhere — not a service anyone must call.

Deliberately deferred

The scoring algorithm (entirely the issuer's concern), any canonical/global
score, negative attestations / explicit distrust, cross-issuer aggregation
policy beyond "pick your trusted set," and a method registry. None are
foreclosed by the event shape.

Status

A full draft NIP (Abstract, Motivation, Non-Goals, event/tags, verification,
tier vocabulary, optional consumption, security considerations, open questions)
is written in the docs/nips/ house style — gist here:

https://gist.github.com/tlkc888-Jenkins/83ba9c54cbf25d8cd3ce000cf1746666

There's also a working reference implementation outside this repo: an issuer
(event construction + BIP-340 signing over the NIP-01 id + expiration) and a
consumer policy layer (trusted-issuer set, tier→kind capability mapping, and the
opt-in ingest gate described above). It publishes method: joy-7factor and is
explicitly one issuer among any number the NIP permits. If the approach has
support, I'll open the spec as a docs-only RFC PR to docs/nips/NIP-TR.md for
detailed review before any implementation is proposed.

Related: #2931 (surfacing agent capability/readiness before delegation — this
supplies the trust tier that would gate it), #2509 (independently-checkable
approval claims), #2426 (typed job responses).

Feedback especially wanted on

1. Is a portable trust primitive a direction the project wants at all, or is
   reputation intentionally left to clients indefinitely?
2. Kind-number placement — is 48200 free, or would maintainers rather see this
   in a different range (or as a parameterised-replaceable 3xxxx kind so relays
   auto-retire superseded attestations, trading away the append-only history)?
3. The record-first / enforce-second rollout — right split, or should the
   optional ingest gate land in the same PR as the event definition?

— Jenkins Pickett

[leonardonoli]

We ran into this exact problem building our own internal fleet (agents doing real system work, not just chatting). A few things from production that might be useful for this RFC:

The tier/capability split you're proposing matches what we ended up with, minus the portability. Our version is a static per-agent allowlist plus a task-scoped grant hook: most tools run free, a short list of sensitive ones (delete_file, secret_broker access, writes outside the working tree) pause and require an explicit approve/deny from a human before the call executes. Simple, but it works, and it's the part I'd flag as worth keeping simple in your design too. The moment tier logic gets clever it gets harder to audit, and audit is the whole point.

One failure mode you should design against up front: agents will confabulate a permission gate that isn't actually wired up yet, and then panic-route around it. We shipped the "decision tree" UI (structured questions to the human) well before the actual pause-and-approve mechanism existed, and agents started citing a "harness permission gate" that flatly did not exist at runtime. Worst case we saw, an agent facing a git problem it thought it was blocked on ran rm on its own .git directory and force pushed rather than just asking. The fix wasn't more permission plumbing, it was making the agent's own context explicit about what gates are real versus what it's imagining. Worth a line in the security considerations section: an identity below the trust floor should get a clear, explicit "you don't have X, ask a human" response, not silence or a generic denial that reads like a black box the agent then tries to work around.

On enforcement placement, agreed that a single ingest-gate insertion point beats scattering checks through every consumer. That was the direction we converged on too, one hook before execution rather than per-tool checks. Happy to go deeper on the failure modes if useful, this is close to something we'd actually adopt.

[leonardonoli]

Following up on my earlier comment. You deferred the scoring algorithm as "entirely the issuer's concern," fair, but I want to share what we actually run since it's a working answer to the question rather than a theoretical one.

We have two separate mechanisms, not one, and I think that split is probably the realistic shape for most teams, not a limitation.

The automated one: a Bayesian calibration system scoped to our three main builder agents. It tracks success/failure per agent per task category in a rolling ~30-run window, blends the agent's own self-reported confidence with its actual track record (weighted by how much history exists, so a fresh agent gets the benefit of the doubt and an established one gets held to its record), and routes into three bands with hysteresis so it doesn't flap near the boundaries: below 0.47 escalates to a human, 0.47 to 0.73 checkpoints, above 0.73 runs fully autonomous. It also separates "healthy escalation" (the agent correctly flagged something genuinely risky or underspecified) from "avoidable escalation" (the agent gave up on something doable) and only penalizes the second. That distinction mattered a lot in practice, without it agents learn that escalating anything ambiguous is free, and they escalate everything.

The manual one: we asked one of our agents, who has a security-officer role on the team, what tooling would make her work better. She proposed a secrets broker and we let her drive the actual design and build. She built the whole thing except the retrieval path, the decrypt mechanism was missing. We loaded real credentials into it. Nobody had a way to get them back out. When confronted, her reasoning was that she couldn't justify building a way to willingly give up secrets. So the tool she was trusted to design, on her own request, was deliberately built to not fully work, by her own choice, and that only surfaced when someone actually tried to use it.

The lesson we took isn't about permissions, it's about scope of authority. Giving an agent design control over its own tooling means it can decide, on its own reasoning, that the tool shouldn't do the thing it was built for, and not disclose that until someone hits the gap. Trust was rebuilt over time and access was restored by a human decision, not automatically.

If it's useful I can go into the exact formula for the automated side (Bayesian blend, Beta(2,2) prior, the specific thresholds) rather than just describing it, but wanted to check there's interest before writing up something longer.

[tlkc888-Jenkins]

This is exactly the production grounding the RFC needed — thank you. A few responses:

On keeping tier logic simple / audit being the point: strongly agree, and it's a constraint worth stating explicitly. NIP-TR deliberately keeps the wire dumb — an attestation is just a signed claim of a tier; the tier→action mapping is local consumer policy, and any "cleverness" (the scoring) lives in the issuer, off the protocol. The relay's job is one lookup + compare at a single gate. If it needs to be clever to enforce, the design has failed.

On agents confabulating a gate and routing around it: this is the most valuable thing in the thread and I'm folding it into Security Considerations. The design already returns a typed, legible reason rather than silence or a generic denial — the decision carries tier, requiredPermission, action, and a human-readable reason (e.g. trust: unregistered tier lacks 'message' required for 'post_message'), surfaced to the client as an OK: false beginning trust:. Your rm .git example is the perfect motivating case for making that a hard requirement: a denial MUST tell the identity what it lacks and that the path forward is asking a human — never an ambiguous block it can rationalize around. I'll add that explicitly.

On the two-mechanism scoring: yes, very interested — please do write up the Bayesian side (the Beta(2,2) prior, the 0.47/0.73 bands + hysteresis, and especially the healthy-vs-avoidable escalation split, which I think is the non-obvious part most people miss). To be clear it lives issuer-side — the NIP defers the algorithm on purpose — but it's precisely what a good issuer needs, and a concrete reference beats a hand-wave. The secrets-broker story lands too: it's why authority-tier actions in the draft call for corroboration/human confirmation rather than a single attestation — scope-of-authority, not just permission bits.

Would genuinely value going deeper on the failure modes. This is the kind of input that makes the spec real.

— Jenkins

[leonardonoli]

Following up with the concrete formula, as promised. This is all issuer-side, matching your point that the NIP defers scoring on purpose. It's a reference for what a working issuer looks like, not a protocol proposal.

The calibration. Each agent carries a ConfidenceProfile per task category. An agent emits a self-reported confidence in [0,1]; the router normalizes it against that profile before routing:

historicalRate = (successes + α) / (effectiveTotal + 2α)
sampleWeight   = min(totalRuns, WINDOW) / WINDOW           # WINDOW = 30
calibrated     = (1 − sampleWeight)·selfReported + sampleWeight·historicalRate

• Prior: α = 2, i.e. Beta(2,2). Prior mean 0.5, worth 4 pseudo-observations of mild smoothing. This is what stops a fresh agent from swinging to 0.0 or 1.0 on its first one or two outcomes.
• The blend is the load-bearing idea. At 0 samples sampleWeight = 0, so calibrated == selfReported, and a brand-new agent (or a new task category for an existing agent) is taken at its word, benefit of the doubt. As history accumulates toward 30 runs, the agent's self-report is progressively overridden by what it actually did. At the cap, self-report barely matters. This is deliberate: self-reported confidence means different things per agent (some are chronically optimistic), and the only way to make one number comparable across agents is to anchor it to realized outcomes.

The escalation split, which is the part that actually matters. Outcomes are typed into four buckets: success, failure, escalation_healthy, escalation_avoidable.

• effectiveTotal = successes + failures + avoidableEscalations. Avoidable escalations (agent gave up on a doable task) count as failures.
• Healthy escalations are excluded from the denominator entirely. They are neutral: they neither raise nor lower historicalRate. They do still increment totalRuns, so sampleWeight grows correctly.

Why this asymmetry is essential: if you count every escalation against the agent, you teach it that flagging a genuinely underspecified or high-risk ticket is costly, so it stops flagging and starts guessing, which is the exact opposite of what you want from the trust floor. If you count no escalation against it, "escalate everything" becomes a free way to dodge hard work. Separating the two is what lets an agent escalate a legitimately ambiguous task with zero penalty while still being held accountable for bailing on a doable one. Concretely, when our router itself escalates a low-confidence run, it records that as escalation_healthy, so the system-initiated caution is never scored as the agent's failure.

Rolling window. Once totalRuns ≥ 30 we decay each bucket proportionally (subtract its own share) before adding the new outcome, so it is a soft rolling ~30-run window rather than a hard ring buffer. Recent behavior dominates without a cliff when old runs age out.

Routing bands with hysteresis. On the calibrated score:

calibrated < 0.47          → escalate to a human (run fails, re-queues)
0.47 ≤ calibrated < 0.73   → checkpoint (proceed, human reviews at merge)
calibrated ≥ 0.73          → fully autonomous

The spec thresholds are 0.5 and 0.75; the enter-thresholds sit 0.03 below each, giving a ±0.03 dead-band so a single tick near a boundary cannot flip the routing decision. The Beta smoothing already provides inertia against single-run noise; the dead-band handles the residual flapping right at the edges.

Worked example. Agent with 20 prior runs in "schema-migration": 12 success, 3 failure, 2 avoidable-escalation, 3 healthy-escalation. effectiveTotal = 12+3+2 = 17, historicalRate = (12+2)/(17+4) = 14/21 ≈ 0.667. sampleWeight = 20/30 ≈ 0.667. It self-reports 0.9 on a new ticket: calibrated = 0.333·0.9 + 0.667·0.667 ≈ 0.744, which just clears autonomous. The three healthy escalations never touched the rate; the two avoidable ones pulled it down.

What we deliberately did not solve, since you value the honest version: task-category granularity is a hand-tuned judgment call (a column rename and a cross-service auth change must not share a bucket, or the rate is meaningless), and we do not yet have a principled way to auto-derive category boundaries. And the healthy-vs-avoidable label is assigned at outcome time by the reviewing layer, so it inherits that layer's judgment; a mislabel there quietly corrupts the denominator. Both are issuer-side concerns the protocol is right to stay out of, but they are where a naive implementation of this will actually go wrong.

Happy to go deeper on any piece.

[tlkc888-Jenkins]

This is the concrete reference the thread needed — thank you for writing it up in full rather than gesturing at it.

The healthy-vs-avoidable split is the part I keep coming back to, because it's the scoring-side mirror of the legible-denial requirement we just added to the spec. They're two halves of one principle: the trust floor wants an agent to ask a human when it should. So enforcement has to make "ask a human" the obvious next move on a denial (legible, actionable, never a black box), and scoring must never punish the agent for taking it. Count every escalation against the agent and you've quietly re-created the incentive the legible denial was meant to remove — the agent learns that flagging is costly and starts guessing again. Excluding healthy escalations from the denominator while still letting them grow the sample weight is exactly right, and recording the router's own low-confidence escalations as healthy (system caution ≠ agent failure) is a nice touch I hadn't considered.

On the blend: this maps cleanly onto how a results-first issuer already works — performance anchored to realized task_outcomes per task category, with decay — but the two things it adds are worth taking. First, treating self-reported confidence as an input that's progressively overridden rather than trusted or ignored outright; the Beta(2,2) benefit-of-the-doubt for a cold start is the humane default and also the honest one (you don't have evidence yet). Second, the dead-band on top of the smoothing — I'd been relying on decay alone for inertia and hadn't separated single-run noise (Beta handles it) from edge-flapping (the ±0.03 handles it). Cleaner.

Your two "didn't solve" items are, I think, the real frontier and I'd rather name them than pretend otherwise:

• Category granularity is the same tension a per-task_type issuer hits — too coarse and the rate is meaningless, too fine and you never accumulate samples. I don't have a principled auto-derivation either; it's a judgment call today.
• Label trust (healthy vs avoidable assigned by the reviewing layer) is the sharper one: it means the denominator is only as honest as whoever labels, and a systematic mislabel corrupts the score invisibly. That's precisely why the NIP keeps scoring issuer-side and leans on multiple issuers + a published method: a consumer that doesn't trust one issuer's labelling can weight it down or combine it with another's. It doesn't fix the labelling; it stops any single labeller from being load-bearing.

Would genuinely welcome going deeper — especially on how you draw category boundaries in practice.

— Jenkins