Repository navigation
TRP 0.5.3 ships with Verifiable Trust Receipt (VTR) version 0.5. We list every behavior that differs from 0.5.2 under Changed and Fixed.
Added
- Verifiable Trust Receipt (VTR) version 0.5 in
vtr/, with the
specification, a JSON Schema, a Python reference library and emitter,
independent Python and Node.js verifiers, Merkle batch signing under RFC 9162
and 45 test vectors that both verifiers must agree on.standingis an object with aresultbeside the band and the response.
The result tells an incomplete evaluation apart from a failing one. A
receipt whose required evidence is missing, or whose hard rule is
inconclusive, carries the resultincompletewith a null band and a null
response. A triggered hard rule is a known violation, so it gives
does_not_meeteven when other evidence is missing. A critical
condition, a value in the critical range, or a signal missing with
on_missing: critical, sets the worst band whatever else is missing, so
it also givesdoes_not_meetbesides missing evidence whenever that band
does not meet the profile. A signal with weight 0 never changes the
standing, so its absence never givesincomplete.result_digestis the digest of the evaluator output with its
evaluationmember removed. A verifier with the profile and the evidence
re-runs either reference evaluator and compares one digest.- Coverage counts a requirement as checked only when its outcome is
something other thaninconclusive, so a hard rule whose field is
present with the wrong type does not count as checked. - The valid vectors carry every standing result and cover every kind and
every mode, including a hard rule field of the wrong type and a triggered
hard rule or a critical signal beside a missing signal. The invalid
vectors include an unknown key identifier, a missing subject, a coverage
outside 0 to 1, an incomplete standing altered tomeetsand a triggered
hard rule or a critical signal reported asincomplete.
- The TRP gate signs a VTR for every decision when given a receipt key.
profile_from_dictintools/evaluate.pyandprofileFromObjectin
tools/evaluate.js, so a parsed profile can be evaluated without a file.
The Node.js evaluator can now be loaded as a module.adapters/gate/trp_gate.py, one TRP gate that judges each AI agent tool
call before it runs and each tool result before the model sees it, with a
decision log chained by SHA-256 and a session file shared across servers and
frameworks.adapters/mcp/trp_mcp_checkpoint.py, a TRP checkpoint in front of any Model
Context Protocol (MCP) server on stdio, with the reference profile
adapters/mcp/profiles/mcp-tool-checkpoint.trp.jsonand policies for the
official filesystem and fetch servers.- Adapters for LangChain, LangGraph, the OpenAI Agents SDK, Google ADK, CrewAI
and AG2 inadapters/frameworks, each checked by running the real framework
with a scripted model. - An evidence sample can name the profile it runs against in
_profile.
Samples without it still run againstfull-featured.json. - Conformance tests for
on_missing: ignore, rounding ties, a critical
signal beside a missing signal or an inconclusive hard rule, and weight 0
signals (48 tests, up from 42).
Changed
- Both reference evaluators honor
on_missing: criticalwhen they assign the
band. Section 3.3 treats such an absence as a critical condition, so the
standing is the worst band, the same as a value in the critical range. In
0.5.2 the evaluators added a penalty that no output was used and left the band
unchanged.tests/evidence/missing-critical-signal.json, published in
0.5.2 with the expected standingtrustedand the responsefull, now
expectsuntrustedandsuspended.
tests/evidence/missing-ignored-signal.jsoncoverson_missing: ignore. - Both evaluators assign the band in a stated order of precedence. A
triggered hard rule comes first, then a critical condition, then an
incomplete assessment, then a warning. A value in the critical range takes
the same precedence as a signal missing withon_missing: critical, so a
critical condition sets the worst band whatever other evidence is missing.
The band is the same as in 0.5.2 for every input that does not involve
on_missing: criticalor weight 0, and
tests/evidence/critical-signal-and-missing-signal.jsonand
tests/evidence/critical-signal-and-inconclusive-rule.jsonpin it. - A signal with weight 0 is informational in both evaluators and never moves
the band, including a signal whose value is in the critical range or that
is missing withon_missing: critical. In 0.5.2 such a signal could set
the worst band or the middle band. Section 3.3 states the rule once, where
weight is defined.tests/profiles/valid/zero-weight-signal.jsongains a
weight 0 signal declaredon_missing: critical, and
tests/evidence/zero-weight-critical-value.jsonand
tests/evidence/zero-weight-missing.jsonexpectgoodagainst it. result_hashin both evaluators is the SHA-256 of the RFC 8785 canonical
JSON form of the output without itsevaluationmember, so the Python and
Node.js evaluators write the same hash byte for byte. In 0.5.2 the Python
evaluator hashedjson.dumpswith sorted keys, and the Node.js evaluator
passed the top-level keys toJSON.stringifyas a filter, which dropped
every nested member, so the two hashes differed whenever the output listed
a signal or a rule. Aresult_hashrecorded under 0.5.2 can differ from
the 0.5.3 value for the same input. For an evaluation with no drift
history, it equals a VTRresult_digestin hexadecimal. The
cross-evaluator check in continuous integration (CI) now compares
result_hashon every profile and evidence pair.- A non-finite number has no JSON form, so both evaluators now stop with an
error when the output would carry one, such as a scored signal value that
converts to not a number (NaN). In 0.5.2 both kept going and wrote the
value asNaNornull. - The specification header, the schema title and description, and the
README give version 0.5.3, and the VTR library names the evaluator version
0.5.3. The schema$idand thespec_versiona profile declares stay
0.5.
Fixed
tools/evaluate.jsrounds severity, penalty, drift moving averages, and generated sample values half to even on the exact value of each number, the way Python'srounddoes. It used to round a tie such as 0.03125 up, and near ties such as 0.95075 could round the other way, so the two evaluators could disagree.tests/evidence/rounding-tie.jsonlands on both cases.
Removed
- A compiled Python file committed by mistake, with a
.gitignoreto keep such files out.