RFC: MCP Reliability Evidence Standard — Draft v0.2.0

[aliengineering-byte]

MCP-RES Draft v0.2.0 is now available as a project-defined reliability-evidence standard and prerelease implementer kit.

MCP-RES remains independent of the official MCP specification. It is not official MCP conformance, an official certification program, a security certification, a guarantee, or an endorsement by the MCP project.

Why v0.2 exists

The v0.1 draft established a useful evidence envelope, but executable review found several places where a superficially correct verdict could overstate what had actually been tested. Most importantly, a negative test could reject for an earlier or unrelated reason and still look green. Producer-written execution booleans also lacked observation binding, scenario identity was not separated from the individual run, coverage of bypassed surfaces was implicit, and authenticity was not represented independently from content integrity.

v0.2 keeps the v0.1 normative tree immutable and introduces an explicitly versioned contract rather than reinterpreting historical results.

Evidence semantics

A v0.2 negative control binds the property under test, prerequisite reachability, intended and observed stop reason, oracle evidence, and optional deliberately wrong-reason mutant. Rejection is insufficient when the target property was never reached. Stable diagnostics distinguish property-not-reached, wrong-stop-reason, missing-prerequisite, vacuous-control, and surviving-mutant failures.

Consequential claims are derived from integrity-bound observations. Required surfaces are declared as instrumented, observed indirectly, intentionally bypassed, uninstrumented, unknown, or not applicable. An unknown or uninstrumented required surface prevents an unqualified PASS.

Scenario identity versus run identity

The scenario fingerprint captures the stable test contract: profile, protocol revision, subject/configuration digests, operation and fault classes, recovery and side-effect policy, resource limits, validator policy, and seed. It excludes timestamps, paths, hostnames, users, PIDs, and CI job IDs.

The execution-instance digest binds that scenario to one actual run, its runner and timing, observations, outcome, and artifact manifest. Equivalent reruns can therefore share a scenario fingerprint while remaining distinct executions; cross-scenario and cross-run substitution are rejected.

Integrity, authenticity, and trust

Content integrity, producer authenticity, trust policy, reliability result, and security certification are separate dimensions. A genuine runtime result may still be UNSIGNED_INTEGRITY_ONLY; signing does not promote its evidence class or rescue invalid evidence.

The optional offline-capable attestation envelope uses an in-toto statement/predicate separation with DSSE-style signature binding. Disposable test keys exercise altered payloads/subjects, wrong or missing keys, unsupported algorithms, expired/revoked/untrusted policies, identity mismatch, and a valid signature over invalid evidence. GitHub-hosted build and SBOM attestations protect the prerelease assets but are not a SLSA-level claim.

Protocol, transports, async work, and authorization

Profiles preserve exact protocol-revision and requirement-set identity and cover modern stateless behavior, server/discover, _meta, resultType, safe integers, MRTR/input_required, Tasks, subscriptions, progress, cancellation, Streamable HTTP interruption, stdio framing/lifecycle, cache behavior, extension negotiation, and MCP Apps boundaries. Roadmap-dependent behavior stays provisional or experimental.

The OAuth boundary profile uses only synthetic local/loopback fixtures. It covers resource/audience binding, metadata discovery, PKCE S256, redirect/state/mix-up/code binding, token-endpoint identity, CIMD policy recording, scope/refresh boundaries, token passthrough, SSRF resistance, bounded redirects, collision behavior, and proxy consent/client binding. It is reliability evidence, not a security certification.

Official-conformance attachment

MCP-RES can preserve sanitized output from the pinned official conformance tool as an attachment. The bridge records exact tool commit/version, requirement sets, modes, checks, expected failures, stale baselines, uninstrumented paths, warnings, and original-result digest. It never converts an incomplete or warned official run into an MCP-RES conformance claim. The published Windows reference attachment remains INCOMPLETE because its original libuv assertion is preserved.

Profiles and status

The machine-readable registry contains normative requirements, schema/vector digests, validator compatibility, ownership, history, and lifecycle state. Current runtime profiles are deliberately PROVISIONAL or EXPERIMENTAL; registry validation rejects duplicate or incompatible IDs, missing artifacts, undocumented status promotion, and mutation without a version bump.

Migration and independent implementation status

The deterministic v0.1 → v0.2 migration records v0.1 producer booleans only as LEGACY_SELF_ASSERTED; they cannot self-promote into stronger v0.2 evidence. v0.1 reproduction and dual-version CI remain active. The existing UTF-16 canonicalization stays authoritative for historical identity and v0.2 binding; RFC 8785 is evaluated as a separately named optional digest where the accepted domain proves cross-language agreement, not described as equivalent.

A Python validator with no JavaScript or ResiliReplay imports consumes only the public specification inputs and agrees 100% with the JavaScript implementation on the published decision, diagnostic-family, canonical-byte, and digest corpora. It is maintained in the same repository, so it is a second implementation but not external independence. Public field evidence contains four distinct pinned subject artifacts across Windows/Linux, multiple languages and transports; those subjects were tested by this project and are not adopters. Current independent adopters: 0.

Exact draft and release

• Source commit: https://github.com/aliengineering-byte/resilireplay/tree/fef71ab4d8b258910a948ff3c56e367f2d6d36a1/docs/standards/mcp-res/v0.2.0
• Normative draft: https://github.com/aliengineering-byte/resilireplay/blob/fef71ab4d8b258910a948ff3c56e367f2d6d36a1/docs/standards/mcp-res/v0.2.0/MCP_RES.md
• Implementer kit: https://github.com/aliengineering-byte/resilireplay/tree/fef71ab4d8b258910a948ff3c56e367f2d6d36a1/docs/standards/mcp-res/implementer-kit
• Draft prerelease: https://github.com/aliengineering-byte/resilireplay/releases/tag/mcp-res-v0.2.0-draft.1
• Gap ledger: https://github.com/aliengineering-byte/resilireplay/blob/fef71ab4d8b258910a948ff3c56e367f2d6d36a1/docs/standards/mcp-res/v0.2.0/GAP_LEDGER.md
• Independent implementation challenge: Independent MCP-RES implementation interoperability challenge #55

Feedback requested

Please focus technical feedback on wrong-reason controls and causal reachability, observation coverage, cross-language canonicalization edge cases, profile boundaries/status, official-conformance attachment semantics, migration behavior, operational budgets, and the evidence required before any stable 1.0 proposal. Counterexamples and independently produced public vectors are especially useful.

Do not submit credentials, private traces, production data, or destructive proofs. The published validators are inert evidence readers and do not execute submitted subjects.