feat: add client conformance tests for SEP-2575#270
Open
anubhav756 wants to merge 7 commits into
Open
Conversation
commit: |
cd07785 to
4592cc7
Compare
pcarleton
reviewed
May 13, 2026
Member
pcarleton
left a comment
There was a problem hiding this comment.
Thanks for tackling this. SEP-2575 is a big one. I ran /new-sep 2575 to build the full requirement-traceability YAML and opened it as #273 so we have a shared reference for both this PR and #271.
Coverage: the YAML has 12 client-side check: rows; this PR currently covers 3 (client-populates-meta, the version-header pair). The 5 stdio-only rows can be a follow-up scenario, but the 3 client-declares-{elicitation,roots,sampling}-capability MUSTs and the client-retry-supported-version SHOULD are HTTP-reachable and would fit nicely here.
Correctness bits:
client-consistent-version— I couldn't find spec backing for this. SEP-2575 makes each request self-contained; nothing forbids a client from changingprotocolVersionbetween requests. TheflippingVersionClientnegative test would fail a conformant client. Suggest dropping.client-cancels-by-notification— spec scopesnotifications/cancelledto stdio only ("Streamable HTTP: …Nonotifications/cancelledmessage is required or expected"). This scenario is HTTP, so the check can't legitimately fire here.server/discovergap — the example's discover call (everything-client.ts:98) sends no_metaand noMCP-Protocol-Versionheader, but the harness marks it as passing.client-sends-version-headeris guarded byif (currentVersion)andclient-populates-metaruns after the discover early-return. I think we want the example to send the header and the check order to catch it if an example doesn't do it.- Check IDs — we're shifting to check ids like
sep-<NNNN>-<slug>(seesep-2164-error-codein resources.ts); renaming to match #273's slugs will let the traceability tooling line them up automatically. specReferencesURLs are empty — #273 has the per-row anchors you can lift.
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds client-side conformance tests for stateless MCP (SEP-2575).
Note
This is a companion PR to the server-side tests in #271.
Context
SEP-2575 removes the
initializehandshake and carries version and capabilities on a per-request basis. This PR adds the conformance tests required to verify that client implementations correctly follow these new stateless rules.How Has This Been Tested?
I created the
StatelessScenarioinsrc/scenarios/client/stateless.tsand verified it locally:everything-client.ts. Running it against the harness passes all checks.stateless.test.tsto verify the harness correctly catches non-compliant clients that:_metaparameter.MCP-Protocol-VersionHTTP header.All unit tests pass locally.
Breaking Changes
None.
Types of changes
Checklist
Additional context
None.