Skip to content

fix(sep-2243): map custom-header check IDs to their requirements#303

Merged
pcarleton merged 1 commit into
mainfrom
paulc/sep-2243-requirement-check-ids
May 22, 2026
Merged

fix(sep-2243): map custom-header check IDs to their requirements#303
pcarleton merged 1 commit into
mainfrom
paulc/sep-2243-requirement-check-ids

Conversation

@pcarleton
Copy link
Copy Markdown
Member

@pcarleton pcarleton commented May 22, 2026

What

SEP-2243's traceability showed 6 of 20 requirements tested even though the x-mcp-header / Mcp-Param / Base64 half of the SEP has had scenarios since #259. The scenarios emit one check ID per test case while src/seps/sep-2243.yaml declares one per normative requirement, and the traceability join is an exact ID match — so the custom-header checks never connected to their requirements. #287 collapsed the standard-header IDs but intentionally left the custom-header ones per-case.

This finishes the rename and adds the DECLARED_CHECK_IDS backfill pattern (from request-metadata.ts) so every requirement-level ID is emitted on every run — exercised, backfilled as FAILURE, or SKIPPED when the server under test exposes no x-mcp-header tool. The IDs therefore reach checks.json even when the reference SDK gates the feature off, which is what kept the manifest stuck at 6/20.

SEP-2243 traceability: 6/20 → 18/20. The remaining 2 (server-not-expect-null, server-reject-missing-required) have no test case yet and will be a follow-up.

Renames

before (per-case) after (per-requirement)
param-header-{name} (15 params) client-mirrors-designated-params / client-encode-values / client-base64-unsafe, picked by what the parameter exercises
no-mirror-unannotated client-mirrors-designated-params
param-header-tool-call-gate client-supports-custom-headers (now also emitted SUCCESS when the annotated tool is called with ≥1 Mcp-Param-* header)
reject-invalid-tool loop (10 tools) the specific constraint each tool violates: x-mcp-header-not-empty / -charset / -unique / -primitive-only; client-reject-invalid-tool keeps the "valid tool was not over-rejected" check
server-accepts-valid-base64 server-decode-base64
server-rejects-invalid-base64-{padding,chars} server-reject-invalid-param-chars
server-literal-missing-base64-{prefix,suffix} server-validate-param-match
server-rejects-missing-custom-header server-validate-param-match
every custom-header rejection's -error-code check server-reject-param-mismatch
standard-header reject per-case IDs (catch path only) server-reject-invalid-headers

Judgment calls (so they don't get re-litigated)

  • Truncated base64 padding → server-reject-invalid-param-chars: every character is in the base64 alphabet; the defect is padding. "Invalid characters" is a stretch but it's the closest requirement; server-validate-param-match was the alternative.
  • Missing custom header (value present in body) → server-validate-param-match: treated as the limiting case of a header/body mismatch. The separate server-reject-missing-required row (param absent from body and header) remains untested.
  • Gates, setup markers, and the RFC 9110 OWS whitespace check keep their scenario-specific IDs and stay in untracked — they are not SEP requirements.

Test plan

  • npm run typecheck, npm run lint, npx vitest run src/ — 599/599 pass.
  • New http-custom-headers.test.ts pins the declared-ID sets: zero-interaction runs emit exactly the declared IDs, conforming calls map each parameter kind to its requirement ID, and violations land on the violated constraint's ID.
  • Ran all 5 SEP-2243 scenarios against typescript-sdk@main and fed the results through collectEmittedIdscomputeTraceability: 18 tested / 2 untested / 4 excluded / 3 untracked.
  • src/seps/traceability.json is not regenerated here — the next manifest refresh will pick up the new IDs.

The x-mcp-header / Mcp-Param / Base64 half of SEP-2243 was already tested,
but the scenarios emitted one check ID per test case while sep-2243.yaml
declares one per normative requirement, so the traceability join (exact ID
match) reported 14 of 20 requirements as untested. #287 collapsed the
standard-header IDs but left the custom-header ones per-case.

This finishes the rename and adds DECLARED_CHECK_IDS backfill (the
request-metadata.ts pattern) so every requirement-level ID is emitted on
every run — exercised, backfilled as FAILURE, or SKIPPED when the server
under test exposes no x-mcp-header tool. SEP-2243 traceability goes from
6/20 to 18/20 tested; the remaining 2 (server-not-expect-null,
server-reject-missing-required) have no test case yet.

Client scenarios:
- param-header-{name} -> client-mirrors-designated-params /
  client-encode-values / client-base64-unsafe, picked by what the
  parameter exercises
- no-mirror-unannotated -> client-mirrors-designated-params
- param-header-tool-call-gate -> client-supports-custom-headers, now also
  emitted SUCCESS when the annotated tool is called with Mcp-Param headers
- the invalid-tool loop emits the specific x-mcp-header constraint each
  tool violates (not-empty / charset / unique / primitive-only)

Server scenarios:
- accepts-valid-base64 -> server-decode-base64
- rejects-invalid-base64-{padding,chars} -> server-reject-invalid-param-chars
- literal-missing-base64-{prefix,suffix} -> server-validate-param-match
- rejects-missing-custom-header -> server-validate-param-match
- every custom-header rejection's -32001 check -> server-reject-param-mismatch
- standard-header reject cases pass the coarse ID through the catch path too
@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new Bot commented May 22, 2026

Open in StackBlitz

npx https://pkg.pr.new/@modelcontextprotocol/conformance@303

commit: 8c0813d

@pcarleton pcarleton merged commit 909ea0b into main May 22, 2026
8 checks passed
@pcarleton pcarleton deleted the paulc/sep-2243-requirement-check-ids branch May 22, 2026 22:37
pcarleton added a commit that referenced this pull request May 22, 2026
Regenerated from a full client+server suite run against
typescript-sdk@258f1a04a5d3 on current main:

- sep-2468: the two new client checks are emitted (tested) and the two
  authorization-server rows move to excluded (#304)
- sep-2243: 12 rows flip to tested now that the custom-header check IDs
  match their requirements (#303)
pcarleton added a commit that referenced this pull request May 22, 2026
…305)

* chore: refresh SEP traceability manifest after #303 and #304

Regenerated from a full client+server suite run against
typescript-sdk@258f1a04a5d3 on current main:

- sep-2468: the two new client checks are emitted (tested) and the two
  authorization-server rows move to excluded (#304)
- sep-2243: 12 rows flip to tested now that the custom-header check IDs
  match their requirements (#303)

* chore: regenerate the manifest on top of SEP-2106 (#295)

Re-ran both suites after rebasing onto main so the manifest also picks
up the new sep-2106 block (1 tested, 4 excluded, 3 untracked check IDs
emitted by the server scenario with no yaml row yet).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant