Skip to content

Conformance: add endpoint-binding check (assert wrong-endpoint calls are rejected) #28

Description

@mickdarling

Summary

The conformance runner (src/conformance.ts, feat/parity-runner) can connect to any adapter over stdio but currently runs only four schema/introspection checks. It does not verify the spec's CRUDE endpoint-binding MUST — that an operation invoked via the wrong mcp_aql_* endpoint is rejected, not executed (spec/docs/operations.md:494, crude-pattern.md:269, endpoint-modes.md:151). This gap let a hand-built adapter ship the violation (MCPAQL/makerperks-adapter#93) with a green build.

This realizes the slice of the existing TODO: add live CRUD proxy operation exercises (conformance.ts:~69) that covers endpoint binding.

Why this is the highest-leverage fix

The generator already emits an endpoint check, but hand-built adapters bypass it. A conformance check catches the violation regardless of how the adapter was built — generated, hand-rolled, or reference. It's also small, because the harness already introspects and connects.

What to build

Add an endpoint-binding check category to the runner, driven by introspection (no static vectors needed for this check):

  1. Call introspect to learn every operation and its semantic category / endpoint.
  2. Determine the set of registered CRUDE endpoints. If only one endpoint is registered (e.g. a read-only adapter), record the check as n/a (single endpoint) — not pass (don't imply coverage that wasn't exercised).
  3. Pick a READ operation (side-effect-free even if wrongly executed) and invoke it through every other registered endpoint. For each:
  4. Control: assert the same READ op via its correct endpoint succeeds.

Acceptance criteria

  • New endpoint-binding check in conformance.ts + surfaced in conformance-cli.ts output and conformance-report.json.
  • A non-enforcing adapter (e.g. makerperks-adapter@main) fails this check; the fixed version (PR #94 / fix/crude-endpoint-enforcement) and the generated example adapters pass (after the generator code-name is aligned).
  • Single-endpoint adapters report n/a, not pass.
  • Asserts the canonical VALIDATION_ENDPOINT_MISMATCH.

Dependencies / sequencing

Context

Triggered by MCPAQL/makerperks-adapter#93 (CRUDE endpoint enforcement missing). Current-state finding: the runner exists and is pointable at any adapter, but no existing check (conformance.ts, parity-runner.ts) ever issues a cross-endpoint call.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions