Skip to content

Add the portable interoperability testbed - #40

Merged
callumalpass merged 1 commit into
mainfrom
agent/interop-testbed
Jul 29, 2026
Merged

Add the portable interoperability testbed#40
callumalpass merged 1 commit into
mainfrom
agent/interop-testbed

Conversation

@callumalpass

Copy link
Copy Markdown
Collaborator

What changed

  • Defines a versioned, product-neutral black-box testbed protocol with Draft 2020-12 schemas for adapters, scenarios, fixtures, transcripts, requests, and evidence.
  • Adds 15 canonical scenarios across contract, live interoperability, and durable runtime rings.
  • Ships @callumalpass/mdbase-testbed with a shell-free process runner, a real reference adapter, exact transcript comparison, useful first-difference diagnostics, and digest-bound evidence.
  • Extends v0.3 conformance claims with testbed_transcript evidence and documents the authority boundary.
  • Runs the reference suite and package checks on Linux, macOS, and Windows.

Why

The normative fixtures prove individual requirements, but ecosystem implementations also need comparable observable evidence across languages and product boundaries. The adapter boundary lets implementations keep their own architecture while testing the same contract, multicast, provider-resolution, validation, replay, recovery, and fencing behavior.

Developer and user impact

Implementers can add two process commands (describe and run) and exercise the suite without linking to a particular language runtime. Users can inspect neutral JSON scenarios and attach portable transcript evidence to a conformance claim. Passing never grants authority and does not replace a complete profile claim.

Validation

  • npm test in packages/testbed (15 scenarios; 9 reference scenarios)
  • npm pack --dry-run from a clean generated dist boundary
  • python3 scripts/check_v03_tests.py (61 executable tests, 95 adapter targets)
  • npm run build --prefix site
  • Downstream black-box adapters passed in mdbase TypeScript, mdbase Rust, mdbase Connect, TaskNotes Workflows, Canvas Bases, and the local TaskNotes v5 branch.

@callumalpass
callumalpass marked this pull request as ready for review July 29, 2026 01:45
@callumalpass
callumalpass merged commit 9155a3d into main Jul 29, 2026
18 checks passed
@callumalpass
callumalpass deleted the agent/interop-testbed branch July 29, 2026 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant