Skip to content

[feature] Public mock fixture suite (or keeperhub-mock Docker image) for adapter testing #53

@B2JK-Industry

Description

@B2JK-Industry

Friction

Building sbo3l-keeperhub-adapter, every CI run of the live submission test must either:

  1. Burn a real workflow execution by hitting a real wfb_<token>-protected webhook. Costs money (KH platform billing per execution), pollutes the workflow's run history with synthetic CI traffic, and requires the operator to provision long-lived test credentials with cross-team permissions.
  2. Mock the entire HTTP surface in-process (crates/sbo3l-keeperhub-adapter/src/lib.rs#L97-L99KeeperHubExecutor::local_mock()). Cheap and deterministic, but the mock's response shape is OUR guess at what KH returns. We've reverse-engineered it from curl -v against one workflow; it diverges from production in ways we'll only discover when adapter authors using our crate hit prod for real.

Both options are unsafe at scale. Adapter authors after us will write their own bespoke mocks (cost: hours per author × N authors), each with subtle drift from production.

Reproduction

# Our adapter's local_mock returns:
cargo run --example submit_signed_receipt -p sbo3l-keeperhub-adapter
# == mock execute result ==
# sponsor=keeperhub mock=true execution_ref=kh-01HTAWX5K3R8YV9NQB7C6P2DGZ

# But we don't know if production returns:
#   { "executionId": "..." }       ← what our mock assumes
#   { "id": "..." }                ← what our parser falls back to
#   { "execution_id": "..." }      ← snake_case variant?
#   { "data": { "id": "..." } }    ← nested envelope?
# The submit_live_to parser falls back through `executionId` → `id`
# (lib.rs#L317-L319), but a public canonical answer is missing.

Proposed fix (pick one or both)

Option A — keeperhub-mock Docker image

Publish ghcr.io/keeperhub/mock:latest — a tiny HTTP server that responds to /api/workflows/<any-id>/webhook with deterministic fixtures matching the production webhook contract. Adapter authors run it as a sidecar in CI:

services:
  keeperhub-mock:
    image: ghcr.io/keeperhub/mock:latest
    ports: ['18080:8080']

Then the adapter test points at http://localhost:18080/api/workflows/<id>/webhook instead of prod. Drift between mock and prod is detectable by version-pinning the image.

Option B — public JSON fixture suite

Publish a versioned keeperhub/fixtures repo (or KeeperHub/cli/testdata/) with one JSON file per webhook response shape:

fixtures/
├── webhook_submit_201_minimal.json    # canonical 201 success
├── webhook_submit_200_with_metadata.json  # 200 with optional fields
├── webhook_submit_400_envelope_malformed.json
├── webhook_submit_422_schema_mismatch.json
├── webhook_submit_429_rate_limited.json
└── webhook_submit_503_transient.json

Adapter authors use these as serde_json::from_str(include_str!(...)) to validate their parser without any network. Pairs naturally with issue #52 (error code catalog) — every fixture corresponds to one row in the catalog.

Why both options together?

  • Docker image = integration testing (real HTTP, real timeouts, real connection failures)
  • JSON fixtures = unit testing (parser correctness, no network)

Most adapters need both. Other webhook-platform vendors (Stripe's stripe-mock, Auth0's fixtures) ship both for the same reason.

Reference PR draft (consumer side)

B2JK-Industry/SBO3L-ethglobal-openagents-2026 PR — branch agent/dev2/kh-bf-additional-pr-draft-2: shows our adapter's test suite migrated from local_mock() (in-process stub) to KeeperHubMockServer::from_image() (Option A) plus golden-fixture comparisons (Option B). Removes ~80 lines of bespoke mock code.

Context

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions