Context
spec-sdk-tests/ is a contract test suite that validates the Outpost API against the generated TypeScript SDK. It exercises CRUD on all destination types, merge-patch semantics, pagination, event delivery flow, and the publish → event → attempt path.
No CI workflow runs it today. Verified across .github/workflows/ and .speakeasy/workflow.yaml. The only signal anyone gets is when a developer manually runs ./spec-sdk-tests/scripts/run-tests.sh against a local or managed Outpost.
This gap let real drift accumulate on main:
- TS SDK
dist/ went stale from Oct 2025 → May 2026 unnoticed
- A Speakeasy CLI bump (1.741.7 → 1.753.0) silently changed the paginator response shape, breaking test compilation
- A wire-serialization bug in destination PATCH (#920) was undiagnosed until a manual run
What we want
A workflow that runs spec-sdk-tests against a freshly-started local Outpost, gated on PRs that touch any of:
docs/apis/openapi.yamlsdks/outpost-typescript/**spec-sdk-tests/**internal/apirouter/** (handler shape changes).speakeasy/**
Plus workflow_dispatch for ad-hoc runs and a low-frequency cron as drift-safety net.
Implementation notes
- Local Outpost, not managed. Tests create and delete real resources; pointing at
api.outpost.hookdeck.com would bleed test-tenant state into prod and hit rate limits. docker compose config exists in deployments/; consider a slimmed compose.test.yml.
- Wait for
/healthz before running the suite; tests already support SKIP_HEALTH_CHECK=true for managed (don't use that here).
- Topics:
TEST_TOPICS must include user.created,user.updated,order.created,heartbeat. Configure via env or config file at server startup.
- API key: shared between server config and
spec-sdk-tests/.env.
- Don't enable Speakeasy's built-in
sdk-test.yaml — that's a different system; spec-sdk-tests is the authoritative contract test for this repo.
- Run time: ~1 minute against managed Outpost; should fit cleanly within a CI job budget once local startup is amortized.
Related
- #920 — the bug that surfaced this gap
- Branch
feat/spec-sdk-fixes — umbrella for current spec/SDK fixes; this CI workflow lands separately
🤖 Generated with Claude Code
Context
spec-sdk-tests/is a contract test suite that validates the Outpost API against the generated TypeScript SDK. It exercises CRUD on all destination types, merge-patch semantics, pagination, event delivery flow, and the publish → event → attempt path.No CI workflow runs it today. Verified across
.github/workflows/and.speakeasy/workflow.yaml. The only signal anyone gets is when a developer manually runs./spec-sdk-tests/scripts/run-tests.shagainst a local or managed Outpost.This gap let real drift accumulate on
main:dist/went stale from Oct 2025 → May 2026 unnoticedWhat we want
A workflow that runs
spec-sdk-testsagainst a freshly-started local Outpost, gated on PRs that touch any of:docs/apis/openapi.yamlsdks/outpost-typescript/**spec-sdk-tests/**internal/apirouter/**(handler shape changes).speakeasy/**Plus
workflow_dispatchfor ad-hoc runs and a low-frequency cron as drift-safety net.Implementation notes
api.outpost.hookdeck.comwould bleed test-tenant state into prod and hit rate limits.docker composeconfig exists indeployments/; consider a slimmedcompose.test.yml./healthzbefore running the suite; tests already supportSKIP_HEALTH_CHECK=truefor managed (don't use that here).TEST_TOPICSmust includeuser.created,user.updated,order.created,heartbeat. Configure via env or config file at server startup.spec-sdk-tests/.env.sdk-test.yaml— that's a different system;spec-sdk-testsis the authoritative contract test for this repo.Related
feat/spec-sdk-fixes— umbrella for current spec/SDK fixes; this CI workflow lands separately🤖 Generated with Claude Code