Skip to content

test(vessel,vesseld): quality + e2e black-box suites#64

Merged
lIang70 merged 1 commit into
mainfrom
feat/vesseld-e2e-tests
May 7, 2026
Merged

test(vessel,vesseld): quality + e2e black-box suites#64
lIang70 merged 1 commit into
mainfrom
feat/vesseld-e2e-tests

Conversation

@lIang70

@lIang70 lIang70 commented May 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

PR-3 of the vessel/vesseld v0.1 series. Adds two new test modules covering the runtime + daemon end-to-end. No production-code changes.

What's in here

`tests/quality/vessel/` (in-process integration suite)

In go.work, race-enabled, no GOWORK trickery. Pinned in go.mod to:

Module Version
sdk v0.2.7
vessel v0.1.0-rc.2

Coverage:

  • Captain submit/drain/stop lifecycle
  • Restart loop (incl. `MaxRestarts` exhaustion → `PhaseStopped`)
  • Kanban agent-as-tool delegation
  • Sidecars
  • History modes (independent / shared)
  • Hooks — incl. the new `Handle.OnTerminate` ordering contract
  • Probes (liveness/readiness, panic isolation, restart escalation)
  • Concurrency (multi-handle Wait, gate semantics)
  • Fleet-level dispatch

`tests/e2e/vesseld/` (black-box subprocess suite)

GOWORK=off, build-tagged `//go:build e2e`, `make test-e2e`. Each test boots a vesseld subprocess on a per-test temp socket against an in-process mock OpenAI server — no network, no API key, no shared state across tests.

Coverage (~4.7k LOC):

  • API contract (`/healthz`, `/readyz`, `/metrics`, `/v1/{plan,vessels,runs}`, full plan/runs/vessels surface)
  • Auth (TCP bearer token + unix-socket file-mode)
  • SSE log conformance (envelope shape, run lifecycle ordering, delta replay)
  • Validate-only / bad-config rejection (duplicate vessel, missing engine ref, …)
  • Tool allowlist enforcement
  • Sidecar reject path
  • Drain semantics
  • Run-status pagination & filtering
  • Restart policy
  • Kanban delegation across vessels
  • Multi-vessel fleet
  • Rate limiting (per-LLMProfile token bucket)
  • Chaos (kill-during-run)
  • CLI flags + OS-specific edge cases

Wiring

  • `go.work`: `+ ./tests/quality/vessel`
  • `Makefile`: vessel / cmd/vesseld / tests/quality/vessel join `MODULES_WORK`; tests/e2e/vesseld joins `MODULES_OFFWORK`; `make test-e2e` + `make ci-e2e`
  • `.gitignore`: allowlist `cmd/` and `vessel/` at the repo root
  • `.github/workflows/ci.yml`: new `test-vessel-quality` + `test-vesseld-e2e` jobs (Go 1.25 + 1.26, -race); both gated through `ci-pass`; lint extended

Test plan

  • `make vet` green
  • `go test -race -count=1 ./...` in `tests/quality/vessel/`
  • `GOWORK=off go test -tags=e2e -count=1 -race ./...` in `tests/e2e/vesseld/`
  • `gofmt -l` empty across both new dirs
  • CI green

Out of scope

  • PR-4: `chore: examples + docs/quickstart` — chained on this PR

Made with Cursor

Two new test modules, both keyed off the just-released vessel
v0.1.0-rc.2 / cmd/vesseld v0.1.0-rc.1 surface:

tests/quality/vessel/ — in-process integration suite for the
vessel runtime (in go.work, race-enabled, no GOWORK trickery).
Covers Captain submit/drain/stop, restart loop, kanban
agent-as-tool, sidecars, history modes, hooks (incl. the new
Handle.OnTerminate ordering contract), probes/restart escalation.
Pins sdk v0.2.7 / vessel v0.1.0-rc.2 in require lines so go.work
gives us local source while go mod tidy still produces a coherent
graph.

tests/e2e/vesseld/ — black-box subprocess suite for the vesseld
binary. GOWORK=off, build-tagged //go:build e2e. Each test boots
a vesseld subprocess on a per-test temp socket against an
in-process mock OpenAI server (no network, no API key).
Coverage: API contract, plan/runs/vessels endpoints, auth (token
+ unix-socket file mode), conformance (SSE log envelopes incl.
delta replay), validate-only / bad-config rejection, allowlist,
sidecar reject, drain semantics, run-status pagination, restart
policy, kanban delegation, multi-vessel fleet, rate limiting,
chaos (kill-during-run), CLI flags, OS-specific edge cases.
Total: ~4.7k LOC of black-box coverage.

Wiring: - go.work: add ./tests/quality/vessel
  - Makefile: vessel/cmd/vesseld/tests/quality/vessel join
    MODULES_WORK; tests/e2e/vesseld joins MODULES_OFFWORK; add
    `make test-e2e` and `make ci-e2e` targets
  - .gitignore: allowlist cmd/** and vessel/** at the repo root
    (previously these new top-level dirs would have been ignored)
  - .github/workflows/ci.yml: new test-vessel-quality and
    test-vesseld-e2e jobs (Go 1.25 + 1.26, -race), wired into the
    ci-pass gate; lint extended to vet+gofmt the new dirs
Co-authored-by: Cursor <cursoragent@cursor.com>
@lIang70
lIang70 merged commit c1b52e8 into main May 7, 2026
17 checks passed
@lIang70
lIang70 deleted the feat/vesseld-e2e-tests branch May 7, 2026 07:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant