Replace Rust E2E mock tests with Bun lifecycle test books by ractive · Pull Request #14 · ractive/hoppy

ractive · 2026-03-19T16:23:05Z

Summary

Remove 103 wiremock-based Rust E2E tests (tests/e2e_*.rs, tests/e2e_support/, tools/e2e-report.sh) and associated dev-dependencies (assert_cmd, wiremock, predicates)
Add 7 Bun test books in testbooks/ that exercise full resource lifecycles (create → get → list → update → verify → delete) against the live bunny.net API with snapshot testing
Test books cover: auth, pull-zone, dns-zone (+ records + MX), storage-zone (+ file ops), stream (library + collection), script (+ variables + secrets + releases), shield (zone + WAF + rate limits + access lists + bot detection)

Test books

File	Describe blocks	Steps
`auth.test.ts`	Auth (1)	check
`pull-zone.test.ts`	Lifecycle (7) + Errors (2)	create → get → list → update → verify → purge → delete
`dns-zone.test.ts`	Zone (6) + Record (7) + Record Types (3)	zone CRUD + A record CRUD + MX with priority
`storage-zone.test.ts`	Zone (6) + File Ops (7)	zone CRUD + upload → ls → download+verify → rm
`stream.test.ts`	Library (7) + Collection (8)	library CRUD + collection CRUD
`script.test.ts`	Script (10) + Variable (7) + Secret (7)	script lifecycle + code + publish + releases + var/secret CRUD + upsert
`shield.test.ts`	Full lifecycle (24)	pull zone → shield zone → WAF profiles/rules → rate limits → access lists → bot detection → cleanup

Not covered (intentional)

Containers — deferred (cost/complexity, per roadmap)
Stream video — deferred (requires real video files + async processing)
script statistics — needs actual traffic data
script rotate-deployment-key — destructive, not reversible
Pagination/search params — mechanical pass-through, not worth lifecycle test time

Running

cargo build
BUNNY_API_KEY=xxx bun test testbooks/
BUNNY_API_KEY=xxx bun test testbooks/ --update-snapshots  # capture/update snapshots

Test plan

bun run typecheck passes (TypeScript compilation)
cargo build succeeds (no Rust regressions from removed dev-deps)
cargo test --workspace passes (crate-level wiremock tests unaffected)
BUNNY_API_KEY=xxx bun test testbooks/ passes against live API
Snapshots captured and committed after first live run

🤖 Generated with Claude Code

Remove 103 wiremock-based Rust E2E tests and replace with 7 Bun test books that exercise full resource lifecycles (create → get → list → update → delete) against the live bunny.net API with snapshot testing. Test books: auth, pull-zone, dns-zone, storage-zone, stream, script, shield — covering all CLI commands and flags except containers (deferred) and stream video (requires real video files). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Fix cleanupFns.reverse() mutating in place — use spread copy - Document shared cleanup singleton assumption in helpers.ts - Remove empty [dev-dependencies] section from Cargo.toml - Pin @types/bun to ^1.3.11 instead of "latest" - Add trailing newline to Bugs.md - Sync roadmap per-test-book checklists with implementation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

This PR replaces the prior Rust wiremock-based CLI E2E suite with Bun “test books” that run ordered, stateful lifecycle tests against the live bunny.net API, using snapshots to lock output shapes while keeping crate-level wiremock tests intact.

Changes:

Removed the Rust E2E harness (wiremock + assert_cmd + predicates), its helpers, and the report script.
Added Bun-based lifecycle “test books” in testbooks/ with a shared CLI runner + cleanup registry and initial snapshots.
Updated roadmap/knowledgebase docs and trimmed Rust dev-dependencies accordingly.

Reviewed changes

Copilot reviewed 30 out of 33 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
tools/e2e-report.sh	Removed mock E2E report generator script.
tests/e2e_support/server.rs	Removed wiremock server helper.
tests/e2e_support/mod.rs	Removed E2E support module and live-mode skip macro.
tests/e2e_support/cmd.rs	Removed assert_cmd-based CLI runner for mock server/live mode.
tests/e2e_stream.rs	Removed stream CLI mock E2E tests.
tests/e2e_storage_zone.rs	Removed storage-zone CLI mock E2E tests.
tests/e2e_storage.rs	Removed storage file-operation CLI mock E2E tests.
tests/e2e_shield.rs	Removed shield CLI mock E2E tests.
tests/e2e_script.rs	Removed script CLI mock E2E tests.
tests/e2e_pull_zone.rs	Removed pull-zone CLI mock E2E tests.
tests/e2e_global.rs	Removed global CLI mock tests (help/version/errors).
tests/e2e_dns.rs	Removed DNS CLI mock E2E tests.
tests/e2e_container.rs	Removed container CLI mock E2E tests.
tests/e2e_auth.rs	Removed auth CLI mock E2E tests.
testbooks/tsconfig.json	Added TypeScript config for Bun testbooks.
testbooks/stream.test.ts	Added live lifecycle tests for Stream libraries/collections.
testbooks/storage-zone.test.ts	Added live lifecycle tests for storage zones + file ops.
testbooks/shield.test.ts	Added live lifecycle tests for shield (zone/WAF/rate-limit/access-list/bot-detection).
testbooks/script.test.ts	Added live lifecycle tests for scripts + variables + secrets + releases.
testbooks/pull-zone.test.ts	Added live lifecycle tests + basic error cases for pull zones.
testbooks/package.json	Added Bun testbooks package metadata and scripts.
testbooks/helpers.ts	Added shared Bun harness for invoking the built `hoppy` binary + cleanup helpers.
testbooks/dns-zone.test.ts	Added live lifecycle tests for DNS zones and records (incl. MX).
testbooks/bun.lock	Added Bun lockfile for testbooks dependencies.
testbooks/auth.test.ts	Added live auth check testbook.
testbooks/snapshots/pull-zone.test.ts.snap	Added initial Bun snapshot for pull-zone create response.
testbooks/.gitignore	Added testbooks-local ignores (node_modules, caches, env files, etc.).
hoppy-knowledgebase/development-roadmap.md	Documented the transition from mock E2E to Bun testbooks and how to run them.
hoppy-knowledgebase/Bugs.md	Added a knowledgebase note about a container-app decode error and debug expectations.
Cargo.toml	Removed Rust dev-dependencies used only by the deleted mock E2E suite.
Cargo.lock	Updated lockfile to reflect removed Rust dev-dependencies.
.gitignore	Removed ignores for outputs produced by the deleted E2E report script.
.claude/agents/bun-test-engineer.md	Added a Claude agent profile for Bun test engineering.

You can also share your feedback on Copilot code review. Take the survey.

- Add monotonic counter to testName() to prevent same-millisecond collisions - Add --timeout 60000 to test scripts for live API calls - Move typescript from peerDependencies to devDependencies - Support HOPPY_BIN env var and Windows binary name in helpers - Fix stream.test.ts architecture description to note video is deferred - Update pull-zone error cases checklist entry Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Move 95 JSON fixture files from crate-local tests/fixtures/ to shared fixtures/{core,storage,stream,shield,compute,containers}/ directory - Update all include_str! paths in 10 test files to use shared fixtures - Add read_body() helper to all 6 API client crates that buffers the response and logs status + body to stderr when --debug is enabled - Add CAPTURE_FIXTURES=1 mode to Bun test helpers that captures raw API responses from --debug output and writes them as fixture files - Add adding-a-feature.md checklist to knowledgebase - Update development roadmap with fixtures architecture and capture docs Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Documents the three-layer test architecture (wiremock unit, wiremock CLI, live E2E), shared fixture/mock setup, implementation phases, and test patterns using assert_cmd + insta + wiremock. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Implements phases 1-4 of the Rust E2E rewrite plan: infrastructure (assert_cmd, insta, wiremock dev-deps), shared test helpers, and 88 wiremock-backed integration tests across 8 test files covering auth, pull-zone, storage-zone, storage, DNS, stream, script, and shield. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Delete 3 fabricated fixture files (pullzone_create, shield_zone_create, dnsrecord_add_mx) and update tests to use only real captured fixtures. Add 7 missing tests to match Bun coverage: MX record with priority, pull-zone update 404, script variable list/update/upsert, and script secret update/upsert. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add global --record=<dir> flag that records API response bodies to JSON fixture files. The flag threads through CLI → commands → auth → all 6 client crates, piggy-backing on the existing debug logging path. Add 15 live API lifecycle tests gated behind #[cfg(feature = "live-api")] covering auth, pull-zone, DNS, storage-zone, storage, stream, script, and shield. Tests use run_lifecycle() with CleanupStack for panic-safe resource cleanup. Delete testbooks/ (Bun-based tests) now fully replaced by Rust tests. Update knowledgebase docs to reflect the new test architecture. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Add camelCase serde aliases to Stream API types (Collection, PaginatedList, StatusMessage) to handle inconsistent API casing - Fix paginated list assertions across all test files to access ["Items"] instead of treating response as bare array - Add Id field to UpdateEdgeScript and skip_serializing_if for optional fields to match compute API requirements - Implement read-modify-write pattern for Shield WAF/rate-limit updates (API requires all fields on PATCH) - Remove skip_serializing_if from Shield WAF/rate-limit rule types where API requires fields even when empty - Handle free-plan limitations in Shield tests (skip WAF custom rules and rate limits when plan returns id=0) - Update test snapshots for Shield WAF and rate-limit responses Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Add 50 tests (49 wiremock mock + 1 live API lifecycle) covering all container subcommands: app, template, endpoint, volume, registry, region, node, pod, limits, and log-forwarding - Fix --format flag conflict on log-forwarding create/update by renaming to --syslog-format - Add --image-name/--image-namespace/--image-tag/--registry-id flags to container app create (API requires at least one container template) - Fix nullable fields in containers types: imageDigest (null -> ""), exposedPort (null -> None) - Live test exercises endpoint creation to cover exposedPort:null deserialization path Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Suppress dead_code warnings for shared test helpers that are only used by some test binaries, and remove unnecessary mut in hoppy_live_cmd. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

This PR refactors the project’s end-to-end/CLI testing approach by removing the old tests/e2e_*.rs wiremock suite and introducing a new snapshot-based CLI testing setup with shared fixtures, plus adds a --record flag to capture live API responses for fixture generation.

Changes:

Removed legacy Rust E2E harness (tests/e2e_*, tests/e2e_support/*) and the tools/e2e-report.sh report generator.
Added a global --record <DIR> CLI flag and plumbed it through command handlers into API clients to record successful JSON responses.
Centralized fixture JSON under top-level fixtures/ and added/updated many insta snapshots for CLI output; added a live-api test feature.

Reviewed changes

Copilot reviewed 153 out of 250 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
`tools/e2e-report.sh`	Removed obsolete E2E report script.
`src/cli.rs`	Adds global `--record` flag; extends container CLI options and renames syslog flag field to avoid `format` ambiguity.
`src/main.rs`	Threads `record` into all command dispatch paths.
`src/auth.rs`	Extends client constructors to accept `record` and call client `.with_record(...)`.
`src/commands/auth.rs`	Passes `record` through to core client usage.
`src/commands/pull_zone.rs`	Passes `record` through to core client usage.
`src/commands/storage_zone.rs`	Passes `record` through to core client usage.
`src/commands/storage.rs`	Passes `record` into storage client builder and applies `.with_record(...)`.
`src/commands/dns.rs`	Passes `record` through to core client usage.
`src/commands/stream.rs`	Passes `record` through; resolves stream client with optional recording.
`src/commands/script.rs`	Passes `record` through to compute client usage.
`src/commands/shield.rs`	Passes `record` through; adjusts create/update payload construction for Shield PATCH requirements.
`tests/e2e_support/{mod.rs,cmd.rs,server.rs}`	Removed legacy E2E support module.
`tests/e2e_{global,auth,storage,storage_zone}.rs`	Removed legacy E2E tests.
`tests/cli_auth.rs`	Adds snapshot-based CLI tests using wiremock + `insta`.
`tests/support/mod.rs`	Adds shared helpers for mock/live CLI runs and fixtures loading.
`Cargo.toml`	Adds `live-api` feature and `insta` dev-dependency.
`README.md`	Documents mock vs live API testing and env overrides.
`.gitignore`	Removes ignores for the deleted E2E report outputs.
`hoppy-knowledgebase/adding-a-feature.md`	Adds a feature checklist including `--record` fixture capture guidance.
`hoppy-knowledgebase/Bugs.md`	Adds a bug note about missing debug JSON on decode failure.
`crates/bunny-api-storage/src/client.rs`	Refactors response handling to read bytes once and adds response recording support.
`crates/bunny-api-shield/src/client.rs`	Refactors response handling to read bytes once and adds response recording support.
`crates/bunny-api-containers/src/client.rs`	Refactors response handling to read bytes once and adds response recording support.
`crates/bunny-api-containers/src/types.rs`	Makes `image_digest` null-tolerant; makes `exposed_port` optional.
`crates/bunny-api-stream/src/types.rs`	Adds serde aliases to accept alternate casing from fixtures/responses.
`crates/bunny-api-compute/src/types.rs`	Adds `id` field to `UpdateEdgeScript` payload.
`crates//tests/_api.rs`	Updates test fixtures to load from shared top-level `fixtures/` paths.
`crates/*/Cargo.toml`	Adds `bytes` dependency to support body buffering/recording changes.
`fixtures/**`	Adds many shared JSON fixtures for core/storage/stream/shield/compute/containers.
`tests/snapshots/**`	Adds many `insta` snapshots for CLI output across services.

- Extract duplicated recording logic into shared bunny-api-recording crate, fixing Mutex poisoning (unwrap_or_else) and adding eprintln on write errors - Restore skip_serializing_if on shield request types (CreateCustomWafRule, UpdateCustomWafRule, CreateRateLimitRule, UpdateRateLimitRule, WafRuleConfiguration, RateLimitRuleConfiguration) to avoid sending nulls - Restore body text in stream client parse_response error context - Fix compute update_script test to expect new Id field - Fix clippy warnings: assert_eq!(x, true), unnecessary to_string, too_many_arguments Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ractive requested a review from Copilot March 19, 2026 16:23

Copilot AI reviewed Mar 19, 2026

View reviewed changes

ractive and others added 9 commits March 19, 2026 17:29

Fix test support warnings for both default and live-api features

81449b6

Suppress dead_code warnings for shared test helpers that are only used by some test binaries, and remove unnecessary mut in hoppy_live_cmd. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ractive requested a review from Copilot March 19, 2026 22:42

Copilot started reviewing on behalf of ractive March 19, 2026 22:43 View session

Copilot AI reviewed Mar 19, 2026

View reviewed changes

ractive and others added 2 commits March 20, 2026 00:01

Add secrets warning to --record fixture capture docs

c76ceb4

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

ractive merged commit 0905bd7 into main Mar 19, 2026

ractive deleted the iter-11/e2e-lifecycle-testbooks branch March 19, 2026 23:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace Rust E2E mock tests with Bun lifecycle test books#14

Replace Rust E2E mock tests with Bun lifecycle test books#14
ractive merged 13 commits into
mainfrom
iter-11/e2e-lifecycle-testbooks

ractive commented Mar 19, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

ractive commented Mar 19, 2026

Summary

Test books

Not covered (intentional)

Running

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants