test(e2e): parallel-safe e2e suite + API-key coverage + preview-gateway secret-drift fix - #579
Merged
Merged
Conversation
…rage - web/e2e: replace CI-run-id naming with a worker+retry+random uniqueSlug() helper so parallel workers can't collide on resource names; give project-create.spec.ts a try/finally cleanup (it previously never deleted the project it created); flip playwright.config.ts to fullyParallel: true. - apps/temps-e2e: makeRunId() now mixes in a random suffix, not just a millisecond timestamp, so concurrent scenario/examples runs can't collide. - web/e2e: add api-key-create.spec.ts — drives the real 3-step RBAC wizard and verifies the minted secret actually authenticates against the API, not just that a row was created. - crates/temps-agents: fix preview gateway secret-drift bug found while verifying the above locally. The reconciler only checked *presence* of PREVIEW_GATEWAY_SHARED_SECRET on the shared temps-preview-gateway container, never its value. Since that container has one fixed name across every local `temps serve` instance, whichever instance created it first "wins" and every other instance's proxy injects a secret the container will never accept, with no way to self-heal. container_matches now compares the value and triggers recreation on drift. - web/e2e/README.md: document the parallel-safety conventions, a coverage matrix against the feature catalog's 6 subsystems, and why vercel-labs/emulate can't be wired into temps' own provider/webhook config (the SSRF guard in validate_external_url is deliberate).
📓 Changelog previewThis is what your commits will add to the generated ## [Unreleased]
### Testing
- **e2e:** Make console/CLI e2e suite parallel-safe, add API-key coverage |
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
web/e2e: replaced CI-run-id-only resource naming with a worker+retry+randomuniqueSlug()helper (the old naming was identical across every worker in one run, so it collides the moment specs run concurrently).project-create.spec.tspreviously never deleted the project it created — addedtry/finallycleanup. Flippedplaywright.config.tstofullyParallel: true.apps/temps-e2e:makeRunId()now mixes in a random suffix, not just a millisecond timestamp — concurrentscenario/examplesruns (parallel CI shards, two dev terminals) could otherwise land on the exact same run id.web/e2e/authenticated/api-key-create.spec.ts— the security-auth subsystem's first end-to-end spec. Drives the real 3-step RBAC wizard through the UI and, critically, verifies the minted secret actually authenticates against/api/projectswith a live bearer token, not just that a row was created.crates/temps-agents/src/preview_gateway.rs— the reconciler that keeps the sharedtemps-preview-gatewayDocker container in sync only checked presence ofPREVIEW_GATEWAY_SHARED_SECRET, never its value. That container has one fixed name shared across every localtemps serveinstance on a machine, so whichever instance created it first "wins" the secret — every other instance's proxy injects its own DB secret, the gateway holds a different one forever, and every preview request gets rejected with "missing or invalid X-Temps-Preview-Token" with no way to self-heal.container_matchesnow compares the secret value and triggers recreation on drift. Confirmed via log:preview gateway drift detected — recreating.web/e2e/README.mdnow documents the parallel-safety conventions, a coverage matrix against the feature catalog's 6 subsystems (observability, data-storage RBAC-depth, and platform-commerce webhooks/notifications are the remaining gaps), and whyvercel-labs/emulatecan't be wired into temps' own provider/webhook config — every such URL passes through the deliberate SSRF guard invalidate_external_url, which is a security boundary, not something to route around for test convenience.Test plan
cd apps/temps-e2e && bun run typecheckcd web && bunx tsc --noEmitcd web && bunx eslinton all touched files (clean after--fixfor formatting)$(command -v cargo) clippy -p temps-agents --all-targets -- -D warnings(0 warnings)web/e2esuite withfullyParallel: true— 21/21 passapps/temps-e2e scenarioprocesses concurrently — distinct run ids despite starting in the same millisecond, both torn down cleanlypreview_gateway.rsfix verified live: forced the drift condition, confirmed the reconciler detects and recreates