[vitest] Isolate test data and improve cleanup#1895
Merged
VaguelySerious merged 1 commit intomainfrom May 2, 2026
Merged
Conversation
Move the @workflow/vitest harness from process-wide env vars to project-scoped provided context. workflow() resolves cwd, rootDir, dataDir, and outDir once and passes them through Vitest's per-project context; global-setup and setup-file read that resolved state when building bundles and creating the local test world. On the recovery side, world-local now keeps fileIdFilter applied while paginating and scopes startup recovery to the active tag. Adds an explicit recoverActiveRuns opt-out and disables recovery in the Vitest harness so worker startup does not re-enqueue stale runs before direct handlers are registered. Also tightens cleanup in two e2e dev tests so the deferred-builder / Turbopack interaction does not leave the dev server in a broken state for subsequent test files (which previously caused Windows e2e to hang for the full 30-min job timeout). Co-Authored-By: Tom Dale <tom@vercel.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
🦋 Changeset detectedLatest commit: 7ff87ae The changes in this PR will be included in the next version bump. This PR includes changesets to release 19 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Contributor
Contributor
🧪 E2E Test Results✅ All tests passed Summary
Details by Category✅ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
|
Contributor
📊 Benchmark Results
workflow with no steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 1 step💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 10 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 25 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 50 sequential steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express Promise.all with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express Promise.all with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express Promise.all with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express Promise.race with 10 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express Promise.race with 25 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express Promise.race with 50 concurrent steps💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 10 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 25 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 50 sequential data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 10 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 25 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express workflow with 50 concurrent data payload steps (10KB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express Stream Benchmarks (includes TTFB metrics)workflow with stream💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express stream pipeline with 5 transform steps (1MB)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express 10 parallel streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express fan-out fan-in 10 streams (1MB each)💻 Local Development
▲ Production (Vercel)
🔍 Observability: Express SummaryFastest Framework by WorldWinner determined by most benchmark wins
Fastest World by FrameworkWinner determined by most benchmark wins
Column Definitions
Worlds:
|
VaguelySerious
changed the title
TooTallNate
approved these changes
May 2, 2026
Member
TooTallNate
left a comment
TooTallNate
left a comment
There was a problem hiding this comment.
Three coherent fixes wrapped into one PR — vitest project-context isolation, world-local recovery scoped by tag, and Windows-specific dev-test cleanup hardening. Verified each piece independently.
Vitest project-context isolation
The shape is right: WorkflowTestOptions resolves to a ResolvedWorkflowTestOptions shape (cwd / rootDir / dataDir / outDir, all absolute), gets passed via Vitest's provide/inject (per-project context), and is consumed by both global-setup.ts and setup-file.ts. No process env vars touched. The new test confirms process.env.WORKFLOW_VITEST_* stays clean.
resolveWorkflowTestOptions is idempotent (resolving an already-resolved object is a no-op since path.resolve of an absolute path returns itself), so the double-resolution from readProvidedWorkflowTestOptions is safe.
Path resolution: cwd defaults to process.cwd(); rootDir defaults to cwd (resolved relative to cwd if provided); dataDir/outDir default to <rootDir>/.workflow-data and <rootDir>/.workflow-vitest (resolved relative to cwd if provided). Consistent with the docs table.
Recovery scoped by tag
The bug existed: on main, createLocalWorld() calls createStorage() twice (once stored as storage, once spread into the world). They're separate instances with separate instrumentObject wrappers. Recovery (storage.runs) wouldn't see updates that consumers made through the spread instance, and adding a fileIdFilter wrapper to storage.runs wouldn't be visible to consumers. The PR collapses this to a single storage and uses ...storage for the spread, fixing the latent duplication.
The new LocalListWorkflowRunsParams extends the public ListWorkflowRunsParams with an internal fileIdFilter. Kept off the public Storage['runs']['list'] surface via a separate LocalRunsStorage type, with LocalStorage structurally assignable to Storage. This is the right way to add a backend-private field without leaking it to @workflow/world consumers.
The paginatedFileSystemQuery change is the load-bearing piece: filteredFileIds is now applied to both the no-cursor branch and the cursor-paginated branch (via candidateFileIds = filteredFileIds). Test "keeps fileIdFilter applied on later cursor pages" exercises this directly with multiple ULID-stamped files, verifying that page 2 still respects the filter.
recoverActiveRuns: false opt-out for the vitest harness is the right control: tests register direct handlers AFTER setupWorkflowTests() starts, and stale runs in the data dir would otherwise be dispatched before handlers are wired up. The reordering of world.start() to happen after world.registerHandler() (with the inline comment explaining the future hazard if anyone re-enables recovery) is good defensive practice.
The new hasTag(fileId, tag) helper is a clean primitive — wrun_ABC.vitest-0 matches vitest-0, untagged fileIds never match. Three new tests cover: tag isolation between two workers, tag filtering across pagination (25 tagged + 5 untagged + 5 differently-tagged runs), and recovery skip when recoverActiveRuns: false.
Dev-test cleanup hardening
These are Windows-specific Turbopack flake fixes, well-explained by inline comments at every behavior change:
afterEachreorders restore-then-delete to avoid leaving the dev server with broken imports between deletes (Turbopack on Windows caches the parse failure).- The "imported step hot-reload" test catches
triggerWorkflowRunfailures during polling, rewrites the api file to invalidate Turbopack's cache, and continues polling. Conservative — only rewrites on error, doesn't bail. - The "discovered via workflow imports" test does in-test cleanup (instead of
afterEach) so the deferred builder has time to drop the discovered step before the next test file's dev server requests start failing.
Timeout was bumped 30s → 60s on the third one. Reasonable for the additional polling.
Test verification
Ran on the PR branch:
pnpm testinpackages/vitest: 5 new tests pass.pnpm testinpackages/world-local: all 339 tests pass (including the new pagination filter test and the three tag-recovery tests).
Incidental cleanup
vitest moves from a non-listed-but-locked dependency to a proper devDep. Consumers continue to resolve it via the existing peerDependencies (vitest: >=3.0.0). No surface change for users of @workflow/vitest.
Nits (non-blocking)
The two changesets swift-cobras-repair.md and moody-rivers-play.md have somewhat overlapping content — one's the package-version bumps, the other's the empty-changeset note about the dev-test cleanup. Slightly confusing to have both, but it preserves attribution from the original PR(s) so probably worth keeping.
pranaygp
added
the
backport-stable
Contributor
|
Backport to To resolve manually: git fetch origin stable
git checkout stable
git cherry-pick 2f52d14f3844c999f6b89baeb8e04289d6dd34a9
# Fix conflicts, then:
git cherry-pick --continue
git push origin stable |
pranaygp
added a commit
that referenced
this pull request
May 2, 2026
…ignal * origin/main: [vitest] [world-local] Fix local-world data recovery isolation (#1895) ci: switch Vercel deployment-protection bypass to OIDC Trusted Sources (#1882) Add additional tests for event consumer fixes for hook/sleep/step race conditions (#1528) # Conflicts: # packages/core/src/workflow.test.ts
This was referenced May 2, 2026
VaguelySerious
added a commit
that referenced
this pull request
May 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #1818
Squashes the contents of #1818 (vitest project-context isolation + world-local recovery scoping by Tom Dale) together with two follow-up fixes for the e2e dev tests that surfaced once the vitest changes restored test isolation:
@workflow/vitestproject-scoped context.workflow()resolvescwd,rootDir,dataDir, andoutDironce and passes them through Vitest's per-project provided context.global-setupandsetup-fileconsume that resolved state instead of reading process-wide env vars, so workspace projects and config reloads no longer leak paths into one another. Adds a unit test suite for the harness.world-localrecovery scoped by tag.paginatedFileSystemQuerynow keepsfileIdFilterapplied across cursor pages (previously dropped after page one).start()gains arecoverActiveRunsopt-out, used by the Vitest harness so worker startup does not re-enqueue stale runs before direct handlers are registered.should include steps discovered from workflow importsnow tears down in-test and waits for the deferred builder to drop the discovered step from the manifest before the next test file runs, instead of relying onafterEach.should rebuild on imported step dependency changeswallows the Turbopack-on-WindowsMODULE_UNPARSABLEflake by rewriting the api file to invalidate Turbopack's bad cache and retrying. Both addressed cases where the Windows E2E job would burn its full 30-minute timeout polling stuck workflow runs.Test plan
pnpm exec vitest run packages/vitest/src/index.test.ts packages/world-local/src/fs.test.ts packages/world-local/src/reenqueue.test.tspnpm exec tsc -p packages/vitest/tsconfig.json --noEmitpnpm exec tsc -p packages/world-local/tsconfig.json --noEmite2e.test.tssucceed against the same dev server (verified locally on macOS)🤖 Generated with Claude Code