Skip to content

[e2e] Backport: separate event-log-race-repro harness noise from real regressions#2194

Merged
VaguelySerious merged 1 commit into
stablefrom
peter/event-log-race-repro-classify-infra-stable
Jun 1, 2026
Merged

[e2e] Backport: separate event-log-race-repro harness noise from real regressions#2194
VaguelySerious merged 1 commit into
stablefrom
peter/event-log-race-repro-classify-infra-stable

Conversation

@VaguelySerious
Copy link
Copy Markdown
Member

Backport of #2190 to stable.

The event-log-race-repro job previously dumped harness/timing/transport non-completions (HOOK_RESUME_FAILED, NO_WAKE_BRANCH, cancellations, transport errors) into the other bucket and gated the job on all of them — so a green SDK could still fail the run because the test harness lost a hook-resume timing race.

This backport:

  • Routes harness/timing/transport non-completions into a non-gating infra bucket; the --check gate now fails only on real event-log regressions (CORRUPTED_EVENT_LOG / USER_ERROR / RUNTIME_ERROR / stuck / other).
  • Sorts regressions ahead of infra in the failures table and annotates infra as non-gating in the summary.
  • Raises the hook/sleep iteration ceiling 5→8 so the resume reliably lands inside the sleep budget (free — the run short-circuits on wake).
  • Adds a coverage guard and makes the renderer unit-testable (node:test, new CI Scripts Tests job).

CI-only / test-only change; empty changeset.

🤖 Generated with Claude Code

@vercel
Copy link
Copy Markdown
Contributor

vercel Bot commented Jun 1, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
example-nextjs-workflow-turbopack Ready Ready Preview, Comment Jun 1, 2026 9:20am
example-nextjs-workflow-webpack Ready Ready Preview, Comment Jun 1, 2026 9:20am
example-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workbench-astro-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workbench-express-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workbench-fastify-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workbench-hono-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workbench-nitro-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workbench-nuxt-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workbench-sveltekit-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workbench-tanstack-start-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workbench-vite-workflow Ready Ready Preview, Comment Jun 1, 2026 9:20am
workflow-docs Ready Ready Preview, Comment, Open in v0 Jun 1, 2026 9:20am
workflow-swc-playground Ready Ready Preview, Comment Jun 1, 2026 9:20am
workflow-tarballs Ready Ready Preview, Comment Jun 1, 2026 9:20am
workflow-web Ready Ready Preview, Comment Jun 1, 2026 9:20am

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Jun 1, 2026

🦋 Changeset detected

Latest commit: c1d7bab

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 1, 2026

🧪 E2E Test Results

Some tests failed

Summary

Passed Failed Skipped Total
❌ ▲ Vercel Production 898 3 67 968
✅ 💻 Local Development 970 0 86 1056
✅ 📦 Local Production 970 0 86 1056
✅ 🐘 Local Postgres 970 0 86 1056
✅ 🪟 Windows 88 0 0 88
❌ 🌍 Community Worlds 135 87 0 222
✅ 📋 Other 492 0 36 528
Total 4523 90 361 4974

❌ Failed Tests

▲ Vercel Production (3 failed)

fastify (1 failed):

  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_01KT182YACK70KTRE6AYH5XZEQ | 🔍 observability

hono (2 failed):

  • DurableAgent e2e experimental_onStepStart (GAP) completes but callbacks are not called (GAP)
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context | wrun_01KT181988MW550677DZ4RH8NM | 🔍 observability
🌍 Community Worlds (87 failed)

mongodb (11 failed):

  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KT17NCRDVSWJ4R5FGT31E4TW
  • sleepingWorkflow | wrun_01KT17NQNM71HTQP75JH2R02H2
  • outputStreamWorkflow no startIndex (reads all chunks)
  • outputStreamWorkflow negative startIndex (reads from end)
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns correct index after stream completes
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns -1 before any chunks are written
  • outputStreamWorkflow - getTailIndex and getStreamChunks getStreamChunks returns same content as reading the stream
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions | wrun_01KT17S13AWAMYZAMB9PRYMEZY
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KT17XDZTH6VCW8NCJD6WSWT0
  • pages router sleepingWorkflow via pages router
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_01KT183NCD2RSF9XHVGME34YXV

redis (9 failed):

  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KT17NCRDVSWJ4R5FGT31E4TW
  • sleepingWorkflow | wrun_01KT17NQNM71HTQP75JH2R02H2
  • outputStreamWorkflow negative startIndex (reads from end)
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns correct index after stream completes
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns -1 before any chunks are written
  • outputStreamWorkflow - getTailIndex and getStreamChunks getStreamChunks returns same content as reading the stream
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KT17XDZTH6VCW8NCJD6WSWT0
  • pages router sleepingWorkflow via pages router
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_01KT183NCD2RSF9XHVGME34YXV

turso-dev (1 failed):

  • dev e2e should rebuild on imported step dependency change

turso (66 failed):

  • addTenWorkflow | wrun_01KT17M8N4JD0F0T8NYN1J5YM4
  • addTenWorkflow | wrun_01KT17M8N4JD0F0T8NYN1J5YM4
  • wellKnownAgentWorkflow (.well-known/agent) | wrun_01KT17PT3HE40S0VEY2SMW5H53
  • should work with react rendering in step
  • promiseAllWorkflow | wrun_01KT17MFMVJ0CRR6GQE14TZHS2
  • promiseRaceWorkflow | wrun_01KT17MNAQDF6BZ9E7M5HTPJZ3
  • promiseAnyWorkflow | wrun_01KT17MQHEF4Y6TBVRFD4GBJTH
  • importedStepOnlyWorkflow | wrun_01KT17Q39YM4JRGMP2EG04JD42
  • readableStreamWorkflow | wrun_01KT17MSRKZT1VJDS1NM4VS45Y
  • hookWorkflow | wrun_01KT17N5TXGH3TZ2534R1R39K1
  • hookWorkflow is not resumable via public webhook endpoint | wrun_01KT17NCRDVSWJ4R5FGT31E4TW
  • webhookWorkflow | wrun_01KT17NHA4JHK4Z0FBT04SRACF
  • sleepingWorkflow | wrun_01KT17NQNM71HTQP75JH2R02H2
  • parallelSleepWorkflow | wrun_01KT17P6W8EYD2BJPAKA14MZBA
  • nullByteWorkflow | wrun_01KT17PAQMZH52W6DPXCKYE45A
  • workflowAndStepMetadataWorkflow | wrun_01KT17PCXJWRRSKXZB05AJY9CR
  • outputStreamWorkflow no startIndex (reads all chunks)
  • outputStreamWorkflow positive startIndex (skips first chunk)
  • outputStreamWorkflow negative startIndex (reads from end)
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns correct index after stream completes
  • outputStreamWorkflow - getTailIndex and getStreamChunks getTailIndex returns -1 before any chunks are written
  • outputStreamWorkflow - getTailIndex and getStreamChunks getStreamChunks returns same content as reading the stream
  • outputStreamInsideStepWorkflow - getWritable() called inside step functions | wrun_01KT17S13AWAMYZAMB9PRYMEZY
  • fetchWorkflow | wrun_01KT17SE8TG22PQW3QGM1KN8BV
  • promiseRaceStressTestWorkflow | wrun_01KT17SHW8SH6Q93FATZ08D2RF
  • error handling error propagation workflow errors nested function calls preserve message and stack trace
  • error handling error propagation workflow errors cross-file imports preserve message and stack trace
  • error handling error propagation step errors basic step error preserves message and stack trace
  • error handling error propagation step errors cross-file step error preserves message and function names in stack
  • error handling retry behavior regular Error retries until success
  • error handling retry behavior FatalError fails immediately without retries
  • error handling retry behavior RetryableError respects custom retryAfter delay
  • error handling retry behavior maxRetries=0 disables retries
  • error handling catchability FatalError can be caught and detected with FatalError.is()
  • error handling not registered WorkflowNotRegisteredError fails the run when workflow does not exist
  • error handling not registered StepNotRegisteredError fails the step but workflow can catch it
  • error handling not registered StepNotRegisteredError fails the run when not caught in workflow
  • hookCleanupTestWorkflow - hook token reuse after workflow completion | wrun_01KT17X18G0AJM64G2WPKAAA2E
  • concurrent hook token conflict - two workflows cannot use the same hook token simultaneously | wrun_01KT17XDZTH6VCW8NCJD6WSWT0
  • hookDisposeTestWorkflow - hook token reuse after explicit disposal while workflow still running | wrun_01KT17XYHKF5S62GCDEDNEJP1G
  • stepFunctionPassingWorkflow - step function references can be passed as arguments (without closure vars) | wrun_01KT17YG32HTW9405W21W0Q6AE
  • stepFunctionWithClosureWorkflow - step function with closure variables passed as argument | wrun_01KT17YSCW0VPKFMTTNV5FZ99W
  • closureVariableWorkflow - nested step functions with closure variables | wrun_01KT17YZES28GW64BSK0S8KBR8
  • spawnWorkflowFromStepWorkflow - spawning a child workflow using start() inside a step | wrun_01KT17Z1YRWS8YCTA3GJE4PGEG
  • health check (queue-based) - workflow and step endpoints respond to health check messages
  • health check (CLI) - workflow health command reports healthy endpoints
  • pathsAliasWorkflow - TypeScript path aliases resolve correctly | wrun_01KT17ZJ49KJEDK5EQK3AH5CC2
  • Calculator.calculate - static workflow method using static step methods from another class | wrun_01KT17ZR04HV2PXKWBGR8H9855
  • AllInOneService.processNumber - static workflow method using sibling static step methods | wrun_01KT17ZYVEAXK6N845QW2CAXZF
  • ChainableService.processWithThis - static step methods using this to reference the class | wrun_01KT1805MAEMG387Z96PYVM4MB
  • thisSerializationWorkflow - step function invoked with .call() and .apply() | wrun_01KT180CMAP1D1FA9CHT2PSE74
  • customSerializationWorkflow - custom class serialization with WORKFLOW_SERIALIZE/WORKFLOW_DESERIALIZE | wrun_01KT180MRX8DP7RMCMJ2RF4AQV
  • instanceMethodStepWorkflow - instance methods with "use step" directive | wrun_01KT180VGWM0CJGTS6TCDR4CKH
  • crossContextSerdeWorkflow - classes defined in step code are deserializable in workflow context | wrun_01KT181988MW550677DZ4RH8NM
  • stepFunctionAsStartArgWorkflow - step function reference passed as start() argument | wrun_01KT181HAAM6KMWGQJZDDPP0W5
  • cancelRun - cancelling a running workflow | wrun_01KT181R434TVHK4PB9GBEWC0J
  • cancelRun via CLI - cancelling a running workflow | wrun_01KT1821H6FVSBCW6585RJ3QVZ
  • pages router addTenWorkflow via pages router
  • pages router promiseAllWorkflow via pages router
  • pages router sleepingWorkflow via pages router
  • hookWithSleepWorkflow - hook payloads delivered correctly with concurrent sleep | wrun_01KT182E90VNR2E3PS8CFAXGEG
  • sleepInLoopWorkflow - sleep inside loop with steps actually delays each iteration | wrun_01KT182YACK70KTRE6AYH5XZEQ
  • sleepWithSequentialStepsWorkflow - sequential steps work with concurrent sleep (control) | wrun_01KT1839YJJJHKF2F6YA94J87D
  • importMetaUrlWorkflow - import.meta.url is available in step bundles | wrun_01KT183GXP5R83CX4D0A4J44TP
  • metadataFromHelperWorkflow - getWorkflowMetadata/getStepMetadata work from module-level helper (#1577) | wrun_01KT183K85T2VMWR7XV4YCVCX4
  • resilient start: addTenWorkflow completes when run_created returns 500 | wrun_01KT183NCD2RSF9XHVGME34YXV

Details by Category

❌ ▲ Vercel Production
App Passed Failed Skipped
✅ astro 81 0 7
✅ example 81 0 7
✅ express 81 0 7
❌ fastify 80 1 7
❌ hono 79 2 7
✅ nextjs-turbopack 86 0 2
✅ nextjs-webpack 86 0 2
✅ nitro 81 0 7
✅ nuxt 81 0 7
✅ sveltekit 81 0 7
✅ vite 81 0 7
✅ 💻 Local Development
App Passed Failed Skipped
✅ astro-stable 82 0 6
✅ express-stable 82 0 6
✅ fastify-stable 82 0 6
✅ hono-stable 82 0 6
✅ nextjs-turbopack-canary 69 0 19
✅ nextjs-turbopack-stable 88 0 0
✅ nextjs-webpack-canary 69 0 19
✅ nextjs-webpack-stable 88 0 0
✅ nitro-stable 82 0 6
✅ nuxt-stable 82 0 6
✅ sveltekit-stable 82 0 6
✅ vite-stable 82 0 6
✅ 📦 Local Production
App Passed Failed Skipped
✅ astro-stable 82 0 6
✅ express-stable 82 0 6
✅ fastify-stable 82 0 6
✅ hono-stable 82 0 6
✅ nextjs-turbopack-canary 69 0 19
✅ nextjs-turbopack-stable 88 0 0
✅ nextjs-webpack-canary 69 0 19
✅ nextjs-webpack-stable 88 0 0
✅ nitro-stable 82 0 6
✅ nuxt-stable 82 0 6
✅ sveltekit-stable 82 0 6
✅ vite-stable 82 0 6
✅ 🐘 Local Postgres
App Passed Failed Skipped
✅ astro-stable 82 0 6
✅ express-stable 82 0 6
✅ fastify-stable 82 0 6
✅ hono-stable 82 0 6
✅ nextjs-turbopack-canary 69 0 19
✅ nextjs-turbopack-stable 88 0 0
✅ nextjs-webpack-canary 69 0 19
✅ nextjs-webpack-stable 88 0 0
✅ nitro-stable 82 0 6
✅ nuxt-stable 82 0 6
✅ sveltekit-stable 82 0 6
✅ vite-stable 82 0 6
✅ 🪟 Windows
App Passed Failed Skipped
✅ nextjs-turbopack 88 0 0
❌ 🌍 Community Worlds
App Passed Failed Skipped
✅ mongodb-dev 5 0 0
❌ mongodb 58 11 0
✅ redis-dev 5 0 0
❌ redis 60 9 0
❌ turso-dev 4 1 0
❌ turso 3 66 0
✅ 📋 Other
App Passed Failed Skipped
✅ e2e-local-dev-nest-stable 82 0 6
✅ e2e-local-dev-tanstack-start-stable 82 0 6
✅ e2e-local-postgres-nest-stable 82 0 6
✅ e2e-local-postgres-tanstack-start-stable 82 0 6
✅ e2e-local-prod-nest-stable 82 0 6
✅ e2e-local-prod-tanstack-start-stable 82 0 6

📋 View full workflow run


Some E2E test jobs failed:

  • Vercel Prod: failure
  • Local Dev: success
  • Local Prod: success
  • Local Postgres: success
  • Windows: success

Check the workflow run for details.

⚠️ Community world tests failed (non-blocking):

  • Community Worlds: failure

Check the workflow run for details.

@VaguelySerious VaguelySerious marked this pull request as ready for review June 1, 2026 09:22
@VaguelySerious VaguelySerious requested a review from a team as a code owner June 1, 2026 09:22
@VaguelySerious VaguelySerious merged commit ac59dc3 into stable Jun 1, 2026
89 of 94 checks passed
@VaguelySerious VaguelySerious deleted the peter/event-log-race-repro-classify-infra-stable branch June 1, 2026 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant