[miniflare] Surface workerd fatal crash reports instead of filtering them out

[petebacondarwin]

Describe your change...

fixtures/dev-registry has been failing on Windows CI at a high rate. Across 19 recent test-and-check runs where the fixture actually executed (it is often a turbo cache hit), the Windows fixtures job failed on it 7 times (37%), and a workerd crash was present in 11 (58%). macOS and Linux: 0.

The failure is always the same test — Dev Registry: vite dev <-> vite dev > supports exported handler fetch over service binding — and always Test timed out in 50000ms on both attempts. Every failing log contains this in exactly one of the vite dev child processes:

➜  Local:   http://localhost:5173/
*** std::terminate() called with no exception
9:43:12 AM [vite] server restarted.
*** std::terminate() called with no exception

That is workerd aborting, #handleWorkerdCrash respawning it, and the Vite plugin's unsafeHandleRuntimeRestart restarting the dev server — which then calls setOptions() and respawns workerd a second time. It costs ~30s per crash, which is what eats the test budget. (Notably it always hits the first-started session in the test, i.e. the one on port 5173, whichever config that happens to be.)

The problem is that the crash carries no diagnostic information, so there is nothing to act on. messageClassifiers.isInternal discards anything matching /stack:( (0|[a-f\d]{4,})){3,}/ — a pattern added for Windows hex stacks — and kj emits the crash stack as a separate line immediately after the banner. So the stack was being thrown away.

This PR fixes that, ahead of fixing the crash itself:

• Recognise kj's fatal crash banners (*** std::terminate(), *** Fatal uncaught, *** Received signal, *** Uncaught exception) and keep the stack: line and missing-$LLVM_SYMBOLIZER notice that follow one, reporting the whole report at error level. Address-in-use and access-violation handling keep priority, so their existing messages are unchanged.
• Create the wrapped log handler once per stream instead of once per chunk, so the latch survives the banner and its stack trace arriving in separate data events.
• Warn when the runtime is restarted after a crash. Recovery was completely silent, so a user on Windows today sees their dev server restart for no stated reason. The crash count distinguishes a one-off from a restart loop.

Before / after, from test/index.spec.ts:

# before (stdout, level "log")
*** Fatal uncaught kj::Exception: workerd/server/server.c++:4180: failed: abortIsolate() called, terminating process; reason = test crash

# after (stderr, level "error")
*** Fatal uncaught kj::Exception: workerd/server/server.c++:4180: failed: abortIsolate() called, terminating process; reason = test crash
stack: 10451606b 104c193a7 104846dd3 10588bb4b 105889d3f ...

This is a user-facing improvement in its own right, and it has already done its job here. With a temporary diagnostic on top (since reverted) the Windows fixtures job captured the crash for the first time:

  ➜  Local:   http://localhost:5173/          <- stdout
12:02:16 PM [vite] server restarted.          <- stdout
*** std::terminate() called with no exception <- stderr
stack: 7ff845f81249 7ff697ab63a5 7ff845f9baff 7ff845f83a9c 7ff845f9ae8c 7ff845f82ce1
       7ff845f9b8f7 7ff854e6700e 7ff854d0e592 7ff845f9b3d5 7ff845f81c1f 7ff845f82dd7
       7ff845f9b8f7 7ff854e66f8e 7ff854d12326 7ff854d0a960 7ff8525375f9 7ff845f855a8
       7ff69914045f 7ff697ab5f63 7ff699145512 7ff69914c416 7ff699145a6b 7ff69914c416
       7ff699142fbb 7ff697aa1e77 7ff69af7219e 7ff8540fe8d6 7ff854d8c53b
The Workers runtime crashed unexpectedly and is being restarted (crash #1). Any additional runtime output above may indicate the cause.

Two things fall straight out of that, neither of which was visible before:

• The stack is byte-identical across all 11 crashes in the run, so this is a deterministic code path rather than corruption or resource exhaustion. It bottoms out at BaseThreadInitThunk/RtlUserThreadStart with CRT and ntdll unwind frames on top, i.e. a C++ exception escaping a workerd worker thread, and it contains a repeated frame (…c416 twice) so something re-enters.
• It always hits the first-started session (port 5173), whichever config that happens to be, across both instrumented runs. That session is the only one that starts with an empty registry and has to learn about its peers via setRegistry pushes; later sessions boot with the registry already baked into their config. That asymmetry is the live lead.

Note on reading the excerpt above: the test helper accumulates stdout and stderr into separate buffers and the diagnostic printed them one after the other, so the relative order of the Vite line and the crash lines there is an artifact of concatenation, not real interleaving. (This change is what moved the banner from stdout to stderr, by reclassifying it to error.) Ordering has to be read from pre-change logs, where both appear on stdout — and those show the first crash arriving before any Vite restart, with the server restarted. in between being the crash recovery itself.

Follow-up work will fix the crash and reduce the recovery cost (one crash currently costs two workerd spawns plus a full Vite restart, ~30s, with no backoff).

---

• Tests
  • Tests included/updated
  • Automated tests not possible - manual testing has been completed as follows:
  • Additional testing not necessary because:
• Public documentation
  • Cloudflare docs PR(s):
  • Documentation not necessary because: this changes internal diagnostic output only. There is no new API, flag or configuration; users simply get the stack trace that workerd already emitted, plus a warning when the runtime is restarted after a crash.

A picture of a cute animal (not mandatory, but encouraged)

Note

This is a contribution from an AI agent: OpenCode, claude-opus-5.

[changeset-bot]

🦋 Changeset detected

Latest commit: 843e6da

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 8 packages

Name	Type
miniflare	Patch
@cloudflare/deploy-helpers	Patch
@cloudflare/pages-shared	Patch
@cloudflare/remote-bindings	Patch
@cloudflare/runtime-types	Patch
@cloudflare/vite-plugin	Patch
@cloudflare/vitest-pool-workers	Patch
wrangler	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[ask-bonk]

abortIsolate is an established pattern in existing tests. The new test follows the exact same pattern as the pre-existing crash-recovery tests.

I've completed a thorough review. The changes are well-designed:

• The isFatalCrash regex correctly matches kj's banner formats and is anchored with ^.
• Moving wrapStructuredLogsHandler out of the per-chunk callback correctly preserves the reportingFatalCrash latch across data events — this is the core fix and is correct.
• stdout/stderr use separate handler instances, but kj writes both banner and stack to stderr, so the shared latch works.
• Address-in-use and access-violation handling retain priority (checked before the latch), preserving existing behavior — verified by tests.
• The latch is cleared on the first ordinary log line, and the tests cover this.
• The crash-count warning is a genuine UX improvement; the test follows the established abortIsolate pattern.
• The changeset is correctly scoped to miniflare as a patch.

No logic bugs, security issues, backward-compatibility violations, or incorrect API behavior found. The edge cases I considered (a non-crash structured log whose message begins with *** , or a stray symbolizer notice being surfaced while a latch is stuck open) are theoretical and not realistic given workerd's actual output.

LGTM

github run

[pkg-pr-new]

@cloudflare/autoconfig

npm i https://pkg.pr.new/@cloudflare/autoconfig@14989

@cloudflare/build-output-utils

npm i https://pkg.pr.new/@cloudflare/build-output-utils@14989

@cloudflare/config

npm i https://pkg.pr.new/@cloudflare/config@14989

create-cloudflare

npm i https://pkg.pr.new/create-cloudflare@14989

@cloudflare/deploy-helpers

npm i https://pkg.pr.new/@cloudflare/deploy-helpers@14989

@cloudflare/kv-asset-handler

npm i https://pkg.pr.new/@cloudflare/kv-asset-handler@14989

miniflare

npm i https://pkg.pr.new/miniflare@14989

@cloudflare/pages-functions

npm i https://pkg.pr.new/@cloudflare/pages-functions@14989

@cloudflare/pages-shared

npm i https://pkg.pr.new/@cloudflare/pages-shared@14989

@cloudflare/unenv-preset

npm i https://pkg.pr.new/@cloudflare/unenv-preset@14989

@cloudflare/vite-plugin

npm i https://pkg.pr.new/@cloudflare/vite-plugin@14989

@cloudflare/vitest-pool-workers

npm i https://pkg.pr.new/@cloudflare/vitest-pool-workers@14989

@cloudflare/workers-auth

npm i https://pkg.pr.new/@cloudflare/workers-auth@14989

@cloudflare/workers-editor-shared

npm i https://pkg.pr.new/@cloudflare/workers-editor-shared@14989

@cloudflare/workers-utils

npm i https://pkg.pr.new/@cloudflare/workers-utils@14989

wrangler

npm i https://pkg.pr.new/wrangler@14989

commit: 843e6da

[github-actions]

✅ All changesets look good

[workers-devprod]

Codeowners approval required for this PR:

• ✅ @cloudflare/wrangler

Show detailed file reviewers

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

[workers-devprod]

Codeowners reviews satisfied