fix(index): yield during resolution so the liveness watchdog can't kill a valid large index (#1091)#1105
Merged
Conversation
…ll a valid large index (#1091) The #850 liveness watchdog SIGKILLs a process whose main-thread event loop stalls past its window (60s default). It was extended to `index`/`init` in #999, but reference resolution and callback-edge synthesis run synchronously on that same thread — so on a large repo a legitimate, in-progress index gets killed, and users had to disable the watchdog entirely (CODEGRAPH_NO_WATCHDOG=1). Make the long synchronous spans yield cooperatively so the heartbeat keeps firing during real work, while a genuinely wedged span (which never reaches a yield) still trips the watchdog: - synthesizeCallbackEdges yields between its whole-graph passes, and the heavy scanners (closure-collection, event-emitter, JSX-child, object-registry, field-channel) yield within their loops; - batched resolution sub-chunks each batch with yields; - the deferred chained-call and this-member post-passes yield per ref. Behaviour-preserving — only timing changes; node/edge counts are identical. Validated end-to-end with the real watchdog armed at the default 60s: the released build is SIGKILLed partway through indexing the Swift compiler (27k files, ~1.1M edges) and the TypeScript compiler, while the fixed build indexes both to completion. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem (#1091)
Indexing a large project gets SIGKILLed partway through with "Main thread unresponsive for ~60s — killing the wedged process (#850)", even though the index is healthy and progressing. Users were forced to set
CODEGRAPH_NO_WATCHDOG=1. Reported on both a 22k-file repo (bar at 100% then wedge) and a 134-file repo (88%).Root cause
The #850 liveness watchdog SIGKILLs a process whose main-thread event loop stalls past its window (60s default). Its heartbeat is a
setIntervalon that thread. It was wired intoindex/initin #999 — but unlike theservedaemon (whose heavy work is off-thread in the parse worker / shelled out), reference resolution and callback-edge synthesis run synchronously on that same thread. On a large repo those spans legitimately run for minutes, so the heartbeat can't fire and the watchdog kills valid, in-progress work. The progress bar freezes wherever it last rendered (88% / 100%) when a synchronous span begins.Fix
Make the long synchronous spans yield cooperatively (
src/resolution/cooperative-yield.ts, a budget-gatedmaybeYield()), so the heartbeat keeps firing during real work while a genuinely wedged span — which never reaches a yield — still trips the watchdog:synthesizeCallbackEdgesyields between its ~33 whole-graph passes, and the heavy scanners (closure-collection, event-emitter, JSX-child, object-registry, field-channel) yield within their loops;Behaviour-preserving — only timing changes; node/edge counts are identical.
Validation (real watchdog armed at the default 60s)
mainFull test suite green (1890). New regression test
__tests__/cooperative-yield.test.tspins the yielder's budget contract and locks the async structure of the three spans.Closes #1091.
🤖 Generated with Claude Code