Skip to content

fix(resolution): 20+ min PHP/JS indexing wedge at "Resolving refs 97%" — gate closure-collection synthesis to Swift/Kotlin (#1235)#1237

Merged
colbymchenry merged 1 commit into
mainfrom
fix/1235-closure-collection-lang-gate
Jul 10, 2026
Merged

fix(resolution): 20+ min PHP/JS indexing wedge at "Resolving refs 97%" — gate closure-collection synthesis to Swift/Kotlin (#1235)#1237
colbymchenry merged 1 commit into
mainfrom
fix/1235-closure-collection-lang-gate

Conversation

@colbymchenry

Copy link
Copy Markdown
Owner

Fixes #1235.

Symptom

A 12,860-file PHP/JS app that indexed in 1m11s on 0.9.9 takes 24+ minutes on 1.3.1, apparently stuck at "Resolving refs 97%". With the watchdog on, the run is killed ("Main thread unresponsive for ~60s", #850); with CODEGRAPH_NO_WATCHDOG=1 it grinds for 24 minutes and counting. Reproduced on CRMEB (a 2,913-file ThinkPHP mall app with Chinese-language content): CODEGRAPH_SYNTH_TIMINGS=1 + --cpu-prof showed closureCollEdges: 127191ms127s of a 166s index in one synthesis pass, producing zero edges.

Root cause

closureCollectionEdges (the Alamofire-shape closure-collection dispatch synthesizer) scanned every method/function node in every language, but its dispatcher patterns ({ $0( / { it( ) are Swift/Kotlin trailing-closure syntax — on a PHP/JS repo the pass cannot emit a single edge, yet .push(/.add( fires its registrar gate on nearly every JS function. Three compounding costs:

  1. No language gate — the scan sliced and regexed ~17k node sources for nothing (CRMEB: 0 Swift/Kotlin nodes, 0 edges possible).
  2. lineAt was O(source-length) per regex match (src.slice(0, idx).split('\n')) — quadratic on match-dense generated functions, and roughly 2× worse on two-byte (e.g. Chinese) content (V8 FindTwoByteStringIndices was 40% of the whole index in the profile).
  3. Yields only every 128 nodes, never inside a node's match loops — one giant generated function blocks the main thread past the watchdog's 60s window, so the watchdog kills a healthy index.

The pass itself is unchanged since 0.9.x — what regressed is input volume: 1.x extraction produces far more method/function nodes on the same repo, pushing the always-latent quadratic over the cliff. The frozen "97%" is the last batched-ref progress update; synthesis runs after it with no progress reporting.

Fix

  • Language gate: skip nodes whose language isn't swift/kotlin before any file I/O. This is also a precision guard — pairing a JS .push( registrar with a Swift dispatcher's field name would be a wrong edge, not a missed one.
  • makeLineAt(): lazy newline index + binary search replaces the per-match slice+split, here and in the two emitter passes' per-file lineOf.
  • Per-match yields (every 256 regex matches) so a single pathological function can never starve the codegraph cause 100% CPU #850 watchdog.

Verification

Repo Before After Graph
CRMEB (ThinkPHP, 2,913 files) 166.8s (closureColl 127.2s) 72.7s byte-identical (32,220 nodes / 177,791 edges)
Alamofire (the canonical closure-collection repo) 9 closure-collection edges 9 edges, byte-identical (streams/validators/finishHandlers/requestsToRetry) identical totals
Drupal core (15,005-file control) byte-identical (142,319 nodes / 381,490 edges)

Full suite: 132/132 files, 2,257 passed. New tests: a JS fixture containing every textual trigger (.forEach, .push(, a { $0( lookalike in a string) must produce zero closure-collection edges, and the existing Swift fixture now pins the exact wiring-site line so the binary-search resolver is held to the old answer.

🤖 Generated with Claude Code

…d de-quadratic its line accounting (#1235)

closureCollectionEdges scanned every method/function node in every
language, but its dispatcher patterns ({ $0( / { it( ) are Swift/Kotlin
trailing-closure syntax — on PHP/JS repos the pass can never emit an
edge, yet .push(/.add( fired its append gate on nearly every function.
Per match it computed the line via src.slice(0, idx).split('\n'), which
is O(source) per match and goes quadratic on match-dense generated
functions (two-byte content roughly doubles it). On a 12,860-file
PHP/JS app that was 20+ minutes of the "Resolving refs" tail — frozen
at 97% — and a #850 watchdog kill; profiled on CRMEB it was 127s of a
166s index for zero edges.

- Skip nodes whose language isn't swift/kotlin before any file I/O.
- makeLineAt(): lazy newline index + binary search, shared with the
  emitter passes' per-file lineOf.
- Yield every 256 regex matches inside the scan's match loops so a
  single pathological function can't starve the watchdog.

CRMEB (ThinkPHP, 2,913 files): 166.8s -> 72.7s, graph byte-identical.
Alamofire: closure-collection edges byte-identical (9 edges, 4 fields).
Drupal core control: graph byte-identical.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@colbymchenry colbymchenry merged commit e76a355 into main Jul 10, 2026
@colbymchenry colbymchenry deleted the fix/1235-closure-collection-lang-gate branch July 10, 2026 01:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

codegraph 1.3.1 init takes 23min and still processing

1 participant