fix(resolution): 20+ min PHP/JS indexing wedge at "Resolving refs 97%" — gate closure-collection synthesis to Swift/Kotlin (#1235)#1237
Merged
Conversation
…d de-quadratic its line accounting (#1235) closureCollectionEdges scanned every method/function node in every language, but its dispatcher patterns ({ $0( / { it( ) are Swift/Kotlin trailing-closure syntax — on PHP/JS repos the pass can never emit an edge, yet .push(/.add( fired its append gate on nearly every function. Per match it computed the line via src.slice(0, idx).split('\n'), which is O(source) per match and goes quadratic on match-dense generated functions (two-byte content roughly doubles it). On a 12,860-file PHP/JS app that was 20+ minutes of the "Resolving refs" tail — frozen at 97% — and a #850 watchdog kill; profiled on CRMEB it was 127s of a 166s index for zero edges. - Skip nodes whose language isn't swift/kotlin before any file I/O. - makeLineAt(): lazy newline index + binary search, shared with the emitter passes' per-file lineOf. - Yield every 256 regex matches inside the scan's match loops so a single pathological function can't starve the watchdog. CRMEB (ThinkPHP, 2,913 files): 166.8s -> 72.7s, graph byte-identical. Alamofire: closure-collection edges byte-identical (9 edges, 4 fields). Drupal core control: graph byte-identical. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1235.
Symptom
A 12,860-file PHP/JS app that indexed in 1m11s on 0.9.9 takes 24+ minutes on 1.3.1, apparently stuck at "Resolving refs 97%". With the watchdog on, the run is killed ("Main thread unresponsive for ~60s", #850); with
CODEGRAPH_NO_WATCHDOG=1it grinds for 24 minutes and counting. Reproduced on CRMEB (a 2,913-file ThinkPHP mall app with Chinese-language content):CODEGRAPH_SYNTH_TIMINGS=1+--cpu-profshowedclosureCollEdges: 127191ms— 127s of a 166s index in one synthesis pass, producing zero edges.Root cause
closureCollectionEdges(the Alamofire-shape closure-collection dispatch synthesizer) scanned every method/function node in every language, but its dispatcher patterns ({ $0(/{ it() are Swift/Kotlin trailing-closure syntax — on a PHP/JS repo the pass cannot emit a single edge, yet.push(/.add(fires its registrar gate on nearly every JS function. Three compounding costs:lineAtwas O(source-length) per regex match (src.slice(0, idx).split('\n')) — quadratic on match-dense generated functions, and roughly 2× worse on two-byte (e.g. Chinese) content (V8FindTwoByteStringIndiceswas 40% of the whole index in the profile).The pass itself is unchanged since 0.9.x — what regressed is input volume: 1.x extraction produces far more method/function nodes on the same repo, pushing the always-latent quadratic over the cliff. The frozen "97%" is the last batched-ref progress update; synthesis runs after it with no progress reporting.
Fix
swift/kotlinbefore any file I/O. This is also a precision guard — pairing a JS.push(registrar with a Swift dispatcher's field name would be a wrong edge, not a missed one.makeLineAt(): lazy newline index + binary search replaces the per-match slice+split, here and in the two emitter passes' per-filelineOf.Verification
streams/validators/finishHandlers/requestsToRetry)Full suite: 132/132 files, 2,257 passed. New tests: a JS fixture containing every textual trigger (
.forEach,.push(, a{ $0(lookalike in a string) must produce zero closure-collection edges, and the existing Swift fixture now pins the exact wiring-site line so the binary-search resolver is held to the old answer.🤖 Generated with Claude Code