fix(c, cpp): bound the type token run in function declarations - #4462
Merged
joshgoebel merged 2 commits intoAug 9, 2026
Merged
Conversation
The `FUNCTION_DECLARATION` matcher looked for `(TYPE[*&\s]+)+NAME(`, with an unbounded outer quantifier. On text that never reaches a function name the group first consumes the whole run of words, and the engine then retries the function name at every token boundary of that run before giving up. Since the scan repeats this at every start offset, highlighting cost grows with the square of the document size: 16 KB of `'a '` took ~7s and 64 KB took ~2min. Bound the repetition instead. Real declarations use a handful of type tokens, so 12 leaves plenty of room, and it makes the work per start offset constant, which puts highlighting back to linear time. Output is unchanged. Fixes highlightjs#4362
joshgoebel
approved these changes
Aug 9, 2026
Build Size ReportChanges to minified artifacts in 9 files changedTotal change +30 B View Changes
|
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #4362
Changes
FUNCTION_DECLARATIONinc.js/cpp.js(and soarduino) matches a run of type tokens followed by a function name:The outer quantifier is unbounded, so on text that never reaches a function name the group first consumes the whole run of words, and the engine then retries
FUNCTION_TITLEat every token boundary of that run before giving up. The scan repeats that at every start offset, so the cost grows with the square of the document size.Worth noting the ambiguity is not between
[\*&\s]+and the outer+, as the issue guesses —FUNCTION_TYPE_REcan't start with whitespace, so each token/separator split is forced and shortening[\*&\s]+always leaves the next iteration on a separator, where it fails immediately. The quadratic term is purelyO(start offsets) × O(iterations to backtrack), which is why the fix has to bound the iteration count: for any(A)+B, a backtracking engine retriesBonce per iteration whenBfails, so limiting how far the group can reach is the only way to make the work per start offset constant.Bounding it to 12 tokens leaves plenty of headroom —
static const volatile unsigned long long int * const * restrict fn(void)is 11 — and puts highlighting back to linear time.Why not
[\*&\s]*On the suggestion in the thread (comment) to relax the inner
+to*: that one makes things considerably worse. With*,FUNCTION_TYPE_REcan repeat with no separator in between, and(ident)+is ambiguous for any run of word characters — so it turns polynomial backtracking into exponential.'a'.repeat(n), no function name in sight:That's ~4× per 2 characters, so a 40-character identifier is already hours. It's also rejected by our own regex suite — with
*in place,ONLY_LANG=c npx mocha test/regexfails both checks:The current
+version passes those checks, which is presumably why this survived: the exponential test only looks atAvsA{2,}disjointness, and the polynomial test only compares adjacent single-character quantifiers, so neither can see the(A sep)+ Bshape this issue is about.Verification
hljs.highlight('a '.repeat(n), { language, ignoreIllegals: true }), node 22, same machine:ccppandarduinobehave the same;arduinowas the worst before, at 22.6s for 32 KB.Highlighting is unchanged. I hashed the rendered HTML of a corpus before and after — all 230 C headers and 192 libc++ headers in the macOS SDK, plus every
c/cpp/arduinofixture undertest/, each run through all three grammars (2442 outputs, 18.5 MiB) — and all 2442 hashes are identical. As a side effect that corpus now highlights in 8.5s instead of 22.8s, since real headers contain plenty of comment prose that was hitting the backtracking path.npm testis green (1597 passing),npm run lintandnpm run lint-languagesclean.Not fixed here
A single long identifier (
'a'.repeat(n), no spaces) is also quadratic, but that one isn't specific to this grammar —javascriptandjavashow the same curve, and it comes from\w*being re-scanned at every start offset rather than from this quantifier. Unchanged by this PR; happy to open a separate issue if that's useful.Checklist
test/parser/function-declaration-backtracking.jsasserts a 32 KB adversarial payload highlights well inside a loose 1s budget for all three languages, and that a declaration with 11 type tokens still gets its title highlighted. Reverting just the grammar change fails the three timing assertions (18.4s / 16.9s / 16.6s against the 1s budget) while the title assertion keeps passing.CHANGES.md