fix(hir): route node-core module spread calls through the variadic dispatch - #7726
Conversation
…spatch `path.join(...parts)` threw `TypeError [ERR_INVALID_ARG_TYPE]` while the identical non-spread call succeeded (#7720). Every native-module fast path in `lower_call` consumes its arguments POSITIONALLY, and a spread argument is lowered as one expression holding the whole array. `path.join(...parts)` therefore reached the `args.len() == 1` arm as `PathNormalize(<array>)`; the same fold made `util.format(...args)` inspect its array instead of formatting it and `fs.existsSync(...args)` test an array for existence. Decline the whole fast-path chain when the callee is a node-core module namespace method (or a named export of one) and any argument is spread. The fall-through tail then builds an `Expr::CallSpread` over the namespace member — the lowering the value-read form (`const j = path.join; j(...parts)`) already takes, which materializes the argument array and dispatches through `js_native_call_method` -> `dispatch_native_module_method`. That dispatcher is variadic by construction, so it gets both the valid case and Node's `ERR_INVALID_ARG_TYPE` for an invalid one right. Generalizes the per-module bail #6668 added for `crypto`. Scoped to node-core modules: an ext/npm native module (mysql2, redis, node-forge) has no by-name runtime dispatcher behind its codegen-wired rows, so declining its fast path would trade a wrong answer for no answer. Native CLASS statics (`Buffer.concat`, `URL.parse`) are excluded for the same reason. Tests: `native_module_spread_tests.rs` asserts the verdict in both directions — a spread call is diverted, a non-spread call still gets `PathJoin` — because both a fixed and a fully-disabled fast path produce correct output. Behaviour is byte-compared against node in three new node-suite fixtures.
7d9bec2 to
c175383
Compare
|
Caution Review failedAn error occurred during the review process. Please try again later. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Merging as v0.5.1421The best thing in this PR is the observation that the existing coverage could not fail, and I verified it before trusting it. for (const args of [["a", 1], ["a", null], ["a", {}], ["a", []]] as any[]) {
try { console.log("join:", path.join(...args)); } catch (err: any) { ... }
}Every case contains a non-string Node rejects, so the broken lowering (which rejects the whole array) and the correct one (which rejects the offending element) throw the identical The root cause is general, and the table proves it isn't a Declining the fast path rather than teaching each one about spread is the right shape. The fall-through already builds The out-of-scope section is the part I'd want kept. Both exclusions are argued from what the alternative would cost, not from effort:
The two #6668 crypto guards are correctly retained (verified at Verification
Full The |
|
Marking draft: a same-host A/B (this tree vs. the same tree with the guard forced to Cause: the predicate recurses through any sub-namespace receiver, but only some sub-namespaces have a runtime dispatch bucket ( Running a 32-case A/B matrix over the node-core spread surface (both arms + node oracle) to set that list from measurement rather than guesswork; will push the narrowed predicate and re-request review. |
…tring bridge Follow-up to #7726, which routed node-core module spread calls through the variadic runtime dispatch. A 32-case A/B matrix (this tree vs. the same tree with the guard forced to `false`, both against node 26.5.1) found two calls that were CORRECT before and `undefined` after, plus two wrong-to-differently-wrong conversions. All four are addressed here; the matrix is now 17 fixed / 13 same / 2 wrong-to-wrong / 0 regressed. 1. `querystring.escape(...args)` regressed to `undefined`. Root cause is older and wider than the spread bail: `nm_dispatch_querystring` advertises escape/unescape/stringify/encode/parse/decode, but the stdlib bridge it calls (`js_querystring_native_dispatch`) implemented ONLY `unescapeBuffer` and fell to `_ => undefined` for everything else. So on main every indirect form was already silently undefined — `const d: any = qs; d.escape("a b")`, `const e = qs.escape; e("a b")` — while the statically dispatched `qs.escape("a b")` was correct. #7726 merely routed spread calls onto that hole. Wire the remaining six names to the `js_querystring_*` entry points that already existed (`encode`/`decode` are Node's aliases for `stringify`/`parse`), which fixes the regression and the pre-existing captured/dynamic forms together. 2. `fs.promises` / `dns.promises` are not dispatch buckets. The predicate recursed through any sub-namespace receiver and treated any `<module>/<export>` that happened to be a node-core module name as a namespace. `nm_module_index` has DOTTED tags only for `path.posix`, `path.win32`, `util.types`, `crypto.subtle`/`webcrypto` and `punycode.ucs2`; there is no `fs.promises` bucket. Diverting the bucket-less ones produced a silent `undefined` (`dns.promises.lookup(...args)`) and a synchronous `TypeError: value is not a function` where a rejected promise used to arrive (`import { promises } from "node:fs"`). Replace the derivation with an explicit allowlist, and reject the slash sub-module tags: the direct import (`import fsp from "node:fs/promises"`) already reaches the generic tail without the bail, measured identical on both arms, so excluding them costs nothing. Known and deliberate: `events.listenerCount(...args)` still changes a bogus `ERR_INVALID_ARG_TYPE` throw into `undefined` — `nm_dispatch_events` implements only `init` and `EventEmitterAsyncResource`, so the dispatcher has no arm to reach. Both forms are wrong (node returns a count); completing that dispatcher is its own change. Tests: `sub_namespace_allowlist_is_the_runtime_bucket_set` pins the allowlist against exactly the re-derivation that shipped, `bucketless_sub_namespaces_keep_ their_lowering` pins the two HIR verdicts that changed, and `node-suite/querystring/aliases/dynamic-dispatch.ts` byte-compares the static, captured, dynamic and spread forms of six querystring methods against node.
…tring bridge (#7734) * fix(hir,stdlib): narrow the #7726 spread bail and complete the querystring bridge Follow-up to #7726, which routed node-core module spread calls through the variadic runtime dispatch. A 32-case A/B matrix (this tree vs. the same tree with the guard forced to `false`, both against node 26.5.1) found two calls that were CORRECT before and `undefined` after, plus two wrong-to-differently-wrong conversions. All four are addressed here; the matrix is now 17 fixed / 13 same / 2 wrong-to-wrong / 0 regressed. 1. `querystring.escape(...args)` regressed to `undefined`. Root cause is older and wider than the spread bail: `nm_dispatch_querystring` advertises escape/unescape/stringify/encode/parse/decode, but the stdlib bridge it calls (`js_querystring_native_dispatch`) implemented ONLY `unescapeBuffer` and fell to `_ => undefined` for everything else. So on main every indirect form was already silently undefined — `const d: any = qs; d.escape("a b")`, `const e = qs.escape; e("a b")` — while the statically dispatched `qs.escape("a b")` was correct. #7726 merely routed spread calls onto that hole. Wire the remaining six names to the `js_querystring_*` entry points that already existed (`encode`/`decode` are Node's aliases for `stringify`/`parse`), which fixes the regression and the pre-existing captured/dynamic forms together. 2. `fs.promises` / `dns.promises` are not dispatch buckets. The predicate recursed through any sub-namespace receiver and treated any `<module>/<export>` that happened to be a node-core module name as a namespace. `nm_module_index` has DOTTED tags only for `path.posix`, `path.win32`, `util.types`, `crypto.subtle`/`webcrypto` and `punycode.ucs2`; there is no `fs.promises` bucket. Diverting the bucket-less ones produced a silent `undefined` (`dns.promises.lookup(...args)`) and a synchronous `TypeError: value is not a function` where a rejected promise used to arrive (`import { promises } from "node:fs"`). Replace the derivation with an explicit allowlist, and reject the slash sub-module tags: the direct import (`import fsp from "node:fs/promises"`) already reaches the generic tail without the bail, measured identical on both arms, so excluding them costs nothing. Known and deliberate: `events.listenerCount(...args)` still changes a bogus `ERR_INVALID_ARG_TYPE` throw into `undefined` — `nm_dispatch_events` implements only `init` and `EventEmitterAsyncResource`, so the dispatcher has no arm to reach. Both forms are wrong (node returns a count); completing that dispatcher is its own change. Tests: `sub_namespace_allowlist_is_the_runtime_bucket_set` pins the allowlist against exactly the re-derivation that shipped, `bucketless_sub_namespaces_keep_ their_lowering` pins the two HIR verdicts that changed, and `node-suite/querystring/aliases/dynamic-dispatch.ts` byte-compares the static, captured, dynamic and spread forms of six querystring methods against node. * docs(changelog): fragment for #7734 * chore: bump version to 0.5.1426 Claude-Session: https://claude.ai/code/session_01Y1QZ5wUP9gRSwpiweT4Wix --------- Co-authored-by: Ralph Küpper <ralph@skelpo.com>
Fixes #7720.
The bug
Every native-module fast path in
lower_callconsumes its argumentspositionally, and a spread argument is lowered as a single expression
holding the whole array.
path.join(...parts)therefore reached theargs.len() == 1arm and becamePathNormalize(<array>), which rejected thearray as a non-string.
It is not a
pathbug. The same positional fold is why, onmain:path.join(...['a','b'])TypeError [ERR_INVALID_ARG_TYPE]a/butil.format(...['x=%s','X'])[ 'x=%s', 'X' ]x=Xfs.existsSync(...['/tmp'])falsetrueThe fix
When any argument is spread and the callee is a node-core module namespace
method (or a named export of one), decline the whole fast-path chain. The
fall-through tail then builds an
Expr::CallSpreadover the namespace member —exactly the lowering the value-read form already takes:
Codegen materializes the argument array and dispatches through
js_closure_call_apply_with_spread→js_native_call_method→dispatch_native_module_method, which is variadic by construction. It gets thevalid case right and reproduces Node's
ERR_INVALID_ARG_TYPE(with thematching
code) for an invalid one.This generalizes the per-module bail #6668 added for
crypto; those twocrypto guards stay, because they also cover the bare
cryptoglobalreceiver, which is not an import and so is invisible to the new predicate.
What is deliberately out of scope
NativeMethodCallrows are wired in codegen with no by-name runtimedispatcher behind them, so declining their fast path would trade a wrong
answer for no answer.
Buffer.concat(...list),URL.parse(...)). Adifferent lowering family whose dynamic dispatch does not cover the same
surface —
Buffer.concatis already broken through the dynamic path for theplain
B.concat(list)call, independent of spread, so routing spread onto itwould swap a loud failure for a silent wrong answer.
Both exclusions are asserted by tests, not just described here.
Tests
crates/perry-hir/src/lower/expr_call/native_module_spread_tests.rs(6 tests,cargo test -p perry-hir --lib, so per-PR-visible) asserts which lowering acall got, in both directions:
CallSpread;path.join('a','b')still lowers toPathJoin.The second half matters: the generic dispatch is a correct fallback, so a
regression that disabled the fast path everywhere would still print the right
answer. Both halves were sabotage-checked — with the guard forced to
false,3 of the 6 fail; with the fast path removed, the other half fails.
This is also why the existing
node-suite/path/join/type-errors-extra.tsalready contained
path.join(...args)and stayed green through the whole lifeof the bug: it only spreads segments Node rejects, so both the broken and the
correct lowering throw
ERR_INVALID_ARG_TYPE. CLAUDE.md's fourth way a gatecan be unable to fail — the gate ran, its subject never did. The new
node-suite/path/join/spread.tssupplies the valid-segment cases the issueasked for.
Byte-compared against node
26.5.1:test-parity/node-suite/path/join/spread.ts— namespace / namespace-import /require-alias / named-import /
posix/win32/ sub-namespace alias /value-read / in-loop forms, plus mixed, trailing, single, empty and
normalizing segment lists
test-parity/node-suite/path/resolve/spread.ts— the reset-on-absolute siblingtest-parity/node-suite/util/format/spread.tsVerification
run_parity_tests.sh --suite node-suite --module path— 94/94, 100%(including the two new fixtures)
cargo test -p perry-hir --lib— greenutil.format/fs.existsSync/os.homedir/fs.writeFileSync/fs/promises.readFile/crypto.createHash/named-import and
node:console/node:processnamespace forms, all nowbyte-identical to node
Math.min(...xs),arr.push(...xs),console.log(...xs)One unrelated pre-existing gap surfaced while testing and is not touched
here:
Object.assign(target, ...sources)passes the sources array as oneargument (
{"0":{...},"1":{...}}instead of a merge). Different family(
Objectstatics, not a node module) — worth its own issue.