feat(compile): #348 Phase A — native CommonJS support for compilePackages#359
Merged
Conversation
…ages (v0.5.433) Closes the React-class blocker surfaced in the #348 scoping pass: any package shipping CJS (the dominant shape on npm — React 18, react-dom, react-reconciler, scheduler, lots of older libs) hit `Undefined symbols: _perry_fn_<pkg>_<file>__<name>` at link time because Perry's native `compilePackages` pipeline only handles ESM, and `module.exports = require('./cjs/<file>')` lowered `module` / `require` to bare-identifier-zero — every named export silently vanished. Implementation: source-level CJS-to-ESM wrap before SWC parses, modeled on the V8-fallback wrap at perry-jsruntime/src/modules.rs:481. New crates/perry/src/commands/compile/cjs_wrap.rs: - is_commonjs heuristic (matches module.exports / exports. / require( without import). - wrap_commonjs hoists each `require('X')` as `import _req_N from 'X'`, wraps the body in an IIFE that defines module = { exports: {} } plus a synchronous require(specifier) dispatching to the _req_N bindings, returns module.exports as _cjs, then emits `export default _cjs` plus `export const X = _cjs.X` for each detected named export. - Two named-export sources unioned: `exports.X = ...` patterns in the file itself (the existing jsruntime regex), AND for "trivial re-export wrappers" (`module.exports = require('./X')`) the patterns of the recursively-required target file. Without the second source, react/index.js (whose only meaningful statements are two conditional `module.exports = require(...)` calls inside a process.env.NODE_ENV check) produces zero named exports of its own and the link still fails. Wired in one place: collect_modules.rs:114 after read_to_string, gated on is_in_compiled_pkg so user TypeScript and ESM-shaped packages skip the wrap untouched. Verified end-to-end on a react-only sandbox: import { createContext, useState } from "react"; const Ctx = createContext(null); console.log(typeof createContext, typeof useState, typeof Ctx); compiles to a 2.2 MB single-file native binary, runs to exit 0, prints `function function function`. Matches Node on lines 1-2; line 3 returns `function` instead of Node's `object` — separate cross-module CJS-call return-value typeof bug, follow-up under #348 Phase B. End-to-end ink compile flips 46/67 modules from JS-runtime fallback to native (was 0/67 — every react.*/ink/build/* file now compiles natively). The remaining 21 modules still routed to V8 fallback are ink's transitive deps not declared in compilePackages (chalk / scheduler / react-reconciler / yoga-layout / etc) — Phase B. Out of scope (separate sub-issues under #348): - dynamic `require(someVar)` — throws at IIFE runtime, unrepresentable as static ESM - process.env.NODE_ENV static evaluation — both branches' requires hoist as imports, harmless modulo compile time - cross-module CJS-call typeof bug - ink's transitive deps (process import resolution, dynamic import(), yoga, react-reconciler) 7 unit tests in cjs_wrap.rs covering: detection on each shape, named- export extraction skipping __esModule, require-specifier dedup, end-to- end wrap round-trip with default + named exports + require import hoisting. New direct dep `regex = "1"` in crates/perry/Cargo.toml (already in the transitive surface via perry-jsruntime).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the React-class blocker surfaced in the #348 scoping pass: any package shipping CJS (the dominant shape on npm — React 18, react-dom, react-reconciler, scheduler, lots of older libs) hit
Undefined symbols: _perry_fn_<pkg>_<file>__<name>at link time because Perry's nativecompilePackagespipeline only handled ESM.module.exports = require('./cjs/<file>')loweredmodule/requireto bare-identifier-zero, so every named export silently vanished.Approach
Source-level CJS-to-ESM wrap before SWC parses, modeled on the V8-fallback wrap at
perry-jsruntime/src/modules.rs:481.New
crates/perry/src/commands/compile/cjs_wrap.rs(~210 LOC + 7 unit tests):is_commonjsheuristic — matchesmodule.exports/exports./require(withoutimport.wrap_commonjs— hoists eachrequire('X')asimport _req_N from 'X', wraps the body in an IIFE that definesmodule = { exports: {} }plus a synchronousrequire(specifier)dispatching to the_req_Nbindings, returnsmodule.exportsas_cjs, then emitsexport default _cjsplusexport const X = _cjs.Xfor each detected named export.exports.X = ...patterns in the file itself (the existing jsruntime regex), AND for trivial re-export wrappers (module.exports = require('./X')) the patterns of the recursively-required target file. Without the second source,react/index.js(whose only meaningful statements are two conditionalmodule.exports = require(...)calls inside aprocess.env.NODE_ENVcheck) produces zero named exports of its own and the link still fails.Wired in one place:
collect_modules.rs:114afterread_to_string, gated onis_in_compiled_pkgso user TypeScript and ESM-shaped packages skip the wrap untouched.Verified
End-to-end on a
react-onlysandbox:compiles to a 2.2 MB single-file native binary, runs to exit 0, prints
function function function. Matches Node on lines 1–2; line 3 returnsfunctioninstead of Node'sobject— separate cross-module CJS-call return-value typeof bug, follow-up.End-to-end ink compile flips 46/67 modules from JS-runtime fallback to native (was 0/67 — every
react.*/ink/build/*file now compiles natively). The remaining 21 modules still routed to V8 fallback are ink's transitive deps not declared incompilePackages(chalk / scheduler / react-reconciler / yoga-layout / etc) — separate follow-up.Out of scope (separate sub-issues under #348)
require(someVar)— throws at IIFE runtime, unrepresentable as static ESM.process.env.NODE_ENVstatic branch evaluation — both branches'requires hoist as imports, harmless modulo compile time.processimport resolution, dynamicimport(), yoga, react-reconciler).Test plan
cargo build --release -p perrycleancjs_wrap.rs(detection heuristics, regex extraction, end-to-end wrap shape)react-onlysandbox compiles + runs + matches Node on lines 1–2inksandbox: 46/67 native (was 0/67)Issue: #348