Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

Perry is a native TypeScript compiler written in Rust that compiles TypeScript source code directly to native executables. It uses SWC for TypeScript parsing and LLVM for code generation.

**Current Version:** 0.5.432
**Current Version:** 0.5.433


## TypeScript Parity Status
Expand Down Expand Up @@ -150,6 +150,7 @@ First-resolved directory cached in `compile_package_dirs`; subsequent imports re

Keep entries to 1-2 lines max. Full details in CHANGELOG.md.

- **v0.5.433** — Issue #348 Phase A — first cut: native CommonJS support for `compilePackages`. Closes the React-class blocker that made `import { useState } from "react"` link-fail with `Undefined symbols: _perry_fn_node_modules_react_index_js__useState` for any package shipping CJS — surfaced as the dominant bucket in the #348 scoping pass. New `crates/perry/src/commands/compile/cjs_wrap.rs` (~210 LOC + tests) lifted from the `perry-jsruntime/src/modules.rs:481` V8-fallback wrap pattern: heuristic `is_commonjs` (matches `module.exports` / `exports.` / `require(` without `import `), source-level `wrap_commonjs` that hoists each `require('X')` as `import _req_N from 'X';`, wraps the original body in an IIFE that defines `module = { exports: {} }` plus a synchronous `require(specifier)` dispatching to the `_req_N` bindings, and emits `export default _cjs;` plus `export const X = _cjs.X;` for each detected named export. Two named-export sources are unioned: `exports.X = ...` patterns in the file itself (the existing jsruntime regex), AND for **trivial re-export wrappers** (`module.exports = require('./X')`) the patterns of the recursively-required *target* file — without this, `react/index.js` (whose only meaningful statements are two conditional `module.exports = require(...)` calls inside `process.env.NODE_ENV === 'production'`) produces zero named exports of its own and the link still fails. Wired in one place: `collect_modules.rs:114` after `read_to_string`, gated on `is_in_compiled_pkg` so user TypeScript and ESM-shaped packages skip the wrap untouched. New `regex = "1"` direct dep in `crates/perry/Cargo.toml` (already in the transitive surface via `perry-jsruntime`). **Verified end-to-end** on a `react-only` sandbox (`import { createContext, useState } from "react"; const Ctx = createContext(null); console.log(typeof createContext, typeof useState, typeof Ctx)`) — compiles to a 2.2 MB single-file native binary, runs to exit 0, prints `function / function / function` (matches Node on lines 1-2; `Ctx` `typeof` returns `function` instead of Node's `object` — separate bug in cross-module CJS-call return-value typeof, follow-up). End-to-end ink compile gets 46/67 modules native (was 0/67 — every `react.*` and `ink/build/*` file now compiles natively); the remaining 21 modules going to V8 fallback are ink's transitive deps not in `compilePackages` (chalk / scheduler / react-reconciler / yoga-layout / etc) — Phase B work, separate sub-issues. **Out of scope for this slice**: dynamic `require(someVar)` (throws at IIFE runtime — unrepresentable as static ESM); `process.env.NODE_ENV` static evaluation (both branches' requires hoist as imports — both files compile, only one is reached at module init, harmless modulo compile time); cross-module CJS-call typeof bug; ink's transitive deps (`process` import resolution, dynamic `import()`, yoga, react-reconciler).
- **v0.5.432** — Phase 2 v12: 4 more ArkUI widgets in `perry-codegen-arkts`. **Tabs(`[{label, body}, ...]`)** → ArkUI `Tabs() { TabContent() {<body>}.tabBar('<label>'); ... }`. Each tab spec is an Object or `__AnonShape_*` New (same closed-shape support as v5 styling). Tab bodies recursively harvest like any other widget tree, so closure-bearing children compose with the v2 callback registry transparently. **Modal/Dialog** emits a placeholder with a Phase 2 v12.5 hand-off comment — real `AlertDialog.show({...})` integration needs a runtime FFI bridge (mirroring the showToast drain pattern) and is its own follow-up. **Menu/ContextMenu(`[{label, action}, ...]`)** → emits each action as a Button via the existing v2 emit_button pipeline so closure registration works automatically. Real `.bindMenu(...)` modifier integration on a triggering widget is v12.5. **Grid(columns, items)** → ArkUI `Grid() { GridItem() {...} }` with `.columnsTemplate('1fr 1fr 1fr')` (column count clamped 1..=12). 5 new unit tests (`tabs_emits_tabcontent_per_spec`, `menu_emits_buttons_per_item`, `grid_emits_columns_template_and_griditems`, `modal_emits_placeholder_with_runtime_hint`, plus closure-slot verification on Menu items). 42/42 perry-codegen-arkts unit tests pass total (was 37).
- **v0.5.431** — Phase 2 v8: HarmonyOS NEXT documentation page (`docs/src/platforms/harmonyos.md`). 200+ line guide covering architecture (harvest model + NAPI bridge + drain queue pattern), supported widgets/events/reactivity/styling/dynamic lists, setup via `perry setup harmonyos`, the canonical compile + splice + run workflow, known limitations, and per-version history. New SUMMARY.md entry alphabetically after Android. No code changes.
- **v0.5.430** — Closes #25 + cuts macOS doc-tests cross-compile cost. **(1) NJOBS=6 + flock** — restores compile-smoke's parallelism from NJOBS=3 to NJOBS=6 (the v0.5.384 retreat). The race that forced the retreat was: two `perry compile` workers targeting the same `target/perry-auto-<hash>/` invoke cargo concurrently, cargo's own `.cargo-lock` serializes the BUILD but worker B's clang opens `libperry_runtime.a` mid-rename when worker A's link step finishes — clang sees `errno=2`. Fix in `crates/perry/src/commands/compile/optimized_libs.rs::build_optimized_libs`: acquire an OS file lock on `<target_dir>/.perry-auto-build.lock` BEFORE the cargo invocation, hold it through the build. New direct dep `fslock = "0.2"` (already a transitive — gzip-header pulls it via bindgen, so no new crate cost) provides cross-platform `flock`/`LockFileEx`. Per-hash so different feature combos still parallelize; same hash serializes. Best-effort: dir creation and lock acquisition are both `match` arms that fall through unguarded on failure (no behavior regression vs pre-fix). NJOBS bumped 3 → 6 in `.github/workflows/test.yml::compile-smoke`; the retry-once safety net stays as belt-and-suspenders. Sequential baseline ~26 min; NJOBS=3 ~10-12 min; NJOBS=6 ~6-8 min on the ubuntu-latest 4-vCPU runner. **(2) macOS doc-tests cross-compile trim** — `cmd_xcompile_blocking` for the `macos-14` matrix entry stripped `--xcompile-only-target=web --xcompile-only-target=wasm`, keeping only `--xcompile-only-target=ios-simulator` (which needs Apple SDK and only macOS can run). The `ubuntu-24.04` entry already verifies web+wasm at 1× billing weight; removing the redundant macOS verification saves ~5-8 min × 10× = **50-80 multiplier-min/run**. Total per-run impact for v0.5.430 vs v0.5.428 baseline: compile-smoke 18 → ~6-8 ubuntu-min (saves 10-12 ubuntu-min) + doc-tests-macos drops 5-8 min × 10× = ~60 macos-min cut. **No code change beyond the locking helper; no parity change.**
Expand Down
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ opt-level = "s" # Optimize for size in stdlib
opt-level = 3

[workspace.package]
version = "0.5.432"
version = "0.5.433"
edition = "2021"
license = "MIT"
repository = "https://github.com/PerryTS/perry"
Expand Down
1 change: 1 addition & 0 deletions crates/perry/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ zip.workspace = true
jsonwebtoken = "9.3"
dotenvy = "0.15"
rayon = "1.10"
regex = "1"

# Cross-platform file locking (Unix flock + Windows LockFileEx) — used by
# the auto-optimize driver in commands/compile/optimized_libs.rs to
Expand Down
1 change: 1 addition & 0 deletions crates/perry/src/commands/compile.rs
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ use crate::OutputFormat;
// `compile/` directory. The `compile.rs` orchestrator stays as the
// public API surface; helpers move to focused modules so unrelated
// changes don't churn this file.
mod cjs_wrap;
mod collect_modules;
mod library_search;
mod link;
Expand Down
225 changes: 225 additions & 0 deletions crates/perry/src/commands/compile/cjs_wrap.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,225 @@
//! CommonJS-to-ESM source-level transformation for `compilePackages`.
//!
//! Closes the React-class blocker for issue #348 (ink-as-compilePackages).
//!
//! React 18 ships as CommonJS — `node_modules/react/index.js` does
//! `module.exports = require('./cjs/react.production.min.js')`, and the
//! actual implementation file uses `exports.useState = function() {...}`
//! patterns. Perry's native pipeline is ESM-only — `module`/`require` lower
//! to bare-identifier-zero, so the entire react module compiles to a no-op
//! and every downstream `import { useState } from "react"` link-fails with
//! `Undefined symbols: _perry_fn_node_modules_react_index_js__useState`.
//!
//! This module detects CJS at module-read time and rewrites the source to
//! ESM-shaped code before SWC parses it. The wrap pattern (modeled after
//! `perry-jsruntime/src/modules.rs:481` which already does this for the V8
//! fallback) is:
//!
//! 1. Hoist every `require('X')` call as `import _req_N from 'X';`.
//! 2. Wrap the CJS body in an IIFE that defines `module = { exports: {} }`,
//! a synchronous `require(specifier)` that dispatches to the hoisted
//! `_req_N` bindings, runs the original code, and returns
//! `module.exports`. The IIFE result is bound to `_cjs`.
//! 3. Emit `export default _cjs;` plus `export const X = _cjs.X;` for each
//! detected named export.
//!
//! Two named-export sources are unioned:
//!
//! - `exports.X = ...` patterns *in this file* (regex; the existing
//! jsruntime heuristic).
//! - For "trivial re-export wrappers" (`module.exports = require('./X')`,
//! optionally inside a `process.env.NODE_ENV` conditional), the
//! `exports.X = ...` patterns of the recursively-required *target* file.
//! Without this, react/index.js — whose only meaningful statements are
//! two conditional `module.exports = require(...)` calls — produces zero
//! named exports of its own and the link still fails. The recursion
//! follows up to a small depth (2 levels) to handle one level of env
//! switching; deeper indirection is rare and gets the no-op fallback.

use std::path::Path;

/// Heuristic CJS detection. Same shape as
/// `perry-jsruntime/src/modules.rs::is_commonjs`. False negatives are
/// acceptable (the file just falls through to the existing ESM-only
/// pipeline); false positives on a real ESM file would be more painful but
/// require a file that uses neither `module.exports` nor `exports.` nor
/// `require(` — i.e., an ESM file that *also* contains those tokens. Real
/// hybrid cases are rare and would need a `"type": "module"` package.json
/// override, which is the next refinement if this trips a real package.
pub(super) fn is_commonjs(source: &str) -> bool {
source.contains("module.exports")
|| source.contains("exports.")
|| (source.contains("require(") && !source.contains("import "))
}

/// Wrap CJS source as ESM. `source_path` is the absolute path of the file
/// being wrapped — used to resolve `require('./relative')` targets when
/// peeking at re-export wrappers' transitive named exports.
pub(super) fn wrap_commonjs(source: &str, source_path: &Path) -> String {
let require_specs = extract_require_specifiers(source);

let imports = require_specs
.iter()
.enumerate()
.map(|(i, spec)| format!("import _req_{} from '{}';", i, spec))
.collect::<Vec<_>>()
.join("\n");

let require_cases = require_specs
.iter()
.enumerate()
.map(|(i, spec)| format!(" if (specifier === '{}') return _req_{};", spec, i))
.collect::<Vec<_>>()
.join("\n");

let mut named_exports = extract_exports_from_source(source);

// For trivial re-export wrappers (`module.exports = require('./X')`),
// recursively pull in the target's named exports. Without this,
// react/index.js — which has zero `exports.X =` patterns of its own —
// produces zero named exports and downstream `import { useState } from
// "react"` link-fails.
let parent = source_path.parent();
if let Some(parent) = parent {
for spec in &require_specs {
if !spec.starts_with("./") && !spec.starts_with("../") {
continue;
}
let target = parent.join(spec);
if let Ok(target_source) = std::fs::read_to_string(&target) {
for name in extract_exports_from_source(&target_source) {
if !named_exports.contains(&name) {
named_exports.push(name);
}
}
}
}
}

let named_export_decls = if named_exports.is_empty() {
String::new()
} else {
named_exports
.iter()
.map(|n| format!("export const {} = _cjs.{};", n, n))
.collect::<Vec<_>>()
.join("\n")
};

format!(
r#"{imports}
const _cjs = (function() {{
const module = {{ exports: {{}} }};
const exports = module.exports;
function require(specifier) {{
{require_cases}
throw new Error('require() is not supported: ' + specifier);
}}

{source}

return module.exports;
}})();

export default _cjs;
{named_export_decls}
"#
)
}

/// Extract `require('X')` / `require("X")` specifiers, preserving order and
/// deduping. Only matches static string literal arguments — dynamic
/// `require(someVar)` is unrepresentable as ESM and the bound `require`
/// inside the IIFE will throw at runtime if hit.
fn extract_require_specifiers(source: &str) -> Vec<String> {
let re = regex::Regex::new(r#"require\s*\(\s*['"]([^'"]+)['"]\s*\)"#).unwrap();
let mut specs = Vec::new();
for cap in re.captures_iter(source) {
if let Some(m) = cap.get(1) {
let s = m.as_str().to_string();
if !specs.contains(&s) {
specs.push(s);
}
}
}
specs
}

/// Extract `exports.X = ...` named-export patterns. Skips `__esModule`
/// (the interop marker injected by Babel/TypeScript that consumers use to
/// detect "this is a CJS module pretending to be ESM" — we don't want to
/// re-export a boolean as if it were a named binding).
fn extract_exports_from_source(source: &str) -> Vec<String> {
let re = regex::Regex::new(r"exports\.([A-Za-z_$][A-Za-z0-9_$]*)\s*=").unwrap();
let mut names = Vec::new();
for cap in re.captures_iter(source) {
if let Some(m) = cap.get(1) {
let name = m.as_str();
if name == "__esModule" {
continue;
}
let owned = name.to_string();
if !names.contains(&owned) {
names.push(owned);
}
}
}
names
}

#[cfg(test)]
mod tests {
use super::*;
use std::path::PathBuf;

#[test]
fn detects_module_exports_assignment() {
assert!(is_commonjs("module.exports = function() {};"));
}

#[test]
fn detects_exports_dot_pattern() {
assert!(is_commonjs("exports.foo = 1;"));
}

#[test]
fn detects_require_without_import() {
assert!(is_commonjs("var x = require('foo');"));
}

#[test]
fn does_not_detect_pure_esm() {
assert!(!is_commonjs("import x from 'foo'; export const y = 1;"));
}

#[test]
fn extracts_named_exports() {
let src = "exports.foo = 1; exports.bar = function() {}; exports.__esModule = true;";
let names = extract_exports_from_source(src);
assert_eq!(names, vec!["foo".to_string(), "bar".to_string()]);
}

#[test]
fn extracts_require_specifiers_dedup() {
let src = r#"var a = require('./a'); var b = require("./b"); var c = require('./a');"#;
let specs = extract_require_specifiers(src);
assert_eq!(specs, vec!["./a".to_string(), "./b".to_string()]);
}

#[test]
fn wraps_simple_cjs_as_esm() {
let src = "exports.foo = 42;";
let wrapped = wrap_commonjs(src, &PathBuf::from("/tmp/test.js"));
assert!(wrapped.contains("export default _cjs;"));
assert!(wrapped.contains("export const foo = _cjs.foo;"));
assert!(wrapped.contains("const _cjs = (function()"));
}

#[test]
fn wrap_hoists_require_as_import() {
let src = "var dep = require('./dep'); module.exports = dep.value;";
let wrapped = wrap_commonjs(src, &PathBuf::from("/tmp/test.js"));
assert!(wrapped.contains("import _req_0 from './dep';"));
assert!(wrapped.contains("if (specifier === './dep') return _req_0;"));
}
}
15 changes: 14 additions & 1 deletion crates/perry/src/commands/compile/collect_modules.rs
Original file line number Diff line number Diff line change
Expand Up @@ -111,9 +111,22 @@ pub(super) fn collect_modules(
}

// It's a TypeScript file to compile natively
let source = fs::read_to_string(&canonical)
let raw_source = fs::read_to_string(&canonical)
.map_err(|e| anyhow!("Failed to read {}: {}", canonical.display(), e))?;

// Issue #348: when a `compilePackages` target ships CommonJS (e.g. React
// 18's `module.exports = require('./cjs/react.production.min.js')`),
// rewrite the source as ESM before SWC parses it. Only fires for files
// inside a `compilePackages` target — user TypeScript and ESM-shaped
// packages skip the wrap. See `cjs_wrap.rs` for the wrap shape.
let source = if is_in_compiled_pkg
&& super::cjs_wrap::is_commonjs(&raw_source)
{
super::cjs_wrap::wrap_commonjs(&raw_source, &canonical)
} else {
raw_source
};

// Record the source hash for V2.2's per-module object cache. Computed here
// (instead of in the rayon codegen loop) so the cache key doesn't force a
// second read of the source bytes — we already have them.
Expand Down
Loading