feat: enhanced SEA mode with walker integration and VFS#229
feat: enhanced SEA mode with walker integration and VFS#229robertsLando merged 49 commits intomainfrom
Conversation
Evolves the --sea flag from a simple single-file wrapper into a full packaging pipeline that reuses the walker for dependency discovery, maps files as SEA assets, and provides a runtime VFS bootstrap using @platformatic/vfs for transparent fs/require/import support. - Add seaMode to walker: skips bytecode compilation and ESM-to-CJS transform, but still discovers all dependencies via stepDetect - Add sea-assets.ts: generates SEA asset map and manifest JSON from walker output (directories, stats, symlinks, native addons) - Add sea-bootstrap.js: runtime bootstrap with lazy SEAProvider, native addon extraction, and process.pkg compatibility - Add seaEnhanced() to sea.ts: walker → refiner → asset gen → blob → bake pipeline with Node 25.5+ --build-sea detection - Route --sea to enhanced mode when input has package.json and target Node >= 22; falls back to simple mode otherwise - Add 4 test suites: multi-file project, asset access, ESM (skipped until node:vfs lands), and VFS fs operations Closes #204 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…mpat @platformatic/vfs requires Node >= 22 but CI runs on Node 20. Since the package is only needed at build time (esbuild bundles it into the sea-bootstrap) and the bundle step is skipped on Node < 22, making it optional prevents yarn install failures on older Node versions. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Node.js 20 reached EOL in April 2026. This simplifies the project by removing the Node 20 CI matrix, test scripts, and engine check. It also allows @platformatic/vfs to be a regular dependency (requires Node 22+) and removes the conditional build script for sea-bootstrap. - Update engines to >=22.0.0 in package.json - Remove node20 from CI matrix (ci.yml, test.yml) - Update update-dep.yml to use Node 22 - Move @platformatic/vfs from optionalDependencies to dependencies - Simplify build:sea-bootstrap script (no Node version check) - Update README examples and SEA requirements - Update @types/node to ^22.0.0 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Simple SEA mode was introduced in Node 20, not Node 22. The guard should reflect when the Node.js feature was added, not the project minimum. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The shared assertSeaNodeVersion() is used by both simple and enhanced SEA modes. Simple SEA was introduced in Node 20 — the check should reflect that. Enhanced mode's Node 22 requirement is enforced separately by the routing logic in index.ts. Also fixes README --sea description to explain both modes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
This PR upgrades pkg’s experimental --sea flag from a single-file wrapper into an “enhanced SEA” pipeline that reuses the existing walker/refiner and embeds a VFS-backed runtime bootstrap, while also raising the project’s minimum supported Node.js version to 22.
Changes:
- Add enhanced SEA orchestration (
seaEnhanced) that walks dependencies, generates SEA assets + a manifest, builds a SEA blob, and bakes it into target Node executables. - Introduce SEA VFS runtime bootstrap (
prelude/sea-bootstrap.js) bundled via esbuild, and add@platformatic/vfsas a dependency. - Drop Node 20 from engines/CI/scripts and add new SEA-focused tests.
Reviewed changes
Copilot reviewed 32 out of 34 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| yarn.lock | Adds lock entry for @platformatic/vfs. |
| package.json | Adds @platformatic/vfs, bumps @types/node to 22, builds SEA bootstrap bundle, removes Node 20 test script, bumps engines to Node >= 22. |
| lib/types.ts | Centralizes Marker/WalkerParams types and adds SeaEnhancedOptions. |
| lib/walker.ts | Adds seaMode behavior to skip bytecode/ESM transforms and keep dependency detection. |
| lib/sea-assets.ts | New: converts walker/refiner output into SEA asset map + manifest JSON. |
| lib/sea.ts | Adds seaEnhanced(), shared tempdir helper, execFile usage, and shared macOS signing helper. |
| lib/index.ts | Routes --sea to enhanced mode when appropriate; factors buildMarker(); reuses macOS signing helper. |
| prelude/sea-bootstrap.js | New: SEA runtime bootstrap that mounts a VFS from SEA assets and supports native addon extraction. |
| eslint.config.js | Ignores generated prelude/sea-bootstrap.bundle.js. |
| .gitignore | Ignores generated prelude/sea-bootstrap.bundle.js. |
| README.md | Updates SEA docs and examples to reflect enhanced mode and Node >= 22 focus. |
| .github/workflows/ci.yml | Removes Node 20 from matrix; updates lint run condition to Node 22. |
| .github/workflows/test.yml | Removes Node 20 from test matrix. |
| .github/workflows/update-dep.yml | Updates workflow to use Node 22. |
| test/utils.js | Adds assertSeaOutput() helper for SEA tests. |
| test/test-85-sea-enhanced/* | New enhanced SEA “multi-file project” test fixture. |
| test/test-86-sea-assets/* | New enhanced SEA “fs.readFileSync assets” test fixture. |
| test/test-87-sea-esm/* | New enhanced SEA ESM test fixture (currently skipped). |
| test/test-89-sea-fs-ops/* | New enhanced SEA filesystem ops test fixture. |
| plans/SEA_VFS_IMPLEMENTATION_PLAN.md | Adds design/analysis and implementation plan document. |
Move dlopen patching, child_process patching, and process.pkg setup into a shared module used by both the traditional and SEA bootstraps. This eliminates duplication and ensures both modes handle native addons, subprocess spawning, and process.pkg identically. - Replace copyFolderRecursiveSync with fs.cpSync (Node >= 22) - Add REQUIRE_SHARED parameter to packer wrapper for traditional mode - SEA bootstrap consumes shared module via esbuild bundling - Remove dead code (ARGV0, homedir import, copyFolderRecursiveSync) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Handle non-numeric nodeRange (e.g. "latest") in enhanced SEA gating - Bump assertSeaNodeVersion to require Node >= 22 matching engines - Guard stepStrip in walker to only strip JS/ESM files in SEA mode, preventing binary corruption of .node addons - Replace blanket eslint-disable in sea-bootstrap.js with targeted no-unused-vars override in eslint config - Wire symlinks through to SEA manifest and bootstrap provider - Document --build-sea Node 25 gating assumption Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Instead of silently falling back to simple SEA mode when the input is a package.json/config but targets are below Node 22, throw a clear error. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Detailed comparison of traditional pkg mode vs enhanced SEA mode covering build pipelines, binary formats, runtime bootstraps, VFS provider architecture, shared code, performance, and code protection. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The SEA manifest uses POSIX keys but VFS passes platform-native paths on Windows. Normalize all paths to POSIX in the SEAProvider before manifest lookups, SEA asset lookups, and MemoryProvider storage. Remove the now- redundant symlink normalization block. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Explains that traditional mode with --no-bytecode produces a similar code protection profile to enhanced SEA (plaintext source), while still retaining compression and custom VFS format advantages. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Replace fs.cpSync with recursive copy using patched fs primitives so native addon extraction works through VFS in SEA mode - Use toPosixKey instead of snapshotify for symlink manifest keys to match the key format used by directories/stats/assets - Make assertSeaOutput log explicit skip on unsupported platforms instead of silently passing Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move VFS tree dump and fs call tracing from prelude/diagnostic.js into bootstrap-shared.js as installDiagnostic(). Both bootstraps now share the same diagnostic implementation. Security: diagnostics are only available when built with --debug / -d. In SEA mode this is controlled via manifest.debug flag set at build time; without it, installDiagnostic is never called. - Delete prelude/diagnostic.js (replaced by shared installDiagnostic) - Packer injects small DICT-dump snippet + shared call when --debug - SEA bootstrap gates on manifest.debug before calling installDiagnostic - Fix assertSeaNodeVersion: restore check to Node >= 20 (simple SEA) - Fix withSeaTmpDir: restore tmpDir cleanup (was commented out) - Update architecture docs with diagnostic usage Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Bump assertSeaNodeVersion minimum from Node 20 to 22 (consistent with engines) - Extract generateSeaBlob() helper with --build-sea fallback for Node 25.x - Accept separate defaultEntrypoint in setupProcessPkg for process.pkg compat - Update ARCHITECTURE.md Simple SEA min Node from 20 to 22 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Worker thread support: - Extract VFS setup into shared sea-vfs-setup.js (used by both main and worker threads, no duplication) - Bundle sea-worker-entry.js separately via esbuild — workers get the same @platformatic/vfs module hooks as the main thread - Monkey-patch Worker constructor to intercept /snapshot/ paths and inject the bundled VFS bootstrap via eval mode - Add test-90-sea-worker-threads test Walker fix: - Skip stepDetect (Babel parser) for non-JS files in SEA mode — only run on .js/.cjs/.mjs files, matching the existing stepStrip guard. Eliminates thousands of spurious warnings on .json, .d.ts, .md, .node files in real-world projects. Build: - Extract build:sea-bootstrap into scripts/build-sea-bootstrap.js for two-step bundling (worker entry → string, then main bootstrap) TODO: Remove node_modules/@platformatic/vfs patches once platformatic/vfs#9 is merged and released. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The VFS mount point must always be POSIX '/snapshot' because @platformatic/vfs internally uses '/' as the path separator. The Windows prototype patches convert C:\snapshot\... and V:\snapshot\... to /snapshot/... before they reach the VFS. This was broken during the sea-vfs-setup.js extraction where SNAPSHOT_PREFIX was incorrectly set to 'C:\snapshot' on Windows. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
So, comparing |
|
Thanks for the feedback! There are some important differences between this PR and fossilize that are worth highlighting: 1. Virtual Filesystem (VFS) — transparent Fossilize bundles everything into a single file with esbuild and requires you to use Our enhanced SEA mode uses 2. Native addon support Fossilize cannot handle 3. Worker thread support We monkey-patch the 4. Dependency walker integration Fossilize relies entirely on esbuild's bundling, which means apps must be fully bundleable. Our approach reuses pkg's battle-tested dependency walker ( 5. Stock Node.js binaries — no more patched builds This is actually a shared advantage with fossilize, and one of the main motivations for this PR. Traditional pkg requires patched Node.js binaries built and released via the yao-pkg/pkg-fetch project. These patches are hard to create, time-consuming to build for every Node.js release, and can lag behind upstream — users often have to wait for new builds before they can use a new Node.js version. With enhanced SEA mode, users can use official Node.js binaries directly from nodejs.org — no patches, no waiting for pkg-fetch releases, no maintenance burden. 6. On the performance comparison You're right about the numbers — SEA with bundle vs Standard PKG with bundle is 261ms vs 381ms startup and 111 MB vs 145 MB binary. That's a meaningful but not dramatic difference, and in return you gain stock Node.js binaries (no pkg-fetch dependency), ESM support, and a much simpler maintenance story. The tradeoff for losing bytecode/source protection is real, but for many use cases (CLI tools, internal services, IoT deployments) it's not a concern — and the elimination of the patched-binary dependency is a significant operational win. |
…ck, CRLF normalization - Restore unconditional native addon copy in patchDlopen with per-file SHA-256 checksums, matching original vercel/pkg behavior (PRs vercel#1492, vercel#1611). The existsSync guard on destFolder was a regression — OS cleanup can delete files inside the cache while leaving the directory intact. - Narrow generateSeaBlob fallback to only catch unsupported --build-sea flag errors (exit code 9 / "bad option"), rethrow real failures. - Normalize CRLF in assertSeaOutput and test-00-sea for Windows compat. Uncomment previously disabled Windows SEA assertion. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- bootstrap-shared.js: standalone addon branch now uses writeFileSync with the already-read moduleContent (reuses hash too), so extraction works through VFS in SEA mode instead of calling copyFileSync. - bootstrap-shared.js: extract cpRecursive helper out of patchDlopen for readability. - docs/ARCHITECTURE.md: drop stale diagnostic.js reference and clarify that installDiagnostic ships in the bundle but is only invoked when the binary is built with --debug. - .prettierignore: ignore generated prelude/sea-bootstrap.bundle.js so post-build lint stays clean.
Split the SEA bootstrap into a shared core plus two thin wrappers so ESM
entrypoints work on any supported Node.js target:
- `sea-bootstrap-core.js` — VFS mount, shared patches, worker interception,
diagnostics. Exports `{ manifest, entrypoint }` with no entry execution.
- `sea-bootstrap.js` (CJS) — `Module.runMain()` for CJS entries; dynamic
`import(pathToFileURL(entry))` fallback for ESM entries on Node < 25.7.
- `sea-bootstrap-esm.js` (ESM) — static `import core` + top-level
`await import(entry)` for Node >= 25.7 with `mainFormat: "module"`.
`lib/sea.ts` picks the right bundle and sea-config based on entry format
and the smallest target Node major. When the ESM-on-old-Node fallback is
used, a build-time `log.warn` explains the limitations (one microtask
delay, no sync require of ESM-with-TLA deps) and points at Node 25.7+ as
the fix. No runtime warning is emitted.
Also refresh the symlink + read-once-Buffer improvements to `cpRecursive`
in bootstrap-shared.js (prevents infinite recursion and unwanted content
escape on addon packages with symlinks; halves I/O on the hash-mismatch
path).
Add tests:
- `test-91-sea-esm-entry` — ESM entry with relative `.mjs` imports
- `test-92-sea-tla` — ESM entry using top-level await
Update `docs/ARCHITECTURE.md` with the dual-wrapper layout, the
bootstrap-selection table, the TLA section, and the 3-step esbuild flow.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Clarify the enhanced SEA bullet list: ESM entries work on any supported Node target (native on Node >= 25.7 via mainFormat:"module", dynamic import() fallback on Node 22-25.6 with a build-time warning). Also note that DEBUG_PKG diagnostics are available but only when built with --debug. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ator selection, cross-plat - Always use CJS bootstrap: Node 25.5+ embedder dynamic-import callback only resolves builtins (nodejs/node#62726), so native ESM SEA main cannot import the user entrypoint. ESM entries are dispatched through a vm.Script compiled with USE_MAIN_CONTEXT_DEFAULT_LOADER, which routes dynamic import() to the default ESM loader and supports top-level await. Drop the now-unused sea-bootstrap-esm.js and its bundle step. - Pick SEA blob generator binary by host/target major: use process.execPath when host major matches minTargetMajor (works for cross-platform builds), else the downloaded target binary (works cross-major when host can exec it). Restores cross-platform same-major SEA builds. - Split host/target node version checks: assertHostSeaNodeVersion covers the pkg host, resolveMinTargetMajor validates targets separately. - Reject useSnapshot:true in enhanced SEA mode — incompatible with the VFS bootstrap (snapshot build has no __pkg_archive__ asset). - Trust in-memory body in sea-assets when present: walker invariant guarantees body equals shippable bytes in seaMode (ESM→CJS and type:module rewrites are gated on !seaMode). Removes bodyModified flag and a redundant disk re-read, and makes the build race-safe against mid-build source edits. - stepStrip leaves record.body untouched on no-op so Buffer identity is preserved for the sea-assets reuse path. - perf.finalize() moved into each bootstrap dispatcher's finally block so module-loading timings reflect real entrypoint completion (including async / top-level-await apps) and still print on thrown entrypoints. Error paths set process.exitCode instead of process.exit() so the finally runs. - _resolveSymlink now throws ELOOP instead of returning the last hop, matching fs semantics for broken symlink cycles. - insideSnapshot refactored to a prefix table; behavior unchanged. - Share public / no-dict / publicPackages WalkerParams between the SEA and traditional pipelines in lib/index.ts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…-write loop - bootstrap-shared.js: cpRecursive() falls back to recursive copy when symlinkSync throws EPERM/EACCES on Windows (no admin / dev mode), so native addon extraction no longer aborts silently. - sea-vfs-setup.js: validate manifest offsets before Buffer.subarray() — corrupt or truncated manifests now throw instead of returning silently truncated bytes. Also hoist MAX_SYMLINK_DEPTH to a named const and stop reaching the module-scope provider from perf.finalize(). - sea-assets.ts: writeAll() helper loops on FileHandle.write bytesWritten for both buffer and stream paths, keeping manifest offsets byte-exact even under filesystem backpressure. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ESM/TLA now runs through one CJS wrapper using vm.Script + USE_MAIN_CONTEXT_DEFAULT_LOADER on every Node >= 22 target. Native ESM SEA main (mainFormat:"module") is not used — nodejs/node#62726 blocks embedder dynamic-import of the user entry. Also documents useSnapshot incompatibility, --experimental-sea-config-only blob generation, and the host/target blob generator picker. File reference + ecosystem tables refreshed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bake() previously reloaded the prep blob from disk per target inside Promise.all, causing N redundant reads and N peak buffer copies on multi-target builds. Read once before the loop and pass the shared Buffer into bake(). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…k, test cleanup
- lib/sea.ts: add assertSingleTargetMajor() and call from both sea() and
seaEnhanced(). SEA prep blobs are Node-major specific, so mixing majors
in one run (e.g. -t node22-linux-x64,node24-linux-x64) silently produced
broken executables. Reject up front instead.
- prelude/sea-bootstrap.js: restore the original process.emitWarning as
soon as the SEA loader warning is suppressed, so user code does not
observe a permanently wrapped emitWarning.
- test/utils.js: filesAfter() gains { tolerateWindowsEbusy } option that
only swallows EBUSY on win32 during cleanup.
- test/test-85..92: drop copy-pasted try/catch noop around filesAfter and
use the new option. test-91/92 also switch from inlined spawn+CRLF
logic to the shared assertSeaOutput helper.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
package-lock.json was created by accident in #229 alongside the yarn lockfile. pkg's canonical package manager is yarn (see yarn.lock); the accidental npm lockfile drifts silently and confuses contributors. - Remove package-lock.json and gitignore it at the repo root so it cannot reappear. - Rewrite the dev-command references in CLAUDE.md, .claude/rules/*, .github/copilot-instructions.md and docs-site/development.md from "npm run <x>" to "yarn <x>", and spell out the split: pkg uses yarn, docs-site is the only place npm is used. User-facing install instructions (README, guide/getting-started, guide/ migration, guide/api) are unchanged — end users still "npm install -g @yao-pkg/pkg" and the CI recipe example still shows npm, since that reflects how consumers package their own projects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: host documentation on GitHub Pages with VitePress Move README content into a proper documentation site built with VitePress and deployed to https://yao-pkg.github.io/pkg/ via GitHub Actions. - Landing page with pkg identity (amber/pink palette from logo) - Guide split into 21 focused pages (getting started, targets, config, bytecode, compression, SEA mode, snapshot FS, native addons, ESM, API, troubleshooting, advanced) - New SEA vs Standard page highlighting the stock-binary benefit and linking the pkg-fetch elimination roadmap (#231) - DEVELOPMENT.md and docs/ARCHITECTURE.md canonical sources moved into docs-site/, roots replaced with stub pointers to avoid drift - README trimmed to install + quick start + links to hosted docs - .github/workflows/docs.yml builds and deploys on push to main Manual follow-up: enable GitHub Pages with Source = GitHub Actions in repo settings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update GitHub Actions to use latest versions of checkout, setup-node, and deploy-pages * docs: address self-review feedback on docs site PR - gate docs workflow on PRs (build-only) to catch VitePress breakage before merge - switch workflow to npm ci with cache-dependency-path for reproducible installs - read site version from package.json so nav dropdown no longer rots each release - reconcile SEA vs Standard messaging on the guide landing page - fix typos and stale DEVELOPMENT.md references in copilot-instructions - mark root DEVELOPMENT.md and docs/ARCHITECTURE.md stubs as non-canonical - replace non-standard -webkit-linear-gradient with standard linear-gradient Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: major docs site overhaul — tutorials, recipes, mermaid, migration New pages: - guide/migration.md — drop-in migration from archived vercel/pkg fork - guide/recipes.md — copy-paste cookbook for common tasks - changelog.md — CHANGELOG.md mirrored into the docs site via include Rewritten pages: - guide/getting-started.md — first-binary tutorial with code groups + full CLI table; no more raw --help dump - guide/sea-mode.md — leads with a hello-world walkthrough - guide/configuration.md — full schema table + production example - guide/targets.md — code groups + tip callout + alpine/musl section - guide/environment.md — inlined pkg-fetch vars + PKG_EXECPATH clarification - guide/troubleshooting.md — NODE_OPTIONS dead end replaced with a full fix, check-this-first preface added - guide/bytecode|compression|options|build|output|snapshot-fs|native-addons| esm|api|packaged-app|custom-node|detecting-assets|advanced-debug-vfs| advanced-windows-metadata — expanded + frontmatter + next-step links VitePress config: - withMermaid for mermaid flowchart support - sitemap.hostname for SEO - outline level raised to [2, 4] - Collapsed singleton sidebar groups into a Cookbook section - Added Recipes to the top nav - Version dropdown links to migration + changelog Landing page: - vercel/pkg fork banner - Social badges (npm version, downloads, stars, CI, license) - Code groups for CLI vs package.json vs Node.js API - Standard vs SEA side-by-side Architecture: - Top-level mermaid overview comparing both build paths - Mermaid flowcharts for traditional + SEA build pipelines - Mermaid flow for VFS provider path resolution - ASCII diagrams kept for readers who prefer them Development page: - Full docs-site local dev workflow - Structure, mermaid how-to, new-page checklist - Canonical-source note for the root stubs Polish: - Responsive hero image on small viewports - Fork-banner CSS - .nvmrc pinning docs-site to Node 22 - Frontmatter (title + description) on every guide page for SEO - Twitter card meta Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add hero terminal GIF and fix badge stacking on landing page - Record a real 15s pkg session (mkdir → echo → pkg index.js → ls → run) with VHS as docs-site/public/hero.gif (98KB). Tape-based so it is reproducible in CI if we want to regenerate it later. - Embed the GIF in the landing page as a figure under the fork banner. - Replace the inline HTML badge anchors with Markdown image links on a single line so VitePress renders them inside a single <p>. The old setup put each anchor on its own line which the markdown processor wrapped in separate paragraphs, stacking the badges vertically. - Add .landing-badges / .landing-demo / .landing-body CSS: flex layout for the badge row, shadow + border-radius on the demo GIF, consistent max-width + padding on the content block. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: re-record hero GIF with a working sandboxed pkg wrapper The previous recording ran VHS with a PATH that did not include the nvm node bin dir, so the /tmp/bin/pkg wrapper's `env ... node ...` invocation failed with "env: node: No such file or directory" and the session showed three cascading errors instead of a successful build. Fix: hardcode the absolute node path inside the wrapper so it works under any restricted PATH. Re-recorded — now shows the real pkg build producing three binaries and running the linux one. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: update mermaid diagram styles for better readability and adjust SEA mode documentation * docs: enhance dark mode visibility for mermaid diagrams and update native addon notes * docs: trim docs-site verbosity and split architecture doc Audience split: keep the detailed pipelines, binary layouts, VFS provider, worker bootstrap and patch tables as docs/ARCHITECTURE.md (contributor / agent reference), and replace docs-site/architecture.md with a lean ~110 line user overview that links back to it. Also dedup SEA-vs-standard and walker explanations across index pages, collapse the pkg-fetch roadmap down to a pointer to #231, share the reference sidebar config, and add a jump TOC to the recipes page. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: clarify cross-compile support and Node 22 regression Standard cross-compile is confirmed broken on Node 22 targets (builds succeed but linux-arm64 crashes with UNEXPECTED-20 and win-x64 exits silently, matching #87 and #181). Node 20 and Node 24 are fine in both Standard and Enhanced SEA modes. Rewrite the targets.md Cross-compilation section as a node-version-keyed support table with the regression called out, and reference it from the landing page, getting-started, and the sea-vs-standard feature matrix. Verified on a Linux x86_64 host against linux-x64, linux-arm64 (via docker + QEMU) and win-x64 (via docker + Wine) for each of Node 20, 22, and 24 using the pkg-xcompile-test harness. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(skills): add pkg-xcompile-test skill Project-local skill (under .claude/skills/) that runs a full (mode × target) cross-compile matrix for pkg and reports which combinations work. Default modes are std, std-public (--public-packages "*" --public) and sea; targets cover linux-x64, linux-arm64 (via docker + QEMU), win-x64 (via docker + Wine), and macos-* (build only — runtime verification needs a real Mac). The run-matrix.sh script auto-switches nvm to match the target node major, resolves PKG_BIN from its own location (<repo>/lib-es5/bin.js), and writes binaries to $PKG_XCOMPILE_WORKDIR (default /tmp/pkg-xcompile) so the working tree stays clean. SKILL.md records the known results for node 20/22/24 as of 2026-04-15 and documents the wine-in-docker stdout gotcha. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: add cross-compile matrix workflow Adds a GitHub Actions workflow that builds hello.js from every supported host runner (linux-x64, linux-arm64, macos-x64, macos-arm64, win-x64) for every target triple, in Standard / Standard+public / SEA modes on Node 22. Each produced binary is then executed on its target runner, and a per-mode matrix of the host-to-target results is posted as a sticky PR comment. Structured as three stages: 1. build — matrix of (host × mode × node); each job compiles all targets from that host and uploads the binaries. 2. run — one job per target runner; downloads all artifacts, filters by target, runs each binary, writes result JSON. 3. collate — renders the markdown matrix and upserts it as a PR comment using a <!-- xcompile-matrix --> marker so re-runs update instead of spamming. Build failures are emitted as ".failed" sentinel files so the run stage can distinguish BUILD_FAIL from RUN_FAIL. Uses yarn (the repo's package manager) and pins actions to the versions already used elsewhere in the repo (checkout@v6, setup-node@v6, upload/download- artifact@v4, github-script@v7). Triggers on PRs touching lib/, prelude/, dictionary/, scripts/, package.json, yarn.lock, or the workflow itself, and on workflow_dispatch for manual runs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address deep-review findings on docs site branch - docs: fix broken cross-page anchors in bytecode, recipes, build and sea-mode guides; replace raw /pkg/ prefixed link in index.md with VitePress-managed markdown link. - ci(xcompile): switch cross-compile matrix to workflow_dispatch only (manual trigger is safer and avoids burning macOS/Windows minutes on every PR), wire the node-majors input via a setup job, add timeouts, upload per-cell build logs, short-circuit collate when no results, fix empty-array expansion under bash 3.2 / set -u on macOS runners, add spawnSync timeout, sanitise backticks in PR summary, and drop the dead PR-comment path. - ci(docs): split concurrency so PR builds no longer queue behind production deploys, scope pages/id-token permissions to the deploy job, add timeout-minutes, and add an offline lychee link-check step to catch dead anchors before they ship. - chore(skills): make run-matrix.sh fail fast (set -euo pipefail), write per-cell logs under $WORKDIR/logs so a failing cell is not overwritten, print log paths on stderr, add an unknown-target case, and document the debugging workflow in SKILL.md. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: drop root package-lock.json and standardize on yarn for pkg package-lock.json was created by accident in #229 alongside the yarn lockfile. pkg's canonical package manager is yarn (see yarn.lock); the accidental npm lockfile drifts silently and confuses contributors. - Remove package-lock.json and gitignore it at the repo root so it cannot reappear. - Rewrite the dev-command references in CLAUDE.md, .claude/rules/*, .github/copilot-instructions.md and docs-site/development.md from "npm run <x>" to "yarn <x>", and spell out the split: pkg uses yarn, docs-site is the only place npm is used. User-facing install instructions (README, guide/getting-started, guide/ migration, guide/api) are unchanged — end users still "npm install -g @yao-pkg/pkg" and the CI recipe example still shows npm, since that reflects how consumers package their own projects. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * ci: fix xcompile matrix false failures on macos and build-fail reporting First run of the matrix on #234 surfaced three harness bugs (all cells that failed under Standard mode were already matching the known Node 22 regression — those were expected). The CI-side bugs were: 1. macos-arm64 std jobs died on the first target with `EXTRA[@]: unbound variable`. Bash 3.2 ships on GitHub's macOS runners, and `set -u` + `"${EMPTY_ARRAY[@]}"` is a nounset error there. std-public/sea worked only because EXTRA was non-empty. Expand it with `${EXTRA[@]+"${EXTRA[@]}"}` instead, which degrades gracefully on old bash. 2. All macos-x64 build and run jobs were cancelled by the runner pool — GitHub has retired the Intel macOS runner (macos-13). Drop macos-x64 as a build host entirely, and route the macos-x64 *target* runs to macos-14, which pre-installs Rosetta 2 and executes x86_64 Mach-O binaries transparently. 3. The run-stage reader iterated binaries and skipped .failed sidecars, so a pkg failure that left no output at all (vs. a 0-byte stub) dropped the combo out of the matrix as `:grey_question:` instead of showing `:x: build`. Also capture pkg's real exit code via `|| status=$?` so `set -e` doesn't eat it, always leave an empty binary on failure so the reader enumerates every combo, and pull the last 5 lines of the build log into the BUILD_FAIL detail so the collated summary surfaces the actual error without downloading artifacts. Verified by parsing the workflow YAML: 4 build hosts (linux-x64, linux-arm64, macos-arm64, win-x64), 5 run targets including macos-x64@macos-14. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(ci): drop lychee link-checker from docs workflow The step was a belt-and-braces defence against dead anchors, but lychee fights VitePress's /pkg/ base-URL rewrite and extensionless URLs — it reports ~2000 false positives for resources that are served fine on the live site. Not worth the complexity; VitePress's own build-time dead-link check already catches markdown link typos, and the five broken anchors that motivated the step have been fixed and are verified against the rendered HTML. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: proofread development.md Address Copilot review feedback on grammar in docs-site/development.md: fix "create release", "have build", "runned", "It install", and "Foreach" phrases flagged by the reviewer. * ci: skip heavy jobs on docs-only PRs Add a `changes` job that uses dorny/paths-filter to detect whether anything outside docs-site/ was touched. The build matrix and every test.yml call still run so their required status check contexts are reported, but their steps are gated on the flag — docs-only PRs get green checks in seconds instead of burning the full OS/node matrix. --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
Implements #204 — evolves the
--seaflag from a simple single-file wrapper into a full packaging pipeline with walker integration and VFS support.Enhanced SEA mode
Walker SEA mode — reuses the existing dependency walker with a
seaModeflag that skips bytecode compilation and ESM→CJS transform while still discovering all dependencies viastepDetect. InseaMode,stepStripis a true no-op (preservesrecord.bodyBuffer identity) and the walker guarantees that the in-memory body equals the shippable bytes — the asset generator trusts this invariant and skips a disk re-read.SEA asset generator (
lib/sea-assets.ts) — concatenates all discovered files into a single__pkg_archive__blob and emits a__pkg_manifest__.jsonwith directory listings, stats, symlinks, native-addon list,entryIsESMflag, and byte offsets into the archive for zero-copy extraction. Race-safe against mid-build source edits (uses the walker's in-memory body rather than re-reading from disk).Runtime bootstrap — self-contained bootstrap bundled with
@roberts_lando/vfsthat provides transparentfs/fs/promises/require/importsupport via a lazySEAProvider, native addon extraction, andprocess.pkgcompatibility. Split into:prelude/sea-bootstrap-core.js— shared setup (VFS mount, shared patches, worker interception, diagnostics,perf.start('module loading')). Exports{ manifest, entrypoint, perf }without running the entry.prelude/sea-bootstrap.js— single CJS wrapper used for every entry format. Dispatches based onmanifest.entryIsESM:Module.runMain()(goes through the real CJS loader;require(esm)on Node 22.12+ transparently handles trivial ESM).import("file:///path/to/entry")viavm.ScriptwithimportModuleDynamically: vm.constants.USE_MAIN_CONTEXT_DEFAULT_LOADER. Dynamicimport()inside that script is routed to the default ESM loader, so file URLs resolve and top-level await in the main module works on every Node ≥ 22 target. TheExperimentalWarningforUSE_MAIN_CONTEXT_DEFAULT_LOADERis filtered out via aprocess.emitWarningshim so it doesn't leak into packaged-app stderr.prelude/sea-vfs-setup.js—SEAProvider, archive loading, VFS mount, Windows path normalization. Shared by main thread and workers.ESM entrypoints with top-level await (uniform path) — native ESM SEA main (
mainFormat: "module", Node 25.7+) is not used. On Node 25.5+, the embedderimportModuleDynamicallyForEmbeddercallback only resolves builtin modules (nodejs/node#62726), so a native ESM main cannot dynamically import the user entrypoint at all. Instead, the CJS wrapper handles ESM entries through thevm.Script+USE_MAIN_CONTEXT_DEFAULT_LOADERpath, which supports TLA on every Node ≥ 22 target — no version split, no build-time warning, no fallback tier.sea-bootstrap.bundle.jsModule.runMain()sea-bootstrap.bundle.jsvm.ScriptwithUSE_MAIN_CONTEXT_DEFAULT_LOADER→ dynamicimport()perf.finalize()is moved into each dispatcher'sfinallyblock so module-loading timings reflect real entrypoint completion (including async / TLA apps) and still print when the entry throws. Errors on the ESM path setprocess.exitCode = 1instead of callingprocess.exit()so thefinallyruns.Worker thread support — monkey-patches
Workerso/snapshot/...workers get the sameSEAProvider+@roberts_lando/vfsmount as the main thread (same archive blob, sameBuffer.subarray()extraction, same 164+ fs intercepts). Worker VFS setup is reused fromsea-vfs-setup.js, no code duplication.DEBUG_PKG diagnostics — the existing
DEBUG_PKG/SIZE_LIMIT_PKG/FOLDER_LIMIT_PKGdiagnostics now work in SEA mode too. Diagnostic code is only invoked when built with--debug, so release SEA builds can't be coerced into dumping the VFS tree via environment variables.Enhanced orchestrator (
seaEnhanced()inlib/sea.ts) — full pipeline: walker → refiner → asset generator → blob generation → bake → optional macOS signing.node --experimental-sea-configto generate the prep blob.--build-sea(Node 25.5+) is intentionally not used: it produces a finished executable and bypasses the prep-blob + postject flow that pkg needs for multi-target injection.host major === minTargetMajor, pkg usesprocess.execPath(always executable regardless of target platform/arch — this is the only path that works for cross-platform same-major SEA builds, e.g. Linux x64 host producing a Windows x64 SEA). Otherwise it falls back to the downloaded target-platform binary (works cross-major when the host can exec it, e.g. same platform/arch or via QEMU/Rosetta).assertHostSeaNodeVersion()validates the pkg host,resolveMinTargetMajor()validates targets separately.useSnapshot: trueis rejected — incompatible with the VFS bootstrap (SEA snapshot mode runs the main script at build time inside a V8 startup snapshot context and expects av8.startupSnapshot.setDeserializeMainFunction()entry; the pkg bootstrap doesn't do that andsea.getRawAsset('__pkg_archive__')does not exist at build time).useCodeCacheis still forwarded — it only caches V8 bytecode for the bootstrap script and doesn't touch the runtime VFS path.CLI routing —
--seaautomatically uses enhanced mode when the input haspackage.jsonand all targets are Node >= 22; falls back to the existing simple mode otherwise. Public / no-dict / publicPackagesWalkerParamsare now shared between the SEA and traditional pipelines inlib/index.ts.Backward compatible — existing simple
--seamode (single pre-bundled file) works unchanged.Drop Node.js 20 support
>=22.0.0inpackage.json.ci.yml,test.yml) andtest:20script.update-dep.ymlworkflow to use Node 22.@roberts_lando/vfs(requires Node 22+) added as a regular dependency.@types/nodebumped to^22.0.0.Bug fixes
stepDetect(Babel parser) for non-JS files in SEA mode. Previously produced thousands of spurious warnings on.json,.d.ts,.md,.nodefiles.require.resolve()interception, trailing-slash specifiers, andmain-pointing-to-directory. See platformatic/vfs#9 for upstream VFS fixes.cpRecursiveinbootstrap-shared.js— now useslstatSyncand recreates symlinks at the destination instead of dereferencing them. Fixes infinite recursion and unwanted content escape when addon packages contain symlinks (npm dedup, nestednode_modules). Also reads each file once as aBuffer, hashes the buffer directly, and reuses it forwriteFileSync— halves I/O on the hash-mismatch path and drops the unnecessary'binary'encoding conversion.SEAProvider._resolveSymlink— throwsELOOPinstead of returning the last hop on broken symlink cycles, matching real-fs semantics.insideSnapshot— refactored to a prefix table (behavior unchanged, faster check).Documentation
A full technical comparison of traditional mode vs enhanced SEA mode lives in
docs/ARCHITECTURE.md— covers build pipelines, binary formats, runtime bootstrap, VFS provider architecture, worker thread support, performance characteristics, code protection tradeoffs, and the Node.js ecosystem dependencies (including thenode:vfsmigration path). Updated for the single-bootstrap architecture.Node.js ecosystem
@roberts_lando/vfsas VFS polyfill (Node 22+), with built-in migration path tonode:vfswhen nodejs/node#61478 lands.node --experimental-sea-config.--build-seaandsea-config mainFormat: "module"are deliberately not used (see nodejs/node#62726).Upstream dependencies
@platformatic/vfsmodule hooks (built-in shadowing, resolution order,require.resolve, trailing slash, main-as-directory).Test plan
test-00-sea— simple SEA backward compat (no regression)test-85-sea-enhanced— multi-file CJS project with walker integrationtest-86-sea-assets— non-JS asset access viafs.readFileSynctest-89-sea-fs-ops— VFS ops (readdir,stat,existsSync)test-90-sea-worker-threads— workers spawned with/snapshot/...pathstest-91-sea-esm-entry— ESM entrypoint with relative.mjsimportstest-92-sea-tla— ESM entrypoint using top-level await (uniformvm.Scriptpath)🤖 Generated with Claude Code