fix: #835 #846 — auto-link providing crate when codegen emits its FFIs#868
Merged
Conversation
When TS code (often inside a `perry.compilePackages` package) emits a call to an FFI whose implementation lives in `perry-stdlib` or a `perry-ext-*` crate, but no matching `import "node:…"` shows up in `ctx.native_module_imports`, the link step picks "runtime-only" or omits the well-known crate flip, and the linker fails with: - `Undefined symbols: _js_readable_stream_*` / `_js_stream_unwrap_handle` when Effect's `Stream` lowers `new ReadableStream(...)` calls via paths that don't trip `uses_fetch` (#835). - `Undefined symbols: _js_node_http_create_server` when Express's compiled output emits `js_node_http_create_server` from a code path that didn't surface `node:http` in `ctx.native_module_imports` (#846). Codegen already knows the FFI name at call-emission time. Adds a process-wide registry mapping each affected FFI symbol to its providing crate (`Stdlib` or `WellKnown("http")` / `WellKnown("streams")` / …), hooks `LlBlock::call` / `call_void` to record every emission against the registry, and folds the drained set into `ctx.needs_stdlib` + `ctx.native_module_imports` right before `build_optimized_libs` runs. The existing well-known flip + minimal- stdlib feature computation pick up the augmented set unchanged. - `crates/perry-codegen/src/ext_registry.rs` — new module hosting the symbol→owner table and the mutex-guarded `USED_PROVIDERS` collector. Covers all stream / writer / reader FFIs and every `js_node_http*` / `js_node_https*` / `js_node_http2*` server symbol shipped by `perry-ext-http-server`. - `crates/perry-codegen/src/block.rs` — single-line hook in `LlBlock::call` and `LlBlock::call_void`. Zero behavioural change for symbols not in the registry. - `crates/perry/src/commands/compile.rs` — drains the collector once at the start of `run_with_parse_cache` (in case a prior `perry dev` rebuild left stale entries) and once before `build_optimized_libs`, routing `OwnerKind::Stdlib` → `ctx.needs_stdlib`, and `OwnerKind::WellKnown(key)` → `ctx.native_module_imports.insert(key)` + `ctx.needs_stdlib`. Validation: - New unit test `ext_registry::tests::registry_dispatch_routes_to_correct_owner` exercises the Stdlib + WellKnown lookup paths plus a non-registered FFI control. - Full workspace test sweep passes (`cargo test --release --workspace` with the cross-host UI exclusions from CLAUDE.md). - Spot-check gap tests (`test_gap_async_advanced`, `_class_advanced`, `_array_methods`, `_bigint`) still link runtime-only and match Node byte-for-byte — the registry is inert when no listed FFI is emitted. - Repros: `new ReadableStream({...})` and `createServer((req,res) => …)` link cleanly under `target/release/perry compile` with no explicit `node:*` import in the entry TS. Closes #835 Closes #846
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When TS code (often inside a
perry.compilePackagespackage) emits a call to an FFI whose implementation lives inperry-stdlibor aperry-ext-*crate, but no matchingimport "node:…"shows up inctx.native_module_imports, the link step picks "runtime-only" (or omits the well-known flip), and the linker fails with the symptoms reported on #835 / #846:needs_stdlibnot set when compiled-package code (Effect) references ReadableStream FFIs #835:Undefined symbols: _js_readable_stream_new/_js_readable_stream_controller_*/_js_stream_unwrap_handle— Effect'sStreamlowersnew ReadableStream(...)and stream-controller methods via paths that don't tripuses_fetch, soneeds_stdlibnever flips.js_node_http_create_servercodegen'd butperry-ext-http-servernot linked #846:Undefined symbols: _js_node_http_create_server— Express's compiled output emits the server-create FFI from a path that didn't surfacenode:httpin the import set, soperry-ext-httpnever joins the link line.Codegen already knows the FFI name at call-emission time. This PR records every such emission against a small registry (
OwnerKind::StdliborOwnerKind::WellKnown("http")/"streams"/ …) and folds the drained set intoctx.needs_stdlib+ctx.native_module_importsright beforebuild_optimized_libsruns — the existing well-known flip + minimal-stdlib feature computation pick up the augmented set unchanged.Design (per the design note in the issue)
crates/perry-codegen/src/ext_registry.rs:FFI_REGISTRYtable mapping each affected FFI symbol → providing crate.Mutex<HashSet<OwnerKind>>collector populated by codegen.take_used_providers()drain consumed by the CLI driver.crates/perry-codegen/src/block.rs: single-line hook inLlBlock::callandLlBlock::call_void. Inert for symbols not in the registry.crates/perry/src/commands/compile.rs: drains at the start ofrun_with_parse_cache(so a staleperry devset can't bleed in) and once beforebuild_optimized_libs, routing entries through the existing flip mechanisms.Registry coverage today: all Web Streams (
js_readable_stream_*/js_writable_stream_*/js_writer_*/js_reader_*/js_transform_stream_*/js_stream_unwrap_handle) →Stdlib; alljs_node_http*/js_node_https*/js_node_http2*server FFIs →WellKnown("http")(routes toperry-ext-httpwhich rlibs inperry-ext-http-server).Test plan
cargo test --release -p perry-codegen ext_registry— new unit test covers Stdlib + WellKnown routing plus a non-registered FFI control.cargo test --release --workspace --exclude perry-ui-ios --exclude perry-ui-tvos --exclude perry-ui-watchos --exclude perry-ui-visionos --exclude perry-ui-android --exclude perry-ui-windows --exclude perry-ui-gtk4— full sweep, no regressions.test_gap_async_advanced,_class_advanced,_array_methods,_bigint) still link asruntime-onlyand match Node byte-for-byte — the hook is inert when no listed FFI is emitted.cargo fmt --all --checkclean./tmp/perry-b1-repro-{835,846}/—new ReadableStream({...})andcreateServer((req,res) => …)link cleanly with no explicitnode:*import in the entry TS.PERRY_DISABLE_WELL_KNOWN=1still surfaces the original express link-fails:js_node_http_create_servercodegen'd butperry-ext-http-servernot linked #846 link error (confirms the well-known flip is the consumer my fix feeds).Closes #835
Closes #846