Skip to content

fix: #835 #846 — auto-link providing crate when codegen emits its FFIs#868

Merged
proggeramlug merged 1 commit into
mainfrom
fix/835-846-auto-link-codegen-ffis
May 16, 2026
Merged

fix: #835 #846 — auto-link providing crate when codegen emits its FFIs#868
proggeramlug merged 1 commit into
mainfrom
fix/835-846-auto-link-codegen-ffis

Conversation

@proggeramlug
Copy link
Copy Markdown
Contributor

Summary

When TS code (often inside a perry.compilePackages package) emits a call to an FFI whose implementation lives in perry-stdlib or a perry-ext-* crate, but no matching import "node:…" shows up in ctx.native_module_imports, the link step picks "runtime-only" (or omits the well-known flip), and the linker fails with the symptoms reported on #835 / #846:

Codegen already knows the FFI name at call-emission time. This PR records every such emission against a small registry (OwnerKind::Stdlib or OwnerKind::WellKnown("http") / "streams" / …) and folds the drained set into ctx.needs_stdlib + ctx.native_module_imports right before build_optimized_libs runs — the existing well-known flip + minimal-stdlib feature computation pick up the augmented set unchanged.

Design (per the design note in the issue)

  • New module crates/perry-codegen/src/ext_registry.rs:
    • Static FFI_REGISTRY table mapping each affected FFI symbol → providing crate.
    • Process-wide Mutex<HashSet<OwnerKind>> collector populated by codegen.
    • take_used_providers() drain consumed by the CLI driver.
  • crates/perry-codegen/src/block.rs: single-line hook in LlBlock::call and LlBlock::call_void. Inert for symbols not in the registry.
  • crates/perry/src/commands/compile.rs: drains at the start of run_with_parse_cache (so a stale perry dev set can't bleed in) and once before build_optimized_libs, routing entries through the existing flip mechanisms.

Registry coverage today: all Web Streams (js_readable_stream_* / js_writable_stream_* / js_writer_* / js_reader_* / js_transform_stream_* / js_stream_unwrap_handle) → Stdlib; all js_node_http* / js_node_https* / js_node_http2* server FFIs → WellKnown("http") (routes to perry-ext-http which rlibs in perry-ext-http-server).

Test plan

  • cargo test --release -p perry-codegen ext_registry — new unit test covers Stdlib + WellKnown routing plus a non-registered FFI control.
  • cargo test --release --workspace --exclude perry-ui-ios --exclude perry-ui-tvos --exclude perry-ui-watchos --exclude perry-ui-visionos --exclude perry-ui-android --exclude perry-ui-windows --exclude perry-ui-gtk4 — full sweep, no regressions.
  • Spot-check gap tests (test_gap_async_advanced, _class_advanced, _array_methods, _bigint) still link as runtime-only and match Node byte-for-byte — the hook is inert when no listed FFI is emitted.
  • cargo fmt --all --check clean.
  • Repro programs in /tmp/perry-b1-repro-{835,846}/new ReadableStream({...}) and createServer((req,res) => …) link cleanly with no explicit node:* import in the entry TS.
  • Inverse check with PERRY_DISABLE_WELL_KNOWN=1 still surfaces the original express link-fails: js_node_http_create_server codegen'd but perry-ext-http-server not linked #846 link error (confirms the well-known flip is the consumer my fix feeds).

Closes #835
Closes #846

When TS code (often inside a `perry.compilePackages` package) emits a
call to an FFI whose implementation lives in `perry-stdlib` or a
`perry-ext-*` crate, but no matching `import "node:…"` shows up in
`ctx.native_module_imports`, the link step picks "runtime-only" or
omits the well-known crate flip, and the linker fails with:

- `Undefined symbols: _js_readable_stream_*` / `_js_stream_unwrap_handle`
  when Effect's `Stream` lowers `new ReadableStream(...)` calls via
  paths that don't trip `uses_fetch` (#835).
- `Undefined symbols: _js_node_http_create_server` when Express's
  compiled output emits `js_node_http_create_server` from a code path
  that didn't surface `node:http` in `ctx.native_module_imports` (#846).

Codegen already knows the FFI name at call-emission time. Adds a
process-wide registry mapping each affected FFI symbol to its
providing crate (`Stdlib` or `WellKnown("http")` / `WellKnown("streams")`
/ …), hooks `LlBlock::call` / `call_void` to record every emission
against the registry, and folds the drained set into
`ctx.needs_stdlib` + `ctx.native_module_imports` right before
`build_optimized_libs` runs. The existing well-known flip + minimal-
stdlib feature computation pick up the augmented set unchanged.

- `crates/perry-codegen/src/ext_registry.rs` — new module hosting the
  symbol→owner table and the mutex-guarded `USED_PROVIDERS` collector.
  Covers all stream / writer / reader FFIs and every `js_node_http*`
  / `js_node_https*` / `js_node_http2*` server symbol shipped by
  `perry-ext-http-server`.
- `crates/perry-codegen/src/block.rs` — single-line hook in
  `LlBlock::call` and `LlBlock::call_void`. Zero behavioural change
  for symbols not in the registry.
- `crates/perry/src/commands/compile.rs` — drains the collector once
  at the start of `run_with_parse_cache` (in case a prior `perry dev`
  rebuild left stale entries) and once before `build_optimized_libs`,
  routing `OwnerKind::Stdlib` → `ctx.needs_stdlib`, and
  `OwnerKind::WellKnown(key)` → `ctx.native_module_imports.insert(key)`
  + `ctx.needs_stdlib`.

Validation:
- New unit test `ext_registry::tests::registry_dispatch_routes_to_correct_owner`
  exercises the Stdlib + WellKnown lookup paths plus a non-registered
  FFI control.
- Full workspace test sweep passes (`cargo test --release --workspace`
  with the cross-host UI exclusions from CLAUDE.md).
- Spot-check gap tests (`test_gap_async_advanced`, `_class_advanced`,
  `_array_methods`, `_bigint`) still link runtime-only and match Node
  byte-for-byte — the registry is inert when no listed FFI is emitted.
- Repros: `new ReadableStream({...})` and `createServer((req,res) =>
  …)` link cleanly under `target/release/perry compile` with no
  explicit `node:*` import in the entry TS.

Closes #835
Closes #846
@proggeramlug proggeramlug merged commit 9f3e817 into main May 16, 2026
9 checks passed
@proggeramlug proggeramlug deleted the fix/835-846-auto-link-codegen-ffis branch May 16, 2026 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant