Skip to content

[codex] Guard cyclic inheritance walks in codegen#4261

Draft
andrewtdiz wants to merge 29 commits into
PerryTS:mainfrom
andrewtdiz:codex/fix-pointer-local-type-growth
Draft

[codex] Guard cyclic inheritance walks in codegen#4261
andrewtdiz wants to merge 29 commits into
PerryTS:mainfrom
andrewtdiz:codex/fix-pointer-local-type-growth

Conversation

@andrewtdiz
Copy link
Copy Markdown
Contributor

What changed

  • Add visited-set guards to codegen/lowering/type-analysis parent-chain walks so cyclic native class parent links cannot spin indefinitely during native-facts/codegen.
  • Guard class accessor and this-as-value collector inheritance traversal, including focused cyclic-parent regression tests.
  • Preserve the large-graph codegen object streaming/thread-cap work already needed for the OpenCode graph scale path on this branch.

Why

The OpenCode Perry-native graph moved past the previous default-stack codegen overflow, reached codegen ... visited=2760/2760, and then exited 137 with no output binary. Reduction showed a smaller minimatch/OpenCode util graph getting stuck after codegen in native-facts work. The hang came from cyclic lowered class parent chains: helper walks such as accessor lookup and this-as-value collection repeatedly followed the same parent closure IDs.

The fix makes these inheritance walks terminate on cycles instead of treating parent chains as guaranteed trees.

Validation

  • cargo test -p perry-codegen cyclic_parent_chain -- --nocapture passed: 3 tests.
  • cargo build -p perry --release passed.
  • Focused minimatch/OpenCode reducer passed with the rebuilt release binary:
    • command from /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-mLd9Ee
    • PERRY_NO_CACHE=1 PERRY_CODEGEN_THREADS=1 PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry compile -v --no-auto-optimize minimatch-core-entry.ts -o /tmp/perry-minimatch-rebased-bin
    • result: STATUS=0, wrote /tmp/perry-minimatch-rebased-bin, 20.6 MB, max RSS 764887040.
  • Full OpenTUI/OpenCode graph rerun with this fix moved past the prior post-codegen exit 137/no-binary point, completed object emission, then failed with the next compiler/codegen blocker:
    • exit 1, no binary
    • ✗ 24 module(s) failed to compile — REFUSING TO LINK
    • examples: duplicate LLVM definitions in several Remeda chunks and OpenCode schema modules, array.push_spread expects exactly 1 arg, got 2, continue statement outside any loop, and one @__perry_ns__ undefined-value clang failure.

This PR does not complete the OpenTUI/OpenCode migration. Actual OpenCode Perry-native still has no binary and no UI/input/update/teardown proof.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Follow-up for the OpenCode refusal-to-link blocker:

  • Added 01870b3c3 Fix export alias function symbol collisions.
  • Root cause: Remeda-style modules can have a private local function named a while exporting a different local function as a (export { l as a }). Perry emitted both the private helper and exported wrapper as perry_fn_<module>__a, producing duplicate LLVM definitions and contributing to REFUSING TO LINK.
  • Fix: private-rename only colliding local helper symbols (__local_<id>) and have export wrappers call the resolved function symbol.

Validation:

  • cargo test -p perry-codegen export_alias_can_collide_with_private_local_function_name -- --nocapture: passed.
  • cargo build -p perry --release: passed. Built binary: perry 0.5.1112, sha256 1657bcc1199f3f33393c6220a55ea2920cad5574027cd19404f1560af476b7e0.

Full OpenTUI/OpenCode rerun with the rebuilt binary still produced no binary. It timed out at the configured 1200000ms before object emission, last visible progress around ai/src/types/provider-metadata.ts with visited=1555 collected=1528; kept temp dir /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5. Actual OpenCode Perry-native remains incomplete/no binary.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Updated this PR with a follow-up fix for the new pre-object timeout from the patched OpenCode graph.

Root cause:
rerun_collect_with_class_field_types built a graph-wide class-field seed map and then every module lowering context cloned it through seed_imported_class_fields. On the OpenCode graph the second collection pass spent seconds per otherwise tiny module and timed out before object emission. The fix stores the graph-wide map behind an Arc and resolves imported class fields lazily as a fallback, while preserving local class shadowing semantics.

Validation:

  • cargo test -p perry-hir imported_class_fields_are_shared_and_used_as_fallback -- --nocapture
  • cargo build -p perry
  • cargo build -p perry --release
  • release binary: perry 0.5.1112, sha256 6342e7f0a100f5948d7b4e3ea02f828d27b6732673a638df7de16b37ebbb9d7b
  • direct OpenCode compile from /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5 with PERRY_PROGRESS_TIMINGS=1 completed collection and codegen (visited=2759/2759 collected=2759) and wrote object files before failing at the next blocker. Timing: 960.87 real, max RSS 1687470080, peak footprint 1950045632.

New first blocker after this fix:
17 module(s) failed to compile — REFUSING TO LINK; no binary was produced. First failed module is @anthropic-ai/sdk/src/core/streaming.ts, where clang rejects duplicate LLVM function definition ...__iterator. Other failures include duplicate LLVM functions in npmcli/octokit/openai/remeda/schema modules, too-long object filenames for OpenTelemetry machine-id modules, array.push_spread expects exactly 1 arg, got 2, continue statement outside any loop, and a 52-arg namespace static-method closure call.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Updated this PR with two more OpenCode full-graph movements.

Commits:

  • 2fb2aea82 Disambiguate duplicate local function symbols
  • 3100e654e Separate static and instance method symbols

What moved:

  • Fixed duplicate local function LLVM symbols from repeated nested function names like the Anthropic/OpenAI async function* iterator() shape. Public/exported function symbols stay stable; colliding internal functions now get __local_<FuncId> suffixes.
  • Fixed the next @npmcli/package-json/lib/index.js blocker where PackageJson has both static create(...) and instance create(...). Static method symbols now live in a separate registry from instance method symbols, so perry_static_*__PackageJson__create no longer clobbers perry_method_*__PackageJson__create.

Validation:

  • cargo test -p perry-codegen --test duplicate_function_symbols -- --nocapture
  • cargo build -p perry
  • cargo build -p perry --release
  • release binary: perry 0.5.1112, sha256 9ecf31c068490f947e56ff6fe54767174ce23b203c0c0fcee5c35db47c6fbe5b
  • direct OpenCode compile from /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5 with that release binary completed codegen to visited=2759/2759 collected=2759, wrote object files, and no longer reports the @anthropic-ai/sdk/OpenAI __iterator or @npmcli/package-json PackageJson.create duplicate-symbol failures.

Current remaining terminal result:

  • Still no binary. The graph now refuses to link with 14 module(s) failed to compile — REFUSING TO LINK.
  • New first failed module: @octokit/graphql/dist-bundle/index.js, duplicate LLVM function ...__graphql (/var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/perry_llvm_7513_1780516843300599000_483.ll:3120:15).
  • Other remaining failures include the existing opentelemetry object filename/path failures, Remeda duplicate __a, two array.push_spread expects exactly 1 arg, got 2, duplicate Started/make, two continue statement outside any loop, the 52-arg namespace static-method closure call, and undefined @__perry_ns__ in worktree overlay.
  • Direct compile timing: 852.89 real, max RSS 1688125440, peak footprint 1950226048.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Updated with commit 0f9e04b34 (Reserve renamed value export symbols).

This fixes the current @octokit/graphql/dist-bundle/index.js duplicate LLVM __graphql blocker. Root cause: export { graphql2 as graphql } emitted a public value-export getter symbol ...__graphql, while the local function graphql(...) also used the same public symbol. Codegen now reserves renamed named-export public symbols when assigning local function symbols, so the exported getter keeps ...__graphql and the local function gets a __local_<func_id> suffix.

Focused validation:

cargo test -p perry-codegen --test duplicate_function_symbols -- --nocapture
# 3 passed
cargo build -p perry
cargo build -p perry --release
# passed
./target/release/perry --version
# perry 0.5.1112
shasum -a 256 ./target/release/perry
# 8a94b6c794cb28628fe124720307030c5a8b2ee4e199f36e56038ee616f8d859  ./target/release/perry

Direct OpenCode compile rerun:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5
/usr/bin/time -l env PERRY_PROGRESS_TIMINGS=1 \
  PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
  /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o ./opencode-perry-native-smoke-release-export-alias \
  > /tmp/perry-opencode-0PGQN5-release-export-alias.log 2>&1

Result: codegen reached visited=2759/2759, wrote objects, and @octokit/graphql wrote its object successfully:

Wrote object file: ...__octokit_graphql_9_0_2_node_modules__octokit_graphql_dist_bundle_index_js.o
✗ 13 module(s) failed to compile — REFUSING TO LINK

No binary was produced. The graph moved from 14 failed modules to 13. New first representative failure in the log is the remaining Remeda duplicate local symbol:

Error compiling module '.../remeda/dist/chunk-ANXBDSUI.js' with --backend llvm: clang -c failed
.../perry_llvm_68739_1780520164151374000_1693.ll:2680:15: error: invalid redefinition of function '...chunk_ANXBDSUI_js__a'

Other remaining blockers include array.push_spread expects exactly 1 arg, got 2, continue statement outside any loop, namespace static-method closure call with 52 args, and undefined @__perry_ns__ in opencode-source-overlay/src/worktree/index.ts.

Actual OpenCode Perry-native still has no binary.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Updated with commit 67fa59625 (Deduplicate exported value getter symbols).

This fixes the Remeda chunk-ANXBDSUI.js duplicate LLVM __a blocker. Reduced source shape:

var s = ..., a = ...;
export { s as a, a as b };

Before the fix, codegen emitted two value getter functions named perry_fn_<module>__a: one for the exported alias s as a, and one for the local exported binding a. Codegen now tracks emitted value getter symbols and skips later duplicate getter definitions, while still emitting the distinct alias getter (__b).

Focused validation:

cargo test -p perry-codegen --test duplicate_function_symbols -- --nocapture
# 4 passed
cargo build -p perry
cargo build -p perry --release
# passed
./target/release/perry --version
# perry 0.5.1112
shasum -a 256 ./target/release/perry
# bf40f10fa549caa1bad3a0626fdafd2e07760fc357427451821eec91be89f63d  ./target/release/perry

Direct OpenCode compile rerun:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5
/usr/bin/time -l env PERRY_PROGRESS_TIMINGS=1 \
  PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
  /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o ./opencode-perry-native-smoke-release-value-getter-alias \
  > /tmp/perry-opencode-0PGQN5-release-value-getter-alias.log 2>&1

Result: codegen reached visited=2759/2759, Remeda objects wrote successfully, and the failure count moved from 13 to 9:

Wrote object file: ...remeda_dist_chunk_ANXBDSUI_js.o
✗ 9 module(s) failed to compile — REFUSING TO LINK

No binary was produced. New first blocker is object filename length for OpenTelemetry machine-id modules:

Error writing object file '...getMachineId_darwin_js.o' ... File name too long (os error 63)
Error writing object file '...getMachineId_linux_js.o' ... File name too long (os error 63)
Error writing object file '...getMachineId_unsupported_js.o' ... File name too long (os error 63)

Remaining compile/codegen failures after those include array.push_spread expects exactly 1 arg, got 2, continue statement outside any loop, namespace static-method closure call with 52 args, and undefined @__perry_ns__ in opencode-source-overlay/src/worktree/index.ts.

Actual OpenCode Perry-native still has no binary.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Moved the OpenTelemetry object filename blocker with 5514a0d7b (Shorten generated object filenames).

What changed:

  • Object file stems are now capped at 200 bytes and get a stable 64-bit SHA-256 suffix when truncated.
  • Short names keep the old sanitized stem behavior.
  • Added focused regression coverage for long OpenTelemetry-style machine-id paths staying under the filesystem component limit while remaining distinct.

Validation:

  • cargo test -p perry object_file_stem -- --nocapture passed (2 passed).
  • cargo build -p perry && cargo build -p perry --release passed before the direct graph rerun.
  • Release binary: /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
  • Version/hash: perry 0.5.1112, sha256 c090826b85aecddd0e5e817cfc316e3f2b0b4e0e807953b4fb91b357c35c5b27

Direct OpenCode compile rerun:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5
rm -f /tmp/perry-opencode-0PGQN5-release-safe-obj-names.log ./opencode-perry-native-smoke-release-safe-obj-names
/usr/bin/time -l env PERRY_PROGRESS_TIMINGS=1 \
  PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
  /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o ./opencode-perry-native-smoke-release-safe-obj-names \
  > /tmp/perry-opencode-0PGQN5-release-safe-obj-names.log 2>&1

Result:

  • Completed codegen to visited=2759/2759 collected=2759.
  • The previous File name too long (os error 63) object write failures are gone.
  • Objects now write through that area successfully.
  • Still no binary: Perry refuses to link because 6 module(s) failed to compile.

Current next first blocker from /tmp/perry-opencode-0PGQN5-release-safe-obj-names.log:

Error compiling module '.../node_modules/.bun/zod@4.1.8/node_modules/zod/src/v3/types.ts' ... lowering getter 'ParseInputLazyPath::path': lowering body of method 'ParseInputLazyPath::__get_path': array.push_spread expects exactly 1 arg, got 2

Other remaining failures in that run:

.../packages/core/src/config/plugin/agent.ts: array.push_spread expects exactly 1 arg, got 2
.../packages/sdk/js/src/gen/core/serverSentEvents.gen.ts: continue statement outside any loop
.../packages/sdk/js/src/v2/gen/core/serverSentEvents.gen.ts: continue statement outside any loop
.../opencode-source-overlay/src/effect/app-runtime.ts: namespace static-method closure call with 52 args (max 16)
.../opencode-source-overlay/src/worktree/index.ts: clang -c failed; use of undefined value '@__perry_ns__'

Actual OpenCode Perry-native still has no binary; this only moved the object filename blocker.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Moved the Zod/property-receiver array.push_spread arity blocker with 0a03cebd5 (Pack mixed property push spreads).

What changed:

  • Generic expr.push(...) lowering now treats any spread arg as spread-aware, not only when the first arg is spread.
  • Single-spread calls keep the old push_spread(source) shape.
  • Mixed calls like this._cachedPath.push(...this._path, this._key) are packed into one ArraySpread source and then lowered through existing array.push_spread codegen, preserving the property-receiver writeback path.
  • Added a focused HIR regression mirroring the Zod ParseInputLazyPath::path getter shape.

Validation:

  • cargo test -p perry-hir --test array_push_mixed_spread -- --nocapture passed (3 passed).
  • cargo build -p perry && cargo build -p perry --release passed.
  • Release binary: /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
  • Version/hash: perry 0.5.1112, sha256 0fa5137d74383607d7fdd6b8add711f27df0fd39133886381ec7b785735c7506

Direct OpenCode compile rerun:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5
rm -f /tmp/perry-opencode-0PGQN5-release-push-spread.log ./opencode-perry-native-smoke-release-push-spread
/usr/bin/time -l env PERRY_PROGRESS_TIMINGS=1 \
  PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
  /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o ./opencode-perry-native-smoke-release-push-spread \
  > /tmp/perry-opencode-0PGQN5-release-push-spread.log 2>&1

Result:

  • Completed codegen to visited=2759/2759 collected=2759.
  • Previous zod/src/v3/types.ts and packages/core/src/config/plugin/agent.ts array.push_spread expects exactly 1 arg, got 2 failures are gone.
  • Still no binary: Perry refuses to link because 4 module(s) failed to compile.

Current next first blocker from /tmp/perry-opencode-0PGQN5-release-push-spread.log:

Error compiling module '.../packages/sdk/js/src/gen/core/serverSentEvents.gen.ts' ... lowering closure func_id=9: lowering closure body func_id=9: continue statement outside any loop

Other remaining failures in that run:

.../packages/sdk/js/src/v2/gen/core/serverSentEvents.gen.ts: continue statement outside any loop
.../opencode-source-overlay/src/effect/app-runtime.ts: namespace static-method closure call with 52 args (max 16)
.../opencode-source-overlay/src/worktree/index.ts: clang -c failed; use of undefined value '@__perry_ns__'

Actual OpenCode Perry-native still has no binary; this only moved the mixed property-receiver push_spread blocker.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Moved the generated SSE continue statement outside any loop blocker with commit 5b5173b40 (Fix async generator catch route loop control).

What changed:

  • Fix loop break/continue sentinel resolution for catch routes produced while linearizing yielded for/while bodies.
  • Rewrite synthetic state-dispatch continue pairs when lifted async-generator catch routes are emitted into standalone .throw() closures / direct async step bodies, so they do not escape outside the state-machine loop.
  • Added focused regression: async_generator_catch_route_break_does_not_emit_bare_continue.

Validation:

cargo test -p perry-transform async_generator_catch_route_break_does_not_emit_bare_continue -- --nocapture
# test result: ok. 1 passed; 0 failed

cargo build -p perry --release
# Finished release profile

shasum -a 256 /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
# 52d0e33a8760442b6f897a0edb71fa7a6e850eb349c45bfeb283c5fffe9e74ca

Isolated SSE compile now writes an object:

PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/debug \
/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/debug/perry \
  compile --no-link --no-auto-optimize \
  /private/var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opencode-perry-source.u32cWx/packages/sdk/js/src/gen/core/serverSentEvents.gen.ts \
  -o /tmp/perry-sse-fixed
# Found 1 module(s): 1 native, 0 JavaScript
# Wrote object file: serverSentEvents_gen_ts.o

Full direct OpenCode compile moved from 4 module(s) failed with first blocker packages/sdk/js/src/gen/core/serverSentEvents.gen.ts: continue statement outside any loop to 2 module(s) failed:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5
PERRY_PROGRESS_TIMINGS=1 \
PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o ./opencode-perry-native-smoke-release-continue-route \
  > /tmp/perry-opencode-0PGQN5-release-continue-route.log 2>&1

Terminal result:

[progress] stage=codegen .../opencode-source-overlay/src/worktree/index.ts visited=2759/2759 collected=2759
Error compiling module '.../opencode-source-overlay/src/effect/app-runtime.ts' ... perry-codegen: namespace static-method closure call with 52 args (max 16)
Error compiling module '.../opencode-source-overlay/src/worktree/index.ts' ... clang -c failed ... use of undefined value '@__perry_ns__'
✗ 2 module(s) failed to compile — REFUSING TO LINK
Error: 2 module(s) failed to compile (see errors above); set PERRY_ALLOW_PARTIAL_CODEGEN=1 to link empty stubs anyway
966.33 real ... 1222557696 maximum resident set size ... 1962104448 peak memory footprint

The previous serverSentEvents.gen.ts modules now codegen/write objects in the full graph:

Wrote object file: ...packages_sdk_js_src_gen_core_serverSentEvents_gen_ts.o
Wrote object file: ...packages_sdk_js_src_v2_gen_core_serverSentEvents_gen_ts.o

Actual OpenCode Perry-native still has no binary; the next first blocker is now opencode-source-overlay/src/effect/app-runtime.ts / namespace static-method closure call with 52 args (max 16).

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update: moved the remaining app-runtime.ts 52-arg namespace static-method closure blocker.

New commit: c8b9b1c45 Route large imported closure calls through spread helper

What changed:

  • Shared the imported/namespace var-shaped closure call path through a helper that keeps js_closure_call0..16 for small calls.
  • For >16 positional args, it now lowers args into a stack buffer and calls the existing js_closure_call_apply_with_spread(..., spread=0) runtime helper, so rest-parameter closures like Effect Layer.mergeAll(...52 layers) do not hit the fixed arity cap or get double-packed.
  • Added tests/test_namespace_var_closure_many_args.sh covering export * as Layer + Layer.mergeAll(24 args).

Validation:

  • cargo check -p perry-codegen passed.
  • PERRY_BIN=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry ./tests/test_namespace_var_closure_many_args.sh => PASS.
  • cargo build -p perry --release passed.
  • Release binary sha256: 18dc95cbc20e3427eaf3b88e497349bdf5f762b7b768733e1acf9397632d379f.

Full direct OpenCode graph rerun from the kept temp dir:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5
/usr/bin/time -l env PERRY_PROGRESS_TIMINGS=1 \
  PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
  /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o ./opencode-perry-native-smoke-release-many-arg-closure \
  > /tmp/perry-opencode-0PGQN5-release-many-arg-closure.log 2>&1

Result: moved from 2 module(s) failed to 1 module(s) failed; opencode-source-overlay/src/effect/app-runtime.ts now writes successfully. Actual OpenCode Perry-native still has no binary.

Current next first blocker:

✗ 1 module(s) failed to compile — REFUSING TO LINK
  - private/var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5/opencode-source-overlay/src/worktree/index.ts
...
/var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/perry_llvm_91681_1780530462304334000_2758.ll:139595:29: error: use of undefined value '@__perry_ns__'
 139595 |   %r2003 = load double, ptr @__perry_ns__

The direct compile exited rc=1, reached visited=2759/2759 collected=2759, and produced no opencode-perry-native-smoke-release-many-arg-closure binary.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update: fixed/moved the sole @__perry_ns__ blocker in opencode-source-overlay/src/worktree/index.ts.

Commit: 0df627b61 Resolve dot namespace re-export targets

Root cause: Perry did not classify bare . / .. as relative import specifiers, so export * as Worktree from "." did not resolve/rewrite to the current index.ts module. The dynamic namespace path then sanitized . to an empty module prefix and emitted bare @__perry_ns__.

Fix: treat ., .., ./..., and ../... as relative specs in compile import resolution. Added regression tests/test_dynamic_import_self_namespace_reexport.sh for a dynamic-imported index.ts containing export * as Self from ".".

Validation:

  • cargo build -p perry passed
  • PERRY_BIN=target/debug/perry ./tests/test_dynamic_import_self_namespace_reexport.sh => PASS
  • PERRY_BIN=target/debug/perry ./tests/test_namespace_var_closure_many_args.sh => PASS
  • cargo build -p perry --release passed
  • cargo build --release -p perry-wasm-host passed
  • PERRY_BIN=target/release/perry ./tests/test_dynamic_import_self_namespace_reexport.sh => PASS
  • PERRY_BIN=target/release/perry ./tests/test_namespace_var_closure_many_args.sh => PASS

Fresh release hashes after rebase:

  • target/release/perry: be510e1307142f6d7f535eee22e91b531ef0f7c871e86ce372b0f17df05a6d57
  • target/release/libperry_wasm_host.a: f21f67be53f10c9aee6fc3cf688883a73aafca2d14ab1630030628d3fea0262d

Direct full OpenCode compile rerun with the fresh release binary:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5
/usr/bin/time -l env PERRY_PROGRESS_TIMINGS=1 \
  PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
  /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o ./opencode-perry-native-smoke-release-dot-ns-rebased \
  > /tmp/perry-opencode-0PGQN5-release-dot-ns-rebased.log 2>&1

Result: moved the @__perry_ns__ module failure. Evidence:

  • stage=codegen ... opencode-source-overlay/src/worktree/index.ts visited=2759/2759 collected=2759
  • Wrote object file: ...opencode_source_overlay_src_worktree_index_ts.o
  • then entered Linking (with stdlib)...

New current blocker: final linker undefined symbols, no output binary:

  • Undefined symbols for architecture arm64
  • examples include missing wrapped/default Perry function symbols such as ___perry_wrap_...__leichtgewicht_ip_codec...__default, ___perry_wrap_...__lukeed_ms...__default, Octokit default wrappers, OpenTelemetry enum/value symbols, plus runtime/native symbols like _js_atomics_load, _js_http_request_overload, _js_net_is_ip, _js_node_http_create_server_with_options, and _ptr.
  • terminal: ld: symbol(s) not found for architecture arm64, clang: error: linker command failed with exit code 1, Error: Linking failed
  • max RSS: 2058387456

Actual OpenCode Perry-native still has no binary.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update: moved the first final-link undefined-symbol blocker with 2716349d9 (Use namespace imports for CJS require of ESM).

Root cause: the CJS wrapper hoisted every require(...) as a default ESM import. In the graph, dns-packet/index.js does const ip = require('@leichtgewicht/ip-codec'), but @leichtgewicht/ip-codec/index.mjs only has named exports. Perry then referenced a nonexistent ...ip_codec_index_mjs__default wrapper at link time.

Fix: CJS wrapping now resolves each require target when module-collection context is available. If the target is a native-compiled non-CJS module, the hoisted import is import * as reqN from '...'; unresolved/interpreted/CJS targets keep the existing default import path. Added tests/test_cjs_require_esm_namespace.sh for CJS require('./esm.mjs') over an ESM named-export namespace.

Validation:

  • cargo test -p perry cjs_wrap -- --nocapture => pass, 54 tests
  • PERRY_BIN=target/release/perry ./tests/test_cjs_require_esm_namespace.sh => PASS
  • PERRY_BIN=target/release/perry ./tests/test_dynamic_import_self_namespace_reexport.sh => PASS
  • PERRY_BIN=target/release/perry ./tests/test_namespace_var_closure_many_args.sh => PASS
  • cargo build -p perry --release passed before the final focused reruns
  • release hashes: perry b5d48a76187288c726a539f06b80d43b037a6ba0c72113e3bf78591f0e35ef40; libperry_wasm_host.a f21f67be53f10c9aee6fc3cf688883a73aafca2d14ab1630030628d3fea0262d

Full direct graph rerun:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-0PGQN5
/usr/bin/time -l env PERRY_PROGRESS_TIMINGS=1 \
  PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
  /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o ./opencode-perry-native-smoke-release-cjs-require-esm \
  > /tmp/perry-opencode-0PGQN5-release-cjs-require-esm.log 2>&1

Result: link still fails and actual OpenCode Perry-native still has no binary, but the requested first blocker moved. The previous log had 69 undefined symbols and started with:

1 ___perry_wrap_...__leichtgewicht_ip_codec_index_mjs__default
2 ___perry_wrap_...__lukeed_ms_dist_index_mjs__default
...
9 ___perry_wrap_...__walk_up_path_dist_esm_index_js__default
10 ___perry_wrap_...packages_core_src_database_migration_ts__migrations

The new log has 56 undefined symbols; the leichtgewicht object writes successfully and the old default-wrapper undefined symbols are gone. New first linker blocker:

1 ___perry_wrap_perry_fn_private_var_folders_8q_vx4z9tqd08jd1b5_bykzbdkr0000gn_T_opencode_perry_source_u32cWx_packages_core_src_database_migration_ts__migrations
2 ___perry_wrap_perry_fn_private_var_folders_8q_vx4z9tqd08jd1b5_bykzbdkr0000gn_T_opentui_perry_native_0PGQN5_opencode_source_overlay_src_cli_cmd_run_runtime_ts__INTERACTIVE_INPUT_ERROR
3 _js_atomics_load
...

The direct compile completed object writing and failed at final link in 827.30 real, max RSS 2050228224. Compatibility boundaries remain unchanged; this is only a Perry compiler/link blocker move, not OpenCode migration completion.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Updated #4261 with commit 2047e31aa (Append extensions for dotted import basenames).

Root cause for the first linker undefined symbol:

  • import { migrations } from "./migration.gen" in packages/core/src/database/migration.ts was resolved as migration.ts instead of migration.gen.ts.
  • Path::with_extension("ts") turns a dotted extensionless basename like migration.gen into migration.ts; since that sibling exists, Perry skipped the real producer module and later linked an undefined wrapper for database_migration_ts__migrations.
  • The resolver now only replaces known source extensions; dotted basenames append extensions instead (./migration.gen -> migration.gen.ts).

Regression/validation:

cargo test -p perry dotted_extensionless_relative_import_appends_extension -- --nocapture
# result: ok. 1 passed

cargo build -p perry
# result: finished successfully

PERRY_BIN=target/debug/perry ./tests/test_dotted_extensionless_import.sh
# result: PASS

PERRY_BIN=target/debug/perry ./tests/test_cjs_require_esm_namespace.sh
# result: PASS

PERRY_BIN=target/debug/perry ./tests/test_dynamic_import_self_namespace_reexport.sh
# result: PASS

cargo build -p perry --release
# result: finished successfully

PERRY_BIN=target/release/perry ./tests/test_dotted_extensionless_import.sh
# result: PASS

shasum -a 256 target/release/perry
# e3be7ae3d7738edf5e1afb79356a862f25fa048bf98ba3c3c2d1a270d9098005  target/release/perry

Full graph rerun evidence:

cd /Users/andrew/Documents/opentui/packages/core && \
PERRY_BIN=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
bun scripts/opencode-source-perry-smoke.ts \
  --source-dir /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opencode-perry-source.u32cWx \
  --perry-native --perry-check-deps --perry-verbose \
  --compile-timeout-ms 1200000 --keep-temp \
  --opencode-compat-disable-external-plugin-loader-import \
  --opencode-compat-static-readline-import \
  --opencode-compat-inline-text-import-assets \
  --opencode-compat-disable-image-resizer-import \
  --opencode-compat-disable-debug-agent-eval \
  --opencode-compat-disable-gray-matter-js-engine-eval \
  --opencode-compat-disable-shell-parser-wasm-import \
  --opencode-compat-disable-msgpackr-new-function-reader

Result: the old linker undefined ___perry_wrap_...packages_core_src_database_migration_ts__migrations is moved. The patched compile now resolves and collects migration.gen.ts:

[progress] stage=resolve-import module=.../packages/core/src/database/migration.ts name=migration.ts import=./migration.gen ...
[progress] stage=collect-module module=.../packages/core/src/database/migration.gen.ts ...

New current first blocker: the graph now correctly resolves additional dotted modules, including opencode-source-overlay/src/cli/cmd/run/runtime.lifecycle.ts, and hits Perry's no-runtime-JS guard before codegen/link:

[progress] stage=resolve-import module=.../opencode-source-overlay/src/cli/cmd/run/runtime.ts name=.../runtime.ts import=./runtime.lifecycle ...
[progress] stage=collect-module module=.../opencode-source-overlay/src/cli/cmd/run/runtime.lifecycle.ts ...
Error: JavaScript runtime (V8) support has been removed. This build of Perry compiles TypeScript ahead-of-time only and cannot evaluate JavaScript modules at runtime. The build pulled in a JS runtime via the following file(s):
  - /Users/andrew/Documents/opentui/node_modules/.bun/marked@17.0.1/node_modules/marked/lib/marked.esm.js [marked]
  - /Users/andrew/Documents/opentui/node_modules/.bun/bun-ffi-structs@0.2.2+1fb4c65d43e298b9/node_modules/bun-ffi-structs/dist/index.js [bun-ffi-structs]
  - /Users/andrew/Documents/opentui/node_modules/.bun/solid-js@1.9.12/node_modules/solid-js/dist/solid.js [solid-js]
  - .../opencode-perry-source.u32cWx/node_modules/.bun/solid-js@1.9.10/node_modules/solid-js/dist/solid.js [solid-js]
  - .../opencode-perry-source.u32cWx/node_modules/.bun/solid-js@1.9.10/node_modules/solid-js/store/dist/store.js [solid-js]
  - .../opencode-perry-source.u32cWx/node_modules/.bun/@opentui+solid@0.3.0.../node_modules/@opentui/solid/index.js [@opentui/solid]
  - .../opencode-perry-source.u32cWx/node_modules/.bun/@opentui+solid@0.3.0.../node_modules/@opentui/solid/chunk-7fkmdv3h.js [@opentui/solid]

Kept temp dir from the harness rerun: /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-BVu46p.

Actual OpenCode Perry-native still has no binary. This update only moves the database_migration_ts__migrations missing-producer linker blocker and exposes the next compatibility/package-routing blocker.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update after the OpenTUI/OpenCode no-runtime-JS guard rerun:

  • Added commit f7204c762 (Skip runtime JS edges for type-only named imports).
  • Root cause moved here: Perry treated import { type ... } from "marked" as a runtime edge when every named specifier in the declaration was type-only. HIR now marks an import declaration as runtime type-only when it has no value-bearing specifiers, while preserving mixed imports as runtime imports.
  • Also added runtime-JS guard edge diagnostics, so the V8-free refusal names the native source/import specifier that introduced each JS implementation file.
  • Added tests/test_runtime_js_guard_import_edge.sh, covering both all-type named imports from a JS package (must compile without routing to runtime JS) and value imports from an untrusted JS package (must still fail and name the native edge).

Validation:

cargo test -p perry diagnostic_mentions_native_import_edge_for_runtime_js_module -- --nocapture
cargo test -p perry-hir --lib import -- --nocapture
cargo build -p perry
PERRY_BIN=target/debug/perry ./tests/test_runtime_js_guard_import_edge.sh
cargo build -p perry --release

Fresh release binary:

/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
sha256 99bdf20b6c2880e0f99beaa97e4955330e0fb7e0e421417564238fbfa0e37e42

OpenTUI harness rerun with the fresh release binary still does not produce an OpenCode Perry-native binary, but the false Markdown.ts type-only marked edge is gone. Current first true no-runtime-JS blocker:

- marked
  imported by /Users/andrew/Documents/opentui/packages/core/src/renderables/markdown-parser.ts via `marked`
- bun-ffi-structs
  imported by /Users/andrew/Documents/opentui/packages/core/src/zig-structs.ts via `bun-ffi-structs`
- solid-js
  imported by /Users/andrew/Documents/opentui/packages/keymap/src/solid/index.ts via `solid-js`
- solid-js
  imported by .../opencode-source-overlay/src/cli/cmd/tui/keymap.tsx via `solid-js`
- solid-js/store
  imported by .../opencode-source-overlay/src/cli/cmd/tui/context/theme.tsx via `solid-js/store`
- @opentui/solid
  imported by .../opencode-source-overlay/src/cli/cmd/tui/context/theme.tsx via `@opentui/solid`

Package hint remains: @opentui/solid, bun-ffi-structs, marked, solid-js are still routed to runtime JavaScript because they are not in perry.compilePackages / perry.allow.compilePackages. This looks like a real package-routing/native-target gap now, not another dotted resolver regression.

Harness log: /tmp/perry-opencode-harness-type-only-js-import.log
Kept temp dir: /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-pgbLD0

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Follow-up on the current OpenTUI/OpenCode blocker after f7204c762 (Skip runtime JS edges for type-only named imports):

Status: the false Markdown.ts via marked edge is moved, but actual OpenCode Perry-native still has no binary.

Authoritative harness rerun with release binary sha256 99bdf20b6c2880e0f99beaa97e4955330e0fb7e0e421417564238fbfa0e37e42 failed the no-runtime-JS guard with real value imports:

  • marked imported by /Users/andrew/Documents/opentui/packages/core/src/renderables/markdown-parser.ts via marked
  • bun-ffi-structs imported by /Users/andrew/Documents/opentui/packages/core/src/zig-structs.ts via bun-ffi-structs
  • solid-js imported by /Users/andrew/Documents/opentui/packages/keymap/src/solid/index.ts via solid-js and OpenCode TUI overlay modules
  • @opentui/solid imported by OpenCode TUI overlay modules

I inspected the generated temp host config at /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-pgbLD0/package.json: perry.compilePackages and perry.allow.compilePackages contain 165 reviewed packages, but none of marked, bun-ffi-structs, solid-js, or @opentui/solid. The local source package aliases for @opentui/core and @opentui/keymap are present, but Perry should not implicitly trust/compile their third-party JS dependencies; that would weaken the #497/no-runtime-JS boundary.

Temp-only opt-in experiment: I backed up and edited only the kept temp package.json to add those four packages to both perry.compilePackages and perry.allow.compilePackages, then ran:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-pgbLD0
PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o ./opencode-perry-native-smoke-explicit-native-js-packages \
  2>&1 | tee /tmp/perry-opencode-pgbLD0-explicit-native-js-packages.log

That moved the guard and exposed the first native package blocker in bun-ffi-structs:

Error: dynamic import() in module index.js (/Users/andrew/Documents/opentui/node_modules/.bun/bun-ffi-structs@0.2.2+1fb4c65d43e298b9/node_modules/bun-ffi-structs/dist/index.js): path argument references a binding that is not statically resolvable to a literal

Small reducer:

cd /tmp/perry-bun-ffi-structs-native-repro
PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts -o ./repro-bin

Reducer shape:

import { defineEnum } from "bun-ffi-structs"
const Example = defineEnum({ one: 1 }, "u8")
console.log(Example.to("one"))

with package.json opt-in:

{
  "type": "module",
  "perry": {
    "compilePackages": ["bun-ffi-structs"],
    "allow": { "compilePackages": ["bun-ffi-structs"] }
  }
}

Root source in bun-ffi-structs/dist/index.js:

var backend = await loadBackend();
async function loadBackend() {
  if (typeof process !== "undefined" && "bun" in process.versions) {
    return createBunBackend(await importModule("bun:ffi"));
  }
  try {
    return createNodeBackend(await importModule("node:ffi"));
  } catch (error) { ... }
}
function importModule(specifier) {
  return import(specifier).then((module) => module.default ?? module);
}

I also probed a direct await import("bun:ffi") shape. Perry warns Could not resolve import 'bun:ffi' and later links with an undefined _js_module_dynamic_import_apply_hooks under --no-auto-optimize, so simply teaching dynamic-import folding about literal helper parameters would not be enough to make bun-ffi-structs native-safe. The next real fix is a reviewed native strategy for this dependency: either an OpenTUI/Perry-native replacement for bun-ffi-structs that avoids the backend dynamic import, or Perry support for the relevant FFI/struct-layout surface. The generated temp config was restored to the harness-generated state after the experiment.

Latest authoritative coordinator rerun command remains:

cd /Users/andrew/Documents/opentui/packages/core && \
PERRY_BIN=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
bun scripts/opencode-source-perry-smoke.ts \
  --source-dir /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opencode-perry-source.u32cWx \
  --perry-native --perry-check-deps --perry-verbose \
  --compile-timeout-ms 1200000 --keep-temp \
  --opencode-compat-disable-external-plugin-loader-import \
  --opencode-compat-static-readline-import \
  --opencode-compat-inline-text-import-assets \
  --opencode-compat-disable-image-resizer-import \
  --opencode-compat-disable-debug-agent-eval \
  --opencode-compat-disable-gray-matter-js-engine-eval \
  --opencode-compat-disable-shell-parser-wasm-import \
  --opencode-compat-disable-msgpackr-new-function-reader

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update: pushed ffda496e9 Fix exported no-init var bindings to move the TS-emitted enum/no-init export producer-symbol blocker.

Root cause: export var TraceFlags; was pre-registering a module var but, without an initializer, HIR lowering did not emit module init binding/export metadata/exported object getter. Consumers of enum-IIFE patterns like export var X; (function (X) { ... })(X || (X = {})); linked against perry_fn_<module>__X with no producer-side symbol.

Fix: lower no-init exported identifier vars through the normal var-decl path and register the named export/exported object. Added tests/test_export_var_enum_iife.sh covering direct import plus barrel re-export.

Validation:

  • cargo build -p perry passed.
  • cargo build -p perry-runtime passed.
  • PERRY_BIN=target/debug/perry PERRY_RUNTIME_DIR=target/debug ./tests/test_export_var_enum_iife.sh => PASS.
  • cargo build -p perry-runtime --release && cargo build -p perry --release && PERRY_BIN=target/release/perry PERRY_RUNTIME_DIR=target/release ./tests/test_export_var_enum_iife.sh => PASS.
  • release target/release/perry sha256: 58e3b42bed7e5c5869ee8945e3ece6c543f9d2ae9af63261d5b7714a50056eaa.

Full graph evidence with corrected OpenTUI win32 FFI guard:

  • Harness log: /tmp/perry-opencode-opentui-native-export-var-enum.log.
  • Kept temp dir: /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW.
  • Harness got to visited=2945/2945 collected=2945, then stopped because local release runtime archives were incomplete: WebAssembly.* used but libperry_wasm_host.a not found.
  • After building the needed release archives (perry-wasm-host plus ext libs), direct compile from the kept temp reached final native link. Log: /tmp/perry-opencode-opentui-native-export-var-enum-with-libs-direct.log.
  • The previous enum/export-shaped undefineds (TraceFlags, ElementType, DecodingMode, QuoteType) are no longer present in the linker undefined list.

Current next blocker: final native link still fails, first undefined is _js_webassembly_module_new referenced from Undici lib/dispatcher/client-h1.js; next visible symbols include _perry_fn_Users_andrew_Documents_opentui_packages_core_src_perry_native_ts__Yoga, Effect NodeSocket NodeWS, residual short export values, and Undici dispatcher destroy methods. Actual OpenCode Perry-native still has no binary.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update after 7327fec3a (Fix namespace materialization for OpenCode graph):

What changed:

  • Materialize namespace globals for static namespace re-export sources, including self-aliases such as export * as AccountV2 from "./account".
  • Route export * as NodeWS from "ws"-style well-known/out-of-tree namespace re-exports through native module namespace construction instead of trying to load a nonexistent @__perry_ns_ws global.
  • Deduplicate nested namespace extern declarations when the same namespace source is needed by both namespace materialization and namespace re-export getter emission.
  • Added focused codegen regressions covering self namespace re-export globals, foreign nested namespace extern declaration, and the duplicate extern recurrence.

Validation:

  • cargo fmt --package perry --package perry-codegen
  • cargo test -p perry-codegen --test duplicate_function_symbols namespace -- --nocapture -> 4 passed
  • cargo check -p perry -> passed with existing warnings
  • cargo build -p perry --release -> passed with existing warnings
  • release binary: /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
  • sha256: a5b46b48d5a563a96f5e911cab7991e4e77f045c1ca03a9e2d7624ab74f796e3

Direct OpenCode graph compile validation:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW
PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW/opencode-perry-native-smoke-ns-dedupe

Result:

  • log: /tmp/perry-opencode-opentui-native-ns-dedupe.log
  • time: /tmp/perry-opencode-opentui-native-ns-dedupe.time
  • reached visited=2945/2945 collected=2945
  • wrote opencode-source-overlay/src/worktree/index.ts.o
  • previous duplicate namespace extern codegen blocker is gone
  • no output binary was produced; actual OpenCode Perry-native still has no product proof
  • terminal blocker is final native link undefined symbols

Current first linker undefined symbol:

Undefined symbols for architecture arm64:
  "_perry_fn_...domhandler_lib_esm_index_js__hasChildren", referenced from:
      _...domutils_lib_esm_index_js__init in ...domutils...o

Next visible related symbols include domhandler predicates (isCDATA, isComment, isDocument, isTag, isText), Effect alias symbols (Context.ts__a, Effect.ts__a, Option.ts__a), Remeda/y18n/Zod no-init export symbols, and Undici Dispatcher.destroy methods. This is a new final-link blocker, not product completion.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Updated with a0c1c6e51 (Fix export-all function forwarders).

What changed:

  • Adds producer-side callable forwarders for function exports that arrive through export * barrels.
  • The OpenCode representative was domhandler/lib/esm/index.js re-exporting predicates from ./node.js, while downstream code referenced perry_fn_<domhandler-index>__hasChildren and friends.
  • Includes the forwarder metadata in the object-cache key so stale barrel objects are invalidated.
  • Adds focused codegen regression export_all_barrel_emits_callable_function_forwarder.

Validation:

  • cargo test -p perry-codegen --test duplicate_function_symbols export_all_barrel_emits_callable_function_forwarder -- --nocapture
  • cargo test -p perry compile::object_cache::object_cache_tests::key_changes_with_dynamic_import_metadata -- --nocapture
  • cargo test -p perry-codegen --test duplicate_function_symbols -- --nocapture
  • cargo check -p perry
  • cargo build -p perry --release

Release binary:

  • /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
  • sha256 79aae80e2a164452b47be4e528685f154477ea2e5f12581a34f782f219082ab6

Direct OpenCode compile result:

  • cwd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW
  • output attempted /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW/opencode-perry-native-smoke-exportall-fn
  • log /tmp/perry-opencode-opentui-native-exportall-fn.log
  • time 887.17s user 78.23s system 99% cpu 16:10.28 total
  • rc 1, no output binary

Moved blocker:

  • The domhandler missing symbols (hasChildren, isCDATA, isComment, isDocument, isTag, isText) are no longer present in the linker failure.

Current next first blocker:

  • final native link still fails. First undefined symbols are now Effect short export aliases:
    • _perry_fn_...effect_src_Context_ts__a
    • _perry_fn_...effect_src_Effect_ts__a
    • _perry_fn_...effect_src_Option_ts__a
  • Later remaining symbols include Remeda chunk_WIMGWYZL_js__c, y18n default, Zod regexes_ts___null / ___undefined, and Undici Dispatcher*__destroy.

Actual OpenCode Perry-native still has no binary and no product truth proof.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update after commit 8775fbec1 (Fix scoped namespace import aliases):

  • Added scoped namespace member origin metadata so namespace re-export aliases like import { Context } from "effect"; Context.omit(...) do not pollute flat bare-import symbol suffixes such as Remeda omit(...)/map(...) in the same consumer module.
  • Added codegen coverage for scoped namespace member prefix/origin lookup and object-cache coverage for the new metadata.
  • Validation:
    • cargo test -p perry-codegen --test duplicate_function_symbols -- --nocapture
    • cargo test -p perry codegen_thread_tests::export_origin_override_ -- --nocapture
    • cargo test -p perry object_cache_tests::key_changes_with_namespace_member -- --nocapture
    • cargo check -p perry
    • cargo build -p perry --release
  • Release binary sha256: f7afa830c9ee8a6e60673dd07a4016454e9d847a1edb5fd8d0445bd7a6432b01.
  • Direct OpenCode compile rerun: /tmp/perry-opencode-opentui-native-effect-scoped-origin.log, output target opencode-perry-native-smoke-effect-scoped-origin, time real 968.44 user 881.42 sys 78.89.
  • Result: all 2945 modules reached final native link. The previous Effect short alias undefineds (Context_ts__a, Effect_ts__a, Option_ts__a) are gone. Actual OpenCode Perry-native still has no binary.
  • New first linker blocker is Remeda: _perry_fn_...node_modules__bun_remeda_2_26_0_node_modules_remeda_dist_chunk_WIMGWYZL_js__c, referenced from the chunk_WMCGP7PY_js__o wrapper. Remaining linker symbols after that include y18n index_js__default, Zod regexes_ts___null / ___undefined, and Undici Dispatcher*.destroy methods.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Remeda chunk_WIMGWYZL_js__c linker blocker is moved by fa5655160 Fix aliased import compatibility collisions.

Root cause: the compile import map kept an old compatibility entry keyed by an imported symbol's exported name. In the Remeda shape:

import { a as n, c as a } from "./chunk-ANXBDSUI.js"
import { a as t } from "./chunk-WIMGWYZL.js"

the later a as t import registered exported-name key a -> WIMGWYZL, shadowing the real local binding a -> ANXBDSUI while the origin suffix still came from export c. That produced a call/declaration for nonexistent WIMGWYZL__c. The fix keeps local-name entries, but skips the exported-name compatibility alias when that exported name is already a real import-local binding in the same module.

Validation:

  • cargo test -p perry codegen_thread_tests::exported_name_compat_alias_does_not_shadow_import_local -- --nocapture
  • cargo test -p perry codegen_thread_tests::export_origin_override_ -- --nocapture
  • cargo test -p perry-codegen --test duplicate_function_symbols -- --nocapture
  • cargo build -p perry --release
  • Release binary: target/release/perry, sha256 39ea69dba78c1d036078d191c0fd0c7cc931ae1576d1b1165783517691d86e27
  • Focused /tmp/perry-remeda-alias-collision-repro compile with --trace llvm --no-link --no-cache writes IR with no wimg_js__c; local a resolves to the first module's exported c, local t resolves to wimg_js__a.

Full direct OpenCode compile:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW
PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize entry.ts \
  -o /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW/opencode-perry-native-smoke-remeda-alias-compat

Log: /tmp/perry-opencode-opentui-native-remeda-alias-compat.log

Result: reached final native link, no output binary. The previous Remeda undefined symbol _perry_fn_...remeda_dist_chunk_WIMGWYZL_js__c is gone. New first linker blocker is:

_perry_fn_private_var_folders_8q_vx4z9tqd08jd1b5_bykzbdkr0000gn_T_opencode_perry_source_u32cWx_node_modules__bun_y18n_5_0_8_node_modules_y18n_build_lib_index_js__default

Remaining visible linker symbols after that are Zod regexes_ts___null, Zod regexes_ts___undefined, and Undici Dispatcher1Wrapper__destroy / Dispatcher__destroy methods.

Actual OpenCode Perry-native still has no binary/product proof.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update for the y18n linker blocker:

  • Fixed package exports array resolution in resolve_exports_with_conditions. Perry now walks array entries in order and chooses the first matching conditional target, instead of falling through to the legacy module field.
  • Root cause: y18n@5.0.8 has exports["."] as an array where the import target is ./index.mjs; Perry skipped the array form, fell back to module: ./build/lib/index.js, and then yargs/default-import consumers referenced a build/lib/index.js__default producer that does not exist.
  • Commit: 6c63c3143 Fix package exports array resolution
  • Validation:
    • cargo test -p perry package_exports_array_prefers_import_condition_before_module_field -- --nocapture
    • cargo build -p perry --release
    • release binary sha256: 6946d72a06972b5953ca269a88bbeb3d4f6afdd9af0ec41f7075ebe86b4e6a1e
  • Direct OpenCode graph compile from /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW reached final native link again. It collected/codegen'd/wrote y18n build/lib/index.js, build/lib/platform-shims/node.js, and index.mjs; the old _perry_fn_...y18n_build_lib_index_js__default undefined symbol is gone.
  • Full run command:
    PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry compile -v --no-auto-optimize entry.ts -o /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW/opencode-perry-native-smoke-y18n-exports-array
  • Full run log: /tmp/perry-opencode-opentui-native-y18n-exports-array.log
  • Timing: real 952.67, user 874.79, sys 76.23

New first blocker after this fix: final native link undefineds now start with Zod regexes.ts exported null/undefined symbols, followed by Undici Dispatcher1Wrapper.destroy / Dispatcher.destroy methods. No OpenCode Perry-native binary yet.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update: moved the Zod reserved-name export linker blocker.

New commit on this branch:

  • a841a9823 Fix renamed value export namespace getters

What changed:

  • Producer-side codegen now emits a local-name getter alias for renamed value exports when the public exported getter exists but namespace-origin metadata can still ask for the local binding name.
  • Concrete OpenCode/Zod shape:
    const _null = /^null$/i;
    export { _null as null };
    const _undefined = /^undefined$/i;
    export { _undefined as undefined };
    Full graph namespace materialization referenced perry_fn_<zod regexes>___null / ___undefined, while the producer only emitted __null / __undefined before this fix.

Validation:

  • cargo test -p perry-codegen --test duplicate_function_symbols renamed_value_export_emits_local_namespace_getter_alias -- --nocapture
  • cargo test -p perry-codegen --test duplicate_function_symbols -- --nocapture
  • cargo build -p perry --release
  • release binary: /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
  • release sha256: ce29d165b25907d4b4dcec0c136ff90758dcf1612551ef4209184cb158a92aa1

Focused compiler-path check:

  • Real Zod regexes.ts repro now emits object symbols for both public and local getters:
    • _perry_fn_regexes_ts__null
    • _perry_fn_regexes_ts___null
    • _perry_fn_regexes_ts__undefined
    • _perry_fn_regexes_ts___undefined

Full direct OpenCode compile rerun:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW
/usr/bin/time -p sh -c 'PERRY_NO_CACHE=1 PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry compile -v --no-auto-optimize entry.ts -o /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW/opencode-perry-native-smoke-zod-reserved-alias' > /tmp/perry-opencode-opentui-native-zod-reserved-alias.log 2>&1

Result:

  • reached visited=2947/2947 collected=2947
  • all module objects wrote, including opencode-source-overlay/src/worktree/index.ts
  • no output binary
  • final native link failed
  • previous Zod undefineds zod_src_v4_core_regexes_ts___null / ___undefined are gone
  • time: real 961.97, user 878.51, sys 78.10

New current first linker blocker:

Undefined symbols for architecture arm64:
  "_perry_method_...undici_index_js__Dispatcher1Wrapper__destroy", referenced from:
      _perry_closure_...@effect_platform_node_src_NodeClusterSocket_ts__6
      _perry_closure_...@effect_platform_node_src_NodeHttpClient_ts__4
      _perry_closure_...@effect_platform_node_src_NodeHttpClient_ts__54
      _perry_closure_...@effect_platform_node_src_NodeHttpClient_ts__57
  "_perry_method_...undici_index_js__Dispatcher__destroy", referenced from:
      _perry_closure_...@effect_platform_node_src_NodeClusterSocket_ts__6
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1
Error: Linking failed

Actual OpenCode Perry-native still has no binary/product proof.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Updated with 5ca8ff2ca Fix CJS class re-export method origins.

What changed:

  • Teach the CJS wrapper to recognize const X = require("leaf"); module.exports.X = X / exports.X = X barrels and emit direct named re-exports from the leaf default. This covers Undici's index.js export shape for Dispatcher and Dispatcher1Wrapper.
  • Preserve class aliases while normalizing imported class method producer prefixes to the canonical class-defining module when a source class id is known. This prevents typed method calls through a barrel namespace from inventing missing method symbols like undici_index_js__Dispatcher__destroy when the producer is the leaf dispatcher module.

Focused validation:

  • cargo fmt --check
  • cargo test -p perry cjs_wrap -- --nocapture -> 57 passed
  • cargo build -p perry
  • cargo build -p perry --release
  • Local reducer for a CJS class barrel + namespace import compiled with --no-link; nm showed leaf Dispatcher__destroy / Dispatcher1Wrapper__destroy producers and no barrel index_js__...__destroy method symbols.

Release binary:

  • /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
  • sha256 ab9b4c86b3dde90493cdb533151fd96564bb24e0676d3937b0ec48e01ba2bd64

Full direct OpenCode graph rerun:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW
/usr/bin/time -p sh -c 'PERRY_NO_CACHE=1 PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry compile -v --no-auto-optimize entry.ts -o /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-xptSVW/opencode-perry-native-smoke-undici-dispatcher-canonical-class' > /tmp/perry-opencode-opentui-native-undici-dispatcher-canonical-class.log 2>&1

Result:

  • Reached visited=2947/2947 collected=2947 and final native link.
  • The previous Undici undefineds ...undici_index_js__Dispatcher1Wrapper__destroy and ...undici_index_js__Dispatcher__destroy are gone.
  • No output binary was produced.
  • New first linker blocker is Effect method producer/origin symbols, starting with _perry_method_...effect_src_SchemaAST_ts__FiberImpl_A_E__addObserver, followed by related FiberImpl_A_E methods such as children, evaluate, getCont, getRef, interruptUnsafe, pipe, pollUnsafe, setContext, and yieldWith.
  • Runtime: real 1470.44, user 1028.97, sys 120.26.

Actual OpenCode Perry-native still has no binary; this only moves the Undici dispatcher method linker blocker.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update for the Effect FiberImpl_A_E linker blocker.

Committed/pushed dff77f463 (Fix canonical class origins for generic id collisions) to codex/fix-pointer-local-type-growth.

Diagnosis:

  • The missing symbols were references like SchemaAST_ts__FiberImpl_A_E__addObserver, while the producer object defines internal_effect_ts__FiberImpl_A_E__addObserver.
  • The native compile command builds a canonical class-origin map in compile.rs. It was keyed by ClassId only.
  • Original lowered class ids are graph-wide, but monomorphized generic class ids are allocated per module from local max id + 1000. In the full OpenCode graph, an unrelated class in SchemaAST.ts and the specialized Effect FiberImpl_A_E can share a numeric id, so the id-only map rewrote imported method metadata to the wrong source prefix.
  • The fix keys canonical class origins by (ClassId, class name) and uses that key for parent closure lookup, canonical export filtering, and imported-class prefix normalization.

Validation:

  • cargo fmt --check
  • cargo test -p perry class_canonical_key_keeps_monomorph_collisions_separate -- --nocapture
  • cargo build -p perry
  • cargo build -p perry --release
  • Release binary: /Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
  • Version/sha256: perry 0.5.1116 / a043e1bc7eebdc1f26cbe5bc79d43874ecdaebc48c8eacc23e72d4c6d4e4878a

Full graph rerun caveat from this worker:

  • Attempted both the harness command and direct kept-temp compile.
  • Harness failed immediately with error: An unknown error occurred (Unexpected) before spawning the long graph.
  • Direct kept-temp compile now stops at visited=2484 collected=2473 with Error: Failed to read /Users/andrew/Documents/opentui/packages/keymap/src/extras/index.ts: Operation not permitted (os error 1).
  • A local probe can stat that file but cannot read its contents (head also gets Operation not permitted), so this worker cannot currently complete the authoritative OpenTUI rerun. This does not prove the full graph is fixed; it only blocks verification from this thread.

Coordinator rerun command should use the existing OpenTUI harness and include the Win32 FFI no-op flag:

cd /Users/andrew/Documents/opentui/packages/core && \
PERRY_BIN=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
bun scripts/opencode-source-perry-smoke.ts \
  --source-dir /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opencode-perry-source.u32cWx \
  --perry-native --perry-check-deps --perry-verbose \
  --compile-timeout-ms 1200000 --keep-temp \
  --opencode-compat-disable-external-plugin-loader-import \
  --opencode-compat-static-readline-import \
  --opencode-compat-inline-text-import-assets \
  --opencode-compat-disable-image-resizer-import \
  --opencode-compat-disable-debug-agent-eval \
  --opencode-compat-disable-gray-matter-js-engine-eval \
  --opencode-compat-disable-shell-parser-wasm-import \
  --opencode-compat-disable-msgpackr-new-function-reader \
  --opencode-compat-disable-win32-ffi-console-control

Actual OpenCode Perry-native still has no binary/product proof from this update.

@andrewtdiz
Copy link
Copy Markdown
Contributor Author

Update after the latest coordinator timeout:

  • Added commit 6a3e0c903 (Avoid class origin lookup allocations).
  • This keeps the dff77f463 canonical class-origin correctness fix keyed by class id + class name, but changes the backing map to ClassId -> { class_name -> source_path } so hot codegen lookups can use &str without allocating a fresh (ClassId, String) key each time.
  • Rationale: the coordinator's latest full OpenCode graph timed out at 1200000ms around codegen visited=2726/2947, with the last visible module opencode-source-overlay/src/effect/runner.ts. In the kept temp, only 2724 .o files existed and effect_runner_ts.o had not been written. A temp-root reduced entry importing @/effect/runner compiled and linked successfully, so this looked like aggregate codegen overhead rather than a standalone runner lowering hang.

Validation on this worker:

cargo fmt --check
cargo test -p perry class_canonical_paths_keep_monomorph_collisions_separate -- --nocapture
cargo build -p perry
cargo build -p perry --release

Reduced OpenCode-temp smoke with rebuilt release binary:

cd /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-CcHmFS && \
PERRY_RUNTIME_DIR=/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release \
/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry \
  compile -v --no-auto-optimize runner-entry.ts \
  -o /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-CcHmFS/runner-entry-bin-canonical-path-map

Result: wrote /var/folders/8q/vx4z9tqd08jd1b5_bykzbdkr0000gn/T/opentui-perry-native-CcHmFS/runner-entry-bin-canonical-path-map (37.6MB) in real 20.01.

Release binary for coordinator rerun:

/Users/andrew/.codex/worktrees/perry-opencode-compile-timeout/target/release/perry
sha256 8775ca30b872e62b76eeec8dcab22fa726ffa43235c6058805e0cc22d919c520
perry 0.5.1116

I could not run the authoritative full OpenTUI harness from this worker because direct compile of the kept temp hits Operation not permitted (os error 1) reading /Users/andrew/Documents/opentui/packages/keymap/src/extras/index.ts. Actual OpenCode Perry-native still has no binary/product proof; this commit needs the coordinator full graph rerun.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant