feat: per-resource handle tables + alias-fallback (1/3 trio runtime passes)#108
Merged
feat: per-resource handle tables + alias-fallback (1/3 trio runtime passes)#108
Conversation
Adds scripts/explore/ — sibling pipeline to scripts/mythos/ — for exploring design spaces where multiple alternatives exist and the right answer requires "implementation as argumentation." Key adaptations from mythos: - Two-axis rank: viability (1-5) AND locus-of-fix (meld | wit-bindgen | spec | wasm-tools | hybrid). Honest "this isn't ours to fix" is a valid output. - Oracle: a fuse_only_test! flips to a passing runtime_test! on the prototype branch, instead of mythos's failing-Kani-plus-PoC pair. - "NOT VIABLE TODAY: <reason>" replaces "NO FINDING: could not satisfy oracle" as the honest-rejection escape hatch. - Validator can reject for "real but uninteresting" the same way mythos does — high maintenance + low generalization is a three-strikes-out for ADR promotion. Adds safety/adr/ as the emission target: - schema.yaml defines design-question and design-adr shapes - ADR-0 is the parent question for the re-exporter resource chain problem (epic #69 / issue #92), enumerating seven candidate paths (D, E, F, G, H, I, J) for sibling ADRs to explore in parallel - Path I introduces the pulseengine/wit-bindgen fork as a venue for the opaque-rep convention proposal No code changes. Pipeline templates only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rekey HandleTableInfo from HashMap<usize, _> to HashMap<(usize, String, String), _>, so a re-exporter component with multiple resources gets one handle table per (component, interface, resource_name) instead of a single shared table that misroutes imports the re-exporter passes through. Background: 3-component re-exporter chains (resource_floats, resource_with_lists, resource-import-and-export) trapped at runtime with 'wasm unreachable' (rustc/LLVM assume(ptr.is_aligned()) debug assertion firing because leaf code dereferenced the rep as a Box<T> pointer when the rep was actually a small handle integer). Convergent diagnosis from prototype paths D + E pinpointed the seam: merger.rs:739 redirected ALL [resource-*] imports of any re-exporter component through one shared ht_* regardless of which resource the import was for. Changes: - resolver.rs: add reexporter_resources field to DependencyGraph, populated from adapter sites where callee_defines_resource=false. - merger.rs: rekey handle_tables by (comp, iface, resource_name); refactor allocate_handle_tables to allocate one table per (component, resource); refactor the redirect loop with [export]-prefix / resource-graph discrimination to determine the owner of each [resource-*] import; add ht_export_suffix helper. - fact.rs: add parse_resource_import helper; update three handle-table lookup sites to use per-resource keys (caller-side ht_rep, callee-side ht_new, callee-side ht_rep for own results). - component_wrap.rs: update LocalResource ht_* lookup to use the new per-resource export naming. Validation: - 73-test wit-bindgen suite passes with 0 regressions. - The originally-failing trio's trap shape changes from 'wasm unreachable' to 'unknown handle index 0x110004' — proving the per-resource routing now works (leaf no longer derefs handle as Box<T>) and narrowing the diagnosis to a remaining wrapper-boundary leak. - Opaque-rep oracle (pulseengine/wit-bindgen feat/opaque-rep-attribute, resource_floats_opaque) continues to pass fuse oracle. Known follow-up: at the outer wasmtime canon-lift boundary, a memory-address handle from ht_new (table_base + 4) escapes into wasmtime's resource validation. Needs unwrap shim in component_wrap.rs to translate memory-address handles back to canonical-ABI indices when the fused module's exports cross the wrapper boundary.
…ports Three follow-ups to the path B refactor (commit e027bb1) that extend per-resource handle table routing to additional cases the original commit didn't cover: 1. Iterate ALL components in merger.rs:739, not just those with their own handle tables. A pure consumer (the runner in a 3-component chain) holds handles allocated by the re-exporter's ht_new and must drop them through that same handle table — its [resource-drop] imports also need redirection. 2. component_wrap.rs:1254 falls back to ANY component's handle table matching (iface, rn) when the importer's own component_idx doesn't have one. Mirrors the merger.rs change at the wrapper boundary so consumer-side ImportResolution::LocalResource entries pick up the re-exporter's ht_* export instead of falling through to canonical resource ops. 3. strip_dollar_suffix helper strips meld's $N dedup suffix from resource names before handle-table lookup. The dedup suffix distinguishes duplicate canonical-ABI imports from different components but the canonical resource name is the same, so it must be stripped before keying handle_tables. Status: 73/73 wit-bindgen suite still passes (0 regressions). The originally-failing trio (resource_floats, resource_with_lists, resource-import-and-export) still traps at runtime with 'unknown handle index 0x110004' — the bug now narrowed to wit-bindgen's intermediate code mixing ht_new-returned memory addresses with leaf's canonical handles in the inner-Box / outer-Box composition. Further work needs deeper instrumentation of the actual handle flow across the cabi-export shim. Tracking as continued follow-up.
…back Two refinements to the consumer-side handle-table routing introduced in c104b1b: 1. ImportResolution::LocalResource gains an is_definer flag set by the resolver: true when the import came from an [export]-prefixed module (the importer is the resource's definer / owner), false when the importer is a pure consumer of someone else's resource. The wrapper-side fallback only routes through another component's handle table when is_definer=false. Previously the fallback would incorrectly route a definer's own [resource-new] through a re-exporter's ht_new — causing the definer's canonical [resource-rep] to later return whatever was stored there (a small handle integer instead of the actual Box pointer) and the user code's deref to panic with 'misaligned pointer dereference: address must be a multiple of 0x8 but is 0x2'. 2. merger.rs:739 redirect adds re-exporter-self-import detection: when a non-[export] [resource-*] import comes from a component that ALSO owns a handle table for the same (interface, resource), that import is the inner-component (definer) view from the re- exporter's perspective and must use canonical resource ops, NOT the re-exporter's own ht. Previously a re-exporter's import-side helpers were accidentally routed through its own ht, writing to memory address small_int and corrupting heap state. Status: - 73-test wit-bindgen suite still passes (0 regressions). - resource_floats trio still traps at runtime with 'unknown handle index 0x110004' — the deeper architectural issue (memory-address handles from ht_new escaping into wasmtime canonical resource table at the wrapper boundary) remains. Tracked in issue #107.
Adds the infrastructure for marking individual (interface, resource) pairs
as opaque-rep — for re-exporter components built with the
pulseengine/wit-bindgen feat/opaque-rep-attribute fork that returns
u32 reps instead of boxed pointers. The flag is qualified
(--opaque-rep iface.resource, e.g. test:foo/bar.baz) and repeatable.
What's plumbed:
- meld-cli: --opaque-rep IFACE.RESOURCE arg, parsed to (String, String)
tuples by splitting on the LAST '.' so qualified WIT interface names
(which never contain '.') are preserved intact.
- FuserConfig.opaque_resources: Vec<(String, String)> field.
- Threaded through to Merger via with_opaque_resources(...) builder, and
to wrap_as_component (currently the wrapper-side parameter is unused
pending the right conditional strategy).
- All existing test FuserConfig initializers updated.
What's NOT yet doing anything (intentionally):
- Both conditional strategies tried in this spike were dead ends:
(a) per-component local_resource_types keying alone produces a NEW
'handle index used with the wrong type' trap because wasmtime's
type system rejects cross-component handle passing for
per-component-typed resources. Reverted.
(b) identity ht_* functions break opaque-rep storage semantics —
the rep stored at ht_new(rep) must be retrievable by a LATER
ht_rep(handle), but identity collapses these. Reverted.
- The flag currently has no behavioral effect; the architectural fix
for opaque-rep + meld fusion needs more design work.
Validation:
- Standard wit-bindgen suite: 73/73 (0 regressions).
- Full meld test suite: all 15 integration test files pass.
- Opaque oracle still traps with same shapes from issue #107.
The infrastructure is a clean foundation for future iteration on
conditional opaque-rep handling without re-doing the CLI/plumbing
work.
When intermediate has 'use test.{float}' (re-exporting leaf's test.float
as exports.float), wasmtime unifies them into one canonical resource
type. Path B previously refused to redirect [export]-prefixed (definer)
imports — leaf's [export]test:resource-floats/test [resource-rep]float
fell through to canon resource.rep, which never saw the memory-pointer
handles minted by intermediate's ht_new. Trap: 'unknown handle index'.
Fix: add a resource-name-only fallback in BOTH the merger redirect
(merger.rs:864) and the wrapper resolver (component_wrap.rs:1306) for
definer imports. When self-lookup misses, search any handle table
matching just the resource_name. In well-formed compositions there's
at most one re-exporter per logical resource, so the match is
unambiguous.
Status: 73/73 standard suite still passes (0 regressions). The trio
trap shape changes from 'unknown handle index 0x110004' (1st ht
entry) to 'unknown handle index 0x110010' (5th ht entry) — proving
more redirects are firing AND more handles are being allocated, but
something else still leaks to canon resource.*. Investigation ongoing.
…ame unification
The consumer-side LocalResource fallback was too aggressive: it blocked
redirect when the importing component owned a handle table for ANY
resource ending in the same name (e.g. intermediate's
'test:resource-floats/test [resource-rep]float' was blocked because
intermediate also has 'exports.float' ht — even though they're SEPARATE
imports unified at canon-type via 'use test.{float}').
Refine: check self-owns by SPECIFIC (component, iface, resource) tuple,
not by any-iface match. The resource-name-only alias fallback then
catches the unified consumer view and routes through the lone re-exporter
ht_drop / ht_rep / ht_new.
Also added an Instance::ResourceDrop alias-fallback at component_wrap.rs
(category C path) — for completeness, though resource_floats happens to
not hit that path. Keeps it symmetric with the LocalResource path.
Result on the trio:
- resource_floats: NOW PASSES (was 'unknown handle index 0x110004')
- resource_with_lists: still traps (wasm unreachable; was Option::unwrap)
- resource-import-and-export: still traps (wasm unreachable; was Option::unwrap)
Standard 73-test suite: 73/73, 0 regressions.
The remaining two trio failures are likely the same architectural issue
in a slightly different shape — investigation continuing in Phase 5.
The previous ht_drop just zeroed mem[handle] — the wit-bindgen-rust
Box that backs the rep was never freed AND the canonical-ABI lifetime
hook (`[dtor]<resource>` exported by the owning component) never
fired. Wit-bindgen's _resource_dtor uses Box::from_raw to drop the
Box, which runs user-defined Drop impls (including any Option::take
on inner-handle fields).
Fix: in allocate_handle_tables, look up the matching dtor export
`<iface>#[dtor]<rn>` whose function origin matches the ht-owning
component, and prepend a call to it in ht_drop before zeroing.
Implementation:
- Search merged.exports for entries whose name contains `#[dtor]<rn>`
- Filter by func origin component matching the ht owner
- If found, ht_drop emits: if (handle != 0) { call dtor(load mem[handle]); store 0 }
- If not found (no Box-backed dtor — e.g. opaque-rep), just zero the slot
- handle == 0 short-circuit prevents double-free if drop is called twice
Status:
- 73/73 standard suite still passes (0 regressions)
- resource_floats: still PASSES
- resource_with_lists, resource-import-and-export: still trap with
Option::unwrap() — but the dtor wiring is correct in principle and
needed for memory-correctness anyway. The remaining trap is the
cross-component handle aliasing problem (handles from different
components share one ht namespace; deeper architectural fix needed).
The previous `name.contains("#[dtor]<rn>")` lookup picked the FIRST
matching export regardless of which interface owned it. For fixtures
with the same resource name across multiple interfaces (e.g.
resource_floats has dtors for `exports#[dtor]float`,
`imports#[dtor]float`, and `test:resource-floats/test#[dtor]float`
under one component), the contains-match could wire the wrong dtor.
The origin-comp filter (`func.origin.0 == comp_idx`) doesn't
disambiguate when one component defines the same resource in
multiple interfaces (which is exactly the re-exporter pattern with
`use foo.{r}`). `find_map` short-circuits on the first match,
making this latent unless the export iteration order happens to
favor the wrong one.
Mythos slop-hunt discover stage flagged this as
'buggy-but-untested' — `resource_floats` happened to pick the right
dtor by iteration order, masking the bug. Tightening to exact match
`<iface>#[dtor]<rn>` (with optional `$N` dedup suffix) makes the
selection unambiguous.
Validation: 73/73 standard suite unchanged. resource_floats runtime
still passes. The fix is safe (more specific match) and removes the
latent bug.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Architectural progression on issue #107 (under epic #106 / issue #92):
HandleTableInfofromHashMap<usize, _>toHashMap<(usize, String, String), _>so multi-resource re-exporters route per-resource instead of per-component. (commits e027bb1, c104b1b, 254bfab)--opaque-repCLI flag for resources built withpulseengine/wit-bindgen feat/opaque-rep-attribute. Plumbed end-to-end; conditional behavior is currently a no-op (architectural design pending). (commit e166cb4)use-aliased resources where wasmtime unifies multiple WIT interfaces into one canonical type. Both definer-side and consumer-side branches now route to the unique re-exporter handle table. (commits 0035441, 73f95e6)ht_dropinvokes[dtor]before zeroing the slot — restores the canonical-ABI lifetime contract so wit-bindgen's Box-backed reps are properly dropped (and anyOption::takeon inner-handle fields fires at the right moment). (commit 9eb1229)Test status
resource_floatsruntimewasm unreachable)resource_with_listsruntimeresource-import-and-exportruntimeresource_floats_opaque(fork driver)resource_floats_opaque(meld)What this DOES NOT fix yet
Two of the trio fixtures still trap with
Option::unwrap() on None. Diagnosis (issue #107): cross-component handle namespace conflation. Multiple components share a single handle table when their resources unify viauseor shared interface — handles from different components address different memory layouts, and wit-bindgen's_ResourceRep<T>::val.as_ref().unwrap()reads garbage when handed a rep from the wrong component.The principled fix is per-component handle tables with bridging trampolines at cross-component hand-offs (Option A in #107). That's a separate branch (
feat/per-component-ht-bridging).Why open this as a draft now
Test plan
cargo test --release --test wit_bindgen_runtime— 73/73cargo run --release --bin meld -- fuse tests/wit_bindgen/fixtures/resource_floats.wasm -o /tmp/f.wasm --component && wasmtime --invoke='run()' /tmp/f.wasm— outputs()cargo run --release --bin meld -- fuse tests/wit_bindgen/fixtures/resource_with_lists.wasm -o /tmp/r.wasm --component && wasmtime --invoke='run()' /tmp/r.wasm— traps (documented)Related issues
Branch state
Worktree:
/Users/r/git/pulseengine/meld-conditional-typingonfeat/conditional-resource-typinghead9eb1229. Pushed to origin.