feature: HOF native closure dispatch (Phase 2 PR3)#387
Merged
Conversation
Phase 2 PR3 starts the HOF native dispatch migration. partition 2 previously round-tripped every per-element callback through the tree-bridge (OP_CALL_BUILTIN_TREE), paying a NanVal-to-Value cost per call and depending on the ACTIVE_AST_PROGRAM TLS being live. Now partition emits its own foreach loop using OP_CALL_DYN per element, mirroring the shape of the existing flt 2 native lift but threading two accumulator lists. The closure-aware OP_CALL_DYN from the Phase 2 PR1/PR2 work means capturing lambdas already work here without any further plumbing. The remaining bridge HOFs (srt 2, grp 2, uniqby 2, mapr 2, ct 2, rsrt 2) need dedicated finalizer opcodes for sort/group/dedup or short-circuit-result handling, and are deferred to follow-up PRs.
Nine regression tests cover partition 2 across the three Phase 2 shapes (non-capturing inline lambda, capturing inline lambda, named fn ref) plus text values, empty input, all-pass, all-fail, and the non-bool predicate error path. All nine run against --run-tree, --run-vm, and --run-cranelift so the engines stay in lockstep. Updates examples/partition.ilo to drop the now-obsolete engine-skip: jit annotation, and adds examples/partition-closure- native.ilo which demonstrates the new native closure path with single, double, and text-typed captures.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This was referenced May 18, 2026
danieljohnmorris
added a commit
that referenced
this pull request
May 18, 2026
Phase 2 PR3c completes the HOF native dispatch migration. The 2-arg
forms of srt, grp, and uniqby previously round-tripped every per-
element callback through the tree-bridge (OP_CALL_BUILTIN_TREE),
paying a NanVal-to-Value cost per call and depending on the
ACTIVE_AST_PROGRAM TLS being live.
Each now emits its own foreach loop using OP_CALL_DYN per element to
fill two parallel scratch lists (keys + values), then hands them to a
dedicated finalizer opcode:
OP_SRT_BY_KEY (179) — sort vals by key; mixed-type-key fallback
folds to Ordering::Equal, matching the tree
walker exactly.
OP_GRP_BY_KEY (180) — bucket vals into a Map<MapKey, List<value>>;
numeric keys floor to i64; non-finite numbers
and non-t/n/b keys raise a typed runtime err.
OP_UNIQ_BY_KEY (181) — dedup using HashSet<String> with type-prefixed
keys (t:/n:/b:) so values from disjoint
domains never alias, matching the tree
scheme.
Each finalizer ships a free fn that the VM dispatch arm and Cranelift
JIT/AOT helpers share, so the two paths can't drift. The closure-
aware OP_CALL_DYN from Phase 2 PR1/PR2 means capturing lambdas work
here without further plumbing.
A single emit_hof_keyed_finalize helper covers all three HOFs at the
compiler layer; the only per-HOF variation is which finalizer opcode
to emit.
mapr already shipped its native lift in PR3b (#389), so this completes
the Phase 2 HOF migration that started with partition in PR3 (#387).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 2 PR3 of 5 in the HOF native closure dispatch sweep. Migrates the first of the 2-arg HOFs off the tree-bridge so closure callbacks dispatch natively via OP_CALL_DYN per element instead of re-entering the tree interpreter on every call.
Pre-PR3 the OP_CALL_BUILTIN_TREE bridge routed every callback through the tree-walker. That cost a NanVal-to-Value round-trip per call and depended on ACTIVE_AST_PROGRAM TLS being live, which has been a source of cross-engine bugs. Post-PR3 partition does its own foreach loop and uses the closure-aware OP_CALL_DYN from PR1/PR2 (#384/#385).
What landed
[pass, fail]result. Capturing lambdas work for free via OP_MAKE_CLOSURE + closure-aware OP_CALL_DYN. Dropped fromis_tree_bridge_eligible.What's still on the bridge
The other HOFs the PR3 plan called out (srt 2, grp 2, uniqby 2, mapr 2, plus neighbours ct 2 and rsrt 2) each need a dedicated finalizer opcode for the post-callback assembly step:
Rather than ship a half-finished foundation for those, this PR locks in the easiest migration (partition's bool-predicate shape, identical to the existing flt 2 lift) and leaves the dedicated finalizer-opcode work for follow-up PRs. Time-boxed to keep the surface area reviewable.
Bench
The Phase 2 PR3 bio canonical bench (
flt all-h (window k seqs)) runs unchanged: flt 2 already had a native lift pre-PR3, so the bench numbers don't move on this partition-only migration. Bench rerun will be meaningful once srt/grp/uniqby land in a follow-up PR.What's in the diff
Per-commit:
vm: lift partition 2 off tree-bridge to OP_CALL_DYN(src/vm/mod.rs)test: cross-engine coverage for partition native dispatch(tests + examples)Test plan
cargo build --release --features craneliftcleancargo test --release --features cranelift(6852 pass, 0 fail; up 9 from 6843)Follow-ups