linq_fold: PR C — SourceAdapter + 4 decs planner migrations#2885
Merged
Conversation
Step 1 — Kernel: - SourceAdapter gains Decs : tuple<DecsBridgeShape?; string> variant - New adapter helpers: adapter_bind_name (it/decs_tup), adapter_element_type, adapter_wrap_source_loop (Array for-loop vs Decs for_each_archetype + build_decs_inner_for_pruned), adapter_wrap_invoke (Array source-arg-invoke vs Decs zero-arg-invoke + optional .to_sequence_move() outer wrap) - New decs_source predicate (extract_decs_bridge != null) Step 2 — Refactor 7 emit fns to read bindName from adapter and dispatch source-loop + invoke wrap via helpers (no behavior change on Array adapter): - emit_reverse_counter, emit_reverse_walk_overwrite_scalar, emit_reverse_buffer_inplace - emit_hashtable_dedup (take(N) case retains per-adapter inline: Array uses break, Decs uses for_each_archetype_find with bool return) - emit_streaming_min, emit_bounded_heap, emit_fused_prefilter Array-side regression suite all green: - test_linq_sorting (74), test_linq_fold (385), test_linq_fold_ast (228) - test_linq_fold_order_family (12), test_linq_fold_terminal_select (28) - test_linq_fold_theme45_quick_wins (32), test_linq_fold_theme8 (36) - test_linq_fold_loop_or_count (22), test_linq_fold_iterator_wrap (34) - test_linq_fold_collapse_chained_wheres (18), test_linq_fold_pattern_walker (16) - test_linq_fold_theme2/3/6/7 all green - mcp__daslang__lint: clean Next: Decs-arm dispatch in emit_loop_or_count_lane + 4 decs planner stubs + delete imperative decs bodies + per-archetype tests. See plan file: ~/.claude/plans/inherited-strolling-lollipop.md Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Step 3 — emit_loop_or_count_lane_decs:
Reconstructs flatten_linq-shaped calls array from captures (head + range
ops + term) and dispatches to existing emit_decs_* lane fns. Per-adapter
state-hoist shape stays (array binds source as invoke arg, decs zero-arg
with state above for_each_archetype) — D1 keeps the 4 array lane fns
untouched.
Step 4+5 — 4 decs planners now thin pattern-table stubs:
plan_decs_unroll → plan_loop_or_count_patterns + Decs adapter
plan_decs_order_family → plan_order_family_patterns + Decs adapter
(Row 4 buffer_helper_dispatch gated by array_source;
decs cascades to Row 3 fused_prefilter)
plan_decs_reverse → plan_reverse_patterns + Decs adapter
(backward-walk rows already array_source-gated)
plan_decs_distinct → plan_distinct_patterns + Decs adapter
(emit_hashtable_dedup carries per-adapter take(N)
branch — Decs for_each_archetype_find, Array break)
All 4 imperative bodies hard-deleted: -1173 LOC, +492 LOC = -681 LOC net.
Skip-into-tail preserved (emit_decs_reverse_skip_into_tail):
Decs `reverse |> take(N) |> to_array` fast path lifted into a dedicated
emit fn that emit_reverse_buffer_inplace pre-checks before the general
buffer path. Preserves PR #2834's 5.2× perf gain on multi-archetype
decs sources.
Test fixes (tests/linq/test_linq_from_decs.das):
6 splice-shape assertions updated to match unified naming:
decs_buf → order_buf (order family) / `buf (distinct)
decs_seen → order_seen (order family) / `seen (distinct)
decs_best → order_best, decs_taken → `taken, decs_acc → `acc
Verification:
- mcp__daslang__lint: clean
- test_linq_from_decs (198), test_linq_fold (385), test_linq_fold_ast (228)
- all themes 2/3/6/7/8 + pattern_walker + collapse_chained_wheres
- test_queries, test_queries_comprehensive (decs core)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Masterplan (daslib/linq_fold.md):
- Status row: PR C flipped to complete
- Kernel snippet: SourceAdapter widened with Decs : tuple<DecsBridgeShape?;string>
- Migration phases table: PR C row expanded with shipped scope
- PR C — shipped section: kernel changes, 7 emit fn refactors, decs-arm
dispatch (emit_loop_or_count_lane_decs), 4 stub migrations, ~970 LOC
deletion, fast-path preservation (count_archsize + reverse_skip_into_tail)
- Decision log: 8 new entries (D1–D6 plus 2 impl notes on per-adapter
take(N) inlining + calls reconstruction)
- Open questions: SourceAdapter method surface answered (4 helpers)
linq_fold_patterns.rst (Decs-source patterns section):
- Added note documenting PR C migration: plan_decs_* are now thin
pattern-table stubs reusing array-side rows + emit fns via Decs adapter
- Noted decs-specific fast paths preserved (count_archsize,
reverse_skip_into_tail)
- Noted Row 4 (buffer_helper_dispatch) gated to Array via array_source
- Noted D6 deferred (reverse |> distinct[_by] on decs)
Bench refresh deferred — pure refactor expected to be byte-identical or
strictly faster at the per-arm level. The unified emit fns produce the same
splice shapes the imperative bodies did (modulo cosmetic ID names like
decs_buf → order_buf), and the only logic change is the calls-array
reconstruction inside emit_loop_or_count_lane_decs which dispatches to
the same emit_decs_* lane fns unchanged. Will refresh post-merge if any
bench shows a regression.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Phase 3 of the linq_fold pattern-table refactor: widens SourceAdapter to support both Array and Decs sources, refactors shared emit archetypes to be adapter-driven, and migrates four plan_decs_* planners to thin pattern-table stubs reusing the array-side pattern rows. Docs and Decs AST-shape tests are updated to reflect the unified naming.
Changes:
- Add Decs support to
SourceAdapterand introduce adapter helpers to unify source-loop + invoke wrapping across Array/Decs. - Refactor shared emit archetypes to consume the adapter and migrate
plan_decs_reverse/distinct/order_family/unrollto pattern-table stubs. - Update docs and Decs splice-shape assertions to match unified emitted symbol names.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
daslib/linq_fold.das |
Implements SourceAdapter::Decs, adapter helpers, emit refactors, and decs planner stub migrations. |
tests/linq/test_linq_from_decs.das |
Updates AST-shape assertions for unified emitted variable naming on Decs paths. |
doc/source/reference/linq_fold_patterns.rst |
Documents the PR C refactor and how Decs now reuses array-side pattern rows via the adapter. |
daslib/linq_fold.md |
Updates the masterplan/status/decision log to reflect PR C’s delivered scope and design decisions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+2082
to
2084
| var retType : TypeDeclPtr | ||
| if (oc.selectLam != null) { | ||
| let outBufName = qn("order_proj_buf", at) |
16 tasks
borisbat
added a commit
that referenced
this pull request
May 26, 2026
…educer-spec table Continues the linq_fold pattern-table refactor (PRs A / B1 / B2 / C merged). PR D1 is the first half of PR D's masterplan scope: tackle the group_by family. Pure refactor — semantics unchanged, expected behavior identical. ## Changes **Bundled R1 follow-up to PR #2885:** drop dead `retType` assignments in emit_bounded_heap (3 LOC). Return type correctly inferred from inner `return <-` statements; `null` was already passed to adapter_wrap_invoke. **Partial GroupBy ↔ SourceAdapter reconciliation:** new `to_source_adapter(GroupBySourceAdapter) : SourceAdapter` projection maps array/decs/decs-join onto PR C's variant. `adapter_emit_source_loop` keeps decs-join inline shape (~40 LOC) but delegates array + plain-decs to `adapter_wrap_source_loop`. `adapter_finalize_emission` collapses to a one-line `adapter_wrap_invoke` call. Full unification deferred to PR D2. **`plan_group_by` + `plan_decs_group_by` migrated to pattern-table stubs:** 2 SplicePattern rows (group_by_array, group_by_decs) capture the tail-pop recognizer shape: head segments, optional upstream join (decs only), group_by_lazy, optional having_, select(groupproj), optional trailing where_, optional trailing order_by[_descending], optional count terminator. `Slot.arity` enforces args-length guards. New `order_by_family` alias (excludes bare order/order_descending — no key). New `decs_join_invariants` predicate enforces v1 join-shape limits (empty head, no having, no trailing_where when upstream_join captured). Shared `emit_group_by` reconstructs head calls from c.many["head"] (mirrors emit_loop_or_count_lane_decs precedent), builds the GroupBySourceAdapter per source-shape (Array / Decs / Decs-with-join), and delegates to plan_group_by_core unchanged. Both planner bodies (~165 LOC) hard-deleted. **`emit_reducer_branches` 13-arm if/elif → `reducer_emitters` data table:** table<string; ReducerEmitterFn> with 10 named `mk_reducer_*` fns. Shared `mk_strictly_preferred(workhorse, isMin, valExpr, cmpExpr)` helper collapses the 4-arm workhorse split between mk_reducer_min_max and mk_reducer_min_max_inner. `mk_*` naming (vs `emit_*`) marks these as sub-codegen building blocks, not pattern-table emit fns. ## Tests - All 624 tests across 13 group_by / theme / decs / fold-ast test files pass (test_linq_group_by 18, test_linq_from_decs 198, test_linq_fold_theme3_c2 14, test_linq_fold_theme3_decs_join_groupby 14, test_linq_fold_theme2 22, test_linq_fold_theme45 32, test_linq_fold_ast 228, test_linq_fold_walker 16, test_linq_fold_terminal_select 28, test_linq_fold_theme7 18, test_linq_fold_theme8 36, test_linq_fold_order_family 12). - Lint clean, compile clean. - INTERP + JIT bench matrix for 16 group_by + join_groupby benches refreshed — all deltas within run-to-run noise (~±1 ns INTERP, ~±0.3 ns JIT). ## Docs - Masterplan: PR D row split into D1 (complete) + D2 (deferred); PR D1 shipped scope detail; 7 new decision log entries; reducer-spec data table open question answered. - linq_fold_patterns.rst: group_by rows rewritten as "pattern `<name>` (sub-codegen `plan_group_by_core`) handles …". - benchmarks/sql/results.md: group_by rows refreshed against PR D1 branch. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 3 of the linq_fold pattern-table refactor (see
daslib/linq_fold.md— PRs A/B1/B2 already merged as #2878/#2881/#2883).Array | Decs. Four new adapter helpers (adapter_bind_name,adapter_element_type,adapter_wrap_source_loop,adapter_wrap_invoke) abstract the source-loop and invoke-wrap shape so emit fns work for both adapters.plan_decs_reverse/_distinct/_order_family/_unroll, ~970 LOC) replaced with thin pattern-table stubs that reuse the existing array-side rows.emit_loop_or_count_lane(newemit_loop_or_count_lane_decs) reconstructs a calls array from captures and routes to the existingemit_decs_*lane fns — D1: 1st-order adapter, lane fns untouched (state-hoist-above-for_each_archetypeshape stays per-adapter).emit_decs_count_archsize(barecount()) and PR linq_fold: trivial-let elision + reverse_take skip-into-tail — closes the m4 ladder #2834'sreverse |> take(N) |> to_arrayskip-into-tail (lifted into a dedicatedemit_decs_reverse_skip_into_tailhelper).plan_order_family_patterns(buffer_helper_dispatch) gated to Array viaarray_sourcepredicate; Decs cascades to Row 3 (fused_prefilter) which materializes — matches the imperative decs behavior.Net: −610 LOC of production code. Pure refactor — runtime semantics preserved across all 198 tests in
tests/linq/test_linq_from_decs.das. 6 splice-shape AST assertions updated to the unified naming (decs_buf→order_buf/`buf,decs_seen→order_seen/`seen, etc.).Deferred (D6):
reverse |> distinct[_by]on decs sources cascades to tier-2 — array R-2a row uses backward index walk (random access), decs has none. Future small PR.Masterplan + decision log +
linq_fold_patterns.rstrefreshed.Test plan
mcp__daslang__lint daslib/linq_fold.das— clean (verified)mcp__daslang__compile_check daslib/linq_fold.das— clean (verified)tests/linq/test_linq_from_decs.das(198 tests — covers all 4 migrated decs planners)tests/linq/test_linq_fold.das(385),test_linq_fold_ast.das(228)test_linq_fold_order_family(12),test_linq_fold_loop_or_count(22),test_linq_fold_terminal_select(28),test_linq_fold_iterator_wrap(34)test_linq_fold_pattern_walker(16),test_linq_fold_collapse_chained_wheres(18)tests/decs/test_queries,test_queries_comprehensive— green (parity for decs core)🤖 Generated with Claude Code