Skip to content

linq_fold: PR D1 — group_by adapter reconciliation + 2 migrations + reducer-spec table#2886

Merged
borisbat merged 1 commit into
masterfrom
bbatkin/linq-fold-pattern-table-prd1
May 26, 2026
Merged

linq_fold: PR D1 — group_by adapter reconciliation + 2 migrations + reducer-spec table#2886
borisbat merged 1 commit into
masterfrom
bbatkin/linq-fold-pattern-table-prd1

Conversation

@borisbat
Copy link
Copy Markdown
Collaborator

Summary

Continues the linq_fold pattern-table refactor (PRs A / B1 / B2 / C merged). PR D1 is the first half of PR D's masterplan scope: tackle the group_by family. Pure refactor — semantics unchanged.

  • Bundled R1 follow-up to linq_fold: PR C — SourceAdapter + 4 decs planner migrations #2885: drop dead retType in emit_bounded_heap (3 LOC). Return type was already being inferred; the assignment was dead.
  • Partial GroupBy ↔ SourceAdapter reconciliation: new to_source_adapter(GroupBySourceAdapter) : SourceAdapter projection. adapter_emit_source_loop keeps decs-join inline (~40 LOC, no SourceAdapter variant for it yet) but delegates array + plain-decs to PR C's adapter_wrap_source_loop. adapter_finalize_emission collapses to a one-liner over adapter_wrap_invoke. Full unification deferred to PR D2 (when plan_zip / plan_decs_join migrate and Zip/DecsJoin variants join).
  • plan_group_by + plan_decs_group_by → pattern-table stubs: 2 SplicePattern rows (group_by_array, group_by_decs) + 1 shared emit_group_by delegating to plan_group_by_core (unchanged sub-codegen). Slot.arity enforces the args-length guards (count=1, join=5, group_by_lazy=2, order_by[descending]=2, where=2). New order_by_family alias excludes bare order / order_descending (no key arg). New decs_join_invariants predicate enforces v1 join-shape limits (empty head, no having, no trailing_where when upstream_join captured). ~165 LOC of imperative tail-pop hard-deleted across both planners.
  • emit_reducer_branches 13-arm if/elif → reducer_emitters data table: table<string; ReducerEmitterFn> lookup with 10 named mk_reducer_* fns. Shared mk_strictly_preferred(workhorse, isMin, valExpr, cmpExpr) helper collapses the 4-arm workhorse split (workhorse × isMin) shared between mk_reducer_min_max and mk_reducer_min_max_inner. mk_* naming marks these as sub-codegen building blocks, not pattern-table emit fns.

Architectural decisions

  • D1-A — Partial reconciliation only. GroupBySourceAdapter stays load-bearing (per-mode field bags diverge; plan_group_by_core reads initialBindName / initialElemType directly per mode). Projection unlocks helper reuse without restructuring.
  • D1-B — decs-join → Decs(null, "djoin_jres") is safe at finalize: adapter_wrap_invoke's Decs branch reads only the variant tag, never the tuple fields. Array path's invoke shifts untyped→typed; plan_group_by_core unconditionally sets retType so the inference shift is a no-op.
  • D1-C — Decs+join fits as a SINGLE row with optional upstream_join slot (not 2 separate rows). Predicate-gated invariants moved from emit-fn-land to slot/predicate-land via Slot.arity + decs_join_invariants.
  • D1-D — Reducer table LOC delta is ~+25 net (helper savings less than the agent's ~30 estimate); win is uniformity + lint-ability + future extensibility, not bytes.

Full decision log in daslib/linq_fold.md (7 new entries: D1-A through D1-E + 2 impl notes).

Out of scope (deferred to PR D2)

  • plan_zip (352 LOC) + plan_decs_join (189 LOC) migrations
  • SourceAdapter widening to Array | Decs | DecsJoin | Zip
  • Full GroupBySourceAdapter unification (once DecsJoin variant exists)
  • reverse |> distinct[_by] on decs sources (D6 from PR C plan)

Files

Test plan

  • mcp__daslang__compile_check daslib/linq_fold.das — clean
  • mcp__daslang__lint daslib/linq_fold.das — clean
  • tests/linq/test_linq_group_by.das — 18 passed
  • tests/linq/test_linq_fold_theme3_c2_group_by_order_by.das — 14 passed
  • tests/linq/test_linq_fold_theme3_decs_join_groupby.das — 14 passed
  • tests/linq/test_linq_fold_theme2_trailing_where.das — 22 passed (HAVING shapes)
  • tests/linq/test_linq_fold_theme45_quick_wins.das — 32 passed
  • tests/linq/test_linq_from_decs.das — 198 passed
  • tests/linq/test_linq_fold_ast.das — 228 passed
  • tests/linq/test_linq_fold_pattern_walker.das — 16 passed
  • tests/linq/test_linq_fold_terminal_select.das — 28 passed
  • tests/linq/test_linq_fold_theme7_chained_select.das — 18 passed
  • tests/linq/test_linq_fold_theme8_fusion_arms.das — 36 passed
  • 624 tests pass across 13 affected files
  • INTERP + JIT bench matrix for 16 group_by + join_groupby benches refreshed — all deltas within run-to-run noise (~±1 ns INTERP, ~±0.3 ns JIT)
  • Full CI matrix on push (Linux / macOS / Windows, INTERP + AOT + JIT, modulo the 27 documented master JIT-quote regressions)

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings May 26, 2026 16:59
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR continues the linq_fold pattern-table refactor by migrating the group_by planner family onto pattern-table rows, partially reconciling GroupBySourceAdapter with the newer SourceAdapter helpers, and converting reducer emission dispatch into a data-table lookup. It also updates the corresponding documentation and refreshes benchmark result matrices for the affected group-by benchmarks.

Changes:

  • Migrate plan_group_by / plan_decs_group_by to pattern-table stubs backed by shared emit_group_byplan_group_by_core.
  • Introduce to_source_adapter(GroupBySourceAdapter) and reuse adapter_wrap_source_loop / adapter_wrap_invoke for non-join group-by sources.
  • Replace emit_reducer_branches if/elif chain with reducer_emitters table + mk_reducer_* helpers; update docs/bench outputs accordingly.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
daslib/linq_fold.das Core refactor: group_by patterns/emitter, adapter projection, reducer emitter table, and minor cleanup in bounded-heap emission.
daslib/linq_fold.md Masterplan/status + decision log updates documenting the D1 refactor decisions and scope split (D1/D2).
doc/source/reference/linq_fold_patterns.rst Updates pattern documentation to reflect new group_by pattern rows and reducer dispatch mechanism.
benchmarks/sql/results.md Refreshes benchmark matrix rows for group_by/join_groupby and updates the generation metadata.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread daslib/linq_fold.das
…educer-spec table

Continues the linq_fold pattern-table refactor (PRs A / B1 / B2 / C merged).
PR D1 is the first half of PR D's masterplan scope: tackle the group_by family.
Pure refactor — semantics unchanged, expected behavior identical.

## Changes

**Bundled R1 follow-up to PR #2885:** drop dead `retType` assignments in
emit_bounded_heap (3 LOC). Return type correctly inferred from inner
`return <-` statements; `null` was already passed to adapter_wrap_invoke.

**Partial GroupBy ↔ SourceAdapter reconciliation:** new
`to_source_adapter(GroupBySourceAdapter) : SourceAdapter` projection maps
array/decs/decs-join onto PR C's variant. `adapter_emit_source_loop` keeps
decs-join inline shape (~40 LOC) but delegates array + plain-decs to
`adapter_wrap_source_loop`. `adapter_finalize_emission` collapses to a
one-line `adapter_wrap_invoke` call. Full unification deferred to PR D2.

**`plan_group_by` + `plan_decs_group_by` migrated to pattern-table stubs:**
2 SplicePattern rows (group_by_array, group_by_decs) capture the tail-pop
recognizer shape: head segments, optional upstream join (decs only),
group_by_lazy, optional having_, select(groupproj), optional trailing
where_, optional trailing order_by[_descending], optional count terminator.
`Slot.arity` enforces args-length guards. New `order_by_family` alias
(excludes bare order/order_descending — no key). New `decs_join_invariants`
predicate enforces v1 join-shape limits (empty head, no having, no
trailing_where when upstream_join captured).

Shared `emit_group_by` reconstructs head calls from c.many["head"]
(mirrors emit_loop_or_count_lane_decs precedent), builds the
GroupBySourceAdapter per source-shape (Array / Decs / Decs-with-join),
and delegates to plan_group_by_core unchanged. Both planner bodies
(~165 LOC) hard-deleted.

**`emit_reducer_branches` 13-arm if/elif → `reducer_emitters` data table:**
table<string; ReducerEmitterFn> with 10 named `mk_reducer_*` fns. Shared
`mk_strictly_preferred(workhorse, isMin, valExpr, cmpExpr)` helper collapses
the 4-arm workhorse split between mk_reducer_min_max and
mk_reducer_min_max_inner. `mk_*` naming (vs `emit_*`) marks these as
sub-codegen building blocks, not pattern-table emit fns.

## Tests

- All 624 tests across 13 group_by / theme / decs / fold-ast test files pass
  (test_linq_group_by 18, test_linq_from_decs 198, test_linq_fold_theme3_c2
  14, test_linq_fold_theme3_decs_join_groupby 14, test_linq_fold_theme2 22,
  test_linq_fold_theme45 32, test_linq_fold_ast 228, test_linq_fold_walker
  16, test_linq_fold_terminal_select 28, test_linq_fold_theme7 18,
  test_linq_fold_theme8 36, test_linq_fold_order_family 12).
- Lint clean, compile clean.
- INTERP + JIT bench matrix for 16 group_by + join_groupby benches refreshed
  — all deltas within run-to-run noise (~±1 ns INTERP, ~±0.3 ns JIT).

## Docs

- Masterplan: PR D row split into D1 (complete) + D2 (deferred); PR D1
  shipped scope detail; 7 new decision log entries; reducer-spec data
  table open question answered.
- linq_fold_patterns.rst: group_by rows rewritten as
  "pattern `<name>` (sub-codegen `plan_group_by_core`) handles …".
- benchmarks/sql/results.md: group_by rows refreshed against PR D1 branch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@borisbat borisbat force-pushed the bbatkin/linq-fold-pattern-table-prd1 branch from 7042ee9 to c601219 Compare May 26, 2026 17:20
@borisbat borisbat requested a review from Copilot May 26, 2026 17:21
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

@borisbat borisbat merged commit 7874386 into master May 26, 2026
32 checks passed
@borisbat borisbat deleted the bbatkin/linq-fold-pattern-table-prd1 branch May 30, 2026 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants