linq_fold: plan_decs_eager_bridge — count + to_array splice past from_decs(_template)

[borisbat]

Summary

Phase 4 sub-PR 4a of the LINQ-to-DECS plan. New plan_decs_eager_bridge planner that recognizes the post-expansion eager-bridge shape (produced by both from_decs(...) and from_decs_template(type<Foo>)) and splices past it for count and to_array terminators — bypassing array<tuple> materialization.

[decs_template(prefix="particle_")]
struct Particle { val : int }

// Before: eager bridge allocates array<tuple<val:int>>, walks every entity,
// returns to_sequence, then count walks the iterator.
// After: arch.size shortcut — sums per-archetype entity counts, no
// allocation, no inner for-loop.
let n = _fold(from_decs_template(type<Particle>).count())

How it works

_fold sees the FULLY-EXPANDED AST — both from_decs_template AND query have lowered before the planner runs. What _fold receives:

invoke($() : iterator<tuple<val:int>> {
    var res : array<tuple<val:int>>
    for_each_archetype(req_hash, erq_factory, $(arch) {
        for (val in get_ro(arch, "particle_val", type<int>)) {
            res |> push(tuple(val))
        }
    })
    return res.to_sequence()
})

The planner pattern-matches this shape (no marker annotation needed — the shape itself is deterministic and unique to FromDecs(Template)Macro). When matched + a supported terminator is found:

count splice (with arch.size shortcut, since no chain ops):

invoke($() : int {
    var acc = 0
    for_each_archetype(req_hash, erq_factory, $(arch) {
        acc += arch.size
    })
    return acc
})

to_array splice (skip the to_sequence iterator roundtrip):

invoke($() : array<tuple<val:int>> {
    var buf : array<tuple<val:int>>
    for_each_archetype(req_hash, erq_factory, $(arch) {
        for (val in get_ro(arch, "particle_val", type<int>)) {
            buf |> push(tuple(val))
        }
    })
    return <- buf
})

(req_hash and erq_factory are the same ExprConstUInt64 + ExprAddr from the original bridge, reused via clone_expression — no rebuild of EcsRequest needed.)

Architecture notes

• No decs_boost changes. The shape is the marker; pure pattern match in the planner. linq_fold stays decs-unaware at the dependency level.
• Splice fires on BOTH from_decs($(...){}) AND from_decs_template(type<Foo>) — same post-expansion shape.
• Cascade slot: between plan_group_by and plan_zip (bridge shape is more specific than zip's generic source).
• Safe degradation everywhere. Any shape mismatch returns null; tier-2 (eager bridge) runs unchanged. No way to break existing behavior.

Out of scope for 4a (lands in follow-ups)

• 4b: accumulator (sum / min / max / average / long_count) + early-exit (first / first_or_default / any / all / contains / take) terminators
• 4c: chain-op fusion (where_ / select / take / skip / take_while / skip_while between source and terminator)
• 4d: state-table terminators (distinct / group_by / reverse / order_by) — hoist state above outer

Files

• daslib/linq_fold.das (+139) — new plan_decs_eager_bridge planner + extract_eager_bridge + emit_decs_count_splice + emit_decs_to_array_splice + get_call_short_name helper
• tests/linq/test_linq_from_decs.das (+158) — 4 functional parity tests + 3 AST-shape gates
• benchmarks/sql/LINQ_TO_DECS.md (+204) — design doc updates: Phase 1-3 status, spike findings (macro expansion order, marker round-trip, for_each_archetype_find block-return), Phase 4 sub-PR breakdown

Test plan

• mcp__daslang__lint — clean on touched files
• mcp__daslang__format_file — clean
• mcp__daslang__run_test tests/linq/test_linq_from_decs.das — 14/14 pass (7 existing + 7 new splice tests)
• Interpret sweep tests/linq — 1145/1145 green
• AOT regen + rebuild + sweep tests/linq — 1145/1145 green
• Decs regression tests/decs — 239/239 green
• CI

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR adds a new LINQ _fold planner optimization that recognizes the expanded “DECS eager-bridge” shape emitted by from_decs(...) / from_decs_template(type<...>) and splices through it for count and to_array, avoiding intermediate array<tuple> materialization and iterator round-trips.

Changes:

• Add plan_decs_eager_bridge to daslib/linq_fold.das, including eager-bridge extraction and specialized emitters for count and to_array.
• Add functional parity tests plus AST-shape “splice fired” gates for the new planner behavior.
• Update LINQ→DECS design notes in benchmarks/sql/LINQ_TO_DECS.md.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
daslib/linq_fold.das	Introduces the eager-bridge recognizer and emits optimized splices for count/to_array, wired into the _fold planner cascade.
tests/linq/test_linq_from_decs.das	Adds new splice parity tests and AST-shape gates intended to ensure the optimization triggers (and cascades when unsupported).
benchmarks/sql/LINQ_TO_DECS.md	Updates the Phase 4 design writeup/status, but currently describes a marker-based approach that conflicts with this PR’s implementation.

Comments suppressed due to low confidence (2)

tests/linq/test_linq_from_decs.das:252

• The AST gate test silently returns when find_module_function_via_rtti fails, which can make the test pass without asserting anything (e.g., when RTTI is disabled). Please assert func != null (e.g., t |> success(func != null, ...)) before early-returning so this test can’t become a no-op.

        var func = find_module_function_via_rtti(compiling_module(), @@target_splice_to_array_template_fold)
        if (func == null) return
        var body_expr : ExpressionPtr
        let r = qmatch_function(func) $() {
            return <- $e(body_expr)
        }

tests/linq/test_linq_from_decs.das:269

• The AST gate test silently returns when find_module_function_via_rtti fails, which can make the test pass without asserting anything (e.g., when RTTI is disabled). Please assert func != null (e.g., t |> success(func != null, ...)) before early-returning so this test can’t become a no-op.

        var func = find_module_function_via_rtti(compiling_module(), @@target_splice_cascade_select_count_fold)
        if (func == null) return
        var body_expr : ExpressionPtr
        let r = qmatch_function(func) $() {
            return <- $e(body_expr)
        }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

benchmarks/sql/LINQ_TO_DECS.md:14

• In the status table, piece 4 is described as “plan_from_decs_template splice + Stage B marker”, but later in this doc Phase 4a is explicitly “Option C (chosen)… pure pattern-match on the post-expansion eager-bridge AST (no marker)”. Please update this row to match the current Phase 4 approach (or clearly label the marker plan as a superseded/alternative path).

| 3. `from_decs_template(type<Foo>)` macro — eager bridge (Stage A) | 🟡 in flight | #2745 |
| 4. `plan_from_decs_template` splice in `linq_fold` + Stage B marker (the real payoff) | ⬜ next | — |
| 5. sqlite_linq N-ary zip recognition | ⬜ deferred (no concrete use case) | — |

benchmarks/sql/LINQ_TO_DECS.md:467

• The “Updated Phase 4 plan (post-validation)” section proposes adding a from_decs_template_meta annotation and a plan_from_decs_template planner, which contradicts the earlier “Decision: Option C (chosen, implemented in PR #2748): Pure pattern-match…” and the implementation in this PR (no annotation, no decs_boost changes). This section should be removed, reworded as a rejected/superseded spike note, or updated to reflect the Option C implementation so the design doc remains internally consistent.

## Updated Phase 4 plan (post-validation)

The work is now:

1. **Add `[block_macro(name="from_decs_template_meta")]` to `daslib/decs_boost.das`.** Class extends `AstBlockAnnotation`. `apply` is a no-op (return `true`). Just a tag.
2. **Modify FromDecsTemplateMacro** to attach the tag annotation to the lambda body block as part of the eager-bridge emission. Annotation arg: `prefix = "<resolved prefix>"`. No other behavior change.
3. **Add `plan_from_decs_template` planner** in `daslib/linq_fold.das`. Slot between `plan_group_by` and `plan_zip`. Walks `flatten_linq(expr)` for ExprInvoke whose body block has `from_decs_template_meta` annotation. When found:

[borisbat]

Superseded by #2750. The new PR uses a cleaner architecture (named-tuple bind + fold_linq_cond peel) that covers the same case plus chain-aware 0 0///. The terminator from this PR is deferred to a Slice 3 follow-up on the same architecture.

[borisbat]

Replacement for the earlier mangled close-comment (shell ate backticks):

Superseded by #2750. The new PR uses a cleaner architecture — named-tuple bind + fold_linq_cond peel instead of renameVariable on the cloned for-body — that covers the same count case (now via arch.size shortcut) plus chain-aware sum / long_count / _where / _select. The to_array terminator from this PR is deferred to a Slice 3 follow-up on the same architecture.