feat(asap-aware-mapping): add workload-aware fine-to-coarse rollups

[zzylol]

Summary

Closes #254. Part of #33.

This PR adds workload-aware group-by-lattice roll-up reuse. When two aggregates compute a compatible measure over identical input IR and one grouping is strictly finer than the other, the planner can derive the coarse result by re-aggregating the fine result instead of independently aggregating the raw input again.

The implementation includes both pieces required for the feature to be reachable:

• RollupStrategy: validates a fine/coarse pair and constructs the rewrite.
• Workload search integration: after CSE and target discovery, both search_workload and search_workload_with build the strategy from all discovered aggregate siblings and include its candidates automatically.

PR #259, which introduced ReplacementStrategy and workload search, is already merged; this PR is no longer stacked on an unmerged dependency.

Example workload

Both queries are considered together:

-- Q_fine
SELECT job, region, SUM(bytes)
FROM metrics
GROUP BY job, region;

-- Q_coarse
SELECT job, SUM(bytes)
FROM metrics
GROUP BY job;

Names used below:

• S: input Scan(metrics)
• F: fine aggregate grouped by (job, region)
• C: original independent coarse aggregate grouped by job
• R: coarse roll-up that consumes F

Core detection algorithm

1. Run CSE over all workload roots.
2. Discover every reachable QueryExpr target.
3. Collect all Aggregate nodes as the RollupStrategy sibling set.
4. For each aggregate target C, compare candidate finer aggregates F:
   a. both are single-measure Reduction::Reduce aggregates with no HAVING;
   b. their child IR is pointer-identical or structurally equal;
   c. their aggregate intents and source columns are compatible;
   d. coarse_by is a strict set subset of fine_by;
   e. F has a provable unique output key;
   f. a sound roll-up combinator exists.
5. Remap coarse group-key ColumnIds from S's schema into F's output schema.
6. Emit Rewrite(R), where R consumes F instead of S.

For aggregates sharing one input, pairwise matching costs approximately O(k² × g), where k is the number of sibling aggregates and g is the number of grouping keys.

Input identity and column safety

A fast Rc::ptr_eq check handles children already shared by CSE. Full QueryExpr::PartialEq is the fallback for independently lowered but identical inputs, including identical schemas. This fallback is necessary because CSE conservatively does not pointer-share a scan without a declared unique key.

ColumnId values are compared only after child equality proves both aggregates index the same input shape. Structurally different inputs or schemas are rejected.

Strict grouping relation

The comparison is set-based:

coarse_by ⊂ fine_by

Equality is not a roll-up, unrelated key sets are rejected, and duplicate keys cannot fake a strict superset.

Supported combinators

Fine measure	Roll-up measure	Status
Sum(x)	Sum(F.sum_x)	Supported
Min(x)	Min(F.min_x)	Supported
Max(x)	Max(F.max_x)	Supported
Exact Count	Sum(F.count)	Supported
Approximate Count	—	Conservatively rejected in this PR
Avg, Rate, and unsupported intents	—	Rejected

Exact count uses Sum, not another Count: counting rows in F would count fine groups, while summing their exact counts recovers the original coarse row count.

Approximate coarse queries are not inherently ineligible. They require composing finer error bounds or merging compatible sketch state and proving that the result satisfies the coarse query's own AccuracyTarget. Equal finer/coarse epsilon values alone are not sufficient. This follow-up is tracked by #278.

Strategy candidates for both queries

Before this PR:

Q_fine candidates:
  [independent Summary(F over S)]

Q_coarse candidates:
  [independent Summary(C over S)]

After this PR:

Q_fine candidates:
  existing strategies: [independent Summary(F over S)]
  RollupStrategy(F):   []
    # No strictly finer sibling exists.

Q_coarse candidates:
  existing strategies: [independent Summary(C over S)]
  RollupStrategy(C):   [Rewrite(R over F)]

Combined candidate space:

Q_fine:   [independent Summary(F over S)]
Q_coarse: [independent Summary(C over S), roll-up Rewrite(R over F)]

RollupStrategy proposes the alternative; it does not choose it. Cost/search or a downstream planner decides whether to use the independent candidate or the roll-up candidate.

Complete two-query workload DAG

Every arrow below represents data flow, from producer to consumer/result.

Before selecting the roll-up

flowchart LR
    S["S: Scan(metrics)"] --> F["F: fine aggregate<br/>GROUP BY job, region<br/>SUM(bytes)"]
    S --> C["C: independent coarse aggregate<br/>GROUP BY job<br/>SUM(bytes)"]
    F --> QF["Q_fine result"]
    C --> QC["Q_coarse result"]

    classDef result fill:#eef,stroke:#668;
    classDef aggregate fill:#efe,stroke:#484;
    classDef scan fill:#fee,stroke:#844;
    class QF,QC result;
    class F,C aggregate;
    class S scan;

Loading

S -> F -> Q_fine
S -> C -> Q_coarse

S feeds two independent aggregates.

After selecting the roll-up

flowchart LR
    S["S: Scan(metrics)"] --> F["F: fine aggregate<br/>GROUP BY job, region<br/>SUM(bytes)"]
    F --> QF["Q_fine result"]
    F --> R["R: coarse roll-up<br/>GROUP BY job<br/>SUM(F.sum_bytes)"]
    R --> QC["Q_coarse result"]

    classDef result fill:#eef,stroke:#668;
    classDef aggregate fill:#efe,stroke:#484;
    classDef scan fill:#fee,stroke:#844;
    class QF,QC result;
    class F,R aggregate;
    class S scan;

Loading

S -> F -> Q_fine
     F -> R -> Q_coarse

F appears once and becomes the shared producer. The original independent coarse node C is removed; its S -> C flow is replaced by F -> R.

This is the selected pre-ASAP QueryExpr DAG. Materializing a final post-ASAP or physical plan remains a downstream decision.

Implementation details

• RollupStrategy owns cloned Rc handles for its workload sibling set.
• is_legal_rollup_source remains a standalone predicate for reuse by other strategies.
• Coarse grouping columns are repositioned into the fine aggregate's by ++ measures output schema.
• Rewrites preserve the original coarse aggregate's explicit output names.
• without(...), Reduction::PerEntity, multi-measure aggregates, and aggregates with HAVING are rejected.

Tests

• Legality: strict subset, equality, unrelated groupings, duplicate keys, without(...), intent mismatch, unsupported combinators, and missing fine unique key.
• Construction: Sum, exact Count -> Sum, column remapping, shared fine child, and output-name preservation.
• Accuracy: approximate Count is rejected both directly and through workload search.
• Input matching: pointer-shared and structurally identical non-aliased inputs.
• Integration: two independently built workload roots pass through default search and produce the F -> R roll-up candidate.

Validation

• cargo test -p asap-aware-mapping: 117 passed.
• cargo check --workspace: passed.
• cargo clippy -p asap-aware-mapping --all-targets --all-features -- -D warnings: passed.
• GitHub format-and-lint and test checks: passed.

[milindsrivastava1997]

Didn't understand the example in the PR. Is that the output of both queries? A single query? Is it just showing the strategy replacement sub dag?

[zzylol]

Addressed the roll-up review findings in a78cc7d:

• workload search now discovers post-CSE aggregate siblings and automatically invokes an owning RollupStrategy from both search_workload and search_workload_with;
• Count -> Sum is restricted to AccuracyTarget::Exact so approximate error contracts are not discarded;
• identical inputs may match by Rc identity or full QueryExpr equality, allowing independently lowered scans without declared unique keys while still requiring identical schemas;
• strict group-by containment now compares sets, so duplicate keys cannot fake a finer grouping;
• added two-query end-to-end search tests plus negative approximate-count and duplicate-key coverage.

Local validation: 117 asap-aware-mapping tests pass, workspace check passes, and package clippy passes with -D warnings. GitHub CI is running.

[zzylol]

Opened #278 to track accuracy-aware approximate Count roll-ups.

PR #262 now notes that its exact-only restriction is conservative: approximate coarse queries may become legal when finer error composition or mergeable sketch state can be proven to satisfy the coarse query's own AccuracyTarget. Equal epsilon values alone are not sufficient. The follow-up covers preserving the coarse target, unknown fanout, epsilon/delta composition, and positive/negative legality tests.