Push limit to leaf stage by default for DISTINCT / no-aggregate GROUP BY

[yashmayya]

For SELECT DISTINCT col ... LIMIT n and GROUP BY col ... LIMIT n without aggregate functions, the multi-stage engine currently ships every distinct group key from each server to the intermediate stage before applying the limit. This pushes the limit (and order-by-on-key) down to the leaf aggregate by default, so each server emits at most limit groups.

This is safe for the no-aggregate case: each leaf produces complete group keys (no partial aggregation), so leaf-level trimming is exact for ordered queries and a valid subset for unordered ones. Queries with aggregate functions are unchanged — they remain gated behind the existing is_enable_group_trim hint/config. Limited to a single group set, so ROLLUP / CUBE / GROUPING SETS are excluded. Opt out per query with /*+ aggOptions(is_enable_group_trim='false') */.

Note: for an unordered ... LIMIT (no ORDER BY), the specific rows returned may differ from before — this is already unspecified in SQL and was non-deterministic previously.

Covered by new planner plan tests (DISTINCT/GROUP BY + LIMIT, ORDER BY on key, HAVING, OFFSET, opt-out hint, and a multi-group-set negative case) and GroupByOptionsTest integration tests for paginated DISTINCT/GROUP BY.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 60.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.29%. Comparing base (c2561d4) to head (0d584fc).

Files with missing lines	Patch %	Lines
...el/rules/PinotAggregateExchangeNodeInsertRule.java	75.00%	0 Missing and 1 partial ⚠️
...t/calcite/rel/rules/PinotLogicalAggregateRule.java	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #18598      +/-   ##
============================================
+ Coverage     56.82%   64.29%   +7.47%     
- Complexity        7     1137    +1130     
============================================
  Files          2567     3335     +768     
  Lines        149066   206012   +56946     
  Branches      24103    32134    +8031     
============================================
+ Hits          84700   132453   +47753     
- Misses        57178    62906    +5728     
- Partials       7188    10653    +3465     

Flag	Coverage Δ
custom-integration1	100.00% <ø> (ø)
integration	100.00% <ø> (ø)
integration1	100.00% <ø> (ø)
integration2	0.00% <ø> (?)
java-21	64.29% <60.00%> (+7.47%)	⬆️
temurin	64.29% <60.00%> (+7.47%)	⬆️
unittests	64.29% <60.00%> (+7.47%)	⬆️
unittests1	56.78% <60.00%> (-0.03%)	⬇️
unittests2	36.82% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Jackie-Jiang]

I think we can push down the limit for the following scenarios:

• Distinct (no aggregates)
• Aggregates with order-by and order-by doesn't include aggregates

I don't follow why we cannot push down limit when there are multiple group keys

[yashmayya]

In the current PR we're pushing down limit for distinct and group by without aggregate (with or without order by). We're pushing down limit for the multiple group keys case too - the condition here is to exclude queries with multiple group sets (ROLLUP / CUBE / GROUPING SETS - probably not supported today, but if they are in the future, this push down won't be logically valid for those cases).