Describe the bug
LimitPushdown can treat an operator-level fetch on a multi-partition plan as
if it satisfied a global LIMIT.
For operators that do not combine partitions, ExecutionPlan::fetch() limits
each output partition independently. Removing the remaining global enforcement
can therefore return up to fetch * partition_count rows. Related state
transitions can also place GlobalLimitExec directly above multiple partitions
for OFFSET-only plans or combine a per-partition LocalLimitExec fetch with a
pending global fetch.
To Reproduce
Optimize a physical plan shaped like:
GlobalLimitExec: skip=0, fetch=5
TestScan: partitions=2, supports_fetch=true
The current rule produces:
TestScan: partitions=2, fetch=5
Each partition may emit five rows, so the plan can return ten rows even though
the query has a global limit of five.
The same scope mismatch appears when the child already has a smaller
per-partition fetch, with OFFSET-only, and when a pending global limit reaches a
multi-partition LocalLimitExec.
Expected behavior
A fetch on a multi-partition operator should remain only an early-stop hint.
LimitPushdown should retain or create a single-partition enforcement boundary
(CoalescePartitionsExec or SortPreservingMergeExec) before considering the
global requirement satisfied. Global and per-partition local fetches should not
be combined before that boundary.
Additional context
The standard physical optimizer pipeline often inserts a partition-combining
boundary before GlobalLimitExec, which can hide this problem. It is observable
when optimizing physical extension/distributed plan shapes and when the rule
itself removes or moves limit nodes. The fix can preserve per-partition fetches
for early termination while retaining the global cap.
Describe the bug
LimitPushdowncan treat an operator-levelfetchon a multi-partition plan asif it satisfied a global
LIMIT.For operators that do not combine partitions,
ExecutionPlan::fetch()limitseach output partition independently. Removing the remaining global enforcement
can therefore return up to
fetch * partition_countrows. Related statetransitions can also place
GlobalLimitExecdirectly above multiple partitionsfor OFFSET-only plans or combine a per-partition
LocalLimitExecfetch with apending global fetch.
To Reproduce
Optimize a physical plan shaped like:
The current rule produces:
Each partition may emit five rows, so the plan can return ten rows even though
the query has a global limit of five.
The same scope mismatch appears when the child already has a smaller
per-partition fetch, with OFFSET-only, and when a pending global limit reaches a
multi-partition
LocalLimitExec.Expected behavior
A fetch on a multi-partition operator should remain only an early-stop hint.
LimitPushdownshould retain or create a single-partition enforcement boundary(
CoalescePartitionsExecorSortPreservingMergeExec) before considering theglobal requirement satisfied. Global and per-partition local fetches should not
be combined before that boundary.
Additional context
The standard physical optimizer pipeline often inserts a partition-combining
boundary before
GlobalLimitExec, which can hide this problem. It is observablewhen optimizing physical extension/distributed plan shapes and when the rule
itself removes or moves limit nodes. The fix can preserve per-partition fetches
for early termination while retaining the global cap.