Skip to content

LimitPushdown can mistake per-partition fetch for a global limit #23799

Description

@discord9

Describe the bug

LimitPushdown can treat an operator-level fetch on a multi-partition plan as
if it satisfied a global LIMIT.

For operators that do not combine partitions, ExecutionPlan::fetch() limits
each output partition independently. Removing the remaining global enforcement
can therefore return up to fetch * partition_count rows. Related state
transitions can also place GlobalLimitExec directly above multiple partitions
for OFFSET-only plans or combine a per-partition LocalLimitExec fetch with a
pending global fetch.

To Reproduce

Optimize a physical plan shaped like:

GlobalLimitExec: skip=0, fetch=5
  TestScan: partitions=2, supports_fetch=true

The current rule produces:

TestScan: partitions=2, fetch=5

Each partition may emit five rows, so the plan can return ten rows even though
the query has a global limit of five.

The same scope mismatch appears when the child already has a smaller
per-partition fetch, with OFFSET-only, and when a pending global limit reaches a
multi-partition LocalLimitExec.

Expected behavior

A fetch on a multi-partition operator should remain only an early-stop hint.
LimitPushdown should retain or create a single-partition enforcement boundary
(CoalescePartitionsExec or SortPreservingMergeExec) before considering the
global requirement satisfied. Global and per-partition local fetches should not
be combined before that boundary.

Additional context

The standard physical optimizer pipeline often inserts a partition-combining
boundary before GlobalLimitExec, which can hide this problem. It is observable
when optimizing physical extension/distributed plan shapes and when the rule
itself removes or moves limit nodes. The fix can preserve per-partition fetches
for early termination while retaining the global cap.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions