Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add optimizer rule to move a limit beneath a union #15212

Merged
merged 2 commits into from
Dec 20, 2023
Merged

Conversation

mkleen
Copy link
Contributor

@mkleen mkleen commented Dec 19, 2023

Summary of the changes / Why this improves CrateDB

This adds the optimizer rule optimizer_move_limit_beneath_union which reduces the number of intermediate rows created when a LIMIT clause is used in combination with a UNION operator, by moving the limit beneath the union operator e.g.:

 SELECT a FROM t1 UNION ALL SELECT b FROM t2 LIMIT 10;

has the logical plan:

Limit[10;0]
  └ Union[a, b]
     ├ Collect[doc.t1 | [a] | true]
     └ Collect[doc.t2 | [b] | true]

becomes after the rule application:

Limit[10;0]
  └ Union[a, b]
    ├ Limit[10;0]
    │  └ Collect[doc.t1 | [a] | true]
    └ Limit[10;0]
       └ Collect[doc.t2 | [b] | true]		

Checklist

  • Added an entry in the latest docs/appendices/release-notes/<x.y.0>.rst for user facing changes
  • Updated documentation & sql_features table for user facing changes
  • Touched code is covered by tests
  • CLA is signed
  • This does not contain breaking changes, or if it does:
    • It is released within a major release
    • It is recorded in the latest docs/appendices/release-notes/<x.y.0>.rst
    • It was marked as deprecated in an earlier release if possible
    • You've thought about the consequences and other components are adapted
      (E.g. AdminUI)

@mkleen mkleen changed the title Add optimizer rule to move a limit beneath a union operator Add optimizer rule to move a limit operator beneath an union operator Dec 19, 2023
@mkleen mkleen force-pushed the mkleen/union-limit branch 2 times, most recently from 378f8fe to 6e178f9 Compare December 20, 2023 10:17
@mkleen mkleen changed the title Add optimizer rule to move a limit operator beneath an union operator Add optimizer rule to move a limit beneath a union Dec 20, 2023
@mkleen mkleen force-pushed the mkleen/union-limit branch 2 times, most recently from f30ec9c to 18c4d25 Compare December 20, 2023 10:44
@mkleen mkleen marked this pull request as ready for review December 20, 2023 10:52
@mkleen mkleen requested review from seut and matriv December 20, 2023 11:00
Copy link
Contributor

@matriv matriv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! LGTM

@mkleen mkleen added the ready-to-merge Let Mergify merge the PR once approved and checks pass label Dec 20, 2023
This adds the optimizer rule ``optimizer_move_limit_beneath_union`` which
reduces the number of intermediate rows created when a ``LIMIT`` clause is
used in combination with a ``UNION`` operator, by moving the limit beneath
the union operator.
@mergify mergify bot merged commit bc0c84f into master Dec 20, 2023
18 checks passed
@mergify mergify bot deleted the mkleen/union-limit branch December 20, 2023 16:32
@mkleen mkleen linked an issue Dec 21, 2023 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-to-merge Let Mergify merge the PR once approved and checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Extend Optimizer to push Limits down to more plans
2 participants