Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement random/targeted bucketing for cte materialization #22781

Open
jaystarshot opened this issue May 18, 2024 · 2 comments · May be fixed by #22882
Open

Implement random/targeted bucketing for cte materialization #22781

jaystarshot opened this issue May 18, 2024 · 2 comments · May be fixed by #22882

Comments

@jaystarshot
Copy link
Member

jaystarshot commented May 18, 2024

Currently the first projection column is taken as the bucketing column for cte materialization. However it makes more sense to implement random bucketing instead since that will remove skew from readers and writers and improve performance.
We could also look for downstream exchanges and add bucketing on that exchange column and potentially remove the exchange but that will be an overkill

Expected Behavior or Use Case

Presto Component, Service, or Connector

Possible Implementation

Example Screenshots (if appropriate):

Context

@jaystarshot
Copy link
Member Author

cc: @feilong-liu @aditi-pandit

@jaystarshot
Copy link
Member Author

Could be related to #13008

@jaystarshot jaystarshot linked a pull request May 31, 2024 that will close this issue
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant