Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Increase dynamic filter size limits for FTE #16104

Closed
arhimondr opened this issue Feb 14, 2023 · 2 comments
Closed

Increase dynamic filter size limits for FTE #16104

arhimondr opened this issue Feb 14, 2023 · 2 comments
Assignees
@arhimondr

Comments

@arhimondr
Copy link
Contributor

In FTE limits have to be higher as dynamic filters are collected before shuffle boundary, thus the number of distinct values per driver is expected to be higher (same as size of dynamic filter collected by each driver)

Depends on: #16102, #16100
@arhimondr arhimondr mentioned this issue Apr 4, 2023
Closed
@arhimondr arhimondr mentioned this issue Jun 9, 2023
Merged
@losipiuk
Copy link
Member

losipiuk commented Oct 6, 2023

@raunaqmorarka I think you have context on this one. Is there sill some outstanding work?

@losipiuk
Copy link
Member

losipiuk commented Oct 9, 2023

Initial work was already done in #17831.
Per @raunaqmorarka @Dith3r is currently working on more general improvements which are not limited to FTE.

@losipiuk losipiuk closed this as completed Oct 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@arhimondr arhimondr

Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@losipiuk @arhimondr