Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Turn on memory limits for parquet reader #10334

Closed
revans2 opened this issue Jan 30, 2024 · 0 comments · Fixed by #10406
Closed

[FEA] Turn on memory limits for parquet reader #10334

revans2 opened this issue Jan 30, 2024 · 0 comments · Fixed by #10406
Assignees
@revans2
Labels
feature request New feature or request

Comments

@revans2
Copy link
Collaborator

revans2 commented Jan 30, 2024

Is your feature request related to a problem? Please describe.
#9991 added in support for using memory limits on the chunked parquet reader. But it is kind of late in the release cycle to have it on by default right now. This is to enable the feature by default in 24.04 so we get some testing/soak time.
@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify labels Jan 30, 2024
@revans2 revans2 mentioned this issue Jan 30, 2024
Merged
@gerashegalov gerashegalov assigned revans2 and unassigned revans2 Jan 30, 2024
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Jan 30, 2024
@revans2 revans2 mentioned this issue Feb 9, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@revans2 revans2

Labels
feature request New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Enabled subPage chunking by default revans2/spark-rapids
2 participants
@revans2 @mattahrens