Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix worker OOM #9949

Merged
merged 1 commit into from
Jan 11, 2022
Merged

Fix worker OOM #9949

merged 1 commit into from
Jan 11, 2022

Conversation

XuPengfei-1020
Copy link
Member

The io.trino.orc.reader.SliceDictionaryColumnReader#dictionaryData could retaine hundreds MB memory.
Sometime caused worker OOM.

@cla-bot cla-bot bot added the cla-signed label Nov 12, 2021
@cla-bot
Copy link

cla-bot bot commented Dec 23, 2021

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: x624464767@hotmail.com.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email email@example.com
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@cla-bot cla-bot bot removed the cla-signed label Dec 23, 2021
@cla-bot cla-bot bot added the cla-signed label Dec 23, 2021
sopel39
sopel39 approved these changes Jan 5, 2022
View reviewed changes
Copy link
Member

@sopel39 sopel39 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm % comments

lib/trino-orc/src/main/java/io/trino/orc/reader/SliceDictionaryColumnReader.java Show resolved Hide resolved
lib/trino-orc/src/main/java/io/trino/orc/reader/SliceDictionaryColumnReader.java Show resolved Hide resolved
lib/trino-orc/src/test/java/io/trino/orc/TestSliceDictionaryColumnReader.java Outdated Show resolved Hide resolved
lib/trino-orc/src/test/java/io/trino/orc/TestSliceDictionaryColumnReader.java Outdated Show resolved Hide resolved
@XuPengfei-1020
Account for memory usage of dictionary in SliceDictionaryColumnReader
b8aec8b 
SliceDictionaryColumnReader#dictionaryData could retain hundreds
MB memory, sometime can cause worker OOM, especially in the
situation of small worker and large concurrency.
@sopel39 sopel39 merged commit f722686 into trinodb:master Jan 11, 2022
@sopel39 sopel39 mentioned this pull request Jan 11, 2022
Closed
@github-actions github-actions bot added this to the 368 milestone Jan 11, 2022
@mosabua mosabua mentioned this pull request Jan 11, 2022
Merged
@maltesander maltesander mentioned this pull request Apr 12, 2022
Closed
7 tasks
@MnO2 MnO2 mentioned this pull request Mar 8, 2023
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sopel39 sopel39 sopel39 approved these changes

@dain dain Awaiting requested review from dain

@skrzypo987 skrzypo987 Awaiting requested review from skrzypo987

@losipiuk losipiuk Awaiting requested review from losipiuk

@martint martint Awaiting requested review from martint

@raunaqmorarka raunaqmorarka Awaiting requested review from raunaqmorarka

Assignees
No one assigned
Labels
cla-signed
Milestone
368
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@XuPengfei-1020 @martint @raunaqmorarka @sopel39 @skrzypo987