Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve](restore) Release useless info for the finished job to reduce mem usage #34513

Merged
merged 3 commits into from
May 8, 2024

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented May 8, 2024

Proposed changes

Issue Number: close #xxx

The info about snapshot files is not visible in the result of SHOW RESTORE, but it costs a lot of memory.

image

Here is an example of a restore job about a large table, the main memory usage is in :

  • jobInfo
    • backupOlapTableObjects
    • tabletBeMap
    • tabletSnapshotPathMap
  • fileMapping
    • mapping
    • overwriteMap

This PR clear these fields when task is finished/cancelled, to reduce the memory usage.

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@w41ter
Copy link
Contributor Author

w41ter commented May 8, 2024

run buildall

@gavinchou
Copy link
Contributor

LGTM

Copy link
Contributor

github-actions bot commented May 8, 2024

PR approved by anyone and no changes requested.

@w41ter
Copy link
Contributor Author

w41ter commented May 8, 2024

run buildall

@w41ter
Copy link
Contributor Author

w41ter commented May 8, 2024

run buildall

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label May 8, 2024
Copy link
Contributor

github-actions bot commented May 8, 2024

PR approved by at least one committer and no changes requested.

@dataroaring dataroaring merged commit 6f4738c into apache:master May 8, 2024
24 of 26 checks passed
@w41ter w41ter deleted the improve/reduce_mem_usage branch May 8, 2024 12:42
w41ter added a commit to w41ter/incubator-doris that referenced this pull request May 8, 2024
dataroaring pushed a commit that referenced this pull request May 8, 2024
yiguolei pushed a commit that referenced this pull request May 10, 2024
ByteYue pushed a commit to ByteYue/doris that referenced this pull request May 15, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.10-merged dev/3.0.0-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants