Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory check before checkpoint is inaccurate #1881

Closed
morningman opened this issue Sep 26, 2019 · 0 comments
Closed

Memory check before checkpoint is inaccurate #1881

morningman opened this issue Sep 26, 2019 · 0 comments
Assignees
Milestone

Comments

@morningman
Copy link
Contributor

Currently, Doris will check whether the jvm memory is enough to hold the checkpoint image. But the algorithm is inaccurate, it only concern the OlapTable objects. The Views, load jobs and other metadata are not included.
And inaccurate memory checking may cause OOM when doing checkpoint.

So now, I will use JVM old mem pool used percent to check the memory usage. And add to new FE config

metadata_checkopoint_memory_threshold. default is 60(%), if the mem pool used percent exceed this percent, checkpoint will not work.

force_do_metadata_checkpoint. default is false. In case the auto memory check has some problem, we can switch this config to do the checkpoint forcibly, ignore the memory usage.

@morningman morningman self-assigned this Sep 26, 2019
@morningman morningman added this to the 0.11 milestone Sep 26, 2019
@imay imay closed this as completed Oct 11, 2019
SWJTU-ZhangLei pushed a commit to SWJTU-ZhangLei/incubator-doris that referenced this issue Jul 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants