Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[hailctl] Add --debug-mode flag to hailctl dataproc start #9420

Merged
merged 1 commit into from Sep 8, 2020

Conversation

tpoterba
Copy link
Contributor

@tpoterba tpoterba commented Sep 8, 2020

Currently the only debug feature is the heap dump flag, but probably best to control debug features we might add in the future as a unit.

Copy link
Contributor

@johnc1231 johnc1231 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great inclusion, thanks. I think another thing that could be helpful is to periodically do a jmap -hist:live to count / inspect sizes of objects. That's a little more complicated, but might be helpful to avoid having to download full sized heap dumps. I'm also not sure if we are going to get heap dumps if yarn kills a process for using too much memory.

@tpoterba
Copy link
Contributor Author

tpoterba commented Sep 8, 2020

yeah, that's a good idea. I think we'll get a heap dump on driver OOM, but not worker.

@danking danking merged commit c631301 into hail-is:main Sep 8, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants