Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add an option to disable the idle GC #5589

Closed
benjaminp opened this issue Jul 13, 2018 · 2 comments
Closed

add an option to disable the idle GC #5589

benjaminp opened this issue Jul 13, 2018 · 2 comments
Assignees
@meisterT
Labels
type: feature request

Comments

@benjaminp
Copy link
Collaborator

If a Bazel server is idle for 10 seconds after finishing a command, it will invoke System.gc(). On HotSpot, System.gc() triggers an aggressive full GC. In practice, I see System.gc() in a Bazel server with a 2GB heap fully consume 4 cores for several seconds. This behavior is problematic for workflows that involve building a binary and then immediately running it; if the binary itself is resource intensive and long-running, Bazel will steal CPU from it. One can effectively disable the idle GC today with --host_jvm_args=-XX:+DisableExplicitGC, but this is hacky and makes instrumentation commands like bazel info used-heap-size-after-gc silently useless. So, I propose introducing an official startup flag to disable idle GC.

A new option is my conservative proposal. Going further, I'd be interested in learning the reasons for Bazel's idle GC behavior. To me, it's polite to only use lots of CPU when Bazel is synchronously processing a command. I suppose in theory an idle GC saves GC work during synchronous work, but can the value of that be measured? If a user is iterating by running build or test on the same target, most server state should be essentially persistent and won't benefit from being repeatedly walked by a full GC. If the idle GC is valuable, could Bazel use a whole-system idleness check in lieu of the current 10 second heuristic?
@dslomov
Copy link
Contributor

dslomov commented Jul 13, 2018

@meisterT for triage since this is perf related. I do agree that "every 10 seconds full gc" is not awesome.

@buchgr
Copy link
Contributor

buchgr commented Jul 13, 2018

I think we should delete this code. I have yet to see a benchmark showing that the application logic choosing to do a GC results in better performance than the JVM triggering the GC itself.

benjaminp added a commit to benjaminp/bazel that referenced this issue Jul 16, 2018
@benjaminp
Remove idle GC.
ef44beb 
If a Bazel server was idle for 10 seconds, it would trigger a full-scale Java GC
via System.gc(). This behavior didn't have clear benefits and caused Bazel to
steal resources from whatever the user did after invoking Bazel.

Fixes bazelbuild#5589.
@benjaminp benjaminp mentioned this issue Jul 16, 2018
Closed
benjaminp added a commit to benjaminp/bazel that referenced this issue Jul 18, 2018
@benjaminp
Disable idle gc by default.
8a290be 
If a Bazel server was idle for 10 seconds, it would unconditionally trigger a full-scale Java GC via System.gc(). This behavior doesn't have clear benefits and causes Bazel to steal resources from whatever the user did after invoking Bazel. This CL adds a startup option, --idle_gc, to toggle the idle GC behavior and disables it by default.

Also, add some logging for when idle GC is enabled, so it's easier to evaluate its effects. Example of logging:
```
180718 17:43:04.609:I 247 [com.google.devtools.build.lib.server.IdleServerTasks.lambda$idle$0] [Idle GC] used: 157MB -> 15MB, committed: 421MB -> 422MB
```

Fixes bazelbuild#5589.
@benjaminp benjaminp mentioned this issue Jul 18, 2018
Closed
benjaminp added a commit to benjaminp/bazel that referenced this issue Jul 19, 2018
@benjaminp
Add an option to disable idle GC.
1a9d41e 
If a Bazel server is idle for 10 seconds, it would unconditionally trigger a full-scale Java GC via System.gc(). This behavior doesn't have clear benefits and causes Bazel to steal resources from whatever the user does after invoking Bazel. This CL adds a startup option, --idle_gc, to toggle the idle GC behavior.

Also, add some logging for when idle GC is enabled, so it's easier to evaluate its effects. Example of logging:
```
180718 17:43:04.609:I 247 [com.google.devtools.build.lib.server.IdleServerTasks.lambda$idle$0] [Idle GC] used: 157MB -> 15MB, committed: 421MB -> 422MB
```

Fixes bazelbuild#5589.
benjaminp added a commit to benjaminp/bazel that referenced this issue Jul 31, 2018
@benjaminp
Add an option to disable idle GC.
594630c 
If a Bazel server is idle for 10 seconds, it would unconditionally trigger a full-scale Java GC via System.gc(). This behavior doesn't have clear benefits and causes Bazel to steal resources from whatever the user does after invoking Bazel. This CL adds a startup option, --idle_server_tasks, to toggle the idle GC behavior.

Also, add some logging for when idle GC is enabled, so it's easier to evaluate its effects. Example of logging:
```
180718 17:43:04.609:I 247 [com.google.devtools.build.lib.server.IdleServerTasks.lambda$idle$0] [Idle GC] used: 157MB -> 15MB, committed: 421MB -> 422MB
```

Fixes bazelbuild#5589.
benjaminp added a commit to benjaminp/bazel that referenced this issue Jul 31, 2018
@benjaminp
Add an option to disable idle GC.
8013b19 
If a Bazel server is idle for 10 seconds, it would unconditionally trigger a full-scale Java GC via System.gc(). This behavior doesn't have clear benefits and causes Bazel to steal resources from whatever the user does after invoking Bazel. This CL adds a startup option, --idle_server_tasks, to toggle the idle GC behavior.

Also, add some logging for when idle GC is enabled, so it's easier to evaluate its effects. Example of logging:
```
180718 17:43:04.609:I 247 [com.google.devtools.build.lib.server.IdleServerTasks.lambda$idle$0] [Idle GC] used: 157MB -> 15MB, committed: 421MB -> 422MB
```

Fixes bazelbuild#5589.
benjaminp added a commit to benjaminp/bazel that referenced this issue Aug 6, 2018
@benjaminp
Add an option to disable idle GC.
be70ab9 
If a Bazel server is idle for 10 seconds, it would unconditionally trigger a full-scale Java GC via System.gc(). This behavior doesn't have clear benefits and causes Bazel to steal resources from whatever the user does after invoking Bazel. This CL adds a startup option, --idle_server_tasks, to toggle the idle GC behavior.

Also, add some logging for when idle GC is enabled, so it's easier to evaluate its effects. Example of logging:
```
180718 17:43:04.609:I 247 [com.google.devtools.build.lib.server.IdleServerTasks.lambda$idle$0] [Idle GC] used: 157MB -> 15MB, committed: 421MB -> 422MB
```

Fixes bazelbuild#5589.
iirina pushed a commit that referenced this issue Aug 9, 2018
@benjaminp @iirina
Add a option to disable idle gc.
206d554 
If a Bazel server is idle for 10 seconds, it unconditionally triggers a full-scale Java GC via System.gc(). This behavior doesn't have clear benefits and causes Bazel to steal resources from whatever the user does after invoking Bazel. This CL adds a startup option, --idle_server_tasks, to toggle the idle GC behavior.

Also, add some logging for when idle GC is enabled, so it's easier to evaluate its effects. Example of logging:
```
180718 17:43:04.609:I 247 [com.google.devtools.build.lib.server.IdleServerTasks.lambda$idle$0] [Idle GC] used: 157MB -> 15MB, committed: 421MB -> 422MB
```

Fixes #5589.

Closes #5628.

PiperOrigin-RevId: 207869996
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@meisterT meisterT

Labels
type: feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@benjaminp @jin @meisterT @buchgr @dslomov