New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: runtime: add way for applications to respond to GC backpressure #29696

Open
josharian opened this Issue Jan 11, 2019 · 4 comments

Comments

Projects
None yet
5 participants
@josharian
Copy link
Contributor

josharian commented Jan 11, 2019

This is marked as a proposal, but I'd like it to start out as a discussion. Although I believe the need is clear, the correct design is not at all obvious.

Many applications have caches that they could easily drop in response to GC backpressure. However, there is currently no good way to do this.

One obvious example is sync.Pool. If we also had per-P storage, we could rebuild sync.Pool outside the standard library using GC backpressure. Better, folks could customize the sync.Pool emptying strategy in response to their own needs.

Another example is interning strings.

As to design, there are a few aspects. What does the API look like? When and how does the runtime trigger it? And what system dynamics might result.

One API ideas:

package runtime

// The API below needs better names. And the docs are a mix of user-facing and internal details.

// GCBackpressureC returns a channel that the runtime will close to indicate GC backpressure.
// One the channel has been closed, another call to GCBackpressureC will return a new channel.
// There is only a single, shared channel across all calls to GCBackpressureC.
// This makes the runtime side cheap: a single, lazily allocated channel, and a single close call per round of backpressure.
// The downside is that it could cause a stampede of memory releases.
// Another downside is that there could be logical races in which clients miss backpressure notifications by not obtaining a new channel quickly enough.
func GCBackpressureC() <-chan struct{}

// GCBackpressureC returns a channel that the runtime will send values on to indicate GC backpressure.
// The values have no meaning here, but maybe there's something useful we could send?
// The runtime side here is more expensive. It has O(n) channels to keep track of, and does O(n) work to send signals to everyone.
// On the other hand, by not sending signals to everyone simultaneously it can help avoid stampedes.
func GCBackpressureC() <-chan struct{}

As to when the runtime should indicate backpressure, there are several options. It could do it once per GC cycle, as a matter of course. (This matches existing sync.Pool behavior.) It could do it in response to heap growth. Or maybe something else or some combination.

It's unclear what effect this might have on system dynamics. To avoid messy situations like #22950, someone should probably do some hysteresis and/or incremental work. It is unclear whether that should reside in the runtime or in user code.

cc @aclements @rsc @dvyukov @dsnet @cespare

@gopherbot gopherbot added this to the Proposal milestone Jan 11, 2019

@gopherbot gopherbot added the Proposal label Jan 11, 2019

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

ianlancetaylor commented Jan 11, 2019

See also #16843.

@josharian

This comment has been minimized.

Copy link
Contributor

josharian commented Jan 11, 2019

Thanks, Ian. Looks like the primary difference between this request and that one is that #16843 seems focused on dealing with heap limits, whereas I mostly had in mind improving the management of long-lived shared caches.

Another obvious use case is monitoring and metrics.

Another observation: you don’t want the runtime to block on these. So for the queue-per-consumer model you probably need it to be best effort only and use a buffered channel. Also, for the queue-per-consumer model, you could use the sent value to provide information about the memory event (GC cycle? Regular backpressure? > 50% on our way to heap growth? > 90% on our way to heap growth? Etc.)

cc @RLH

@CAFxX

This comment has been minimized.

Copy link
Contributor

CAFxX commented Jan 12, 2019

@josharian the closest thing I managed to do is https://github.com/CAFxX/gcnotifier, but it fires a notification at the end of a GC cycle, not when it starts (as is the case of the clearpools hook).

@dvyukov

This comment has been minimized.

Copy link
Member

dvyukov commented Jan 14, 2019

One thought about the interface until it's too late:

func GCBackpressureC() <-chan struct{}

Async interface based on chans can limit its usefulness. It won't be possible to understand when the callbacks have actually finished to do the scanning. The previous round may not have finished when we start another one. Sync callbacks may be more useful in preventing heap growth.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment