Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: runtime: add way for applications to respond to GC backpressure #29696

josharian opened this issue Jan 11, 2019 · 6 comments


Copy link

commented Jan 11, 2019

This is marked as a proposal, but I'd like it to start out as a discussion. Although I believe the need is clear, the correct design is not at all obvious.

Many applications have caches that they could easily drop in response to GC backpressure. However, there is currently no good way to do this.

One obvious example is sync.Pool. If we also had per-P storage, we could rebuild sync.Pool outside the standard library using GC backpressure. Better, folks could customize the sync.Pool emptying strategy in response to their own needs.

Another example is interning strings.

As to design, there are a few aspects. What does the API look like? When and how does the runtime trigger it? And what system dynamics might result.

One API ideas:

package runtime

// The API below needs better names. And the docs are a mix of user-facing and internal details.

// GCBackpressureC returns a channel that the runtime will close to indicate GC backpressure.
// One the channel has been closed, another call to GCBackpressureC will return a new channel.
// There is only a single, shared channel across all calls to GCBackpressureC.
// This makes the runtime side cheap: a single, lazily allocated channel, and a single close call per round of backpressure.
// The downside is that it could cause a stampede of memory releases.
// Another downside is that there could be logical races in which clients miss backpressure notifications by not obtaining a new channel quickly enough.
func GCBackpressureC() <-chan struct{}

// GCBackpressureC returns a channel that the runtime will send values on to indicate GC backpressure.
// The values have no meaning here, but maybe there's something useful we could send?
// The runtime side here is more expensive. It has O(n) channels to keep track of, and does O(n) work to send signals to everyone.
// On the other hand, by not sending signals to everyone simultaneously it can help avoid stampedes.
func GCBackpressureC() <-chan struct{}

As to when the runtime should indicate backpressure, there are several options. It could do it once per GC cycle, as a matter of course. (This matches existing sync.Pool behavior.) It could do it in response to heap growth. Or maybe something else or some combination.

It's unclear what effect this might have on system dynamics. To avoid messy situations like #22950, someone should probably do some hysteresis and/or incremental work. It is unclear whether that should reside in the runtime or in user code.

cc @aclements @rsc @dvyukov @dsnet @cespare

@gopherbot gopherbot added this to the Proposal milestone Jan 11, 2019

@gopherbot gopherbot added the Proposal label Jan 11, 2019


This comment has been minimized.

Copy link

commented Jan 11, 2019

See also #16843.


This comment has been minimized.

Copy link
Contributor Author

commented Jan 11, 2019

Thanks, Ian. Looks like the primary difference between this request and that one is that #16843 seems focused on dealing with heap limits, whereas I mostly had in mind improving the management of long-lived shared caches.

Another obvious use case is monitoring and metrics.

Another observation: you don’t want the runtime to block on these. So for the queue-per-consumer model you probably need it to be best effort only and use a buffered channel. Also, for the queue-per-consumer model, you could use the sent value to provide information about the memory event (GC cycle? Regular backpressure? > 50% on our way to heap growth? > 90% on our way to heap growth? Etc.)

cc @RLH


This comment has been minimized.

Copy link

commented Jan 12, 2019

@josharian the closest thing I managed to do is, but it fires a notification at the end of a GC cycle, not when it starts (as is the case of the clearpools hook).


This comment has been minimized.

Copy link

commented Jan 14, 2019

One thought about the interface until it's too late:

func GCBackpressureC() <-chan struct{}

Async interface based on chans can limit its usefulness. It won't be possible to understand when the callbacks have actually finished to do the scanning. The previous round may not have finished when we start another one. Sync callbacks may be more useful in preventing heap growth.


This comment has been minimized.

Copy link

commented Mar 12, 2019

My big question is what does "GC backpressure" mean? In the context of SetMaxHeap, this is fairly clear: the GC is under pressure when it switches from the amortized proportional regime to the heap-limited regime. But outside of that context, the garbage collector is always under the same amount of pressure, by design.

If the point is to release caches, what you really want to know is what the trade-off is between using that memory for your caches versus giving the GC less memory to mark. I have no idea how you compute the trade-off.

In the case of sync.Pool, the current behavior leads to weird GC dynamics and I'm not sure I want to let users construct that sort of behavior without any runtime visibility into what's going on.

Related to @dvyukov's comment, it's not clear when you actually invoke this signal, but maybe defining backpressure would clarify that. Calling it when GC starts is too late for that GC cycle to collect anything that gets released. So you have to call it before GC starts, but how long before? You probably don't want to block GC until it's done (that would cause either deadlocks or unbounded heap growth). You could call it right after marking finishes, when the GC knows the targets for the next GC, but maybe that's too early.

FWIW, we've been thinking of splitting up SetMaxHeap and its notification channel because almost everyone who's tried SetMaxHeap has ignored or misused the channel and becayse tying the channel to SetMaxHeap may limit future evolution that could need the same type of notification for different reasons.


This comment has been minimized.

Copy link

commented Aug 13, 2019

@aclements, what is the state of this proposal?
Is it the same as the current SetMaxHeap implementation sketch?
Something different?
And what is the relationship between this issue and #23044 and #16843?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
7 participants
You can’t perform that action at this time.