proposal: runtime: heap adaptive resizing with gc throughput control

[superajun-wsj]

Proposal Details

From ByteDance Programming Language Team

Summary

This proposal proposes an Adaptive GC Control Algorithm based on GC Throughput, aimed at addressing the lack of adaptability in the current GC mechanism across different workload scenarios.

GC Throughput is used to evaluate the impact of GC on application performance. It is defined as the ratio of CPU costs consumed by the application to the process CPU costs within one gc cycle, process CPU cost is the sum of application CPU cost and GC CPU cost:

$$ GC\ Throughput = \frac{Application\ CPU\ Cost}{Application\ CPU\ Cost + GC\ CPU\ Cost} $$

The algorithm uses GC throughput as its control target, establishing a dynamic feedback control system. By monitoring the application's GC throughput in real-time and comparing it with a target GC throughput (for example, 96%), the system dynamically adjusts GOGC based on the deviation. This maintains GC throughput within the desired range and stabilizes application performance.

The target GC throughput is not constant and is dynamically adjusted at runtime according to GC CPU overhead. GC CPU overhead is the ratio of GC CPU cost to Total CPU Time within one gc cycle, Total CPU Time is the sum of process CPU cost and idle CPU time:

$$ GC\ CPU\ overhead = \frac{GC\ CPU\ Cost}{Total\ CPU\ Time} $$

GC CPU overhead is used as an indicator of workload intensity. A higher GC CPU overhead implies a busier workload and a stronger demand for GC throughput; therefore, a higher target GC throughput should be set to satisfy the workload’s requirements. Conversely, a lower GC CPU overhead indicates a more idle workload with reduced throughput demands, allowing a lower target GC throughput to be set to conserve system resources.

In summary, this control mechanism addresses the following issues:

1. Adaptability: Enhances GC adaptability across various scenarios by enabling self-adjusting GOGC, eliminating the need for manual tuning.
2. Resource Optimization: Automatically searches for optimal memory resource consumption while maintaining stable performance, achieving a balance between performance and memory usage.
3. Safety: Works synergistically with the Soft Memory Limit mechanism, setting an upper bound for GOGC adjustments to prevent OOM.

Motivation

Prior to Go 1.19, the GC pacer primarily estimated the trigger threshold for the next cycle based on GOGC and the live heap size. Since live heap sizes vary significantly across different workloads, the default GOGC often lacked universality. A small live heap could lead to frequent GC cycles, impacting performance, while a large live heap could consume excessive memory, leading to OOM. Consequently, users had to manually tune GOGC for different scenarios. Go 1.19 introduced the Soft Memory Limit (GOMEMLIMIT), which triggers GC when memory usage reaches a specified limit, effectively solving the OOM issue.

Based on the current GC mechanism, users typically adopt one of three configuration patterns:

1. GOMEMLIMIT Disabled, GOGC Enabled: GC behavior is determined solely by GOGC and live memory. As mentioned, this requires manual tuning. For example, in a scenario with a high allocation rate but low survival rate, the default GOGC=100 triggers frequent GCs. A significant portion of CPU resources is consumed by GC (overhead), severely impacting mutator performance. Users must manually increase GOGC to raise the trigger threshold and reduce frequency. Furthermore, there is a lack of clear standards for determining the "correct" GOGC value, resulting in high tuning costs.

2. GOMEMLIMIT Enabled, GOGC Disabled (or set to off): GC behavior is determined solely by GOMEMLIMIT. This means the application's RSS will consistently hover around the limit. If a workload exhibits periodic peaks and valleys, memory resources are not released during valleys (low load periods), which is detrimental to resource utilization in cloud-native hybrid deployment scenarios. Additionally, selecting an appropriate GOMEMLIMIT value is challenging and often requires multiple adjustments.

3. GOMEMLIMIT Enabled, GOGC Enabled: GC behavior is primarily driven by GOGC, with GOMEMLIMIT acting as a safety net (cap) to prevent OOM. While this solves the safety issue of the first pattern, it retains the defect of fixed configurations: a static GOGC cannot adapt to dynamic workload requirements.

Historically, users have manually adjusted GOGC or GOMEMLIMIT based on GC frequency and the gc_cpu_fraction metric. For example, users would increase GOGC when GC frequency is high, and decrease it otherwise. However, GC frequency itself does not accurately reflect the degree to which GC impacts application performance, rendering the tuning process largely subjective. Additionally, the gc_cpu_fraction metric merely indicates the absolute ratio of GC CPU utilization, which also fails to accurately represent the true impact on performance.

We introduce GC Throughput to quantify the degree to which GC impacts application performance. By dynamically adjusting GOGC to maintain a target GC throughput derived from a sigmoid model based on GC CPU overhead, the algorithm stabilizes mutator performance and automatically discovers optimal memory usage. Furthermore, this metric signals the suitability of GOMEMLIMIT: consistently missing the target suggests the limit is too restrictive(capping GOGC), while meeting the target with low memory usage suggests the limit is over-provisioned and can be reduced.

Detailed design

At the end of each GC cycle (during the second STW phase), the system calculates the real-time GC throughput and GC CPU overhead based on the recorded application CPU cost, GC CPU cost and total CPU time in this GC cycle.

• GC CPU cost = (GCAssistTime + GCDedicatedTime + GCIdleTime + GCPauseTime + ScavengeTotalTime) in this GC cycle.

• Application CPU cost = UserTime in this GC cycle.

• Total CPU time = TotalTime in this GC cycle.

We model the relationship between GC CPU overhead and the target GC throughput using the right half of a sigmoid function. When the GC CPU overhead is non-negative (x ≥ 0), the sigmoid output lies in the range [0.5, 1), which naturally captures the monotonic increase in throughput demand as GC overhead grows. The sigmoid output is further linearly mapped to a bounded target GC throughput range [Min, Max], ensuring that the adjusted throughput remains within predefined limits.

The feedback control algorithm continuously compares the actual GC throughput with the target given by the sigmoid model in each GC cycle. If GC throughput exceeds the target, the algorithm gradually decreases gc percent; if it falls below, gc percent is incrementally raised to compensate. After several cycles of adjustment, the algorithm gradually aligns the gc throughput with the target.

• if real-time GC throughput is close to the target with a margin of error: keep gc percent unchanged

• if real-time GC throughput > target: decrease gc percent by a step

• if real-time GC throughput < target: increase gc percent by a step

The adjustment step size can be dynamically modified based on the deviation between real-time GC throughput and the target. For example, if the deviation is large, a larger step size can be used to adjust gc percent more quickly. Conversely, if the deviation is small, a smaller step size can be used to adjust gc percent more gradually.

Take three classic scenarios to illustrate how the algorithm works:

1. High object allocation rate with a very low marking survival rate: This scenario triggers frequent GC cycles, usually with a high GC CPU overhead, significantly disrupting normal operations while failing to fully utilize available memory resources. The sigmoid model will give a high target GC throughput. The feedback control algorithm increases gc percent to improve GC throughput, thereby reducing the frequency of garbage collection cycles and improving application performance. After several adjustment cycles, the application reaches a new steady state.

2. Slow object allocation rate with a high marking survival rate: In this case, GC cycles tend to be longer, and GC CPU overhead remains low. The sigmoid model will give a relatively low target GC throughput. The feedback control algorithm decreases gc percent to reduce memory consumption, while ensuring the application’s performance requirements are still met. After several adjustment cycles, the application reaches a new steady state.

3. Workload with periodic peaks and valleys: During low-load periods, the application requires only a small amount of memory to maintain performance. However, if GOGC remains unchanged, memory may not be released back to the OS, leading to inefficient resource usage. During low-load periods, low GC CPU overhead will give a relatively low target GC throughput. The feedback control algorithm decreases gc percent to release unused memory back to the system, ensuring efficient resource utilization and preventing unnecessary memory retention. During peak loads, high GC CPU overhead will give a relatively high GC throughput overhead, when GC throughput declines, the algorithm increases gc percent to sustain throughput and preserve application performance.

The algorithm dynamically regulates GC throughput, keeping it within the target range to maintain a predictable and stable impact on application performance. This ensures a balance between GC overhead and memory efficiency, preventing excessive memory consumption while avoiding noticeable performance degradation. Additionally, it could integrate the soft limit memory mechanism to effectively protect the program from running out of memory (OOM).

Simulations

We conduct sigmoid simulations to evaluate the goal GC throughput with different GC CPU overhead. The result shows that the sigmoid model with K = 3.0 predicts the target GC throughput very well, providing a reliable basis for the feedback control algorithm.

With sigmoid function with K = 3.0, the target GC throughput is linearly mapped to the defined range [0.66, 0.95].
When the GC CPU overhead is 1%, the target GC throughput is 92.3%. Additional example mappings are listed in the table below.

GC CPU Overhead (%)	Target GC Throughput (%)
0.1	70.7
0.25	76.7
0.5	84.6
1	92.3
1.5	94.3
2.0	94.8
2.56	94.9

Implementation

https://github.com/superajun-wsj/gc-adaptive

Stress test results

scenarios one: High object allocation rate with a very low marking survival rate

This scenario simulates a situation with a high object allocation rate but a very low mark-and-sweep survival rate. Under these conditions, the application frequently triggers garbage collection, typically resulting in high GC CPU overhead and significant performance disruption. Three configurations were tested: the first with the feedback control algorithm enabled, the second with the default settings (GOGC=100), and the third with GOGC=-1 and GOMEMLIMIT=10G. The results show that under the default settings, the workload experiences low GC throughput; with the third configuration, GC throughput improves significantly but at the cost of high memory consumption. When the feedback control algorithm is enabled, GC throughput is substantially increased—roughly comparable to the third configuration—while memory resource consumption is significantly reduced.

The test results under the three configurations are as follows:

Configuration	GC Throughput (%)	Memory Usage (GB)	Overall Requests/sec	Avg Req Time (ms)
GC Throughput Control	95.3	5.1	2726.16	18.333
Default Settings (GOGC=100)	67.5	1.0	1969.47	25.377
GOMEMLIMIT=10G (GOGC=off)	97.5	9.0	2755.03	18.141

scenarios two: Very slow object allocation rate after a period of high object allocation rate

This scenario simulates a workload that has been running under high load for an extended period, after which the load suddenly drops to a very low level. The purpose is to observe whether the feedback control algorithm can promptly adjust the target GC throughput and reduce GOGC accordingly, thereby lowering memory overhead. The test results show that after the workload enters an extremely low-load state, memory usage gradually decreases under the control of the feedback algorithm, effectively releasing memory resources.

scenarios three: Workload with periodic peaks and valleys

This scenario simulates a workload with periodic peaks and valleys. The purpose is to evaluate the performance of the feedback control algorithm during low-load and peak-load periods, and to assess whether it can promptly adjust GOGC—reducing memory usage during valleys and increasing memory availability during peaks to maintain stable performance. The test results show that during low-load periods, the feedback control algorithm effectively reduces memory usage by lowering GOGC, while during peak-load periods, it increases GOGC to maintain GC throughput and ensure stable performance. This demonstrates the algorithm's ability to adapt to dynamic workload changes and optimize resource utilization.

References

Paper(Heap Size Adjustment with CPU Control): https://dl.acm.org/doi/epdf/10.1145/3617651.3622988
OpenJDK G1 GC JEP: https://openjdk.org/jeps/8359211
OpenJDK G1 GC Heap Resizing issue: https://bugs.openjdk.org/browse/JDK-8374026?jql=labels%20%3D%20gc-g1-heap-resizing
Java ZGC: https://inside.java/2023/06/20/optimizing-memory-utilization-zgc/

[gabyhelp]

Related Issues

• proposal: runtime: GC pacer redesign #44167 (closed)
• proposal: runtime: add a mechanism for specifying a minimum target heap size #23044 (closed)
• runtime: GC pacer problems meta-issue #42430
• runtime: throttle heap growth #14735
• proposal: runtime/debug: user-configurable memory target #44309 (closed)

Related Documentation

• A Guide to the Go Garbage Collector > The GC cycle > GOGC

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ianlancetaylor]

CC @golang/runtime

[mknyszek]

Thanks for the detailed proposal. The idea of helping programs find better trade-offs in the GOGC space automatically is an interesting idea in general. I think there's certainly merit to this proposal.

At a high level, my concern is that today, GOGC is defined such that memory usage is somewhat predictable: it's some proportion over your live heap, which is mostly inherent to your program. My worry about a fully adaptive approach is making understanding memory usage more complicated. When someone asks "why this how much memory my program is using," ideally we don't have to discuss heuristics. Furthermore, memory is finite in a way that CPU time is not. There's more CPU time in the future, but if you run out of memory, you're done. It seems more important to me in many cases to keep that predictable.

My other high level concern is violating the spirit of GOGC. I don't think we can just opt-in everyone to this. It would just be far too disruptive. Should this be a new setting? Like GOGC=auto?

Actually, given that this entire proposal is based on building on top of GOGC, could it be implemented as a library first? After all, we have runtime/debug.SetGCPercent, and a whole bunch of GC CPU metrics in the runtime/metrics package (they're updated once per GC cycle, but that seems reasonable for this?). I feel like that would be a great place to start; we could get a much broader part of the community to try it out and gather feedback from production. There's also an interesting opportunity here to use your technique to suggest a better GOGC value to users, even if it's not adjusted automatically.

At a lower level, I have one more specific question: what motivated the sigmoid model based on GC overhead? I'm having trouble building an intuition for that choice. Perhaps this is answered in your links to what the JVM does here.

[superajun-wsj]

1. The ideal way for this feature to operate is in conjunction with the soft memory limit mechanism, enabling adaptive adjustments within the configured memory bound. This helps effectively prevent OOM situations.

2. Using GOGC=auto as the switch to enable this feature is a good idea.

3. Our internal implementation resides in the runtime package. It may also be possible to implement this as a library—for example, by leveraging object finalizer callbacks to ensure that the computation and update of GOGC occur once per GC cycle. However, the most appropriate time to adjust GOGC would be during the second STW phase of the GC cycle.

4. Neither Java G1 GC nor ZGC establishes a sigmoid-based mapping between GC CPU overhead and target GC throughput. The fundamental motivation for introducing a sigmoid model in this proposal is to allow the target throughput to adapt dynamically to workload conditions, rather than remaining a fixed constant (e.g., 95%). Maintaining a 95% GC throughput target is reasonable under heavy workload conditions. However, when the application is relatively idle but happens to retain a large live heap, the computed GC throughput may appear low—for example, GC CPU cost = 100 and application CPU cost = 150, yielding a GC throughput of 60%. In such cases, a control algorithm that strictly targets 95% throughput would continue increasing GOGC. Under low-load conditions, however, aggressively pursuing a 95% throughput target is unnecessary and may prevent memory from being reclaimed effectively. Therefore, we use GC CPU overhead as an indicator of application load and dynamically adjust the target GC throughput based on this value.

@mknyszek do you have any comments or suggestions please?

[samborkent]

Hi @superajun-wsj, I think this is a very interesting proposal. I liked @mknyszek's idea of implementing it as a library, so I made an attempt here: https://github.com/samborkent/adaptive-gc.

It uses runtime.AddCleanup as trigger. If this would be implemented in the std lib, it could perhaps be triggered during the 2nd STW phase as suggested. I calculate runtime metrics over a 3 cycle average, as suggested in the linked paper.

Here is the sigmoid model I used for the target GC throughput: https://www.desmos.com/calculator/bxhsgvtsyj. It corresponds to the model in your plot.

After implementing this, I have a few questions:

• How did you decide on this sigmoid model? How was K = 3.0 (actually 300 if you're working with GC overhead percentages) determined?
• How are the optimal minimal and maximum GC throughput values determined? In your plot you use [0.66, 0.95], where do these limits come from?
• Should the minimum and maximum GC throughput values be set by the user? In that case doesn't that make it more complicated to configure GC correctly? Because you now have to set 2 values (min and max GC throughput) instead of 1 (GOGC).
• How do you scale the step size for adjusting GOGC based on the GC throughput error?
• In my testing it seems that incorrectly picking the minimum and maximum throughput or incorrectly scaling the GOGC adjustment step size can lead to instability. How is stability guaranteed in your model?

Thank you in advance!

[superajun-wsj]

Hi @samborkent — I’ve implemented this proposal as a standalone library (https://github.com/superajun-wsj/gc-adaptive), based on advice from @mknyszek. The repo includes three stress-test examples showing how to integrate and use the library, and they reproduce the previously discussed effects in three representative scenarios.

[superajun-wsj]

@samborkent according to your questions:

• To make the target GC throughput vary with GC CPU overhead, we found that a sigmoid curve (k = 3.0) provides a good mapping between the two.

• In our initial implementation, the target GC throughput was constant, with a default of 95% (a common rule of thumb from Java GC). This target is configurable. The value 0.66 comes from the upper bound of the target throughput multiplied by 0.7, where 0.7 is an empirical factor.

• In normal cases, users only need to enable this feature via GC_ADAPTIVE=true. Compared to asking users to tune GOGC, this feature uses GC throughput as the control objective and can adapt to a much wider range of workload patterns. Ideally, it should be used together with GOMEMLIMIT: within the heap range defined by GOMEMLIMIT, it will automatically find a memory consumption level that meets the target GC throughput. In addition, both the upper and lower bounds of the target GC throughput can be set via environment variables; the defaults are [0.66, 0.95].

• Please refer to https://github.com/superajun-wsj/gc-adaptive/blob/main/gc_adaptive_controller.go.

• Our implementation can easily drive the system into a stable state; the three examples under examples/stress can be used to validate this.

If you have any questions, feel free to reach out and discuss.

[ethanmontgomery148-hub]

Could u share more details about your setup

…

On Tue, Mar 17, 2026, 3:46 AM Jeffery ***@***.***> wrote: *superajun-wsj* left a comment (golang/go#77587) <#77587 (comment)> @samborkent <https://github.com/samborkent> according to your questions: - To make the target GC throughput vary with GC CPU overhead, we found that a sigmoid curve (k = 3.0) provides a good mapping between the two. - In our initial implementation, the target GC throughput was constant, with a default of 95% (a common rule of thumb from Java GC). This target is configurable. The value 0.66 comes from the upper bound of the target throughput multiplied by 0.7, where 0.7 is an empirical factor. - In normal cases, users only need to enable this feature via GC_ADAPTIVE=true. Compared to asking users to tune GOGC, this feature uses GC throughput as the control objective and can adapt to a much wider range of workload patterns. Ideally, it should be used together with GOMEMLIMIT: within the heap range defined by GOMEMLIMIT, it will automatically find a memory consumption level that meets the target GC throughput. In addition, both the upper and lower bounds of the target GC throughput can be set via environment variables; the defaults are [0.66, 0.95]. - Please refer to https://github.com/superajun-wsj/gc-adaptive/blob/main/gc_adaptive_controller.go . - Our implementation can easily drive the system into a stable state; the three examples under examples/stress can be used to validate this. If you have any questions, feel free to reach out and discuss. — Reply to this email directly, view it on GitHub <#77587 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/B7BKMDSJZF4MAGI577OXXZL4REGHZAVCNFSM6AAAAACU7K3UMCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DANZTGI4DQMZSHA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>