Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/debug: don't STW GC on SetGCPercent #19076

Closed
aclements opened this issue Feb 14, 2017 · 5 comments
Closed

runtime/debug: don't STW GC on SetGCPercent #19076

aclements opened this issue Feb 14, 2017 · 5 comments
Assignees
Milestone

Comments

@aclements
Copy link
Member

@aclements aclements commented Feb 14, 2017

As of Go 1.8, runtime/debug.SetGCPercent forces a full STW GC. This is an impediment to responding to memory pressure/overload.

Assuming we're not already over the new GOGC value, it should instead adjust the GC pacing parameters without necessarily triggering a GC. This is complicated by the fact that we may be mid-sweep or mid-mark, so we have to account for current progress.

@aclements aclements added this to the Go1.9Early milestone Feb 14, 2017
@aclements aclements self-assigned this Feb 14, 2017
@gopherbot
Copy link

@gopherbot gopherbot commented Apr 6, 2017

CL https://golang.org/cl/39835 mentions this issue.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 6, 2017

CL https://golang.org/cl/39834 mentions this issue.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 6, 2017

CL https://golang.org/cl/39831 mentions this issue.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 6, 2017

CL https://golang.org/cl/39832 mentions this issue.

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 6, 2017

CL https://golang.org/cl/39833 mentions this issue.

gopherbot pushed a commit that referenced this issue Apr 21, 2017
Currently, the computations that derive controls from the GC trigger
are spread across several parts of the mark termination code.
Consolidate computing the absolute trigger, the heap goal, and sweep
pacing into a single function called at the end of mark termination.

Unlike the code being consolidated, this has to be more careful about
negative gcpercent. Many of the consolidated code paths simply didn't
execute if GC was off.

This is a step toward being able to change the GC trigger ratio in the
middle of concurrent sweeping and marking. For this commit, we try to
stick close to the original structure of the code that's being
consolidated, so it doesn't yet support mid-cycle adjustments.

For #19076.

Change-Id: Ic5335be04b96ad20e70d53d67913a86bd6b31456
Reviewed-on: https://go-review.googlesource.com/39831
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
gopherbot pushed a commit that referenced this issue Apr 21, 2017
Currently, proportional sweep maintains its own count of how many
bytes have been allocated since the beginning of the sweep cycle so it
can compute how many pages need to be swept for a given allocation.

However, this requires a somewhat complex reimbursement scheme since
proportional sweep must be done before a span is allocated, but we
don't know how many bytes to charge until we've allocated a span. This
means that the allocated byte count used by proportional sweep can go
up and down, which has led to underflow bugs in the past (#18043) and
is going to interfere with adjusting sweep pacing on-the-fly (for #19076).

This approach also means we're maintaining a statistic that is very
closely related to heap_live, but has a different 0 value. This is
particularly confusing because the sweep ratio is computed based on
heap_live, so you have to understand that these two statistics are
very closely related.

Replace all of this and compute the sweep debt directly from the
current value of heap_live. To make this work, we simply save the
value of heap_live when the sweep ratio is computed to use as a
"basis" for later computing the sweep debt.

This eliminates the need for reimbursement as well as the code for
maintaining the sweeper's version of the live heap size.

For #19076.

Coincidentally fixes #18043, since this eliminates sweep reimbursement
entirely.

Change-Id: I1f931ddd6e90c901a3972c7506874c899251dc2a
Reviewed-on: https://go-review.googlesource.com/39832
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
gopherbot pushed a commit that referenced this issue Apr 21, 2017
This changes gcSetTriggerRatio so it can be called even during
concurrent mark or sweep. In this case, it will adjust the pacing of
the current phase, accounting for progress that has already been made.

To make this work for concurrent sweep, this introduces a "basis" for
the pagesSwept count, much like the basis we just introduced for
heap_live. This lets gcSetTriggerRatio shift the basis to the current
heap_live and pagesSwept and compute a slope from there to completion.
This avoids creating a discontinuity where, if the ratio has
increased, there has to be a flurry of sweep activity to catch up.
Instead, this creates a continuous, piece-wise linear function as
adjustments are made.

For #19076.

Change-Id: Ibcd76aeeb81ff4814b00be7cbd3530b73bbdbba9
Reviewed-on: https://go-review.googlesource.com/39833
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
gopherbot pushed a commit that referenced this issue Apr 21, 2017
The current SetGCPercent test is, shall we say, minimal.

Expand it to check that the GC target is actually computed and updated
correctly.

For #19076.

Change-Id: I6e9b2ee0ef369f22f72e43b58d89e9f1e1b73b1b
Reviewed-on: https://go-review.googlesource.com/39834
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
@gopherbot gopherbot closed this in 227fff2 Apr 21, 2017
@golang golang locked and limited conversation to collaborators Apr 21, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants