Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: the Go compiler uses a hack to pretend it has a large startup heap #56546

Open
dr2chase opened this issue Nov 3, 2022 · 0 comments
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@dr2chase
Copy link
Contributor

dr2chase commented Nov 3, 2022

This is a placeholder bug to collect comment, input, experience concerning the compiler's hack. Anyone who sees that code and says "wow, I want this, I will try it myself", we'd like to hear about your experience, either positive or negative, because that is useful for determining if this just stays a hack or if it eventually gets turned into a GC knob. It's important to remember that the compiler is a little bit special; in general its heap use grows and grows (the last phases are some of the most memory-hungry) and it has been carefully tuned and tweaked to reuse allocated objects and avoid generating garbage. Its heap use is very input-dependent, and it also runs on a wide variety of hardware (ODroids and Raspberry Pi, busy laptops shared with greedy browsers, sole-tenant build containers, and 100+ processor and GB servers). Your use case is probably not like the Go compiler, it looks weird in all the GC benchmarks that we do.

What version of Go are you using (go version)?

This is true starting in Go 1.20.

What operating system and processor architecture are you using (go env)?

This is true all places where people run the Go compiler.

What did you do?

In CL 436235, in cmd/compile/internal/base/base.go, the function AdjustStartingHeap does exactly that, using runtime metrics and finalizers to monitor the heap's live size. After-ish each GC, it estimates a new value of GOGC that will generate the requested heap "goal" (the maximum heap size, attained just as a GC cycle completes), and if that is larger than 125, it sets that GOGC and continues monitoring and adjusting. Otherwise, it sets GOGC to 100 and stops the process. This seems to work really well for the compiler, but this is yucky, and relies on finalizers running promptly, which they usually do. This tweak currently has an interaction with manually requested GC, so, be aware of that.

@dr2chase dr2chase added this to the Backlog milestone Nov 3, 2022
@dr2chase dr2chase self-assigned this Nov 3, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Nov 3, 2022
@dr2chase dr2chase added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. and removed compiler/runtime Issues related to the Go compiler and/or runtime. labels Nov 3, 2022
@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Nov 3, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
Status: Todo
Development

No branches or pull requests

2 participants