New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: clearpools causes excessive STW1 time #22331

Open
bryanpkc opened this Issue Oct 18, 2017 · 1 comment

Comments

Projects
None yet
5 participants
@bryanpkc
Contributor

bryanpkc commented Oct 18, 2017

Our application uses a large number of goroutines, mutexes, and defers. Using go 1.8.3 on linux amd64, we observed that the STW1 time is dominated by the runtime.clearpools function in mgc.go. Here is an excerpt from the GC trace:

gc 12 @27.641s 4%: 16+865+0.82 ms clock, 522+6169/9551/1514+26 ms cpu, 8375->8694->5362 MB, 8943 MB goal, 48 P

By instrumenting the runtime, we determined that, out of the 16ms STW1 time, 9.3ms was spent in the loop that disconnects sched.sudogcache linked list, and 6.7ms was spent in the loop that disconnects the linked lists in sched.deferpool. These O(n) loops end up causing a very long pause.

I understand the reason for originally introducing the loops in #9110. However, looking at tip around the call sites of releaseSudog and freedefer, I couldn't see a case where a released object would still be referenced by a live pointer somewhere else in the system. Are these loops still really necessary?

FWIW, I replaced the loops with a simple zeroing of the heads of the linked lists, and test/fixedbugs/issue9110.go still passed.

I propose making these loops optional to avoid the excessive STW1 time, and enabling them only with a GODEBUG option for debugging purposes.

@bradfitz

This comment has been minimized.

Member

bradfitz commented Oct 18, 2017

@bradfitz bradfitz changed the title from runtime.clearpools causes excessive STW1 time to runtime: clearpools causes excessive STW1 time Oct 18, 2017

@bradfitz bradfitz added this to the Go1.10 milestone Oct 18, 2017

@rsc rsc modified the milestones: Go1.10, Go1.11 Nov 22, 2017

@ianlancetaylor ianlancetaylor modified the milestones: Go1.11, Go1.12 Jul 9, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment