Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: scheduler: go-routine starvation #21053

prasannavl opened this issue Jul 17, 2017 · 7 comments

runtime: scheduler: go-routine starvation #21053

prasannavl opened this issue Jul 17, 2017 · 7 comments


Copy link

prasannavl commented Jul 17, 2017

What version of Go are you using (go version)?

go version go1.8.3 windows/amd64

What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=D:\Workspace\Golang
set GOROOT=D:\Apps\Scoop\apps\go\current
set GOTOOLDIR=D:\Apps\Scoop\apps\go\current\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set PKG_CONFIG=pkg-config
set CGO_CFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2

What did you do?

Consider the below program:

package main

import (

func main() {

	var ops uint64 = 0
        // This is the key value. When this is >= to the num 
        // of physical cores, the program will never terminate. 
        // Otherwise it should work as expected.
        iters := 4
	for i := 0; i < iters; i++ {
		go func() {
			for {
				atomic.AddUint64(&ops, 1)

	// Wait a second to allow some ops to accumulate.

	opsFinal := atomic.LoadUint64(&ops)
	fmt.Println("ops:", opsFinal)

I was trying to get an estimate of the ops of atomics and mutexes. I wrote the above example. When instead of atomics, mutexes are used, it works as expected. However, when using atomics, when the number of go-routines are greater than the number of the actual physical threads on the system, the goroutine that's waiting on the timer (the main go-routine) is starved - I'm guessing is because of the CAS semantics of atomics that staves the runtime due to SpinWaits.

What did you expect to see?

The program actually completes in a near a second.

What did you see instead?

The program never ends using 100% of the CPU.

This tells me that the scheduler ends up starving the goroutine that's waiting on the timer. Is it possible to improve the scheduler to detect this somehow and not end up starving? This is a unique problem due to the nature of go-routines. Coming from a .NET eco-system, where the Task units are run from a thread pool, which uses physical threads, the OS scheduler handles this smoothly, and this was quite a surprise to me - but makes sense considering the nature of go-routines.


Surprisingly, looks like it doesn't actually have anything to do with atomics. Just turning it into an empty loop, causes the main go-routine to be starved. Quite confused on what to make of this. Is this some kind of a scheduler bug?

@prasannavl prasannavl changed the title Scheduler starvation when using atomics scheduler: go-routine starvation when using atomics Jul 17, 2017
Copy link

ALTree commented Jul 17, 2017

This is #10958.

Note that if you build with GOEXPERIMENT=preemptibleloops, the program terminates. On my 4-cores machine:


$ go run prova.go
  [does not terminate]

go1.9beta2, built with preemptibleloops:

$ ~/go/bin/go run prova.go
ops: 30388899

This will be fixed when (if) preemptibleloops-by-default lands.

@prasannavl prasannavl changed the title scheduler: go-routine starvation when using atomics scheduler: go-routine starvation Jul 17, 2017
Copy link

@ALTree - Ah. Glad to see it work in progress. Shall I close this as dup?

Copy link

Yes, this is a dup. It's targeted to Go 1.10.

Copy link

@ALTree, does the GOEXPERIMENT apply to all platforms? I tried it in windows - doesn't seem to work. Still hangs.

PS D:\Workspace\Labs\go-play> go version
go version go1.9beta2 windows/amd64
PS D:\Workspace\Labs\go-play> $env:GOEXPERIMENT
PS D:\Workspace\Labs\go-play> go run .\play.go

Copy link

You can't just set the environment variable, IIRC. You have to set the variable and then rebuild the Go toolchain.

Copy link

ALTree commented Jul 17, 2017

Yeah you need to rebuild the toolchain from source with the variable enabled, e.g. GOEXPERIMENT=preemptibleloops ./make.bash in go/src

Copy link

@bradfitz, ah - thanks. Thought it was a runtime config. Rebuilding the toolchain now.

@mikioh mikioh changed the title scheduler: go-routine starvation runtime: scheduler: go-routine starvation Jul 21, 2017
@golang golang locked and limited conversation to collaborators Jul 21, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
None yet

No branches or pull requests

4 participants