Skip to content

runtime: running cmd.exec within goroutine sometimes leaves process with 100% CPU #53863

Closed
@ostrowr

Description

@ostrowr

What version of Go are you using (go version)?

Reproducible on at least 1.18.1 and 1.18.4 on Apple Silicon (M1)

go version go1.18.1 darwin/arm64
go version go1.18.4 darwin/arm64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/robbie/Library/Caches/go-build"
GOENV="/Users/robbie/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/robbie/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/robbie/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/homebrew/Cellar/go/1.18.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/homebrew/Cellar/go/1.18.4/libexec/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.18.4"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/my/x46mws81707gx7sxgsjcwc2w0000gn/T/go-build881887791=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

See the repro repo here for a few scripts that might help you reproduce the issue.

  1. Build the below program – go build -o repro main.go
  2. Run repro a bunch of times at once – e.g. for i in $(seq 500); do "./repro" &; done
  3. run ps aux | grep repro
  4. Notice that some small percentage of the time, there are leftover processes either called repro or (repro) that continue to run and hog 100% CPU.

Screen Shot 2022-07-13 at 10 35 38 PM

This is the smallest reproduction I can find but haven't been able to diagnose what this process is actually doing. kill -ABRT is ignored; the only way I can get rid of these hanging processes is with a kill -9.

No idea if this has to do with running a goroutine in an init function and whether this is replicable without running exec.Command or whether this is an exec.Command bug independent of init semantics.

package main

import (
	"math/rand"
	"os/exec"
	"time"
)

func main() {
	go func() {
		exec.Command("echo").Run()
	}()
	time.Sleep(time.Duration(rand.Intn(20)) * time.Millisecond)
}

Old reproduction
package main

import (
	"math/rand"
	"os/exec"
	"time"
)

func init() {
	go func() {
		exec.Command("echo").Run() // doesn't have to be `echo`; reproduces with other commands
	}()
}

func main() {
	time.Sleep(time.Duration(rand.Intn(20)) * time.Millisecond)
}

What did you expect to see?

The process to exit and not take up much CPU

What did you see instead?

A small percentage of the time, the process remains, either with its own name or its name in parentheses in the process table, hanging and taking up 100% CPU.

Metadata

Metadata

Assignees

Labels

FrozenDueToAgeNeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.OS-Darwincompiler/runtimeIssues related to the Go compiler and/or runtime.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions